Standing on the Shoulders of Giant Frozen Language Models
condense relevant information from 100+ retrieved documents into the input sequence length of the frozen LM reader. We show that can reach and surpass leading ﬁne tuning approaches on Natural…
Value-based Search in Execution Space for Mapping Instructions to Programs
- Dor Muhlgay, Jonathan Herzig, Jonathan Berant
- Computer ScienceNorth American Chapter of the Association for…
- 2 November 2018
A search algorithm that uses the target world state, known at training time, to train a critic network that predicts the expected reward of every search state, and dramatically improves performance on all three domains compared to standard beam search and other baselines.
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning
This work describes a neuro-symbolic architecture with multiple neural models, complemented by discrete knowledge and reasoning modules, dubbed the Modular Reasoning, Knowledge and Language (MRKL), and some of the technical challenges in implementing it, and Jurassic-X, AI21 Labs’ MRKL system implementation.
Tackling Spuriousness with Similarity
Training the parser on examples labeled only with their correct output, and not with the correct formula that produced it, facilitates the training process but comes at the price of learning from spurious formulas, logical forms which execute to the correct output but do not capture the meaning of the original utterance.