Skip to search formSkip to main contentSkip to account menu
Injecting Numerical Reasoning Skills into Language Models
TLDR
This work shows that numerical reasoning is amenable to automatic data generation, and thus one can inject this skill into pre-trained LMs, by generating large amounts of data, and training in a multi-task setup.
View on ACL
Break It Down: A Question Understanding Benchmark
TLDR
This work introduces a Question Decomposition Meaning Representation (QDMR) for questions, and demonstrates the utility of QDMR by showing that it can be used to improve open-domain question answering on the HotpotQA dataset, and can be deterministically converted to a pseudo-SQL formal language, which can alleviate annotation in semantic parsing applications.
View on ACL
Approaching the Chasm at Depth Four
TLDR
Any homogeneous depth four arithmetic circuit with bottom fanin bounded by √n computing the permanent (or the determinant) must be of size exp(Ω(√n)).
View on ACM
Algebraic Geometric Techniques for Depth-4 PIT & Sylvester-Gallai Conjectures for Varieties
  • Ankit Gupta
  • Mathematics
    Electron. Colloquium Comput. Complex.
  • 2014
TLDR
An algebraic-geometric approach for devising a deterministic polynomial time blackbox identity testing (PIT) algorithm for depth-4 circuits with bounded top fanin and a Dvir-Shpilka [DS06]-like approach to resolve the degenerate case is presented.
View Paper
Arithmetic Circuits: A Chasm at Depth Three
We show that, over Q, if an n-variate polynomial of degree d = n<sup>O(1)</sup> is computable by an arithmetic circuit of size s (respectively by an arithmetic branching program of size s) then it…
View on IEEE
Arithmetic Circuits: A Chasm at Depth 3
TLDR
If you can prove a lower bound of $\exp(\omega(\sqrt{d} \cdot \log d)$ on the size of any $\Sigma \Pi \Sigma$ circuit computing the $d \times d$ permanent $\mathsf{Perm}_d$, then you get superpolynomial lower bounds for the sizeof any arithmetic branching program computing $\Mathsf{ perm]_d$.
View via Publisher
Memory-efficient Transformers via Top-k Attention
TLDR
This work proposes a simple yet highly accurate approximation for vanilla attention that leads to accuracy that is nearly-identical to vanilla attention in multiple setups including training from scratch, fine-tuning, and zero-shot inference.
View on ACL
GMAT: Global Memory Augmentation for Transformers
TLDR
This work proposes to augment sparse Transformer blocks with a dense attention-based $\textit{global memory}$ of length $M$ ($\ll L$) which provides an aggregate global view of the entire input sequence to each position, and empirically shows that this method leads to substantial improvement on a range of tasks.
View PDF on arXiv
Reconstruction of depth-4 multilinear circuits with top fan-in 2
TLDR
This work presents a randomized algorithm for reconstructing multilinear depth-4 circuits with fan-in 2 at the top + gate, which is the first reconstruction result for any model of depth- 4 arithmetic circuits.
View on ACM
Random arithmetic formulas can be reconstructed efficiently
TLDR
A randomized algorithm that given black-box access to the polynomial f computed by an unknown/hidden arithmetic formula ϕ reconstructs, on the average, an equivalent or smaller formula that computes an ANF formula of size s computing f.
View on IEEE
...
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy, Terms of Service, and Dataset License