• Corpus ID: 212633677

Teaching Temporal Logics to Neural Networks

@article{Finkbeiner2020TeachingTL,
  title={Teaching Temporal Logics to Neural Networks},
  author={Bernd Finkbeiner and Christopher Hahn and Markus N. Rabe and Frederik Schmitt},
  journal={ArXiv},
  year={2020},
  volume={abs/2003.04218}
}
We show that a deep neural network can learn the semantics of linear-time temporal logic (LTL). As a challenging task that requires deep understanding of the LTL semantics, we show that our network can solve the trace generation problem for LTL: given a satisfiable LTL formula, find a trace that satisfies the formula. We frame the trace generation problem for LTL as a translation task, i.e., to translate from formulas to satisfying traces, and train an off-the-shelf implementation of the… 
[PDF] Semantic Reader
29 Citations
Highly Influential Citations
1
Background Citations
17
Methods Citations
10

Figures and Tables from this paper

29 Citations

Proof Artifact Co-training for Theorem Proving with Language Models

PACT is proposed, a general methodology for extracting abundant self-supervised data from kernel-level proof terms for co-training alongside the usual tactic prediction objective and applied to Lean, an interactive proof assistant which hosts some of the most sophisticated formalized mathematics to date.

Mathematical Reasoning via Self-supervised Skip-tree Training

It is found that models trained on the skip-tree task show surprisingly strong mathematical reasoning abilities, and outperform modelstrained on standard skip-sequence tasks.

Conversational Neuro-Symbolic Commonsense Reasoning

An interactive conversational framework built on the authors' neuro-symbolic system, that conversationally evokes commonsense knowledge from humans to complete its reasoning chains, and presents a neuro-Symbolic theorem prover that extracts multi-hop reasoning chains for this problem.

Checking LTL Satisfiability via End-to-end Learning

This paper trains different neural networks for keeping three logical properties of LTL, i.e., recursive property, permutation invariance, and sequentiality, and demonstrates that neural networks can indeed capture some effective biases for checking LTL satisfiability.

Deep Learning for Temporal Logics

This work reports on current advances in applying deep learning to temporal logical reasoning tasks, showing that models can even solve instances where competitive classical algorithms timed out.

Neural Circuit Synthesis from Specification Patterns

This paper considers a method to generate large amounts of additional training data, i.e., pairs of specifications and circuits implementing them, and shows that hierarchical Transformers trained on this synthetic data solve aSignificant portion of problems from the synthesis competitions, and even out-of-distribution examples from a recent case study.

Towards the Automatic Mathematician

This extended abstract summarizes recent developments of machine learning in mathematical reasoning and the vision of the N2Formal group at Google Research to create an automatic mathematician.

Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples

A novel, fully explainable neural approach to synthesis of combinatorial logic circuits from input-output examples, which succeeds in learning a number of arithmetic, bitwise, and signal-routing operations, and hints at a wider promise for synthesis and reasoning-related tasks.

FOLIO: Natural Language Reasoning with First-Order Logic

The results show that one of the most capable Large Language Model (LLM) publicly available, GPT-3 davinci, achieves only slightly better than random results with few-shot prompting on a subset of FOLIO, and the model is especially bad at predicting the correct truth values for False and Unknown conclusions.

OCTAL: Graph Representation Learning for LTL Model Checking

A novel GRL-based framework OCTAL, is designed to learn the representation of the graph-structured system and specification, which reduces the model checking problem to binary classification in the latent space.

References

SHOWING 1-10 OF 52 REFERENCES

Learning to Solve SMT Formulas

This work phrases the challenge of solving SMT formulas as a tree search problem where at each step a transformation is applied to the input formula until the formula is solved, and synthesizes a strategy in the form of a loop-free program with branches to guide the SMT solver to decide formulas more efficiently.

NeuroCore: Guiding High-Performance SAT Solvers with Unsat-Core Predictions

This work trains a simplified NeuroSAT architecture to directly predict the unsatisfiable cores of real problems and modify several state-of-the-art SAT solvers to periodically replace their variable activity scores with neuroSAT’s prediction of how likely the variables are to appear in an unsatisfiable core.

Learning to Represent Programs with Graphs

This work proposes to use graphs to represent both the syntactic and semantic structure of code and use graph-based deep learning methods to learn to reason over program structures, and suggests that these models learn to infer meaningful names and to solve the VarMisuse task in many cases.

Attention is All you Need

A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Deep Network Guided Proof Search

Experimental evidence is given that with a hybrid, two-phase approach, deep learning based guidance can significantly reduce the average number of proof search steps while increasing the number of theorems proved.

Dynamic Neural Program Embedding for Program Repair

A novel semantic program embedding that is learned from program execution traces is proposed, showing that program states expressed as sequential tuples of live variable values not only captures program semantics more precisely, but also offer a more natural fit for Recurrent Neural Networks to model.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.

Mathematical Reasoning in Latent Space

The experiments show that graph neural networks can make non-trivial predictions about the rewrite-success of statements, even when they propagate predicted latent representations for several steps, a strong indicator for the feasibility of deduction in latent space in general.

Graph Representations for Higher-Order Logic and Theorem Proving

This paper presents the first use of graph neural networks (GNNs) for higher-order proof search and demonstrates that GNNs can improve upon state-of-the-art results in this domain. Interactive,…

Analysing Mathematical Reasoning Abilities of Neural Models

This paper conducts a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and finds notable differences in their ability to resolve mathematical problems and generalize their knowledge.
...