An Empirical Study on Practicality of Specification Mining Algorithms on a Real-World Application

  title={An Empirical Study on Practicality of Specification Mining Algorithms on a Real-World Application},
  author={Mohammad Jafar Mashhadi and H. Hemmati},
  journal={2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)},
Dynamic model inference techniques have been the center of many research projects recently. [...] Key Method We tried to apply some of the existing model inference techniques in a real-world industrial context to support program comprehension for debugging. Our initial experiments have shown many limitations both in terms of implementation as well as the algorithms. The paper will discuss the root cause of the failures and proposes ideas for future improvement.Expand
Survey on Learning-Based Formal Methods: Taxonomy, Applications and Possible Future Directions
This paper is not a comprehensive survey of learning-based techniques in formal methods area, but rather as a survey of the taxonomy, applications and possible future directions in learning- based formal methods. Expand
Behavioral Model Inference of Black-box Software using Deep Neural Networks
This paper proposes a hybrid deep neural network that accepts as input a multivariate time series and applies a set of convolutional and recurrent layers to learn the non-linear correlations between signals and the patterns over time, and shows how this approach can be used to accurately detect state changes. Expand
An Interview Study of how Developers use Execution Logs in Embedded Software Engineering
The empirical evidence provided by the study implies the need to support log inspection and comparison with multiple levels of abstraction, categorize log differences, and recover links between different types of logs. Expand


Automatic mining of specifications from invocation traces and method invariants
Using invariants to infer an initial model significantly improves model quality, increasing precision by 4% and recall by 41%, on average; effective invariant filtering is crucial for quality and scalability of strategies that use invariants; and using traces in combination with invariants greatly improves robustness to input noise. Expand
Symbolic mining of temporal specifications
  • Mark Gabel, Z. Su
  • Computer Science
  • 2008 ACM/IEEE 30th International Conference on Software Engineering
  • 2008
This paper precisely defines this form of specification mining and shows that its general form is NP-complete, and introduces a symbolic algorithm, based on binary decision diagrams (BDDs), that exploits a great deal of regularity in the representation and tracking of all possible combinations of system components. Expand
Learning extended FSA from software: An empirical assessment
This is the first work that quantifies both the effect of adding data-flow information within automata and the effectiveness of the techniques when varying sparseness of traces, produced by different software systems. Expand
Inferring extended finite state machine models from software executions
A novel EFSM inference technique is presented that addresses the problems of inflexibility and non-determinism, and is adapted from the field of Machine Learning and applies it to three diverse software systems. Expand
Behaviourally Adequate Software Testing
  • G. Fraser, Neil Walkinshaw
  • Computer Science
  • 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation
  • 2012
A new test generation technique that is founded on behavioural adequacy is presented, which combines a model evaluation framework from the domain of statistical learning theory with search-based white-box test generation strategies. Expand
Leveraging existing instrumentation to automatically infer invariant-constrained models
It is formally prove that Synoptic always produces a model that satisfies exactly the temporal invariants mined from the log, and it is argued that it does so efficiently. Expand
Mining modal scenario-based specifications from execution traces of reactive systems
A novel method, framework, and tool, for mining inter-object scenario-based specifications in the form of a UML2-compliant variant of Damm and Harels Live Sequence Charts (LSC), which extends the classical partial order semantics of sequence diagrams with temporal liveness and symbolic class level lifelines, in order to generate compact and expressive specifications. Expand
Inferring models of concurrent systems from logs of their behavior with CSight
Algorithms for inferring CFSM models from traces of concurrent systems, proves them correct, provides an implementation, and evaluates the implementation in two ways: by running it on logs from three different networked systems and via a user study that focused on bug finding. Expand
Black-Box Test Generation from Inferred Models
This paper provides a proof-of-concept inference-driven testing framework, along with evidence of the efficacy of its test sets on three programs, and presents an openly available framework to facilitate experimentation in this area. Expand
Automatically Generating Test Cases for Specification Mining
To the knowledge, this is the first combination of systematic test case generation and typestate mining-a combination with clear benefits: on a sample of 800 defects seeded into six Java subjects, a static typestate verifier fed with enriched models would report significantly more true positives and significantly fewer false positives than the initial models. Expand