• Corpus ID: 1287308

Differential Testing for Variational Analyses: Experience from Developing KConfigReader

  title={Differential Testing for Variational Analyses: Experience from Developing KConfigReader},
  author={Christian Kastner},
Differential testing to solve the oracle problem has been applied in many scenarios where multiple supposedly equivalent implementations exist, such as multiple implementations of a C compiler. If the multiple systems disagree on the output for a given test input, we have likely discovered a bug without every having to specify what the expected output is. Research on variational analyses (or variability-aware or family-based analyses) can benefit from similar ideas. The goal of most variational… 

Figures from this paper

RoBin: Facilitating the Reproduction of Configuration-Related Vulnerability

RoBin is implemented, a binary similarity-based building configuration inference tool to infer the specific building configurations via the binary from crash report that can help developers reproduce and diagnose the vulnerability, and finally, patch the programs.

Finding broken Linux configuration specifications by statically analyzing the Kconfig language

A software model checking framework for building Kconfig static analysis tools and a bug finder that takes kclause models and leverages automated theorem proving to find unmet dependency bugs, which is evaluated for its precision, performance, and impact on kernel development for a recent version of Linux.

Quantifying the Potential to Automate the Synchronization of Variants in Clone-and-Own

There is potential to automate the synchronization of cloned variants through existing patching techniques through existingpatching techniques, as demonstrated in this empirical study.

Tseitin or not Tseitin? The Impact of CNF Transformations on Feature-Model Analyses

It is illustrated that some CNF transformations do not scale to complex feature models or even lead to wrong results for model-counting analyses, and the choice of the CNF transformation can substantially influence the performance of subsequent analyses.

Causal Testing: Understanding Defects' Root Causes

Causal Testing is presented, a new method of root-cause analysis that relies on the theory of counterfactual causality to identify a set of executions that likely hold key causal information necessary to understand and repair buggy behavior.

Simulating the Evolution of Clone-and-Own Projects with VEVOS

VEVOS is presented, a benchmark generation framework that picks up requirements and, given the version history of a software product line, enables the simulation of the evolution of cloned variants, and provides meta-data serving as ground truth.

A BDD for Linux?: the knowledge compilation challenge for variability

The goal of this challenge is to guide research on these computationally complex problems and to foster the exchange between researchers and practitioners.

Using Extended Logical Primitives for Efficient BDD Building

It is proved, among other results, that the number of nodes created during the process for XOR groups is reduced from quadratic to linear for the affected clauses, and the XOR primitive is tested against eight logical models, two from industry and six from Kconfig-based open-source projects.

A BDD for Linux?

The goal of this challenge is to guide research on these computationally complex problems and to foster the exchange between researchers and practitioners.

Product Sampling for Product Lines: The Scalability Challenge

A set of large real-world feature models with up to 19 thousand features, which are supposed to be sampled, are provided to initiate a community effort and a well-performing sampling algorithm achieves full t-wise coverage, while minimizing the other properties as best as possible.



Do #ifdefs influence the occurrence of vulnerabilities? an empirical study of the linux kernel

It is suggested that vulnerable functions have higher variability than non-vulnerable ones and are also constrained by fewer configuration options, which suggests that developers are inclined to notice functions appear in frequently-compiled product variants.

Controllable Combinatorial Coverage in Grammar-Based Testing

The notion of controllable combinatorial coverage and a corresponding algorithm for test- data generation are described and implemented in the C#-based test-data generator Geno, which has been successfully used in projects that required differential testing, stress testing and conformance testing of grammar-driven functionality.

Morpheus: Variability-Aware Refactoring in the Wild

Morpheus, a sound, variability-aware refactoring engine for C code with preprocessor directives, is developed and applied to three substantial real-world systems showing that it scales reasonably well, despite of its heavy reliance on satisfiability solvers.

Randomized Differential Testing as a Prelude to Formal Verification

A flight software development effort incorporating large-scale (biased) randomized testing on commodity desktop hardware shows that use of a reference implementation, hardware simulation with fault injection, a testable design, and test minimization enabled a high degree of automation in fault detection and correction.

Finding and understanding bugs in C compilers

Csmith, a randomized test-case generation tool, is created and spent three years using it to find compiler bugs, and a collection of qualitative and quantitative results about the bugs it found are presented.

Generating range fixes for software configuration

This paper proposes a novel concept, range fix, for software configuration, which specifies the options to change and the ranges of values for these options, and designs an algorithm that automatically generates range fixes for a violated constraint.

Variability-aware parsing in the presence of lexical macros and conditional compilation

A novel variability-aware parser that can parse almost all unpreprocessed code without heuristics in practicable time is contributed and paves the road for further analysis, such as variability- aware type checking.

SuperC: parsing all of C by taming the preprocessor

This paper presents a complete, performant solution to the problem of parsing C itself and the preprocessor, and presents a simple algorithm for table-driven Fork-Merge LR parsing and four novel optimizations.

A Comparison of 10 Sampling Algorithms for Configurable Systems

A comparative study of 10 state-of-the-art sampling algorithms regarding their fault-detection capability and size of sample sets found that sampling algorithms with larger sample sets are able to detect higher numbers of faults, but simple algorithms with small sample set are the most efficient in most contexts.

On essential configuration complexity: Measuring interactions in highly-configurable systems

A dynamic analysis for Java based on variability-aware execution and monitor executions of multiple small to medium-sized programs is developed to better understand interactions in practice and finds that the essential configuration complexity of these programs is much lower than the combinatorial explosion of the configuration space indicates.