Dissection of a bug dataset: Anatomy of 395 patches from Defects4J

@article{Sobreira2018DissectionOA,
  title={Dissection of a bug dataset: Anatomy of 395 patches from Defects4J},
  author={Victor Sobreira and Thomas Durieux and Fernanda Madeiral Delfim and Monperrus Martin and Marcelo de Almeida Maia},
  journal={2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER)},
  year={2018},
  pages={130-140}
}
  • Victor SobreiraThomas Durieux M. Maia
  • Published 19 January 2018
  • Computer Science
  • 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER)
Well-designed and publicly available datasets of bugs are an invaluable asset to advance research fields such as fault localization and program repair as they allow directly and fairly comparison between competing techniques and also the replication of experiments. These datasets need to be deeply understood by researchers: the answer for questions like "which bugs can my technique handle?" and "for which bugs is my technique effective?" depends on the comprehension of properties related to… 

Figures and Tables from this paper

Is the Ground Truth Really Accurate? Dataset Purification for Automated Program Repair

DPTEST, an automated DatasEt Purification technique from the perspective of triggering Tests, can automatically identify and filter out the code changes irrelevant to the bug exposed by triggering tests and has great potential in assisting in the construction of datasets of accurate bug fixes.

AVATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations

This study builds the AVATAR APR system, which exploits fix patterns of static analysis violations as ingredients for patch generation, and highlights the relevance of static bug finding tools as indirect contributors of fix ingredients for addressing code defects identified with functional test cases.

Towards an automated approach for bug fix pattern detection

PPD, a detector of repair patterns in patches, which performs source code change analysis at abstract-syntax tree level, is designed and implemented and evaluated and it is concluded that PPD has the potential to detect as many repair patterns as human manual analysis.

Where were the repair ingredients for Defects4j bugs?

A large scale empirical study on the execution results of 24 repair systems evaluated on realworld bugs from Defects4J provides a comprehensive view on the distribution of repair ingredients that are relevant for these bugs.

BUGSJS: a benchmark and taxonomy of JavaScript bugs

BugsJS is a benchmark of 453 real, manually validated JavaScript bugs from 10 popular JavaScript server‐side programs, comprising 444k lines of code (LOC) in total, and a classification of the bugs according to their nature is performed, which shows that the taxonomy is adequate for characterizing the bugs in BugsJS.

GumTree Pattern MiningData PreprocessingData Collection Projects Commits Static analysis tool Fixed static analysis violations Buggy tree Fixed tree Edit scripts

This study builds the AVATAR APR system, which exploits fix patterns of static analysis violations as ingredients for patch generation, and highlights the relevance of static bug finding tools as indirect contributors of fix ingredients for addressing code defects identified with functional test cases.

Constructing Regression Dataset from Code Evolution History

This work addresses the challenges of identifying potential regression-fixing commits from the code evolution history, migrating the test and its code dependencies over the history, and minimizing the compilation overhead during the regression search, and builds the largest replicable regression dataset within shortest period.

An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation

An empirical study to assess the feasibility of using Neural Machine Translation techniques for learning bug-fixing patches for real defects finds that such a model is able to fix thousands of unique buggy methods in the wild.

How Different Is It Between Machine-Generated and Developer-Provided Patches? : An Empirical Study on the Correct Patches Generated by Automated Program Repair Techniques

This study conducts an empirical study towards understanding the correct patches that are generated by existing state-of-the-art APR techniques, aiming at providing guidelines for future assessment of patches.

Defexts: A Curated Dataset of Reproducible Real-World Bugs for Modern JVM Languages

Defexts is introduced, a family of bug datasets currently containing child datasets for Kotlin and Groovy, which contains reproducible real-world bugs and their corresponding patches scraped from real- world projects.
...

References

SHOWING 1-10 OF 36 REFERENCES

Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset

The result of the experiment shows that the considered state-of-the-art repair methods can generate patches for 47 out of 224 bugs, however, those patches are only test-suite adequate, which means that they pass the test suite and may potentially be incorrect beyond the test-Suite satisfaction correctness criterion.

Automatic patch generation learned from human-written patches

A novel patch generation approach, Pattern-based Automatic program Repair (Par), using fix patterns learned from existing human-written patches to generate program patches automatically, which is more acceptable than GenProg.

Extraction of bug localization benchmarks from history

iBUGS is presented, an approach that semiautomatically extracts benchmarks for bug localization from the history of a project and demonstrates the relevance of the dataset with a case study on the bug localization tool AMPLE.

Defects4J: a database of existing faults to enable controlled testing studies for Java programs

Defects4J, a database and extensible framework providing real bugs to enable reproducible studies in software testing research, and provides a high-level interface to common tasks in softwareTesting research, making it easy to con- duct and reproduce empirical studies.

Do automated program repair techniques repair hard and important bugs?

Strengths and shortcomings of the state-of-the-art of automated program repair along new dimensions are identified and can drive research toward improving the applicability of automated repair techniques to hard and important bugs.

A Deeper Look into Bug Fixes: Patterns, Replacements, Deletions, and Additions

A large-scale study of bug-fixing commits in Java projects is conducted, focusing on assumptions underlying common search-based repair approaches and common and uncommon statement modifications in human patches and the applicability of previously-proposed patch construction operators in the Java context.

A learning-to-rank based fault localization approach using likely invariants

This work proposes Savant, a new fault localization approach that employs a learning-to-rank strategy, using likely invariant diffs and suspiciousness scores as features, to rank methods based on their likelihood to be a root cause of a failure.

The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs

The need for a new set of benchmarks is outlined, requirements are outlined, and two datasets, ManyBugs and IntroClass, consisting between them of 1,183 defects in 15 C programs are presented, designed to support the comparative evaluation of automatic repair algorithms asking a variety of experimental questions.

Elixir: Effective object-oriented program repair

ELIXIR is able to increase the number of correctly repaired bugs in Defects4J and in Bugs.jar by 85% and significantly out-performing other state-of-the-art repair techniques including ACS, HD-Repair, NOPOL, PAR, and jGenProg.

Precise Condition Synthesis for Program Repair

A novel program repair system, ACS, that could generate precise conditions at faulty locations and can perform a repair operation that is previously deemed to be too overfitting: directly returning the test oracle to repair the defect.