• Corpus ID: 17381714

Replication , Reproduction and Re-analysis : Three ways for verifying experimental

  title={Replication , Reproduction and Re-analysis : Three ways for verifying experimental},
  author={Omar S. G{\'o}mez and Natalia Juristo},
The verification of a previously observed finding is important. Being able to verify a finding provides some guarantee that it is not product of chance but that the observed phenomenon is stable or regular. The most well done way of verifying experimental findings is through replication. But, is this the only method for verifying the observations of an experiment? We have studied different ways of verifying experimental findings that are used in other disciplines. From the 18 replication… 

Tables from this paper

Replications of software engineering experiments

This chapter starts with an introduction to the importance of replication and the state of replication in ESE, and discusses replication from both the statistical and scientific viewpoint.

ARRESTT: A framework to create reproducible experiments to evaluate software testing techniques

It is concluded that ARRESTT enhances reproducibility of an experiment and does not demand a lot of effort to configure and execute an experiment.

Replication and Knowledge Production in Empirical Software Engineering Research

This dissertation dissects the problem of replication into four primary concerns, and enables the empirically and statistically-grounded reconciliation and generalization of otherwise contradictory results across dissimilar replications—which problem has previously remained unsolved in software engineering.

Efficiency of Software Testing Techniques: A Controlled Experiment Replication and Network Meta-analysis

It is suggested that software engineers should have a clear understanding of the software product to be verified and apply either black-box or white-box testing techniques as they yield similar defect detection rates.

Investigations about replication of empirical studies in software engineering: preliminary findings from a mapping study

This research analyzed 36 papers reporting empirical and non-empirical studies about replications in SE published in the last 17 years and explored different topics related to concepts and classifications, presented guidelines, and discussed theoretical issues that are relevant for the understanding of replication in the authors' field.

Evolution of design patterns: a replication study

Findings confirm that patterns change more frequently when they play a crucial role in the software and when in newer releases they support more advanced features.

Replication of Empirical Studies in Software Engineering: Preliminary Findings from a Systematic Mapping Study

The preliminary findings of this review are presented, concentrating on the studies reporting replications and the related original studies, about the current state of the replication work of empirical studies performed in software engineering research.

An Initiative to Improve Reproducibility and Empirical Evaluation of Software Testing Techniques

The development of a tool that enables automatic execution and analysis of experiments producing a reproducible research compendia as output that is, in turn, shared among researchers is proposed.

Reproducibility in Scientific Computing

The many objectives and meanings of reproducibility are discussed within the context of scientific computing, and technical barriers to reproducible work are described.



Replication as a Verification Technique in Survey Research: A Paradigm

Replication is one of several methodological techniques that contributes to the verification of survey research data. A critique of the literature on replication reveals that limited systematic

CHAPTER 3 A Process Model Of Replication Studies : On The Relation Between Different Types Of Replication *

This chapter argues that the "received view" of replication, the exact or algorithmic repetition of an original study has become obsolete, and that methodologically replicability is considered to be one of the most important cornerstones of science.

The Design of Replicated Studies

Abstract Replication is little discussed in the statistical literature nor practiced widely by statistically minded researchers. It is needed not merely to validate one's findings, but more

Experimental Reproducibility and the Experimenters' Regress

  • H. Radder
  • Education
    PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association
  • 1992
In his influential book, "Changing Order", H.M. Collins puts forward the following three claims concerning experimental replication. (i) Replication is rarely practiced by experimentalists; (ii)

An Empirical Evaluation of Three Defect-Detection Techniques

The results suggest that inexperienced subjects can apply a formal verification technique (code reading) as effectively as an execution-based validation technique, but they are most efficient when using functional testing.

Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences

Replication is one of the most important tools for the verification of facts within the empirical sciences. A detailed examination of the notion of replication reveals that there are many different

What is the value of replicating other studies

In response to a question on the value of replication in social science research, the author undertook a search of the literature for expert advise on the value of such an activity. Using the

The Desperate Need for Replications

An overemphasis on creativity for evaluating research has lead to a serious devaluation of replication studies. However, we need a total sample size of N = 153,669 to estimate a causal effect to two


This study proposes a taxonomy of replication designs and applies it to a representative sample of experimental communication research reports in Communication Monographs and Human Communication

Can results from software engineering experiments be safely combined?

  • James Miller
  • Computer Science
    Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403)
  • 1999
The treatise of the paper describes a number of recommendations for controlling and reporting empirical work to advance the discipline towards a position where meta-analysis can be profitably employed.