The Dagstuhl beginners guide to reproducibility for experimental networking research

@article{Bajpai2019TheDB,
  title={The Dagstuhl beginners guide to reproducibility for experimental networking research},
  author={Vaibhav Bajpai and Anna Brunstr{\"o}m and Anja Feldmann and Wolfgang Kellerer and Aiko Pras and Henning Schulzrinne and Georgios Smaragdakis and Matthias W{\"a}hlisch and Klaus Wehrle},
  journal={Comput. Commun. Rev.},
  year={2019},
  volume={49},
  pages={24-30}
}
Reproducibility is one of the key characteristics of good science, but hard to achieve for experimental disciplines like Internet measurements and networked systems. This guide provides advice to researchers, particularly those new to the field, on designing experiments so that their work is more likely to be reproducible and to serve as a foundation for follow-on work by others. 

Tables from this paper

Benchmarking data dissemination protocols for opportunistic networks
TLDR
Improve the reproducibility and cross-comparability of OppNets data dissemination protocols by defining a benchmark suite with realistic mobility and traffic scenarios, mostly based on real traces.
[Tool] Designing Replicable Networking Experiments With Triscale
TLDR
This paper provides networking researchers with a rational and concrete experimental methodology rooted in sound statistical foundations, the first of its kind, and implements this methodology in a software framework called TriScale.
Reproducible Computer Network Experiments: A Case Study Using Popper
TLDR
This paper shows how a recently proposed reproducibility tool called Popper facilitates the reproduction of networking experiments, and presents a generic workflow for carrying out network simulation experiments.
DESIGNING REPLICABLE NETWORKING EXPERIMENTS WITH TriScale
TLDR
This paper provides networking researchers with a rational and concrete experimental methodology rooted in sound statistical foundations and implements this methodology in a software framework called TriScale, which quantifies the replicability of evaluations.
Validating the Sharing Behavior and Latency Characteristics of the L4S Architecture
TLDR
The DualPI2 scheduler is benchmarked, a reference implementation of DQC AQM, to validate some of the experimental result(s) and attribute the difference in the results to the sensitivity of the L4S architecture to traffic bursts and the burst sending pattern of the Linux kernel.
Measuring DNS over TLS from the Edge: Adoption, Reliability, and Response Times
TLDR
It is found that nearly all DoT requests take at least 100 ms to return a response, showing an inflation in response times of more than 100 ms compared to Do53, and while Do53 failure rates for most resolvers individually are consistent across continents, DoT failure rates have much higher variation.
Bluetooth Mesh under the Microscope: How much ICN is Inside?
TLDR
This paper compares BT mesh with ICN both conceptually and in real-world experiments, and identifies synergies and sketches a design of a BT-ICN that benefits from both worlds.
Measuring DNS over TCP in the era of increasing DNS response sizes
TLDR
In this study, a view at DoTCP from the Edge is presented, issuing 12.1M DNS requests from 2,500 probes toward Public as well as Probe DNS recursive resolvers, and it is observed that Do TCP is generally slower than DoUDP, where the relative increase in Response Time is less than 37% for most resolver.
Evaluating QUIC Performance Over Web, Cloud Storage, and Video Workloads
TLDR
This work evaluates QUIC performance over the web, cloud storage, and video workloads and compares them to traditional TLS/TCP and observes that QUIC tends to depict better video content delivery with reduced stall events and up to 50% lower stall durations due to its lower latency overheads.
...
...

References

SHOWING 1-10 OF 33 REFERENCES
PlanetLab: an overlay testbed for broad-coverage services
TLDR
This paper discribes the initial implementation of PlanetLab, including the mechanisms used to impelment virtualization, and the collection of core services used to manage PlanetLab.
Difficulties in simulating the internet
TLDR
Two key strategies for developing meaningful simulations in the face of the global Internet's great heterogeneity are discussed: searching for invariants and judiciously exploring the simulation parameter space.
Encouraging Reproducibility in Scientific Research of the Internet (Dagstuhl Seminar 18412)
TLDR
The seminar discussed challenges to improving reproducibility of scientific Internet research, and developed a set of recommendations that people both from academia and industry can undertake to initiate a cultural change toward reproducible research.
A Survey on Internet Performance Measurement Platforms and Related Standardization Efforts
TLDR
This paper provides a taxonomy of large-scale performance measurement platforms on the basis of their deployment use-case by exploring their coverage, scale, lifetime, deployed metrics and measurement tools, architecture and overall research impact.
Open collaborative hyperpapers: a call to action
TLDR
This note announces initial experiments with Internet measurement hyperpapers, which address issues with reproducibility and verifiability of research in Internet science and measurement, and calls to action to others to come build out this vision with us.
Avoiding traceroute anomalies with Paris traceroute
TLDR
A new publicly-availabletraceroute is provided, called Paris traceroute, which controls packet header contents to obtain a more precise picture of the actual routes that packets follow, and to suggest possible causes for others.
Proceedings of the ACM SIGCOMM workshop on Models, methods and tools for reproducible network research, MoMeTools '03, Karlsruhe, Germany, August 25-27, 2003
TLDR
The goal of this workshop was to critically assess the current models, methods and tools for identifying shortcomings of the state-of-the-art, and to discuss approaches for improvements and innovation.
Thoughts and Recommendations from the ACM SIGCOMM 2017 Reproducibility Workshop
TLDR
A side workshop has been organized with the specific purpose to tackle reproducibility of results as an essential part of research carried out by members of the ACM SIGCOMM community, with the objective being to trigger discussion and activity in order to craft recommendations on how to introduce incentives for authors to share their artifacts.
A Survey on Artifacts from CoNEXT, ICN, IMC, and SIGCOMM Conferences in 2017
TLDR
A survey among the papers published at leading ACM computer networking conferences in 2017 to assess the current state of artifact availability and reproducibility based on a survey is conducted.
Strategies for sound internet measurement
Conducting an Internet measurement study in a sound fashion can be much more difficult than it might first appear. We present a number of strategies drawn from experiences for avoiding or overcoming
...
...