Continuously Testing Distributed IoT Systems: An Overview of the State of the Art

  title={Continuously Testing Distributed IoT Systems: An Overview of the State of the Art},
  author={Jossekin Beilharz and Philipp Wiesner and Arne Boockmeyer and Lukas Pirl and Dirk Friedenberger and Florian Brokhausen and Ilja Behnke and Andreas Polze and Lauritz Thamsen},
  booktitle={ICSOC Workshops},
The continuous testing of small changes to systems has proven to be useful and is widely adopted in the development of software systems. For this, software is tested in environments that are as close as possible to the production environments. When testing IoT systems, this approach is met with unique challenges that stem from the typically large scale of the deployments, heterogeneity of nodes, challenging network characteristics, and tight integration with the environment among others. IoT… 

Tables from this paper

A Domain-Specific Language for Simulation-Based Testing of IoT Edge-to-Cloud Solutions

A domain-specific language (DSL) for generating edge-to-cloud simulators, named IoTECS, is proposed and implemented, ensuring the scalability of cloud applications by putting them under extreme loads from IoT devices connected to the edge.

IoTreeplay: Synchronous Distributed Traffic Replay in IoT Environments

The IoTreeplay tool embeds TCPLivePlay into an environment that allows the synchronous replaying of network traffic with multiple endpoints and connections, and presents the resulting implementation, able to accurately replay connections within a maximum transmission rate but struggles with deviations from regular TCP connections.



Software-Defined Virtual Testbed for IoT Systems

This paper proposes the concept of a software-defined IoT testbed which enables an IoT system to be immersed and tested in a virtual environment in order to evaluate its behavior under controllable conditions.

Héctor: A Framework for Testing IoT Applications Across Heterogeneous Edge and Cloud Testbeds

Héctor is presented, a framework for the automatic testing of IoT applications that allows the automated execution of user-defined experiments on agnostic IoT testbeds and is able to generate virtual test beds with adjustable network properties.

IoT-TaaS: Towards a Prospective IoT Testing Framework

IoT Testing as a Service- IoT-TaaS, a novel service-based approach for an automated IoT testing framework aims to resolve constraints regarding coordination, costs, and scalability issues of traditional software testing in the context of standards-based development of IoT devices, is introduced and its design and implementation explored.

Towards a Staging Environment for the Internet of Things

  • Jossekin BeilharzPhilipp Wiesner L. Thamsen
  • Computer Science
    2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)
  • 2021
Marvis is presented, a framework that orchestrates hybrid testbeds, co-simulated domain environments, and a central network simulation for testing distributed IoT applications and preliminary results include an open source prototype and a demonstration of a Vehicle-to-everything (V2X) communication scenario.

EMU-IoT - A Virtual Internet of Things Lab

It is shown how easy and simple it is to set up experiments with EMU-IoT and the usefulness is demonstrated by conducting experiments in the lab and using virtualization and container technologies to demonstrate an experimentation infrastructure.

ELIoT: Design of an emulated IoT platform

The design of an IoT emulation platform (ELIoT) is presented, which aim on making the deployment of IoT test environments easier and more flexible, and exploiting the lightweight and flexible characteristics of emerging virtualization technologies, such as Docker containers, for emulating both simple and complex IoT devices.

UiTiOt v3: A Hybrid Testbed for Evaluation of Large-Scale IoT Networks

This paper introduces the third version of the UiTiOt testbed, which is actually a hybrid system which puts virtual wireless nodes together with real physical devices to carry out a variety of IoT experiments.

Simulators, Emulators, and Test-beds for Internet of Things: A Comparison

  • N. PatelB. MehtreR. Wankar
  • Computer Science
    2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)
  • 2019
This paper compares sixteen simulators, four emulators, and six test-beds for IoT on the basis of Scope, Type, Programming language, IoT layers, Scale of Operation, IoT Standards, API Integration, Cyber Resilience, Service Domain, and Security Measures to support researchers and developers for choosing suitable tools for their application.

Internet of Things (IoT): Research, Simulators, and Testbeds

A comparative analysis of existing simulation tools categorized based on the scope of coverage of the IoT architecture layers is presented, and existing large-scale IoT testbeds that have been adopted by researchers for examining the physical IoT prototypes are compared.

IOTSim: A simulator for analysing IoT applications