Effective test suites for mixed discrete-continuous stateflow controllers
@article{Matinnejad2015EffectiveTS, title={Effective test suites for mixed discrete-continuous stateflow controllers}, author={Reza Matinnejad and Shiva Nejati and Lionel Claude Briand and Thomas Bruckmann}, journal={Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering}, year={2015} }
Modeling mixed discrete-continuous controllers using Stateflow is common practice and has a long tradition in the embedded software system industry. Testing Stateflow models is complicated by expensive and manual test oracles that are not amenable to full automation due to the complex continuous behaviors of such models. In this paper, we reduce the cost of manual test oracles by providing test case selection algorithms that help engineers develop small test suites with high fault revealing…
Figures and Tables from this paper
25 Citations
Multi-objective black-box test case selection for cost-effectively testing simulation models
- Computer ScienceGECCO
- 2018
This paper proposes a cost-effective approach for test case selection that relies on black-box data related to inputs and outputs of the system and demonstrates that this approach managed to improve Random Search by 26% on average in terms of the Hypervolume quality indicator.
Pareto efficient multi-objective black-box test case selection for simulation-based testing
- Computer ScienceInf. Softw. Technol.
- 2019
Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior
- Computer ScienceIEEE Transactions on Software Engineering
- 2019
A black-box test generation approach, implemented based on meta-heuristic search, that aims to maximize diversity in test output signals generated by Simulink models, and a test prioritization algorithm to automatically rank test cases generated by the test generation algorithm according to their likelihood to reveal a fault.
Automated Test Suite Generation for Time-Continuous Simulink Models
- Computer Science2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE)
- 2016
This paper proposes a test generation approach that is applicable to Simulink models built for both purposes of simulation and code generation, and is implemented as a meta-heuristic search algorithm and is guided to produce test outputs with diverse shapes according to the proposed notion of diversity.
Effective fault localization of automotive Simulink models: achieving the trade-off between test oracle effort and fault localization accuracy
- Computer ScienceEmpirical Software Engineering
- 2018
A prediction model is developed to help stop test generation when adding test cases is unlikely to improve fault localization and is able to maintain almost the same fault localization accuracy while reducing the average number of newly generated test cases by more than half.
Improving fault localization for Simulink models using search-based testing and prediction models
- Computer Science2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER)
- 2017
This paper identifies three test objectives that aim to increase test suite diversity and uses a search-based algorithm to generate diversified but small test suites, and develops a prediction model to stop test generation when adding test cases is unlikely to improve fault localization.
Faster Multi-Goal Simulation-Based Testing Using DoLesS (Domination with Least Square Approximation)
- Computer ScienceArXiv
- 2021
DoLesS (Domination with Least Squares Approximation) is a fast method to find minimal test suites for multi-goal cyberphysical systems and performs as well or even better as state-of-the-art, while running 80-360 times faster on average.
SimCoTest: A Test Suite Generation Tool for Simulink/Stateflow Controllers
- Computer Science2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C)
- 2016
SimCoTest uses meta-heuristic search to maximize the likelihood of presence of specific failure patterns in output signals and to maximize diversity of output signal shapes, and is applicable to Simulink/Stateflow models in their entirety.
Seeding strategies for multi-objective test case selection: an application on simulation-based testing
- Computer ScienceGECCO
- 2020
The authors' evaluation suggests that these strategies are indeed helpful for solving the multi-objective test case selection problem, and two of the proposed seeding strategies outperformed the NSGA-II algorithm without seeding population with statistical significance for 92.8 and 96% of the problems.
Profiting from Unit Tests for Integration Testing
- Computer Science2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)
- 2016
OutFIT tool targets superfluous or missing functionality and untested exception/fault handling in Matlab Simulink models and generated code and is effective in finding a potential defect, and efficiently analyzes all evaluated components within a worst case execution time of 110 minutes.
References
SHOWING 1-10 OF 60 REFERENCES
Automated Model-in-the-Loop Testing of Continuous Controllers Using Search
- Computer ScienceSSBSE
- 2013
This paper identifies a set of common requirements characterizing the desired behavior of continuous controllers, and develops a search-based technique to automatically generate test cases for these requirements, which generates better test cases faster than random test case generation.
Prototype-based tests for hybrid reactive systems
- Computer Science14th IEEE International Workshop on Rapid Systems Prototyping, 2003. Proceedings.
- 2003
It is argued that rapid prototyping approaches fit well with the use of models that serve as specifications, as a basis for test case generation, or as a base for production code generation.
An integrated test generation tool for enhanced coverage of Simulink/Stateflow models
- Computer Science2012 Design, Automation & Test in Europe Conference & Exhibition (DATE)
- 2012
This paper discusses SmartTestGen and the different test generation techniques utilized - random testing, constraint solving, model checking and heuristics, and compared the performance of the tool with that of two prominent commercial tools.
Randomized directed testing (REDIRECT) for Simulink/Stateflow models
- Computer ScienceEMSOFT '08
- 2008
RedIRECT (RandomizEd DIRECted Testing), the proposed testing method, uses a careful combination of heuristics and certain combinations of random and directed testing to achieve better coverages as compared to the existing commercial tools.
Automatic test case generation from Simulink/Stateflow models using model checking
- Computer ScienceSoftw. Test. Verification Reliab.
- 2014
This paper outlines the test generation methodology of AutoMOTGen and gives a comparative study with a commercial, primarily random input‐based, test generation tool on the same set of examples, indicating that in terms of coverage, model checking‐based techniques complement the random input-based techniques.
Search-based automated testing of continuous controllers: Framework, tool support, and case studies
- Computer ScienceInf. Softw. Technol.
- 2015
Specification-based test oracles for reactive systems
- Computer ScienceInternational Conference on Software Engineering
- 1992
It is argued that test oracles should be derived from specifications in conjunction with testing criteria, represented in a common form, and their use made integral to the test- ing process.
Empirical Investigation of the Effects of Test Suite Properties on Similarity-Based Test Case Selection
- Computer Science2011 Fourth IEEE International Conference on Software Testing, Verification and Validation
- 2011
The ideal situation in which a similarity-based selection works best is identified, which is useful for devising more effective similarity functions, and a solution based on rank scaling is proposed to alleviate this problem.
The Oracle Problem in Software Testing: A Survey
- Computer ScienceIEEE Transactions on Software Engineering
- 2015
This paper provides a comprehensive survey of current approaches to the test oracle problem and an analysis of trends in this important area of software testing research and practice.
Formal Semantics and Analysis Methods for Simulink Stateflow Models
- Computer Science
- 2001
The formal semantics of a fragment of Stateflow based on a modular representation called communicating pushdown automata is described and it is shown how this semantics can be used to analyze simple properties of State flow models.