Effective test suites for mixed discrete-continuous stateflow controllers

@article{Matinnejad2015EffectiveTS,
  title={Effective test suites for mixed discrete-continuous stateflow controllers},
  author={Reza Matinnejad and Shiva Nejati and Lionel Claude Briand and Thomas Bruckmann},
  journal={Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering},
  year={2015}
}
Modeling mixed discrete-continuous controllers using Stateflow is common practice and has a long tradition in the embedded software system industry. Testing Stateflow models is complicated by expensive and manual test oracles that are not amenable to full automation due to the complex continuous behaviors of such models. In this paper, we reduce the cost of manual test oracles by providing test case selection algorithms that help engineers develop small test suites with high fault revealing… 
View on ACM
orbilu.uni.lu
25 Citations
Highly Influential Citations
2
Background Citations
9
Methods Citations
9
Results Citations
2

25 Citations

Citation Type

Citation Type

Multi-objective black-box test case selection for cost-effectively testing simulation models
TLDR
This paper proposes a cost-effective approach for test case selection that relies on black-box data related to inputs and outputs of the system and demonstrates that this approach managed to improve Random Search by 26% on average in terms of the Hypervolume quality indicator.
Pareto efficient multi-objective black-box test case selection for simulation-based testing
Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior
TLDR
A black-box test generation approach, implemented based on meta-heuristic search, that aims to maximize diversity in test output signals generated by Simulink models, and a test prioritization algorithm to automatically rank test cases generated by the test generation algorithm according to their likelihood to reveal a fault.
Automated Test Suite Generation for Time-Continuous Simulink Models
TLDR
This paper proposes a test generation approach that is applicable to Simulink models built for both purposes of simulation and code generation, and is implemented as a meta-heuristic search algorithm and is guided to produce test outputs with diverse shapes according to the proposed notion of diversity.
Effective fault localization of automotive Simulink models: achieving the trade-off between test oracle effort and fault localization accuracy
TLDR
A prediction model is developed to help stop test generation when adding test cases is unlikely to improve fault localization and is able to maintain almost the same fault localization accuracy while reducing the average number of newly generated test cases by more than half.
Improving fault localization for Simulink models using search-based testing and prediction models
  • Bing Liu, Lucia, S. Nejati, L. Briand
  • Computer Science
    2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER)
  • 2017
TLDR
This paper identifies three test objectives that aim to increase test suite diversity and uses a search-based algorithm to generate diversified but small test suites, and develops a prediction model to stop test generation when adding test cases is unlikely to improve fault localization.
Faster Multi-Goal Simulation-Based Testing Using DoLesS (Domination with Least Square Approximation)
TLDR
DoLesS (Domination with Least Squares Approximation) is a fast method to find minimal test suites for multi-goal cyberphysical systems and performs as well or even better as state-of-the-art, while running 80-360 times faster on average.
SimCoTest: A Test Suite Generation Tool for Simulink/Stateflow Controllers
TLDR
SimCoTest uses meta-heuristic search to maximize the likelihood of presence of specific failure patterns in output signals and to maximize diversity of output signal shapes, and is applicable to Simulink/Stateflow models in their entirety.
Seeding strategies for multi-objective test case selection: an application on simulation-based testing
TLDR
The authors' evaluation suggests that these strategies are indeed helpful for solving the multi-objective test case selection problem, and two of the proposed seeding strategies outperformed the NSGA-II algorithm without seeding population with statistical significance for 92.8 and 96% of the problems.
Profiting from Unit Tests for Integration Testing
TLDR
OutFIT tool targets superfluous or missing functionality and untested exception/fault handling in Matlab Simulink models and generated code and is effective in finding a potential defect, and efficiently analyzes all evaluated components within a worst case execution time of 110 minutes.
...
1
2
3
...

References

SHOWING 1-10 OF 60 REFERENCES
Automated Model-in-the-Loop Testing of Continuous Controllers Using Search
TLDR
This paper identifies a set of common requirements characterizing the desired behavior of continuous controllers, and develops a search-based technique to automatically generate test cases for these requirements, which generates better test cases faster than random test case generation.
Prototype-based tests for hybrid reactive systems
TLDR
It is argued that rapid prototyping approaches fit well with the use of models that serve as specifications, as a basis for test case generation, or as a base for production code generation.
An integrated test generation tool for enhanced coverage of Simulink/Stateflow models
TLDR
This paper discusses SmartTestGen and the different test generation techniques utilized - random testing, constraint solving, model checking and heuristics, and compared the performance of the tool with that of two prominent commercial tools.
Randomized directed testing (REDIRECT) for Simulink/Stateflow models
TLDR
RedIRECT (RandomizEd DIRECted Testing), the proposed testing method, uses a careful combination of heuristics and certain combinations of random and directed testing to achieve better coverages as compared to the existing commercial tools.
Automatic test case generation from Simulink/Stateflow models using model checking
TLDR
This paper outlines the test generation methodology of AutoMOTGen and gives a comparative study with a commercial, primarily random input‐based, test generation tool on the same set of examples, indicating that in terms of coverage, model checking‐based techniques complement the random input-based techniques.
Search-based automated testing of continuous controllers: Framework, tool support, and case studies
Specification-based test oracles for reactive systems
TLDR
It is argued that test oracles should be derived from specifications in conjunction with testing criteria, represented in a common form, and their use made integral to the test- ing process.
Empirical Investigation of the Effects of Test Suite Properties on Similarity-Based Test Case Selection
  • H. Hemmati, A. Arcuri, L. Briand
  • Computer Science
    2011 Fourth IEEE International Conference on Software Testing, Verification and Validation
  • 2011
TLDR
The ideal situation in which a similarity-based selection works best is identified, which is useful for devising more effective similarity functions, and a solution based on rank scaling is proposed to alleviate this problem.
The Oracle Problem in Software Testing: A Survey
TLDR
This paper provides a comprehensive survey of current approaches to the test oracle problem and an analysis of trends in this important area of software testing research and practice.
Formal Semantics and Analysis Methods for Simulink Stateflow Models
TLDR
The formal semantics of a fragment of Stateflow based on a modular representation called communicating pushdown automata is described and it is shown how this semantics can be used to analyze simple properties of State flow models.
...
1
2
3
4
5
...