• Corpus ID: 239009881

Identifying Similar Test Cases That Are Specified in Natural Language

  title={Identifying Similar Test Cases That Are Specified in Natural Language},
  author={Markos Viggiato and Dale Paas and Christian Buzon and Cor-Paul Bezemer},
Software testing is still a manual process in many industries, despite the recent improvements in automated testing techniques. As a result, test cases are often specified in natural language by different employees and many redundant test cases might exist in the test suite. This increases the (already high) cost of test execution. Manually identifying similar test cases is a time-consuming and error-prone task. Therefore, in this paper, we propose an unsupervised approach to identify similar… 

Figures and Tables from this paper


Clustering test steps in natural language toward automating test automation
The approach includes domain-specific word embedding training along with measurement based on Relaxed Word Mover’sDistance to analyze the similarity of test steps, and includes a technique to combine hierarchical agglomerative clustering and K-means clustering post-refinement to derive high-quality and manually-adjustable clustering results.
RTCM: a natural language based, automated, and practical test case generation framework
A TCS language, named as Restricted Test Case Modeling (RTCM), which is based on natural language and composed of an easy-to-use template, a set of restriction rules and keywords is proposed and a test case generation tool (aToucan4Test) is proposed, which takes TCSs in RTCM as input and generates either manual test cases or automatically executable test cases, based on various coverage criteria defined on R TCM.
Cluster-based test suite functional analysis
This work reports on the practical experience in automated analysis of real-world free text test suites from six different industrial companies, finding that the novel, cluster-based approach provides significant time savings for the analysis of the test suites, thus enabling functional analysis in many cases where manual analysis is infeasible in practice.
On Using k-means Clustering for Test Suite Reduction
This paper introduces a machine learning based algorithm for test suite reduction that combines k-means clustering with binary search and presents experimental results using small to larger Java programs with different types of inputs and outputs.
Automatic generation of system test cases from use case specifications
Use Case Modelling for System Tests Generation (UMTG), an approach that automatically generates executable system test cases from use case spec- ifications and a domain model, the latter including a class diagram and constraints.
Improving Test Execution Efficiency Through Clustering and Reordering of Independent Test Steps
By applying the proposed test case synthesis method in a case study at Mercedes-Benz Passenger Car Development, a test load reduction of 15% is observed due to removing redundant test steps and an additional reduction of at least 3% for rearranging test steps.
Automatic Generation of Acceptance Test Cases from Use Case Specifications: an NLP-based Approach
This paper presents UMTG, an approach that supports the generation of executable, system-level, acceptance test cases from requirements specifications in natural language, with the goal of reducing the manual effort required to generate test cases and ensuring requirements coverage.
Semantic analysis technique of logics retrieval for software testing from specification documents
A Semantic Analysis Technique of Logics Retrieval for Software Testing from Japanese Public Sector's Specification Documents is proposed, which is a new logics retrieval from harmonization between natural language processing technique and software testing.
A learning-to-rank based fault localization approach using likely invariants
This work proposes Savant, a new fault localization approach that employs a learning-to-rank strategy, using likely invariant diffs and suspiciousness scores as features, to rank methods based on their likelihood to be a root cause of a failure.
A Natural Language Programming Approach for Requirements-Based Security Testing
This paper proposes, applies and assess Misuse Case Programming (MCP), an approach that automatically generates security test cases from misuse case specifications (i.e., use case specifications capturing the behavior of malicious users).