Time-travel Testing of Android Apps

@article{Dong2020TimetravelTO,
  title={Time-travel Testing of Android Apps},
  author={Zhen Dong and Marcel B{\"o}hme and Lucia Cojocaru and Abhik Roychoudhury},
  journal={2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)},
  year={2020},
  pages={481-492}
}
Android testing tools generate sequences of input events to exercise the state space of the app-under-test. Existing search-based techniques systematically evolve a population of event sequences so as to achieve certain objectives such as maximal code coverage. The hope is that the mutation of fit event sequences leads to the generation of even fitter sequences. However, the evolution of event sequences may be ineffective. Our key insight is that pertinent app states which contributed to the… 

Figures and Tables from this paper

TimeMachine: Time-travel Testing of Android Apps
TLDR
A prototype of an automated Android app testing technique that saves program states in the execution and restores an interesting state previously saved for further exploration when progress is slow so as to continuously explore new program behavior efficiently, which outperforms the-state-of-art techniques including Sapienz.
Automated, Cost-effective, and Update-driven App Testing
TLDR
This paper presents ATUA, a model-based approach that synthesizes App models with static analysis, integrates a dynamically-refined state abstraction function and combines complementary testing strategies, enabling ATUA to generate a small set of inputs that exercise only the code affected by the updates.
Preference-Wise Testing of Android Apps via Test Amplification
TLDR
In an experimental evaluation conducted on real-world Android apps, amplified test cases produced by Prefest from automatically generated test cases covered significantly more code of the apps and detected 7 real bugs, and the tool's test amplification time was at the same order of magnitude as the running time of the input test cases.
Automated Functional Fuzzing of Android Apps
TLDR
Independent view fuzzing is introduced, the first automated approach for detecting functional bugs in Android apps, to leverage the commonly-held independent view property of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties.
Benchmarking automated GUI testing for Android against real-world bugs
TLDR
This paper offers the first ground-truth empirical evaluation of automated GUI testing for Android, and identifies five major common challenges that these tools face, and reveals additional findings such as factors affecting these tools in bug finding and opportunities for tool improvements.
ProMal: Precise Window Transition Graphs for Android via Synergy of Program Analysis and Machine Learning
  • Changlin Liu
  • Computer Science
    2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)
  • 2021
TLDR
ProMal, a "tribrid" analysis that synergistically combines static analysis, dynamic analysis, and machine learning to construct a precise window transition graph (WTG), is proposed.
Vet: identifying and avoiding UI exploration tarpits
TLDR
Vet, a general approach including a supporting system for the given specific Android UI testing tool on the given given specific app under test (AUT), which identifies exploration tarpits by recognizing their patterns in the UI traces, enabling each evaluated tool to achieve higher code coverage and improve crash-triggering capabilities.
Speculating Ineffective UI Exploration via Trace Analysis
TLDR
Vet is presented as a general framework for applying the idea of trace analysis on UI testing history to identify ineffective exploration behavior for a given UI testing tool on a given app and enables enhancing the testing tool by guiding the exploration to avoid ineffective exploration.
SetDroid: Detecting User-Configurable Setting Issues of Android Apps via Metamorphic Fuzzing
  • Jingling Sun
  • Computer Science
    2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)
  • 2021
TLDR
This work designed and introduced, setting-wise metamorphic fuzzing, the first automated testing technique to overcome the oracle problem in detecting setting issues in Android apps, and applied it on 26 popular, open-source Android apps.
COSMO: Code Coverage Made Easier for Android
TLDR
COSMO is presented, the first fully automated Android app instrumenter publicly available that operates at the source code level in a completely transparent way, making it fully compatible with existing system level testing technologies and Android test generators.
...
1
2
3
...

References

SHOWING 1-10 OF 39 REFERENCES
Systematic execution of Android test suites in adverse conditions
TLDR
This work proposes a new methodology for testing by leveraging existing test suites such that each test case is systematically exposed to adverse conditions where certain unexpected events may interfere with the execution.
Guided, stochastic model-based GUI testing of Android apps
TLDR
The results show that the models produced by Stoat cover 17~31% more code than those by existing modeling tools; and Stoat detects 3X more unique crashes than two state-of-the-art testing tools, Monkey and Sapienz.
Reducing Combinatorics in GUI Testing of Android Applications
TLDR
This paper presents TrimDroid, a framework for GUI testing of Android apps that uses a novel strategy to generate tests in a combinatorial, yet scalable, fashion and is backed with automated program analysis and formally rigorous test generation engines.
EvoDroid: segmented evolutionary testing of Android apps
TLDR
EvoDroid overcomes a key shortcoming of using evolutionary techniques for system testing, i.e., the inability to pass on genetic makeup of good individuals in the search, and has the ability to achieve significantly higher code coverage than existing Android testing tools.
Android Testing via Synthetic Symbolic Execution
TLDR
This work builds a dynamic symbolic execution engine for Android apps, built on top of JDART, that shows that the library models obtained from program synthesis are often more accurate than the semi-manual models in JPF-Android, and could reach more branch targets, as compared to using the JPf-Android models.
Practical GUI Testing of Android Applications Via Model Abstraction and Refinement
TLDR
On 15 large, widely-used apps from the Google Play Store, APE outperforms the state-of-the-art Android GUI testing tools in terms of both testing coverage and the number of detected unique crashes.
An Empirical Study of Android Test Generation Tools in Industrial Cases
  • Wenyu Wang, Dengfeng Li, Tao Xie
  • Computer Science
    2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE)
  • 2018
TLDR
Monkey, a state-of-the-practice tool from Google, achieves the highest method coverage on 22 of 41 apps whose method coverage data can be obtained, and Stoat, astate-of the-art tool, is able to trigger the highest number of unique crashes on 23 apps.
Automated Test Input Generation for Android: Are We There Yet? (E)
TLDR
A thorough comparison of the main existing test input generation tools for Android is performed, evaluating the effectiveness of these tools, and their corresponding techniques, according to four metrics: ease of use, ability to work on multiple platforms, code coverage, and ability to detect faults.
Dynodroid: an input generation system for Android apps
TLDR
Dynodroid views an app as an event-driven program that interacts with its environment by means of a sequence of events through the Android framework, and monitors the reaction of an app upon each event in a lightweight manner, using it to guide the generation of the next event to the app.
Targeted and depth-first exploration for systematic testing of android apps
TLDR
The key insight of the approach is to use a static, taint-style, dataflow analysis on the app bytecode in a novel way, to construct a high-level control flow graph that captures legal transitions among activities (app screens).
...
1
2
3
4
...