GLIB: towards automated test oracle for graphically-rich applications

  title={GLIB: towards automated test oracle for graphically-rich applications},
  author={Ke Chen and Yufei Li and Yingfeng Chen and Changjie Fan and Zhipeng Hu and Wei Yang},
  journal={Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  • Ke Chen, Yufei Li, +3 authors Wei Yang
  • Published 19 June 2021
  • Computer Science
  • Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
Graphically-rich applications such as games are ubiquitous with attractive visual effects of Graphical User Interface (GUI) that offers a bridge between software applications and end-users. However, various types of graphical glitches may arise from such GUI complexity and have become one of the main component of software compatibility issues. Our study on bug reports from game development teams in NetEase Inc. indicates that graphical glitches frequently occur during the GUI rendering and… Expand


Improving random GUI testing with image-based widget detection
This work proposes a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques and provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. Expand
Reducing Combinatorics in GUI Testing of Android Applications
This paper presents TrimDroid, a framework for GUI testing of Android apps that uses a novel strategy to generate tests in a combinatorial, yet scalable, fashion and is backed with automated program analysis and formally rigorous test generation engines. Expand
Automated model-based Android GUI testing using multi-level GUI comparison criteria
  • Young-Min Baek, Doo-Hwan Bae
  • Computer Science
  • 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)
  • 2016
A set of multi-level GUI Comparison Criteria (GUICC) that provides the selection of multiple abstraction levels for GUI model generation and can alleviate the inherent state explosion problems of existing a single-level GuICC for behavior modeling of real-world Android apps by flexibly manipulating GUICC. Expand
Owl Eyes: Spotting UI Display Issues via Visual Understanding
This work proposes a novel approach, OwlEye, based on deep learning for modelling visual information of the GUI screenshot, which can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. Expand
Guided, stochastic model-based GUI testing of Android apps
The results show that the models produced by Stoat cover 17~31% more code than those by existing modeling tools; and Stoat detects 3X more unique crashes than two state-of-the-art testing tools, Monkey and Sapienz. Expand
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
This paper presents an approach that automates this process by enabling accurate prototyping of GUIs via three tasks: detection, classification, and assembly, and implemented this approach for Android in a system called ReDraw. Expand
GUIFetch: Supporting App Design and Development through GUI Search
  • Farnaz Behrang, S. Reiss, A. Orso
  • Computer Science
  • 2018 IEEE/ACM 5th International Conference on Mobile Software Engineering and Systems (MOBILESoft)
  • 2018
GUIFetch is a technique that takes as input the sketch for an app and leverages the growing number of open source apps in public repositories to identify apps with GUIs and transitions that are similar to those in the provided sketch. Expand
Seeking the user interface
A system that uses code search to simplify and automate the exploration of Java-based graphical user interface solutions and lets programmers interact with the matched solutions and return the running code for the solutions they choose. Expand
Seenomaly: Vision-Based Linting of GUI Animation Effects Against Design-Don't Guidelines
This work proposes an unsupervised, computer-vision based adversarial autoencoder that learns to group similar GUI animations by “seeing” lots of unlabeled real-application GUI animations and learning to generate them, and builds the datasets of synthetic and realworld GUI animations. Expand
Reverse Engineering Mobile Application User Interfaces with REMAUI (T)
  • T. Nguyen, C. Csallner
  • Computer Science
  • 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE)
  • 2015
The first technique to automatically Reverse Engineer Mobile Application User Interfaces (REMAUI) is introduced, which identifies user interface elements such as images, texts, containers, and lists, via computer vision and optical character recognition (OCR) techniques. Expand