Porta: Profiling Software Tutorials Using Operating-System-Wide Activity Tracing

  title={Porta: Profiling Software Tutorials Using Operating-System-Wide Activity Tracing},
  author={Alok Shankar Mysore and Philip J. Guo},
  journal={Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology},
  • A. Mysore, Philip J. Guo
  • Published 11 October 2018
  • Computer Science
  • Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
It can be hard for tutorial creators to get fine-grained feedback about how learners are actually stepping through their tutorials and which parts lead to the most struggle. To provide such feedback for technical software tutorials, we introduce the idea of tutorial profiling, which is inspired by software code profiling. We prototyped this idea in a system called Porta that automatically tracks how users navigate through a tutorial webpage and what actions they take on their computer such as… 

Figures and Tables from this paper

Docable: evaluating the executability of software tutorials
This work conducts an empirical study on software tutorials, finding a naive execution strategy achieves an overall executability rate of only 26% and a human-annotation-based execution strategy, while doubling executability, yields no tutorial that can successfully execute all steps.
Software Developers Learning Machine Learning: Motivations, Hurdles, and Desires
A survey to the website of the TensorFlow.js ML framework found that many developers wanted to learn ML for aspirational reasons rather than for immediate job needs, and desired frameworks to provide more basic ML conceptual support.
Detecting User-Perceived Failure in Mobile Applications via Mining User Traces
  • Deyu Tian
  • Computer Science
    2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)
  • 2021
The insight behind the algorithm is that when user-perceived failure occurs on an app page, the users will backtrack and revisit the certain page to retry, and the approach can achieve good detection performance on a dataset collected from real world users.
Comprehension analysis considering programming thinking ability using code puzzle
A method to estimate the learners’ understanding from the learner’s process to solve the code puzzles that require programming thinking as well as knowledge and showed that the proposed method could estimate with the accuracy of 80% or more.
Towards a dynamic multiscale personal information space: beyond application and document centered views of information
An international research community is catalyzed to rethink the nature of information as a basis for radically advancing the human-centered design of information-based work and helping to ensure the future is one of convivial, effective, and humane systems.
Towards a Dynamic Multiscale Personal Information Space
A new view of information systems is proposed, cognitive requirements for a human-centered information space are discussed, and an international research community is catalyzed to rethink the nature of information as a basis for radically advancing the human- centered design of information-based work.
Extraction of Poor Learning Items with Automatic Labeling in Fill-in-the-blank Test
A method to automate the systematic analysis of grading results in fill-in-the-blank tests by evaluates appropriateness for labels from learning histories of learners to find the most suitable labels for blanks.
Hybrid Microgenetic Analysis: Using Activity Codebooks to Identify and Characterize Creative Process
A design method is introduced that leverages time-series data (interaction logs, physical sensor, and biosignal data) to isolate unique actions and behaviors between groups of users to better understand tacit knowledge.
Segmentation ( A ) Sensing & Data Collection Multivariate Timeseries Matrix Process Chromatogram Codebook Algorithm ( C ) Qualitative Analysis
  • 2019


Torta: Generating Mixed-Media GUI and Command-Line App Tutorials Using Operating-System-Wide Activity Tracing
Torta is an end-to-end system that automatically generates step-by-step GUI and command-line app tutorials by demonstration, provides an editor to trim, organize, and add validation criteria to these tutorials, and provides a web-based viewer that can validate step-level progress and automatically run certain steps.
Codetrail: Connecting source code and web resources
  • Max Goldman, Rob Miller
  • Computer Science
    2008 IEEE Symposium on Visual Languages and Human-Centric Computing
  • 2008
Codetrail is presented, a system that demonstrates how the developerpsilas use of Web resources can be improved by connecting the Eclipse IDE and the Firefox Web browser.
HyperSource: bridging the gap between source and code-related web sites
Hyper source comprises a browser extension that logs visited pages; an IDE that tracks user activity and maps pages to code edits; a source document model that tracks visited pages at a character level; and a user interface that enables interaction with these histories.
ZIPT: Zero-Integration Performance Testing of Mobile App Designs
This paper introduces a design performance testing approach that leverages existing app implementations and crowd workers to enable comparative testing at scale and demonstrates that designers can use ZIPT's aggregate data and visualizations to understand the relative performance of interaction patterns found in the wild, and identify usability issues in existing Android apps.
Addressing misconceptions about code with always-on programming visualizations
Theseus, an IDE extension that visualizes run-time behavior within a JavaScript code editor, finds that users quickly adopted strategies that are unique to always-on, real-time visualizations, and used the additional information to guide their navigation through their code.
Bifröst: Visualizing and Checking Behavior of Embedded Systems across Hardware and Software
A new development environment designed to illuminate the boundary between embedded code and circuits called Bifröst, which automatically instruments and captures the progress of the user's code, variable values, and the electrical and bus activity occurring at the interface between the processor and the circuit it operates in.
Community enhanced tutorials: improving tutorials with multiple demonstrations
A user study found that users perform significantly better using the FollowUs system with a library of multiple demonstrations in comparison to its equivalent baseline system with only the original authored content.
Dynamic Instrumentation of Production Systems
DTrace features the ability to dynamically instrument both user-level and kernel-level software in a unified and absolutely safe fashion and features a C-like high-level control language to describe the predicates and actions at a given point of instrumentation.
ATOM: a system for building customized program analysis tools
ATOM (Analysis Tools with OM) is a single framework for building a wide range of customized program analysis tools and has been used to build a diverse set of tools for basic block counting, profiling, dynamic memory recording, instruction and data cache simulation, pipeline simulation, evaluating branch prediction, and instruction scheduling.
ERICA: Interaction Mining Mobile Apps
ERICA, a system that takes a scalable, human-computer approach to interaction mining existing Android apps without the need to modify them in any way, and collects interaction traces from over a thousand popular Android apps.