Directed Automated Memory Performance Testing

@inproceedings{Chattopadhyay2017DirectedAM,
  title={Directed Automated Memory Performance Testing},
  author={Sudipta Chattopadhyay},
  booktitle={TACAS},
  year={2017}
}
Understanding software non-functional properties e.g. time, energy and security requires deep understanding of the execution platform. The design of caches plays a crucial role in impacting software performance for low latency of caches and software security for cache being used as a side channel. We present CATAPULT, a novel test generation framework to systematically explore the cache behaviour of an arbitrary program. Our framework leverages dynamic symbolic execution and satisfiability… 
Testing Cache Side-Channel Leakage
TLDR
The proposed test generation methodology is a method that systematically explores the program input space and it adapts based on the observed cache performance in the executed tests, which effectively reveals cache side-channel leakage in such real-world programs.
Exposing cache timing side-channel leaks through out-of-order symbolic execution
TLDR
A symbolic execution-based technique, named SymO3, for exposing cache timing leaks under the context of out-of-order execution, and finds that, in general, program transformation from compiler optimizations shrink the surface to timing leaks.
SPECUSYM: Speculative Symbolic Execution for Cache Timing Leak Detection
TLDR
This work proposes a new symbolic execution based method, Specusym, for precisely detecting cache timing leaks introduced by speculative execution and implemented SpecuSym atop KLEE, which successfully detected from 2 to 61 leaks in 6 programs under 3 different cache settings.
Eliminating timing side-channel leaks using program repair
TLDR
The method is implemented in LLVM and validated on a large set of applications, which are cryptographic libraries with 19,708 lines of C/C++ code in total, and ensures that the number of CPU cycles taken to execute any path is independent of the secret data.
An exploration of effective fuzzing for side‐channel cache leakage
TLDR
This paper proposes a test‐generation methodology, which, in both timing‐based and access‐based dimensions, systematically discovers the cache side‐channel leakage of an arbitrary software program.
Adversarial symbolic execution for detecting concurrency-related cache timing leaks
TLDR
This work develops a new method, named adversarial symbolic execution, that systematically explores both the feasible program paths and their interleavings while modeling the cache, and leverages an SMT solver to decide if there are timing leaks.
Quantifying the Information Leakage in Cache Attacks via Symbolic Execution
TLDR
At the core of CHALICE is a novel approach to quantify information leakage that can highlight critical cache side-channel leakage on arbitrary binary code.
Efficient Symbolic Execution of Concurrent Software
TLDR
An Assertion Guided Symbolic Execution method to identify and eliminate redun-related executions in AGSE, which is complementary to standard partial order reduction techniques in that it relies on property-specific information to reduce the state space.
Path cost analysis for side channel detection
Side-channels have been increasingly demonstrated as a practical threat to the confidentiality of private user information. Being able to statically detect these kinds of vulnerabilites is a key
Automated Performance Testing Based on Active Deep Learning
TLDR
An automated test generation method called ACTA for black-box performance testing based on active learning, which means that it does not require a large set of historical test data to learn about the performance characteristics of the system under test, and dynamically chooses the tests to execute using uncertainty sampling.
...
...

References

SHOWING 1-10 OF 17 REFERENCES
Automatic Quantification of Cache Side-Channels
TLDR
A novel method for automatically deriving upper bounds on the amount of information about the input that an adversary can extract from a program by observing the CPU's cache behavior by using a novel technique for efficient counting of concretizations of abstract cache states.
MESS: Memory Performance Debugging on Embedded Multi-core Systems
TLDR
MESS systematically discovers the order of memory-access operations that expose performance bugs due to shared caches, and proposes an approximate solution that dramatically reduces debugging time, at the cost of a reasonable amount of false positives.
The worst-case execution-time problem—overview of methods and survey of tools
TLDR
Different approaches to the determination of upper bounds on execution times are described and several commercially available tools1 and research prototypes are surveyed.
Quantifying the Information Leak in Cache Attacks through Symbolic Execution
TLDR
At the core of CHALICE is a novel approach to quantify information leak that can highlight critical cache side-channel leaks on arbitrary binary code.
CacheAudit: A Tool for the Static Analysis of Cache Side Channels
TLDR
The results obtained exhibit the influence of cache size, line size, associativity, replacement policy, and coding style on the security of the executables and include the first formal proofs of security for implementations with countermeasures such as preloading and data-independent memory access patterns.
Automated software testing of memory performance in embedded GPUs
TLDR
GUPT, a novel test generation framework that systematically explores and detects poor memory performance of applications running on embedded GPUs, is proposed and the efficacy of the framework is suggested by exposing numerous memory-performance issues in a reasonable time.
Toddler: Detecting performance problems via similar memory-access patterns
TLDR
Toddler is presented, a novel automated oracle for performance bugs, which enables testing for performance Bugs to use the well established and automated process of testing for functional bugs, and is implemented for Java.
Understanding and detecting real-world performance bugs
Developers frequently use inefficient code sequences that could be fixed by simple patches. These inefficient code sequences can cause significant performance degradation and resource waste, referred
Fast and Precise WCET Prediction by Separated Cache and Path Analyses
TLDR
This paper shows how the microarchitecture analysis can be separated from the path analysis in order to make the overall analysis fast and shows that the approach can be used to analyse executables created by a standard optimising compiler.
SimpleScalar: An Infrastructure for Computer System Modeling
TLDR
The SimpleScalar tool set provides an infrastructure for simulation and architectural modeling that can model a variety of platforms ranging from simple unpipelined processors to detailed dynamically scheduled microarchitectures with multiple-level memory hierarchies.
...
...