L. Bulej

In this paper we study the impact of CPU pinning on performance interference and energy efficiency for pairs of collocated workloads. Expand

We show that this approach ignores an inherent and unavoidable nondeterminism in the initial state of the system that is evaluated, often leading to an implausible estimate of result precision. Expand

This paper is motivated by two observations, one related to performance testing and one to software testing activities in general: – Software performance testing is widely recognized as an essential software quality assurance tool. Expand

We present Renaissance, a new benchmark suite composed of modern, real-world, concurrent, and object-oriented workloads that exercise various concurrency primitives of the JVM. Expand

We point out common causes of such imprecision related to the gathering of timing information and the effects of warm-up, randomization, cross talk and delayed or hidden functionality that can distort the results of the evaluation. Expand

We show how to tackle the problem of nondeterminism inherent to contemporary computer systems and execution environments with statistical methods. Expand

We investigate the difference between the behavior of a real garbage collector implementation and a simplified model on a selection of workloads, focusing on the accuracy achievable with particular input information (sizes, references, lifetimes). Expand

The paper outlines the concept of regression benchmarking as a variant of regression testing focused at detecting performance regressions. Expand

We use various metrics to characterize the dynamic behavior of a variety of programs written in three dynamic languages (Clojure, Python, and Ruby) executing on the JVM. Expand

We propose a method to estimate the impact of code changes with additional allocations on garbage collection time, which can be applied, e.g., when assessing the overall performance impact of alternative changes. Expand