Multi-objectivizing software configuration tuning

@article{Chen2021MultiobjectivizingSC,
  title={Multi-objectivizing software configuration tuning},
  author={Tao Chen and Miqing Li},
  journal={Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  year={2021}
}
  • Tao Chen, Miqing Li
  • Published 31 May 2021
  • Computer Science
  • Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
Automatically tuning software configuration for optimizing a single performance attribute (e.g., minimizing latency) is not trivial, due to the nature of the configuration systems (e.g., complex landscape and expensive measurement). To deal with the problem, existing work has been focusing on developing various effective optimizers. However, a prominent issue that all these optimizers need to take care of is how to avoid the search being trapped in local optima β€” a hard nut to crack for… 

Figures and Tables from this paper

MMO: Meta Multi-Objectivization for Software Configuration Tuning
TLDR
A meta multi-objectivization (MMO) model is proposed that considers an auxiliary performance objective (e.g., throughput in addition to latency) and is used to make similarly-performing while different configurations less comparable, thus preventing the search from being trapped in local optima.
Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes
  • Jing Gong, Tao-An Chen
  • Computer Science
    2022 IEEE/ACM 19th International Conference on Mining Software Repositories (MSR)
  • 2022
TLDR
This paper empirically compared the widely used encoding schemes for software performance learning, namely label, scaled label, and one-hot encoding, to provide a better understanding of which is better and under what circumstances for the community.
Planning Landscape Analysis for Self-Adaptive Systems
  • Tao Chen
  • Sociology
    2022 International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)
  • 2022
To assure performance on the fly, planning is arguably one of the most important steps for self-adaptive systems (SASs), especially when they are highly configurable with a daunting number of…
The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software Engineering
TLDR
The key finding is that weighted search reaches a certain level of solution quality by consuming relatively less resources at the early stage of the search; however, Pareto search is at the majority of the time significantly better than its weighted counterpart, as long as the authors allow a sufficient, but not unrealistic search budget.
Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?
  • Tay-Yu Chen
  • Computer Science
    2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
  • 2022
TLDR
Experimental results on three real-world SASs show that the concept of explicitly handling dynamic as part of the search in the SAS planning is effective, as LiDOS outperforms its stationary counterpart overall with up to 10Γ— improvement.

References

SHOWING 1-10 OF 72 REFERENCES
Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm
TLDR
This paper proposes an approach for automatically finding an optimal ORM configuration using a multi-objective genetic algorithm and finds that this approach finds near-optimal configurations in 360-450 seconds for PetClinic and in 9-12 hours for ZK.
Finding Faster Configurations Using FLASH
TLDR
Flash is introduced, a sequential model-based method that sequentially explores the configuration space by reflecting on the configurations evaluated so far to determine the next best configuration to explore, which reduces the effort required to find the better configuration.
BestConfig: tapping the performance potential of systems via automatic configuration tuning
TLDR
Best Config is a system for automatically finding a best configuration setting within a resource limit for a deployed system under a given application workload and is designed with an extensible architecture to automate the configuration tuning for general systems.
MRONLINE: MapReduce online performance tuning
TLDR
This work proposes an online performance tuning system that monitors a job's execution, tunes associated performance-tuning parameters based on collected statistics, and provides fine-grained control over parameter configuration, and designs a gray-box based smart hill climbing algorithm that can efficiently converge to a near-optimal configuration with high probability.
Evaluating the Role of Optimization-Specific Search Heuristics in Effective Autotuning ?
TLDR
This work presents a transformation-aware (TA) search algorithm which effectively combines optimization-specific heuristics to explore the configuration space of six optimizations, parallelization via OpenMP, cache blocking, array copying, unrolland-jam, scalar replacement, and loop unrolling, to improve the multithreading, memory system and microprocessor performance of three linear algebra routines.
Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software
TLDR
This study study of configuration settings of real-world users, including thousands of customers of a commercial storage system, and hundreds of users of two widely-used open-source system software projects, reveals a series of interesting findings to motivate software architects and developers to be more cautious and disciplined in configuration design.
Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction
TLDR
This study proposes and investigates two alternatives to the weighted sum scalarization and investigates the multi-objectivization, which decomposes the fitness function into several optimization objectives as an attempt to increase test case diversity.
Taming parallel I/O complexity with auto-tuning
  • Babak Behzad, Huong Luu, M. Snir
  • Computer Science
    2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC)
  • 2013
TLDR
An auto-tuning system that uses a genetic algorithm to search a large space of tunable parameters and to identify effective settings at all layers of the parallel I/O stack that substantially improved write performance over default system settings is presented.
ParamILS: An Automatic Algorithm Configuration Framework
TLDR
An automatic framework for this algorithm configuration problem is described and methods for optimizing a target algorithm's performance on a given class of problem instances by varying a set of ordinal and/or categorical parameters are provided.
Automatic Configuration of Multi-objective Optimizers and Multi-objective Configuration
TLDR
This chapter reviews two main aspects of research on automatic configuration of multi-objective optimizers and the possibility of using multiple performance metrics for the evaluation of algorithm configurations, and details a case study that shows how these approaches can be used to design new, high-performing multi- objective evolutionary algorithms.
...
...