Users beware: preference inconsistencies ahead

@article{Behrang2015UsersBP,
  title={Users beware: preference inconsistencies ahead},
  author={Farnaz Behrang and Myra B. Cohen and Alessandro Orso},
  journal={Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering},
  year={2015}
}
The structure of preferences for modern highly-configurable software systems has become extremely complex, usually consisting of multiple layers of access that go from the user interface down to the lowest levels of the source code. This complexity can lead to inconsistencies between layers, especially during software evolution. For example, there may be preferences that users can change through the GUI, but that have no effect on the actual behavior of the system because the related source… 

Figures and Tables from this paper

Software Configuration Engineering in Practice Interviews, Survey, and Systematic Literature Review
TLDR
It is found that developers face a variety of challenges for all nine configuration engineering activities, starting from the creation of options, which generally is not planned beforehand and increases the complexity of a software system, to the non-trivial comprehension and debugging of configurations, and ending with the risky maintenance of configuration options, since developers avoid touching and changing configuration options in a mature system.
What should your run-time configuration framework do to help developers?
TLDR
A prototyping approach is used to derive and empirically evaluate requirements for tool support able to deal with 13 configuration engineering challenges, and finds that a configuration framework satisfying the four requirements enables developers to perform more accurately and more swiftly in 70% and 60% of the configuration engineering tasks than a state-of-the-practice framework not satisfying the requirements.
ORPLocator: Identifying Read Points of Configuration Options via Static Analysis
TLDR
This work proposes an approach for detection of inconsistencies between source code and documentation based on static analysis, which automatically identifies source code locations where options are read, and for each such location retrieves the name of the option.
STAD: Stack Trace Based Automatic Software Misconfiguration Diagnosis via Value Dependency Graph
TLDR
STAD can successfully diagnose all misconfigurations with less average number of false positives compared with existing approaches and does not require software crash reproduction, and users do not need to provide configuration options and their option read points, making debugging more efficient.
Does the Choice of Configuration Framework Matter for Developers? Empirical Study on 11 Java Configuration Frameworks
TLDR
This paper analyzes almost 2,000 Java projects on GitHub to investigate the features and properties of 11 major Java configuration frameworks and tries to identify links between the maintenance effort involved with the usage of these frameworks and the frameworks' properties.
ConfGuru - A System for Fully Automated Debugging of Configuration Errors
  • A. Andrzejak, Matthias Iacsa
  • Computer Science
    2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)
  • 2018
TLDR
The evaluation shows that ConfGuru can diagnose configuration errors and extract ORPs from a variety of applications with an accuracy matching previous semi-automated approaches, and has low response time, making it suitable as a practical configuration error diagnosis tool or a service for real-world scenarios.
Software Variability Through C++ Static Polymorphism
TLDR
This paper shows how static polymorphism can be used to implement variability, through a case study of IBM's open-source Eclipse OMR project, and discusses the current open problems and challenges this variability implementation mechanism raises and highlights technology gaps for reasoning about variability in OMR.
ConfEx: Towards Automating Software Configuration Analytics in the Cloud
TLDR
This paper introduces ConfEx, a framework that enables discovery and extraction of text-based configurations in multi-tenant cloud platforms and cloud image repositories for configuration analysis and validation and demonstrates a use case of ConfEx for detecting injected misconfigurations via outlier analysis.
Understanding and discovering software configuration dependencies in cloud and datacenter systems
TLDR
The results show that configuration dependencies are more prevalent and diverse than previously reported and should henceforth be considered a first-class issue in software configuration engineering.
Characterizing and Detecting Configuration Compatibility Issues in Android Apps
TLDR
A systematic study by analyzing 196 real-world issues collected from 43 popular apps identified common patterns of Android framework code changes that induce configurations compatibility issues and proposes CONFDROID that can automatically extract rules for detecting configuration compatibility issues.
...
1
2
...

References

SHOWING 1-10 OF 40 REFERENCES
Which configuration option should I change?
TLDR
This paper presents a technique (and its tool implementation) to troubleshoot configuration errors caused by software evolution, called ConfSuggester, which uses dynamic profiling, execution trace comparison, and static analysis to link the undesired behavior to its root cause.
Automated diagnosis of software configuration errors
TLDR
This work presents a technique (and its tool implementation, called ConfDiagnoser) to identify the root cause of a configuration error - a single configuration option that can be changed to produce desired behavior.
iTree: Efficiently Discovering High-Coverage Configurations Using Interaction Trees
TLDR
The improved iTree algorithm is highly scalable and can identify a high-coverage test set of configurations more effectively than existing methods, and the key improvements are based on the use of composite proto-interactions - a construct that improves iTree's ability to correctly learn key configuration option combinations.
Configurations everywhere: implications for testing and debugging in practice
TLDR
This paper analyzes a highly-configurable industrial application and two open source applications in order to quantify the true challenges that configurability creates for software testing and debugging, finding that all three applications consist of multiple programming languages.
PrefFinder: getting the right preference in configurable software systems
TLDR
PrefFinder is a natural language framework that finds (and changes) user preferences tied into an application's preference system and static documentation and can potentially save days or even weeks of time.
Mining configuration constraints: static analyses and empirical results
TLDR
An approach which uses build-time errors and a novel feature-effect heuristic to automatically extract configuration constraints from C code and argues that this approach, tooling, and experimental results support researchers and practitioners working on variability model re-engineering, evolution, and consistency-checking techniques.
Precomputing possible configuration error diagnoses
  • A. Rabkin, R. Katz
  • Computer Science
    2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011)
  • 2011
TLDR
This work builds a map from each program point to the options that might cause an error at that point, which reduces the number of false positives by nearly a factor of four for Hadoop, at the cost of approximately one minute's work per unique query.
SPLat: lightweight dynamic analysis for reducing combinatorics in testing configurable systems
TLDR
SPLat is presented, a new way to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables, which achieves an optimal reduction in the number of configurations.
Covering arrays for efficient fault characterization in complex configuration spaces
TLDR
The results strongly suggest that sampling via covering arrays allows us to characterize option-related failures nearly as well as if the authors had tested exhaustively, but at a much lower cost.
Configuration selection using code change impact analysis for regression testing
TLDR
This work proposes a configuration selection approach for regression testing that discards 15% to 60% of configurations as redundant, and saves 20% to 55% of the regression testing time, while retaining the same fault detection capability and code coverage of the retest-all approach.
...
1
2
3
4
...