• DOI:10.4204/EPTCS.259.2
• Corpus ID: 21893585

ACCBench: A Framework for Comparing Causality Algorithms

@inproceedings{Rehwald2017ACCBenchAF,
  title={ACCBench: A Framework for Comparing Causality Algorithms},
  author={Simon Rehwald and Amjad Ibrahim and Kristian Beckers and Alexander Pretschner},
  booktitle={CREST@ETAPS},
  year={2017}
}

• Simon Rehwald, Amjad Ibrahim, +1 author A. Pretschner
• Published in CREST@ETAPS 10 October 2017
• Computer Science

Modern socio-technical systems are increasingly complex. A fundamental problem is that
the borders of such systems are often not well-defined a-priori, which among other problems
can lead to unwanted behavior during runtime. Ideally, unwanted behavior should
be prevented. If this is not possible the system shall at least be able to help determine
potential cause(s) a-posterori, identify responsible parties and make them accountable for
their behavior. Recently, several algorithms addressing these concepts have been proposed.
However, the applicability of the corresponding approaches, specifically their effectiveness
and performance, is mostly unknown. Therefore, in this paper, we propose ACCBench, a
benchmark tool that allows to compare and evaluate causality algorithms under a consistent
setting. Furthermore, we contribute an implementation of the two causality algorithms
by [7] and [6] as well as of a policy compliance approach based on some concepts of [16].
Lastly, we conduct a case study of an Intelligent Door Control System, which exposes concrete
strengths and weaknesses of all algorithms under different aspects. In the course of
this, we show that the effectiveness of the algorithms in terms of cause detection as well as
their performance differ to some extent. In addition, our analysis reports on some qualitative
aspects that should be considered when evaluating each algorithm. For example, the
human effort needed to configure the algorithm and model the use case is analyzed.