On the Road to Benchmarking BPMN 2.0 Workflow Engines

@article{Skouradaki2015OnTR,
  title={On the Road to Benchmarking BPMN 2.0 Workflow Engines},
  author={Marigianna Skouradaki and Dieter Roller and Frank Leymann and Vincenzo Ferme and Cesare Pautasso},
  journal={Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering},
  year={2015}
}
  • M. Skouradaki, D. Roller, C. Pautasso
  • Published 28 January 2015
  • Computer Science
  • Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering
Workflow Management Systems (WfMSs) provide platforms for delivering complex service-oriented applications that need to satisfy enterprise-grade quality of service requirements such as dependability and scalability. In this paper we focus on the case of benchmarking the performance of the core of WfMSs, Workflow Engines, that are compliant with the Business Process Model and Notation 2.0 (BPMN 2.0) standard. We first explore the main challenges that need to be met when designing such a… 
View on ACM
www2.informatik.uni-stuttgart.de
32 Citations
Highly Influential Citations
3
Background Citations
15
Methods Citations
12

Figures from this paper

32 Citations

Towards Workflow Benchmarking: Open Research Challenges

TLDR
The case for benchmarking the performance of WfMSs that are compliant with the Business Process Model and Notation 2.0 standard is made and most of the challenges that one must tackle when constructing such benchmarks are explored.

A Framework for Benchmarking BPMN 2.0 Workflow Management Systems

TLDR
This paper presents a framework for automatic and reliable calculation of performance metrics for BPMN 2.0 WfMSs by pinpointing performance bottlenecks, and aims to empower end users to make informed decisions when selecting a WFMS.

Lessons Learned from Evaluating Workflow Management Systems

TLDR
The findings of two research initiatives that deal with WfMSs conformance and performance benchmarking are synthesized to distill a set of lessons learned and best practices and provide useful advice for practitioners who plan to evaluate and use WFMSs and for Wf MS vendors that would like to foster wider adoption of process-centric service composition middleware.

Micro-Benchmarking BPMN 2.0 Workflow Management Systems with Workflow Patterns

TLDR
Experiments on a mix of workflow patterns indicated that there are no unexpected performance side effects when executing different workflow patterns concurrently, although the duration of the individual workflows that comprised the mix was increased, and the first micro-benchmark for WfMSs that can execute BPMN 2.0 workflows is proposed.

IT-Centric Process Automation: Study About the Performance of BPMN 2.0 Engines

TLDR
This chapter presents the challenges of establishing the first standard benchmark for assessing and comparing the performance of WfMSs in a way that is compliant to the main requirements of a benchmark: portability, scalability, simplicity, vendor neutrality, repeatability, efficiency, representativeness, relevance, accessibility, and affordability.

“ A Decision Support System for the Performance Benchmarking of Workflow Management Systems ”

TLDR
This work proposes a Decision Support System to ease the decision making of the desigh of benchmarks for Workflow Management Systems, and presents the conceptual models of the Decision support System and provides a prototypical implementation of it.

A Container-centric Methodology for Benchmarking Workflow Management Systems

TLDR
This paper shows how Containers, originally developed to ease the automated deployment of Cloud application components, can be used in the context of a benchmarking methodology, and takes advantage of emerging Cloud technologies to address technical challenges.

On the Evolution of BPMN 2.0 Support and Implementation

TLDR
It is found that the implementation of the BPMN 2.0 standard is more or less seen as finished by vendors and it is unlikely that features which are not available by now will be implemented in the future.

BPMN in the Wild: A Reprise

TLDR
This paper presents the repository mining approach for generating a corpus of open source BPMN models by systematically analyzing software projects on GitHub.com, and describes the resulting dataset, containing 79,713 distinct BPMn models from 18,534 open source projects.

Process Engine Benchmarking with Betsy - Current Status and Future Directions

TLDR
This work presents the work on process engine benchmarking, which discusses what has been achieved so far and point out future directions that deserve further investigation.

References

SHOWING 1-10 OF 26 REFERENCES

Throughput improvements for BPEL engines: implementation techniques and measurements applied to SWoM

TLDR
A set of optimization techniques for a state-of-the-art WfMS have been developed that delivers the required robustness with the best achievable performance and are complemented by a set of optimized techniques that improve performance by relaxing some of the stringent robustness requirements.

BPEL conformance in open source engines

TLDR
An evaluation of the standard conformance of five open source BPEL engines is presented and it is demonstrated that full standard conformity in contemporary engines is still far from given.

All that glitters is not gold : selecting the right tool for your BPM needs

TLDR
As the BPM marketplace continues its rapid evolution, there is an increasing array of technology offerings available for modeling and enacting business processes, but it is more difficult than ever to select an appropriate tool on which to base their business processes.

Business Process Control-Flow Complexity: Metric, Evaluation, and Validation

  • J. Cardoso
  • Computer Science
    Int. J. Web Serv. Res.
  • 2008
TLDR
A metric is presented and described to analyze the control-flow complexity of business processes and is evaluated in terms of Weyuker’s properties in order to guarantee that it qualifies as good and comprehensive.

A Workload Model for Benchmarking BPEL Engines

This paper describes a benchmarking workload model for business process execution language (BPEL) engines for Web services. The proposed model is based on simulation of real world traffic conditions…

SOABench: performance evaluation of service-oriented middleware made easy

SOABench is a framework for the automatic generation, execution and analysis of testbeds for evaluating the performance of service-oriented middleware. Testbeds can be characterized in terms of the…

Sliver: A BPEL Workflow Process Execution Engine for Mobile Devices

TLDR
Sliver is presented, a BPEL workflow process execution engine that supports a wide variety of devices ranging from mobile phones to desktop PCs and the design decisions that allow Sliver to operate within the limited resources of a mobile phone or PDA.

Monitoring and Analyzing Influential Factors of Business Process Performance

TLDR
This work provides a framework for performance monitoring and analysis of WS-BPEL processes, which consolidates process events and Quality of Service measurements, and uses machine learning techniques in order to construct tree structures, which represent the dependencies of a KPI on process and QoS metrics.

Process Fragment Libraries for Easier and Faster Development of Process-based Applications

TLDR
This work claims that process fragments allow for an easier and faster development of process- based applications, and discusses the potential impact of using process fragment libraries in cross-enterprise collaboration and application integration.

"BPELanon": Anonymizing BPEL Processes

TLDR
This work’s objective is to propose a method (“BPELanon”) for BPEL process anonymization, which transforms a process to preserve its original structure and runtime behaviour, while completely anonymizing its business semantics.