Performance Comparison Between BPMN 2.0 Workflow Management Systems Versions

TitlePerformance Comparison Between BPMN 2.0 Workflow Management Systems Versions
Publication TypeConference Proceedings
Year of Publication2017
AuthorsVincenzo Ferme, Marigianna Skouradaki, Ana Ivanchikj, Cesare Pautasso, and Frank Leymann
Conference Name18th Working Conference on Business Process Modeling, Development, and Support (BPMDS 2017)
Conference LocationEssen, Germany
KeywordsBenchFlow, BPMN, Performance Regression, Performance Testing, workflow engine, Workflow Management Systems

Software has become a rapidly evolving artifact and Workflow Management Systems (WfMSs) are not an exception. WfMSs' changes may impact key performance indicators or resource consumption levels may change among different versions. Thus, users considering a WfMS upgrade need to evaluate the extent of such changes for frequently issued workload. Deriving such information requires running performance experiments with appropriate workloads.
In this paper, we propose a novel method for deriving a structurally representative workload from a given business process collection, which we later use to evaluate the performance and resource consumption over four versions of two open-source WfMSs, for different numbers of simulated users. In our case study scenario the results reveal relevant variations in the WfMSs' performance and resource consumption, indicating a decrease in performance for newer versions.

Citation Key2017:benchflow:bpmds
Refereed DesignationRefereed