Development and evaluation of a three-dimensional multi-level model for visualising the workflow composite score in a health IT benchmark

Abstract

Background: Clinical information logistics is the backbone of care workflows inside and outside of hospitals. Due to the great potential of health IT to support clinical processes its contribution needs to be regularly monitored and governed. IT benchmarks are a well-known instrument to optimise the availability and use of IT by guiding the decision making process. The aim of this study was to translate IT benchmarking results that were grounded on a hierarchical workflow scoring system into an appropriate visualisation concept.

Methods: To this end, a three-dimensional multi-level model was developed, which allowed the decomposition of the highly aggregated workflow composite score into score views for the individual clinical workflows concerned and for the descriptors of these workflows. Furthermore this multi-level model helped to break down the score views into single and multiple indicator views.

Results: The results could be visualised per hospital in comparison to the results of organisations of similar size and ownership (peer reference groups) and in comparison to different types of innovation adopters. The multi-level model was implemented in a benchmark of 199 hospitals and evaluated by the chief information officers. The evaluation resulted in high ratings for the comprehensibility of the different types of views of the scores and indicators.

Conclusions: The implementation of the multi-level model in a large benchmark of hospitals proved to be feasible and useful in terms of the overall structure and the different indicator views. There seems to be a preference for less complex and familiar views.