A multiple-criteria framework for evaluation of decision support systems

  • Gloria E. Phillips-Wrena, Eugene D. Hahnb, Guisseppi A. Forgionnec
  • Published 2004


In the literature, decision support systems (DSSs) have typically been evaluated on only a single criterion such as the outcome from decision making. However, it is clear that DSSs simultaneously have a critical impact on the process-oriented aspects of decision making, suggesting that a combination of both outcome and process criteria are highly relevant for DSS evaluation. Indeed, process characteristics are particularly crucial for web-based and real-time DSSs because of their ability to deliver timely, current information through features such as just-in-time information, real-time processing, on-line transaction processing, connectivity and globally up-to-date information. In this underexplored area, we propose a framework to evaluate DSSs that combines outcomeand process-oriented evaluation measures. The approach is demonstrated in the context of a real-time threat criticality detection DSS. Investigations are conducted using a multicriteria decision-making method called the Analytic Hierarchy Process (AHP) as well as a newly developed stochastic enhancement of AHP. We 6nd that the real-time DSS o7ered a signi6cant improvement in terms of process-related characteristics. However, it did not o7er a statistically signi6cant improvement in terms of outcome-related characteristics. The importance of simultaneously addressing both sets of considerations is discussed. ? 2004 Elsevier Ltd. All rights reserved.

7 Figures and Tables


Citations per Year

56 Citations

Semantic Scholar estimates that this publication has 56 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{PhillipsWrena2004AMF, title={A multiple-criteria framework for evaluation of decision support systems}, author={Gloria E. Phillips-Wrena and Eugene D. Hahnb and Guisseppi A. Forgionnec}, year={2004} }