Frequent commentaries in the literature have stated that certain critical success factors (CSFs) have to be accomplished in an organisation for an enterprise resource planning (ERP) system project to be successful. In this study we argue and demonstrate empirically that success in implementing an ERP system and in gaining performance improvement should be conceptualised as two separate dependent variables. The distinction is made because the former aspect is based upon project delivery outcomes, while the latter assesses post-ERP project performance. We question whether some factors labelled as ‘critical’ success factors for ERP projects are in practice actually critical for achieving success in implementation and improving output performance. To examine this we report an empirical study that has investigated whether four major CSFs are in practice critical for achieving organisational performance improvements, and the role that successful implementation may play in influencing the relationship between CSFs and improvements in organisational performance. A conceptual model was devised and then analysed using structural equation modelling based on data collected from 217 organisations. We found that some CSFs were not critical to achieve success in ERP implementation but were critical to help an organisational achieve performance improvement from an ERP system. Additionally, we also found that achieving successful ERP system implementation mediates the degree to which a CSF affects output performance improvement. The managerial and research implications of these findings are discussed and the limitations of the study noted. & 2013 Elsevier B.V. All rights reserved.