Combining Estimates with Planning Poker--An Empirical Study

  title={Combining Estimates with Planning Poker--An Empirical Study},
  author={Kjetil Mol{\o}kken-{\O}stvold and Nils Christian Haugen},
  journal={2007 Australian Software Engineering Conference (ASWEC'07)},
Combination of expert opinion is frequently used to produce estimates in software projects. However, if, when and how to combine expert estimates, is poorly understood. In order to study the effects of a combination technique called planning poker, the technique was introduced in a software project for half of the tasks. The tasks estimated with planning poker provided: 1) group consensus estimates that were less optimistic than the mechanical combination of individual estimates for the same… 

Figures and Tables from this paper

Using Bayesian Networks to Obtain the Task's Parameters for Schedule Planning in Scrum
The identification and validation of the most important factors taken into account by Scrum teams to assign complexity and importance to tasks are proposed, which are two important factors to consider for task scheduling.
Planning Poker in cost estimation in Agile methods: Averaging Vs. Consensus
The main aim of this paper is to present the results of a case study research which has been carried out to compare the accuracy of both disciplines and show less accuracy when team get an average size of the User Stories compared to coming to consensus about the size of User Stories.
The Benefits and Challenges of Planning Poker in Software Development: Comparison Between Theory and Practice
The research includes identification of several different estimation process patterns, such as a gradual changing of the benefit of promoting group interaction is achieved well in practice, whereas avoidance of the “anchoring” effect and promoting individual participation, are largely not achieved.
Supporting Feature Estimation with Risk and Effort Annotations
The results show that the use of annotations in planning poker increases estimation accuracy, and it is proposed to annotate user stories with symbols that highlight risk and effort drivers to give all team members a better understanding of a project's challenges.
A solution for estimates in software development projects
This dissertation aims to contribute to more realistic estimates, by identifying shortcomings in the current estimation process and supporting the self-improvement of the process, by gathering as much relevant information as possible from each finished project.
Empirical research on Software Effort Estimation Accuracy
To increase estimation accuracy, software contractors should involve the customer, and nurture the customer relationship, and add some repeatable structure to the estimation process, but not too much.
Identification of Inaccurate Effort Estimates in Agile Software Development
The emphasis of this paper is on "Planning Poker", a widely used estimation technique in agile context, and a set of metrics is proposed, which can be used to evaluate accuracy of estimates.
An investigation of feature weighting algorithms and validation techniques using blind analysis for analogy-based estimation
A novel and efficient feature weighting algorithm is developed that shows that blinding can be a relatively straightforward procedure and also highlights various statistical analysis decisions which ought not be guided by the hunt.
A Study of Software Development Cost Estimation Techniques and Models
This study supports the SDCE by exploring its techniques and models and collecting them in one place, and recommends a hybrid approach for SDCE because the limitations of one model and technique are complemented by the merits of the other model/technique.


Group Processes in Software Effort Estimation
An experiment in which experts with different backgrounds combined their estimates in group discussion found that the groups submitted less optimistic estimates than the individuals, and the group discussion-based estimates were closer to the effort expended on the actual project than the average of the individual expert estimates.
An empirical study of using planning poker for user story estimation
The results show that planning poker improved the team's estimation performance in most cases, but that it increased estimation error in the extreme cases.
Expert Opinions in Forecasting: The Role of the Delphi Technique
A number of principles are developed here to indicate how to conduct structured groups to obtain good expert judgments and it is anticipated that by following these principles, forecasters may be able to use structuredgroups to harness effectively expert opinion.
An experimental study of individual subjective effort estimations and combinations of the estimates
  • Martin Höst, C. Wohlin
  • Economics
    Proceedings of the 20th International Conference on Software Engineering
  • 1998
Interview techniques dealing with which type of questions to ask are evaluated and techniques for combining estimates from individuals into one estimate are compared in an experiment, showing that the interview technique is not as important as the combination technique.
A framework for the analysis of software cost estimation accuracy
It is shown that a selection of cost estimation studies frequently do not take the necessary actions to ensure meaningful interpretations of estimation error data, and a general framework is proposed that will improve analyses of software cost estimation error.
The Delphi technique as a forecasting tool: issues and analysis
Expert judgement as an estimating method
  • R. Hughes
  • Computer Science
    Inf. Softw. Technol.
  • 1996
Software Metrics: Measurement for Software Process Improvement
This book sets out an approach to the formal validation of measures and the theory of statistical data analysis for students and contains many examples of the use of real data in real projects.