Evaluating Design Solutions Using Crowds

  title={Evaluating Design Solutions Using Crowds},
  author={Yasuaki Sakamoto and Jeffrey V. Nickerson},
Crowds can be used to generate and evaluate design solutions. To increase a crowdsourcing system’s effectiveness, we propose and compare two evaluation methods, one using five-point Likert scale rating and the other prediction voting. Our results indicate that although the two evaluation methods correlate, they have different goals: whereas prediction voting focuses evaluators on identifying the very best solutions, the rating focuses evaluators on the entire range of solutions. Thus… CONTINUE READING

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.
Showing 1-10 of 22 extracted citations


Publications referenced by this paper.
Showing 1-10 of 32 references

Crowdsourcing creativity: Combining ideas in networks

  • J. V. Nickerson, Y. Sakamoto
  • Workshop on Information in Networks
  • 2010
2 Excerpts

Harnessing crowds: Mapping the genome of collective intelligence

  • T. W. Malon, R. Laubacher, C. Dellarocas
  • MIT Sloand School Working Paper 4732-09,
  • 2010

Similar Papers

Loading similar papers…