Assessing Collaborative Problem Solving with Simulation Based Tasks

  title={Assessing Collaborative Problem Solving with Simulation Based Tasks},
  author={J. Hao and Lei Liu and Alina A. von Davier and Patrick Kyllonen},
Assessing collaborative problem solving (CPS) is an integrated part of the computer-supported collaborative learning (CSCL). We present some preliminary results from a project developed for assessing the CPS using web-based simulation. In the simulation, two participants collaborate via a chat box to complete a task on volcano science. By comparing the responses from 486 individuals and 278 teams (dyads) recruited from Amazon Mechanical Turk, we found the performance from the teams (dyads) is… 

Figures and Tables from this paper

An analysis of collaborative problem-solving activities mediated by individual-based and collaborative computer simulations
Analysis of how students solved a physics problem using individual-based and collaborative simulations to understand their effects on science learning found that students using the two simulations collaborated with peers to solve the problem in significantly different patterns.
Collaborative Problem Solving: Innovating Standardized Assessment
The goal of this symposium is to examine research questions that are of interest to the CSCL community, such as how CPS skills and collaborative patterns interact with performance outcomes, and how prior content knowledge and personality of team members affect the collaboration process and outcomes.
Characterizing Interactive Communications in Computer-Supported Collaborative Problem-Solving Tasks: A Conditional Transition Profile Approach
It is found that participants in the same team tend to show similar CTP compared to participants from different teams and that team members who showed more “negotiation” after the partner “shared” information tended to show more improvement after the collaboration.
Exploring graduate students’ collaborative problem-solving in engineering design tasks
This study explored seven engineering graduate students’ collaborative problem-solving (CPS) skills while working in interdisciplinary teams. Students worked in two different teams, in face-to-face
Initial Steps Towards a Standardized Assessment for Collaborative Problem Solving (CPS): Practical Challenges and Strategies
This chapter outlines some practical challenges for developing a large-scale, standardized assessment for CPS and presents some strategies to address those challenges and illustrates these strategies with the Collaborative Science Assessment Prototype (CSAP) developed at Educational Testing Service.
Interactive animation construction to measure students’ collaborative problem solving skills in learning earthquake
Investigation of students’ collaborative problem solving skills (CPS) through constructed interactive animation in Public Junior High School in Bandung Indonesia finds perspectives and abilities of team members are discovered and the type of collaborative interaction required and goals are established.
Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
The use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency and computational linguistic methods for automatically characterizing students’ display of various CPS skills in the task are described.


A Tough Nut to Crack: Measuring Collaborative Problem Solving
The development and implementation of a conceptual model to measure individuals’ cognitive and social skills through collaborative activities and the potential feasibility of using game-like environments with avatar representation is investigated.
An overview of the research conducted in several fields of work related to collaboration is given, a framework for the assessment of cognitive skills through collaborative problem-solving tasks is proposed, and several statistical approaches to model the data collected from collaborative interactions are proposed.
The cognitive underpinnings of effective teamwork: a meta-analysis.
There is indeed a cognitive foundation to teamwork; team cognition has strong positive relationships to team behavioral process, motivational states, and team performance, and meta-analytic regressions indicate that team cognition explains significant incremental variance in team performance after the effects of behavioral and motivational dynamics have been controlled.
Assessment and Teaching of 21st Century Skills
Rapidand seemingly acceleratingchanges in the economies of developed nations are having a proportional effect on the skill sets required of workers in many new jobs. Work environments are often
Assessing Science Inquiry Skills Using Trialogues
Trialogue-based tasks can be used to gather evidence that may be difficult to obtain using traditional assessment approaches, such as embedded questions. However, more research needs to be done in
Workforce Readiness: Competencies and Assessment.
Given the changes in the workplace over the last decade due in large part to the increase in technology and the rise of globalisation, the author suggests that there is a potential skill gap for the
Evidence for a Collective Intelligence Factor in the Performance of Human Groups
A psychometric methodology for quantifying a factor termed “collective intelligence” (c), which reflects how well groups perform on a similarly diverse set of group problem-solving tasks, and finds converging evidence of a general collective intelligence factor that explains a group’s performance on a wide variety of tasks.