Validity of Simulation-Based Assessment for Accreditation Council for Graduate Medical Education Milestone Achievement

  title={Validity of Simulation-Based Assessment for Accreditation Council for Graduate Medical Education Milestone Achievement},
  author={Robert S. Isaak and Fei Chen and Susan M. Martinelli and Harendra Arora and David A. Zvara and Gene W. Hobbs and Marjorie Podraza Stiegler},
  journal={Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare},
Introduction The Accreditation Council for Graduate Medical Education requires biannual evaluation of anesthesiology residents on 25 subcompetency milestones. Some milestone domains are particularly challenging to repeatedly and reliably observe during clinical care. Simulation-Based Milestones Assessment (SBMA) may help overcome these challenges. However, few studies have examined the external validation of simulation assessment scores (ie, the relationships between simulation-based assessment… Expand
Simulation-based evaluation of anaesthesia residents: optimising resource use in a competency-based assessment framework
Simulation-based assessment is becoming an increasingly important tool for assessing the competency of medical residents in conjunction with other assessment methods and G-analysis can be used to assist in planning for optimal resource use and cost-efficacy. Expand
Comparing Real-time Versus Delayed Video Assessments for Evaluating ACGME Sub-competency Milestones in Simulated Patient Care Environments
The findings support the use of either real-time or delayed video evaluation for assessing milestones in a simulated patient care environment, as both assessment methods yielded comparable scores. Expand
Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring
Assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace, suggesting that resuscitation performance in simulation setting may be an indicator of competence inThe clinical setting. Expand
One (Not So Small) Step for Simulation-Based Competency Assessment in Critical Care.
The strength of the study by Boyle et al (4) lies in the rigorous systematic approach to development, which can serve as an example of evidence-based design and evaluation methodology for simulation-based assessment for frontline providers. Expand
Experience Is the Teacher of All Things: Prior Participation in Anesthesiology OSCEs Enhances Communication of Treatment Options With Simulated High-Risk Patients.
Findings from this study provide preliminary evidence that anesthesiology-based OSCE experience is associated with improved performance in an OSCE scenario, specifically regarding discussions of treatment options with high-risk patients. Expand
Creation of Simulation-Based Curriculum of Perioperative Emergencies for Residents in Anesthesiology
A simulation-based curriculum of perioperative emergencies for anesthesiology residents is feasible to implement, viewed as worthwhile by trainees, and can foster education in a different learning environment. Expand
Modules for the Technical Skills Section of the OSCE Component of the American Board of Anesthesiology APPLIED Examination
Video modules and a curriculum for anesthesiology resident OSCE preparation for the Interpretation of Monitors andinterpretation of Echocardiography components and more than 90% of participants reported that the modules were useful, and over 70% reported that they now felt prepared for these stations of the OSCE. Expand
Society for Neuroscience in Anesthesiology & Critical Care (SNACC) Neuroanesthesiology Education Milestones for Resident Education
The SNACC Neuroanesthesiology milestones provide a framework for reviewing resident performance and are expected to facilitate improved use of ACGME milestones during Neuroanesthesia subspecialty training. Expand
European Section/Board of Anaesthesiology/European Society of Anaesthesiology consensus statement on competency-based education and training in anaesthesiology.
CBMET programmes should enable establishment of a culture of lifelong learning for the anaesthesiology community and should clearly document core competencies, defined end-points, proficiency standards, practical descriptions of teaching and assessment practices and an assessment plan. Expand
Educating Anesthesiologists During the COVID-19 Pandemic and Beyond.
This vision paper discusses pre-COVID-19 educational methodologies and best evidence, adaptations required under CO VID-19 and evidence for these modifications and suggests future directions for anesthesiology education. Expand


Standardized Mixed-Fidelity Simulation for ACGME Milestones Competency Assessment and Objective Structured Clinical Exam Preparation
A full-scale, standardized, mixed-fidelity simulation training program for residents that serves multiple functions including the following: efficient and reliable means to meet the bi-annual ACGME reporting standards for milestone competency assessments. Expand
External Validation of Simulation-Based Assessments With Other Performance Measures of Third-Year Anesthesiology Residents
It is hypothesized that technical and nontechnical performance in the simulated environment is related to other various criterion measures, providing evidence to support the validity of the scores from the performance-based assessment. Expand
Simulation-based Assessment in Anesthesiology: Requirements for Practical Implementation
A broad overview of the use of simulation for measuring physician skills and competencies is provided to provide a model for those individuals and organizations that are responsible for developing, scoring, or validating simulation-based education and assessment programs in anesthesiology. Expand
Simulation-based Assessment to Identify Critical Gaps in Safe Anesthesia Resident Performance
Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (&rgr;2 = 0.81) and expected performance differences between first-year resident and fellow participants. Expand
Simulation for Anesthesiology Milestones
  • E. Sinz
  • Medicine
  • International anesthesiology clinics
  • 2015
The new Milestones requirement for all Accreditation Council for Graduate Medical Education (ACGME)-approved residency training was introduced by the ACGME with the goal of changing programExpand
Simulated and standardized patients in OSCEs: achievements and challenges 1992-2003
  • G. Adamo
  • Psychology, Medicine
  • Medical teacher
  • 2003
The author discusses achievements and challenges in the use of the objective structured clinical examination (OSCE) and simulated patient (SP) for medical education and assessment and the founding of the Association of Standardized Patient Educators (ASPE), a specialist professional organization. Expand
Mapping Direct Observations From Objective Structured Clinical Examinations to the Milestones Across Specialties.
The ICS-OSCE scores for PGY-1 residents appear to relate to beginner-level milestones for physician-patient communication across multiple specialties. Expand
Barriers to use of simulation-based education
PurposeBarriers to simulation-based education in postgraduate and continuing education for anesthesiologists have not been well studied. We hypothesized that the level of training may influenceExpand
Designing and Implementing the Objective Structured Clinical Examination in Anesthesiology
The history of OSCE and its current application in medical education and in different medical and surgical specialties are reviewed and the psychometrics of test design and implementation with emphasis on reliability and validity measures as they relate to OSCE are discussed. Expand
Simulation-based assessment of pediatric anesthesia skills.
A multiple-scenario, simulation-based assessment of pediatric perioperative care was designed and administered to residents and fellows and indicated the content was relevant and that raters could reliably score the scenarios. Expand