..........................................................................................................................................................................3 MEASURING TEACHERS’ EFFICACY WORKING WITH DIVERSE STUDENT NEEDS: .....................................4 TESTING A MEASUREMENT MODEL ...........................................................................................................................4 PURPOSE ............................................................................................................................................................................8 METHODS ............................................................................................................................................................................8 DEVELOPING THE SURVEY INSTRUMENT .........................................................................................................................8 PILOTING THE SURVEY INSTRUMENT ...............................................................................................................................9 EVALUATING THE STABILITY OF THE SURVEY INSTRUMENT ........................................................................................12 ANALYZING THE MODEL FIT ..........................................................................................................................................13 RESULTS ............................................................................................................................................................................13 PART 1: TECHNICAL ADEQUACY ....................................................................................................................................14 TESTING OF A MEASUREMENT MODEL...........................................................................................................................24 RESULTS ON SURVEY DURING PILOT TESTING: HOW TEACHERS RESPONDED.............................................................33 TEACHERS’ ESTIMATION OF THE LIKELIHOOD OF THEIR STUDENTS FAILING THE STATE TEST..................................37 DISCUSSION ......................................................................................................................................................................40 APPENDIX A: FEEDBACK FORM USED DURING CONTENT REVIEW................................................................41 APPENDIX B: TECHNICAL SPECIFICATIONS OF SURVEY ...................................................................................46 APPENDIX C: SURVEY USED IN PILOT STUDY .......................................................................................................49 APPENDIX D: MPLUS OUTPUT FILES FOR MEASUREMENT MODEL ................................................................60 APPENDIX E: REVISED SURVEY (POST PILOT STUDY) ........................................................................................88 REFERENCES ....................................................................................................................................................................95 Teacher Efficacy Survey 3 Abstract This technical report describes the development, pilot testing, and revision of a survey instrument designed to measure secondary school teachers’ perceptions of their efficacy working with students from diverse backgrounds. A brief review of relevant literature frames the currentThis technical report describes the development, pilot testing, and revision of a survey instrument designed to measure secondary school teachers’ perceptions of their efficacy working with students from diverse backgrounds. A brief review of relevant literature frames the current study in the context of survey development that is technically adequate. Exploratory factor analysis is used to identify factors within each of four hypothesized latent constructs (alignment, inclusivity, organization, and general efficacy), and the measurement model fit is evaluated to explain their representation. Results suggest that the survey instrument (once revised) is reliable and that the measurement model may adequately fit the data. Teacher Efficacy Survey 4 Measuring Teachers’ Efficacy Working with Diverse Student Needs: Testing a Measurement Model School improvement, as mandated by the federal government and implemented by nearly all state education agencies, is a continuing effort on the part of school staff to improve student achievement on standardized measures (Buttram & Waters, 1997). The past 25 years has seen a growing movement toward school improvement throughout the United States. Beginning with A Nation at Risk (National Commission on Educational Excellence, 1983), continuing through the President’s Education Summit in 1996 (Buttram & Waters, 1997), and, more recently, the No Child Left Behind Act of 2001 (2002) significant public resources have been dedicated to improving school effectiveness through standards-based accountability. In the case of school improvement and accountability, the construct under investigation may be improving student achievement, but efforts toward that end generally take the form of interventions, not with the students, but with the teachers (Barth, 1990, Rosenholz, 1989). This approach reflects the broadly held position that improving student achievement relies on changing what happens in the classroom (Rudy & Conrad, 2004). Researchers at the University of Oregon have undertaken an effort to identify, and to subsequently measure, elements of teacher attitudes and competencies that have an impact on the effectiveness of staff development efforts in school improvement. The elements described here were identified as consequential to a staff development effort targeting instructional change toward concept-based instruction (CBI) as opposed to more common fact-based instruction (Carnine, 2002; McCoy & Ketterlin-Geller, 2004; Twyman, Ketterlin-Geller, McCoy, & Tindal, 2003). Our focus has been in the secondary school content areas. Teacher Efficacy Survey 5 Briefly, concept-based instruction attempts to identify concepts common across large portions of the curriculum and to explicitly address these concepts rather than to focus upon the facts of individual and independent instances throughout curricula. We focus on four constructs of school improvement as reported by teachers in conversation following training and implementation of concept-based instruction in the classroom. Teachers participating in staff development in the use of concept-based instruction anecdotally report improvement in their self-efficacy in these areas and often see associated improvement in student academic achievement. The challenge, then, is to identify or create an instrument that can measure teachers’ self-efficacy in these areas before and after their involvement in staff development so that the change might be more easily identified and its relationship to student academic achievement more explicitly described. This technical report describes the survey instrument developed to measure these four latent constructs. Alignment. As a result of the push toward academic achievement as measured by standardized assessments, it is critical to establish alignment among state standards, curriculum, instruction, and assessment. To improve student outcomes we must evaluate student performance (American Educational Research Association (AERA), 2003; Rudy & Conrad, 2004). Alignment of these four constructs provides an assurance, first, that the tests appropriately measure student knowledge and, second, that the results of those tests can be used to modify both curriculum and instruction to better address the standards and improve student achievement (AERA, 2003). Without alignment among these elements of schooling, data may be suspect as school and district staff modify their practices to facilitate improvement. Although responsibility for aligning statewide testing to state standards falls reasonably to state department of education staff (AERA, 2003), aligning the curriculum and associated instruction must occur at the local Teacher Efficacy Survey 6 level and focus on curriculum and instruction to standards and associated assessments. Raising a teacher’s perception of efficacy in alignment would enhance student instruction toward the standards and therefore toward statewide assessments. By aligning the curriculum to standards, teachers can better support the learning of all students relative to these standards. Despite efforts at mandating content standards as the basis for curriculum-design decisions, textbooks often dictate the curriculum rather than these standards (Diegmueller, 1995). Exacerbating matters, the content in these textbooks typically is not aligned to the standards, nor do textbooks present information on an intellectual level called for by state standards (Tomlinson, 2000; Ketterlin-Geller, McCoy, Twyman, & Tindal, 2003). Inclusivity. The No Child Left Behind Act of 2001 (NCLB) (2002) presents schools with the challenge of helping all students, regardless of special needs status, to meet the same high standards of academic performance. This challenge has been educators’ concern for some time and is reflected in the trend toward inclusion of special needs students in regular education classrooms (Alper, Schloss, Etscheidt, & Macfarlane, 1995). Inclusion is intended to provide students with a functional instructional environment that is based on their needs (e.g. classified special needs, at risk, homeless, or gifted) (Stainback, Stainback, & Jackson, 1992). Unfortunately, teachers not yet comfortable with their inclusive classroom now are confronted with the demands of NCLB. Inclusivity, then, is influenced by teacher’s propensity to manage their classrooms in an encompassing manner based on curriculum adaptation and instructional scaffolds rather than mere presence of students from v