...................................................................................................................................2 TABLE OF CONTENTS..............................................................................................................4 INTRODUCTION..........................................................................................................................5 BACKGROUND AND SIGNIFICANCE....................................................................................5 LITERATURE REVIEW.............................................................................................................8 PROCEDURES............................................................................................................................11 RESULTS.....................................................................................................................................15 DISCUSSION...............................................................................................................................19 RECOMMENDATIONS.............................................................................................................22 REFERENCES.............................................................................................................................24 APPENDIX A (Performance Standards)...................................................................................26 APPENDIX B (Performance Based Evaluation Check Sheet)................................................29 5 INTRODUCTION The Idaho Falls Fire Department places a high priority on training. An important part of our training program is the practice of engine company evolutions. Department members are expected to participate in these evolutions to maintain basic fire combat skills. The problem is that the Idaho Falls Fire Department does not have a systematic program to evaluate personnel with regards to engine company or individual task performance standards. The purpose of this research project is to develop a program of evaluating the performance of all engine companies and individuals in standard evolutions, tasks, and department requirements. Action research was used in an effort to develop such a program. The following research questions had to be answered: 1. How can the Idaho Falls Fire Department evaluate the ability of individuals to perform necessary tasks at a level of acceptable fire service standards? 2. How can the Idaho Falls Fire Department evaluate the ability of engine companies to perform necessary tasks at a level of acceptable fire service standards? 3. What format would be effective in implementing a performance based evaluation system? 4. What method of documentation can be implemented to increase the effectiveness of a performance based evaluation system? 6 BACKGROUND AND SIGNIFICANCE The Idaho Falls Fire Department, hereafter referred to as the department, has historically attempted to introduce programs designed to confirm competency. In the early 1980s the department adopted what was then termed a physical agility test. All department members, on an annual basis, were required to perform a series of tasks designed to simulate actual fire combat tasks such as donning a self contained breathing apparatus, advancing a hose line, and climbing a ladder. The annual test did manage to confirm a department member’s baseline ability to perform fire related tasks, but the program had numerous shortcomings. The test was comprised of two sections, and firefighters were allowed 20 minutes to complete each section. The time allowed proved excessive, as the more motivated and competent firefighters could complete the tasks in around five minutes. Those department members less motivated and/or incompetent could complete the series of tasks, but they used all the allowed time. It was clear to most of the department’s management that the leniency of the physical agility test rendered it a poor indicator of a firefighter’s actual ability to perform in combat situations. As the department’s administration changed, an attempt was made to increase the test’s validity by decreasing the time allowed. This posed no difficulty for the more motivated and competent firefighters, but the less motivated now complained of numerous injuries, so the physical agility test was abandoned. In retrospect the most prominent weakness of the abandoned testing program was not the time factor, but the lack of any connection to a corresponding training program. The department’s training officer emphasized engine company evolutions, but department 7 management made no attempt to mandate any performance level. The more motivated company officers devoted time to practicing the evolutions at the drill field, but the less motivated officers placed a much lower priority on practical training. Management’s failure to mandate training or test for competency created a broad range of competency levels. The success of fire ground operations varied widely depending upon which crew arrived on the scene. The individual firefighters assigned to an unmotivated officer were not provided with the practical drill field experience to assure their effectiveness in an actual fire situation. The department clearly had a need for a program that would encourage all companies and individuals to train and also allow for confirmation of the effectiveness of that training. “A department that establishes performance standards or timed evolutions for engine and ladder companies and then trains utilizing those criteria will be better prepared to handle the varied problems that occur at an incident scene” (Smith, 1996, p. 16). Besides the practical need for a performance standard program there is also the legal need. As the department moved towards computerized record keeping, it became obvious that not only was it important to demonstrate our capabilities, it was also important to be able to document that demonstration of appropriate training. Davis (1991) states that: To ease the threat of lawsuit, the prudent fire manager will investigate and consider any sensible protection that might be available. This is especially true today, with the epidemic of lawsuits. It seems that every mishap can result in a legal action. (p. 56) Davis goes on to add, “Also, the documentation that results from performance standards (often called ‘paper trails’) can provide valuable support when legal scrutiny occurs” (p.56). The purpose of this research project, the development of a program to evaluate the performance of engine companies and individuals, relates to Unit 10: Service Quality/Marketing 8 of the Executive Development course taught at the National Fire Academy. While discussing service quality in that setting, Drucker (1999) claimed “it takes far more energy and work to improve from incompetence to mediocrity than it takes to improve from first rate performance to excellence” ( p. 67). The successful implementation of a program that will encourage training and allow the department to evaluate our members’ task performance should eliminate the incompetence and allow for future excellence. LITERATURE REVIEW Training is one of the most often discussed topics in the fire service. The appraisal and assessment of training has also received attention. Past generations of firefighters measured their proficiency under fire; turning a building into a parking lot earned that fire crew a failing grade. The trend towards a more proactive assessment of efficiency has given the concept of performance standards momentum. Performance standards have been around for some time. In the 1970s and 1980s , there were many articles about them. Some have viewed performance standards as a passing fad, but many universities and other educational facilities have developed the concept of competency-based education, which is essentially performance standards in a new wrapper. This can be the best training to effectively maintain competency and optimum employee performance. (Davis, 1991, p. 56) The first research question posed referred to the evaluation of an individual’s ability to perform tasks. Performance standard evaluations originally focused on the engine company. As the concept has developed, proponents of performance standards have become aware of the need 9 to evaluate the individual within an assessed engine company. Lecuyer (1999) warns “be aware that an individual could score poorly even if the company met the time standard, so do not overlook the individual ratings” ( p. 94). The results of focusing only on company success could cause an individual’s deficiencies to be overlooked. That individual might be compensated for when operating on the assigned crew, but if reassigned to another crew those deficiencies could prove problematic. Warren (1998) addresses the individual assessment: The evaluation phase signals the end of the learning process and verifies that the original learning objectives have been attained. The only way to adequately implement this phase is for the company officer to test the student’s ability to perform a task without assistance. Evaluations can be accomplished through . . . performance exams for psychomotor objectives. (p.52) Evaluating the task performance of an individual working within a company operation requires a standardized program. “Standardization lets members assigned to different units work together” (Smith, 1996, p. 16). The assessment of individuals also calls for special attention on the part of the assessors. “The evaluation process requires multiple graders so that at least two (preferably three) raters grade each individual” (Lecuyer, 1999, p. 92). The second research question referred to the evaluation of the engine company’s ability to perform task oriented evolutions. “As chief, one of your responsibilities should be to empower and encourage your company officers to verify that all training goals have been met. Otherwise, the