Oral Examination Is a Requirement for Certification by the American Board of Orthopaedic Surgery

Le résumé fourni par la source

An oral examination is central to the certification process of many of the Member Boards of the American Board of Medical Specialties (ABMS). While there have been some concerns regarding the validity, bias, and reliability of oral examinations, the Boards view this type of examination as the best way to ensure that board-certified physicians are ethical, competent, and knowledgeable. Throughout its existence, the American Board of Orthopaedic Surgery (ABOS) has relied on the oral examination to determine the suitability of orthopaedic surgeons for ABOS Board Certification. History of the ABOS Oral Examination The ABOS was established in 1934. Initial certification was granted to members of the American Orthopaedic Association (AOA), a subset of whom became examiners for nonmember orthopaedic surgeons, or for AOA members who did not gain certification during the “grandfathered” period. Inaugural members of the Board felt that an oral examination was the best method to test knowledge, decision-making ability, and ethical performance in the practice of orthopaedic surgery. The initial requirement for ABOS Board Certification was an oral examination that was partially based on the submission of 25 case histories on which an individual’s practice could be evaluated1,2. In the early years, the oral examination was given by 3 examiners for 15 minutes each, and an essay question also needed to be completed. A candidate’s “orthopaedic work” was evaluated by the examiners, who generally asked a broad array of questions, including some that effectively evaluated the submitted cases. The scoring system used for the oral examination was “BONE”: “B” for excellent, “O” for satisfactory, “N” for borderline, and “E” for failure. The minimal passing grade on the examination was 2 borderline scores and no failures. Three borderline scores or 1 failure resulted in a failure of the overall oral examination and denial of certification. The ABOS has always considered the oral examination to be the best method for evaluating an orthopaedic surgeon’s practice, but examiner reliability and fairness of testing have always been a concern1. From 1937 to 1948, the ABOS Board Certification examination consisted of 2 parts. Part I included both a written and oral examination during training. The Part II oral examination consisted of five 15-minute examinations in each of the following areas: clinical care, pathology, anatomy, cadaver dissection, and osteology. Candidates sat for Part II of the Board Certification examination, which consisted of an oral examination based on the submission of 25 to 50 case histories, after 2 years in practice. Three examiners administered the test. The candidate’s biographical information, including name, practice area, and training program as well as educational background, was known to the examiner. The examination was prone to consisting of a social discussion rather than a true assessment of knowledge or orthopaedic practice, and grading was highly variable, with some very severe and some very lenient examiners. Some examiners spent much of the examination time making the candidate feel comfortable, while others spent most of the examination time making the applicant feel miserable. The non-numerical grading system led to further concerns about the fairness of the examination. Failure rates varied from 15% to 25% during this period1. In 1949, the written portion of the Part I examination was moved to being taken after a candidate had been in practice for 2 years, and it was given in conjunction with the Part II Oral Examination. During that time, examiners who were either very severe or very lenient were identified and either “trained” or dropped from administering the oral examination in order to address examiner fairness and consistency. Subject areas of the oral examination were changed to include anatomy, fractures, pathology, congenital and acquired deformities, and static and postural defects1. Between 1950 and 1970, the oral examination format underwent progressive development. Examiners could bring their own material to use in the examination of individual applicants, and standard cases and questions were adopted in the areas of anatomy and pathology. Two examiners at each examination station caucused to determine the final grade, and the pass/fail grading system was changed to a numerical scoring system. There were continuing efforts to increase the reproducibility and reliability of the oral examination. Starting in 1961, examinations were held in booths in a common area rather than in hotel rooms1. Candidate anonymity was adopted; individuals were identified by number, and the examiners were blinded to the candidate’s training program and institution. Rather than caucusing, the examiners scored each candidate independently in an effort to increase reliability. Examiner variation was assessed by comparing the scores provided by paired examiners who independently scored the same examination. There was a continued effort to standardize the examination, which increased the reliability and validity of the results. Between 1970 and 1989, the oral examination had evolved and became focused on 4 areas: pediatric orthopaedic surgery, adult orthopaedic surgery, trauma, and interpretive skills. In 1979, there was an attempt to include motor skill evaluation as part of the oral examination, but this was dropped after 1 year because of the difficulty in scoring1. During this era, the oral examination was composed of four 30-minute sections. In 1980, the criteria for choosing examiners were modified. Rather than choosing examiners solely because of their status and hospital or medical school affiliation, a broader group of examiners was identified. The number of examiners was increased to prevent examiner fatigue, which was thought to compromise the oral examinations. The examiners were rated in terms of severity and consistency. Although examiners were paired for each examination, they scored each examinee independently. There was agreement between the examiners in 82% of cases, with a 6.5% failure rate, which was consistent from year to year. In the late 1980s, the ABOS Oral Examination Task Force recommended a return to a case-based oral examination in which the individual’s cases were the basis for the examination. In 1990, the examination was changed to consist of 6 standard cases (2 pediatric orthopaedic cases, 2 adult orthopaedic cases, and 2 trauma cases) in addition to 10 selected cases from a Case List that had been submitted from each examinee. Three panels of 2 examiners were used at that time. Starting in 1991, cases were selected from a 6-month consecutive surgical Case List that was submitted by each examinee. By 2000, the standard cases were dropped, and the oral examination was based entirely on an individual’s submitted surgical Case List. The Board was quite concerned about the validity, accuracy, and fairness of the oral examination during this time. Dr. Mary Lunz, a psychometrician and expert in testing, provided extensive consultation to the Board over many years, evaluating the performance of the examiners as well as the overall fairness and validity of the examination. Case selection from the submitted Case Lists initially included 15 cases, and the examinee could drop 5 cases. Over time, case selection became more focused and standardized, requiring the candidate to be examined on nearly all of the selected cases. A scoring rubric was developed and was accompanied by an increased effort in examiner training to improve both consistency and discrimination in scoring. Examiners were handicapped based on their harshness or leniency in scoring so that a candidate’s score would not be adversely affected by examiner severity as long as the oral examiner was consistent. Inconsistent or nondiscriminatory examiners were identified and educated about the use of the scoring rubric. If the individual examiners did not achieve the esta

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.