The last issue of The Diplomate focused on the process of creating Computer Examinations. This issue will describe the development, delivery, and scoring of Oral Examinations.
Oral Examinations have been part of the American Board of Orthopaedic Surgery (ABOS) certification process since almost the beginning. While the structure has changed over the years, they are still an integral component. Candidates must pass the Part II Oral Examination as the final step toward achieving Board Certification. Approximately 750 surgeons take the Part II Examination annually. In addition, approximately 140 orthopaedists take an Oral Recertification Examination each year. Many elect recertification with the Oral Examination over the computerized option while some are required to take the Oral Examination for reasons such as a lapse in certification. While the computer based examinations provide an excellent evaluation of fund of knowledge, the Oral Examination based on the surgeon’s actual practice provides an excellent look that individual’s practice.
Everyone applying to take an Oral Examination submits a six-month case list using the ABOS Scribe system. Those taking a Part II Examination must include all cases from April-September of the year prior to their examination while those recertifying can choose any six consecutive months from two years prior to the examination.
Twelve cases are selected from the submitted case list by volunteer Orthopaedic Surgeons experienced in giving the Oral Examination. The Diplomate then uploads electronic medical records for each of these 12 cases into Scribe. Diplomates receive a list of what records need to be uploaded. The ABOS continues to work diligently on eliminating documents not critical for the examination and decreasing the time and effort required to upload. With the development of an examination based entirely on digital images and records, Diplomates are no longer required to bring large volumes of paper records to the examination. An examination based entirely on electronic records also increased the efficiency of the testing process. Detailed instructional videos on how to upload documents and prepare to take the exam are available on the ABOS website. Staff in the ABOS office are also readily available to answer questions by phone, and information on all application deadlines is available on the ABOS website.
On the day of the examination prior to reporting to their assigned testing stations, examiners and examinees receive separate briefings from ABOS Directors. A bell indicates the start and stop of each 25 minute session. There are four sessions with a break between each. Examiner’s expertise align with the examinees’ practice. Typically the examinee is questioned about three cases in each of the four sessions so that most commonly all 12 uploaded cases are used.
By the end of the entire examination process, a large number of scoring data points are generated for analysis resulting in a reliable exam. Examinees are encouraged to prepare by carefully reviewing their cases and the videos on abos.org. Spending time creating a quality case summary is particularly helpful.
Examiners use a scoring rubric to measure the candidate on:
- Data Gathering
- Diagnosis and Interpretive Skill
- Treatment Plan
- Technical Skill
- Applied Knowledge
- Surgical Indications
- Surgical Complications
- Ethics and Professionalism
Debriefing sessions with examinees after each session provides valuable feedback on the overall process to the ABOS. Information from these sessions along with examiner debriefings has resulted in many of the changes used in current examinations.
After the Oral Examination is complete, the ABOS works extensively with psychometricians to ensure the validity of the examination. Examiners are statistically rated on severity and consistency and adjustments are made so a candidate will have passed or failed based on the examination and not on which examiners they were assigned to. After receiving and analyzing data from psychometricians, the Oral Examination Committee sets the pass/fail point and examinees are notified that their results are posted on their password-protected site on abos.org.
Each year approximately 200 Board Certified Orthopaedic Surgeons volunteer to travel to the Palmer House in Chicago to spend several days working as Oral Examiners. These volunteer examiners come from private and academic practices and represent all regions of the country. Without that generous contribution of time and effort the Oral Board Examination process would not be possible. The dedication and commitment on the part of the volunteers to maintaining a standard of quality and safety in Orthopaedics is commendable. The Oral Examination provides an excellent “snapshot” of a surgeons practice and is an important component of assuring the public about the quality, safety, and competence of a Board Certified Surgeon. That in turn preserves the value of the certificate for the Diplomate. The ABOS is committed to continued refinement of the process.
James Roberson, MD
President, American Board of Orthopaedic Surgery
Executive Medical Director’s Report
Out of the 29,000 American Board of Orthopaedic Surgery (ABOS) Board Certified orthopaedic surgeons, more than 4,000 hold a Subspecialty Certificate in Surgery of the Hand, Orthopaedic Sports Medicine, or both. The ABOS has a method where Diplomates can apply to recertify both their General and one of their Subspecialty Certificates at one time, with one examination.
The Combined Examination, offered as either a Computer or an Oral Examination, allows Diplomates to recertify both certificates with a single examination at a single sitting. Diplomates are required to complete only one application and pay only one fee. The Computer Examination only contains questions in the subspecialty; as of 2017 there are no longer any general orthopaedic questions on Combined or Practice-Profiled ABOS examinations. The Oral Examination is based on a Diplomate’s case list and thus is an excellent way to truly assess a Diplomate’s practice. The ABOS is trying to meet Diplomates where they are and tailor the examination program to fit Diplomate needs.
While the expiration dates on a Diplomate’s General Orthopaedic Certificate and their Subspecialty Certificate may differ (the ABOS is considering ways to remedy this situation), Diplomates should take the Combined Examination based on the date that their General Certificate expires. Once a Diplomate passes the Combined Examination, their General and Subspecialty Certificates will be aligned.
The registration period and other requirements are the same as they are for the other Recertification Examinations: May 1 of the year prior to the examination and a case list from the calendar year two years prior to the year of examination. For those Diplomates wishing to take a Combined Recertification examination in 2018, the deadline to submit an application, application fee, case list, and the required Continuing Medical Education (CME)/Self-Assessment Examination (SAE) credits is May 1, 2017. The case list should be from the 2016 calendar year (any three consecutive months (up to 75 cases) for a Computer Examination and any six consecutive months for an Oral Examination).
Unlike the ABOS General Orthopaedic Recertification Examination and the Practice-Profiled Recertification Examinations in other fields (such as Adult Reconstruction and Spine), the Computer ABOS Combined Recertification Examinations in Surgery of the Hand and Orthopaedic Sports Medicine take place in the late summer/early fall, along with the initial Subspecialty Certification Examinations and the Practice-Profiled Examinations in those areas. The ABOS Combined Oral Examinations take place in July in Chicago in conjunction with the Recertification Oral Examinations.
These Combined Examinations are beneficial to Diplomates and their busy practices. Please do not hesitate to contact me with any questions.
David F. Martin, MD
Executive Medical Director, American Board of Orthopaedic Surgery