A new smartphone-based system developed at Northwestern Medicine simplifies and facilitates the assessments of general surgery trainees.
“We noticed we were having a problem assessing the performance of our residents in the operating room because our busy surgeons don’t always have the time to sit down and fill out a detailed assessment form; yet we know information on how the residents perform in the OR is extremely important,” said Jonathan Fryer, MD, professor of Surgery-Organ Transplantation. “So we brainstormed to come up with a solution which would allow the attending to do the assessment without breaking stride in their busy workday.”
Featured in a recent article in JAMA, Dr. Fryer and his team simplified the assessment by focusing on the most important aspect of training – autonomy – and applied the four-level Zwisch scale: show and tell, active help, passive help and supervision only. Next, they created a smartphone application, the Procedural Autonomy and Supervisions System (PASS), which allowed the attending physician to provide immediate feedback more easily.
“If you do five operations a day, five days a week, a week later you aren’t going to remember how a specific resident performed in a specific case,” Dr. Fryer said. “So we needed a system where you could immediately complete an assessment and provide essential feedback and it had to be easy to do, so that every attending would do it immediately for every single procedure they perform with a resident.”
They integrated PASS with the operating room database; at the end of an operation, the system sends a prompt to the attending physician to fill out the evaluation. The attending physician then opens the app, verifies the resident was on the case, chooses which of the four levels the resident completed and the complexity of the case: hard, typical or easy. Once recorded, the information goes into a central database.
Over the course of eight months, 27 faculty used the system to complete nearly 1,500 operative performance assessments on 31 residents. During this time, they had more than 90 percent compliance from the faculty. Fryer and his colleagues published the data and results of the system in the Journal of Surgical Education in November.
“What we found, which is exciting, is that since the attending gets prompted with every case to assess the residents level of autonomy, they go into the operation thinking ‘what can I do to get the resident to the next level.’ The residents liked it because it gave them a foundation for what their current level of autonomy was and what they needed to do to get to the next level,” Dr. Fryer said. “It’s also a way to evaluate the attending. If for every operation the attending only records ‘show and tell,’(i.e. the lowest level on the autonomy scale) we can talk with the attending about giving the resident the opportunity to do more of the procedure, so they can advance.”
With the success of PASS, the team has developed a modified version aimed for use by other procedural specialties. This system uses a smart phone app, called SIMPL, which is activated once a trainee fills out a self-evaluation; this triggers a text prompt to the attending physician to assess the trainee’s performance. Again, the data goes to a central database where it can be accessed by the residents themselves, attending physicians and the residency program.
“It’s very versatile and it’s valuable on many levels: for the resident it provides valuable feedback on how well they are performing essential procedures, for the faculty it provides feedback on how well they are teaching, and for the program it provides valuable feedback that can be used in curriculum planning,” Dr. Fryer said. “It’s accurate and timely and based on a simple principle, autonomy. Because it is easy to use, it facilitates broad collection of performance data with most if not all of the procedures your residents perform.”
With the support of the American Board of Surgery, Northwestern and nine other programs in the country will soon begin testing the SIMPL system and Dr. Fryer hopes that all general surgery programs in the country will adopt it in the future. He also believes that SIMPL may be useful in other surgical and non-surgical specialties where trainees must learn to do procedures.
Fryer will give a lecture on SIMPL in May as part of Today’s Innovations in Medical Education (TIME) lecture series, sponsored by the Feinberg Academy of Medical Educators.
