It's 11:00 am on a typical, busy day at UHN.
Toronto General and Toronto Western hospitals, and Toronto Rehab are all at Capacity Management Level 4, which means they are at full capacity for admitted patients. The majority of clinics are running full schedules and the Operating Rooms are fully booked. The Emergency Departments are seeing high volumes, with multiple patients waiting for inpatient beds.
All this is taking place when clinical staff across all sites begin to realize they can no longer access Epic, UHN's year-old health information system.
This is the make-believe scenario that was crafted as part of an executive tabletop exercise, designed to help prepare TeamUHN for an emergency Code Grey – Information Systems incident.
The simulated incident was presented to nearly 50 leaders across UHN's clinical, corporate and digital teams to identify potential emergency responses and share learnings.
"This mock exercise enables TeamUHN to practice resiliency techniques in an interactive and immersive way before the stakes get high," says Jackie Kwong Leung, Senior Director, UHN Digital Project Management Office, and co-chair of the IT Downtime Committee.
The objective of this exercise was to test changes made to UHN's Code Grey Incident Response, as a result of an actual Code Grey, which occurred on Jan. 9, 2023 due to a system switch failure.
"By practicing we can collectively mature our understanding of downtime processes, identify what is working and where there may be gaps or opportunities to improve, in a risk-free environment," Jackie says.
Preparing for a Code Grey at UHN
As UHN's technology landscape continues to evolve, and the organization becomes more reliant on electronic systems, it is imperative to prepare for incidents so TeamUHN feels confident in how to proceed through a Code Grey.
"The tabletop exercise was engaging and invaluable to get critical feedback on our response to a potential future event," says Dr. Anil Chopra, UHN's Vice President, Medical Affairs, who participated in the tabletop.
"It helps to gather all inter-professional leaders together to simulate a real-life emergency situation in order to think of pragmatic solutions."
Rachel Boissonneault, Clinical Director, Toronto Western Hospital, had similar feedback to share.
"The case-based exercise, and the ensuing rich conversation, enabled us to thoughtfully consider the strengths and gaps in our current knowledge and downtime processes in a Code Grey situation," Rachel says. "This will result in further productive discussions at the team and department level in continuing to review and strengthen our current procedures."
'Aha' moments after the tabletop exercise
The goal of the tabletop exercise is to test and evaluate the organization's plans and procedures.
Following the exercise, the participants identified three key learnings:
- Calling a Code Grey early when a widespread outage is identified with significant clinical impact is important. This ensures staff are aware of what is going on and that the appropriate teams are investigating the issue.
- Timely and regular communication internally and externally is critical during an unplanned IT downtime.
- Ensuring teams are following the proper decision-making channels through the Incident Management System (IMS) structure when enacted is key to ensuring a coordinated response.
"Being ready for a Code Grey – Information Systems isn't something that can be studied," says Sarah Muttitt, UHN Vice President and Chief Information Officer. “Working through a simulation, where the situation is dynamic and the pressures feel very real helps us, as an organization, think through what we would do and how we would respond without overreacting.
"I want to thank the IT Downtime Committee, and the Clinical Informatics and Emergency Preparedness teams for organizing and assembling such a well-planned and immersive activity."
