Introduction

Healthcare providers around the globe face the daunting task of improving patient outcomes while keeping costs manageable [1]. Advanced analytical technology, driven by the digital revolution, holds promise for enhancing patient care and data-driven decision-making, which promotes a culture of continuous improvement within healthcare organizations [2]. Industries such as manufacturing, finance, and logistics have shown that successful implementation of workflow automation is critical in optimizing operations [3]. Automation techniques can be tailored to address the unique needs and complexities of the healthcare industry, especially when existing methods rely on manual data analysis [1,3]. This customization can significantly enhance efficiency and accuracy in handling healthcare data.

Trauma triage systems play a crucial role in trauma care in the Emergency Department (ED) [4]. Research has consistently shown that appropriate trauma triage improves patient care and outcomes, enhances ED efficiency, and reduces operating costs [5]. Critical resources like blood products, hospital staff, and inpatient beds are limited, making it a complex challenge to balance the need for rapid and effective trauma care with the scarcity of available resources [6]. Trauma team activations (TTA) at the anticipated level of need are essential to ensuring that trauma patients with severe injuries receive timely and adequate care, even in the face of limited resources. In the United States, trauma centers have established TTA protocols to respond promptly to patients with varying degrees of injuries [7]. These protocols rely on criteria of physiologic findings, mechanism of injury, and anatomic damage to approximate injury severity, morbidity, and mortality risk [8].

Though structured, TTA criteria do not consistently anticipate every patient’s needs, often leading to inappropriate triage decisions [9-13]. Overtriage occurs when patients are triaged to a higher level of care than their injury severity warrants, leading to increased healthcare costs. In contrast, undertriage occurs when patients with significant injury burdens do not receive the full or appropriate level of TTA, which can lead to adverse outcomes by preventing or delaying the delivery of definitive care [6,7]. The American College of Surgeons Committee on Trauma (ACS COT) aims for triage error rates of less than 5% for undertriage and between 25% and 50% for overtriage [13,14]. Accurate reporting of overtriage and undertriage rates in trauma activations is imperative for optimizing trauma resource utilization and improving trauma care.

The Standardized Triage Assessment Tool (STAT) is used to evaluate the appropriateness of trauma triage and TTA. It evaluates whether a trauma patient was misclassified in terms of injury severity using the Cribari Matrix Method (CMM) and determines if the patient required major trauma intervention based on Need for Trauma Intervention (NFTI) criteria [6,8,15,16]. CMM calculates the overtriage and undertriage using the Injury Severity Score (ISS) and the level of TTA, while the NFTI criteria assess resource utilization and level of TTA, as outlined in figure 1 [5,6,16].

The STAT is a combination of the Cribari Matrix Method (CMM) and the Need for Trauma Intervention (NFTI) tool. CMM calculates the overtriage and undertriage using the Injury Severity Score (ISS) and the level of TTA—in undertriage, a patient is severely injured with ISS greater than 15 but with a TTA other than full; in overtriage, a patient with a minor to moderate ISS (less than or equal to 15) receives the highest level of TTA (Linder et al., 2019). The NFTI criteria assess resource utilization and trauma-related interventions; a positive NFTI (NFTI+) requires at least one of the following: (1) transfusion of packed red blood cells within four hours of ED arrival; (2) discharge from the ED to the operating room within 90 minutes; (3) discharge from the ED to interventional radiology; (4) discharge from the ED to the intensive care unit (ICU) with ICU length of stay of three or more days; (5) use of non-procedural mechanical ventilation within 72 hours; and, (6) mortality within 60 hours of arrival.

Using insights from successful workflow automation in other industries, the objective of this project was to develop an automated, web-based tool for calculating overtriage and undertriage rates of TTA using the STAT. To our knowledge, this is the first tool specifically designed for this purpose: automating the calculation of triage accuracy and optimizing trauma care decision-making.

Methods

Data collection and analysis

We collected de-identified data sets of 1,578 TTAs from the trauma registry of a single American College of Surgeons Committee on Trauma (ACS COT)-verified Level 1 Trauma Center from November 1, 2022, to October 31, 2023, inclusive. At this institution, Level 1 or ‘Red’ traumas require a full trauma team activation, which includes immediate mobilization of both the Emergency Department (ED) and Trauma Surgery teams, comprising the Trauma Surgery attending and resident teams, as well as the ED attending and resident teams. Level 2 or ‘Yellow’ traumas require activation of the ED team and the resident Trauma Surgery team but do not necessitate immediate involvement of the Trauma Surgery attending. Level 3 or ‘Green’ traumas are managed solely by the ED team, who may consult the Trauma Surgery team if they deem it necessary. Trauma overtriage and undertriage for each TTA were determined according to the criteria of the CMM and NFTI used in the STAT as shown in figure 1. All statistical analyses were done using Python version 2022.3.1. The web tool was designed using the website development programming languages Hypertext Markup Language (HTML), and Cascading Style Sheets (CSS).

Figure 1: Workflow of trauma triage assessment using the Standardized Triage Assessment Tool (STAT), used to determine triage rates, applied to our trauma triage activation (TTA) datasets from our Level 1 Trauma Center [6,16].

Application Prototyping and Development

Figure 2 illustrates the prototyping methodology, which involves an iterative process of planning, designing, building, testing, and debugging the application. During the development stage, a graphical user interface (GUI) was constructed, allowing end-users to interact with the application and perform relevant calculations. The application was designed to receive TTA data through a data upload feature that supports Microsoft Excel spreadsheet format. The application was encrypted and secured behind a login requiring authorized credentials for access. Screenshots of the final application are shown in figure 3.

Figure 2: Prototyping methodology for our web-based data analysis tool using an iterative process. Adapted from Soobia et al., 2019.

Figure 3: Screenshots of the tool's graphical user interface (GUI).

3A: Sign-in page. The user must enter authorized credentials to access the application. 3B: Data upload page. The user must upload a Microsoft Excel spreadsheet of TTA data for analysis. 3C: Analysis summary output. Provides the data visualization of the Trauma Overtriage and Undertriage report. The Overtriage graph (top left) displays the Cribari and STAT Analysis results using the 50% Benchmark and the Undertriage graph (bottom left) displays the Cribari and STAT Analysis results using the 5% benchmark according to the ACS COT 2022 benchmarks. On the right, the user can see further summary data by selecting a subset of patients, either patient who was overtriaged (OT), undertriaged (UT), or appropriately triaged (AT), from a dropdown menu. After selection, a graph displays the distribution of ED disposition of the subset of patents, specifically, the number of patients who went to an inpatient floor, a step-down unit (SDU), a surgical trauma ICU (STICU), the operating room (OR), or to interventional radiology (IR). Below the ED disposition graph are the statistics of the selected triage report (OT/UT/AT). Displayed statistics include the average ISS and six NFTI criteria that go into the STAT calculation, and the percentage OT and UT proportions [17].

Application performance testing

Preliminary testing of the application was performed by testing the tool on a mock data set of 15 TTAs and comparing the results with the manual data analysis process conducted by the programmer (author AA). Intermediate testing used a limited dataset of 104 de-identified TTAs spanning a one-month subset of the TTAs collected from the hospital. Final testing was conducted on the completed TTA dataset collected from the hospital, which included 1,578 de-identified TTAs over one year. For both datasets, the Accuracy and Process Efficiency of the automated and manual analysis methods were evaluated to assess the tool’s performance relative to the manual process. Accuracy was determined by comparing the triage classifications (appropriate, overtriage, or undertriage) from the manual calculations with those from the automated tool. It was anticipated that, once provided with complete data in Excel format and correctly programmed to apply the STAT method, the tool would achieve 100% accuracy. Process Efficiency Improvement was calculated as:

Application evaluation

The application and its functionality were presented to a focus group of trauma providers from a level 1 trauma center. The providers were first trained on performing manual calculations using the STAT method, followed by instructions on how to use our tool with a sample dataset. After this, the participants were asked to complete an evaluation survey using a 5-point Likert scale, evaluating the tool’s performance, and their experience with and perception of the tool. Mean ratings for each question, on a scale of 1 to 5, were calculated and reported for all participants.

Results

Performance testing

As expected, the automated data analysis process had 100% accuracy in determining overtriage and undertriage for the mock and actual datasets. The time to complete the analysis of the full dataset was 26 minutes and 19 seconds (1579 seconds) using the manual process, whereas it took 23 seconds using our automated tool (Table 1). The mean process efficiency of the automated data analysis process was 98.0%. This represents a significant time-saving advantage.

Table 1: Process Efficiency Improvement evaluation of our automated tool, where Process Efficiency Improvement = (1- Automated Process/Manual Process) *100%.

Survey findings

Ten participants completed the online survey to evaluate the tool: two Trauma Program Directors, two Performance Improvement/Clinical Data Analysts, a Trauma Medical Director, an Associate Trauma Medical Director, a Trauma Surgeon, a Trauma Liaison, a Trauma Research Associate, and a consultant. The majority of participants had a significant amount of experience in caring for trauma patients and in trauma center leadership roles: six had over six years of experience in trauma center leadership, three had between three and six years of experience, and one had one to two years of experience.

Table 2: Results of the application evaluation survey after administration to ten trauma providers at our Level 1 trauma center

Table 2 shows the results of the evaluation survey. All participants said the application was easy to use, giving the program an average rating of 4.6. They all thought the application significantly reduced data analysis time, with an average rating of 4.9. They also all agreed that the application assisted in identifying mistreated trauma patients during the performance improvement process (mean rating of 4.7) and noted that the tool effectively automated both overtriage and undertriage analyses, each receiving a mean rating of 4.8. The reliability of the tool was deemed to be high, with an average rating of 4.6. All participants indicated they would consider incorporating this tool into their institution/workflow.

Open-ended participant feedback

Participants expressed enthusiasm about the tool’s ability to enhance workflow efficiency, particularly by reducing the time required for data analysis. One participant highlighted its potential value for trauma research. Several others noted that the tool holds great promise for future advancements in data analysis technology for trauma center leadership. A suggested improvement for future development was to add a feature for identifying outliers in the overtriage and undertriage of TTA.

Discussion

We outlined the development and evaluation of an automated, web-based data analysis tool that calculates the mistriage rates of a dataset of TTAs at a trauma center. Findings indicated that our tool represents an accurate and highly time-efficient method for analyzing mistriage. Feedback from experienced trauma providers was overwhelmingly positive, highlighting the tool's perceived efficiency and reliability in streamlining processes. Overall, participants felt the application was easy to use, accurate in calculating overtriage and undertriage rates, and beneficial performance improvement activities, with all respondents agreeing or strongly agreeing with the evaluated metrics.

From a quality improvement perspective, having a reliable tool that can accurately report on rates and trends of overtriage and undertriage of trauma activations while reducing data analysis processing time allows trauma centers to focus on critical aspects of utilization management. This includes developing actionable plans to enhance the quality of patient care delivery and improving overall trauma center operations.

Limitations

One notable limitation of the project was that there was limited end-user interaction with the tool as the institution had high-level security and permission requirements for any application to be installed on local computers. Additionally, the evaluation of the data analysis tool was conducted with a small sample size of participants, which may affect the generalizability of the results. To enhance the reliability of our evaluation, it is recommended to conduct a larger-scale participant study to minimize potential biases and improve the robustness of the findings.

Future considerations

Further development and refinement of our tool will be necessary before broader institutional and external deployment. Future versions of the data analysis tool will include features for summarizing trends in overtriage and undertriage and identifying outliers, as suggested. Additional functionalities will incorporate an export feature for downloading detailed overtriage and undertriage determinations for each analyzed case, as well as tools for handling incomplete data. Ultimately, we plan to provide access to this tool to all trauma providers, starting with our institution and eventually extending to others. This will involve the engagement of the information technology infrastructure and organizational leadership teams at our trauma center to ensure there are adequate security permissions and approvals before deployment. Successful deployment will enable the tool to be utilized for quality improvement initiatives and research projects, enhancing its impact on trauma care and outcomes.

Author contributions

Conceptualization- AA, BS, and MA; writing—original draft preparation- AA, BS, MC, NDB; writing—review and editing- BS, MA, KT, CG, GA, JD, JM, ZS, SC, and JW; figures and table, AA, BS and MA; supervision- BS and JW; project administration- BS and JW

Acknowledgment

We express our gratitude to Aya Nageh Mohamed, who holds a Bachelor of Science in Communication and Information Engineering from the University of Science and Technology in Zewail City, Egypt, for her contribution to the development of this application.

Funding

There is no grant support or financial relationship for this manuscript.

Institutional Review Board (IRB) Statement

This retrospective study has IRB approval

Conflicts of interest

The authors have no competing interests to declare.

References

1. Gopal G, Suter-Crazzolara C, Toldo L, et al. Digital transformation in healthcare - Architectures of present and future information technologies. Clinical Chemistry and Laboratory Medicine 57 (2019): 328-335.
2. Almazmomi N, Ilmudeen A, Qaffas AA. The impact of business analytics capability on data-driven culture and exploration: achieving a competitive advantage. Benchmarking 29 (2022): 1264-1283.
3. Zayas-Cabán T, Haque SN, Kemper N. Identifying Opportunities for Workflow Automation in Health Care: Lessons Learned from Other Industries. Applied Clinical Informatics 12 (2021): 686-697.
4. Choi J, Carlos G, Nassar AK, et al. The impact of trauma systems on patient outcomes. Current Problems in Surgery 58 (2021): 100849.
5. Linder F, Holmberg L, Eklöf H, et al. Better compliance with triage criteria in trauma would reduce costs with maintained patient safety. European Journal of Emergency Medicine 26 (2019): 283-288.
6. Roden-Foreman JW, Rapier NR, Yelverton L, et al. Avoiding Cribari gridlock: The standardized triage assessment tool improves the accuracy of the Cribari matrix method in identifying potential overtriage and undertriage. Journal of Trauma and Acute Care Surgery 84 (2018): 718-726.
7. American College of Surgeons Committee on Trauma. Resources for Optimal Care of the Injured Patient | ACS (2022).
8. Gabriel B, Kimberly G, Siddhartha N, et al. Need for Trauma Intervention and Improving Under-Triaging in Geriatric Trauma Patients: Under-Triaged or Misclassified. International Journal of Critical Care and Emergency Medicine 8 (2022): 136.
9. Lehmann RK, Arthurs ZM, Cuadrado DG, et al. Trauma team activation: simplified criteria safely reduces overtriage. The American Journal of Surgery 193 (2007): 630-635.
10. Rogers A, Rogers FB, Schwab CW, et al. Increased mortality with undertriaged patients in a mature trauma center with an aggressive trauma team activation system. European Journal of Trauma and Emergency Surgery 39 (2013): 599-603.
11. Shawhan RR, McVay DP, Casey L, et al. A simplified trauma triage system safely reduces overtriage and improves provider satisfaction: a prospective study. The American Journal of Surgery 209 (2015): 856-863.
12. Granström A, Strömmer L, Schandl A, et al. A criteria-directed protocol for in-hospital triage of trauma patients. European Journal of Emergency Medicine 25 (2018): 25-31.
13. Stonko DP, O′Neill DC, Dennis BM, et al. Trauma Quality Improvement: Reducing Triage Errors by Automating the Level Assignment Process. Journal of Surgical Education 75 (2018): 1551-1557.
14. Jeppesen E, Cuevas-ØStrem M, Gram-Knutsen C, et al. Undertriage in trauma: An ignored quality indicator? Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 28 (2020): 1-3.
15. Morris RS, Davis NJ, Koestner A, et al. Redefining the Trauma Triage Matrix: The Role of Emergent Interventions. Journal of Surgical Research 251 (2020): 195-201.
16. Harrigan ME, Boremski PA, Collier BR, et al. Impact of nonphysician, technology-guided alert level selection on rates of appropriate trauma triage in the United States: a before and after study. Journal of Trauma and Injury 36 (2023): 231-241.
17. Saeed S, Jhanjhi NZ, Naqvi M, et al. Analysis of software development methodologies. International Journal of Computing and Digital Systems 8 (2019): 445-460.