Artificial intelligence in prehospital care isn't science fiction — it's becoming a practical clinical tool

1. Introduction

RUNSHEET is a web-based educational platform designed for EMT and paramedic students. The application combines several different learning and decision-support features into a single interface, including patient assessment, symptom-to-impression matching, ECG rhythm recognition, medication reference information, and an AI-powered copilot.

The website describes itself as a study tool rather than a clinical protocol. It emphasizes that its recommendations are based on general EMS reference material and that users should follow their local protocols, treatment guidelines, and medical direction.

This study examines the publicly observable functionality of RUNSHEET and proposes how the system could have been architected and implemented. Because the application's private source code, database, deployment configuration, and development history are not publicly available, implementation details that cannot be directly observed are identified as likely architectural approaches rather than confirmed facts.

2. Purpose of the Application

The primary objective of RUNSHEET appears to be providing EMS students with an interactive environment for practicing patient assessment and reviewing common emergency-care concepts.

Instead of presenting information as a traditional static reference manual, the application allows the user to construct a patient scenario.

The main workflow can be summarized as:

Patient information → Clinical findings → Vitals/ECG → Assessment → Ranked impressions → Educational treatment reference

The homepage describes the central assessment feature as allowing users to enter vital signs and select signs, symptoms, and risk factors. The system then ranks conditions that best match the selected findings and displays associated field-treatment information, oxygen targets, and medication doses.

This makes the application more interactive than a conventional medical reference website.

3. Major Components

The public interface exposes several major components.

3.1 Patient Assessment

The Patient section allows users to enter clinical findings and vital signs.

The application indicates that vital signs are optional and that abnormal values can automatically be added as findings. This suggests that the application contains a rule-based preprocessing layer that interprets numerical inputs before running the assessment.

For example, conceptually:

User enters vital signs

↓

Validate input

↓

Identify abnormal values

↓

Convert abnormalities into findings

↓

Combine with selected symptoms

↓

Calculate condition matches

↓

Rank possible impressions

This is an important design decision because it allows the user to provide information in numerical form while the assessment engine works with higher-level clinical findings.

4. Symptom-to-Impression Engine

One of the most important components appears to be the Symptom → Impression engine.

The user selects signs, symptoms, and risk factors. The application then determines which conditions are most compatible with those findings.

A likely implementation would represent each condition as a collection of associated findings.

For example, a simplified internal structure could look conceptually like:

Condition A

finding_1

finding_2

finding_3

Condition B

finding_2

finding_4

finding_5

When a user selects findings, the application can compare the patient's findings against the condition definitions.

A simple scoring approach could be:

score =

number of matching findings

/ number of relevant findings

The conditions could then be sorted from the highest score to the lowest score.

This would allow RUNSHEET to produce a ranked differential-style educational result without requiring a complex machine-learning model.

The website describes the results as conditions that "best match" the information entered, which is consistent with a matching or scoring architecture.

5. Full Scenario Builder

RUNSHEET also provides a more comprehensive Full Scenario workflow.

The public interface states that users can enter:

• Demographics
• Chief complaint findings
• Vital signs
• ECG information

The system then combines the information and produces a ranked impression with field-treatment information.

A conceptual architecture would therefore be:

┌─────────────────┐

│ Patient Profile │

└────────┬────────┘

│

▼

┌──────────────┐ ┌───────────────┐

│ Chief │──►│ Assessment │

│ Complaint │ │ Engine │

└──────────────┘ └───────┬───────┘

│

┌──────────────┐ │

│ Vital Signs │───────────┤

└──────────────┘ │

▼

┌──────────────┐ ┌──────────────┐

│ ECG Reading │───►│ Ranked │

└──────────────┘ │ Impressions │

└──────┬───────┘

│

▼

Treatment Reference

This architecture allows information from multiple parts of a patient presentation to be considered together.

6. Vital Sign Processing

The application supports several common vital-sign measurements.

The public interface exposes heart rate and SpO₂ information and indicates that abnormal vital signs can automatically become assessment findings.

A likely implementation uses predefined ranges or thresholds.

For example, conceptually:

if heart_rate < lower_limit:

findings.append("bradycardia")

if heart_rate > upper_limit:

findings.append("tachycardia")

if spo2 < oxygen_threshold:

findings.append("low oxygen saturation")

The resulting findings can then be passed to the same assessment engine used for manually selected symptoms.

This design avoids having two separate assessment systems.

7. ECG Rhythm Recognition

Another major feature is the cardiac rhythm reference.

The website provides a Cardiac Monitor & Rhythm Reference section containing a live monitor-style interface and a rhythm library. Users can select a rhythm and view a corresponding monitor trace along with associated vital signs.

The site specifically explains that the rhythm strips are stylized for pattern recognition and are not calibrated clinical tracings.

This suggests that the ECG component was designed primarily as an educational visualization rather than as an actual ECG analysis system.

A likely implementation would use predefined rhythm objects containing information such as:

Rhythm

├── Name

├── Heart rate

├── SpO₂

├── ECG pattern

├── Clinical description

└── Treatment reference

The front end can then render the selected rhythm dynamically.

8. ECG Visualization

The live monitor is likely implemented using browser-based graphics rather than displaying a collection of static images.

A web application could generate an ECG-like waveform using:

• HTML Canvas
• SVG
• CSS animation
• JavaScript rendering

A simplified conceptual process would be:

Selected rhythm

↓

Rhythm configuration

↓

Waveform parameters

↓

JavaScript rendering

↓

Animated monitor display

Different rhythm configurations could produce different waveform shapes.

For an educational application, this approach has the advantage of allowing the same visualization component to display many rhythms without requiring a separate image for every scenario.

9. Medication Reference System

RUNSHEET also contains a medication dosage guide.

The site describes this section as a reference containing common adult emergency doses and provides search/filter functionality.

A likely implementation would store medication information in a structured data source.

For example:

Medication

├── Name

├── Category

├── Adult dose

├── Route

├── Indications

├── Contraindications

└── Notes

The user interface can then filter the dataset when a medication name or category is entered.

The site explicitly warns that the displayed doses are common adult reference ranges and that pediatric dosing, contraindications, routes, and maximum doses can vary.

This is particularly important because medication information is safety-critical.

10. AI Copilot

RUNSHEET includes a Paramedic AI Copilot.

The public page indicates that the copilot provides decision-support answers using the same knowledge base referenced throughout the site.

Interestingly, the publicly visible page embeds the AI component separately rather than displaying the entire AI interface as native page content. The page contains an iframe pointing to a Streamlit application.

This suggests an architecture similar to:

RUNSHEET Website

│

│ iframe

▼

AI Copilot Application

│

▼

Knowledge Base / AI System

Using an iframe allows the main website and AI application to be developed and deployed separately.

The visible page also identifies the AI component as a demonstration copilot and emphasizes that it is not a replacement for training, protocols, or medical direction.

11. Front-End Architecture

Based on the publicly observable interface, RUNSHEET appears to use a modern single-page web application approach.

The interface contains multiple interactive sections:

• Patient assessment
• Full scenario
• Rhythm recognition
• Medication search
• AI Copilot

The page dynamically updates controls and results rather than requiring a separate page for every individual clinical condition.

A likely front-end architecture could therefore consist of reusable components such as:

App

├── Navigation

├── PatientAssessment

│ ├── VitalInputs

│ ├── FindingSelector

│ └── AssessmentResults

├── ScenarioBuilder

├── RhythmReference

│ ├── Monitor

│ └── RhythmLibrary

├── MedicationGuide

└── AICopilot

This component-based architecture would make it easier to maintain and extend the application.

12. Data Architecture

The application requires structured clinical knowledge rather than relying entirely on free-form text.

The knowledge base likely contains relationships between:

Symptoms

↓

Conditions

↓

Treatment References

↓

Medications

For example:

Finding

│

├── Condition A

│ ├── Treatment

│ └── Medication

│

└── Condition B

├── Treatment

└── Medication

This structure allows the same underlying information to be reused by several features.

For example, the assessment engine can use condition data while the AI Copilot can use the same knowledge base to answer educational questions.

The website explicitly states that the AI Copilot uses the same knowledge base referenced throughout the site.

13. Likely Technology Stack

The exact private technology stack cannot be established from the public interface alone. However, some components can be identified directly.

The AI Copilot is embedded from a Streamlit application, indicating that at least that component uses Streamlit.

A plausible overall architecture is therefore:

Layer	Likely Technology/Approach
Main website	Modern HTML/CSS/JavaScript web application
Interactive UI	JavaScript-based components
Clinical knowledge	Structured JSON/database
Assessment engine	Rule-based scoring/matching
ECG visualization	Browser-based graphics/animation
Medication search	Structured searchable dataset
AI Copilot	Python/Streamlit application
AI knowledge	Shared/reference knowledge base
Hosting	Web hosting + separate AI application hosting

The items described as "likely" cannot be confirmed without access to the project's source code or deployment configuration.

14. Why a Rule-Based System Makes Sense

A particularly important architectural observation is that the core patient assessment does not necessarily need machine learning.

For an educational EMS application, a transparent rule-based system has several advantages.

A rule can be inspected:

If finding X is present,

increase the score for condition Y.

This is much easier to explain than an opaque prediction.

It also allows the developer to control exactly which findings are associated with each educational impression.

A simple scoring system could therefore be sufficient:

score = 0

for finding in patient_findings:

if finding in condition_findings:

score += condition_findings[finding]

The conditions can then be ranked according to their scores.

This approach also makes it easier to update educational content when reference material changes.

15. User Experience Design

The application appears to have been designed around the workflow of an EMS student.

Rather than presenting hundreds of pages of reference information, the application organizes information around tasks:

Assess → Interpret → Review → Learn

The user can start with symptoms, build an entire patient scenario, practice rhythm recognition, search medications, or ask the AI Copilot a question.

This modular design makes the website useful as both a reference tool and a study environment.

The homepage also repeatedly labels the system as a study tool rather than a protocol, which establishes an important boundary between educational software and clinical decision-making.

16. Safety Considerations

Because the application deals with medical information, safety is a central part of its design.

The website repeatedly warns users that the information should not replace:

• Training
• Certification standards
• Local protocols
• Medical direction
• Professional clinical judgment

The medication section also specifically warns users to confirm doses against their agency's protocols before administering medication.

This type of disclaimer is particularly important when an application provides medication doses or treatment recommendations.

The application should therefore be understood as an educational reference rather than a medical device or autonomous clinical decision-making system.

17. Proposed Development Process

A project similar to RUNSHEET could be developed in the following stages.

Stage 1 — Requirements

Identify the target audience and define the major features:

• Patient assessment
• Scenario builder
• ECG library
• Medication reference
• AI Copilot

Stage 2 — Knowledge Base

Create structured datasets for:

• Clinical findings
• Conditions
• Treatments
• Medications
• ECG rhythms

Stage 3 — Assessment Engine

Implement the symptom-to-impression matching system and scoring logic.

Stage 4 — Front-End

Build the interactive user interface and connect the controls to the assessment engine.

Stage 5 — ECG Visualization

Create reusable waveform/monitor components for the rhythm library.

Stage 6 — Medication Search

Implement searchable and filterable medication information.

Stage 7 — AI Integration

Develop the AI Copilot separately and connect it to the site's reference knowledge.

The publicly visible Streamlit iframe suggests that the AI component can be deployed independently from the main website.

Stage 8 — Testing

Test:

• Input validation
• Assessment results
• Ranking behavior
• ECG rendering
• Medication searches
• AI responses
• Mobile responsiveness
• Safety messaging

Stage 9 — Deployment

Deploy the main website and AI application and connect the components through the web interface.

18. Example System Architecture

The overall system can be represented as:

RUNSHEET

│

┌──────────────┼──────────────┐

│ │ │

▼ ▼ ▼

Patient Tool Rhythm Tool Medication Tool

│ │ │

▼ ▼ ▼

Assessment Rhythm DB Medication DB

Engine │ │

│ │ │

└───────────────┼──────────────┘

│

▼

Shared Knowledge

│

▼

AI Copilot

│

▼

Educational Response

This architecture separates the major responsibilities while allowing them to share a common clinical knowledge base.

19. Strengths of the Design

Several aspects of the observed design are particularly effective.

Interactive learning

Users actively construct scenarios instead of simply reading information.

Modular architecture

Assessment, ECG, medications, and AI are separate functional areas.

Reusable knowledge

The same clinical information can support multiple features.

Visual learning

The ECG monitor provides a visual way to practice rhythm recognition.

Clear safety boundaries

The application repeatedly explains that it is a study tool and not a replacement for protocols or medical direction.

AI integration

The AI Copilot provides a conversational interface for exploring the same educational knowledge base.

20. Limitations of This Study

This study is based on publicly observable behavior and content from RUNSHEET.

The private source code, database schema, version-control history, backend implementation, AI model configuration, hosting infrastructure, and developer documentation were not available for inspection.

Consequently, statements about specific frameworks or algorithms beyond what can be directly observed should be considered proposed or likely implementations rather than confirmed facts.

The publicly visible site does, however, provide enough information to identify the major functional architecture and to develop a reasonable technical model of how the application could have been constructed.

21. Conclusion

RUNSHEET demonstrates how an EMS educational application can combine structured clinical knowledge, interactive assessment, visualization, searchable references, and artificial intelligence into one web-based platform.

Its core architecture can be understood as several interconnected components: a patient assessment engine, a scenario builder, an ECG rhythm visualization system, a medication reference database, and an independently embedded AI Copilot.

The assessment functionality appears well suited to a rule-based matching architecture, while the ECG system can use predefined rhythm data and browser-based visualization. The medication guide can operate from a structured searchable dataset, and the AI Copilot can be deployed as a separate application connected to the site's educational knowledge.

The result is a system that transforms a conventional EMS reference guide into an interactive learning environment.

Most importantly, RUNSHEET illustrates that an effective medical education application does not necessarily require a complex machine-learning model at its core. A carefully structured knowledge base, transparent rules, intuitive user interface, and appropriately bounded AI system can together provide a powerful educational experience.

Source examined: RUNSHEET — EMS Field Reference, publicly available at runsheet.website. The site identifies itself as an EMS study tool and states that its information should be verified against local protocols and medical direction.

Abstract

Background

Clinical decision-support systems and artificial intelligence (AI)-assisted tools are increasingly being evaluated for their ability to recognize clinical presentations and generate appropriate treatment recommendations. Accurate identification of emergency medical conditions is particularly important because errors in recognition or treatment recommendations may have significant consequences.

Objective

This study evaluated the accuracy of an AI-assisted clinical decision-support software system in identifying appropriate medical treatments across a standardized set of emergency and medical scenarios.

Methods

A benchmark consisting of 50 clinical questions was developed across eight medical conditions representing respiratory, cardiovascular, allergic, gastrointestinal, and abdominal presentations. The conditions evaluated were bronchospasm/asthma, spontaneous pneumothorax, respiratory failure, impending cardiac arrest, severe allergic reaction/anaphylaxis, appendicitis, gastrointestinal bleeding, and gallstones/cholecystitis. Five questions were evaluated for each condition. The software's responses were graded for medical treatment accuracy.

Results

The software achieved a treatment accuracy of 100% across all 50 questions. No treatment recommendation was graded as incorrect in the evaluated benchmark. The system therefore achieved a 50/50 correct treatment-recommendation score, corresponding to an observed accuracy of 100%.

Conclusion

Within the conditions and questions included in this benchmark, the evaluated software demonstrated complete agreement with the predetermined treatment-answer criteria. These findings support further investigation of the system as a potential clinical decision-support tool. However, the results represent performance on a limited 50-question benchmark and should not be interpreted as evidence of clinical safety, efficacy, or readiness for independent use in patient care. Larger, independently developed, clinically representative, and prospectively validated studies are required.

1. Introduction

Artificial intelligence and computer-assisted clinical decision-support systems have the potential to assist healthcare professionals in recognizing medical conditions and selecting appropriate interventions. In emergency and prehospital medicine, rapid recognition of clinical patterns and appropriate treatment selection are particularly important because clinical deterioration may occur rapidly.

Respiratory emergencies such as asthma/bronchospasm, spontaneous pneumothorax, and respiratory failure require recognition of characteristic clinical findings and prompt selection of appropriate interventions. Similarly, cardiovascular emergencies, anaphylaxis, gastrointestinal hemorrhage, appendicitis, and biliary disease require clinicians to distinguish potentially serious presentations and determine appropriate management.

Despite increasing interest in AI-assisted medical decision support, demonstrating performance requires systematic evaluation against predefined clinical criteria. A system that performs well on a controlled benchmark may still perform differently when confronted with more complex patients, incomplete information, atypical presentations, comorbidities, medication interactions, or situations outside the benchmark.

The present study was designed as an initial benchmark evaluation of treatment recommendation accuracy. The objective was to determine whether the evaluated software could provide medically appropriate treatment recommendations across a predefined set of 50 clinical questions.

The primary outcome was the proportion of questions for which the software's treatment recommendation was graded as medically accurate according to the predetermined evaluation criteria.

2. Materials and Methods

2.1 Study Design

This investigation was conducted as a structured benchmark evaluation of an AI-assisted clinical decision-support software system. The evaluation consisted of 50 clinical questions distributed across eight medical conditions.

Five questions were evaluated for each condition.

2.2 Clinical Conditions

The benchmark included the following conditions:

• Bronchospasm/Asthma
• Spontaneous Pneumothorax
• Respiratory Failure
• Impending Cardiac Arrest
• Severe Allergic Reaction/Anaphylaxis
• Appendicitis
• Gastrointestinal Bleeding
• Gallstones/Cholecystitis

These conditions were selected to represent a range of respiratory, cardiovascular, allergic, gastrointestinal, and abdominal clinical presentations.

2.3 Clinical Presentation Domains

The questions incorporated characteristic clinical findings associated with the target conditions.

For bronchospasm/asthma, the benchmark included findings such as prolonged expiration and bilateral wheezing.

For spontaneous pneumothorax, scenarios included sudden shortness of breath and chest pain with diminished breath sounds on the affected side.

Respiratory failure scenarios incorporated findings such as altered mental status, inability to speak in complete sentences, and respiratory arrest requiring positive-pressure ventilation.

Cardiovascular scenarios included symptoms associated with impending cardiac deterioration, including dyspnea, unusual fatigue, chest discomfort, nausea/vomiting, and back pain.

Anaphylaxis scenarios incorporated findings including acute respiratory difficulty, chest tightness, swelling involving the tongue, face, or neck, and urticaria.

Abdominal and gastrointestinal scenarios included characteristic findings of appendicitis, gastrointestinal hemorrhage, and gallbladder disease. These included right-lower-quadrant abdominal pain, melena, hematochezia, hematemesis, and right-upper-quadrant or epigastric pain with characteristic radiation.

2.4 Evaluation Procedure

Each clinical condition was represented by five questions, producing a total of 50 evaluated questions.

The software was provided the clinical questions and its treatment recommendations were evaluated against the predetermined medical answer criteria.

The primary evaluation metric was treatment recommendation accuracy.

A response was classified as correct when the treatment recommendation satisfied the predefined medical-treatment criteria for the corresponding question.

2.5 Outcome Measure

The primary outcome was calculated as:

Treatment Accuracy=Number of Correct Treatment RecommendationsTotal Number of Questions×100\text{Treatment Accuracy} = \frac{\text{Number of Correct Treatment Recommendations}} {\text{Total Number of Questions}} \times 100

The maximum possible score was 50 correct treatment recommendations out of 50 questions.

3. Results

3.1 Overall Performance

The software achieved a 100% treatment recommendation accuracy rate across the 50-question benchmark.

Metric	Result
Total questions	50
Questions graded correct	50
Questions graded incorrect	0
Overall treatment accuracy	100%

The observed accuracy was therefore:

5050×100=100%\frac{50}{50}\times100 = 100\%

No incorrect treatment recommendations were identified within the evaluated benchmark.

3.2 Performance by Clinical Category

Five questions were evaluated for each of the eight clinical conditions.

Clinical condition	Questions	Correct	Accuracy
Bronchospasm/Asthma	5	5	100%
Spontaneous Pneumothorax	5	5	100%
Respiratory Failure	5	5	100%
Impending Cardiac Arrest	5	5	100%
Severe Allergic Reaction/Anaphylaxis	5	5	100%
Appendicitis	5	5	100%
Gastrointestinal Bleeding	5	5	100%
Gallstones/Cholecystitis	5	5	100%
Total	50	50	100%

The system therefore maintained the same observed accuracy across every clinical category represented in the benchmark.

4. Discussion

The principal finding of this benchmark evaluation was that the evaluated software achieved 100% treatment recommendation accuracy across all 50 questions. Every question included in the evaluation received a treatment recommendation that met the predetermined grading criteria.

The finding is notable because the benchmark included multiple clinical domains rather than a single disease category. Respiratory emergencies, cardiovascular deterioration, anaphylaxis, gastrointestinal bleeding, appendicitis, and biliary disease present substantially different clinical management considerations. The software nevertheless achieved complete observed accuracy within the tested question set.

However, the interpretation of a 100% benchmark score requires caution. A perfect score on 50 questions does not establish that the software will achieve 100% accuracy in real-world clinical practice. The benchmark represents a finite sample of possible clinical presentations, and the performance estimate may change substantially with a larger or independently constructed question set.

In addition, clinical cases frequently contain ambiguity, incomplete information, conflicting findings, multiple simultaneous diagnoses, medication contraindications, allergies, age-related considerations, pregnancy, comorbid disease, and rapidly changing patient status. These factors may not be adequately represented by a controlled benchmark.

The distinction between benchmark accuracy and clinical effectiveness is therefore important. The current results demonstrate that the software successfully answered the questions included in this evaluation according to the specified grading criteria. They do not independently establish patient outcomes, clinical safety, diagnostic performance, or superiority over qualified healthcare professionals.

Another important consideration is potential benchmark bias. If the questions, expected answers, or evaluation criteria were known to the system developer or closely reflected the information used to develop the software, performance could overestimate generalization to previously unseen cases. Future studies should therefore use independently constructed test sets and, where possible, blinded external evaluation.

5. Limitations

Several limitations should be considered.

First, the sample size was limited to 50 questions. Although the system achieved perfect observed accuracy, a relatively small benchmark cannot establish reliable performance across the full range of clinical scenarios.

Second, only eight clinical conditions were evaluated. Many important emergency and medical conditions were not included.

Third, the questions may not represent the complexity of actual patient encounters. Real-world patients frequently present with incomplete histories, multiple simultaneous conditions, atypical symptoms, and clinically relevant comorbidities.

Fourth, the evaluation measured treatment recommendation accuracy rather than patient outcomes. The study therefore cannot determine whether use of the software would improve morbidity, mortality, time to treatment, or other clinical outcomes.

Fifth, the grading criteria and question construction may influence the measured accuracy. Independent replication using externally generated cases would provide stronger evidence of generalizability.

Finally, the results do not establish that the software should be used autonomously to diagnose or treat patients. Clinical decision-support systems should be evaluated within appropriate clinical, regulatory, and professional frameworks.

6. Recommendations for Future Research

Future evaluations should expand the benchmark substantially and include a larger number of clinical conditions and patient presentations.

Particularly valuable next steps would include:

• Independent external validation using questions not used during system development.
• Evaluation on several hundred or thousands of clinical cases.
• Inclusion of atypical and ambiguous presentations.
• Evaluation of pediatric and geriatric cases.
• Testing cases involving multiple simultaneous diagnoses.
• Evaluation of medication contraindications and clinically important comorbidities.
• Comparison with practicing physicians, paramedics, nurses, or other appropriate clinicians.
• Assessment of inter-rater agreement among clinical evaluators.
• Prospective evaluation in simulated clinical environments.
• Measurement of sensitivity, specificity, positive predictive value, and negative predictive value where the study design supports these measures.
• Evaluation of treatment omissions, harmful recommendations, and inappropriate treatment escalation separately from simple correctness.
• Assessment of whether the system appropriately recognizes when a case requires emergency escalation or human clinical judgment.

These studies would provide substantially stronger evidence regarding the system's generalizability and clinical utility.

7. Conclusion

In this 50-question benchmark evaluation, the assessed AI-assisted clinical decision-support software achieved 100% treatment recommendation accuracy, with 50 of 50 evaluated questions receiving recommendations that satisfied the predetermined grading criteria.

The system achieved 100% observed accuracy in each of the eight evaluated clinical categories: bronchospasm/asthma, spontaneous pneumothorax, respiratory failure, impending cardiac arrest, severe allergic reaction/anaphylaxis, appendicitis, gastrointestinal bleeding, and gallstones/cholecystitis.

These findings demonstrate strong performance on the evaluated benchmark. Nevertheless, a perfect score on a limited question set should be interpreted as an initial validation result rather than evidence of universal clinical accuracy or safety. Independent, larger-scale, clinically representative validation is necessary before conclusions can be made regarding real-world clinical effectiveness or autonomous clinical use.

8. Data Availability

The underlying benchmark questions, scoring criteria, and complete software outputs should be made available to reviewers or readers where permitted. Providing the complete evaluation dataset would facilitate independent replication of the reported findings.

9. Conflict of Interest

The authors should disclose any financial, employment, ownership, development, or other relationship with the software evaluated in this study.

10. Ethics Statement

Because this evaluation involved a benchmark of clinical questions rather than direct intervention with human patients, formal human-subject research requirements may not apply. The applicable institutional and regulatory requirements should nevertheless be reviewed and reported by the investigators.

11. References

The final manuscript should include authoritative clinical references supporting the diagnostic features and treatment standards used to construct and grade the benchmark. Educational websites or question-bank materials may be useful for identifying source material, but treatment recommendations in a scientific manuscript should preferably be supported by current professional guidelines, peer-reviewed literature, and authoritative clinical references.

RUNSHEET: Development of a Self-Contained, Browser-Based Reference Tool for Medication Reconstitution and Clinical Reasoning in Paramedic Education

Medication reconstitution — combining a powdered drug with a diluent to reach an administrable concentration — sits at the intersection of mathematics, pharmacology, and fine motor skill, and represents a persistent source of error in prehospital and hospital care alike. Paramedic students in particular have been shown to struggle with drug calculation generally, and reconstitution adds a further layer of concentration math, drug-specific technique, and infrequent practice on top of that existing weakness. This paper describes the development and design rationale of RUNSHEET, a free, self-contained, browser-based educational reference tool built to help paramedic students and EMS providers practice symptom-to-condition reasoning alongside medication dosing, reconstitution, and field treatment information. The tool uses a weighted matching system that takes user-entered signs and symptoms and returns ranked likely conditions, each linked to associated risk factors, field treatment steps, oxygen therapy guidance, and — where applicable — medication dosing and reconstitution instructions. This paper situates the tool's design within the existing literature on paramedic dosage calculation performance, the effect of stress on calculation accuracy, and the role of e-learning in supplementing hands-on skills training. No formal evaluation of student outcomes has yet been conducted; this paper is intended as a descriptive account of the tool and a foundation for future pilot evaluation.

emergency medical services; paramedics; education; medication systems; dosage calculation; drug compounding; patient safety; e-learning; clinical decision support; point-of-care systems

Introduction

Medication administration errors in the prehospital setting are well documented. In a survey of paramedics in San Diego County, 9.1% of respondents reported committing a medication error in the previous 12 months, with dose-related errors accounting for the largest share (63%) and contributing factors including infrequent use of the medication in question and dosage calculation error [1]. Reconstitution — the process of mixing a powdered medication with a diluent to reach a usable concentration — compounds these risks, since an error introduced at the preparation stage propagates into every subsequent calculation and administration step even when later arithmetic is performed correctly.

Hospital-based research illustrates how substantial these preparation-stage errors can be. A prospective observational study at a Dutch university hospital found errors in well over half of intravenous admixtures observed, with incorrect preparation technique and incorrect diluent volume among the most frequent problems identified before the medication reached the patient [2]. A separate Brazilian study of drug preparation and administration errors identified instances in which medication was diluted to a volume below the manufacturer's recommendation, directly compromising the intended dose [3]. While these studies were conducted in hospital rather than prehospital settings, the underlying mechanism — an error at the mixing stage undermining every downstream step — applies directly to EMS medication preparation as well.

Two additional factors specific to EMS education make reconstitution a persistent challenge. First, dosage calculation is already a documented area of difficulty for paramedic students independent of reconstitution: research examining mathematics anxiety and numerical ability among paramedic students found that math education prior to university entry and numerical ability were significant predictors of drug calculation performance, and that many students experience measurable anxiety around calculation tasks [4]. Second, calculation accuracy has been shown to deteriorate under stress. In a simulator-based study, paramedics' accuracy in calculating drug dosages fell substantially in stressful scenarios compared to calm conditions, regardless of the participant's level of experience [5]. Reconstitution in EMS practice is rarely performed under calm, unhurried conditions — it typically occurs mid-call, alongside patient assessment and communication demands, which is precisely the kind of environment shown to degrade calculation performance.

Taken together, these findings suggest that reconstitution-related error in EMS is not attributable to any single deficiency, but to the layering of several factors: baseline calculation difficulty, infrequent practice of drug-specific mixing steps, and performance degradation under operational stress. Traditional paramedic education typically addresses reconstitution through a single skills-station demonstration, which offers limited opportunity for the repeated, low-stakes practice that these findings suggest is needed. Digital tools may help close this gap: a recent randomized, quasi-experimental study of paramedic students found that a blended online-and-classroom model produced significantly greater improvement in dosage calculation scores than video-based e-learning alone, suggesting that structured digital practice can meaningfully improve this specific skill set [6]. This paper describes RUNSHEET, a browser-based tool developed to provide this kind of repeatable, self-directed practice.

Tool Description

RUNSHEET is a self-contained HTML, CSS, and JavaScript application that runs entirely in a web browser, requiring no server, account creation, or installation. It was designed for paramedic and EMT students to reference and practice medication and clinical reasoning content outside of scheduled lab or classroom time. Content included in the tool — specific medications, dosing ranges, and treatment protocols — was developed with reference to the National EMS Scope of Practice Model published by the National Highway Traffic Safety Administration (NHTSA), which outlines the knowledge, skills, and medication categories associated with each EMS provider level. Because actual scope of practice, medications carried, and specific protocols vary by state regulation, medical director approval, and local EMS agency, the tool is intended as a general study reference rather than a substitute for an individual program's protocols.

Core Features

Search functionality: Users can search directly for a medication, symptom, or condition rather than browsing a static list, mirroring the speed with which information typically needs to be retrieved in the field.

Condition calculation based on signs and symptoms: The core function of the tool is a weighted matching system. Users enter observed signs and symptoms along with vitals, organized across these categories (respiratory, cardiac/circulatory, neurological, skin/allergic, and gastrointestinal/general) (heart rate, blood pressure systolic/diastolic, respiratory rate, spO2, temperature, and blood glucose level) , and the system returns a ranked list of likely underlying conditions. This is intended to give students practice with the less-rehearsed half of clinical reasoning — working backward from a presentation to a probable cause — rather than only reviewing information about a condition that has already been named.

Medication cards: Each medication is presented in a consistent card format listing indications, contraindications, standard dosing, and route of administration, so that information remains scannable and comparable across the full medication set.

Reconstitution instructions: For medications requiring reconstitution, each card includes step-by-step instructions specifying the correct diluent, volume, and resulting concentration, allowing repeated review of the exact process for infrequently used drugs such as glucagon.

Dosage calculation practice: The tool includes built-in dosage calculation exercises tied to specific medications, giving users a low-stakes setting in which to repeatedly practice the underlying arithmetic rather than only reading about it.

Risk factors: Each condition and medication entry lists relevant patient-specific risk factors (e.g., age, weight, comorbidities, contraindicated combinations) that may increase the likelihood of an adverse outcome or point toward a specific diagnosis.

Field treatment: Each condition entry includes a recommended sequence of field interventions, from initial assessment priorities through medication administration, linking diagnostic reasoning to a concrete treatment pathway.

Oxygen treatment: Relevant condition entries include oxygen therapy guidance — target saturation range, delivery method, and flow rate — reflecting that even high-frequency interventions require patient-specific decision-making.

Potential Benefits for Paramedic Education

Because reconstitution and dosage calculation are skills that appear to decay with infrequent use [1], a resource that students can access repeatedly and without cost may help provide the kind of low-stakes repetition that a single in-person skills station cannot. The symptom-to-condition matching feature is intended to give students additional practice with diagnostic reasoning specifically, rather than only pharmacological recall, and the integration of risk factors, field treatment, and medication information into a single workflow is intended to mirror how these elements are used together on an actual call rather than as isolated topics of study.

Limitations

This tool has several limitations that should be considered. It is a static, browser-based application and therefore cannot replicate the physical component of reconstitution, such as drawing up fluid, mixing technique, or needle handling — skills that require hands-on laboratory practice. The symptom-matching algorithm reflects general clinical patterns encoded by the developer and is not a validated diagnostic instrument; it is intended solely as a study aid. The tool was developed and is maintained by a single individual rather than a clinical education team or institution, and its content has not undergone formal peer review, multi-site testing, or validation against paramedic education outcomes. Most importantly, no formal evaluation of student performance or learning outcomes with this tool has yet been conducted; the benefits described above are proposed based on alignment with existing literature rather than demonstrated empirically.

Future Directions

Planned future work includes formal pilot evaluation of the tool — for example, comparing dosage calculation or condition-recognition performance among paramedic students who use the tool against those who do not, following a design similar to prior blended-learning research [6]. Additional planned enhancements include timed or scenario-based practice modes intended to introduce a controlled degree of time pressure, given that stress has been shown to degrade calculation accuracy even among experienced providers [5]; periodic expansion and updating of the medication and condition library to remain aligned with evolving protocols; and the addition of performance tracking to identify individual students' recurring areas of difficulty.

Conclusion

Medication reconstitution occupies a difficult intersection of mathematics, pharmacology, and physical technique, compounded by the infrequency with which many relevant medications are used and by the performance-degrading effects of operational stress. Existing literature indicates that dosage calculation is already a measurable weak point among paramedic students, that preparation-stage errors are linked to serious patient harm, and that infrequent practice is a recurring contributing factor in real-world medication errors. RUNSHEET was developed as a freely accessible, self-contained tool intended to provide the kind of repeatable, low-stakes practice suggested by this literature, combining diagnostic reasoning, risk factor review, field treatment, and medication dosing and reconstitution practice into a single workflow. It is intended as a supplement to — not a replacement for — hands-on laboratory instruction and clinical supervision, and its educational value remains to be formally evaluated.

Conflicts of Interest

None declared

Funding

The authors received no specific funding for this work.

References

• 1. Vilke GM, Tornabene SV, Stepanski B, Shipp HE, Ray LU, Metz MA, et al. Paramedic self-reported medication errors. Prehosp Emerg Care. 2007;11(1):80-84. doi:10.1080/10903120601021358. PMID: 17169883.
• 2. Jessurun JG, Hunfeld NGM, van Rosmalen J, van Dijk M, van den Bemt PMLA. Prevalence and determinants of intravenous admixture preparation errors: a prospective observational study in a university hospital. Int J Clin Pharm. 2022;44(1):44-52. doi:10.1007/s11096-021-01310-6. PMID: 34363192.
• 3. Mendes JR, Lopes MCBT, Vancini-Campanharo CR, Okuno MFP, Batista REA. Types and frequency of errors in the preparation and administration of drugs. Einstein (São Paulo). 2018;16(3):eAO4146. doi:10.1590/S1679-45082018AO4146. PMID: 30231142.
• 4. Khasawneh E, Gosling C, Williams B. The correlation between mathematics anxiety, numerical ability and drug calculation ability of paramedic students: an explanatory mixed method study. Adv Med Educ Pract. 2020;11:869-878. doi:10.2147/AMEP.S258223. PMID: 33235536.
• 5. LeBlanc VR, MacDonald RD, McArthur B, King K, Lepine T. Paramedic performance in calculating drug dosages following stressful scenarios in a human patient simulator. Prehosp Emerg Care. 2005;9(4):439-444. doi:10.1080/10903120500255255. PMID: 16263679.
• 6. Baran L, Öztürk H. The effects of video-based and blended learning on medication dosage calculation skills of paramedic students: a randomized, quasi-experimental study. Medicine (Baltimore). 2025;104(31):e43651. doi:10.1097/MD.0000000000043651. PMID: 40760542.
• 7. National Highway Traffic Safety Administration. National EMS Scope of Practice Model. Washington (DC): U.S. Department of Transportation; [edition/year to be confirmed and cited directly from NHTSA.gov].

PARAMEDIC AI: Development of a Browser-Based Artificial Intelligence and Machine-Learning Decision-Support Tool for Paramedic Education

Artificial intelligence (AI) and machine learning (ML) are increasingly being incorporated into healthcare education and decision-support systems, creating opportunities for paramedic students to interact with clinical information in ways that extend beyond traditional static reference materials. Paramedic education presents a particularly relevant environment for exploring these technologies because learners must integrate vital-sign interpretation, clinical reasoning, evidence retrieval, communication, and rapid decision-making while maintaining awareness of protocol and scope-of-practice limitations. This paper describes the development and design rationale of Paramedic AI, a browser-based educational and decision-support demonstration designed for EMS learners. The application combines a demonstration machine-learning risk assessment with an AI-powered clinical copilot, voice input, text-to-speech functionality, and a document-based knowledge-management system. Users can enter patient age and vital signs, obtain an estimated ML risk probability and classification, and ask the Paramedic Copilot questions using typed or spoken input. The Copilot retrieves information from an authorized knowledge base and provides responses accompanied by references when available. The Knowledge Manager permits authorized PDF and Word documents to be uploaded, assigned source metadata, and incorporated into the searchable knowledge index. The application is explicitly designed as an educational demonstration rather than a diagnostic or treatment system, and its interface emphasizes that outputs should not replace local EMS protocols, medical direction, scope of practice, or clinical judgment. No formal evaluation of student learning outcomes, diagnostic performance, or clinical safety has yet been conducted. This paper presents the tool's design and educational rationale and provides a foundation for future evaluation of AI-supported learning in paramedic education.

artificial intelligence; machine learning; emergency medical services; paramedic education; clinical reasoning; decision support; EMS education; generative AI; medical education; e-learning; knowledge retrieval; patient assessment

Introduction

Paramedics are required to synthesize multiple sources of information rapidly, including patient history, physical examination findings, vital signs, clinical presentation, protocols, and medical direction. Traditional EMS education provides learners with structured clinical knowledge and hands-on skills practice, but the increasing availability of artificial intelligence creates additional possibilities for learners to interact with clinical information and practice reasoning in a more dynamic environment.

Machine-learning systems and generative AI systems represent two different but potentially complementary approaches to educational support. Machine-learning models can identify patterns in structured data and produce classifications or probability estimates, while generative AI systems can interact with users through natural language and retrieve or synthesize information from reference materials. In paramedic education, these capabilities may provide opportunities for learners to explore the relationship between patient assessment data and predicted risk while simultaneously practicing clinical questioning and evidence evaluation.

The educational value of these technologies depends heavily on how they are implemented. A system that simply produces an answer risks encouraging automation bias, in which users place excessive confidence in an automated recommendation. This is particularly important in EMS, where patient presentations are often incomplete, information can change rapidly, and treatment decisions are governed by local protocols, medical direction, scope of practice, and regulatory requirements.

Paramedic AI was therefore designed as an educational demonstration rather than as an autonomous clinical decision-making system. The application combines structured patient-vital input, a demonstration ML risk model, a retrieval-supported AI Copilot, voice interaction, text-to-speech, and a source-management system. The design intentionally places the learner in the role of evaluator rather than passive recipient of an automated answer.

This paper describes the development and functionality of Paramedic AI, its potential applications in paramedic education, its limitations, and potential directions for future evaluation.

Tool Description

Paramedic AI is a browser-based Streamlit application developed as an EMS education and decision-support demonstration. The application presents a single interface through which users can enter patient vital signs, run a machine-learning risk assessment, interact with an AI Copilot, access retrieved reference information, and manage authorized EMS reference documents.

The application collects seven structured patient variables:

• Age
• Heart rate
• Systolic blood pressure
• Diastolic blood pressure
• Respiratory rate
• SpO₂
• Temperature

These variables are displayed in a patient-vital-sign summary before the learner runs the ML assessment.

The application then passes the entered values to a machine-learning model that generates an estimated risk probability and assigns the result to one of three categories: LOWER RISK, INTERMEDIATE RISK, or HIGHER RISK. The interface explicitly identifies this as a demonstration prediction and states that it is not a clinical diagnosis.

The second major component is the Paramedic Copilot. Users can ask questions using either typed text or speech recognition. Speech input is transcribed and presented to the learner for review before submission, allowing the user to correct transcription errors. The submitted question is then processed through a knowledge-search function before being provided to the Copilot. The application limits both the retrieved context and conversation history before generating the response.

The Copilot response may include references to the sources retrieved from the knowledge base. This design provides an opportunity for learners to evaluate not only the AI-generated answer but also the evidence supporting it.

Core Features

Structured patient assessment: Users enter age and vital-sign information through dedicated assessment fields. The values are then displayed in a standardized patient-vital-sign summary, providing a consistent starting point for ML assessment activities.

Machine-learning risk assessment: The application uses a demonstration ML model to process the seven entered patient variables. The output includes an estimated probability and a categorical risk classification. The system is designed to demonstrate how structured clinical information can be processed by a predictive model rather than to provide a diagnosis.

Risk classification: The model output is translated into three categories: lower risk, intermediate risk, and higher risk. This gives learners an opportunity to examine how continuous probability estimates can be translated into categorical outputs.

Patient-data review: After an assessment is run, the application provides an expandable view of the patient data submitted to the model. This allows learners to verify that the intended values were actually entered and used.

Paramedic Copilot: The Copilot provides a conversational interface through which users can ask EMS-related questions. Rather than functioning solely as a general-purpose chatbot, the implementation first searches the application's knowledge base and supplies retrieved context to the AI response system.

Medication dosing and pharmacology support: The Paramedic Copilot can retrieve medication-related information from the application's authorized EMS knowledge base. Learners can ask questions about medication dosing and related pharmacological information and receive an AI-generated response based on retrieved reference material. This feature is intended to support medication knowledge review and clinical reasoning practice rather than replace agency-specific medication protocols, medical direction, or independent dose verification. Because medication dosing can vary according to patient characteristics, medication formulation, route, jurisdiction, and local protocol, users should verify any dosing information against the current authoritative source before applying it in practice.

Medication-related clinical reasoning: Medication questions can be incorporated into broader patient scenarios, allowing learners to connect patient presentation, vital signs, clinical considerations, and medication information. Rather than treating medication dosing as an isolated memorization exercise, the Copilot can be used to prompt learners to consider why a medication may be indicated, what patient-specific factors could affect its use, and what information should be verified before administration.

Reference-supported medication information: When the Copilot retrieves medication information from the application's knowledge base, associated references can be displayed with the response. This provides an opportunity for learners to compare AI-generated dosing information against the underlying source and reinforces the importance of source verification in medication administration.

Knowledge-grounded responses: The Copilot searches the application's authorized knowledge base before generating an answer. Retrieved material is formatted as context and supplied to the Copilot, creating a mechanism for grounding responses in designated reference material.

Reference display: When available, references associated with retrieved knowledge are appended to the Copilot response. This allows learners to examine the source material behind an AI-generated answer rather than treating the generated response as an independent authority.

Voice input: Users can speak questions through the application's speech-to-text interface. The resulting transcription is displayed for review before submission, allowing users to identify and correct transcription errors.

Response review: The application preserves Copilot conversation history within the current session. This allows learners to review previous questions and answers and observe how conversational context influences subsequent interactions.

Text-to-speech: The application can convert the latest Copilot response into audio. This provides an additional interaction modality and may support accessibility and alternative learning preferences.

Knowledge Manager: Authorized users can upload PDF and Word reference documents. Each source can be associated with metadata including title, jurisdiction, document type, effective date, version, authority, status, and review requirements.

Source-status management: Registered sources can be categorized as active, draft, inactive, retired, or superseded. Sources can also be flagged as requiring review, providing a basic mechanism for communicating the review status of educational material.

Knowledge-index rebuilding: Uploaded documents can be processed through an ingestion and indexing workflow. This allows newly authorized reference material to become available to the knowledge-search component used by the Copilot.

Educational Use Cases

Paramedic AI is designed to support several forms of formative learning.

The first is structured vital-sign reasoning. Learners can enter patient data and observe the output of a predictive model. By changing individual variables while holding others constant, students can explore how structured inputs influence model predictions.

The second is clinical-question formulation. The Copilot provides learners with an opportunity to formulate questions about patient assessment, clinical concepts, and EMS practice using natural language rather than selecting information from a predetermined menu.

The third is evidence evaluation. Because the Copilot retrieves reference material and can display associated sources, learners can be encouraged to distinguish between an AI-generated statement and the authoritative evidence supporting that statement.

The fourth is AI literacy. Students can explore situations in which an AI response may appear plausible while still requiring verification. This can support instruction on hallucination, incomplete information, automation bias, source quality, and appropriate human oversight.

The fifth is repeated low-stakes practice. Unlike a single classroom demonstration, a browser-based application can be accessed repeatedly, allowing students to experiment with patient presentations, questions, and source material without requiring a full simulation-lab setup.

Potential Benefits for Paramedic Education

One potential benefit of Paramedic AI is the integration of multiple learning activities within a single environment. Patient assessment, predictive modeling, natural-language questioning, evidence retrieval, and source evaluation are presented as connected activities rather than isolated topics.

The ML assessment may provide an accessible introduction to predictive modeling by allowing students to observe the relationship between structured patient data and model output. Importantly, the application can also be used to demonstrate why model output should not be equated with clinical diagnosis. Students can compare the information available to the model with the substantially larger information set considered during a complete paramedic assessment.

The Copilot component provides a different educational opportunity. Instead of requiring students to search a static reference manually, the system allows them to formulate questions conversationally and receive an answer grounded in the application's designated knowledge sources. This may support exploratory learning and encourage students to ask follow-up questions.

The reference component is particularly important from an educational perspective. Students can be taught that the quality of an AI answer depends not only on the language-generation system but also on the information supplied to it. Reviewing source metadata can reinforce the importance of jurisdiction, effective dates, document versions, issuing authorities, and source status.

Voice interaction and text-to-speech further expand the possible educational uses of the system. These features may allow learners to interact with educational content using modalities other than conventional keyboard input and written output.

Formative Learning Activities

The application can be incorporated into structured formative exercises.

Vital-sign manipulation: Students can establish a baseline patient assessment and then alter one vital sign at a time. They can record changes in the model's probability and classification and discuss why predictive output should not be interpreted independently of the broader patient presentation.

Clinical reasoning comparison: Students can compare their own initial assessment with the information considered by the ML model. This encourages discussion of the difference between structured numerical prediction and comprehensive clinical reasoning.

AI question formulation: Students can ask the Copilot questions related to an assigned EMS scenario and evaluate whether the response addresses the clinical question appropriately.

Reference verification: Students can identify the sources returned with a Copilot response and determine whether those sources are current, authoritative, and applicable to the relevant jurisdiction.

AI error recognition: Instructors can provide scenarios in which students must identify information in an AI response that requires independent verification.

Source-management exercise: Students can upload an authorized educational reference, enter its metadata, and discuss why document status, version, jurisdiction, and issuing authority matter when constructing an EMS knowledge base.

Safety and Clinical Boundaries

Because the application addresses clinical information, clear boundaries are necessary.

The application's ML prediction is explicitly labeled as a demonstration and not a clinical diagnosis. The Copilot likewise should not be treated as an independent clinical authority. AI-generated information may be incomplete, inaccurate, outdated, or inappropriate for a particular patient or jurisdiction.

The application therefore directs users to follow current local EMS protocols, medical direction, scope of practice, manufacturer instructions, and applicable regulations.

The educational design should emphasize that technology can support learning and information retrieval without transferring clinical responsibility from the practitioner to the software.

The system should also be used without unnecessary patient-identifying information. Educational demonstrations should use simulated or appropriately de-identified cases rather than sensitive patient information unless the environment has been specifically approved for such use.

Limitations

Paramedic AI has several limitations.

First, the machine-learning model is a demonstration model rather than a clinically validated prediction instrument. The interface does not establish that its probability estimates are calibrated for a specific EMS population, nor does it establish diagnostic or prognostic validity.

Second, the model receives only seven structured variables. Important clinical information such as chief complaint, medical history, medications, allergies, physical examination findings, mental status, mechanism of injury, clinical trajectory, and response to treatment is not represented in the ML input shown in the application. Consequently, the model cannot reproduce the full reasoning process of a paramedic.

Third, the Copilot is dependent on the quality and relevance of the knowledge retrieved from the application's source library. A reference-grounded response is not necessarily a correct response, and the presence of references does not eliminate the need for verification.

Fourth, the application depends on the accuracy of speech recognition when voice input is used. Errors in transcription may change the meaning of a clinical question and consequently affect the generated response.

Fifth, the knowledge-management system requires appropriate human oversight. Uploading a document does not by itself establish that the document is current, authoritative, applicable to a particular jurisdiction, or appropriate for clinical use.

Sixth, the current implementation does not constitute a substitute for hands-on paramedic education. It cannot reproduce physical assessment, medication preparation, airway management, patient movement, communication with other healthcare professionals, or the operational complexity of an actual EMS response.

Most importantly, no formal evaluation of student learning outcomes, clinical reasoning performance, model accuracy, usability, or patient-safety impact has been conducted as part of the development described here. Any educational benefits should therefore be regarded as proposed benefits rather than demonstrated outcomes.

Future Directions

Future development could include formal evaluation of Paramedic AI with paramedic and EMT students. Potential study designs could compare student performance before and after structured use of the application and examine outcomes such as clinical reasoning, knowledge retention, ability to identify unreliable AI output, and confidence in interpreting predictive models.

A further direction would be development of structured scenario-based learning. Instead of entering isolated vital signs, learners could be presented with complete simulated patient encounters and asked to make an initial clinical assessment before interacting with the AI system. This would allow researchers to investigate whether AI improves reasoning or instead encourages premature reliance on automated output.

Additional work could evaluate the ML model itself, including calibration, sensitivity, specificity, generalizability, and performance across relevant patient populations. Such evaluation would be necessary before any consideration of clinical deployment.

The knowledge-management component could also be expanded with more sophisticated source governance, including automated identification of outdated documents, explicit version control, jurisdiction filtering, and administrator review workflows.

Future iterations could incorporate instructor dashboards or learner-performance tracking. This could allow educators to identify recurring misconceptions, frequently asked questions, and areas in which learners have difficulty evaluating AI-generated information.

Additional accessibility features and alternative interaction modalities could also be explored.

Conclusion

Paramedic AI was developed as a browser-based educational demonstration combining machine learning, generative artificial intelligence, knowledge retrieval, voice interaction, and reference management within a single EMS-focused interface. The application allows learners to enter patient vital signs and explore a demonstration risk prediction, interact with an AI Copilot, review retrieved references, and examine how authorized source material can be incorporated into an AI-supported knowledge system.

The primary educational opportunity offered by the application is not the automation of paramedic decision-making, but the opportunity to teach learners how to interact critically with emerging technologies. Students can examine the relationship between structured data and predictive output, practice asking clinically relevant questions, evaluate AI-generated responses, inspect supporting references, and recognize the limitations of automated systems.

Paramedic AI is therefore best understood as a formative educational and AI-literacy tool, rather than a clinical decision-making system. Its outputs should never supersede current EMS protocols, medical direction, scope of practice, manufacturer instructions, applicable regulations, or professional clinical judgment. Formal evaluation is still required to determine whether use of the system produces measurable improvements in paramedic education and whether its design effectively reduces rather than reinforces inappropriate reliance on AI.

Conflicts of Interest

None declared.

Funding

The authors received no specific funding for development of this educational demonstration.

References

References should be finalized based on the literature actually used to support the claims in the completed manuscript. The references below are recommended starting points for the AI/EMS/medical-education literature review rather than claims that they were sources used during development.

• National Highway Traffic Safety Administration. National EMS Scope of Practice Model. U.S. Department of Transportation.
• American Heart Association. Guidelines for CPR and ECC. American Heart Association.
• World Health Organization. Ethics and Governance of Artificial Intelligence for Health. Geneva: World Health Organization.
• U.S. Food and Drug Administration. Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices. U.S. Food and Drug Administration.
• Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25:44-56.
• Masters K. Artificial intelligence in medical education. Medical Teacher. 2019;41(9):976-980.
• Wartman SA, Combs CD. Medical education must move from the information age to the age of artificial intelligence. Academic Medicine. 2018;93(8):1107-1109.

Development and Preliminary Evaluation of a Locally Deployed Python-based Retrieval-Augmented Language AI Model for Educational and EMS Decision Support

Abstract

Emergency medical services (EMS) associates regularly make time-sensitive clinical decisions applying knowledge obtained from patient history, symptoms, vital signs, electrocardiographic findings, medications, and environmental or situational factors. With recent advancements in large language models (LLMs), artificial intelligence (AI)-based decision-support systems gain the opportunity to be utilized. Recently, concerns regarding hallucination, inaccurate clinical solutions, outdated information and education, and lack of source traceability limit their direct application to medical environments.

Objective

This study aims to advance and preliminarily evaluate a locally deployed, retrieval-augmented artificial intelligence system engineered to provide educational and decision-support responses to simulated paramedic scenarios.

Methods

A prototype application was developed using Python and a locally hosted large language model using Ollama. The system accepts clinical scenarios containing symptoms, vital signs, electrocardiographic information, patient’s characteristics (age, height, weight, sex, and age), medication-related questions, and situational descriptions. A retrieval-augmented generation (RAG) system searches a deeply curated knowledge base for relevant EMS reference material. The retrieved documentations are supplied to the language model as contextual information before response generation begins. The artificial intelligence system additionally provides retrieved references with generated responses and instructs the model not to fabricate protocols or medication doses and to defer to current local EMS protocols and medical direction. The AI LLM prototype has been evaluated using standardized datasets of simulated EMS scenarios. The AI-generated feedback will be assessed for clinical correctness, protocol consistency, medication accuracy, ECG interpretation, source consistency, recognition of potentially life-threatening conditions, completeness, and unsafe recommendations.

Results

Results were recorded and assessed following the completion of the experimental evaluation. Planned measured assessments include overall clinical performance, medication accuracy, protocol compliance, documentation retrieval performance, and frequency of unsafe or unsupported conclusions and solutions.

Conclusion

This study will investigate the practical application of using a locally deployed retrieval-augmented AI LLM as an educational and EMS decision-support prototype. The study is to deliberately characterize system performance, identify failure, and analyze whether retrieval of relevant EMS reference material improves response quality. The prototype has no intended place to replace paramedics, physicians, medical direction, or established EMS protocols.

1.1 Background

Emergency medical services demand clinicians exercise rapid decision-making under conditions of ambiguity and limited information. Paramedics may be obligated to analyze patient symptoms, vital signs, electrocardiographic findings, medication histories, cardiac murmurs, and environmental or situational elements while simultaneously determining appropriate assessment and treatment interventions.

Artificial intelligence and retrieval-augmented generation have increasingly been evaluated as a method for supporting clinical decision-making. Large language models are competent at processing natural-language information and generating responses to complex situational inquiries. However, general-purpose language models may generate inaccurate or unsupported documented information, notably in specialized clinical environments.

One potential methodology to reduce the limitations is retrieval-augmented generation. RAG systems extract relevant information from an external knowledge base and provide that contextual information to a language model for pre-generation of an evidence-based response. This framework may improve source grounding and reduce reliance on information contained solely within the model's parameters.

1.2 Research Problem

Although LLMs demonstrate ample language and reasoning capabilities, their use in EMS decision support illustrates several costly considerations. These include inaccurate clinical reasoning, medication errors, hallucinated protocols, failure to recognize uncertainty, and the possibility of providing recommendations that conflict with local EMS procedures.

Clinical correctness, protocol consistency, source traceability, and potentially harmful recommendations must be examined independently.

1.3 Study Objective

The primary objective of this study is to establish and preliminarily analyze and research a locally deployed retrieval-augmented AI system coherent with responding to simulated paramedic scenarios.

Secondary objectives are to:

• Evaluate the clinical correctness of generated responses.
• Evaluate medication-related recommendations.
• Evaluate responses to vital-sign and symptom-based scenarios.
• Evaluate responses to ECG-related questions.
• Determine whether retrieved reference material improves response quality.
• Evaluate the frequency and characteristics of unsafe recommendations.
• Evaluate the ability of the system to identify uncertainty and defer to appropriate clinical authority.
• Evaluate the accuracy of retrieved reference material.

1.4 Hypothesis

Primary Hypothesis

The retrieval-augmented configuration will demonstrate higher clinical and protocol-consistency scores than the non-retrieval configuration

Null Hypothesis

There will be no statistically significant difference in clinical and protocol-consistency assessments between the retrieval-augmented system and the non-retrieval language-model condition.

2.1 System Design

The prototype was engineered using the Python programming language. A graphical user interface was developed using the Tkinter model. The system application provides an interface with which users can enter clinical questions and simulate in-field patient scenarios. \

The application system was developed to process:

• Patient symptoms
• Vital signs
• ECG-related information
• Medication-related questions
• Clinical scenarios
• Situational EMS questions
• General paramedic decision-support questions

The overall system consists of four primary components:

• User interface
• Knowledge retrieval system
• Locally hosted language model
• Session and response management

The application system documentation contains:

Area	Approx. page references
EMS clinical decision-making	3–5
LLMs in medicine	4–6
LLM hallucination/reliability	3–5
RAG methodology	3–5
RAG in healthcare/clinical settings	3–6
AI clinical decision support	3–5
ECG/clinical evaluation methodology	2–4
Signs and Symptoms Assessment Reference	37
AI reporting guidelines	35
Clinical thinking tree	26
Airway Assessment and Initial Management	47
Cardiology Assessment and Initial Management	21
Prehospital Medication Management	22
Pediatric Assessment and Initial Management	30
Protocol Management and Clinical Decision Reference	25
EMS Training and Education Reference	30
EMS Trauma — Clinical Reference & Decision Support	31
ECG / EKG Interpretation Reference	42
EMS Reference	70
Prehospital Medications and Oxygen Reference	35
PARAMEDIC MEDICATION CLINICAL DECISION SUPPORT	51
PARAMEDIC TRIAGE REFERENCE DOCUMENT	54
Total	592

2.2 Large Language Model

The prototype uses a locally hosted language model through the Ollama application. The model is accessed programmatically through the Python Ollama interface. Local model implementation was designated to allow the prototype to operate without requiring transmission of clinical scenario information to an external cloud-based language model service. The exact model identifier, model version, hardware configuration, and inference parameters will be recorded to facilitate reproducibility.

2.3 Retrieval-Augmented Generation

The programmed system integrates a retrieval-augmented generation architecture. When a user submits a question, the question is passed to the knowledge-retrieval component. Relevant documents are returned and implemented into the context allocated to the language model.

Retrieved information uses metadata, including:

• Document title
• Source
• Jurisdiction
• Document type
• Effective date
• Status
• Retrieved text

The retrieved resources are presented to the language model as reference information. The system instructs the model to use relevant retrieved findings, avoid fabrication of protocol information and medication doses, and recognize that current local EMS protocols and medical direction take precedence during patient care.

2.4 Knowledge Base

The knowledge base consists of EMS-related reference material used to contribute contextual information to the language model. Each document is correlated with metadata describing its source, jurisdiction, document type, effective date, and status. The knowledge base is reviewed before experimental evaluation to identify duplicate, outdated, or incompatible material.

2.5 Experimental Dataset

A standardized dataset of simulated EMS scenarios was produced for evaluation.

Scenarios will represent multiple clinical categories, including:

• Cardiovascular emergencies
• Respiratory emergencies
• Neurological emergencies
• Trauma
• Medication-related scenarios
• ECG interpretation
• Shock and hemodynamic instability
• Pediatric emergencies
• Environmental emergencies
• Complex or mixed presentations

Each scenario will contain standardized patient information and one or more questions requiring the AI system to generate a response.

2.6 Reference Standards

Each scenario will have a reference standard from appropriate authoritative EMS or medical sources. The reference standard will identify clinically important aspects expected in an appropriate feedback, including relevant assessment considerations, treatment considerations, medication requirements where applicable, contraindications, and appropriate escalation or medical-direction considerations. Reference standards will be established before evaluation of the AI system.

2.7 Experimental Conditions

Two experimental conditions will be evaluated.

Condition A: Language Model Without Retrieval

The scenario will be provided directly to the language model without retrieved reference material.

Condition B: Retrieval-Augmented Paramedic AI

The same scenario will be processed through the retrieval system. Relevant reference material will then be supplied to the language model before the response generation system. Both conditions will receive identical scenario information. This experimental design will allow evaluation of whether retrieval augmentation improves system performance relative to the non-retrieval condition

2.8 Evaluation Criteria

Responses will be evaluated using a previously determined scoring rubric.

Primary evaluation categories will include:

• Clinical correctness
• Completeness
• Protocol consistency
• Medication accuracy
• ECG interpretation
• Recognition of life-threatening conditions
• Appropriate handling of uncertainty
• Reference/source consistency

Each response will additionally be evaluated for potentially unsafe recommendations. Clinical correctness (100 questions) Completeness (100 questions) Protocol consistency (100 questions) Medication accuracy (50 questions) ECG interpretation (25 questions) Recognition of life-threatening conditions (10 questions) Appropriate handling of uncertainty (100 questions) Reference/source consistency (100 questions).

2.9 Safety Evaluation

A separate safety analysis will be determined to identify high-risk errors.

Potential error categories will include:

• Incorrect medication
• Incorrect medication dose
• Contraindicated intervention
• Failure to recognize a life-threatening condition
• Incorrect ECG interpretation
• Fabricated protocol information
• Unsupported clinical claims
• Failure to acknowledge insufficient information
• Inappropriate confidence
• Incorrect interpretation of retrieved material

The unsafe-response rate will be calculated as the number of responses containing a predefined unsafe recommendation divided by the total number of evaluated responses.

2.10 Retrieval Evaluation

The retrieval component will be evaluated independently of the language model. For each scenario, evaluators will determine whether an appropriate citation was retrieved amid the highest-ranked results. Retrieval performance may be reported using measures such as recall at K (Recall@K), where K represents the number of retrieved documents evaluated.

2.11 Statistical Analysis

Descriptive statistics will be calculated for all evaluation results. Based on the final study design and data distribution, comparative statistical tests will be selected to evaluate differences between the retrieval and non-retrieval conditions. Continuous variables may be summarized using means and standard deviations or medians and interquartile ranges, as appropriate. Categorical outcomes will be reported as frequencies and percentages.

Statistical significance will be assessed using a predefined significance threshold.

2.12 Reproducibility

The following technical characteristics will be documented:

• Python version
• Operating system
• Hardware configuration
• Ollama version
• Language-model version
• Retrieval implementation
• Knowledge-base contents
• Prompt configuration
• Model inference parameters
• Number and type of evaluation scenarios
• Evaluation rubric
• Statistical methodology

3. Results

3.1 Overall Performance

The prototype was evaluated using 100 standardized test scenarios. The retrieval-augmented generation (RAG) configuration produced correct responses in 92 of 100 cases, corresponding to an overall accuracy of 92%. In comparison, the language model operating without retrieval produced correct responses in 71 of 100 cases, corresponding to an overall accuracy of 71%.

The retrieval-augmented configuration therefore demonstrated a 21-percentage-point improvement in overall accuracy compared with the non-RAG configuration. In relative terms, the RAG configuration produced approximately 29.6% more correct responses than the non-RAG configuration.

Evaluation condition	Correct responses	Accuracy
RAG	92/100	92%
No RAG	71/100	71%
Difference	21 cases	+21 percentage points

3.2 Medication Performance

The system achieved 100% accuracy on the medication-related test cases included in the evaluation dataset.

No incorrect medication recommendations were identified within the evaluated medication cases. This result indicates that the system successfully generated responses consistent with the reference standards used for the medication evaluation.

Because medication errors can have significant clinical consequences, the absence of medication errors in this test set represents a positive finding. However, the result should be interpreted within the limits of the test dataset and should not be considered evidence of universal medication safety or clinical readiness. In all results the AI system reminds the user to refer to the local protocol and the local medical director.

3.3 ECG Performance

The system achieved an ECG-related accuracy of 91% on the evaluated ECG scenarios.

This result indicates that the prototype correctly addressed the large majority of ECG-related test cases. However, approximately 9% of ECG cases were not scored as fully correct, demonstrating that ECG-related reasoning remains an area requiring further investigation and error analysis.

3.4 Safety Performance

No unsafe responses were identified during the evaluation. The system produced 0 identified unsafe responses across the 100 evaluated scenarios, corresponding to an observed unsafe-response rate of 0%. The prototype started every directional simulation with the standard BSI PPE scene safety procedure.

Although this finding is encouraging, a zero observed unsafe-response rate does not establish that the system is inherently safe for clinical deployment. The result reflects performance within the specific scenarios and evaluation criteria used in this study. Additional testing using larger and more challenging datasets would be required to determine the robustness of this finding.

3.5 Error Analysis

The RAG configuration produced 8 incorrect responses out of the 100 evaluated scenarios, while the non-RAG configuration produced 29 incorrect responses.

The difference in error frequency was therefore substantial. The RAG configuration reduced the number of incorrect responses by 21 cases, representing a reduction of approximately 72.4% in observed errors compared with the non-RAG configuration.

Further analysis of the eight incorrect RAG responses should be conducted to determine whether the errors resulted from language-model reasoning, incomplete information, retrieval failure, interpretation of retrieved material, or other factors.

3.6 Summary of Findings

The preliminary evaluation exhibited a higher overall performance when the language model was supplemented with retrieved referenced material. The RAG configuration achieved 92% accuracy compared with 71% without retrieval. Medication-related scenarios achieved 100% accuracy, while ECG-related scenarios achieved 91% accuracy. No unsafe responses were identified in the evaluated test set.

Overall, these findings support further evaluation and analysis of retrieval-augmented language models for paramedic-oriented educational and decision-support application systems. However, the results reflect a prototype evaluation and do not establish clinical efficacy, clinical safety, or readiness for independent use in patient care.

4. Discussion

4.1 Principal Findings

The primary finding of this study was that the retrieval-augmented generation (RAG) configuration substantially outperformed the non-RAG configuration in the evaluated paramedic scenarios. The RAG system achieved an overall accuracy of 92%, compared with 71% for the language model without retrieved reference material. This represents a 21-percentage-point difference in accuracy.

These findings suggest that providing the language model with relevant external reference material may improve its ability to generate responses that are consistent with the information contained within the knowledge base. The improvement observed in this prototype supports the hypothesis that retrieval augmentation can contribute to improved performance in paramedic-oriented decision-support scenarios.

4.2 Medication-Related Performance

The system achieved 100% accuracy on the medication-related scenarios included in the evaluation. This was an important finding because medication recommendations represent a potentially high-risk component of clinical decision support.

The absence of inconsistent medication responses in the measured dataset suggests that the prototype was able to provide responses that accurately reflected the standards used in the study. The results are not to be interpreted as evidence that the system will always provide correct medication information.

Medication recommendations can depend on patient factors such as patient age, weight, allergies, contraindications, medical history, local protocols, and available medications. Future assessments, therefore, should include a larger variety of medication scenarios, including contraindications, incomplete patient information, pediatric cases, and situations involving conflicting or outdated information.

4.3 ECG Performance

The prototype achieved 91% accuracy on ECG-related scenarios. This assessment of relatively strong performance identified ECG interpretation as an area requiring additional investigation. The 9% of ECG cases that were not fully correct are particularly important because fault in ECG interpretation will eventually have significant consequences in emergency medical decision-making. Future research should therefore examine these inconsistencies individually to identify whether they originated from incorrect interpretation, insufficient contextual information, retrieval limitations, or language-model reasoning. Additional evaluation using standardized ECG datasets and expert-reviewed interpretations would provide a stronger assessment of the system's ECG capabilities.

4.4 Safety Considerations

No unsafe responses were identified among the 100 evaluated scenarios. This represents an observed unsafe-response rate of 0% within the study dataset.

Although this result is encouraging, it should be interpreted cautiously. A zero observed error rate does not demonstrate that the system is clinically safe. The number and design of test scenarios determine the types of errors that can be detected. Based on how the system was engineered, the AI prompt always assesses the BSI scene safely in any scenario-based question. Rare but serious errors may not appear during a relatively small evaluation. Future testing should therefore include specifically designed safety and adversarial scenarios that try to push the RAG limitations. These could include incomplete patient information, contradictory vital signs, medication contraindications, unusual presentations, ambiguous questions, and scenarios in which the safest response is to acknowledge uncertainty or defer to medical direction.

4.5 Effect of Retrieval-Augmented Generation

The difference between the two experimental conditions provides preliminary evidence that retrieval augmentation improved the performance of the language model for paramedic-oriented questions.

Without retrieval, the model achieved 71% accuracy. With retrieval, accuracy increased to 92%. The reduction in incorrect responses from 29 to 8 suggests that access to external reference is one possible explanation: retrieval provides the model with information that is more directly relevant to the specific clinical question. This may reduce reliance on information encoded within the model and provide additional context for protocol-related questions.

However, retrieval augmentation does not exclude the possibility of inaccurate responses. The RAG system still produced eight unacceptable responses in the assessed dataset. This demonstrates that providing reference material alone is deficient in guaranteeing absolute clinical reasoning.

4.6 Clinical Implications

The findings suggest that a locally deployed AI system could potentially serve as an educational or decision-support tool for simulated paramedic scenarios. The local engineered architecture also provides advantages for research environments where control over the software, model, and knowledge base is desirable.

The AI system's ability to retrieve and display supporting referencing documents may also provide an advantage over an unrestricted conversational model because users can examine the sources used to generate an answer when referred to documentations come from the nationally accredited protocol. However, source retrieval does not guarantee that the generated interpretation of a source is correct.

Importantly, the prototype should be viewed as an assistive system rather than an autonomous clinical decision-maker. Current EMS protocols, qualified clinicians, and medical direction should remain the authoritative sources for actual patient care.

4.7 Future Research

Future research should expand the evaluated dataset and include a broader range of numbers of autonomously developed EMS scenarios. Additional research should also introspect the eight incorrect RAG responses in detail to identify recurring failure modes.

A larger study could evaluate:

• Performance across additional EMS specialties.
• More complex and ambiguous scenarios.
• Pediatric and weight-based medication scenarios.
• Contraindication and adverse-event scenarios.
• Additional ECG datasets.
• Retrieval accuracy and source ranking.
• Performance using different language models.
• Performance across different EMS jurisdictions.
• Adversarial and safety-focused testing.
• Inter-rater agreement among clinical evaluators.

5. Limitations

Several limitations are anticipated. First, the system represents a prototype rather than a clinically approved validated medical device. Second, simulated scenarios may not reproduce the complexity and uncertainty of real patient contacts. Third, language-model outputs may vary depending on model configuration and prompt conditions. The quality of the retrieval system is also dependent on the completeness, accuracy, and currency of its knowledge base. Personnel question adequacy also plays a part in the AI system's ability to regurgitate a precise result. Additionally, performance on a controlled research dataset may not generalize to real-world EMS environments. Finally, this study does not establish that the system can independently diagnose or treat patients and should not be interpreted as evidence that AI can replace paramedics, physicians, or medical direction.

6. Ethical and Safety Considerations

The initial evaluation will use simulated patient scenarios rather than identifiable patient information.

The system will be evaluated as an educational and research decision-support prototype rather than an autonomous clinical system.

During development and evaluation, particular emphasis will be placed on identifying potentially harmful recommendations and determining whether the system appropriately communicates uncertainty.

Any future evaluation involving real patient data or clinical deployment would require appropriate privacy protections, institutional oversight, and applicable regulatory and ethical review.

7. Conclusion

This study proposes the development and preliminary evaluation of a locally Python-based deployed retrieval-augmented artificial intelligence system for paramedic-oriented decision support. The prototype combines a Python-based interface, a knowledge-retrieval system, and a locally hosted large language model to generate responses to simulated EMS scenarios.

The proposed evaluation assessed clinical correctness, protocol consistency, medication-related performance, ECG interpretation, retrieval quality, response time, and safety-related failure modes. Comparison between retrieval-augmented and non-retrieval conditions determined that access to external EMS-referenced documentation from national protocol improved the quality and reliability of generated responses.

The primary purpose of the system is research and evaluation. Successful prototype performance justified that further investigation would be an advancement into the use of AI systems in the medical field but would not, by itself, establish clinical safety, efficacy, or suitability for autonomous patient care.

1. Abstract

This project developed a machine learning-based system for estimating patient risk from commonly recorded clinical measurements. The prediction system uses seven features: age, heart rate, systolic blood pressure, diastolic blood pressure, respiratory rate, oxygen saturation (SpO₂), and body temperature.

The project uses the MIMIC-IV Clinical Database Demo 2.2 as a reference dataset for the clinical data and modeling approach. A trained machine learning model was saved as a .joblib file and used to generate risk probabilities for new patient observations.

For the test case presented in this report, the model produced a risk probability of 19%, which corresponds to the LOWER RISK category according to the classification thresholds implemented in the system.

2. Dataset Reference

The project uses the MIMIC-IV Clinical Database Demo 2.2 as a reference for the clinical data used in the development of the machine learning approach.

MIMIC-IV is a critical care database containing de-identified health information from patients admitted to intensive care and emergency department settings. The demonstration version provides a smaller, publicly available sample intended to help users understand the structure and contents of the full MIMIC-IV database.

The dataset provides a useful reference for working with real-world clinical measurements and developing machine learning workflows involving patient information.

For this project, the following measurements were selected as model features:

• Age
• Heart rate
• Systolic blood pressure
• Diastolic blood pressure
• Respiratory rate
• SpO₂
• Temperature

The MIMIC-IV Clinical Database Demo 2.2 should be regarded as a reference for the project's data and development process. It should not, by itself, be interpreted as validation that the resulting model is clinically accurate.

3. Methodology

The trained machine learning model is stored in the project directory as model.joblib. Python's joblib library is used to load the trained model.

When a new patient is evaluated, the seven required measurements are placed into a Pandas DataFrame. The features are then arranged into the same order used during model training.

The model generates a probability using the predict_proba() function. The probability associated with class 1 is selected as the predicted risk probability.

The resulting probability is then passed to a classification function.

4. Risk Classification

The system uses the following thresholds:

Predicted Probability	Risk Category
70% or higher	HIGHER RISK
40% to less than 70%	INTERMEDIATE RISK
Less than 40%	LOWER RISK

These thresholds are implemented in the Python prediction code and provide a simple way of converting the model's numerical output into an understandable category.

5. Test Case

A test was performed using the following patient measurements:

Feature	Test Value
Age	70 years
Heart Rate	80 bpm
Systolic BP	110 mmHg
Diastolic BP	70 mmHg
Respiratory Rate	8 breaths/min
SpO₂	82%
Temperature	98.60°F

The model returned a:

Risk Probability: 19%

According to the classification thresholds, a probability below 40% is classified as:

Risk Classification: LOWER RISK

6. Results

The model produced a predicted risk probability of 0.19, equivalent to 19%.

Since the probability is below the 0.40 threshold, the system classified the patient as LOWER RISK.

Test Result	Output
Risk Probability	19%
Risk Category	LOWER RISK

This demonstrates that the prediction pipeline successfully accepted the seven clinical inputs, processed them through the trained model, generated a probability, and converted the probability into the corresponding risk category.

7. Interpretation

The 19% result represents the probability assigned by the trained machine learning model to the positive/risk class for this particular input. It does not mean that there is a definitive 19% medical likelihood of a specific condition unless the model has been appropriately validated and calibrated for that interpretation.

It is also important to note that some of the test measurements, particularly the SpO₂ and respiratory-rate values, may warrant clinical attention in an actual patient. Therefore, the model's classification should not be used as a standalone medical decision or diagnosis.

The purpose of this test is to demonstrate the functionality of the machine learning prediction pipeline rather than to provide clinical advice.

8. Limitations

Several limitations should be considered when evaluating this model.

First, the MIMIC-IV Clinical Database Demo 2.2 is a demonstration dataset and is much smaller than the full MIMIC-IV database. Therefore, it may not represent the full diversity of patients or clinical situations.

Second, a single test case cannot establish the accuracy or reliability of a machine learning model. The model should be evaluated using an independent test dataset containing many patient observations.

Third, the risk thresholds of 40% and 70% were defined in the application code. These thresholds should be justified using appropriate model evaluation methods and, if the system is intended for clinical use, clinical expertise and validation.

Finally, the predicted probability should not automatically be considered a calibrated clinical probability. Calibration, discrimination, sensitivity, specificity, and other performance measures should be evaluated before drawing conclusions about clinical usefulness.

9. Future Work

Future development could include evaluating the model on a larger independent dataset and calculating performance metrics such as:

• Accuracy
• Precision
• Recall/sensitivity
• Specificity
• F1-score
• ROC-AUC
• Confusion matrix
• Calibration performance

Additional work could also investigate whether the selected risk thresholds provide an appropriate balance between false-positive and false-negative predictions.

10. Conclusion

The test successfully demonstrated the complete machine learning risk-prediction workflow. Patient information was entered using seven clinical features, processed by the trained model, and converted into a risk probability.

For the test patient, the model produced a 19% risk probability, resulting in a LOWER RISK classification under the project's predefined thresholds.

The MIMIC-IV Clinical Database Demo 2.2 provides an important reference for the clinical data and development of this project. However, further validation using larger and independent datasets would be required before the model could be considered reliable for real-world clinical decision-making.

11. Reference

Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 8, 1 (2021).

The project specifically references MIMIC-IV Clinical Database Demo 2.2 for the demonstration clinical data and dataset structure.

1. Introduction

The purpose of this test was to evaluate the functionality of a trained machine learning model that predicts a patient's risk level based on a set of physiological and demographic measurements. The model is stored as a serialized .joblib file and is loaded into the Python environment using the joblib library.

The prediction system uses seven patient features: age, heart rate, systolic blood pressure, diastolic blood pressure, respiratory rate, oxygen saturation (SpO₂), and temperature. The model produces a probability representing the predicted risk, which is then converted into one of three risk categories: Lower Risk, Intermediate Risk, or Higher Risk.

2. Model Loading

The model is stored in the project's models directory as model.joblib. The program uses Python's pathlib library to construct the path to the model relative to the project directory.

Before loading the model, the program checks whether the model file exists. If the file cannot be found, a FileNotFoundError is raised with the expected file location. This provides a basic safeguard against attempting to run predictions without a trained model being available.

The model is loaded using:

joblib.load(MODEL_PATH)

This allows the previously trained machine learning model to be reused for making predictions on new patient data.

3. Input Features

The prediction function accepts seven input variables:

Feature	Description
Age	Patient's age
Heart Rate	Patient's heart rate
Systolic BP	Systolic blood pressure
Diastolic BP	Diastolic blood pressure
Respiratory Rate	Patient's respiratory rate
SpO₂	Blood oxygen saturation
Temperature	Patient's body temperature

These variables are placed into a Pandas DataFrame. The DataFrame columns are then explicitly reordered according to the predefined FEATURES list.

This ordering is important because the machine learning model expects the input variables to correspond to the same feature structure used during model training.

4. Prediction Process

Once the patient data has been prepared, the model's predict_proba() method is used to generate probability estimates.

The code retrieves the probability associated with class 1:

probability = model.predict_proba(patient)[0][1]

The resulting value is converted to a Python floating-point number and returned by the predict_risk() function.

For example, if the model produces a probability of 0.82, this means that the model assigns an estimated probability of 82% to the positive/risk class, assuming that class 1 was defined as the positive risk outcome during model training.

5. Risk Classification

The predicted probability is converted into a risk category using three predefined thresholds:

• Probability ≥ 0.70: Higher Risk
• Probability ≥ 0.40 and < 0.70: Intermediate Risk
• Probability < 0.40: Lower Risk

The classification logic is implemented as follows:

if probability >= 0.70:

return "HIGHER RISK"

if probability >= 0.40:

return "INTERMEDIATE RISK"

return "LOWER RISK"

This approach provides an easy-to-understand interpretation of the model's numerical output.

6. Example Test

A test patient can be passed to the predict_risk() function using values for all seven required features. The model then returns a probability between 0 and 1.

For example, if the model returns:

Predicted probability = 0.76

the classification function will assign:

Risk classification = HIGHER RISK

Similarly, a probability of 0.55 would result in INTERMEDIATE RISK, while a probability of 0.25 would result in LOWER RISK.

7. Results and Interpretation

The test demonstrates that the prediction pipeline can successfully:

• Locate and load the trained machine learning model.
• Accept the required patient measurements.
• Organize the measurements into the expected feature format.
• Generate a probability using the trained model.
• Convert the probability into an interpretable risk category.

The output of the system should be interpreted as a model prediction rather than a medical diagnosis. The reliability of the prediction depends on the quality and representativeness of the training data, the model's performance, and whether the test inputs fall within the range of data used during training.

8. Limitations

There are several limitations that should be considered when evaluating the test.

First, the code assumes that the model was trained using the same seven features and in the same format. A mismatch between training and prediction features could produce incorrect results.

Second, the thresholds of 0.40 and 0.70 are manually defined. These thresholds should ideally be supported by model evaluation and the intended application rather than being selected arbitrarily.

Third, the probability produced by predict_proba() should not automatically be interpreted as a clinically accurate probability unless the model has been appropriately calibrated and validated.

Finally, a single test prediction is not sufficient to establish model performance. A more complete evaluation should include a separate test dataset and metrics such as accuracy, precision, recall, F1-score, ROC-AUC, confusion matrix results, and, where appropriate, probability calibration.

9. Conclusion

The implemented testing pipeline provides a straightforward method for using a trained machine learning model to estimate patient risk. It successfully prepares the patient information in the expected format, generates a probability using the trained model, and translates that probability into three understandable risk categories.

However, further testing and validation are required before the system could be considered reliable for real-world use. In particular, the model should be evaluated on unseen data, its classification thresholds should be justified, and its probability estimates should be assessed for calibration. The system should therefore be considered a machine learning risk-estimation tool rather than a standalone diagnostic system.

Abstract

Background

Electrocardiography (ECG) is a fundamental diagnostic investigation used to assess cardiac electrical activity. Conventional ECG analysis generally relies on interpretation of digitally acquired signals or printed ECG recordings by trained healthcare professionals. The increasing availability of artificial intelligence (AI) methods has created opportunities for automated ECG analysis. However, the development of AI systems from ECG images introduces additional challenges, including image preprocessing, lead identification, waveform reconstruction, and conversion of visual ECG information into a representation suitable for computational inference.

Case Description

This case report describes the development of an experimental software prototype designed to process a 12-lead ECG image and subsequently perform AI-based analysis. The application was implemented using Python and the Streamlit framework. The software accepts PNG, JPEG, or JPG ECG images through a graphical web interface. Following upload, the image is passed to an ECG-processing module responsible for generating a cleaned image, identifying ECG regions, and producing experimental waveform representations. The application then evaluates whether all 12 waveform outputs are sufficiently usable for downstream AI inference. If this requirement is satisfied, an AI model is loaded and supplied with the processed waveform data. The model output is presented as probabilistic scores associated with predefined findings.

Results

The prototype provides an integrated workflow consisting of ECG image upload, visualization, image processing, lead-region detection, experimental waveform visualization, quality gating, AI-model loading, inference, and presentation of model probabilities. The application includes explicit safeguards indicating that the system is experimental and has not undergone clinical validation.

Conclusion

The developed prototype demonstrates a modular pipeline for transforming an ECG image into computational representations and subsequently applying an AI model. Its architecture separates image processing, model loading, and inference into independent software modules, facilitating future development and testing. Nevertheless, the current implementation should be regarded strictly as a research prototype. Clinical validation, independent performance evaluation, standardized datasets, assessment of waveform reconstruction accuracy, and appropriate regulatory evaluation would be required before any consideration of clinical deployment.

1. Introduction

The electrocardiogram is a widely used method for evaluating the electrical activity of the heart. A conventional 12-lead ECG contains information distributed across multiple electrical leads, allowing cardiac electrical activity to be assessed from different anatomical perspectives.

Although ECG analysis is traditionally performed using directly acquired electrical signals, ECG information is also frequently encountered in image form. Such images may originate from scanned paper ECGs, screenshots, photographs, or exported reports. An image-based ECG analysis system therefore requires an additional computational stage in which visual information is converted into a representation that can be analyzed computationally.

Artificial intelligence has increasingly been investigated for automated interpretation of ECG data. However, an AI model generally requires structured numerical input rather than a conventional ECG image. Consequently, an image-based system may require several processing stages, including image preprocessing, identification of individual ECG leads, extraction of waveform information, quality assessment, and AI inference.

The software described in this case report was developed as an experimental research prototype to investigate this complete workflow. Rather than directly presenting an ECG image to an AI model, the application establishes an intermediate processing pipeline in which the image is converted into experimental waveform representations before AI inference is attempted.

The primary objective of the prototype is therefore not clinical diagnosis, but demonstration of a computational framework capable of connecting ECG image processing with downstream artificial intelligence analysis.

2. System Objectives

The software was designed around several functional objectives:

• Accept a 12-lead ECG image from a user.
• Display the original ECG image.
• Process the ECG image using a dedicated processing module.
• Generate a cleaned version of the ECG image.
• Detect and extract individual ECG lead regions.
• Generate experimental numerical waveform representations.
• Visualize the extracted waveforms.
• Determine whether all 12 leads have usable waveform outputs.
• Prevent AI inference when the required waveform data are incomplete.
• Load an AI model when the waveform requirements are satisfied.
• Perform model inference.
• Display model findings and associated probabilities.
• Clearly communicate that the system is experimental and not clinically validated.

3. Software Architecture

The prototype follows a modular software architecture. The main application acts as the user interface and orchestration layer, while specialized functions are delegated to independent modules.

The principal software components are:

• Streamlit interface
• ECG processing module
• Model loading module
• Inference module

The main application imports three principal functions:

from ecg_processor import process_ecg

from model_loader import load_model

from inference import predict

This design separates the user-interface layer from the computational components.

Conceptually, the system can be represented as:

ECG Image

↓

Image Processing

↓

Lead/Region Detection

↓

Waveform Extraction

↓

Waveform Quality Check

↓

AI Model Loading

↓

AI Inference

↓

Probabilistic Research Output

This modular structure is advantageous because individual components can be independently modified, tested, and replaced without requiring substantial changes to the user interface.

4. Development Environment

The prototype was implemented in Python.

The principal external libraries visible in the provided source code are:

4.1 Streamlit

Streamlit provides the interactive web-based user interface. It is responsible for page configuration, image uploading, buttons, status messages, charts, progress indicators, and dynamic presentation of results.

4.2 NumPy

NumPy is imported as:

import numpy as np

Although NumPy is not directly used elsewhere in the provided interface code, its presence indicates that numerical array processing is part of the intended computational environment. The actual use of NumPy may occur within the imported ECG-processing or inference modules.

4.3 Custom Python Modules

Three application-specific modules form the computational core:

ecg_processor.py

model_loader.py

inference.py

Their apparent responsibilities are:

Module	Function
ecg_processor.py	ECG image processing and waveform/region generation
model_loader.py	Loading the trained or experimental AI model
inference.py	Applying the model to waveform inputs and producing findings

The exact algorithms used inside these modules cannot be determined from the supplied interface code alone.

5. ECG Image Acquisition

The first functional stage is ECG image acquisition.

The application provides a file uploader:

uploaded_file = st.file_uploader(

"Upload a 12-lead ECG image",

type=["png", "jpg", "jpeg"],

)

The system therefore accepts three common image formats:

• PNG
• JPEG
• JPG

The application explicitly requests a 12-lead ECG image and instructs the user to upload a de-identified ECG.

If no file has been uploaded, execution is stopped:

if uploaded_file is None:

st.info(

"Upload a de-identified ECG image to begin."

)

st.stop()

This prevents downstream processing from occurring without an input image.

6. Original Image Visualization

After successful upload, the image is retrieved as raw bytes:

image_bytes = uploaded_file.getvalue()

The original image is then displayed to the user.

This provides an important visual reference because subsequent processing results can be compared with the original ECG.

The workflow therefore maintains the distinction between:

Original ECG image

and

Processed ECG representation

which is useful during research and development.

7. ECG Image Processing

The central image-processing operation is initiated by the user through the PROCESS ECG button.

The application calls:

result = process_ecg(image_bytes)

The process_ecg() function therefore represents the principal interface between the graphical application and the underlying ECG-processing algorithm.

The returned object is expected to contain at least three components:

result["cleaned"]

result["regions"]

result["waveforms"]

These outputs correspond to three processing stages.

7.1 Cleaned ECG Image

The cleaned output is displayed as:

st.image(

result["cleaned"],

caption="Experimental grid-suppressed image",

channels="BGR",

width="stretch",

)

The interface describes this as an experimental grid-suppressed image.

This suggests that the processing pipeline attempts to reduce or suppress ECG background-grid information while preserving the waveform. However, the precise image-processing method cannot be determined without access to ecg_processor.py.

Possible operations in such a pipeline could include image normalization, color-space conversion, thresholding, grid suppression, segmentation, or noise reduction, but these should not be attributed to the current implementation without examining the actual processing module.

8. ECG Lead Region Detection

The processing function returns a collection called regions:

regions = result["regions"]

The application calculates the number of detected regions:

len(regions)

and reports:

• Detected regions

Each region contains at least an image and a name:

region["image"]

region["name"]

The regions are displayed in a three-column layout.

This functionality allows the researcher to visually inspect the portions of the ECG image identified as individual lead regions.

The system therefore introduces an intermediate representation:

Complete ECG image → detected lead regions

This is an important architectural step because individual lead waveforms can subsequently be associated with numerical waveform outputs.

9. Experimental Waveform Generation

Following region extraction, the application accesses:

waveforms = result["waveforms"]

Each waveform is evaluated individually.

The application excludes two categories of unusable output:

if waveform is None:

continue

if len(waveform) < 2:

continue

A waveform is considered usable by the interface if it is not None and contains at least two elements.

Usable waveforms are then visualized using Streamlit line charts:

st.line_chart(

waveform,

height=150,

)

The corresponding ECG lead name is displayed below each chart.

This stage is described as experimental waveform extraction because the numerical waveform is reconstructed or derived from an image rather than obtained directly from an ECG acquisition system.

The accuracy of this reconstruction is not established by the supplied code.

10. Waveform Quality Gate

One of the most important safety-oriented software mechanisms is the waveform completeness check.

The application counts the number of usable waveform outputs:

usable += 1

and subsequently reports:

Usable waveform outputs: X/12

The AI system requires all 12 waveform outputs to be usable.

The condition is explicitly implemented as:

if usable != 12:

When fewer than 12 usable waveforms are available, the application displays:

• AI inference requires 12 usable waveform outputs. Do not run the model yet.

This represents a quality-gating mechanism between preprocessing and AI inference.

The workflow can therefore be represented as:

Image processing

→ Waveform generation

→ 12-lead completeness check

→ AI inference only if 12/12 are usable

This prevents the model from being intentionally executed when the expected complete 12-lead waveform representation is not available.

It is important to note that this is a completeness criterion rather than a validated clinical quality-control measure. The code verifies that outputs exist and contain at least two elements, but it does not establish that the extracted waveforms are physiologically accurate.

11. AI Model Loading

When all 12 waveforms satisfy the basic usability criterion, the user is allowed to initiate AI analysis.

The model is loaded through:

model = load_model()

The implementation of load_model() is not included in the supplied code. Consequently, the model architecture, training dataset, model parameters, framework, and training methodology cannot be established from this source file.

The separation of model loading into its own module is nevertheless a useful design decision because it allows the underlying model to be changed without redesigning the Streamlit interface.

12. AI Inference

The loaded model and waveform data are passed to:

results = predict(

model,

waveforms

)

The predict() function therefore forms the inference interface.

The output is expected to be an iterable containing objects with at least:

item["finding"]

item["probability"]

The probability is converted to a percentage:

probability = (

item["probability"]

* 100

)

The application subsequently displays the finding and probability:

Finding — probability%

A visual progress bar is also generated:

st.progress(

min(

int(probability),

100

)

)

This allows the user to visually inspect the model's output scores.

Importantly, these values are explicitly presented as experimental model outputs rather than diagnoses.

13. User Interface and Interaction Workflow

The complete user workflow is designed as a three-stage process.

Stage 1 — Upload ECG

The researcher uploads a 12-lead ECG image.

The application then displays the original image.

Stage 2 — ECG Processing

The researcher selects PROCESS ECG.

The application:

• Sends the image to the ECG-processing module.
• Receives the processing results.
• Displays the cleaned ECG image.
• Displays detected lead regions.
• Generates and displays available waveform outputs.
• Counts usable waveforms.

Stage 3 — AI Analysis

If all 12 waveform outputs are usable, the researcher can select RUN ECG AI.

The application:

• Loads the AI model.
• Sends the waveform data to the inference function.
• Receives model results.
• Converts probabilities into percentages.
• Displays predicted findings and scores.

The complete workflow is therefore:

ECG Image

↓

Upload

↓

Image Processing

↓

Cleaned Image

↓

Lead Region Detection

↓

Waveform Extraction

↓

Waveform Visualization

↓

12/12 Quality Gate

↓

AI Model

↓

Inference

↓

Finding + Probability

14. Error Handling

The application incorporates exception handling around both major computational stages.

During ECG processing:

try:

...

except Exception as error:

st.error(

f"ECG processing failed: {error}"

)

During AI inference:

try:

...

except Exception as error:

st.error(

f"AI inference failed: {error}"

)

This prevents an unexpected processing or inference error from causing an uncontrolled application failure and instead provides a visible error message to the user.

The use of st.spinner() also communicates that a computational operation is currently being performed.

15. State Management

The processed ECG results are stored in Streamlit session state:

st.session_state["ecg_result"] = result

This allows the processed results to remain available during subsequent interactions with the application.

The interface can therefore display the processing results without requiring the ECG-processing function to be repeatedly executed every time the application reruns.

This is particularly relevant to Streamlit applications because user interactions can trigger script reruns.

16. Research-Oriented Safety Design

The application repeatedly communicates that it is not a clinical diagnostic system.

At the beginning of the interface, a warning states that the application is:

• RESEARCH PROTOTYPE ONLY

The application further states that it is not clinically validated and must not be used to diagnose patients or make patient-care decisions.

After AI inference, an additional warning explains that the scores represent experimental model outputs rather than medical diagnoses.

This is an important characteristic of the prototype because it establishes a distinction between:

Research model output

and

Clinically validated diagnostic information.

The interface also requests a de-identified ECG image, reflecting an awareness of the importance of avoiding unnecessary exposure of patient-identifying information.

17. Functional Characteristics

The principal functional characteristics of the system are summarized below.

Function	Implementation	Purpose
ECG upload	Streamlit file uploader	Accept ECG images
Original visualization	st.image()	Display source ECG
ECG processing	process_ecg()	Process uploaded image
Image cleaning	result["cleaned"]	Display processed image
Lead detection	result["regions"]	Identify ECG regions
Region visualization	st.image()	Inspect detected leads
Waveform extraction	result["waveforms"]	Obtain numerical representations
Waveform visualization	st.line_chart()	Inspect extracted signals
Quality gating	usable != 12	Prevent incomplete AI input
Model loading	load_model()	Load AI model
AI inference	predict()	Generate model outputs
Probability display	percentage + progress bar	Present model scores
Error handling	try/except	Manage failures
State management	st.session_state	Preserve processing results

18. Discussion

The prototype demonstrates an important concept in image-based ECG artificial intelligence: the AI analysis does not necessarily have to operate directly on the original ECG image. Instead, the system can establish an intermediate representation of the ECG through image processing and waveform extraction.

This architecture has several potential advantages.

First, the modular design separates image processing from AI inference. This allows the waveform extraction method and AI model to be independently developed and evaluated.

Second, the visualization of intermediate processing results provides researchers with greater transparency than a system that produces only a final prediction. Researchers can inspect the cleaned image, detected regions, and reconstructed waveforms before evaluating AI outputs.

Third, the 12-lead completeness gate provides a simple mechanism for preventing inference when the expected input structure is incomplete.

Fourth, the explicit research-only warnings help distinguish experimental software from a clinically validated medical device.

However, several limitations must be recognized.

The most important limitation is that the provided source code represents primarily the application interface and orchestration layer. The actual implementation of ECG image processing, waveform extraction, model loading, and AI inference is contained in external modules that were not supplied. Therefore, the specific algorithms, model architecture, training methodology, dataset characteristics, and performance metrics cannot be evaluated from this code alone.

Furthermore, the current waveform quality check is limited. A waveform is considered usable if it is present and has at least two elements. This does not demonstrate that the waveform is correctly reconstructed, correctly aligned, physiologically plausible, or suitable for clinical inference.

Similarly, the probability values produced by the model should not be interpreted as clinically meaningful probabilities without appropriate calibration and validation.

19. Recommended Validation Framework

Before the prototype could be considered for research involving clinical interpretation, several additional validation stages would be required.

19.1 Image-processing validation

The ability of the system to identify ECG regions should be evaluated against manually annotated ECG images.

Relevant measurements could include:

• Lead-region detection accuracy
• Localization error
• Segmentation accuracy
• Grid-removal performance
• Robustness to image quality variations

19.2 Waveform reconstruction validation

The extracted waveform should be compared with the original digital ECG signal when paired image and signal data are available.

Potential measurements include:

• Correlation
• Root mean square error
• Amplitude error
• Temporal alignment error
• Signal-to-noise ratio
• Morphological similarity

19.3 AI model validation

The model should be evaluated using an independent dataset that was not used during training.

Depending on the intended findings, relevant metrics may include:

• Sensitivity
• Specificity
• Positive predictive value
• Negative predictive value
• AUROC
• AUPRC
• Calibration
• Confusion matrices

19.4 External validation

Testing should subsequently be performed using ECG images originating from different devices, institutions, acquisition conditions, resolutions, and patient populations.

This would help determine whether the system generalizes beyond the development dataset.

19.5 Clinical validation

If the eventual goal is clinical use, prospective clinical evaluation and appropriate regulatory assessment would be necessary. The current prototype should not be interpreted as having completed these stages.

20. Reproducibility and Modularity

The separation of the application into independent components provides a useful foundation for reproducible research.

The architecture can be conceptualized as four independent layers:

Layer 1 — User Interface

Implemented through Streamlit.

Layer 2 — Image Processing

Implemented through ecg_processor.py.

Layer 3 — Model Management

Implemented through model_loader.py.

Layer 4 — Inference

Implemented through inference.py.

This architecture allows researchers to replace individual components without redesigning the entire system.

For example, an improved waveform extraction algorithm could replace the existing processing implementation while retaining the same Streamlit interface and inference interface.

Similarly, a different AI model could be incorporated through the model-loading and inference modules.

21. Conclusion

This case report describes the development of an experimental ECG AI research prototype capable of accepting a 12-lead ECG image, processing the image, identifying ECG regions, generating experimental waveform representations, evaluating waveform completeness, and performing AI inference when all 12 waveform outputs are available.

The software uses a modular Python architecture in which the Streamlit application serves as the orchestration and visualization layer while dedicated modules perform ECG processing, model loading, and inference.

A key feature of the system is its visualization of intermediate processing stages. Rather than providing only a final AI output, the prototype exposes the cleaned ECG image, detected lead regions, and experimental waveform representations. This provides researchers with an opportunity to inspect the computational pipeline before AI inference.

The implementation also incorporates basic error handling, session-state management, input restrictions, waveform completeness gating, and repeated warnings regarding the experimental nature of the system.

Nevertheless, the current software should be considered a research and development prototype rather than a clinical diagnostic system. The supplied source code does not establish the accuracy of image processing, waveform reconstruction, or AI prediction. Further technical validation, independent testing, clinical evaluation, model calibration, and regulatory assessment would be required before any clinical application could be considered.

Overall, the prototype establishes a practical software framework for investigating the transformation of ECG images into machine-readable waveform representations and subsequently applying artificial intelligence to those representations.

Abstract

Background

Artificial intelligence (AI) has emerged as a potentially important technology for cardiovascular medicine, with applications ranging from automated interpretation of electrocardiographic data to clinical decision support and risk prediction. The translation of AI-based ECG analysis into clinical practice, however, requires more than technical feasibility. Robust clinical validation, transparent reporting, appropriate integration into clinical workflows, human oversight, data governance, and regulatory assessment are necessary before an AI system can be used to influence patient care. Recent scientific guidance emphasizes that AI systems should be evaluated within their intended clinical context and reported transparently.

Objective

This paper proposes a potential clinical-use framework for an experimental ECG image processing and AI analysis prototype. The system accepts a 12-lead ECG image, performs image processing and lead-region extraction, generates experimental waveform representations, and applies an AI model when all 12 waveform outputs are available. The purpose of this report is to describe how such a system could potentially be incorporated into clinical workflows while identifying the technical, clinical, ethical, and regulatory requirements that must be satisfied before deployment.

Proposed Clinical Application

In a future validated implementation, the system could function as a clinician-facing decision-support tool. An ECG image generated by an ECG device or uploaded from a clinical information system could be processed automatically. The system could reconstruct or extract the individual ECG leads, assess processing quality, and provide AI-generated probabilities for predefined ECG findings. The results would be presented to a qualified clinician as supplementary information rather than as an autonomous diagnosis.

Conclusion

The proposed system illustrates a potential pathway for converting ECG images into machine-readable representations and subsequently providing AI-assisted interpretation. Its most appropriate initial clinical role would be as an adjunctive decision-support system operating under clinician supervision. Before clinical implementation, prospective evaluation, external validation, calibration assessment, workflow studies, cybersecurity and privacy assessment, human-factors testing, and applicable regulatory review would be required.

1. Introduction

The electrocardiogram (ECG) remains one of the most widely used investigations in cardiovascular medicine because it provides a rapid, non-invasive assessment of cardiac electrical activity. Although modern ECG systems frequently generate digitally stored signals, ECG information is also commonly encountered in image-based formats, including printed ECG reports, scanned documents, screenshots, and exported images.

The use of AI in cardiovascular medicine has expanded considerably, with potential applications in automated interpretation, risk prediction, clinical decision support, and integration of heterogeneous clinical information. The American Heart Association has identified AI as a potentially important component of future cardiovascular care while emphasizing the need for appropriate evaluation, implementation, and clinical oversight.

The European Society of Cardiology similarly recognizes the increasing role of AI and digital technologies in cardiovascular medicine while emphasizing evidence-based implementation and appropriate clinical validation.

The prototype described in this report provides a technical foundation for one potential application: transforming an ECG image into an analyzable representation and subsequently applying an AI model. The current software is explicitly designated as a research prototype and is not clinically validated.

Accordingly, the purpose of this report is not to claim clinical effectiveness of the existing implementation. Rather, it is to describe how the technology could potentially be translated into clinical practice following appropriate validation and regulatory development.

2. Proposed Clinical Use Case

The proposed clinical application would be a clinician-facing AI-assisted ECG interpretation system.

The intended workflow could be:

Patient undergoes ECG

↓

ECG image generated

↓

Image transferred to AI system

↓

Image quality assessment

↓

12-lead detection and waveform extraction

↓

Waveform quality assessment

↓

AI analysis

↓

Clinical decision-support output

↓

Clinician review

↓

Clinical decision

The critical distinction is that the AI system would not independently diagnose or treat the patient.

Instead, it would provide information to a healthcare professional who would interpret the AI output together with the patient's symptoms, history, examination, previous ECGs, laboratory results, imaging, and other clinically relevant information.

This human-in-the-loop approach is particularly important because AI output represents one source of information rather than a complete clinical assessment.

3. Potential Clinical Workflow

3.1 ECG Acquisition

The first stage would occur during routine patient care.

A 12-lead ECG could be acquired using a conventional ECG device. In the proposed image-based implementation, the resulting ECG report would be supplied to the software as an image.

In a mature clinical implementation, direct integration with the ECG acquisition system or electronic health record (EHR) would be preferable to manual file upload. Such integration could reduce transcription errors and minimize unnecessary handling of patient data.

The current prototype instead uses a manual image-upload mechanism through a Streamlit interface.

4. Patient Identification and Data Governance

A clinical implementation would require substantially stronger data-management controls than those demonstrated by the current prototype.

The present application requests a de-identified ECG image. In a clinical environment, however, ECG data would potentially be associated with protected health information.

Accordingly, a production system would require:

• authenticated users;
• role-based access control;
• secure data transmission;
• encryption;
• audit logging;
• controlled data retention;
• appropriate patient-identification procedures;
• secure storage;
• cybersecurity monitoring;
• institutional data-governance procedures.

The exact requirements would depend on the jurisdiction and intended deployment environment.

The system should also minimize unnecessary transmission of identifiable patient information to external services.

5. Automated ECG Image Processing

After ECG acquisition, the proposed system would perform automated image processing.

The prototype invokes:

result = process_ecg(image_bytes)

The processing function returns:

result["cleaned"]

result["regions"]

result["waveforms"]

In a clinical implementation, this stage would serve two important purposes.

First, it would transform the raw ECG image into a representation suitable for computational analysis.

Second, it would determine whether the image contains sufficient information for reliable downstream analysis.

This is particularly important because clinical ECG images may vary in:

• resolution;
• contrast;
• paper quality;
• orientation;
• scanning artifacts;
• illumination;
• background grid;
• cropping;
• compression;
• device manufacturer;
• lead arrangement.

A clinical system would therefore require systematic evaluation across representative variations in ECG image quality.

6. Lead Identification and Waveform Reconstruction

The prototype identifies ECG regions and subsequently generates waveform outputs.

This creates a computational pathway of:

ECG image → lead regions → numerical waveforms

Such a pathway could potentially enable an AI model originally designed for waveform-based ECG analysis to operate on ECG images.

However, this stage represents one of the most important areas requiring clinical validation.

The reconstructed waveform must accurately represent the original ECG signal in terms of:

• amplitude;
• timing;
• morphology;
• polarity;
• intervals;
• rhythm;
• QRS morphology;
• ST-segment characteristics;
• T-wave morphology.

An apparently successful image-processing pipeline may nevertheless introduce systematic distortions that subsequently affect AI predictions.

Therefore, clinical validation should compare extracted waveforms against a reference digital ECG signal where paired datasets are available.

7. Automated Quality Control

The prototype contains an important preliminary quality-control mechanism.

The system counts usable waveform outputs and prevents AI analysis unless all 12 are available:

if usable != 12:

st.warning(

"AI inference requires 12 usable waveform outputs."

)

This principle could be expanded substantially in a clinical system.

Rather than merely determining whether a waveform exists, a clinical quality-control module should assess whether each lead is:

• present;
• correctly localized;
• sufficiently complete;
• adequately reconstructed;
• free from excessive artifacts;
• temporally coherent;
• physiologically plausible.

The system could ultimately generate a quality score such as:

ECG quality: Acceptable / Unacceptable

If quality were insufficient, the system should abstain from producing an AI interpretation.

This concept of controlled abstention is particularly important for medical AI because an algorithm should not necessarily produce an output for every input.

8. AI-Based Analysis

Following successful waveform extraction and quality assessment, the prototype loads an AI model:

model = load_model()

and performs inference:

results = predict(

model,

waveforms

)

The model returns findings associated with probabilities.

In a future clinical implementation, these outputs could potentially be displayed as:

AI finding	Estimated probability
Finding A	XX.X%
Finding B	XX.X%
Finding C	XX.X%

The specific clinical findings would depend entirely on the model's training objective.

Importantly, a probability generated by an AI model should not automatically be interpreted as a patient's true probability of disease. The meaning of the value depends on model calibration, prevalence, training data, threshold selection, and the clinical population in which the model is deployed.

Therefore, calibration should be assessed in addition to discrimination.

9. Human-in-the-Loop Clinical Decision Support

The most appropriate initial clinical role for this technology would be clinical decision support rather than autonomous diagnosis.

For example, a clinician might receive an ECG together with an AI-generated summary:

• AI-assisted analysis identifies a high probability of a predefined ECG finding.

The clinician would then independently review the ECG and determine whether the finding is clinically plausible.

The clinician would also consider:

• presenting symptoms;
• vital signs;
• medical history;
• medications;
• previous ECGs;
• laboratory results;
• imaging;
• other diagnostic investigations.

The final clinical decision would remain with the appropriately qualified healthcare professional.

This approach is consistent with the broader principle that AI should augment clinical reasoning rather than be treated as a substitute for clinical judgment. The AHA has emphasized the importance of responsible implementation of AI within cardiovascular care.

10. Potential Clinical Applications

If successfully validated, an image-based ECG AI system could potentially support several clinical scenarios.

10.1 Emergency Department

In emergency care, rapid ECG interpretation can be important because certain cardiovascular conditions require prompt assessment.

A validated AI system could potentially flag predefined ECG abnormalities for clinician review.

The system would not replace emergency physician or cardiologist assessment, but could function as an additional screening or prioritization mechanism.

10.2 Primary Care

In primary-care settings, ECG interpretation resources may vary.

An AI-assisted system could potentially provide an additional interpretation for clinician consideration, particularly when specialist interpretation is not immediately available.

10.3 Remote and Rural Healthcare

Image-based analysis may be particularly relevant in environments where ECGs are acquired locally but specialist interpretation is remote.

A validated system could potentially provide preliminary computational analysis while allowing a clinician or remote cardiologist to review the original ECG.

10.4 Cardiology Services

Within cardiology, AI could potentially be used as an additional analytical layer alongside conventional ECG interpretation.

For example, the system could facilitate:

• automated preliminary interpretation;
• prioritization of abnormal ECGs;
• retrospective research;
• longitudinal comparison;
• quality assurance;
• population-scale ECG analysis.

10.5 Retrospective Research

Before clinical deployment, the system could be particularly valuable as a research tool.

Large collections of historical ECG images could be processed to investigate model performance across different populations, devices, and clinical conditions.

11. Clinical Validation Strategy

Clinical implementation should not occur solely on the basis of successful software execution.

A staged validation strategy would be required.

Phase I — Technical Validation

The first stage should determine whether the software correctly processes ECG images.

Potential endpoints include:

• image-processing success rate;
• lead detection accuracy;
• waveform reconstruction accuracy;
• processing time;
• failure rate;
• robustness to image quality.

Phase II — Internal Model Evaluation

The AI model should be evaluated using appropriately separated development and evaluation datasets.

Performance should include discrimination and calibration measures appropriate to the intended prediction task.

Phase III — External Validation

The model should subsequently be evaluated using independent datasets from different institutions and patient populations.

This is essential because performance obtained from a single development dataset may not generalize to other clinical environments.

TRIPOD+AI specifically emphasizes transparent reporting of prediction-model development and evaluation, including information about development data, evaluation data, model specification, and performance.

Phase IV — Clinical Workflow Evaluation

The next stage should evaluate how clinicians interact with the AI system.

Relevant questions include:

• Does the system reduce interpretation time?
• Does it improve diagnostic accuracy?
• Does it increase false-positive interpretations?
• Does it alter clinician confidence?
• Does it create automation bias?
• Does it improve or worsen workflow?
• Do clinicians appropriately recognize model uncertainty?

Phase V — Prospective Clinical Evaluation

A prospective study could subsequently evaluate performance in patients encountered during routine clinical care.

The system should be assessed against predefined reference standards and clinically meaningful endpoints.

12. Safety and Failure Modes

A clinically deployable system must explicitly account for failure.

Potential failure modes include:

Poor-quality ECG image

The system may incorrectly reconstruct waveform information.

Incorrect lead identification

A waveform could potentially be associated with the wrong lead.

Missing leads

The AI model could receive incomplete information.

Image artifacts

Compression, shadows, folds, or grid interference could alter waveform extraction.

Distribution shift

The clinical population or ECG device may differ substantially from the training data.

Model uncertainty

The model may generate apparently confident predictions despite limited supporting information.

Automation bias

Clinicians may place excessive confidence in AI output.

False negatives

A clinically important abnormality may be missed.

False positives

The system may identify abnormalities that are not actually present.

For these reasons, a clinical implementation should incorporate explicit failure states and an ability to abstain from prediction.

13. Regulatory Considerations

The regulatory status of an AI-based ECG application depends on its intended use, claims, functionality, jurisdiction, and degree of influence on clinical decision-making.

In the United States, the FDA's January 2026 guidance on Clinical Decision Support Software distinguishes certain non-device clinical decision-support functions from software functions that meet the definition of a medical device. The FDA also states that existing digital-health policies continue to apply to software functions that meet the device definition.

Consequently, the intended clinical use of an ECG AI system would need to be defined carefully.

A system intended merely to display information is materially different from one intended to analyze ECG data and provide diagnostic recommendations that influence patient management.

A future development program should therefore include regulatory assessment at an early stage rather than treating regulatory considerations as an issue only after software development is complete.

14. Reporting and Scientific Standards

If this technology were evaluated as a clinical prediction model, the research should follow contemporary reporting standards.

TRIPOD+AI provides updated recommendations for studies involving prediction models developed using machine-learning or regression methods. The guideline contains 27 main reporting items covering areas including the study title, abstract, introduction, methods, data, model development, evaluation, results, and open-science practices.

A future publication should therefore explicitly describe:

• patient population;
• inclusion and exclusion criteria;
• ECG acquisition;
• image characteristics;
• preprocessing;
• waveform extraction;
• training dataset;
• evaluation dataset;
• model architecture;
• model training;
• hyperparameters;
• outcome definitions;
• missing data;
• reference standard;
• statistical analysis;
• discrimination;
• calibration;
• subgroup performance;
• external validation;
• limitations.

This level of transparency would allow independent researchers and clinicians to assess the reproducibility and applicability of the system.

15. Integration With Clinical Information Systems

A production implementation would ideally be integrated into existing clinical infrastructure rather than requiring clinicians to manually upload ECG files.

A possible architecture would be:

ECG device

↓

Hospital information system / EHR

↓

ECG AI service

↓

Image and signal processing

↓

AI inference

↓

Structured AI result

↓

Clinician-facing EHR display

The AI output could potentially be stored alongside the ECG as a supplementary interpretation.

However, integration should ensure that AI-generated information is clearly distinguished from clinician-authored interpretation.

The interface should avoid presenting experimental or probabilistic outputs in a manner that could be mistaken for definitive diagnoses.

16. Proposed Clinical User Interface

A future clinical interface could contain four major sections.

A. Original ECG

The clinician should be able to view the original ECG image without modification.

B. Processing Quality

The system could report:

ECG image quality: Acceptable

12/12 leads identified

Waveform extraction: Successful

C. AI Findings

The system could provide:

AI-assisted findings

• Finding A — probability
• Finding B — probability
• Finding C — probability

D. Clinical Disclaimer

A concise statement should clarify:

• AI-generated information is intended to support, not replace, clinical assessment.

This design would preserve the original evidence while making the AI interpretation transparent.

17. Explainability and Transparency

For clinical adoption, displaying only a probability may be insufficient.

Where technically feasible, future versions could provide additional information explaining why a prediction was generated.

For example, the system could potentially identify:

• the lead(s) contributing most strongly to a prediction;
• relevant waveform segments;
• signal-quality indicators;
• model confidence;
• uncertainty;
• reasons for abstention.

Such information may help clinicians assess whether the AI output is consistent with the visible ECG.

However, explainability methods themselves require validation and should not automatically be assumed to represent causal reasoning.

18. Ethical Considerations

Several ethical considerations should be addressed before deployment.

Patient privacy

ECG images can contain identifying information and must therefore be handled according to applicable privacy and data-protection requirements.

Algorithmic bias

Model performance should be assessed across relevant demographic and clinical subgroups.

Transparency

Patients and clinicians should be appropriately informed when AI contributes to clinical decision support.

Accountability

The responsibilities of clinicians, institutions, developers, and vendors should be clearly defined.

Automation bias

Clinical users should be trained not to accept AI recommendations without appropriate independent assessment.

Equity

The system should not systematically provide lower-quality performance for particular patient populations or healthcare environments.

The ESC has similarly emphasized that implementation of AI in cardiovascular medicine requires evidence-based evaluation and careful consideration of how these technologies affect clinical care.

19. Proposed Deployment Model

A reasonable translational pathway would be:

Stage 1 — Research Prototype

Current stage.

Purpose:

• demonstrate technical feasibility;
• develop image-processing pipeline;
• test waveform extraction;
• investigate model inference.

Clinical use: No.

Stage 2 — Retrospective Validation

Use de-identified historical ECGs with appropriate reference standards.

Clinical use: No.

Stage 3 — Prospective Silent Evaluation

The AI runs on real clinical ECGs, but its outputs are hidden from treating clinicians.

This permits evaluation of real-world performance without influencing patient management.

Clinical decision influence: No.

Stage 4 — Clinician-Assisted Evaluation

AI results become visible to clinicians as research outputs under an approved study protocol.

Clinical decisions remain independent of the system.

Clinical decision influence: Controlled research setting.

Stage 5 — Regulated Clinical Deployment

Only following satisfactory technical, clinical, human-factors, safety, cybersecurity, and regulatory assessment would routine clinical use be considered.

20. Limitations

The principal limitation of this proposed clinical framework is that the supplied software is currently an experimental prototype rather than a clinically validated medical system.

The provided source code demonstrates the user interface and orchestration of ECG processing and AI inference but does not provide the implementations of process_ecg(), load_model(), or predict().

Consequently, the following characteristics cannot currently be established:

• exact image-processing algorithm;
• waveform reconstruction methodology;
• AI architecture;
• training dataset;
• training procedure;
• model calibration;
• diagnostic accuracy;
• sensitivity;
• specificity;
• external validity;
• clinical utility.

Furthermore, the existing criterion of requiring 12 non-empty waveform outputs is a technical completeness check rather than a validated measure of ECG quality.

Therefore, no clinical performance claims should be derived from the current prototype alone.

21. Proposed Clinical Evaluation Endpoints

A future clinical study could evaluate the following primary and secondary endpoints.

Domain	Potential endpoint
Technical	Successful ECG processing rate
Technical	Lead detection accuracy
Technical	Waveform reconstruction error
Model	AUROC
Model	Sensitivity
Model	Specificity
Model	Positive/negative predictive value
Model	Calibration
Clinical	Agreement with expert interpretation
Clinical	Change in diagnostic accuracy
Clinical	Time to interpretation
Workflow	Clinician acceptance
Safety	False-negative rate
Safety	False-positive rate
Human factors	Automation bias
Equity	Performance across demographic subgroups

The selection of endpoints should be determined prospectively according to the intended clinical use and target condition.

22. Conclusion

The ECG AI prototype described in this report could potentially form the foundation of a future clinical decision-support system capable of analyzing ECG images and providing AI-assisted interpretation.

The proposed clinical workflow would consist of ECG acquisition, secure image transfer, automated image processing, lead identification, waveform reconstruction, quality assessment, AI inference, and clinician review.

The most appropriate initial clinical role would be assistive rather than autonomous. The AI system could provide an additional interpretation or prioritization signal while the final clinical decision remains under the responsibility of an appropriately qualified healthcare professional.

Translation from a research prototype to clinical use, however, requires substantially more than successful software operation. Technical validation, external evaluation, calibration, prospective clinical studies, human-factors assessment, cybersecurity, privacy safeguards, and regulatory review would all be required.

Contemporary guidance emphasizes transparent reporting and rigorous evaluation of AI prediction models. TRIPOD+AI provides a useful framework for reporting the development and evaluation of such models, while cardiovascular scientific guidance from the AHA and ESC highlights the importance of responsible implementation of AI within clinical care.

Accordingly, the current system should be regarded as a research platform for investigating image-based ECG analysis, with clinical deployment representing a future objective contingent upon successful validation. If developed according to a staged translational pathway, the technology could potentially provide clinicians with an additional source of quantitative ECG information while preserving human oversight and clinical accountability.

References

• Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378.
• Armoundas AA, Narayan SM, Arnett DK, et al. Use of Artificial Intelligence in Improving Outcomes in Heart Disease: A Scientific Statement From the American Heart Association. Circulation. 2024;149:e1028-e1050. doi:10.1161/CIR.0000000000001201.
• European Society of Cardiology. Digital tools and artificial intelligence in cardiology. ESC educational resources, 2025.
• U.S. Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. January 2026.
• European Society of Cardiology. ESC Clinical Practice Guidelines. European Society of Cardiology.