How to Build a Healthcare AI Assistant That Escalates to Doctors Automatically

The difference between a healthcare AI assistant and a liability is almost entirely in how the system handles what it doesn't know.

A general-purpose chatbot hedges. It says "I'm not a doctor" and offers a link to WebMD. A well-built healthcare assistant does something more specific and more useful: it recognises the signals that require human clinical attention, structures those signals into an alert that reaches the right person, and confirms the alert was received before the conversation continues.

That's the system we built for Formula AI, a digital health platform operating in the Gulf region. The stack is Django on the backend, OpenAI's function calling API for structured clinical output, and a tiered escalation pipeline that distinguishes emergency alerts from urgent follow-ups from routine case logging. Here's how it's put together and why the design decisions matter.

Why Function Calling, Not Prompting

The instinct when building an AI assistant is to handle everything through the prompt. Define the assistant's persona, describe how it should respond to symptoms, include instructions for when to escalate. This approach has a fundamental problem in a healthcare context: you're relying on the model to consistently extract the right signal and format the right output under conditions that vary with every conversation.

That's too much non-determinism for a system where failing to escalate a critical symptom has real consequences.

OpenAI's function calling - now called the tools API - gives you a better architecture for this. Instead of asking the model to decide what to do and express that decision in prose, you define a set of typed functions the model can invoke. The model still understands the conversation and determines which function to call. But the output is structured, validated, and handled deterministically by your application code. The AI assesses. The code acts.

For a healthcare assistant, the distinction matters: the model's job is clinical assessment within a defined scope, and the application's job is to take the assessed output and execute the correct action with full determinism. You never want the routing of an emergency alert to be a probabilistic process.

The Tool Schema

The schema is where escalation logic lives, not the system prompt. Here's a simplified version of the core tools we defined:

tools = [
    {
        "type": "function",
        "function": {
            "name": "log_patient_report",
            "description": (
                "Log a structured record of the patient's reported symptoms, "
                "severity assessment, and required follow-up level. "
                "Use this for every clinical exchange. Escalation level must "
                "always be explicitly set."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "symptoms": {
                        "type": "array",
                        "items": {"type": "string"},
                        "description": "List of symptoms as reported by the patient"
                    },
                    "severity": {
                        "type": "string",
                        "enum": ["mild", "moderate", "severe", "critical"],
                        "description": "Clinical severity assessment based on reported symptoms"
                    },
                    "escalation_level": {
                        "type": "string",
                        "enum": ["routine", "urgent", "emergency"],
                        "description": (
                            "routine: non-urgent, schedule follow-up. "
                            "urgent: requires physician review within 4 hours. "
                            "emergency: life-threatening indicators present, "
                            "alert on-call physician immediately."
                        )
                    },
                    "escalation_reason": {
                        "type": "string",
                        "description": "Clinical rationale for the escalation level assigned"
                    },
                    "patient_response": {
                        "type": "string",
                        "description": "The message to display to the patient"
                    }
                },
                "required": [
                    "symptoms", "severity", "escalation_level",
                    "escalation_reason", "patient_response"
                ]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "trigger_emergency_alert",
            "description": (
                "Immediately dispatch an emergency alert to the on-call physician. "
                "Use only when escalation_level is 'emergency'. "
                "This is irreversible and will page the duty physician."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "patient_id": {"type": "string"},
                    "symptom_summary": {"type": "string"},
                    "alert_reason": {"type": "string"},
                    "conversation_id": {"type": "string"}
                },
                "required": [
                    "patient_id", "symptom_summary",
                    "alert_reason", "conversation_id"
                ]
            }
        }
    }
]

A few design decisions embedded in this schema worth calling out.

escalation_level is a required field on every log entry - the model cannot complete the function call without committing to a level. This forces an explicit classification on every clinical exchange, which feeds the audit log and drives the downstream routing.

escalation_reason is also required. This is not just for the log - it becomes part of the alert payload the physician receives. When the on-call doctor gets paged, they see both the symptom summary and the model's stated reason for the escalation. That gives them clinical context before they call the patient back.

trigger_emergency_alert is a separate function from log_patient_report. The model has to explicitly invoke it for emergencies - it doesn't fire automatically when the log shows an emergency level. That separation is intentional: it means the Django view handler can enforce rules like "emergency log entries that don't trigger an alert within N seconds fire an automated fallback alert," rather than trusting the model to always make both calls.

Django View Architecture

On the Django side, a single POST /api/chat/ endpoint handles the conversation loop. The view:

1. Appends the patient's message to the conversation history stored in the session
2. Calls the OpenAI completions endpoint with the full history and tool definitions
3. Parses the response for tool calls
4. Dispatches to the appropriate handler function based on the tool name
5. Appends the tool result to conversation history
6. Runs a second completion to get the patient-facing response if needed
7. Returns the patient response and updates the session

The tool dispatch layer is where clinical routing happens:

def handle_tool_call(tool_name, tool_args, patient_id, conversation_id):
    if tool_name == "log_patient_report":
        entry = ClinicalLog.objects.create(
            patient_id=patient_id,
            conversation_id=conversation_id,
            symptoms=tool_args["symptoms"],
            severity=tool_args["severity"],
            escalation_level=tool_args["escalation_level"],
            escalation_reason=tool_args["escalation_reason"],
        )
        
        if tool_args["escalation_level"] == "urgent":
            schedule_urgent_followup.delay(patient_id, entry.id)
            
        elif tool_args["escalation_level"] == "emergency":
            # Log first - if alert dispatch fails, the log is the fallback
            dispatch_emergency_alert.delay(
                patient_id=patient_id,
                entry_id=entry.id,
                conversation_id=conversation_id
            )

        return {"status": "logged", "entry_id": str(entry.id)}

    elif tool_name == "trigger_emergency_alert":
        dispatch_emergency_alert.delay(
            patient_id=tool_args["patient_id"],
            summary=tool_args["symptom_summary"],
            reason=tool_args["alert_reason"],
            conversation_id=tool_args["conversation_id"]
        )
        return {"status": "alert_dispatched"}

Note that dispatch_emergency_alert is a Celery task, not a synchronous call. The alert dispatch goes into the queue immediately and the patient response is not held while the alert sends. Emergency alerts that fail to dispatch - network error, SMS provider timeout - retry with exponential backoff and trigger a secondary fallback if retries are exhausted.

What Triggers Emergency vs Urgent vs Routine

The model's system prompt defines the clinical criteria for each level. These were written in collaboration with medical advisors and are not something to default your way through. For Formula AI specifically, emergency escalation is triggered on a defined set of clinical presentations:

• Chest pain combined with shortness of breath, sweating, or arm/jaw pain
• Symptoms consistent with stroke: sudden facial drooping, arm weakness, or speech difficulty
• Severe allergic reaction indicators: throat tightening, difficulty breathing after exposure
• Suicidal ideation or self-harm intent - explicit or clearly implied
• Sudden loss of consciousness or convulsions reported by a caregiver
• Severe abdominal pain with fever above 39°C

Urgent escalation - physician review within four hours - covers presentations that aren't immediately life-threatening but require clinical assessment today: high fever in specific populations (infants, immunocompromised patients), pain above a defined threshold with escalating trajectory, symptoms that have worsened within an active care episode despite prescribed treatment.

Routine covers everything else: medication refill questions, general health queries, symptom management guidance for self-limiting conditions, scheduling requests.

The model is given explicit guidance that ambiguous presentations default upward. If the clinical picture could plausibly be urgent rather than routine, it's classified as urgent. The cost of a false positive - an unnecessary alert - is a physician spending five minutes on a non-emergency. The cost of a false negative in a healthcare system is a different category of problem entirely.

The Alert Pipeline

Emergency alerts dispatch across three channels simultaneously: push notification to the on-call physician's app, SMS to their registered number, and an in-app alert in the Formula AI physician dashboard. The on-call schedule is pulled from the rota system at alert time - not hardcoded - so the right person is paged based on current duty assignment.

The alert requires explicit acknowledgement. Not just receipt - active confirmation that the physician has seen it and is taking action. If acknowledgement doesn't arrive within five minutes, the system pages the secondary on-call contact. If that also goes unacknowledged, a clinic administrator is notified and a manual escalation process kicks in.

This acknowledgement loop was one of the most important design decisions in the build. Alerts that fire and never get confirmed are invisible failures. The system has to know the difference between an alert that was received and acted on and one that was sent into the void.

Human Safety as Architecture, Not Feature

The clinical scope of the assistant is defined and enforced at the system level, not in the prompt alone.

The assistant doesn't diagnose. It doesn't tell patients they are or aren't sick. Its patient-facing responses are constrained to acknowledging what the patient reported, explaining what happens next (a physician will be in touch, a follow-up is scheduled, the information has been logged for the care team), and providing general guidance where clinically appropriate and within the scope the platform's medical advisors defined.

The system prompt includes hard prohibitions - things the model is explicitly instructed never to do regardless of how the conversation develops. These include: providing specific diagnostic conclusions, advising patients to ignore symptoms that meet escalation criteria, suggesting medication changes outside the care plan, or reassuring a patient that emergency symptoms are "probably fine." These prohibitions exist in the system prompt and are also validated at the output level - if the patient response text contains certain patterns, the response is flagged for clinical review before it's delivered.

This might sound like over-engineering. It's not. The failure mode in a healthcare AI is not the model producing an obviously wrong answer. It's the model producing a confidently plausible answer in an edge case nobody anticipated during testing. Defensive output validation is what catches those cases.

Audit Logging

Every function call, every tool invocation, every escalation decision, every alert dispatch, and every physician acknowledgement is written to an immutable audit log. This is non-negotiable in a healthcare system.

The log schema records: timestamp, patient ID, conversation ID, tool name, full tool arguments, escalation level assigned, escalation reason, alert dispatch status, and acknowledgement status if applicable. Entries cannot be modified after creation - updates append new entries with a reference to the original.

This log has two purposes. The first is clinical governance - providing a verifiable record of how the system responded to every patient interaction, reviewable by medical supervisors. The second is model evaluation - the escalation decisions the model made across thousands of real interactions are the training signal for improving the system's clinical accuracy over time.

Graceful Degradation

The system has to handle its own failure modes as carefully as it handles clinical ones.

If the OpenAI API is unavailable, the assistant doesn't return an error to the patient. It falls back to a defined static response - "our system is temporarily unavailable, please call the clinic directly if you have urgent symptoms, or we will be in touch shortly" - and creates a manual triage task in the physician dashboard for every conversation that hit the fallback state.

If the escalation alert pipeline fails after retries, a human administrator is notified immediately. The system never silently fails on an alert that was supposed to fire.

These are boring to design and critical to have. Systems that don't degrade gracefully create liability at exactly the moments when they're most needed.

If you're building a healthcare AI system and working through the architecture decisions around escalation, scope definition, or clinical audit requirements, get in touch. This is a domain where the design decisions matter more than the model choice - and we'd rather help you get them right from the start.