By James East, Interim CEO of FirstHx
In 1999, we learned how to make medicine safer. AI is tempting us to forget it.
The newest wave of healthcare AI promises to interpret charts, make decisions and act on our behalf. But we are giving these systems more autonomy before fixing the fragmented, incomplete information underneath them. Medicine learned decades ago that safety depends on standardized, reproducible and transparent processes. The excitement around AI is tempting us to believe a smarter model can compensate for a poorly designed system. It cannot.
In November 1999, the U.S. Institute of Medicine published To Err Is Human: Building a Safer Health System. It estimated that between 44,000 and 98,000 Americans were dying in hospitals every year from preventable medical errors—more than motor vehicle accidents or breast cancer.
That report shook medicine. Its central lesson was simple: good clinicians can cause unintended harm inside poorly designed systems. Safer care requires processes that prevent errors, expose failures and support people in doing the right thing.
More than 25 years later, I worry we’re forgetting that lesson.
Every week brings another promise that AI agents will transform healthcare. Point a model at the chart, let it reason, give it more responsibility and watch outcomes improve.
The promises have moved from documenting care to orchestrating it. The underlying information hasn’t suddenly become more reliable.
I’ve spent 17 years at the bedside in critical care and 15 years in health IT. I want this to work as much as anyone. But healthcare is not a market you can brute-force with compute.
Privacy is only part of the reason.
Set PHIPA — Ontario’s Personal Health Information Protection Act, 2004, which governs the collection, use and disclosure of personal health information — and broader privacy and security requirements aside for a moment. Healthcare remains difficult for structural reasons:
- It is historically siloed. Data lives across EMRs, departments and formats. Even clinicians in the same clinic and subspecialty, using the same EMR, can document very differently.
- It adopts slowly, often for good reason. Clinicians have been promised transformation before. They carry the consequences when it falls short.
- It demands validity before deployment. A tool that is right 92% of the time is not “mostly working” if its failures include missed allergies or missed sepsis.
- It demands real ROI. Health systems run on thin margins. “Interesting” doesn’t get funded. Measurable improvement does.
Whether you’re deploying agentic workflows, clinical decision support or ambient scribes, the question isn’t only “Is the model good?”
It’s “What is the model standing on?”
Applying the lessons of To Err Is Human
The report’s systems approach remains directly relevant:
- Standardize and simplify. Unnecessary variation creates opportunities for error.
- Reduce reliance on memory and inference. Make the right information available at the right time.
- Make errors visible and learnable. You can’t improve what you can’t trace.
- Design for safety from the start. Don’t depend on catching errors downstream.
Now hold the latest AI promises up against those principles.
An inference-based agent operating on incomplete, inconsistently documented charts can compound the problems we have spent decades trying to reduce. It can add variability, fill gaps with inference, obscure the basis for an output and leave clinicians catching errors after the fact.
The more capable these tools become, the easier it is to mistake capability for reliability.
Greater autonomy raises the stakes. It does not lower the standard for trustworthy inputs.
We would never accept “the clinician will catch it” as the safety strategy for a new medication-ordering system. We shouldn’t accept it for AI.
Successful AI depends on what happens upstream.
For AI deployments to be successful, efficient and economical, they need early, high-fidelity data that is:
- Standardized and traceable, with a clear source and meaning
- Testable and validated, so performance is measured rather than assumed
- Dynamic, adapting as evidence, requirements and patient circumstances change
- Locally grounded, reflecting patients’ cultural, social and language realities
Without that foundation, organizations risk spending ever larger sums on compute, hoping more reasoning will compensate for poor inputs.
More reasoning cannot reliably recover information that was never captured. A confident answer does not resolve the uncertainty underneath it.
This is where FirstHx fits.
At FirstHx, we capture the patient’s history directly from the patient, before the visit, in a structured and deterministic way, and anchor it to the chart.
The history is captured in a standardized, reproducible and transparent format. Its source is explicit: what the patient reported, rather than what a model reconstructed from fragments.
Patient-reported information still requires clinical interpretation and, where appropriate, verification. Capturing it consistently gives clinicians and downstream tools a stronger starting point.
Scribes, decision support and agentic workflows all benefit. Reliable inputs create the conditions for better outputs, less rework and safer automation.
This is To Err Is Human, applied to AI:
- Standardized inputs
- Less dependence on inference
- Transparent, traceable data
- Safety designed in from the start
The future of AI in healthcare won’t be decided by who has the biggest model or the most autonomous agent. It will be decided by who builds on a foundation clinicians can trust, regulators can validate and health systems can afford.
More than 25 years ago, we learned that reproducible, transparent, standardized systems are what protect patients. New AI does not make that lesson obsolete. It makes forgetting it more dangerous.