What we are building next

Supermerco is building healthcare scenario infrastructure. You define the situations your software must handle; we construct synthetic patient trajectories with known clinical truth, the record your system is allowed to see, the workflow actions that happened, and any declared errors or delays. Each can be verified independently, used as data on its own, or run against your system and kept as a regression case.

In development Nothing on this page is available unless it is marked Available or Beta.

Programmable healthcare test worlds with verifiable ground truth.

We do not aim to win generic synthetic data or generic AI evaluation. We aim to own the healthcare reality those systems are tested against.

The chart is not necessarily the patient

Each synthetic patient will carry three separate realities, and a scenario can make them disagree on purpose.

Clinical truth
What is actually true in the synthetic world: the medicine taken, the allergy, the condition, whether the patient takes the dose.
Recorded state
What the EHR or downstream system shows. It can be incomplete, stale, delayed, duplicated, contradictory, wrongly linked or simply wrong.
Workflow state
What people and systems actually did: ordered, performed, returned, reviewed, changed, handed off, followed up.

Same patient · identical history until 17:45

09:00blood collected10:00result produced10:03visible in record17:45review time18:10action timeLaterdeclared branch

Clinical truthHigh potassiumMedicine stoppedStable

Recorded stateResult producedResult visibleChange recorded

WorkflowBlood collectedResult reviewedMedicine changedFollow-up arranged

In developmentIllustration of a counterfactual twin. Exactly one declared divergence; every later difference follows from declared rules, and the pair is verified with a machine-readable diff. A declared branch, not a prediction.

Human and process error, as data

Not random corruption. Each error is a declared object with a cause, a consequence and a place in time.

Documentation errors

Allergy omitted
Stale medication list
Wrong dose recorded
Wrong encounter association

Clinical-process failures

Monitoring not ordered
Result available but never reviewed
Missed follow-up
Handoff omission

Patient behaviour

Medication non-adherence
Wrong dose taken
Lab not completed
Late presentation

Integration and system failures

Result delayed
Duplicate message
Wrong-patient reference
Change not propagated

Every error records its type, actor, event time, visible time, affected state, declared cause, declared consequence, detection, correction, provenance and clinical-review status.

Four ways in, without buying everything

Start with data. Add coverage, error trajectories or regression when you need them.

Scenario data

Available

For teams that only need data.

requirement
specification
construct
verify
FHIR and test files
Data Passport

Coverage (gap-fill)

Available

For teams with existing test data.

existing suite
coverage analysis
gaps
build missing cases
merged, verified suite

Clinical reality and error trajectories

In development

For advanced teams.

clinical truth
record state
workflow state
controlled divergence
counterfactual twin

Regression

Beta

For a continuous testing workflow.

Beta for individual failures; suites rerun on every release are in development.

scenario
your system
failure
minimisation
regression case
rerun on each release

Milestones, in order

Milestone A
Clinical reality model
Clinical truth, recorded state and workflow state as separate models, with multiple timestamps, first-class error events and an independent verifier.
The model of truth, record, workflow and system views is in place; the failure scenarios built on it are in development.
Milestone B
Five deep failure trajectories
True allergy absent from the chart; medication discontinued but stale in the record; result produced but never reviewed; handoff information loss; medication recorded active while the patient does not take it.
Milestone C
Counterfactual twins
For every supported error: a baseline and an error branch with identical history before the divergence, an exact diff, and independently verified lineage.
Milestone D
An external customer system
Run trajectories against a real external AI or software product and find a failure the customer did not already have as a regression case.
The connection to external systems works against an open-source FHIR server and a reference system; a customer’s system is next.
Milestone E
Continuous regression
Keep customer-private scenario suites and rerun them across versions.
Later
Deeper clinical trajectories
Complications, recovery branches and multi-year progression, only once clinical-review capacity exists.

The first trajectory families

Medication safety and data integrity first: a few deep, high-value scenarios rather than hundreds of shallow ones.

True allergy missing from the record
Medication prescribed despite a documented allergy
Medication discontinued but still active in the record
Medication recorded active but the patient does not take it
Monitoring not ordered
Monitoring ordered but not performed
Result available but not reviewed
Abnormal result followed by delayed action
Medication or allergy information lost during a handoff
Two systems disagree about the patient’s current state
Wrong encounter or reference linkage
Duplicate patient, or demographic drift
Stale or contradictory medication history
Delayed documentation versus actual event time

Your eval tools run the test. We supply the healthcare world.

In development Generic agent and evaluation tools can already run scenarios, call a system and grade the answer. We would rather work with them than against them: your tools or agents call Supermerco for the healthcare scenario, its hidden ground truth and the evidence, and run it against your system as they do today.

Your eval tools or AI agents
Supermerco: the healthcare test world
Your system

The workflow we are working toward

Five steps are available today, four are in beta, and one is in development. Steps 6 to 9 are the system-testing workflow, offered as a guided beta.

1
Define the situations your system must handle
Available
2
Measure what your existing test data already covers
Available
3
Identify missing and under-covered situations
Available
4
Build the missing cases
Available
5
Verify the resulting coverage independently
Available
6
Run the cases against your AI or software
Beta
7
Identify and attribute failures
Beta
8
Isolate the cause with controlled changes
Beta
9
Save the failure as a reproducible regression case
Beta
10
Rerun the regression suite on every release
In development

Available today, and next

Area by area.

Area	Available today	In development
Test data	Synthetic patients built to your specification; gap-only building against your existing suite.	Standard medical codes in the data; real conditions; values across reviewed ranges.
Verification	Independent verification of every situation; judging your system’s answers: pass / fail / unresolved (beta).	Verification of truth-versus-record test worlds.
Evidence	Data Passport and manifest; Test Passport.	Passports for test worlds.
Testing your system	Runs against a FHIR server and any system that answers over HTTP in JSON (beta).	A first customer system.
Finding causes	Controlled changes that pinpoint which part of a record causes a failure (beta).	Counterfactual twins.
Clinical reality	The model of truth, record, workflow and system views is in place.	The failure situations built on it, starting with three.
Regression	Failures saved and rerun: fixed, reproduced or changed (beta).	Suites rerun on every release, in your CI.
Clinical content	Facts from official sources, named as such; clinician review under way.	Clinician-reviewed content.
Interface	Guided pilots and a command-line tool.	Other interfaces once customers need them.
Comparison	None published yet.	A fair comparison, published whichever way it falls.

What we are working on now

As each item lands, its label changes from In development to Available or Beta.

Standard medical codes in our data: RxNorm for drugs, LOINC for labs, a standard code system for conditions and allergiesData that reads like the records your systems expect, and gap-fill that reads coded suites exactly.
Real conditions in place of placeholder codesUnlocks the indication list and the last two situations.
Lab values that vary across ranges reviewed by a clinician, and doses taken from drug labelsValues that behave like real records.
Clinician sign-off on the key clinical situationsReviewed content for release.
The first three truth-versus-record situations: a true allergy missing from the chart, a stopped medication still active, a result never reviewedThe core of the test-world product.
A first customer system through the regression workflowCustomer evidence for the beta.

Where an AI model might fit, and where it never will

Proposed direction Paid third-party AI services will not be a hard runtime dependency of the product core. Frequent scenario analysis, counterfactual exploration and system testing could become expensive and dependent on third-party costs, quotas, latency and availability. The preferred direction is a strong self-hosted or fine-tuned model for assisted tasks, if it meets measured quality requirements.

Stays deterministic

Scenario constraints and generation planning
Patient construction
Independent verification
Coverage counts
FHIR structural checks
Evidence and manifest hashing
Reproducibility
Regression case identity

Candidate assisted tasks

Turn a plain-language requirement into a draft scenario specification for human review
Propose terminology and code mappings, with deterministic validation and reviewer approval
Summarise or classify outputs from a system under test
Cluster similar failures after deterministic evidence is recorded
Suggest counterfactuals or next tests, with the change and its verification kept deterministic
Write explanatory text for a Test Passport from structured results

A model is never

The sole verifier of whether a patient satisfies the specification
A substitute for clinician review of clinical rules
The source of a regulatory, compliance or safety conclusion
An untracked clinical oracle whose output becomes ground truth
A required network service for basic generation or reproduction

Before any model is chosen, candidates are benchmarked on our exact workload: accuracy, structured output, latency, memory and cost.

What we are deliberately not building

Every substantial feature has to make the healthcare test world more valuable, or be required by a real customer.

A general-purpose AI evaluation or red-teaming tool
A universal digital twin, or a simulator of every disease
Predictions of what will happen to real patients
Ingestion of production patient data
Regulatory sign-off services
Arbitrary “realism” scores

Bring the list of situations your software must handle.

We will turn it into a test specification and show you what a pilot would deliver. Already have FHIR test data? Send a sample and we will show you what it is missing. Or email us at [email protected].

Talk to us about a pilot