The scenario library
21 declared scenarios today: medication safety review (18) and data integrity (3). Below is how each one fared in a real 3,000-patient run.
Run RUN-745029d75136
- Specification
- medication_safety v1
- Population
- 3,000 patients, seed 7
- Measured
- 27 Sep 2026, MacBook Pro, Apple M3, 8 GB
- Result
- 16 of 21 met, 0 short, 5 not verifiable
Intended use, as written in every Passport: “Test a medication safety review agent that reads a patient’s medication list, conditions, allergies and recent labs, and flags safety problems.”
Showing 21 of 21 scenarios
| Scenario | Severity | Target | Found | Found against target | Status |
|---|---|---|---|---|---|
| SCN-MEDSAFE-001Medication prescribed despite documented allergy to it | critical | 50 | 110 | Met | |
| SCN-MEDSAFE-002Renally cleared medication with impaired kidney function | critical | 60 | 85 | Met | |
| SCN-MEDSAFE-003Potassium-raising drug with existing hyperkalaemia | critical | 40 | 0 | Not verifiable | |
| SCN-MEDSAFE-004Duplicate therapy: two active medications, same class | high | 40 | 0 | Not verifiable | |
| SCN-MEDSAFE-005Polypharmacy: eight or more concurrent active medications | high | 50 | 80 | Met | |
| SCN-MEDSAFE-006Five or more active chronic conditions | normal | 40 | 65 | Met | |
| SCN-MEDSAFE-007No allergy record of any kind exists60 built, 90 by chance | high | 60 | 150 | Met | |
| SCN-MEDSAFE-008Medication present with no supporting indication anywhere | high | 40 | 0 | Not verifiable | |
| SCN-MEDSAFE-009Relevant monitoring lab never recorded for a monitored drug | normal | 30 | 0 | Not verifiable | |
| SCN-MEDSAFE-010Medication with no recorded dose | normal | 30 | 55 | Met | |
| SCN-MEDSAFE-011Same medication recorded twice with different doses30 built, 53 by chance | high | 30 | 83 | Met | |
| SCN-MEDSAFE-012Duplicate patient record with drifted demographics | normal | 25 | 25 | Met | |
| SCN-MEDSAFE-013Allergy recorded as both active and refuted | normal | 20 | 20 | Met | |
| SCN-MEDSAFE-014Allergy documented after the allergen was prescribed | high | 30 | 30 | Met | |
| SCN-MEDSAFE-015Medication started before its indication was diagnosed | normal | 25 | 0 | Not verifiable | |
| SCN-MEDSAFE-016Long medication history: 10+ years of records25 built, 1,187 by chance | normal | 25 | 1,212 | Met | |
| SCN-MEDSAFE-017Age 85+ with three or more concurrent medications | high | 40 | 83 | Met | |
| SCN-MEDSAFE-018Extreme lab value outside typical reportable range | critical | 40 | 40 | Met | |
| SCN-DATAINT-001Observation references an encounter that does not exist | high | 20 | 20 | Met | |
| SCN-DATAINT-002Medication references an encounter that does not exist | high | 20 | 20 | Met | |
| SCN-DATAINT-003Medication cites an indication that is not in the record | normal | 20 | 20 | Met |
How to read this
- Met
- Present in the quantity asked for.
- Short
- Present, but in fewer patients than asked for.
- Not verifiable
- The scenario depends on a clinical fact no clinician has signed. The engine refuses to count it. This is not a failure of the data, and the Passport says so.
- Met, mostly by chance
- The coverage report flags scenarios found far above target: MEDSAFE-007 (60 built, 90 by chance), MEDSAFE-011 (30 built, 53 by chance), MEDSAFE-016 (25 built, 1,187 by chance).
- Reviewer status
- 8 scenarios have no named clinical reviewer. Every run prints “NOT RELEASABLE — 8 scenario(s) have no clinical reviewer”. That is by design, and it stays until a clinician signs them.
Met, but mostly by accident
“Met” does not always mean the population was tightly controlled. The report separates patients built for a scenario from patients who satisfy it by chance, so over-coverage cannot hide.
SCN-MEDSAFE-007No allergy record of any kind exists
60 built (target 60) 90 by chance
SCN-MEDSAFE-011Same medication recorded twice with different doses
30 built (target 30) 53 by chance
SCN-MEDSAFE-016Long medication history: 10+ years of records
25 built (target 25) 1,187 by chance
Clinical state
Nothing has been signed by a clinician yet. Finding a clinical reviewer is our top priority.
- Valid-mode run
- 13 of 17 met, 0 short, 4 not verifiable
- Medication-safety quality gate
- 13 of 18
- Clinical checks
- 14 declared: 7 data-consistency rules applied, 7 clinical rules waiting for clinician review
- Drug knowledge
- 91 drug entries across 8 WHO ATC classes, sourced but not yet signed by a clinician
- Scenarios with no named clinical reviewer
- 8
Need situations that are not in this library? New scenarios can be written for your domain. Custom scenario authoring
Bring the list of situations your software must handle.
We will turn it into a scenario specification and show you what a pilot would deliver. Or email us at hello@supermerco.com.