The evidence document delivered with every run. It records what is in the data, what is not, and what nobody tested, and it keeps saying so when the answer is unflattering.
Pick a section to see what it holds. Facts shown are from a real run.
Which generator, specification and population size produced the data, and what it is meant to test.
- Generator: hde.compose.Composer/0.2-construct-fill-repair EVAL-0.1
- Run: RUN-2ebb950f037d
- Specification: medication_safety v1
- Seed: 7
- Population: 3000 patientsWord for word, in every run. They show how we write about what the data does and does not establish.
This assessment measures coverage of a declared specification and the properties listed. It does not establish that the data is anonymous, that its use is lawful, or that any system tested against it is clinically safe.
This rule set is incomplete and covers only the named domain. A dataset can satisfy every rule applied here and still be clinically wrong in ways these rules do not check.
Not tested: attacks using auxiliary external datasets; attacks with access to generator parameters; disclosure through attributes not declared as quasi-identifiers; longitudinal structure beyond the tested columns; any risk arising after the data leaves the environment where it was assessed.
Drug classes: 91 entries verified against an official source (WHO ATC/DDD Index), not reviewed by a clinician.
A real excerpt from a 3,000-patient run. Only the cuts marked … are ours.
We report what we did not measure, instead of returning a reassuring zero or a combined score.
“NaN here means NOT MEASURED, not zero risk and not a low value.” That line is in the Passport itself.
# Data Passport
## 1. Identity and intended use
- Generator: hde.compose.Composer/0.2-construct-fill-repair EVAL-0.1
- Run: RUN-2ebb950f037d
- Specification: medication_safety v1
- Seed: 7
- Population: 3000 patients
## 8. Clinical consistency
…
What the medication knowledge rests on.
- Drug classes: 91 entries verified against an official source (WHO ATC/DDD Index), not reviewed by a clinician.
- Monitoring requirements: 4 entries verified against an official source (US prescribing information on DailyMed), not reviewed by a clinician.
- Indications: none recorded.
## 9. Privacy measurements
- privacy.singling_out_rate = 0.0323 proportion of records unique on the stated quasi-identifiers
- assumptions: those three fields are what an adversary would know in advance. That choice is a judgement and a different choice gives a different number
- privacy.exact_match_rate = NOT MEASURED
- NaN here means NOT MEASURED, not zero risk and not a low value.Beta When we run your system against the test patients, you also receive a Test Passport. It records every record sent and its fingerprint, your system’s name and version as it reported them, each answer and how it was judged, which change to the record caused a failure, the regression case that reproduces it, and what the test did not cover.
We will turn it into a test specification and show you what a pilot would deliver. Already have FHIR test data? Send a sample and we will show you what it is missing. Or email us at [email protected].