Research on a live record: cohorts, consent, the privacy layer and OMOP

How a study is set up, how a cohort is cut and de-identified, what the patient sees and can refuse, and how an OMOP extract reaches an NHS secure data environment. Every screen is the running platform.

1 Start with the person whose data it is

Most research platforms begin with the researcher. This one begins with the patient, because that’s where the trust has to be earned. Robert logs into the patient portal and, under Data & Privacy, sees every research programme his record is contributing to, who runs it, how his data has been treated before it left, and a button to stop it.

The list isn’t a policy page. Each entry is a participation record held on the platform, and the portal reads it live. A programme he has opted out of is greyed and says so. A programme still running shows the plain-English button: I don’t want my data used for this. There is an undo, because people change their minds.

Tapping I want to know more opens the study in the patient’s own terms: what the study is for, whether their record went out pseudonymised or as counts only, the ethics reference, who leads it, when it started and the legal basis. The opt-out sits on the same card, so the decision is made with the facts in front of you.

One more thing the portal doesn’t do: it doesn’t hand the patient a different story to the one the research team sees. The Data Governance tab on the staff side carries a citizen-engagement table keyed to each study, and the messaging changes with the study type. A consented study says consent governs and the national opt-out doesn’t apply. A study under section 251 says the opt-out is checked and honoured before anyone is included. An aggregate study gets a transparency notice and nothing more, because there’s no individual to opt out.

2 A study carries its own approvals

On the staff side, research lives in Clinical Studies. A study has an identifier, a principal investigator, a lawful basis, an ethics reference, a DPIA reference, an approver, a retention period, a data scope and a use type. If any of those is missing the study exists but can’t release anything.

The three demonstration studies are chosen to show the three routes a study can take. One is consented and pseudonymised, under UK GDPR Article 9(2)(j). One runs under section 251 of the NHS Act 2006 and is therefore in scope of the National Data Opt-out, which the platform checks before inclusion. One is aggregate-only under Article 6(1)(e), where nothing that could identify a person ever leaves. The data class column is the important one: identifiable, pseudonymised or aggregate is a property of the study, and the extract path reads it.

Opening a study shows its approvals as structured fields: the REC reference, the DPIA reference, the named approver, the date, the retention period in months, and the data scope as a list (demographics, diagnoses, observations, medications). These aren’t free text. The evidence pack at the end reads them back.

3 Building a cohort

The Cohort Builder is a filter stack. You add filters, they combine with AND, and a preview tells you how many people match before anything is extracted. Filters cover age range, diagnosis by SNOMED code, medication, condition category, frailty score, jurisdiction and time since last contact.

Two things happen at this point that matter later. First, the anonymisation tier is chosen when the cohort is built, before any export. Pseudonymised, k-anonymous with k of at least 5, or synthetic with differential privacy. The tier travels with the cohort and the extract applies it. Second, the cohort is saved against a study, so it inherits that study’s lawful basis, scope and approvals. A cohort with no study can be previewed and can’t be extracted.

4 The privacy layer: tiers, tokens and a vault the analyst can’t open

Anonymisation on this platform is a set of mechanisms, each with a job, and each visible on screen. The Anonymisation & Token Vault tab is where they meet.

Pseudonymisation replaces direct identifiers with stable tokens. The same person gets the same token across extracts for the same study, which is what lets a longitudinal analysis work, and re-identification is only possible through an authorised break-glass. k-anonymity generalises the quasi-identifiers (age band instead of age, area instead of postcode) until every record is indistinguishable from at least four others. Synthetic with differential privacy produces a statistically representative dataset with a mathematical guarantee and no real rows in it, which is why the research portal offers it for feasibility work before a data sharing agreement exists.

Tokenisation is the mechanism under pseudonymisation, and it’s the part people ask about most. The token vault stores no original values. When an NHS number, date of birth, name, email or phone number is tokenised, the platform keeps the token and a one-way hash of the source and nothing else. Tokenising the same value again returns the existing token, so linkage holds. Nobody can read a value back out of the vault, because the value isn’t in it.

The vault runs as a separate service with its own key management and its own audit trail, following the ICO’s anonymisation guidance on keeping the key apart from the data. The access log on the tab shows who touched it and why: a read for a study extract, an audit access for a DPIA review, a scheduled rotation check, and a break-glass with its reason recorded. Analysts working on an extract can’t reach the reverse mapping at all.

5 Access control: who may share what, with whom, on what basis

Tokenisation protects the value. Policy protects the flow. Every route by which data leaves the platform is an egress policy in the Privacy & PET layer, with a named third party, its role, a legal basis, the data categories it may carry, a retention period and a review date.

The research warehouse policy is the one that matters here. It names the recipient as a processor, sets the basis as legitimate interest, limits the categories to demographics, clinical, diagnoses and outcomes, requires pseudonymisation, requires consent, and keeps the data for ten years. An extract that doesn’t fit the policy doesn’t go.

Above the policy sits the contract. The Data Governance tab holds the approval packs for each study and a register of data sharing agreements and processor contracts, with counterparties, signature and expiry dates. The rule printed on that register is the one the extract path enforces: no external share is enabled without a recorded and active agreement.

Re-identification risk is assessed on the same tab, using the ICO’s motivated-intruder approach, at onboarding, on any new linkage, on a change of recipient and periodically. The register records the assessment and its result, and the evidence pack reads it. Role-based access sits underneath all of this: a user sees approved projects and approved datasets and nothing else, and the audit trail records every ingestion, transform, query, export and approval with the user, the project and the purpose.

6 What happens when an extract is requested

This is where the mechanisms above become one decision. A request for an extract runs the cohort, applies the tier, runs disclosure control, checks the agreement, and either releases a file with a hash or blocks and says why. Every outcome, released or blocked, lands in an append-only ledger.

We requested an extract on the frailty cohort during the capture for this page. The platform narrowed the cohort to 247 rows, ran disclosure control, then blocked it. The server’s own words: “No DSA reference: external share requires data sharing agreement.” That row is the second in the ledger above, with no file hash because no file was made. The blocked extract further down was stopped for a different reason: seven rows is below the small-number threshold.

The Evidence Pack tab reads all of the above back as a checklist against the twelve controls an NHS information governance reviewer will ask about: lawful basis and approvals, classification and minimisation, tokenisation and key separation, least privilege, audit and provenance, output controls, re-identification risk, contracts, research integrity, the AI boundary, staff training and incident response. Eleven of the twelve are evidenced from the platform. The twelfth, training, is a human process, and the pack says so instead of ticking it.

7 The NHS specifically: OMOP, the secure data environment, and a day’s work

NHS research increasingly happens inside secure data environments, and the sub-national SDEs have converged on OMOP, the OHDSI common data model, as the shape data arrives in. So the question a research lead asks is simple: can this platform produce an OMOP extract that an SDE will accept?

The answer is on the OMOP CDM Transformation screen. A cohort extract is materialised on demand into OMOP CDM version 5.4, sixteen tables, with field-level provenance recorded for every cell: which extract, which table, which row, which column, and where in the source record it came from. There is no standing warehouse copy. The transformation runs against the cohort, the tier is applied, and the result is a set of Parquet or CSV files with the same ledger entry and hash as any other extract.

The Thames Valley and Surrey work. The Thames Valley and Surrey Secure Data Environment publishes its OMOP data mapper as open source. On 18 September 2026 we were asked to make sure that everything the mapper expects of a source system was present in commvita: which tables, which columns, which merge keys, what a full mapping declaration looks like, how vocabulary resolution behaves, and what an SDE-grade data quality check reports. We read the mapper’s source tree, wrote a field-level mapping declaration for all sixteen tables, made nine of them build directly from that declaration so the documentation and the emitted row can’t drift apart, added per-cell provenance, and wrote the tests and the build check that hold the declaration and the exporter in step. It was committed the same day it was asked for. From a suggestion in conversation to a working, tested export took less than a day.

That speed isn’t a stunt. It’s a consequence of how the platform is built: the record is one model with one ontology, so a new target shape is a declaration over it instead of a rewrite. The declaration-first idea itself is borrowed from the Oxford mapper, and we say so. That tool is ahead of us on a loaded vocabulary, on national datasets and on a death table. What commvita has that it doesn’t is a live cross-setting record, so the extract is cut per cohort, de-identified and gated on demand, with the opt-out honoured before any row exists.

The data quality run is the honest part, and we’d rather you saw it than didn’t. The verdict is review. Only 40.1 per cent of source codes map to a standard concept, because no OMOP standardised vocabulary is loaded: the platform has no concept tables and no import from the OHDSI vocabulary service, and the concept identifiers it emits come from a small internal map that hasn’t been verified against any published release. The screen tells you to re-resolve them against your own vocabulary before use. A thousand people are missing a year of birth in the demonstration data, and the dashboard flags that too.

Provenance is recorded per cell because that’s the question an SDE asks when a value looks wrong. “It came from commvita” isn’t an answer; “it came from that extract, that table, that row, that column, that source record” is. The provenance endpoint answers by table and key. We’ve counted 173 recorded provenance rows on the demonstration extract.

8 The researcher’s side: catalogue, request, approval

A researcher doesn’t start in Clinical Studies. They start in the Research Data Portal, which shows the datasets that exist, what each contains, how many records, which tiers are available and whether ethics and an agreement are needed.

A request goes through six stages: submitted, triage, information governance review, further information if the IG team asks for it, custodian approval, active. The portal shows where each request sits and what the requester has to do next. On the staff side the same requests appear as an activity trail on the Data Governance tab, so an IG lead sees every request, every question raised and every approval in one place.

Nothing in the request path releases data. It ends in an approval that creates a study with its approvals filled in, and the extract path does the rest, with everything in sections 4 to 6 in front of it.

Where it lives in commvita

Live means the screen is backed by platform logic and an API call we exercised while capturing this page. Demonstrated means the screen is a representative surface over seeded data.