The answer to AI's privacy problem: a small language model on the device

Large language models are changing clinical writing. Letters, summaries and report drafts that once took an evening now take minutes. Most of those models run on someone else's servers, and that is where the privacy problem starts. A clinician who pastes a session note into a chatbot has sent a child's name, date of birth and diagnosis to a third party.

Vendors promise not to train on that text. A promise is a policy. What clinical text needs is a control: identifying details that never leave the device in the first place.

A small model that runs where the note is written

The Syntropic research lab has built a small language model for exactly this job. It is fine-tuned for clinical text from an open-source base model, and it is small enough to run on a clinician's own computer, with no network access of its own.

It reads a note, finds every identifying detail and replaces each one with a typed marker:

• names become [NAME]
• dates tied to a person become [DATE]
• addresses, phone numbers, email addresses and record numbers each get a marker of their own

Only the de-identified text goes on to the larger model. The link between each marker and the real value stays on the device, so the draft that comes back is restored there and nowhere else.

Typed markers, not blanks

Blanking a detail breaks the sentence. A typed marker keeps its shape. The larger model still knows a name was there, that a date followed it and that an address belonged to the family, so the draft it returns reads correctly. The clinician sees the real details back in place.

Reading in context

Rules and patterns find a Medicare number. They struggle with a name used inside a sentence, a nickname, or an address split across two lines. A language model reads the note in context, the way a person does. That matters most in paediatric records, where a sibling's name, a school and a suburb sit side by side in almost every note.

Built to the Safe Harbor list, extended for Australia

The model covers all 18 identifier types on the HIPAA Safe Harbor list, the most widely used checklist of what makes a health record identifiable. Australian law sets its own test: the Privacy Act asks whether a person is reasonably identifiable, and treats health information as sensitive information. The Safe Harbor list is the floor, not the ceiling.

The model is now being trained on Australian names, addresses and identifiers, including Medicare numbers, NDIS participant numbers and Individual Healthcare Identifiers.

How it is tested

Every version is benchmarked on synthetic records written in the shape of paediatric health notes. No patient or practice data is used at any stage. Each round adds harder cases: names inside addresses, identifiers split across lines, and details mentioned only in passing.

Where it goes next

The model already runs as a standalone app and has been tested end to end with the Syntropic platform. The next step is to make it the layer that runs before every AI call, so identifying details stay with the clinic by design.

Follow the work in the research lab.

Sources

• hhs.gov
• oaic.gov.au