AI tools and patient data
AI tools and patient data: what to check first
Before identifiable patient data goes into an AI tool, your organization needs an agreement with the vendor that covers health data, and you need to know where the data is stored, who can see it and how long it is kept. De-identifying data reduces the risk but rarely removes it. These are questions for your privacy officer and the vendor, and this guide lists them so you can ask.
The Workup Desk team. Last reviewed November 3, 2026. General guidance on clinical documentation, not medical advice about any individual patient.
Start with whether you need patient data at all
Many useful tasks need no patient data. Questioning a guideline, summarizing a paper and drafting a template all work with published or invented material.
When the task is about a specific patient, add only what the task needs. A referral letter needs the relevant history and results. It rarely needs the patient's address.
Understand what de-identification does and does not do
De-identification removes or alters the details that identify a patient. Direct identifiers include the name, date of birth, address, phone number and record numbers such as a medical record number or a Social Security number.
Removing direct identifiers is not always enough. A rare condition, an unusual occupation, exact dates or a small town in free text can identify a patient to someone who knows them. Under the GDPR, pseudonymized data is still personal data.
In the United States, the HIPAA Safe Harbor method lists 18 identifiers to remove. Other jurisdictions use different tests, so follow your organization's policy rather than a general rule.
- Remove names, addresses, contact details and record numbers.
- Replace exact dates with intervals where the dates do not matter.
- Check free text for occupations, places and family details.
- Treat a rare diagnosis as identifying in a small population.
Get the agreement in place first
When a vendor processes personal data on your organization's behalf, data protection law usually requires a written contract. Under the GDPR this is a data processing agreement between controller and processor. Under HIPAA in the United States it is a business associate agreement.
The agreement covers the accounts your organization manages. A doctor's personal account with the same vendor is outside it. Only add identifiable patient data once the agreement for your jurisdiction is signed and your organization has approved the tool.
Find out where the data is stored
Ask which country or region stores the data, including backups and any subprocessors such as model providers and hosting companies. Some jurisdictions restrict transfers of health data abroad.
Ask how long the vendor keeps uploads, prompts and outputs, and whether you can delete them. Ask whether any of it is used to train models, and whether that can be turned off for your organization.
Find out who can see it
Inside your organization, check whether access is set per patient or whether every user sees everything. Roles such as owner, admin and member, with sharing per workspace, let you limit each record to the people caring for that patient.
Outside your organization, ask which vendor staff can access customer data, in what circumstances, and how that access is approved and recorded.
Ask about the audit trail
An audit trail records who viewed, added, changed, shared or exported a record and when. It lets you answer a patient's question about access and investigate an incident.
Ask what the tool records, how long it keeps the log, and whether you can export it. If the tool records less than your organization requires, that is a gap to raise before you start.
Ask the vendor the same questions every time
A short, consistent list makes vendors comparable and makes gaps obvious. Put the answers in writing and keep them with the approval.
| Area | Question | What a clear answer includes |
|---|---|---|
| Agreement | Will you sign a DPA or BAA for our jurisdiction? | A named agreement and the date it can be in place |
| Storage | Where is our data stored, including backups? | Regions and a list of subprocessors |
| Training | Is our data used to train models? | A clear yes or no, and how to opt out |
| Retention | How long do you keep uploads and outputs? | Periods, and how deletion works |
| Access | Who in your company can see our data? | Roles, approval process and logging |
| Audit | What do you log, and can we export it? | Events recorded, retention and format |
| Incidents | How and when do you notify us of a breach? | A time frame and a named contact |
Questions to check in your own work
- 1.Can I do this task without identifiable patient data?
- 2.Is a data processing agreement or BAA for my jurisdiction signed with this vendor?
- 3.Am I using an account my organization manages?
- 4.Where is the data stored, and is it used to train models?
- 5.Who inside and outside my organization can see it?
- 6.Can I see an audit trail of who viewed and changed the record?
Common questions
Is de-identified data safe to put into any AI tool?
It is lower risk, not no risk. Free text and rare details can re-identify a patient, and your organization's policy may still restrict which tools you use.
Who signs the data processing agreement?
Your organization and the vendor. An individual doctor signing up with a personal account does not create one for the organization.
Is this legal advice?
No. This guide lists the questions to ask. Your privacy officer, compliance team or legal counsel decides what your organization needs.
Try it on one patient's documents
Keep each patient's sources in one workspace, ask questions of them, and draft documents you edit and export. Early access is by invitation.