Building HIPAA- and GDPR-Compliant AI for Healthcare
Compliance is a design input, not a final review. The architectural decisions that make healthcare AI defensible under HIPAA, GDPR and the EU AI Act.
Building AI for healthcare means working with some of the most protected data that exists. In the US, HIPAA governs protected health information. In the UK and EU, GDPR treats health data as a special category requiring an explicit legal basis and extra safeguards. Switzerland's revised Federal Act on Data Protection and Qatar's personal data protection law add their own requirements. On top of this, the EU AI Act introduces obligations for AI systems based on their risk.
None of this makes healthcare AI impractical. It means compliance has to shape the architecture from the first week. This post covers the decisions we make on every healthcare AI project. It is engineering guidance, not legal advice — every project should involve your data protection officer and legal counsel.
Data minimisation: the cheapest control there is
The safest data is data you never collect. For each AI feature, write down exactly which fields the model needs to do its job, and send nothing else. A model summarising a referral letter may not need the patient's name, address or insurance number. A model forecasting appointment no-shows may need history, but not diagnosis.
Minimisation also applies to retention. Decide how long prompts, outputs and logs are kept, and delete on schedule. Logs are a common blind spot: a well-meaning debug log can quietly become the largest store of patient data in the system.
De-identification and pseudonymisation
Where a task can be done without identifying the patient, remove identifiers before data reaches the model. Under HIPAA, de-identification follows defined methods. Under GDPR, pseudonymised data is still personal data, but pseudonymisation is a recognised safeguard that reduces risk. Practical techniques include:
- Replacing direct identifiers with tokens before text is sent to a model, and re-inserting them only in the final, access-controlled view.
- Detecting identifiers in free text — names, dates, addresses, record numbers — with a dedicated step, then reviewing samples to check what it misses.
- Generalising quasi-identifiers such as exact dates of birth or rare conditions that could identify someone in combination.
- Using synthetic or de-identified data for development and testing, so engineers do not need access to real records.
Hosting in-region and choosing model providers
Where data is processed matters as much as how. For UK and EU patients, that usually means hosting in the UK or EU and checking every sub-processor, including the model provider. For US healthcare clients, any vendor handling protected health information needs a business associate agreement. Swiss and Qatari clients often have their own residency expectations.
In practice there are three options: a major cloud provider's model service in a specific region under a suitable agreement; an open model hosted in your own cloud account; or an on-premises deployment. Each has trade-offs in capability, cost and operational effort. The important thing is to make the choice explicitly and document it, and to confirm that prompts and outputs are not retained or used for training by the provider.
Audit trails and access control
Every AI-assisted action in a clinical setting should be reconstructable: who requested it, what data was used, which model and prompt version ran, what it returned, and what the human did next. That record supports incident investigation, patient requests and regulatory review. Keep it tamper-evident and separate from application logs.
Access control applies to AI as much as to any other feature. If a retrieval system searches clinical documents, it must respect the same permissions as the source system, filtering at query time — a pattern we describe in our RAG development guide.
Human in the loop, by design
For anything that influences care, the AI should support a professional, not replace one. That has concrete design consequences: output is presented as a suggestion with its sources and reasoning; the clinician confirms, edits or rejects it; overrides are easy and recorded; and the system makes its uncertainty visible rather than hiding it behind confident prose. GDPR also gives individuals rights around solely automated decisions with significant effects, which human review helps address.
EU AI Act awareness
The EU AI Act classifies AI systems by risk. Many healthcare uses — particularly AI that is part of a regulated medical device — fall into the high-risk category, which brings requirements around risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy and robustness. Obligations are being phased in over several years, so check the current timeline for your use case.
The useful observation for engineering teams is that the practices described above — documented data sources, versioned models and prompts, evaluation reports, logging and human oversight — are exactly the evidence the Act expects. Building them in from the start is far cheaper than reconstructing them later.
Vendor and sub-processor due diligence
A modern AI system involves several third parties: a cloud host, a model provider, perhaps a vector database service, an observability tool and an email or messaging provider. Each one that touches personal data is a processor or sub-processor and needs to be listed, assessed and covered by an agreement. Observability and error-tracking tools are frequently overlooked, yet they can capture full prompts and responses.
Keep a simple register of every service, what data it receives, where it processes that data and under which agreement. Review it whenever a new tool is added or a provider changes its terms. This register is also one of the first documents a hospital procurement or information governance team will ask to see.
A compliance checklist for healthcare AI
- Identify the legal basis and the applicable regimes: HIPAA, UK/EU GDPR, Swiss FADP, Qatari law, medical device rules, EU AI Act.
- Run a data protection impact assessment before building.
- Define the minimum data per feature and a retention schedule, including logs.
- De-identify or pseudonymise wherever the task allows.
- Choose hosting region and model provider explicitly; sign the right agreements.
- Enforce source-system permissions inside retrieval.
- Record a full audit trail of AI-assisted actions.
- Design human review, overrides and visible uncertainty into the interface.
- Version models, prompts and evaluation results; re-test on every change.
How Amygdal can help
We build healthcare software and AI with privacy as a design input, for clients in the UK, Germany, Switzerland, Qatar and the US. See our AI in healthcare service and our broader guide to implementing AI in healthcare, or contact us to discuss your project.
