GxP AI Validation Requirements for Pharma Manufacturing

Most pharmaceutical manufacturers I talk with are not asking whether they can use AI. They are already using it, in visual inspection, deviation triage, predictive maintenance, batch record review, and a growing number of chat tools that someone in quality installed because it saved an hour a day. The question that stalls them is the one an inspector will eventually ask: how do you know it works, and how do you know it still works today?

That is what GxP AI validation answers. This article walks through the regulations that apply, the draft rules written specifically for AI, and a lifecycle you can adapt without starting from scratch.

What Does GxP AI Validation Actually Mean?

GxP AI validation is documented evidence that an AI-enabled system does what you intend it to do, within defined limits, for a GxP-regulated purpose, and that it keeps doing so while it is in use. It is not a new discipline. It is computerized system validation applied to software whose behavior comes from data rather than from code someone wrote line by line.

That last difference is what makes it harder. Traditional software is deterministic, so a test script that passes today will pass tomorrow if nothing changed. A machine learning model can behave differently when the input data drifts, when it is retrained, or when the vendor quietly updates it. The validation question moves from "does the code do what the specification says" to "does the model perform acceptably on the data it will actually see, and how will we notice when it stops?"

In my view, the manufacturers who struggle most are the ones who treat AI as exotic and go looking for a brand-new framework. The existing framework already covers most of it. What is missing is the adaptation.

Which Regulations Apply to AI in Pharmaceutical Manufacturing?

No single regulation says "validate your AI this way," but several existing requirements apply directly, and one new instrument is on the way. Here is how they line up.

Two points are worth pausing on. First, the FDA's January 2025 draft guidance is explicit that it addresses AI used to produce information supporting regulatory decisions about safety, effectiveness, or quality. It is not a manufacturing validation guide, though its credibility framework is a sensible way to think about model risk. Second, the Annex 22 text discussed in this article is the July 2025 consultation draft, published on the Commission's EudraLex Volume 4 page. It is a draft, and this article does not rely on any later version.

A note on Annex 11: the clause numbers cited in this article (1, 3, 10 and 11) refer to the 2011 version. A revision of Annex 11 has been drafted alongside Annex 22, so numbering and wording may change. The lifecycle steps below are our recommended practice, not a regulatory requirement as a whole; where a step rests on a specific clause or draft expectation, it is named in that step.

A line I often repeat to clients: the regulations do not care whether the system is AI, they care whether the quality unit can demonstrate control over it.

What Does Draft Annex 22 Say About AI?

The draft Annex 22 (read the published consultation text for exact wording) is the most important new document for anyone manufacturing for the European market, and it is useful for everyone else as a statement of regulator thinking. As published for consultation in July 2025, it is scoped to AI models used in critical GMP applications, meaning applications that can affect patient safety, product quality, or data integrity.

The draft draws a hard line on the kind of model that is acceptable in those critical applications. It limits them to static models, which do not change their parameters in use, with deterministic outputs, meaning the same input gives the same output. Dynamic models that continue to learn while in operation, and models with probabilistic outputs, are not expected to be used in critical applications. The draft also states that generative AI and large language models should not be used in critical GMP applications. For non-critical applications, where a qualified person reviews the output, the draft allows more flexibility.

The draft also expects:

• A documented intended use that describes the task, the input data, and the limits of the model, reviewed and approved by subject matter experts.
• Test data that is representative of the full range of the intended use, kept independent from training data, and verified for accuracy and correct labeling.
• Predefined acceptance criteria for performance, tied to the performance of the process the model replaces, and not lower than it.
• Explainability and confidence scoring, as the draft describes them, so users can see why a model produced an output and when it is unsure.
• Human oversight where a person is in the loop, with the human responsibilities defined.
• Monitoring of performance and input data drift after release.

If you remember one practical consequence, make it this: the draft pushes the industry toward frozen, well-characterized models in critical steps, and toward humans in the loop for everything else.

How Do I Classify an AI Use Case by GxP Risk?

Risk classification is the first real decision, and it drives everything downstream. ICH Q9(R1) on quality risk management and Annex 11 clause 1 both expect validation effort to be proportionate to risk to patient safety, product quality, and data integrity.

I use a simple three-way sort when I help a team inventory their AI.

The trap is the middle category. A tool introduced as "advisory" becomes load-bearing the moment a busy reviewer stops checking its output. When I assess a site, I ask what happens if the tool is wrong and nobody notices. If the honest answer is "a batch could be affected," then it belongs in the first row, whatever the project charter says.

What Does the Validation Lifecycle Look Like?

GAMP 5 Second Edition describes a risk-based lifecycle with specification, verification, and operation phases. For AI I add several steps that traditional software does not need. Here is the sequence I recommend.

1. Define the intended use

Write down what the model is for, what inputs it takes, what outputs it produces, who uses the output, and what decision it informs. Include what it is not for. A vague intended use, a sentence long, is a common root cause of weak AI validations because every later step depends on it.

2. Assess the supplier

Under Annex 11 clause 3, you remain responsible for a system even when a vendor built it. Request what the vendor can actually show: how the model was trained, what data, how versions are controlled, how updates are communicated, and whether you can freeze a version. If a vendor cannot tell you when the model changes, you cannot hold it in a validated state.

3. Control the data

Data is the specification for a machine learning model, so it needs the same discipline you would give a user requirements document. Document the sources, the selection logic, the labeling method, and who labeled. Keep test data separate from training data, and show it covers the real variation in your process: lines, shifts, lots, lighting, materials, and seasonal effects. ALCOA+ principles apply to training and test data the same way they apply to batch records.

4. Set acceptance criteria before you test

Decide in advance what metrics matter and what thresholds pass. For an inspection model, that might be sensitivity to critical defects and false reject rate. For a classifier, it might be performance per class and not just overall accuracy, because a model can score well overall while missing the rare case you care about. Write the criteria into the protocol, and compare them to the current process, whether that is manual inspection or a legacy system.

An illustrative example (not a regulatory number): if the current manual inspection catches 90% of critical defects, you might set the model's target at 95% sensitivity or better, with a false reject rate agreed with production. To demonstrate at least 95% sensitivity with 95% confidence when the model misses none, you need at least 59 independent critical-defect samples, because 0.95^59 is about 0.05. Each additional miss raises the number of samples needed. Document the rationale for the sample size in the protocol, and use rare, realistic defects rather than synthetic ones where you can.

5. Test and challenge

Execute the protocol against the independent test set. Our recommended practice, not a clause-level requirement, is then to deliberately stress the model with edge cases, borderline defects, degraded inputs, and data from outside the intended range. Record deviations. Show that the model recognizes when an input is outside its domain and handles that safely, for example by routing it to a human.

6. Verify the surrounding system

The model is only one component. Validate the software around it: the data pipeline, the user interface, access controls, and the electronic records it produces. Under 21 CFR 11.10(e), outputs that become GxP records need audit trails that capture who did what and when, including the model version that generated the output. Record the model version with every result, or you will not be able to reconstruct a decision during an investigation.

7. Release under change control

Treat a retrained model as a change. A new training data set, a new threshold, a new vendor version, or a new input source all go through change control with a documented impact assessment and, where needed, revalidation. Annex 11 (2011) clause 10, Change and Configuration Management, states that changes to a computerised system, including configuration, should only be made in a controlled manner in accordance with a defined procedure. The impact assessment and revalidation triggers are our recommended practice.

8. Monitor in operation

Validation is a state you maintain. Define the performance metrics you will track, how often, and what triggers investigation. Watch for data drift, where production inputs slowly move away from what the model was tested on. Annex 11 (2011) clause 11 calls for periodic evaluation to confirm systems remain in a valid state, and the draft Annex 22 expects performance monitoring as well. Feed findings into your CAPA system.

How Do Large Language Models and Generative AI Fit In?

Generative tools raise the hardest questions. They are probabilistic, they can produce different outputs from the same prompt, and vendors update them often. Those properties are hard to square with the draft Annex 22 position that they should not be used in critical GMP applications.

That does not make them unusable. It means placing them where the risk allows. Using an LLM to draft a first version of a deviation summary that a trained investigator reviews, edits, and owns is a very different thing from letting it close a deviation. If you use these tools in a GxP environment, I would want to see:

• A written acceptable-use procedure that says which data may be entered, particularly regarding confidential or patient-related information.
• A named human who is accountable for every output used in a GxP record.
• Records that show the human actually reviewed and changed the output, not just clicked approve.
• Supplier controls covering where data goes and whether it is used to train the vendor's models.

For the broader legal context in Europe, Regulation (EU) 2024/1689, the EU AI Act, adds its own obligations depending on how a system is classified, and those sit on top of GMP requirements rather than replacing them.

How Does Data Integrity Apply to AI?

AI does not suspend data integrity expectations. If anything it widens them. Under 21 CFR 11.10(a), a system must be able to discern invalid or altered records, and under 11.10(e), audit trails must record changes without obscuring earlier entries. For AI, that translates into a few concrete questions an inspector can ask:

• Can you show the model version, input, and output for any decision in the past?
• Is every change to a threshold, a label, or a training data set captured in an audit trail?
• Are the people who can retrain or reconfigure the model controlled by role-based access?
• Can you show that training and test data were not altered after the protocol was approved?

Every answer should be yes. Any question you cannot answer yes to is your gap list.

How Does ISO 42001 Relate to GxP Validation?

ISO/IEC 42001:2023 is an AI management system standard. It is not a GxP validation standard, and certification does not substitute for validating a system. It can add governance for portfolio-level questions, such as who approves a new AI use case. See our page for GMP manufacturers adopting ISO 42001.

What Are the Most Common Validation Mistakes?

These are the problems we most often flag in AI validation programs.

1. Validating the demo, not the intended use. The model was tested on clean data from one line and deployed on three.
2. No independent test set. The same data used to tune the model is used to prove it, which proves very little.
3. Accuracy as the only metric. A rare critical defect can hide inside a high overall score.
4. Ignoring vendor updates. A cloud service changes and nobody is told, so the validated state quietly ends.
5. No plan for drift. The model passes on day one and nobody looks again until an investigation.
6. Human in the loop on paper only. The procedure says a person reviews every output, but the volume makes that impossible.

The last one is the hardest to see from the inside, and because it is easy to miss internally, it deserves a deliberate check.

Where Should a Manufacturer Start?

If you are building this from zero, I would take these steps in order. Inventory every AI-enabled tool in use, including the unofficial ones. Classify each by GxP impact. For anything with direct impact, confirm you can freeze the model version and that you have a written intended use. Then build your validation approach on the GAMP 5 lifecycle you already know, adding the data, acceptance criteria, and monitoring steps above. If your quality system needs broader alignment, our overview of GMP certification shows how these pieces connect.

None of this requires waiting for final guidance. The expectations in Part 11, Annex 11, and 21 CFR 211.68 are in force today, and the draft Annex 22 gives a fairly clear picture of where inspectors are heading. Start with the inventory and classification above.

Last updated: 2026-11-04

Frequently Asked Questions

Does 21 CFR Part 11 apply to AI systems?

Yes, when the AI creates, modifies, or maintains electronic records required under FDA predicate rules. Section 11.10(a) requires validation to ensure accuracy, reliability, and consistent intended performance, and 11.10(e) requires secure, time-stamped audit trails. Part 11 does not mention AI specifically, but it applies to the records regardless of how they were generated.

Can generative AI or LLMs be used in GMP environments?

The draft EU GMP Annex 22 states that generative AI and large language models should not be used in critical GMP applications. For non-critical uses, where a trained person reviews and owns the output, they may be acceptable with acceptable-use controls, supplier assessment, and records showing real human review.

What is the difference between a static and a dynamic AI model in GxP?

A static model does not change its parameters during use and gives the same output for the same input, which makes it possible to validate and hold in a validated state. A dynamic model keeps learning in operation, so its behavior can change without a controlled change event. The draft Annex 22 expects critical GMP applications to use static, deterministic models.

Do I need to revalidate when an AI model is retrained?

Retraining should go through change control with a documented impact assessment, as Annex 11 clause 10 expects for changes to computerized systems. The extent of revalidation depends on risk and on what changed, but a retrained model should never be released into GxP use without evidence that it still meets its predefined acceptance criteria.

Is ISO 42001 certification enough to satisfy GxP AI validation?

No. ISO/IEC 42001:2023 is an AI management system standard that provides governance, risk assessment, and lifecycle controls. It does not replace system-level validation under GMP requirements, though it can help organize oversight across all your AI use cases.