How to Write an AI Governance Policy for Regulated Orgs

Most AI governance policies I read fail in the same way. They are written to be approved, not to be used. They open with a paragraph about "responsible innovation," list six principles nobody can audit, and then say nothing about who decides whether a given AI system goes live. Six months later an employee pastes batch records into a public chatbot, and the policy turns out to have no opinion on it.

If you work in a regulated organization (pharma, medical devices, food, financial services, healthcare, or anything under a notified body or an FDA inspection), your AI governance policy has a harder job. It has to satisfy people who will read it with a highlighter. This guide walks through how I would write one, what it needs to contain, and where the standards and regulations actually put requirements on you.

What Is an AI Governance Policy, and What Is It Not?

An AI governance policy is the top-level document in which management states how the organization will decide whether to develop, buy, deploy, and retire AI systems, and who is accountable for those decisions. It is not a technical specification, a model card, or a list of approved tools. Those sit underneath it.

ISO/IEC 42001:2023 is the clearest model for this. Clause 5.2 requires top management to establish an AI policy that is appropriate to the organization's purpose, provides a framework for setting AI objectives, includes a commitment to satisfy applicable requirements, and includes a commitment to continual improvement of the AI management system. Annex A control A.2.2 repeats the documentation requirement, A.2.3 asks you to determine how the AI policy aligns with other organizational policies, and A.2.4 requires periodic review.

That alignment point matters more than it looks. In a regulated company, an AI policy that does not talk to your quality manual, information security policy, data governance procedures, and supplier management procedures will create conflicts the first time an auditor compares them.

Why Do Regulated Organizations Need a Different Kind of Policy?

Because your AI use is already inside someone else's rulebook. A few examples of where AI quietly lands in an existing regulatory obligation:

• Pharmaceutical manufacturing. An AI tool that screens deviations or drafts investigation reports touches the quality unit's responsibilities under 21 CFR 211.22, and any electronic record it creates or modifies raises 21 CFR Part 11 questions about audit trails and signatures.
• Medical devices. AI-enabled device software is subject to design control and software validation expectations. FDA's final guidance on Predetermined Change Control Plans for AI-enabled device software functions (December 2024) is the document to read if your model will change after release.
• Banking. The Federal Reserve's SR 11-7 (April 2011) on model risk management already treats many AI and machine learning systems as models requiring validation, inventory, and independent challenge.
• Healthcare. The HIPAA Security Rule requires an accurate and thorough risk analysis at 45 CFR 164.308(a)(1)(ii)(A), and an AI vendor that touches protected health information belongs inside that analysis.
• EU market access. Regulation (EU) 2024/1689, the EU AI Act, bans certain practices under Article 5, requires AI literacy measures under Article 4, and sets risk management, data governance, and human oversight requirements for high-risk systems in Articles 9, 10, and 14.

A generic corporate AI policy covers none of this specifically. Yours should point to it.

Which Frameworks Should the Policy Be Built On?

You do not need to pick one framework and ignore the others, but you do need a spine. I usually recommend one certifiable standard as the structure, with the others mapped onto it.

If you already hold ISO 9001 or ISO 13485 certification, the good news is that ISO 42001 uses the same harmonized management system structure, so it can be integrated rather than run as a parallel program. I cover that path for ISO 9001-certified companies separately, and the same logic applies to GMP manufacturers.

How Do You Write the Policy, Step by Step?

Here is the sequence I would follow. The order matters, because each step produces something the next one needs.

Step 1: Inventory the AI you already have

You cannot write a policy for systems you have not found. Start with a plain inventory: every AI system in development, every vendor tool with AI features switched on, and every unapproved tool employees are using anyway. The last category is usually the largest. In my view, the inventory conversation is where most organizations first discover that "we don't really use AI yet" is not true.

Capture for each system the owner, the purpose, the data it touches, whether it makes or informs decisions about people or product quality, and whether it is built in-house or supplied by a third party.

Step 2: Define scope and the regulatory obligations that apply

State plainly which parts of the organization the policy covers and which regulations bind them. ISO 42001 clause 4.1 asks you to determine external and internal issues, and clause 4.2 asks you to identify interested parties and their requirements. For a regulated company, that list includes regulators, notified bodies, customers who audit you, and patients or consumers affected by the output.

Write the obligations into a short register attached to the policy. When a regulation changes, you update the register and not the policy itself.

Step 3: Set the principles, but make each one testable

Every policy has principles such as fairness, transparency, safety, and accountability. The test I apply is simple: can an auditor ask for evidence that this principle was followed on a specific system? "We value transparency" fails. "Each AI system that influences a decision about a person has documentation describing its intended purpose, known limitations, and the point at which a human can override it" passes.

Keep the principle list short. Five that you can evidence beats ten that you can't.

Step 4: Assign roles and decision rights

This is the section most weak policies skip. Name the roles, not the people, and say what each one can approve or block. ISO 42001 clause 5.3 requires that responsibilities and authorities for relevant roles be assigned and communicated, and Annex A.3.2 covers AI roles and responsibilities specifically.

At minimum I would define:

• Executive sponsor, who owns the policy and the resources behind it.
• AI governance committee, drawing from quality, regulatory, IT security, legal, privacy, and the business functions using the AI. It approves high-risk use cases.
• System owner, one named accountable person per AI system.
• Quality or regulatory representative, who confirms the use case fits existing validated-state and change control requirements.

Include an escalation path and a statement that the committee can stop a deployment. A governance body that can only advise is not governing anything.

Step 5: Build a risk-tiering rule

You will not review a spell-check feature the way you review a model that triages complaints or flags adverse events. A tiering rule lets you spend effort where it matters. A workable version looks like this:

ISO 42001 clause 6.1.2 requires a defined AI risk assessment process that produces consistent, valid, and comparable results, and clause 6.1.4 requires an AI system impact assessment process that considers consequences for individuals, groups, and society. Your tiering rule is how you make those two clauses operational without assessing everything to the same depth.

Step 6: Write lifecycle requirements

State what has to happen before and after a system goes live. For regulated organizations, I would require at least these elements, each pointing to a procedure:

1. Documented intended use and a statement of what the system must never be used for.
2. Data requirements, including provenance, quality checks, and handling of personal or regulated data (Annex A.7 covers data for AI systems).
3. Verification and validation proportionate to the tier, using your existing computer system validation or software validation approach where it applies.
4. Human oversight design, including who reviews outputs and what they can override.
5. Change control, so that retraining, version updates, and vendor model swaps go through the same gate as any other change to a validated system.
6. Monitoring after release, with defined triggers for investigation, such as drift or an increase in overrides.
7. Retirement, including what happens to records and data.

The change control point is the one I see missed most often. A vendor can update the model underneath a tool you validated last spring, and nothing in your system will tell you unless your supplier agreement and your policy require it.

Step 7: Cover third-party and generative AI use

Most regulated companies buy more AI than they build. Your policy should require due diligence before procurement, contractual terms covering notification of material model changes, data use restrictions, and audit rights where feasible. Annex A.10 of ISO 42001 addresses third-party and customer relationships.

Add a short, plain section on generative AI. State which tools are approved, what categories of data may never be entered (trade secrets, patient data, unreleased batch or design records), and that any output used in a regulated record must be reviewed and attributed by a qualified person. Staff will follow a rule that fits on one page. They will not follow forty.

Step 8: Add training and competence

EU AI Act Article 4 obliges providers and deployers to take measures to ensure a sufficient level of AI literacy among their staff. ISO 42001 clause 7.2 requires competence for people affecting AI performance. Put a training requirement in the policy and tie it to role: general awareness for everyone, deeper training for system owners and committee members.

Step 9: Define monitoring, incident handling, and review

Say how you will know the policy is working. Set internal audit coverage under clause 9.2, management review under clause 9.3, and an incident process for AI failures, including who decides whether an event is reportable to a regulator. Then set a review cycle. I would tie review to both a calendar (at least annually) and events such as a new regulation, a significant incident, or a change in AI use.

What Should the Finished Document Look Like?

Shorter than you expect. The policy itself should run two to four pages and be signed by top management. The detail goes in supporting documents, which is where regulated organizations tend to do better than most because they already know how to manage a document hierarchy.

A structure that works:

1. Purpose and scope
2. Regulatory and standards context (reference to the obligations register)
3. Principles
4. Roles, responsibilities, and decision authority
5. Risk tiering and approval requirements
6. Lifecycle requirements (with references to procedures)
7. Third-party and generative AI rules
8. Training and competence
9. Monitoring, incidents, and review
10. Related documents and revision history

Underneath sit the AI system inventory, risk and impact assessment procedures, acceptable use standard, supplier assessment procedure, and the validation approach.

What Mistakes Should You Avoid?

A few patterns come up again and again.

Copying a template without fitting it to your regulation. A policy that never mentions your actual regulatory exposure will read as decoration, because that is what it is.

Writing it in the compliance department alone. If the people building and using the systems did not help write it, they will route around it. Include them early.

Treating the policy as the end of the project. A signed policy with no inventory, no risk process, and no one trained is a document, not a program. ISO 42001 certification audits look for operating evidence, and so will a regulator after an incident.

Setting rules you cannot enforce. If you ban all unapproved AI tools but have no way to detect or approve requests quickly, people will stop asking. A fast, simple intake path does more for compliance than a strict ban.

Ignoring the date on which obligations apply. The EU AI Act phases in its obligations over several years, and the timing of the high-risk provisions has been subject to proposed adjustments. Confirm current application dates against the Official Journal and Commission communications before you commit to a compliance schedule.

How Do You Know the Policy Is Ready?

I would ask five questions. Could a new employee find out within a minute whether a given tool is allowed? Can you list every AI system in scope and its risk tier? Does each high-risk system have a named owner and a record of committee approval? Does a vendor model change trigger your change control? And if an inspector asked tomorrow to see how you decided to deploy your most consequential AI system, could you hand them the file?

If any of those answers is no, the gap is probably in the program underneath the policy and not the policy text itself. A structured ISO 42001 gap assessment is a practical way to find out where those gaps are before an auditor or regulator does it for you. If you want help with the policy or the broader management system, my team at Certify Consulting works on ISO 42001 implementation for regulated organizations.

What would your policy say about the AI tool someone in your organization started using last week? If you don't know yet, that is probably where to begin.

Last updated: 2026-11-01

Jared Clark, JD, MBA, PMP, CMQ-OE, CQA, CPGP, Principal Consultant, Certify Consulting

Frequently Asked Questions

What should an AI governance policy include?

At minimum: purpose and scope, applicable legal and regulatory obligations, testable principles, named roles with decision authority, a risk-tiering rule, lifecycle requirements including change control and monitoring, third-party and generative AI rules, training requirements, and a review cycle. ISO/IEC 42001:2023 clause 5.2 sets the baseline content for the top-level AI policy.

Does ISO 42001 require a written AI policy?

Yes. Clause 5.2 requires top management to establish an AI policy, and Annex A control A.2.2 requires it to be documented. The policy must fit the organization's purpose, provide a framework for AI objectives, and commit to meeting applicable requirements and to continual improvement. Annex A.2.4 requires periodic review.

How is an AI governance policy different for a regulated organization?

It must connect to obligations that already apply, such as 21 CFR Part 11 and 21 CFR 211.22 in pharma, design controls and FDA guidance for AI-enabled device software, SR 11-7 for banks, the HIPAA Security Rule for health data, and the EU AI Act for EU market access. It also has to integrate with existing change control, validation, and supplier management procedures.

Who should approve and own the AI governance policy?

Top management should approve and sign it, with an executive sponsor accountable for it. Day-to-day decisions typically sit with a cross-functional AI governance committee that includes quality, regulatory, IT security, legal, privacy, and business representatives, with a named system owner for each AI system.

How often should an AI governance policy be reviewed?

At least annually, and also when triggered by events such as a new or amended regulation, a significant AI incident, or a major change in how the organization uses AI. ISO 42001 Annex A.2.4 expects planned review, and management review under clause 9.3 provides the formal forum.