Intent Debt: Why Is That Step in the Procedure?

Ask an AI to streamline a procedure, and it will find steps that look like waste. Some of them are waste. Others are the only thing standing between you and a failure your company has already had once.

The trouble is that the procedure rarely tells you which is which. It tells people what to do, and it almost never tells them why. In The Hidden Debts of AI, I listed this as intent debt: the goals, constraints and reasons behind a process that were never written down, or were written down somewhere nobody will look.

Quality systems have carried this debt for decades. We got away with it because the people who remembered the reasons were still around, and because changing a procedure was slow. AI changes both of those things. It can revise a hundred procedures in an afternoon, and it judges each step only by what is on the page.

Here's what's inside:

• The fence nobody remembers building. Why a step whose reason was never recorded looks exactly like waste.
• Why AI makes it worse. AI is very good at finding redundancy, and that is the problem.
• Where the reasons actually live. Most of them are already in your quality system, just not next to the step.
• Paying the debt down. Four changes to make before AI starts editing your procedures.

The fence nobody remembers building

G.K. Chesterton described a reformer who comes across a fence built across a road and says, "I don't see the use of this; let us clear it away." Chesterton's answer was that if you don't see the use of it, you are the last person who should clear it away. Go and find out why it was put there first, and then we can talk about removing it.

Quality people know that fence well. Here's a fictional example.

A dietary supplement plant has an encapsulation procedure. In step 3, the operator inspects the machine before use, including the hopper lid and its gasket. In step 9, near the end of setup, a second operator confirms the gasket is in place before the first capsules are made.

A new process engineer asks an AI to streamline the procedure. The AI recommends removing step 9, because the gasket is already checked in step 3, the second check adds a person and several minutes to every changeover, and nothing in the procedure explains why the gasket needs to be checked twice. It's a sensible recommendation, and the reasoning is clear.

What the AI couldn't see is that the gasket comes off during cleaning, and cleaning happens after step 3. A few years ago a gasket was left off after cleaning, and pieces of a damaged seal ended up in finished capsules. A customer complaint led to a CAPA, and the CAPA added step 9. The SOP's revision history says only "Rev 6: added step 9 per CAPA-19-014," and nobody in the department today was there when it happened.

The QA reviewer joined the company two years ago. She reads the AI's reasoning, finds it sound, and approves the change.

Nobody in this story was careless. The procedure told everyone what to do and nobody why, and the AI was simply the first reader to take it at its word.

Why AI makes it worse

A human engineer could make the same mistake. The difference is in how the change gets made.

A person who wants to remove a step usually has to ask around first, and asking around is where the history comes out. Someone says, "Don't touch that one, we had a complaint." That informal check has protected a lot of quality systems, even though nobody ever designed it.

An AI doesn't ask around. It reads what it's given, and a step without a written reason looks the same as a step that really is redundant. Finding duplication is one of the things AI does best, so the steps it flags first will be the ones that seem to repeat something else. Controls added after a failure often do exactly that, because they were added as a second check when the first one wasn't enough.

The AI also explains itself well. A recommendation with a clear, written rationale is easier to approve than an idea someone raised in a meeting, even when the rationale is built on a page that was missing the most important fact.

In Edition 1, I wrote about how hard it is for a reviewer to notice what isn't on the page. This is the same problem from the other direction. There, the AI left a requirement out of a document. Here, it takes a control out of a procedure, and the reviewer has no way of knowing the control was ever needed.

And AI does it at scale. A team that asks AI to simplify its whole SOP library could make dozens of these changes in a month, and each one would look reasonable on its own.

Where the reasons actually live

For most of these steps, the reason isn't lost. It's misplaced.

It sits in a CAPA record, a deviation report, a complaint file, an audit response or an old change control. GMP already requires procedure changes to go through review and approval. For drugs, 21 CFR 211.100(a) says written procedures, "including any changes," must be reviewed and approved by the quality unit. So the paper trail usually exists. What's missing is the link from the step back to the reason.

The rest of the reasons live in people's heads: why a limit is tighter than the compendial one, why one supplier gets an extra test, why the line is cleared twice before an allergen run. When those people retire, the reasons leave with them, and the step becomes a fence nobody can explain.

This is why I think of it as a debt. A control without a recorded reason costs nothing extra to keep. The cost comes due on the day someone, or something, decides the control isn't needed.

The debt also works against us in the other direction. Many SOPs carry steps that were added for a problem that was fixed years ago, and nobody dares to remove them because nobody knows whether they still matter. So procedures keep growing. AI could help us prune them, but only if it knows which steps to keep.

Margaret-Anne Storey has described the same pattern in software, where intent debt is the lost record of what a system's design was meant to achieve. A quality system is a set of designed controls, too, and it can lose its reasons the same way.

Paying the debt down

None of this is an argument against using AI to improve procedures. Many SOPs really are too long, and AI can help. But before AI starts editing them, I would make four changes.

1. Record the why next to the what. For every step that serves as a control, add a short reason and its source. For example: "Added after CAPA-19-014. The gasket was left off after cleaning, and seal fragments reached finished product." One line is enough. Start with the steps that came from CAPAs, because those are the most likely to look redundant.
2. Give the AI the history, not just the procedure. If you ask AI to streamline an SOP, also give it the revision history and the CAPAs, deviations and complaints behind each change. Ask it to trace every step it proposes to remove, and to mark any step it can't trace as "purpose unknown" rather than "redundant."
3. Make "why was it there?" a change-control question. Removing a step should require an answer, with evidence. If nobody can find the reason, the step stays until someone does, or until a risk assessment shows it is safe to remove. "The AI said it was redundant" is not a reason.
4. Capture the reasons while the people are still here. Ask your longest-serving people to walk through the procedures they know best and explain why each unusual step exists. Then write those reasons down next to the steps. It takes an afternoon, and it may save you an investigation later.

These four changes make the reasons visible to everyone who reads the procedure, including the AI. They also make it safe to remove the steps that really are waste.

Find the reasons before the AI finds the steps

This is the work we do at Certify Consulting Group. We help regulated manufacturers prepare their quality systems for AI, including finding the controls whose reasons were never written down, before an AI recommends removing them. If your team is already using AI to draft or revise procedures, our AI in the Quality System Review is a good place to start: a fixed-fee, two-week look at where AI is already writing, checking or deciding inside your quality system, and whether the records it touches would survive an inspection. You can book a free 30-minute call from that page, or send a note through the contact page.

Chesterton's reformer wasn't against removing fences. He only wanted someone to go and find out why a fence was there before clearing it away. With AI, that finding out has to happen first, because AI can clear a lot of road very quickly.

Think of one step in your procedures that looks redundant. Could anyone in your building tell you, with evidence, why it is there?

First published in my LinkedIn newsletter, AI in Regulated Industries.