They Added a New AI Model in 2024. FDA Found It in Their Release Notes in 2026.

In March, FDA investigators inspected O.N. Diagnostics, a small software company in Berkeley, California. Its product, VirtuOst VFA, reads spine CT scans to find vertebral fractures and flag patients at high risk of a future osteoporosis-related fracture. In the company's own software version log and release notes, dated June 2024, the investigators found that version 3.0.0 had introduced a new machine learning model. FDA's warning letter, dated September 1, says that change needed a new 510(k), and it asked the company to stop distributing that version.

The change wasn't hidden. It was written down in the place where software teams write down improvements, and it went out the way improvements go out, in a new version number. What the company didn't do was treat it as the kind of change that needed FDA's review. To FDA, a new model meant the software had gone beyond the device it had cleared.

Most of my readers don't make medical device software, but many of you already use AI inside your quality system to draft SOPs, summarize complaints, sort deviations or check batch records. The models under those tools change too, often without anyone in Quality being told. If a new model makes a device a different device, I think it makes your AI-assisted process a different process, and the question is whether your change control would notice.

Here's what's inside:

• What FDA actually cited. The entry in the release notes, and the sentence in the letter that sets the rule for AI models.
• Why a new model is a new process. The screens and the outputs can look the same while the judgment underneath them has changed.
• Your AI can change even if you never touch it. Vendors update the models behind the tools your team uses, and change control usually doesn't hear about it.
• Four questions for change control. What to settle before a new model goes into a validated process.

What FDA actually cited

VirtuOst VFA was cleared under K171435 to measure vertebral deformities on CT images and classify any fractures it finds, so that a physician can diagnose them and manage the patient's osteoporosis. Version 3.0.0 introduced a new machine learning model for vertebral landmarking, the step that locates key points on each vertebra so the software can measure it.

FDA didn't say the new model was worse. It said the model changed the device enough that FDA had to see the evidence before it went out. The letter puts the rule in two sentences: "A new model trained on different data, with different architecture, or optimized for a different objective may perform differently across patient populations, imaging equipment types, or clinical settings. This new feature fundamentally changes how the device performs, fails, and affects patients in ways that go beyond what was originally cleared and therefore the feature must be adequately validated."

The same version also moved the software from a desktop application to the web and onto a new technology platform. FDA said each of those changes needed a new 510(k) on its own, and it declared version 3.0.0 adulterated and misbranded.

One line in the letter will sound familiar to anyone who read my first edition. Among the harms of an inadequately validated model, FDA listed "overreliance on device output for radiological image interpretation." That is automation bias, the tendency to over-rely on what an automated system tells us, and FDA has now named it as a patient risk in a warning letter.

The letter also found the same habit somewhere that had nothing to do with AI. The company had narrowed the triggers in its CAPA procedure, and FDA wrote that "the impact of these changes was not evaluated prior to implementation." Then it added: "To maintain the integrity of the QMS, all changes must be evaluated to determine their potential impact on it. This evaluation must be conducted before implementing the change." That sentence was written about a CAPA procedure, but in my view it's the whole letter in two lines.

Why a new model is a new process

When a software team fixes a bug, the change is in code someone can read, and they can test it against the requirement it was supposed to meet. A new AI model is a different kind of change. It's the part of the system that makes the judgment, and a new one can be better on average and worse on exactly the cases you care about, like an older scanner or a patient population it saw less of in training. Nothing on the screen tells you which.

Here's a fictional example. A dietary supplement company uses an AI tool to sort incoming customer complaints and flag the ones that might be adverse events. That matters, because a serious adverse event has to be reported to FDA within 15 business days of receiving it. Last year the QA manager validated the tool against 200 past complaints with known outcomes, and it flagged every one that mattered. She signed the validation report, and the tool went into the complaint procedure.

In the spring, the vendor moves the tool to a newer model. The summaries read better, the screens look the same, and nobody at the company is told. The new model is a little more likely to read a vague complaint as product dissatisfaction, so a customer who wrote that the capsules "made my heart race" gets sorted as a quality complaint instead of a possible adverse event.

Nothing in the company's change control records shows that anything happened, because the company didn't change anything. The QA manager's validation report is still accurate about a model that the company no longer uses.

Your AI can change even if you never touch it

O.N. Diagnostics at least made its change itself and wrote it down. Many quality teams using AI tools don't control the model at all. The vendor does, and vendors update and retire models on their own schedule. The model your team validated in the spring may not be the model answering today.

GMP already has the rules for this. They just weren't written with AI in mind. For drugs, 21 CFR 211.68 expects computerized systems to be checked "according to a written program designed to assure proper performance," and 211.100(a) puts procedure changes through the quality unit. For devices, FDA's quality regulation now incorporates ISO 13485, which requires software used in the quality system to be validated before first use "and, as appropriate, after changes to such software or its application." Where it applies, it also expects a written agreement that suppliers will tell you before they make changes that affect what you bought from them. An AI vendor is a supplier, and the same letter cited O.N. Diagnostics for vague supplier monitoring, too.

FDA has also given device makers a way to plan for model changes. Under a predetermined change control plan, described in FDA's final guidance from December 2024, a manufacturer can get planned model updates authorized in advance, along with the methods it will use to make and validate them. Whether or not you make devices, I think that's the right idea: decide before the model changes how you'll prove it still works.

Four questions for change control

None of this is an argument for keeping AI out of the quality system. It's an argument for putting AI under the change control you already have. Before a new model goes into a validated process, or before you find out one already has, I would want clear answers to four questions.

1. Which model are we using today? Record the model and its version for every AI tool in a GMP process, the same way you record the software version of any other computerized system. If a vendor can't tell you, that's worth knowing before an investigator asks.
2. How will we find out when it changes? Put change notification in your supplier agreement where you can. Where you can't, check on a schedule, because a vendor's release notes are no help if nobody in Quality reads them.
3. What will we re-run to show it still works for our use? Keep a small set of real cases with known answers from the original validation, including the hard ones, and run it again on every new model. If the results differ, someone in Quality looks at why before the new model goes back into use.
4. Who decides whether the change matters? That decision belongs in change control, with the quality unit, and not with IT or the vendor. An entry in the release notes records that something changed, but it doesn't record that anyone decided whether the change mattered.

These questions don't slow AI down very much. Most of the work is done once, when the tool is first validated, and after that a model change takes an afternoon to check rather than an inspection to discover.

This is the work we do at Certify Consulting Group. We help regulated manufacturers bring the AI in their quality systems under change control: knowing which models are in use, hearing when they change, and having the evidence ready to show the process still works. If your team uses AI for complaints, CAPAs, document drafting or batch record review, our AI in the Quality System Review is a good place to start: a fixed-fee, two-week look at where AI is already writing, checking or deciding inside your quality system, and whether the records it touches would survive an inspection. You can book a free 30-minute call from that page, or send a note through the contact page.

According to a company statement to AuntMinnie, O.N. Diagnostics is now validating a revised version that "removes the machine-learning functionality identified in the warning letter." That's one way back into compliance, and I suspect other companies will choose it too. I would rather we build change control that can handle the AI, so that taking it out isn't the only option left.

Has an AI tool your team relies on ever changed without anyone telling Quality? How did you find out?

First published in my LinkedIn newsletter, AI in Regulated Industries.