Medtech firms struggle with AI accountability in regulatory work

by Fitri Anggraini • 12 hours ago
Medtech firms struggle with AI accountability in regulatory work

Artificial intelligence has transformed regulatory affairs in the medtech industry, but most organizations lack the governance structures to manage its associated risks effectively. Teams now deploy AI to summarize regulatory guidance, draft deficiency responses, and compare labeling versions, often without formal policies or oversight. What began as individual efficiency measures under pressure has evolved into a pervasive issue known as shadow AI, where employees use public tools to generate polished outputs without documenting the tool, prompt, or verification process.

The core issue is not AI’s capability to accelerate regulatory work but the absence of mechanisms to explain outcomes if errors occur. A deficiency response created by an AI tool may pass internal review without scrutiny, yet if mistakes emerge later, there is no record of how the output was produced, who approved it, or whether it was fact-checked. Regulatory affairs demand more than superficially plausible results—they require clear accountability. When AI influences submissions, labeling discussions, or change assessments, three fundamental questions must have answers: Which tool was used? How was the output verified? Who is responsible? Most organizations cannot provide consistent answers to all three.

AI Governance in Medtech: A Diagnostic Framework for Risk and Accountability

Current AI use in medtech regulatory affairs often lacks documented procedures, leaving organizations unable to explain how outputs were generated or verified. Public AI tools are frequently used to draft responses to regulatory inquiries or summarize guidance without established rules on input handling, output verification, or record retention. This informal approach creates risks when AI influences high-stakes tasks like deficiency responses, as there is no traceable process to confirm accuracy or accountability.

AI governance gaps revealed by maturity model

The Operational AI Capability Maturity Model provides a five-tiered assessment—Unstructured, Defined, Controlled, Governed, and Optimized—to identify where an organization’s practices stand. Unlike aspirational benchmarks, the model serves as a diagnostic to pinpoint gaps in governance. For example, a team using advanced AI tools without documented use cases or review standards remains at the Unstructured level, regardless of the tool’s sophistication. The focus is on understanding current practices and determining the most impactful next steps rather than achieving full maturity.

Gaps often appear in specific domains rather than uniformly. An organization may have secure AI tools but lack clarity on what constitutes a “reviewed output,” revealing a People domain weakness within a Technology-mature environment. Direct questions, such as whether users understand tool limitations or if reviewers follow standardized verification, expose inconsistencies more effectively than formal audits. The model’s value lies in driving actionable improvements, not in reaching an ideal state.

Assessing Organizational Readiness

A new framework called the Operational AI Capability Maturity Model for Regulatory Affairs provides a structured way to evaluate an organization’s preparedness. It outlines five maturity levels, Unstructured, Defined, Controlled, Governed, and Optimized, not as aspirational benchmarks but as diagnostic tools. The focus lies in identifying existing gaps and determining the most critical next steps, rather than aiming for the highest level of maturity.

The model assesses readiness across five domains: People, Process, Technology, Oversight, and Evidence Management. An organization might have access to a secure, enterprise-approved AI platform but still lack clarity on what constitutes “reviewed output”, a People gap within a Technology-focused environment. A single overall score would not reveal such discrepancies. Instead, targeted questions per domain expose weaknesses. For example:

  • People: Do users understand AI’s limitations? Do reviewers know how to verify AI-generated content?
  • Process: Are approved use cases documented? Is review a mandatory step or a personal choice?
  • Technology: Are tools selected specifically for regulatory use? Is access restricted?
  • Oversight: Is there a designated reviewer for high-risk AI outputs?
  • Evidence Management: Can the organization trace AI’s role in regulated decisions?

FDA warning signals medtech’s AI accountability risks

The urgency of these questions intensified in April 2026, when the FDA issued a warning letter for “Inappropriate Use of Artificial Intelligence in Pharmaceutical Manufacturing”, the agency’s first such action. The manufacturer had used AI to generate standard operating procedures, specifications, and production records without sufficient human oversight. When investigators discovered missing process validation, the company argued the AI tool had not flagged the requirement. The same principle applies to medtech: AI does not assume regulatory accountability, people do.

Regulatory guidance has long emphasized proportional risk management, human oversight, and traceability. The FDA’s draft guidance on AI in regulatory decision-making, the EU’s Artificial Intelligence Act, and NIST’s risk management framework all reinforce these principles. Despite this, many medtech firms operate without formal AI governance, treating high-risk tasks, such as drafting deficiency responses, with the same informality as low-risk activities like meeting summaries.

A structured self-assessment across the five domains can reveal where an organization stands. The objective is not perfection but identifying the most consequential next steps. Most teams discover their gaps lie in unexpected areas. The challenge is not whether AI will be used; it already is. The question is whether organizations can operationalize its use without compromising accountability.

One key test derived from risk-based thinking asks: What are the potential consequences if the AI output contains errors? What level of human review is required to mitigate that risk? Who is accountable for the final work product? If these questions lack clear answers, the organization has adopted AI but not yet established governance. The distinction lies not in the tool itself but in the organization’s maturity in managing it.

Evidence gaps leave AI-assisted work unverifiable

Evidence Management frequently exposes the most significant vulnerabilities. Regulatory affairs teams often assume that if an AI tool produces an output, the organization can later trace its use; but this assumption fails in practice. Without structured logging of prompts, tool versions, or verification steps, there is no audit trail to explain how an AI-assisted decision was reached. The FDA’s warning letter to Purolea Cosmetics Lab highlighted this exact failure: investigators found no record of human review for AI-generated standard operating procedures, leaving the company unable to demonstrate compliance.

The issue extends beyond pharmaceuticals, medtech firms using AI to compare labeling versions or draft submissions face the same risk. If an output later requires justification, the absence of evidence management means the organization cannot prove whether AI suggestions were adopted, modified, or discarded. Worse, it cannot distinguish between outputs influenced by AI and those created entirely by human effort, complicating regulatory scrutiny.

Oversight failures further undermine accountability. Even when an organization assigns a named reviewer for high-risk AI outputs, the role’s authority may not be clearly defined. A reviewer might approve an AI-drafted deficiency response without realizing they are also inheriting responsibility for its accuracy, a responsibility that regulatory guidance explicitly places on human oversight, not the tool. The EU’s Artificial Intelligence Act reinforces this principle by requiring lifecycle accountability, meaning every AI-supported decision must be tied to a person who can justify it.

Yet many medtech teams treat oversight as an afterthought, assigning review duties without linking them to documented risk levels. The consequence? A deficiency response drafted with AI assistance might pass internal review only to be flagged by a regulator questioning whether the reviewer understood the tool’s limitations, or whether the output was ever fact-checked against source documents.

LEAVE A REPLY

Your email address will not be published. Required fields are marked *