The Hidden Flaws in How We Audit AI Systems

The Auditors Have No Clothes

Here is a strange fact about the most powerful technology ever built: we have no idea how to check if it is safe.

Not for lack of trying. Governments are scrambling to set up AI oversight bodies. Companies are hiring ethics teams. A whole industry of “AI auditing” has sprung up, complete with checklists, frameworks, and certification badges. But according to a 2023 paper by Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi, published in AI and Ethics, the entire enterprise rests on a flawed assumption. The assumption is that you can audit an AI system the same way you audit a bank or a factory.

You cannot. Large language models are not like banks. They are not like factories. They are more like weather systems that keep changing the rules of meteorology.

The authors propose a three layered approach to fix this. But their paper is not really a solution. It is a confession. It tells us that the people who build these systems are still groping in the dark, and that the tools we have for holding them accountable are embarrassingly primitive.

Why a Single Audit Will Never Catch the Problem

The core problem is that an LLM is not a single thing. It is a platform that can become almost anything.

When OpenAI releases GPT-4, they are not shipping a finished product. They are shipping a bundle of capabilities that other people will reshape into thousands of different applications. One developer uses it to write legal briefs. Another uses it to generate customer service scripts. A third uses it to power a mental health chatbot. Each of these applications creates a different set of risks. A model that passes every safety test in the lab might fail catastrophically when someone deploys it in a hospital.

Mökander and his colleagues call this the “general purpose” problem. They write that existing auditing procedures “fail to address the governance challenges posed by LLMs, which display emergent capabilities and are adaptable to a wide range of downstream tasks” (Mökander et al., 2023). In plain language: you cannot audit a tool that becomes a different tool every time someone uses it.

The paper points to a deeper structural flaw. Most AI auditing today happens at one level only. Either an auditor checks the company that built the model, or they check the model itself, or they check a specific application. But none of these levels alone can catch the full picture. A company might have excellent governance policies and still release a model that generates hate speech. A model might pass every bias test and still cause harm when a third party deploys it in a high stakes setting.

The authors argue that these three levels must be connected. They call it a “three layered approach.” Governance audits check the technology provider. Model audits check the LLM after training but before release. Application audits check the specific use case. Each layer informs the others. A flaw found at the application level should trigger a re audit of the model. A governance failure should raise questions about every model the company releases.

This sounds sensible. But the paper is honest about the limits. Auditing at all three levels is expensive, slow, and requires expertise that barely exists. Most companies do not have people who can audit their own models. Most regulators do not have people who can audit the companies. And nobody has figured out how to make these audits transparent without revealing trade secrets.

The Three Layers, and Where Each One Breaks

Governance Audits: The Paper Trail Problem

A governance audit looks at the organization that builds the AI. It checks whether the company has policies for fairness, safety, and accountability. It looks at who makes decisions, how they document them, and whether there is any oversight.

This sounds like standard corporate governance. But Mökander and his colleagues identify a specific weakness. Governance audits can only verify what a company claims to be doing. They cannot verify what the company is actually doing. A firm can have a beautiful ethics policy and still ship a dangerous model because the policy was never enforced.

The authors found that governance audits work best when they are conducted by independent third parties with access to internal documents. But they also note that most AI companies are secretive by nature. The competitive pressure to release models faster than rivals creates a powerful incentive to cut corners. A governance audit that takes six months might be obsolete by the time it is published.

Model Audits: The Emergence Problem

A model audit tests the LLM itself. Researchers run the model through a battery of benchmarks. They check for bias, toxicity, factual accuracy, and safety. They try to find the model’s failure modes before it is released.

This is where things get strange. LLMs display what researchers call “emergent capabilities.” These are abilities that were not explicitly programmed and that appear only when the model reaches a certain scale. A model that cannot do arithmetic at 10 billion parameters might suddenly become competent at math at 100 billion parameters. Nobody knows exactly why this happens. And nobody can predict which capabilities will emerge next.

Mökander et al. point out that this makes model audits inherently incomplete. You can test a model on thousands of scenarios and still miss the one scenario where it does something dangerous. The paper describes this as a fundamental limitation of auditing “at all.” An audit can tell you what the model did in the past. It cannot tell you what the model will do in the future, especially when the future involves new inputs, new users, and new contexts.

The authors also highlight a practical problem. Model audits are expensive. Running a comprehensive evaluation of a large language model requires massive computational resources. Only a handful of organizations in the world can afford to do it. This creates a situation where the companies being audited are also the only ones that can afford to conduct the audits. That is not a recipe for independence.

Application Audits: The Distribution Problem

An application audit checks how the model behaves in a specific use case. This is the closest thing to what most people imagine when they think of auditing. You take the AI system, put it in a realistic environment, and see what happens.

But Mökander and his colleagues identify a critical flaw here too. Application audits can only test what has been deployed. They cannot test what might be deployed tomorrow. And because LLM based applications are updated constantly, an audit that is valid on Monday might be invalid on Wednesday.

The paper notes that application audits are also vulnerable to “distribution shift.” This is the technical term for what happens when the real world turns out to be different from the test environment. A mental health chatbot that passes every safety test in a controlled study might fail when it encounters a user in actual crisis. The model was not trained on that data. The auditor did not test for that scenario. The failure is discovered only after harm occurs.

What the Paper Does Not Prove

Mökander and his colleagues are careful to state what their three layered approach cannot do.

First, they do not claim that auditing can prevent all harm. They write that it is important to “remain realistic about what auditing can reasonably be expected to achieve.” Auditing is a detection mechanism, not a prevention mechanism. It can find problems. It cannot guarantee that no problems exist.

Second, the paper does not address the question of enforcement. An audit is only as good as the action it triggers. If a company ignores the audit findings, or if regulators lack the power to force changes, the audit is meaningless. The authors note this limitation but do not offer a solution.

Third, the paper does not solve the “black box” problem. LLMs are notoriously opaque. Even their creators cannot always explain why the model produces a particular output. This makes auditing fundamentally different from auditing a financial ledger, where every entry has a clear cause. With an LLM, the auditor is often guessing at the cause.

Fourth, the paper does not address the problem of adversarial pressure. Companies that build LLMs are under enormous market pressure to release quickly. An audit that slows down release is a competitive disadvantage. The authors acknowledge that auditing is “costly and time consuming” but do not propose a way to make it faster.

The Deeper Problem: Auditing Assumes Stability

The most troubling insight from this paper is not in the details. It is in the framing.

Auditing is a governance tool that evolved in stable systems. You audit a financial statement because the rules of accounting are fixed. You audit a factory because the production process is repeatable. You audit a bank because the regulations are clear and the risks are known.

LLMs violate every one of these conditions. The technology is changing so fast that the rules are obsolete before they are written. The risks are unknown because the capabilities are emergent. The systems are not stable because they are updated constantly.

Mökander et al. are essentially saying that we are trying to use a 19th century governance tool on a 21st century technology. The three layered approach is an attempt to patch the tool, not to replace it. But the paper never asks whether auditing is the right framework at all.

Maybe the problem is not that our audits are flawed. Maybe the problem is that we are auditing at all. Perhaps the right response to a technology that is inherently unpredictable is not to check it after the fact, but to constrain it before it is built. That would mean designing models that are inherently safer, rather than trying to catch the dangers after they appear. It would mean building in guardrails at the architecture level, not just at the deployment level.

The paper does not go there. It stays within the auditing framework. But the questions it raises point in a more radical direction.

What This Actually Means

• ▸Auditing is a lagging indicator, not a leading one. If you are waiting for an audit to tell you that a model is dangerous, you are already behind. The paper shows that audits can only detect problems that have already occurred. The real work of safety must happen during design and training, not after release.

• ▸No single audit can cover the full risk surface. The three layered approach is a recognition that risks exist at every level from the company to the model to the application. Any audit that focuses on only one level is dangerously incomplete. Policymakers should require audits at all three levels, and they should mandate that findings from one level trigger re evaluation at the others.

• ▸Independence is not optional. The paper shows that model audits are so expensive that only the companies themselves can afford them. This creates an inherent conflict of interest. The solution is not to ask companies to be more honest. The solution is to fund independent auditing capacity, either through government investment or through a mandatory industry fee.

• ▸Emergent capabilities make static testing obsolete. A model that passes every test today might fail tomorrow because of a new capability that emerged after deployment. This means that auditing must be continuous, not one time. Regulators should require ongoing monitoring, not just pre release certification.

• ▸The biggest risk is not technical failure but governance failure. The paper identifies that the weakest link in the auditing chain is not the model itself but the organizations that build and deploy it. A company with weak governance will produce unsafe models regardless of how many audits it commissions. The most effective intervention may be to focus on the governance layer first, and to treat model and application audits as secondary checks on a system that should already be safe by design.