AI Auditing: How to Audit AI Systems in 7 Steps

What AI auditing covers

The word “system” matters. An AI application includes data, models, prompts, retrieval, tools, infrastructure, people, policies, vendors, and monitoring. A model can pass a benchmark while the deployed product still leaks data, discriminates in practice, takes unauthorized actions, or fails without a safe handoff.

This guide explains how to scope an AI audit, what to test, what evidence to retain, and how to turn a periodic review into continuous assurance. It also includes a dedicated checklist for agentic and voice AI, where a model can affect real systems during a live interaction.

What is AI auditing?

AI auditing is the process of comparing an AI system and its controls with explicit audit criteria, collecting sufficient evidence, testing whether the criteria are met, and reporting exceptions. Those criteria may come from laws, contracts, standards, internal policies, risk tolerances, or documented product requirements.

An audit should answer four questions:

1. Is the system permitted and properly governed?
2. Does it perform as intended for the people and conditions in scope?
3. Are security, privacy, human oversight, and operational controls effective?
4. Can the organization prove those conclusions with reproducible evidence?

AI auditing is not the same as using AI in auditing. The former audits an AI system. The latter uses AI tools to help an auditor analyze documents, transactions, or controls. A team may do both, but each has a different objective and risk model.

AI audit vs. assessment, validation, and red teaming

These activities complement one another, but they are not interchangeable:

An audit can use results from the other activities as evidence. It should still verify the test design, sample, version, environment, and independence of the tester before relying on them.

What should an AI audit cover?

A defensible AI audit follows the deployed decision or action from input to outcome. Its system boundary should include eight areas.

1. Governance and accountability

Confirm that the use case has an owner, an approved purpose, accountable decision-makers, defined risk acceptance, and a path for escalation. Review whether policies cover acquisition, development, deployment, monitoring, incident response, and retirement.

The audit function should not audit its own design work. Builders and operators can test controls; risk and compliance teams can challenge them; internal or external audit can provide independent assurance. The IIA’s AI Auditing Framework applies this governance and assurance lens to AI-related risks and controls.

2. Intended use and human impact

Document who is affected, which decisions the system informs or makes, the consequence of error, and where a human can intervene. Test the real workflow rather than an abstract model. A résumé-ranking system, clinical documentation assistant, and appointment-scheduling voice agent can use the same foundation model while presenting very different risks.

3. Data and knowledge sources

Trace training, tuning, evaluation, retrieval, and production data. Check provenance, permission, quality, representativeness, labeling, retention, access, deletion, and leakage between users or tenants. For retrieval-augmented generation, include the ingestion pipeline, document permissions, freshness, and citation behavior.

4. Model and system performance

Test the metrics that matter to the intended use: accuracy, error rates, false positives and negatives, robustness, calibration, fairness, explainability, latency, availability, and cost. Break aggregate results down by meaningful populations, languages, channels, and operating conditions. A single average can hide a severe failure mode.

5. Security and privacy

Review identities, credentials, network paths, data flows, encryption, logging, vendor access, prompt injection, model extraction, training-data exposure, denial of service, and incident response. For agents, test every path from untrusted content to a tool or side effect. The OWASP Top 10 for Agentic Applications is a useful threat taxonomy, not a replacement for an end-to-end audit.

6. Application and agent controls

Prompts and guardrails influence model behavior, but they are not authorization controls. Verify that deterministic services enforce tool permissions, tenant boundaries, argument validation, transaction limits, approvals, idempotency, and safe retries. Treat a model’s requested action as a proposal that a trusted policy layer may allow or deny.

7. Deployment and change management

Identify the exact model, prompt, knowledge, code, policy, and infrastructure versions under review. Check release approvals, separation of environments, rollback, vendor-change handling, regression tests, and configuration drift. An audit conclusion about one version should not silently carry over to a materially changed system.

8. Monitoring, incidents, and retirement

Verify that production metrics detect the failures identified during risk assessment. Review alert thresholds, investigation records, user complaints, overrides, incident response, kill switches, remediation, and decommissioning. Logs should support reconstruction without retaining sensitive data longer than necessary.

How to conduct an AI audit in 7 steps

The following process works for an internal audit, vendor review, pre-release assurance engagement, or focused algorithmic audit. The depth should scale with impact and uncertainty.

1. Define the audit objective, criteria, and independence

Start with a precise audit question. “Is our AI compliant?” is too broad. Better objectives include:

• determine whether the customer-service agent restricts account changes to authenticated callers;
• assess whether a hiring model meets the organization’s documented fairness thresholds;
• verify that all production AI systems have owners, risk classifications, and monitoring; or
• evaluate readiness against selected ISO/IEC 42001 requirements.

Write down the criteria before testing. Record the laws, contract clauses, standards, policies, risk tolerances, and system requirements that will be used. Define who performs the work, who owns the system, and which conflicts could impair independence.

Also set materiality and severity rules. A 1% error rate can be tolerable for low-impact text suggestions and unacceptable for a high-impact automated decision.

2. Inventory the system and draw its boundary

Create a versioned system record that lists:

• business purpose, owner, users, affected parties, and jurisdictions;
• model and provider, including fallback models;
• prompts, policies, fine-tuning, evaluation sets, and guardrails;
• training, retrieval, session, and analytics data;
• applications, APIs, tools, agents, queues, and human handoffs;
• hosting, telephony or channel providers, vendors, and subprocessors; and
• logs, monitors, credentials, retention rules, and shutdown controls.

Then diagram inputs, transformations, decisions, actions, outputs, and data stores. Mark trust boundaries and every place where data or authority changes. Shadow AI discovered during this step belongs in the inventory even if it was not formally approved.

3. Classify risk and select the control baseline

Classify the use case by impact, autonomy, data sensitivity, scale, reversibility, exposure to attackers, and applicable regulation. Do not infer risk from the model name alone.

Use the classification to choose a control baseline and test depth. The NIST AI Risk Management Framework is a voluntary, cross-sector starting point organized around Govern, Map, Measure, and Manage. NIST’s Generative AI Profile extends that approach for generative AI risks.

Regulation can add mandatory criteria. The EU AI Act, for example, uses a risk-based structure and assigns different duties to providers and deployers. As of September 2026, the European Commission states that transparency rules took effect in August 2026 while its main high-risk-system obligations are scheduled to apply from December 2, 2027. Check the Commission’s current AI Act guidance and obtain legal advice for the specific role, system, and jurisdiction; a framework checklist is not a legal determination.

4. Build the evidence request and test plan

For each criterion, specify the control, expected evidence, test method, sample, and pass threshold. This turns an AI audit checklist into a reproducible plan.

Evidence should identify the system version, environment, time period, owner, and source. Screenshots can support a finding, but structured exports, signed configurations, test code, and immutable logs are stronger.

5. Execute tests across normal, edge, and adversarial conditions

Test effectiveness, not just control existence. A policy document is not evidence that production access is restricted.

Use a mix of:

• document and configuration inspection;
• interviews and walkthroughs;
• reperformance of controls;
• code, prompt, and architecture review;
• statistical and subgroup testing;
• scenario simulations and red teaming;
• production-log and incident sampling; and
• end-to-end tests in a controlled environment.

Cover expected traffic, rare but plausible inputs, and deliberate attacks. Record seeds, model settings, prompts, tool responses, timestamps, and repetitions where model behavior is nondeterministic. Separate a one-off failure from a measured failure rate, but do not dismiss a low-frequency event with catastrophic impact.

6. Rate findings and agree on corrective actions

Each finding should connect evidence to a criterion and business consequence. A useful format is:

• condition: what the auditor observed;
• criterion: what should have happened;
• cause: why the gap exists;
• risk: what could happen and to whom;
• evidence: affected versions, samples, and test results;
• severity: likelihood, impact, detectability, and scale;
• action: a specific control change;
• owner and date: who will fix it and by when; and
• validation: how closure will be retested.

Do not reduce the report to a vague “responsible AI score.” Executives may need a summary, but control owners need the exact failing path and acceptance threshold.

7. Retest, monitor, and define re-audit triggers

Retest the implemented correction rather than accepting a plan or screenshot. Record residual risk and the person authorized to accept it.

Then define monitoring and re-audit triggers, such as:

• a new model, provider, prompt architecture, tool, data source, or jurisdiction;
• a material change in traffic, affected population, or autonomy;
• performance or fairness moving outside approved limits;
• a security, privacy, safety, or rights-related incident;
• repeated complaints or human overrides; or
• a new law, contract, or control requirement.

High-impact and fast-changing systems need continuous control monitoring plus periodic independent review. “Audited last year” says little about a system whose model and tools changed last week.

AI auditing frameworks: which one should you use?

There is no universal framework that makes every AI system safe or compliant. Most organizations combine a general risk framework, an auditable management system, applicable legal criteria, and technical test standards.

Start with the criteria that are actually binding, then use frameworks to close gaps and organize evidence. Mapping the same control once across multiple frameworks is more efficient than running separate checkbox exercises.

AI audit checklist

Use this checklist to plan a first pass. Replace each yes-or-no item with an owner, evidence reference, test, result, and remediation status.

Governance and scope

• Is every production AI system, model, agent, and material vendor recorded in an inventory?
• Does each use have an approved purpose, owner, risk tier, affected population, and jurisdiction?
• Are provider, deployer, operator, reviewer, and auditor responsibilities explicit?
• Are prohibited uses and risk-acceptance authorities documented?
• Is the reviewer sufficiently independent from the control being audited?

Data, privacy, and knowledge

• Can the team trace data provenance, rights, consent or other lawful basis, transformations, and retention?
• Do quality and representativeness tests match the intended population and conditions?
• Are user, tenant, and document permissions enforced before retrieval?
• Can records be corrected or deleted through downstream stores and indexes?
• Are sensitive fields minimized and excluded from prompts, logs, and training where required?

Performance and human impact

• Are success, failure, fairness, robustness, latency, and availability thresholds defined before release?
• Do evaluation cases represent normal, edge, adversarial, multilingual, and accessibility conditions?
• Are results segmented so aggregate scores do not hide group or channel failures?
• Are limitations communicated to users and reviewers?
• Can a human understand, override, stop, and appeal consequential outcomes?

Security and agent actions

• Are model inputs and retrieved content treated as untrusted?
• Are identities, permissions, and approvals enforced outside prompts?
• Does each agent receive only the tools and credentials needed for the current task?
• Are tool arguments validated against schemas, business rules, and resource ownership?
• Are high-impact actions bound to exact approvals, protected from replay, and safely reversible?
• Are prompt injection, data exfiltration, memory poisoning, cross-tenant access, and resource exhaustion tested?

Operations and evidence

• Can the organization reproduce which model, prompt, data, code, tools, and policies handled an event?
• Do regression tests and approvals gate material changes?
• Are production metrics tied to known risks and approved thresholds?
• Are incidents, complaints, overrides, and near misses investigated and fed back into tests?
• Do rate limits, circuit breakers, rollback, credential revocation, and kill switches work?
• Are re-audit triggers and finding-retention periods defined?

How to audit agentic and voice AI

Agentic systems need extra scrutiny because their output can become an action. Voice adds live identity, channel, latency, disclosure, recording, and handoff concerns.

For a production voice agent, include the speech-to-text provider, model, prompt and conversation flow, knowledge sources, text-to-speech provider, telephony, APIs, transfers, analytics, and human operators inside the audit boundary.

Then test complete conversations rather than isolated responses:

1. Identity: Can a caller obtain or change protected information by spoofing caller ID, using details found online, or persuading the model? Caller ID is routing data, not authentication.
2. Disclosure and consent: Does the experience make required AI, recording, and data-use disclosures for the applicable jurisdiction and workflow?
3. Conversation performance: Does the agent handle interruptions, silence, background noise, accents, language changes, ambiguous dates, and poor connections without inventing a successful outcome?
4. Tool execution: Are account lookups and state changes authorized, validated, idempotent, and reconciled against the system of record?
5. Knowledge and privacy: Can one caller retrieve another customer’s data, hidden instructions, credentials, or restricted documents?
6. Handoff: When confidence, identity, policy, or system health falls below a threshold, does the agent transfer or stop with the context a human needs?
7. Evidence: Can a reviewer connect audio or transcript, model turns, tool calls, policy decisions, latency, transfer, and final business state under one trace ID?

Our AI agent security guide provides a deeper control architecture for identity, tools, memory, approval, and audit trails. The main rule is simple: a model may propose an action; a deterministic service must decide whether that action is allowed.

Can AI auditing be automated?

Parts of an AI audit can and should be automated:

• inventory and configuration collection;
• evaluation and red-team suites;
• data-lineage and access checks;
• policy-as-code tests;
• model, prompt, and dependency change detection;
• log sampling and anomaly detection; and
• finding workflow and evidence retention.

Automation improves coverage and repeatability. It does not choose the organization’s risk tolerance, resolve ambiguous legal duties, interview affected people, test whether human oversight works in practice, or provide independent judgment by itself.

Be especially careful with “AI auditing AI.” A model can generate test cases, classify logs, and flag unusual behavior, but its outputs need validation, access controls, versioning, and an audit trail. An AI auditor is another AI system inside the broader control environment—not an automatic source of assurance.

What should an AI audit report contain?

A publishable or internal report should make its conclusion hard to misread. Include:

• objective, criteria, system boundary, versions, period, and locations;
• auditor role, independence, methods, sample sizes, and limitations;
• executive conclusion and residual-risk statement;
• test inventory with evidence references and reproducibility details;
• findings with severity, affected parties, cause, owner, action, and due date;
• management responses and formal risk acceptances;
• retest results and unresolved exceptions; and
• distribution, retention, and next-review triggers.

State what the audit does not prove. A limited security review is not a fairness audit. A management-system certification is not a guarantee that every output is safe. A clean result for one model version is not assurance over the next release.

Build auditability into the AI system

The cheapest audit evidence is evidence the system already produces. Give every model, prompt, workflow, tool, policy, and deployment a version. Correlate inputs, outputs, tool proposals, authorization decisions, side effects, transfers, and final outcomes. Keep evaluation cases alongside release gates. Make sensitive-data retention an explicit choice rather than a logging default.

For voice AI, Dasha’s managed production platform gives technical teams a runtime for conversations, tools, telephony, testing, and production operations. Your application still owns identity, permissions, policy, and the acceptance criteria for its use case. That separation is useful: the agent handles the conversation, while deterministic services control what it may do.

An AI audit should not end with a badge or a score. It should leave the organization able to show which system was tested, against what criteria, with what evidence, what failed, who owns the fix, and what will trigger the next review.

Frequently asked questions

How often should an AI system be audited?

Set frequency by impact and rate of change. High-impact systems may need continuous control monitoring, pre-release testing for every material change, and periodic independent audits. Low-impact, stable systems may justify a longer cycle. Model, data, tool, purpose, incident, and regulatory changes should trigger review outside the calendar.

Who performs an AI audit?

An AI audit is usually cross-functional. Auditors provide independence and evidence discipline; engineering and data teams explain and reproduce the system; security tests threats; privacy and legal teams interpret obligations; domain specialists assess real-world impact; and business owners accept residual risk. No single role covers the full system.

Is there an AI auditing certification?

There is no single credential that authorizes every kind of AI audit. Professional bodies offer AI, risk, security, and internal-audit credentials, while ISO/IEC 42001 certification applies to an organization’s AI management system rather than certifying every model output. Required qualifications depend on the engagement, industry, jurisdiction, and assurance claim.

What is an algorithmic audit?

An algorithmic audit evaluates an automated decision system against criteria such as performance, fairness, transparency, privacy, security, or legal requirements. “AI audit” is often used more broadly to include governance, vendors, infrastructure, human workflows, deployment, and monitoring around the algorithm.

What are AI auditing tools?

AI auditing tools include model-evaluation platforms, fairness and explainability libraries, red-team harnesses, data-lineage systems, security scanners, governance inventories, policy engines, observability tools, and evidence-management systems. Select tools from the audit criteria and architecture; no single product supplies complete assurance.