Auditing AI agent actions: what regulators are starting to ask for

For the past year, AI has been mostly a product question for compliance teams: "Are we training on customer data?" "Do we have a data processing agreement with our AI provider?" "What's our retention policy for AI-generated outputs?"

That's changing. As more companies deploy AI agents — not just AI assistants, but agents that take actions in real systems — regulators and auditors are starting to ask a different and harder set of questions. Questions that most teams haven't prepared for.

The questions auditors are starting to ask

We've talked to dozens of security engineers, compliance leads, and GRC teams at fintech and healthtech companies. Here are the questions they're being asked:

"What did this agent have access to?"

SOC 2 CC6.3 requires that access to information assets is restricted based on roles and least-privilege principles. Auditors are extending this to AI agents. "What was this agent's permission set? Who authorized it? When was it last reviewed?"

Most teams answer this by describing their IAM configuration — but as we explored in our piece on agent permissions and IAM, traditional IAM doesn't capture the full picture of agent access. Auditors are starting to notice.

"What did this agent do?"

This is the most basic audit question, and many teams can't answer it precisely. They have application logs, database write logs, API gateway logs — but not a unified view of "this agent session, in this time window, took these actions."

Reconstructing agent behavior from fragmented logs is painful, time-consuming, and often incomplete. The absence of a unified agent action log is increasingly a finding in SOC 2 reviews.

"How do you know the agent didn't access data outside its intended scope?"

This is the question that stumps most teams. The honest answer is often "we're relying on our prompt to constrain the agent's behavior" — which is not a satisfying answer to an auditor who has read about prompt injection.

The better answer: "The agent ran inside an enclosure with explicit allowlist-only access, so it was architecturally incapable of accessing resources outside its manifest." That answer requires infrastructure, not just documentation.

"Can you show us the agent's decision trail for this transaction?"

This is emerging particularly in financial services and healthcare, where transaction decisions need to be explainable. "An AI agent made this recommendation" is increasingly not sufficient. Auditors want to see: what inputs did the agent receive, what reasoning did it follow, what actions did it take, and what was the output?

This requires capturing not just the final action but the full execution trace — tool calls, intermediate outputs, decision context.

What a defensible agent audit log looks like

Based on the questions above, here's what an agent audit log needs to contain to be useful in a compliance review:

Session-level metadata

• Agent identifier and version
• Enclosure or permission manifest identifier and version
• Initiating user or system
• Start time, end time, termination reason
• Total actions taken, total cost incurred

Action-level records

• Timestamp and sequence number for each action
• Tool name and input parameters
• Output or error received
• Whether the action was within permitted scope (and if not, how it was blocked)

Integrity controls

• Tamper-evident logging (hash chaining or equivalent)
• Immutability — the agent itself should not be able to modify its own logs
• Sufficient retention period for your compliance requirements (typically 1 year for SOC 2, longer for HIPAA depending on context)

The difference between logging and auditing

A log is a record of what happened. An audit trail is a record that can be relied upon in a compliance review or legal proceeding. The difference matters in practice.

Audit trails need to be:

• Complete: No gaps. Every action recorded.
• Tamper-evident: You can demonstrate the log hasn't been modified after the fact.
• Attributable: Each entry is tied to a specific agent session and context.
• Accessible: Reviewable in a reasonable time, including for historical queries.

Application logs typically fail on completeness and tamper-evidence. Scattered API logs fail on attribution. Building a proper audit trail for agent actions requires thinking about these properties explicitly.

HIPAA-specific considerations

For teams operating in healthcare, HIPAA's audit control requirements (§164.312(b)) require "hardware, software, and/or procedural mechanisms that record and examine activity in information systems that contain or use electronic protected health information."

If AI agents access or process ePHI — which they do in any clinical or administrative workflow — they fall squarely under this requirement. The audit controls need to cover the agent's activity, not just the underlying data systems. This is a gap most healthcare AI teams haven't closed.

Building for auditability before you need it

The cost of retrofitting audit infrastructure into a deployed agent system is high. Access logs are scattered, integrity can't be proven retroactively, and session-level attribution requires reconstructing history you didn't capture.

The right time to build audit infrastructure is before your first production deployment — when you can design the logging and permission model together, rather than layering audit on top of a system that wasn't designed for it.

Agent Enclosure captures a complete, tamper-evident execution trace for every agent session by default. Your audit trail is there when you need it — whether for an internal review, a customer security questionnaire, or a formal compliance audit.