Audit of AI agent actions: where traceability is lost
An agentic chain rarely looks like one request and one response. Even a simple task is a sequence of transitions: the model reasons, calls a tool, gets a result, adjusts the plan, and moves on.
Traceability of agent chains holds exactly as long as each such transition is recorded together with its input, output, and the decision made at that step.
A gap appears not inside the model, but at the joints. Unrecorded tool calls are the most common hole: the agent accesses an internal system, its response affects the final action, and only the text of the final message remains in the log. The line exists, but the reason for the decision can no longer be reconstructed from it.
Then retries, parallel branches, and partial failures come into play. A call went out twice, one failed by timeout, the second returned data — and without an end-to-end step identifier, these records cannot be linked. The log shows a set of disconnected events whose order and relationship have to be guessed.
The cost becomes apparent during incident investigation with an agent. Instead of opening a trace and seeing which argument was sent to the tool and what came back, the team reconstructs the picture from indirect signs: fragments of responses, metrics, interviews with witnesses.
This takes days, and the conclusions remain probabilistic — they are hard to present to the customer and the security service. Without a complete trace, the incident cannot be reproduced, which means it is also impossible to verify that the fix closes the problem.
Audit of AI agent actions is built not on the number of logs, but on end-to-end linkage of steps. We explain how this is addressed technically.
What changes after implementation: incident investigation and failures
Time to localize the cause
Previously, investigation began with reconciling scattered logs: agent steps were in one place, tool calls in another, permissions in a third. The cause surfaced over hours, and some episodes remained with an unconfirmed version.
After implementation, the log is built around a trace identifier: step, call, arguments, and response are tied to one session, so the chain is reconstructed in minutes, not over a shift.
The share of incidents with an unconfirmed cause falls to isolated cases — each has input data, the agent's decision, and the result of its verification.
The principle is the same as in service observability OpenTelemetry, only the markup is extended with agent actions: what is visible is not "something failed," but at which step and with whose permission.
Which classes of failures go away
- Repeated agent actions — idempotency keys and call deduplication: two identical charges or emails do not go through.
- Exceeding permissions — permissions are checked on every call, not once at session start; an attempt to go beyond the boundaries is recorded and blocked.
- Data leakage in a response — output checking redacts sensitive fields before sending to the user.
Session replay closes both investigation and regression: inputs, policy versions, and agent decisions are saved, so an episode can be replayed to verify that the fix removed the failure rather than masking it.
Methods of event collection: comparison of options
The completeness of the log depends on the point at which the event is intercepted.
The closer to the agent, the more details get into the record — plans, intermediate reasoning, call arguments; the farther from it, the lower the recording overhead and the lower the chance of losing an event during a failure of the agent itself. In practice, the options do not compete but complement each other.
| Option | When it fits | Limitations |
|---|---|---|
| Event collection inside the agent | Reasoning, plans, and intermediate steps are needed, not just the fact of a call | Depends on code discipline: if a call is missed, the event disappears from the log |
| Interception of tool calls at an intermediate layer | The agent accesses external services, databases, and files | Only tool calls are visible in the record; internal logic remains offscreen |
| Collection on the orchestrator side | There are multiple agents and subtasks in the process; routing matters | Lower detail; the load falls on the orchestration node |
| Stream from the model provider | The model is external; there is no access to its code | The field composition and latency are set by the provider; some details are not exposed |
The choice is dictated by the question: which events must survive a failure and subsequent investigation.
For an incident, two levels are usually taken — interception of tool calls plus the orchestrator log, while internal collection is added where the decision-making logic is needed.
Implementation order: from event schema to handover
We fix the order of work before the first line of code: first we describe what we consider an agent action, and only then set up collection. Otherwise the log comes out either excessive or fails to answer the main question — who made the call and on what basis.
- Analysis of current chains: we go through model calls, tools, and external services, and mark where the session identifier is lost and where a decision is made without a record.
- Event schema: we agree on a closed list — model request, tool call, policy rejection, state change. Each event is assigned mandatory attributes and linkage by chain identifier.
- Log field dictionary: types, requiredness, time format, and personal data masking rules — before records begin to accumulate.
- Event collection setup: we connect sources, buffering in case the storage is unavailable, and protection against record loss during agent failure.
- Access control: roles for reading, export, and administration; access to the log itself is also an event.
- Testing on a test environment: successful call, rejection, connection loss, repeated delivery — we verify record completeness and chain reconstruction by identifier.
- Acceptance of the audit system and handover: we review a sample of real chains, provide documentation, and train the on-call shift to investigate incidents.
The event schema and field dictionary remain with you as a working document: agents are changed and response rules are configured according to them, without contacting us for every revision.
Deliverable package: schemas, policies, regulations
After launch, the customer retains a working package according to which the log lives without us: it can be extended, cleaned, and handed over to new engineers. Each artifact is tied to a specific operation — adding an event, investigating an incident, granting access.
- Event flow schema — the path from the agent calling a tool to the record in storage: interception points, buffer, retries, behavior when the receiver is unavailable. It shows which agent actions do not get into the log.
- Field dictionary — the meaning of each field, requiredness, allowable formats. Without it, incident investigation turns into guesswork about what exactly the agent recorded.
- Log access policies — who reads records and under what conditions, retention period, prohibition of editing. The log is append-only: changing a past record is technically impossible, not "according to instructions."
- Roles and read permissions — separate sets for the on-call shift, incident investigation, audit, and security service: each sees its own slice; full access is not granted by default.
- Data masking rules — what is cut before recording: personal data, access keys, contents of requests. Masking is at the input, so sensitive values do not get into storage.
- Incident investigation regulations — event classification, response time, who decides to stop the agent, how conclusions are recorded. It includes instructions for the on-call shift: what to look at and what to collect before the investigator joins.
We hand over documentation to your repository and support the first changes until the team starts editing it itself.
Case study: incident investigation before and after implementing the log
The trigger was an incident with a support agent: connected to an internal system through tools, it changed statuses and amounts for some orders outside the scenario. We learned about it from customers — the changes had already gone to billing.
The investigation proceeded without a log: the application logs had method calls, but no agent identifier, no instruction version, and no linkage of the call to a session.
Three weeks of manual correlation yielded attribution for less than half of the actions; the incident boundaries remained blurred, and rollback had to be done across the entire database — along with legitimate changes.
What was changed
An immutable log was introduced: records are only added; editing and deletion are unavailable. For each tool call, the agent and its instruction version, input and output data, decision rationale, session identifiers, and parent task are written.
Records are brought to the OpenTelemetry trace format and store the hash of the previous one — omission or substitution is detected, and only the agent runtime can write.
Metrics before and after
- Episode investigation: three weeks of manual search → sample by agent, session, and time interval in hours.
- Action attribution: confirmed for less than half of calls → 100%, with instruction version and initiator at each step.
- Rollback: full database restore → targeted rollback of only affected records.
"A similar case was investigated in a day and the root cause was found — overlapping permissions of two tools, not a model error," — practice lead.
What to do if the system is already running in production?
There is no need to stop agents: event collection is built as an add-on over the points where the agent calls tools and external services.
The agent code is not rewritten — the interceptor records the input and output data of the call and sends the event in a separate stream so that response time does not grow.
Phasing here is not a compromise, but a way to reduce risk. First, the riskiest scenarios come under observation — those that change data, write to external systems, or delete records.
Such actions are irreversible, and they are exactly what people most often ask about: "who did this and on what basis?" The rest are connected in subsequent waves, once the first events have been verified for completeness.
Each wave is configuration, not a new service build. The interceptor is enabled by a flag for a group of scenarios, rules mask personal data at the input, and events go to a queue and then to storage.
If storage is unavailable, they accumulate in a local buffer and are sent after connectivity is restored: a brief failure should not cost the history of actions.
Prioritization is built on two axes — the probability of an error in a scenario and the cost of its consequences. Irreversible actions and external integrations come first; dialog and reference ones come last.
The effect is visible from the first wave: the audit log covers exactly the paths where incident investigation was previously impossible.
Rollback is also provided: removing the flag returns the system to its original behavior, while already collected events remain available.
If your agents are already in operation — tell us which scenarios raise the most questions, and we will return with a phased implementation plan for your environment.
Doubts about the audit log: addressing point by point
Objections to an audit log are usually the same, and they are fair: a record is easy to make incomplete, easy to rewrite, easy to turn into a dump of personal data. Below is how we address the four most common doubts at the level of storage schema and recording rules.
The agent will call the model directly, bypassing the common gateway — will the record be lost?
We collect events from two layers: interception at the transport level and handlers inside the agent process, where tool calls and accesses to external services are recorded.
If an event passed only one path, the record retains a mark of chain incompleteness, and the local buffer sends the missed one at the first opportunity. The gap is visible in the report rather than masked.
Records will contain personal data — what should be done about it?
By default, the log stores the subject identifier and call metadata, not the body of the request and response.
Where the full text is genuinely needed for investigation, we enable field masking and pseudonymization — matching to identity lives separately and is available to a narrow set of roles.
How long should records be stored so the log does not become dead weight?
The period is assigned by event class, not as a single number: operational records live briefly, access and significant decisions live longer. Then the lifecycle rule applies: expired data is exported to cold storage, and later deleted along with keys.
Records can be rewritten retroactively — how can this be excluded?
Each record receives the fingerprint of the previous one and the signature of the node that created it, so an edit in the middle breaks the entire subsequent chain.
We periodically fix the root fingerprint externally, and keep the storage itself in append-only mode — past data cannot be deleted or substituted without leaving a trace.
Let's discuss your task and return with a plan
To move from conversation to a plan, a description of the environment is enough: which agents operate, who initiates calls, which services requests pass through, where storage is located, and who is responsible for infrastructure.
This is enough to break the task into stages and return with an estimate of timelines and scope of work — without a long pre-project cycle.
- Task discussion: we review agent scenarios, who makes decisions, what is considered a sensitive action, and what exactly should go into the audit of AI agent actions.
- Collection schema: we determine interception points — gateway before models, tool calls, database accesses — and the event format: who, what, when, with what context and result.
- Phased implementation plan: piloting on one agent, then scaling, setting up storage and retention periods, connecting search and exports.
- Audit scope of work: list of integrations, event normalization rules, access control to logs, protection against record substitution.
- Acceptance criteria: share of agent actions covered by a record, completeness of context for each step, incident search time, reconstruction of the decision chain from request to result.
- What to send: list of services and agents, example of a typical request and response, regulator or internal service requirements, retention period constraints.
- Next steps: we agree on boundaries, fix the phased plan and acceptance criteria, and then start with the pilot.
Describe the task in this form — we will return with an estimate, phased plan, and scope of work.
