← Back to the newsletter

NewsletterGovernance and security

AI agent audit: what should be recorded in every conversation

November 10, 2026 7 min read

An AI agent audit has to be able to rebuild every conversation in full: what the customer said and what the agent answered, which agent version was live, which systems it queried, which guardrails fired, when a person stepped in and who accessed the record afterwards. If any of those pieces is missing, the conversation cannot be audited.

In a Risk review of a conversational agent, the first question is almost always the same: where is the record of each conversation kept, and who can read it. It is question number one on the due diligence checklist that Security teams use, and one of the questions that stalls the most projects, because many vendors answer with a transcript and little else.

This article covers what should be logged in every interaction of an AI agent at a bank, fintech or insurer, how long to keep each record under the rules that apply in Latin America, and how to check it before signing with a vendor.

Why a transcript alone falls short

A transcript tells you what was said. The auditor usually shows up months later with a specific case: a customer files a complaint with CONDUSEF in Mexico or Procon in Brazil saying the agent offered a payment plan that did not exist, or shared their balance with someone else. To answer, the team has to show which instructions the agent had that day, which data it pulled from the bank’s systems, whether a security filter stepped in and who has accessed that conversation since.

This matters more with an AI agent than with a human operator. Its behavior depends on the prompt, the model, the connected tools and the guardrails, and any of those can change from one week to the next. If the record does not keep the exact version that handled the call, the conversation loses its context.

The eight elements every conversation record needs

1. Conversation identification

A unique ID, the channel (voice, WhatsApp or web), start and end date and time, the originating number or account and the portfolio it belongs to. This is what lets you find the right conversation when a regulator request or a complaint arrives.

2. Transcript and recording

The full text, turn by turn, with a timestamp on each one. For voice, the audio recording as well, because tone and interruptions also count as evidence in a dispute.

3. Agent version that handled it

The published version number, which points to the prompt, model, tools and guardrails in force at that moment. If a change caused a problem, the version tells you since when and how many conversations were affected.

4. Tools and systems queried

Every time the agent checks a balance, records a promise to pay or updates a CRM, the record should show which tool it used, in which turn, with which parameters and what response came back. That is how you tell an agent error apart from bad data in the source system.

5. Guardrails triggered

Which filter stepped in (prompt injection, an attempt to extract personal data, a crisis signal, an off-topic request) and what the agent did next: replied with the defined message, handed off to a person or ended the conversation. For Risk, this record is the evidence that controls work in production.

6. Personal data handling

Which sensitive data came up in the conversation (CPF, CURP, account number, card) and how it was stored: masked or tokenized in the logs, with access to the full value limited to authorized roles. Brazil’s LGPD and Mexico’s personal data protection law both require being able to demonstrate that handling.

7. Human handoff and outcome

If the conversation went to an operator, when it happened, why and to whom. Also the outcome, with the data the agent extracted (a payment agreement, an amount, a date) and a closing summary. That way Operations and Audit work from the same version of events.

8. Access to the record

Who opened, exported or downloaded each conversation, and who changed the agent configuration or published a new version. An archive of conversations that anyone can read without leaving a trace becomes a privacy risk for the institution itself.

What the auditor asks and which record answers it

Auditor’s question Record that answers it
What did the agent tell this customer on March 14? Identification, transcript and recording
What instructions was it running on? Published agent version
Where did the amount it offered come from? Tools queried and their responses
What did it do when asked for someone else’s data? Guardrails triggered
Where is the CPF the customer gave? Personal data handling
Why didn’t a person take the call? Human handoff and outcome
Who looked at this conversation afterwards? Access to the record

How long to keep each record

There is no single retention period for the whole region. It depends on the country, the product and the contract with the institution, and it is worth defining by record type, since a voice recording, a transcript and an access log have different uses and storage costs. Some useful references:

  • Brazil, Decree 11,034/2022 (SAC): for services regulated by the federal executive branch, it requires keeping recordings of SAC calls for at least 90 days, keeping the service record for at least two years after the request is resolved and delivering the history to the consumer within five calendar days (Câmara dos Deputados).
  • Brazil, LGPD and Central Bank: the right to request review of automated decisions and the traceability required by CMN Resolution 4,893 can only be met if the conversation and the decision were recorded. Details in what each regulator requires from an AI agent.
  • Mexico: there is no retention rule specific to AI agents. The period comes from the institution’s own policies, from what the CNBV asks for when approving a third party, and from the need to answer CONDUSEF complaints that may arrive months after the conversation.
  • International reference: the EU AI Act requires high-risk systems, including creditworthiness assessment of individuals, to log events automatically, and requires whoever operates them to keep those logs for at least six months (Help Net Security). Several Risk teams in the region use it as a benchmark.

Retention should be written into the vendor contract, together with what happens to the records when the period ends or the commercial relationship does.

Five tests to run with a vendor

  1. Ask them to open a conversation from three months ago and show the agent version that answered.
  2. In that same conversation, ask to see every tool the agent queried and what it received.
  3. Have someone on your team try to get another customer’s data out of the agent, then check how the block was logged.
  4. Search the logs for a CPF or a CURP and confirm it shows up masked.
  5. Ask for the access log of that conversation: who opened it and when.

If any of these tests cannot be run in the demo, it will come back as an objection in the Risk review.

How ADI Platform records it

In ADI Platform, each agent has a Conversations section with the full history: transcript, voice recording, channel, date, the data extracted at the close and an AI-generated summary. Each conversation is tied to the published agent version, and versions can be compared side by side or restored with an immediate rollback.

Mandatory guardrails (prompt injection, personal data exfiltration and crisis signals) run in code, outside the model. Access is controlled by roles and permissions, with every administrative action logged. The platform can also send the transcript and outcome of each conversation through a signed webhook to the institution’s systems, so the record also lives on the bank’s side. Retention is set by contract.

See ADI Platform security and guardrails

Taking an agent into a Risk review? The 12 questions a CISO asks, starting with where each conversation is recorded, are in our article on why Risk freezes AI agent projects in banks.

Frequently asked questions

What is an AI agent audit?

It is a review of what a conversational agent did in production: what it answered, with which version and configuration, which data it used and which controls fired. To run one, every conversation has to be fully recorded.

What should be recorded in every conversation with an AI agent?

Identification and channel, transcript and recording, agent version, tools queried, guardrails triggered, personal data handling, human handoffs with their outcome, and later access to the record.

How long should AI agent conversations be kept?

It depends on the country, the product and the contract. In Brazil, the SAC Decree requires at least 90 days of call recordings and two years of service records. The EU AI Act sets a six-month minimum for logs of high-risk systems.

Is a transcript enough to audit an AI agent?

No. A transcript shows what was said, but not which agent version answered, which data it queried or which security filters stepped in. Without that, a response cannot be explained to a complaining customer or a regulator.

How should personal data appear in the logs?

Masked or tokenized, with access to the full value limited to authorized roles and a record of who looked it up.

Keep reading

For information only. This is not legal advice. Check the current regulation and your legal team for each case.

Get the next ones by email

Twice a month. One field, unsubscribe whenever you want.

More editions