Voice AI for banking: evaluate a bounded service workflow

A banking voice assistant should be evaluated against one defined service task and the institution's controls. Public branch information is different from a balance inquiry; a balance inquiry is different from changing an address or blocking a card. A natural conversation does not establish authorization for any of them.

Begin with public information

Use an approved source for branch hours, addresses and existing contact channels. A synthetic evaluation can test whether the assistant gives the correct information, acknowledges an unknown answer and avoids inventing account-specific details.

Do not put real account information or payment credentials into a general browser trial. A demonstration is not approval to connect banking records.

Review the third-party relationship

The Federal Reserve's interagency third-party risk guidance addresses how banking organizations manage vendor relationships. The institution should review the proposed provider chain, contracts, security controls, access and ongoing oversight before deployment.

This article does not claim that Burki is certified for banking, provides a core-banking connector or handles regulated transactions out of the box.

Define protected operations

For any proposed account workflow, specify the institution-approved authentication process and what remains unavailable before it succeeds. Caller ID or the sound of a voice should not be treated as sufficient authorization. The connected service must enforce permissions independently of the model's instructions.

For changes, document the allowed operation, required confirmation and authoritative success result. Do not announce that a card is cancelled, a balance is updated or a payment succeeded without that evidence.

Test failure and recovery

Use synthetic accounts to test wrong identity, denied access, a stale record, a tool timeout and a request for staff. Check that sensitive data is not exposed in the conversation or ordinary logs. Reconcile uncertain changes before retrying them.

Human transfer requires the selected route and destination to work, including decline and timeout handling. A browser evaluation does not prove a telephone or human-routing path.

Measure a useful result

Track correct public answers, authorized completed tasks, unresolved failures and staff rework. Customer satisfaction and financial savings require observed evidence. Do not infer them from response speed or a falling transfer count.