Jul 13, 2026

Keeping the call path lean: defer the work, not the data

When people talk about latency in voice agents, they usually mean the pipeline: speech-to-text, the language model, text-to-speech, turn detection. That part matters, but on my platform it's handled by the orchestration layer (VAPI), so it isn't where I spent my effort. The latency win I'm actually proud of came from an architectural decision one level up: doing as little work as possible during the call, and pushing everything else to after it.

The problem with tool calls mid-call

A voice agent for a business needs to do things: look up a customer, check availability, write a record, update a CRM. The obvious way to build this is to have the agent call those tools live, in the middle of the conversation, as the need comes up. It works in a demo. In production it's a latency trap.

Every mid-call tool invocation is a synchronous round trip sitting directly in the conversation path. The caller is on the line, listening to silence, while your agent waits on a database write or a third-party API. Chain a couple of those together in one call and the dead air is obvious and awkward. Worse, the failure modes are terrible: if that API is slow or flaky, the caller experiences it as your agent freezing mid-sentence. You've coupled the responsiveness of a live conversation to the reliability of every backend it touches.

Gather during, process after

So I inverted it. During the call, the agent's job is narrow: hold a good conversation and capture the information, not act on it. Tool calls during the call are kept to the bare minimum needed to actually steer the conversation. Everything else, the record-keeping, the lookups that don't change what the agent says next, the CRM updates, doesn't happen live.

When the call ends, the full transcript is handed off to an async job queue (Redis + BullMQ). A worker picks it up and does the real work off the critical path: parses the transcript, extracts the structured data, runs the lookups, and updates the database. The caller is long gone by the time any of this runs, so none of it costs conversation latency. If a downstream API is slow, the job waits; the call never did.

Why this is the right trade

The insight is that most of what a business call needs to accomplish doesn't have to happen synchronously with the call. The caller needs the agent to sound present and get their intent right. They do not need the database to be updated before they hang up. Once you separate "capture the intent" from "execute on the intent," the live path gets dramatically leaner, and the execution path gets more robust, because a queue gives you retries, backpressure, and isolation for free. A flaky integration becomes a retried job instead of a ruined call.

There's a reliability dividend too. Mid-call tool calls are fragile precisely because they're synchronous and unforgiving. Moving that work into idempotent background jobs means a transient failure is a retry, not a customer-facing incident. The call path has fewer ways to break, and the parts that can break are the parts nobody is waiting on.

What I'd tell someone starting

Ask which work truly has to happen while the caller is on the line, and you'll usually find the honest answer is "less than you think." Keep the conversation path lean, capture everything, and let a queue do the heavy lifting after the call. The best latency optimization is often the work you decided not to do synchronously.