Inworld Inference: Set Up First-Token Fallback

A voice assistant can have a good answer ready too late to be useful. One possible recovery is to ask a second language model when the first has not started replying. The design question is how long to wait, what the replacement may change, and how to tell whether it helped.

Inworld launched Inworld Inference on October 14, 2026. Its launch describes model routing for interactive applications, including fallback when the first token takes too long. This guide checks that setup on October 20. It does not present the launch as today's news or turn the vendor's performance claims into a Burki benchmark. Inworld's launch and measurement notes.

This is a direct-provider developer guide. Burki's current native Inworld speech controls do not establish an Inworld language-model router integration. The examples below were not submitted to a provider. No paid request, configuration change or production call was made for this article.

Download the free fallback planning worksheet. Use it to write down the model order, delay budget, quality checks and evidence you would need before enabling a route.

Define the failure you want to recover from

Start with a narrow task: answer a fictional studio's approved opening-hours question. The first version should return a short text answer. It should not book a visit, change a record or invoke a tool. That makes it easier to inspect the fallback's words before expanding its responsibilities.

Write a small answer contract:

• Use the supplied office facts and retain their exceptions.
• Say when the facts do not answer the question.
• Keep the reply short enough to speak naturally.
• Do not claim an appointment or staff callback has been arranged.

A fallback that returns sooner but changes a closed day into an open day fails this contract. Response availability and answer correctness need separate review.

Also define where recovery ends. First-token fallback addresses a model that has not begun its text response. It should not be treated as a general repair for speech already played, an unavailable phone route or an uncertain booking result. Those are different failure paths.

Use the current request-level fields

Inworld's Chat Completions documentation uses extra_body.models for the ordered fallback list and extra_body.fallback.ttft_timeout for the first-token deadline. The duration must be at least 300ms. When the primary misses it, the router cancels that request and tries the next model. Inspect metadata.attempts in the response to see the models attempted. These are Chat Completions fields; do not assume another API returns the same metadata shape. Request-level routing.

For direct HTTP, the quickstart uses the Standard API credential copied from Portal as the Basic authorization value. Keep it on a server, without encoding the copied credential again. A browser needs a server proxy or an appropriate backend-minted token. Inference authentication and quickstart.

The following is an unexecuted JSON configuration example, not a request receipt. Both identifiers appear in the current model catalog. They illustrate a larger primary and a smaller fallback; their order is not a measured quality recommendation.

{
  "model": "inworld/models/gemma-4-31b-it",
  "messages": [
    {
      "role": "system",
      "content": "Answer only from these fictional office facts: Lark Studio reception is open Monday to Friday, 09:00 to 17:00. It is closed on public holidays. No appointment availability is provided. If the facts are insufficient, say staff must confirm. Do not arrange anything."
    },
    {
      "role": "user",
      "content": "Can I visit reception at noon on a public holiday?"
    }
  ],
  "extra_body": {
    "models": ["inworld/models/gemma-4-26b-a4b-it"],
    "fallback": {"ttft_timeout": "900ms"}
  }
}

900ms is a planning choice for this example, not a recommended setting for every application. Verify the current catalog, account access and request compatibility before authorizing any actual use. Current model identifiers.

A direct request can carry its own routing choices. A saved router is a reusable configuration with additional route and variant behavior; it is another configuration surface to review. Do not create one merely because you want to understand a direct fallback. Saved-router concepts.

Budget the complete wait

If a primary uses the entire first-token allowance before the fallback starts, the fallback cannot erase that wait. Your planning budget needs room for both attempts, transport and whatever happens after the text arrives.

Illustrative sequence, not a recorded trace. A first text token is an intermediate event; it is not proof that speech reached a caller.

Use separate timestamps in a future authorized evaluation: request sent, first text token received, first audio generated and first audio heard on the target channel. A single response-time number hides where the delay occurred. If you only have a provider's attempt metadata, keep the other measurements blank.

Consider a caller who asks a simple office question while a long system prompt is being processed. A shorter deadline may trigger replacements during otherwise healthy requests. A longer deadline may leave too little room for a useful recovery. Decide from the actual workload and accepted waiting experience, rather than copying the smallest documented duration.

Keep cancellation separate from a business operation. In this example the task only produces text. Before any future tool-enabled version, review how the application treats action attempts and uncertain outcomes. A model fallback should not silently become permission to repeat an external action.

Make fallback quality visible

Keep the original question and approved facts identical when comparing the primary and replacement. Otherwise an improved answer might come from a prompt change rather than the model choice.

For the fictional public-holiday question, the expected meaning is that reception is closed on public holidays. The answer must not infer appointment availability from ordinary weekday opening hours. Record both the words returned and the reason a reviewer accepted or rejected them.

Add these acceptance cases before collecting any results:

These are proposed checks. They are not claims about tests performed by Burki or guarantees about provider recovery. The worksheet gives you places to save the configuration, attempt evidence and reviewer decision without filling missing results with assumed successes.

Account for model usage and the rest of the voice stack

The October 20 catalog lists Gemma 4 31B at USD 0.13 per million input tokens and USD 0.38 per million output tokens. Gemma 4 26B A4B lists USD 0.07 input and USD 0.34 output, in the same units. These are the named catalog rates, not an all-model price or a promise that every fallback attempt is free. Model rates.

Inworld describes inference as token-billed usage drawn from shared credits. Account plans and current model rates need review alongside the actual usage record. The minimum advertised input rate is a starting point, not the cost of an entire spoken interaction. Current pricing and credits.

For your estimate, list input tokens, output tokens, any cached-input treatment and the usage evidence available for each attempt. If attempted-request billing is unclear, leave that line unresolved until the provider's applicable terms or usage record answers it. Then add recognition, synthesis, transport, phone-number and platform costs where they apply. The downloadable worksheet is free; production inference and voice use are separate.

Apply the decision to a Burki assistant

In Burki, first distinguish the language model that writes the reply from the provider that speaks it. Selecting an Inworld voice does not configure this Inference router. Current source supports Inworld speech adapters, while the native language-model providers and custom endpoints have their own contracts. No direct Inworld router choice or accepted fallback execution has been verified here.

You can still use the worked example to review an existing Burki assistant's answer contract: approved facts, exception handling and an honest response when information is missing. Start from the AI receptionist use case, then inspect the assistant's actual available model settings. Do not paste the provider JSON above into a speech field or describe a custom endpoint as ready without verifying its authentication, streaming and tool behavior.

For a broader review of response quality, use a model evaluation plan that avoids cherry-picking the demo. For support correlation after an authorized provider request, the Inworld request-ID guide answers a different question: locating an attempt, rather than selecting the replacement model.

The useful outcome is a route specification someone can review: what it answers, what triggers replacement, which model may answer instead, what it might cost and what would count as an accepted result. Until those results exist, it remains a configuration and evaluation plan.