Scribe v3 Live: Let Callers Finish Before You Book
A founder's guide to testing turn prediction, correcting appointment details, and confirming bookings without confusing a finished sentence with permission to act.
ElevenLabs released Scribe v3 and Scribe v3 Live on November 10. The live model adds conversation-aware turn prediction: a way to estimate whether someone has finished speaking, rather than relying only on silence. It is relevant again in today's AI briefing, but this is not a November 13 launch. The release announcement describes the capabilities and vendor evaluation; we have not reproduced its results.
For founders building an appointment receptionist, the useful opportunity is narrower than “more accurate voice AI.” A caller can pause while checking a calendar, correct a date halfway through a sentence, or ask a companion about availability. A system that answers too early can make the conversation frustrating and the appointment wrong.
Our judgment: test whether turn prediction improves the entire booking conversation, while keeping an explicit confirmation step before the calendar changes. Finishing a sentence and authorizing a booking are different events. This guide gives small product teams a concrete scenario, a reusable conversation test sheet, and a bounded rollout decision. It is a proposed evaluation method, not a report of calls we have run or a recommendation based on measured performance.
1. Define the customer problem before changing the model
Imagine a fictional bicycle workshop, Harbor Cycles. Its owner wants a voice receptionist to collect a repair request and offer an available appointment. The caller says, “Could I come Thursday at three ... actually, Friday, after I collect my daughter?” The pause is meaningful because the request is still being formed. If the assistant starts listing Thursday slots during that pause, the caller must interrupt it to regain control.
There are at least three possible product problems here. The system may recognize the words incorrectly. It may recognize them correctly but decide too early that the caller is finished. Or it may understand Friday correctly but save Thursday because an earlier draft already reached the booking system. Changing speech recognition alone does not resolve all three.
Write a specific objective before evaluating a replacement: callers should be able to state and correct their preferred time without losing control of the conversation. Then write an observable outcome: the saved appointment matches the details the caller explicitly confirmed, or the session ends with an accurate explanation that no appointment was created. A pleasant voice is helpful, but it cannot substitute for that outcome.
The workshop is an illustrative scenario throughout this article. No business, call recording, customer result, or revenue improvement is being claimed. Its value is that the task can be inspected from beginning to end by a founder who does not need to read a speech-model leaderboard.
2. Separate speech, turns, and permission
Automatic speech recognition, or ASR, converts audio into text. Voice activity detection, or VAD, identifies periods of speech and silence. Turn prediction estimates whether the speaker has finished a conversational turn. A transcript commit closes a segment of recognized text. A booking confirmation establishes that the customer accepts specific appointment details. These concepts describe different parts of the interaction.
The commit-strategy documentation says manual commitment remains the default. Turn prediction must be enabled explicitly for Scribe v3 Live; changing a model name alone does not establish that the application uses it. The same documentation says silence-based commitment remains active alongside prediction. Therefore, “context-aware” does not mean that pauses can never end a segment.
Ask your builder or implementation partner to demonstrate which events advance the conversation. A committed transcript can justify preparing a response. It should not, by itself, authorize changing a calendar. Likewise, a predicted end of turn says nothing about whether the customer selected an available slot, accepted a cancellation policy, or agreed to the final date.
Keep the handoff explainable in ordinary language: “I heard a request,” “I found an available option,” “you confirmed this option,” and “the calendar accepted it.” If a demonstration jumps from the first statement to the last, ask to see the missing decisions. You can evaluate this behavior without prescribing a particular framework or writing the integration yourself.
3. Read accuracy claims at the right level
The release presents improvements in transcription and speaker labeling, but those claims belong to the vendor's test conditions. We deliberately do not turn a headline error reduction into a predicted reduction in failed appointments. Your product adds dialogue logic, availability checks, calendar updates, and recovery behavior that a transcription benchmark does not measure.
Word error rate measures substitutions, deletions, and insertions relative to a reference transcript. Semantic error scoring instead tries to distinguish mistakes that change meaning. Neither measure alone tells you whether “next Friday” was resolved against the correct date, whether a caller could correct it, or whether the system created a duplicate appointment.
The original FLEURS paper describes a multilingual speech benchmark covering 102 languages. That is useful context for understanding language evaluation, not evidence that every telephone accent, workshop noise level, or bilingual exchange has been validated. Treat a language being listed as supported as an invitation to test your audience, not as proof of equivalent experience across audiences.
Even a perfect transcript can preserve ambiguity. “Friday afternoon” may describe a preference rather than consent to a particular time. The product must resolve that ambiguity through a question. Conversely, a slightly imperfect transcript might still yield a correct appointment after the caller reviews the details. Assess both the transcription and the customer outcome, and keep their denominators separate.
4. Give corrections a clear place in the conversation
For Harbor Cycles, a sensible conversation has five visible stages: gather the request, resolve unclear details, offer a current slot, read back the selected appointment, and confirm the saved result. These are product stages, not requirements to implement five separate services. The caller should understand which stage they are in from what the assistant says.
Consider this proposed exchange. The caller requests Thursday, pauses, then changes to Friday afternoon. The assistant waits, asks which Friday if necessary, and offers a specific available time. Before saving, it says, “I can book your bicycle inspection for Friday, November 20, at 3 p.m. Shall I confirm that?” The wording includes the service, full date, and time; a real deployment must also resolve its local time zone and location where relevant.
If the caller replies, “Yes, but make that the other branch,” the response contains a correction. It should return the conversation to the relevant detail, not treat the opening “yes” as permission to save the old offer. Once the location changes, availability may change too. Read back the updated offer before asking again.
The W3C explanation of error prevention describes confirmation, checking, or reversal for important submissions within its scope. We use that as a design rationale for review and correction, not a claim that this conversational sketch certifies accessibility or determines every appointment's legal obligations. Offer an accessible alternative for people who cannot comfortably use the call flow.
5. Test the moments a smooth demonstration hides
A prepared demo often features one confident speaker, a short request, and no correction. Your evaluation should deliberately include moments when the caller is still thinking. Ask someone to look at a calendar during the call rather than reading a polished sentence. Let them decide between two days, change a service, or explain a constraint in their own words.
Include pauses in different positions: inside a phone number, before a correction, after a complete request, and while the assistant is waiting for confirmation. These situations should not all have the same expected behavior. Waiting longer inside a number may help, while waiting unnecessarily after an unambiguous answer can make the service feel unresponsive.
The speech-to-text capability guide distinguishes live recognition, multilingual features, and transcript controls. That establishes documented options, not performance in your deployment. Test actual supported configurations separately. A feature name in an announcement does not prove your builder exposes it, your plan enables it, or every combination works together.
Use relevant participants and neutral tasks. The GOV.UK moderated-testing guide recommends believable goals and avoiding instructions that reveal the answer. For this evaluation, “Arrange a repair after school pickup” is better than “Say Friday, pause, and then correct Thursday.” Scripted edge cases remain useful for regression checks, but they should not replace observing how people naturally express the task.
6. Reuse this appointment conversation test sheet
Create one sheet per configuration, and one record per attempted task. Before testing, fill in the exact model, builder version, commit strategy, audio path, language setting, calendar environment, and date. If any of these are unknown, mark them unknown. A configuration receipt is more useful than assuming the vendor's newest feature is active.
| Situation | What to observe | Acceptable task result | Failure to record |
|---|---|---|---|
| Pause inside a phone number | Whether the assistant speaks before the number is complete | Full number can be reviewed and corrected | Missing digits accepted silently |
| Calendar-checking pause | Whether the caller retains the floor | Caller can finish the preference | Assistant repeatedly interrupts |
| Date correction | Which date reaches the final offer | Corrected date is read back | Earlier draft survives |
| “Yes, but” response | Whether the correction is handled before saving | Updated offer receives fresh confirmation | Initial yes triggers old booking |
| Companion speaking nearby | Whether unrelated speech changes the offer | Intended caller can review details | Background answer becomes consent |
| Mixed-language request | Whether essential details survive switching | Caller can verify date and service | Unsupported assumption is hidden |
| Slot disappears before save | Whether current availability is checked | Alternative offered without false success | Unavailable appointment announced |
| Connection drops after confirmation | Whether status can be recovered | Saved or unsaved state established | Duplicate created on retry |
| Caller declines | Whether the assistant respects the decision | No booking and clear closure | Persistence turns into unwanted booking |
| Human assistance requested | Whether the route actually works | Transfer or accurate callback instruction | Promised handoff goes nowhere |
This is a proposed worksheet, not a validated benchmark. Add a session identifier, expected outcome, observed outcome, interruption time, response time, correction count, assistance, and final saved state. Use synthetic contact details for early testing and decide what recordings are necessary before collecting them. The sheet should reveal the customer problem without becoming an unnecessary store of private conversations.
Give every row an owner and an action. An interruption might require changing dialogue behavior; a false success requires investigating the booking path. Combining them under “the model failed” makes the next iteration harder to choose. Keep the original observation alongside your diagnosis so later reviewers can disagree with the diagnosis without losing the evidence.
7. Compare waiting time with recovery effort
A shorter response delay is valuable only in context. Measure from the moment the caller actually finishes the relevant turn to the start of a useful response. Also record premature responses separately. Otherwise, a system that interrupts people can appear fast because your clock starts at the wrong event.
For an offline review, have a reviewer mark the intended end of the caller's turn before inspecting the model's behavior where practical. Some turns will be ambiguous; keep those marked as ambiguous rather than forcing false precision. In live sessions, record whether the caller felt interrupted, but do not let that impression replace the observable sequence of speech and actions.
Compare your existing configuration with the candidate using comparable tasks, devices, and environments. Rotate order where possible, since participants learn the task and may shorten their answers the second time. A replay can isolate recognition and timing differences, but it does not recreate a conversation: a real caller changes what they say after hearing the assistant. Use replay and live sessions for different questions.
Make one timing record concrete before collecting a batch. A proposed HC-01 record could identify the test session, configuration revision, caller request, reviewer-marked turn boundary, first audible reply, corrected offer, confirmation, and final calendar result. Leave every time and outcome blank until observed. Link the recording, relevant transcript segment, and test appointment where retention permission allows it. This is a record design, not evidence that Harbor Cycles or Scribe passed a test.
Use those boundaries to locate the waiting. Recognition may finish quickly while the calendar lookup takes longer; a prompt reply may still offer the wrong date. Do not subtract timestamps from separate devices unless their clocks and meanings can be reconciled. Where timing cannot be aligned, report the observation and its uncertainty rather than invent a precise latency figure. An observer can still determine whether the assistant interrupted a correction or announced success before a booking existed.
Preserve the cost of restarting the conversation. If a caller repeats the date after an interruption, count that correction even when the final appointment is right. If staff resolve an uncertain save later, mark the outcome human-assisted and retain the elapsed recovery period separately from the automated response delay. After a configuration change, repeat the affected cases with the same availability fixtures before inviting new callers. Otherwise a less busy calendar or a clearer set of slots can receive credit that belongs to the environment rather than turn prediction.
Report the distribution of delays, the interruptions, and the recovery work instead of a single average. A few very slow calls can matter to a small business, while a rare wrong appointment can matter more than a modest improvement in typical speed. A small pilot can uncover these failure modes; it cannot establish a universal superiority claim or accurately estimate every rare-event rate.
8. Keep speaker labels and saved actions honest
Speaker diarization labels which speaker produced particular words. It is not identity verification. A label such as “speaker one” cannot establish that the person is the account holder, the appointment owner, or someone authorized to approve a change. A founder should not let an appealing multi-speaker demo quietly expand the application's permissions.
The realtime API reference describes speaker information that can update, with final diarization arriving separately after committed text. This matters if your product acts before final speaker information is available. Ask the implementer what happens when a later label differs from an earlier assumption, rather than presuming a committed segment has settled every attribute.
In the workshop example, a companion might say “yes” while the caller is still choosing. The assistant can ask the intended caller to confirm the complete details again. For sensitive account changes, the product needs its own appropriate authorization process. A voice transcript is input to that process, not a substitute for it.
After a calendar request, report success only when the booking system has established the saved state. If the connection drops, show or say that the result is not yet confirmed and recover the status before retrying. The exact mechanism belongs to your integration, but the acceptance question is simple: can the team demonstrate a retry without creating a second appointment? If it cannot, keep that path out of unattended rollout.
9. Decide where voice belongs in the product
Turn prediction is most worth investigating when interrupted or overly delayed conversation is an observed obstacle. If most failures come from unavailable slots, unclear service categories, or an unreliable calendar connection, prioritize those problems first. A speech upgrade may still help, but the evidence should identify which part of the customer experience it changes.
Voice can suit a caller who is away from a keyboard or needs help choosing a service. A short web form may suit someone comparing dates visually, correcting an address, or keeping a written record. Offer a clear route between channels where your operation supports it. Do not advertise an alternative that staff cannot actually fulfill.
This guide is designed for ordinary appointment intake and a bounded pilot. It does not establish readiness for emergency response, clinical triage, financial instructions, or other consequential tasks. Those uses introduce domain-specific requirements that a turn-taking experiment cannot settle. Similarly, a single-language workshop trial should not justify launching a multilingual receptionist nationwide.
Choose the smallest useful scope: one location, a defined service list, known operating hours, and a booking environment the team can inspect. That scope creates a practical tradeoff. You learn less about the whole market initially, but you can diagnose failures and keep the service promise accurate. Expand after you understand the failure patterns, rather than treating the initial scope as an embarrassment to hide.
10. Make the rollout decision from completed tasks
Before the pilot, define what counts as starting and completing a task. For Harbor Cycles, completion might mean an appointment with the intended service, date, location, and contact route, explicitly confirmed by the caller and accepted by the calendar. A call that ends politely without that outcome is not a successful booking. A correctly handled refusal is a separate valid outcome, not necessarily a booking failure.
The GOV.UK completion-rate guidance emphasizes defined transaction boundaries and distinguishes assisted completion. Apply that discipline by reporting started booking attempts, successful bookings, refusals, abandoned attempts, and human-assisted outcomes separately. Do not remove difficult calls from the denominator because the transcript was inconvenient to score. If you exclude a call, document the eligibility rule and reason.
Set release criteria appropriate to the scope before seeing the comparison. Examples of blocking observations are a booking made without clear confirmation, a correction ignored at save time, or a success statement with no saved appointment. These are proposed product gates, not observed defects in Scribe. For waiting and interruption tolerances, use your actual audience and operating needs rather than borrowing an unexplained universal threshold.
Keep the current configuration available for rollback, name the person who reviews failures, and define when the pilot pauses. Include staff recovery time and unsuccessful calls when estimating cost per completed appointment; if none complete, that ratio is undefined, not zero. Separate model charges from telephony, integration, and human handling so an apparent saving does not hide more support work.
The immediate action is to run a bounded comparison with the worksheet, inspect the saved appointment for every attempted booking, and decide whether the observed improvement warrants a wider pilot. Scribe v3 Live offers a documented new way to detect conversational turns. Your product earns trust when the caller can finish, correct the request, and understand exactly what was booked.