The difference between chatting with an LLM and having an assistant is who manages your attention.
A chat answers when you ask. An assistant runs your errands while your attention is somewhere else, and that means every finished piece of work raises a question a chat never faces: does this deserve your attention now, or can it wait?
You can watch shipping products face that question and answer it differently. At the time of this writing, I gave the ChatGPT and Claude voice modes the same paired request: check the web for recent developments in voice LLMs, and tell me the weather in San Francisco. ChatGPT surfaced the weather to a widget on screen and voiced the research summary; when I barged in and later asked it to continue, it finished the research and only then spoke the weather. Two channels for the same answer, delivered in that order, without knowing whether I was looking. Claude spoke both answers in the order I asked, the quick one queued behind the slow one; in one run, even a failure report waited its turn. By the time you read this, both behaviors will probably have changed; ChatGPT's already has: OpenAI swapped its entire voice stack days after I ran this. That is rather the point: the decision about what reaches you, when, and how is being made inside every one of these products, silently, differently and opaquely.
I've spent the last year exploring voice-first UX through nights-and-weekends projects, from games to assistants. I started out thinking the gap between a voice interface and a real conversation was mostly latency: better speech-to-text, a faster model, less dead air. It isn't. Lower latency gets you a snappier IVR, not a conversation. The hard part is owning that decision: what reaches your ears, and when.
Two bugs started giving me clues about the real challenges. The first: a background run finished while I was mid-sentence, but it had started in an earlier session, and its result was silently dropped. The answer existed; I just never heard it. A request-execute-respond assistant, the kind that holds the channel until it answers, never faces that moment. I had the concurrency and still lost. What was missing was not the ability to run many things, it was anything deciding when a finished result should reach me. The second: an accidental barge-in cut the assistant off mid-answer. I was away from the screen, the rest of its reply was never spoken, and it was simply gone. Neither can happen in a text chat. Text doesn't evaporate while you read something else, and half a message doesn't vanish because you cleared your throat.
Voice is not text with a microphone. As an output channel it takes away what a screen gives you for free: many results visible at once, a scrollback to re-read, the freedom to look when you're ready. A screen lets you hold a dozen things in parallel; voice gives you one ephemeral channel, one thing at a time, gone the moment it's spoken. So the instant a voice system is doing more than one thing at once, something has to decide what reaches that channel and when, and hold the rest. I'll call it the attention plane (plane as in control plane or data plane, nothing to do with the transformer's attention). The part of it that decides whose output reaches you, and when, is its arbiter. That plane is the rest of this piece.
Fig. 1: The Attention Plane sits between N background runs and the single spoken channel; its arbiter decides what reaches the user and when, and holds the rest.
The channel is single-stream, the listener isn't
It's worth being precise about what is actually serial here, because it isn't attention. People hold several things in mind at once; one sentence can raise three topics. The single-threaded thing is the spoken channel. You can attend to exactly one auditory stream without serious decrement [1], whereas ten browser tabs sit open at negligible cost, and decades of HCI work describe voice's turn structure as stubbornly single-file [2]. The bottleneck is not the user's attention. It is the medium that delivers things to it.
I learned that the slow way. The first version of my assistant allowed only one active run at a time, on the reasoning that "voice is serial, so do one thing." That was the wrong constraint, and a costly UX compromise. The work behind the voice was already concurrent; the agent harness happily ran two tasks and answered them a second apart, and capping it to one run didn't make voice any more serial; it just did less. What's serial is the single channel carrying the output, not the number of things happening behind it. Letting many runs proceed while still feeding one spoken channel, that gap, is the whole problem.
That matters because the medium is also ephemeral and intrusive. A spoken sentence arrives whether you were ready or not, and once it's gone there's no scrollback. An email waits politely until you look; a result spoken into your ear interrupts by existing. So a voice system that runs more than one task needs rules a text app never needs: what gets spoken, in what order, and what waits.
Nothing shipping has a place to make it
It's worth looking at what's shipping. The voices themselves have never been better, and every launch now pitches voice as the primary interface. But a channel is not first-class because it sounds natural; it is first-class when someone owns what reaches it. Nothing out there ships that yet: nowhere does this decision have a place the user can see or steer. Apple's Siri AI, introduced at WWDC 2026 as "an entirely new version of Siri," rebuilt "from the ground up" [3], can now hold a natural back-and-forth conversation and take actions across your apps. But its model is request, execute, respond: everything Apple describes begins with you asking and ends with Siri answering; nothing keeps working after the reply. So a finished result arriving while you talk about something else is a situation it structurally never faces. Amazon's Alexa has shipped the conservative answer for years: a notification lands as a chime and a yellow ring, and its content is spoken only when you ask [4]. Nothing is decided on your behalf, because nothing is delivered at all; the channel stays silent until you pull.
Google saw the next step a decade ago and patented it, conversation-aware interjection of a single deferred result, graceful transitions and all [5]; for a decade, as far as I could tell, nothing shipping spoke it. Lindy, an AI assistant platform, runs tasks in the background and can text you when something in your inbox looks urgent [6]. It exposes configured urgency criteria layered over an inferred default, and points in the direction I'll argue for: the closer a system gets to running work you didn't wait for, the more the "who decides what reaches you" question shows up as an actual product surface. And open source has the same shape one level down: Voicebox, a popular local voice studio, lets any agent speak to you in a cloned voice with a single call, the agent decides to talk and it renders the audio [7]. That is fine with one agent, and precisely the problem with several.
Then, as I was finishing this piece, OpenAI replaced the engine behind ChatGPT Voice with GPT-Live: full-duplex, deciding many times per second whether to speak, keep listening, pause, or interrupt; when a request needs deeper work, it hands the task to a frontier model in the background, keeps talking, and brings the result back into the conversation "when it's ready" [8]. I re-ran my paired request on it. I asked a question I knew would take several seconds of research, asked for the weather a few seconds later, and got the weather first; the research kept going, and its answer came back into the conversation on its own when it was ready. The moment Google patented, a deferred result delivered into a live conversation, now ships, and it ships well. But look for the place where you say what may interrupt you, and there isn't one. The closest control is a global Background Conversations toggle deciding whether the conversation may keep running outside the app at all; which result deserves to reach you, and when, is decided inside the model, with no surface to set it and no record of what it held back. That is a first-class voice; it is still not a first-class channel.
Four shapes, one shared blind spot: Siri avoids the decision by never needing it, Alexa hands it back to you unmade, Voicebox lets each agent make it, and GPT-Live makes it for you where you can't see it. Lindy is closer, but the layer where the user should sit is opt-in, behind an inference default. In none of them does the arbiter I'm describing have a place to live. So I built the dumbest possible version of it.
What I built: a priority queue
The first version of that plane is deliberately dumb. Every update a background run produces gets tagged high (it's blocked or failed, it needs the user) or low (it finished or produced output, it's informational), and a queue serializes them onto the one channel under a simple policy: never interrupt the user, surface blockers first, and batch near-simultaneous updates into a single utterance.
type Priority = "high" | "low"; // high = blocked/failed (needs you); low = output/done (informational)
interface NarrationItem {
priority: Priority;
kind: string;
text: string;
label?: string;
}
function createNarrationQueue() {
let items: NarrationItem[] = [];
return {
enqueue(item: NarrationItem) {
items.push(item);
},
// Speak only when the channel is idle (open, user not talking, nothing in flight).
// All queued highs first, else all lows. Returned items are removed from the queue.
drain(canSpeak: boolean): NarrationItem[] | null {
if (!canSpeak || items.length === 0) return null;
const highs = items.filter((i) => i.priority === "high");
if (highs.length > 0) {
items = items.filter((i) => i.priority !== "high");
return highs;
}
const batch = items;
items = [];
return batch;
},
};
}It's deliberately source-agnostic: it knows nothing about runs, the model, or the screen. It never barges in, and within a session it never silently drops anything. The full version, with its tests, is published as a gist. Here it is on a two-run test. I asked the assistant to search for recent articles on voice LLMs, the same kind of request I'd given ChatGPT and Claude in the opening, and before the search came back I asked for a quick calculation. It acknowledged both and ran both. The calculation landed almost immediately, and the queue voiced it while the search was still digging; the search summary landed 10 to 15 seconds later and was voiced in its turn. Two runs dispatched in one order and answered in the other, each spoken cleanly under its own name, neither stranding the channel nor getting talked over. When a result lands while the channel is busy, the queue holds it, then voices it in turn, tagged so you know which task it came from. Nothing is dropped, nothing interrupts abruptly. That is exactly what a stack that treats voice as an afterthought gets wrong: it either drops whatever arrives during a spoken utterance or forces everything into dispatch order.
Fig. 2: A two-lane timeline of the worked example. Top lane is dispatch order (the web search first, the quick calculation second); bottom lane is spoken order (the calculation first, the search summary second), each labeled. The queue voices whatever is ready rather than forcing dispatch order.
Making "nothing evaporates" true
Within a session, that is. Across sessions, that last promise had to be earned, and the first bug from the opening is how I found out. The run had started in an earlier session and wasn't in this session's registry, so when its result arrived the queue had nothing to attach it to. The system of record had it the whole time, the append-only ledger held every run and every utterance, but the client never read it back. The fix wasn't new machinery; it was making the queue rehydrate from the ledger on resume, so a result that lands across an interruption, or across a reload, survives instead of evaporating. The ledger was built to make that free; I just had to read it.
The queue can hold things, order them, and not lose them. What it still can't do is tell whether you care about them right now.
Where the dumb queue runs out
The queue treats every low-priority result as equal, so anything that finishes while you're talking about something else still piles onto your one channel and waits its turn to interrupt you, or stacks up unread. With two background runs that's tolerable. With ten it isn't: the channel fills with finished work you don't care about in this moment.
The obvious next move is a smarter scheduler, one that ranks how much each result matters and interrupts you with the winners. That is the mistake. The moment a system tries to deduce how important a result is to you, it is guessing at a value judgment that is yours to make, and it will guess wrong in exactly the cases that matter.
The two axes (the design I'm building toward)
So the version I'm building toward splits the arbiter's decision on two axes, and keeps them strictly separate.
Whether a result is important enough to interrupt you is a value judgment. Only you can make it, so the system takes it as configuration, not as something to infer: a per-task notify setting you can set or change. Whether a result is relevant to what you're talking about right now is a different question, one of plain topical overlap, and the system can and should answer it. Letting an on-topic result fold into the conversation isn't the system deciding what's important to you; it's the system keeping the conversation coherent. One exception proves the rule: a run that is blocked, waiting on input only you can give, is stalling real work, and that's an objective fact, not a guess about importance, so it's allowed to surface more assertively.
// The shape I'm building toward, not what v1 runs today.
// notify is read, never computed. relevance is the only thing the model decides.
type NotifyHint = "immediate" | "queued" | "silent"; // configured per task by the user
type Surfacing = "interject" | "fold_in" | "heads_up" | "defer" | "log";
function decideSurfacing(
o: Output,
focus: Focus,
relevance: RelevanceFn
): Surfacing {
if (o.notify === "silent") return "log"; // recorded, terminal
const hint = o.fromBlockedAgent ? raise(o.notify) : o.notify; // the one allowed bump
if (hint === "immediate") return "interject"; // configured importance wins, no scoring
// Importance didn't force it. Relevance (computed, model-delegated) decides the rest.
const { score, confidence } = relevance(o.topic, focus.activeTopics);
if (score >= FOLD_IN && confidence >= SURE) return "fold_in";
if (score >= MENTION) return "heads_up";
return "defer"; // stays queued, unread, on its own topic
}
Fig. 3: The surfacing decision on two axes. Configured importance (yours to set, never computed) on one axis, computed relevance to what's currently on the floor on the other; a blocked run is the one structural exception allowed to raise assertiveness. The two axes never trade places.
The thing that makes this work, and the thing I'm most careful about, is that relevance stays relatively small. It's a cheap topical-match call, not a second reasoning pass: does this output's topic overlap what's currently on the floor, returning a score and a confidence, nothing more. Keep it small and it's the difference between a scheduler that guesses and a system that defers. Let it grow into a full importance model and you've rebuilt the smart scheduler the whole design exists to avoid.
Why this is a voice problem and not a focus problem
It's fair to push back here: isn't "too many things at once" just a general attention problem, and can't I hit it in text too? You can hold the concerns in your head either way; that part is medium-independent. The difference is that a screen already solves the coordination for you. Tabs, notifications, and scrollback let a dozen results sit in parallel, persist, and wait until you look.
Text agents are the working proof. Anthropic's Dispatch is the closest text analogue to the assistant I'm describing: one continuous thread you reach from your phone; you assign Claude a task, go do something else, and come back to the finished work [9]. In my use it happily runs several tasks at once, so the shape is exactly my problem, one channel and many results arriving in their own time, yet it needs no arbiter: the channel is a transcript, and a finished result just lands and waits for you to look. The same shape appears in Nous Research's Hermes, the same harness my assistant dispatches to, where multiple agents coordinate through a shared Kanban board, another off-channel coordination surface, another problem the screen solves for free [10].
Even here, delivery is the fragile step. A long-running task will sometimes finish without its result ever reaching the thread, and the scrollback makes that failure quiet instead of solving it. "Come back to it" is the one move voice doesn't have. Take the scrollback away, and the decision Dispatch never has to make, whose output reaches you and when, is the entire design problem. Voice strips all three away, so the affordances a screen gives for free have to be rebuilt by hand: the attention plane to decide what speaks, a persistent queue so nothing is lost, and the screen as a pressure valve for everything that shouldn't be spoken.
That last one is worth its own line, because voice and screen are not rivals. The most consequential design decision in a translator prototype I built for iPad, meant to sit in a lobby between a concierge and a visitor, wasn't about translation at all: it was which of the two speakers got the voice channel and which got the screen. Dense, scannable material, a comparison, a list, a long result, belongs on a screen, where the eye can skim and re-read; the voice gives the overview and walks you through one step at a time. Routing dense content to the screen is also how the single spoken channel stays survivable: you stop forcing through it the things it's worst at carrying. The cognitive ceiling is the same as it ever was; what changes is that voice removes the tools you'd normally use to stay under it, and the attention plane plus the screen are how you put them back.
The rest of the iceberg
The attention plane is the part I had to design rather than tune. It sits on a stack of other places where voice refuses to behave like text, each of which is really its own piece. Model choice and latency, recognition accuracy on a domain's own vocabulary (where biasing on in-domain terms cuts the error on the words that matter by 25 to 60 percent relative [11]), the acoustics of the room, isolating the principal speaker, and the biometric weight of audio itself: each is its own engineering problem, and none of them is solved by treating voice as text with a microphone. Each of those gets its own treatment in the longer survey this piece grew out of [12].
What's next
The argument, compressed: the moment a voice system runs more than one thing, something must own what reaches your single spoken channel, and when. Importance is yours to configure, relevance is the system's to compute, a blocked run is the one exception, and nothing evaporates. That plane is what separates an assistant from a chat with a microphone, and it stands on its own. What comes next follows the build: the two-axes arbiter as it lands, the input side of the plane (deciding when the assistant should even speak), and the receipt problem (knowing whether a spoken answer actually reached you). Under all of it sits an infrastructure question I keep circling: where durable workflows belong in a layer like this. My early answer is at the boundaries it hands work to, never inside the arbiter. Each of these becomes its own piece as it becomes real.
References
[1] B. Shinn-Cunningham, "Object-based auditory and visual attention", Trends in Cognitive Sciences, vol. 12, no. 5, pp. 182–186, 2008.
[2] M. Porcheron, J. E. Fischer, S. Reeves, and S. Sharples, "Voice Interfaces in Everyday Life", in Proc. CHI Conf. Human Factors in Computing Systems, 2018.
[3] Apple, "Apple introduces Siri AI, a profoundly more capable and personal assistant", Apple Newsroom, June 8, 2026.
[4] "Notifications Overview", Alexa Voice Service Documentation, Amazon (accessed July 2026).
[5] K. Mixter, D. Colish, and T. Nguyen, "Conversation-aware proactive notifications for a voice interface device", U.S. Patent 10,679,608, Google LLC, granted June 2020.
[6] "Email Triage: Lindy automatically sorts, labels, and prioritizes your inbox", Lindy Documentation, 2026 (accessed July 2026).
[7] Voicebox: open-source local voice studio (jamiepine/voicebox), 2026.
[8] OpenAI, "Introducing GPT-Live", OpenAI, July 8, 2026.
[9] "Assign tasks from anywhere in Claude Cowork", Claude Help Center, 2026 (accessed July 2026).
[10] "Kanban (Multi-Agent Board)", Hermes Agent Documentation, Nous Research, 2026 (accessed July 2026).
[11] R. Huang, M. Yarmohammadi, S. Khudanpur, and D. Povey, "Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation", in Proc. Interspeech, 2024. arXiv:2407.10303.
[12] G. Voulgaris, "Voice Conversation ≠ Text Chat + Audio", voulgaris.net, 2026 (accessed July 2026).