Speculation

Speculation is a way to make an agent feel faster by predicting what it's about to do next and getting a head start on it - similar in spirit to branch prediction in a processor. It's optional, controllable at every level, and defaults to the cheap, safe setting.

The two modes

• Reads (the default). The system predicts an agent's next read-only action - opening a file, checking a calendar - from the pattern of what it's already done, and warms that read ahead of time. This mode never spends any model tokens on a guess, so there's no cost risk: it either saves time, or it doesn't help and costs nothing.
• Turns. When a tool's result is genuinely predictable, the system can go further and start the next model response speculatively, before the tool call it depends on has even finished. If the guess turns out right, you get the answer sooner; if it turns out wrong, the speculative work is discarded ("squashed") and redone properly. This mode does spend tokens on guesses, including ones that get thrown away, so it trades some extra cost for speed.

Staying in control of cost

Turns-mode speculation is never on by default for a new agent, and it respects limits at every level: your own account has a master switch with a kill switch and a running total of what speculation has cost you this month, and (on a team) an admin can cap or forbid it entirely, including by model. Whichever limit is strictest wins.

If you want the speed benefit without any spend risk at all, reads-mode alone already gets you most of it - turn on turns-mode only for agents where a slightly wrong guess is cheap to redo.

Predicted tools

The assistant loads app tools as it needs them. With predicted tools on, it also guesses - before the request starts - which tools you are likely to need, and starts with those already loaded. The guess runs in your browser from the text of your request, and can be worked out while you are still typing; nothing you type is sent anywhere before you press send.

A wrong guess never removes anything: the assistant can still load any other tool it needs, exactly as it would without predictions. Predicted tools are rolled out gradually, so you may not see them yet; once they are on for you, you can switch them off in Settings › General › Early features.

Two guessers work together. A keyword matcher is there from the start. A small trained model (about 50 MB, downloaded once in the background the first time predictions are on for you and kept in the browser's cache) then learns what requests sound like for every tool and app - including apps it never saw in training - and its ranking is combined with the keywords'. Until it has downloaded, the keywords guess alone; it runs on your graphics card when the browser allows it and on the CPU otherwise, and takes a few milliseconds either way.

When predicted tools and turns-mode speculation are both on, a speculative reply is also discarded if the assistant loaded a new tool before the guess resolved - the reply was written without that tool, so it is redone with it.

What's next

• Set an org-wide ceiling on speculation if you're managing a team.
• Watch your spend to see how much speculation is actually costing.