Why long AI conversations get expensive — and slower
Every message you add to an existing thread costs more than the last. On a metered plan, that cost is money. On a flat monthly plan, it is speed and quality instead.
Starting a new chat is almost always the right call. The longer you add to an existing thread, the more the AI has to re-read before it can respond — and the cost of that grows faster than most people expect.
On a metered API plan, the cost is literal: you pay per token, and every exchange in a long thread is billing you for the full history. On a flat monthly subscription — ChatGPT Plus, Claude Pro — the cost arrives as something different: slower responses, more compressed context, and output that drifts from what you actually told it earlier in the conversation.
Why the AI re-reads everything, every time
AI does not have persistent memory between messages the way a person does. Each time you send a message, the model reads from the beginning of the thread: your first message, every response since, any documents you attached, all of it. The response you get is based on everything it has just re-read.
In a short thread, this is fine. Ten exchanges is a small amount to process. But in a thread that has been running for days across dozens of exchanges, the model is now re-reading thousands of words before it can produce a single sentence. The input gets longer with every message. The processing cost accumulates.
What compaction does — and does not solve
AI systems use a process called compaction to manage very long threads. When a conversation grows past a certain length, the system compresses earlier history into a summary, reducing the volume the model has to process.
This helps with cost. It does not help with accuracy. Compressed context is lossy — a summary of what was said earlier is less precise than the original. Instructions given early in a long thread may be summarised away or slightly mis-stated. The model's understanding of what you established at the start degrades over time, which is why long-running threads often feel like the AI has started to forget things. It has, in a sense.
The numbers from agentic coding work
The most precise data on compaction cost comes from long-running agentic tasks — where an AI agent executes code, tests, and iterates in a continuous loop without you sending manual messages. This is a more extreme case than a typical chat, but the mechanism is the same.
In Sean's own work, tasks that had gone through more than 50 compaction cycles were costing 472 times more in tokens per response than a fresh task on the same project. The longest task in those sessions ran 115 compaction cycles and processed 788 times the token load of a median fresh task.
Those figures are specific to agentic coding loops, not casual chat. But the direction is the same for any long thread: cost and overhead compound as the conversation ages. The scale is just more dramatic when the AI is running autonomously across hundreds of exchanges.
What this means on a flat monthly plan
If you pay a monthly subscription rather than by the token, you will not see a larger invoice from a long thread. What you will see is the AI getting slower, its context window filling up, and — if the thread runs long enough — earlier context being dropped entirely.
The quality degradation is the cost. Instructions you set up carefully in message three may be effectively gone by message forty. The AI is not ignoring them; it is working from a compressed version of them. That distinction matters less than the outcome, which is that long threads produce worse output, reliably.
The handover trick
The practical fix is to start a new thread before the thread gets old — but not to lose the context you have built up.
Before closing a long thread, ask it to write a handover: a clean summary of everything it knows about the task, written as if briefing someone who has not seen the previous conversation. Keep this in a file. Start a new thread, share the handover, continue the work.
The new thread opens with all the relevant context, none of the accumulated overhead, and none of the compaction history. The cost — in speed, quality, or money — resets to the fresh-task baseline.
What usually goes wrong
Most people keep threads running because continuity feels safer than starting over. It is not. A well-written handover document carries the meaningful context. What it leaves behind is the accumulated noise: the earlier drafts the AI half-remembers, the directions you walked back, the context that was summarised into something imprecise.
The other failure is waiting until the thread is already degrading before acting. By then, the handover the AI writes is itself built on compressed context — less reliable than one written while the thread is still fresh. The better habit is to start a new thread after each distinct piece of work, not when the current one becomes noticeably worse.
When to keep the thread going
Not every conversation needs a reset. A short exchange — asking a question, getting an answer, refining once or twice — has no compaction problem. The overhead only matters when threads are long enough to trigger compaction, which in practice means multi-day or multi-hour sessions with many exchanges.
The simple test: if you have sent more than 20 or 30 messages in a single thread, and the work is still ongoing, consider whether a handover and fresh start would serve you better than continuing. The answer is usually yes.
The other case where a long thread is fine is when you genuinely need the AI to hold a continuous chain of reasoning — one where the earlier messages directly inform every later one, and where the context cannot be easily summarised without losing something. These threads exist. They are much rarer than the threads people think require continuity, which are usually just habit.
Drawn from chapter 2.6 of AI Magic 2033.
