Clean Slate

Clean Slate

Intent

Clear out the model’s context before continuing a task, or starting a new task.

Motivation

In transformer models, including large language models (LLMs), the self-attention layer transforms the input text stream first into token vectors, then assigns weights to the input tokens in the model’s context, its working store. The model uses these weights to identify the relative importance of items in the input data, and correlate input tokens even if they appear in distant sections of the input stream.

LLMs can process a large amount of input data using a large “context window”, however, increasing the context size comes with multiple costs. Modern approaches make the memory use of the attention layer scale linearly with context length, but the time required scales quadratically with context length. Put another way, doubling the amount of data in the context (in training or inference) causes the time required to calculate attention to increase by a factor of four.

Even if processing the context doesn't take too long, having lots of data in the context window can lead to undesirable behavior.
Models demonstrate positional bias, in which the location of data in the context contributes to the weight that the attention layer calculates. The model can assign too high a score to information in the context that you might consider irrelevant to the task it’s performing, which has the effect of “anchoring” the generated output to insignificant or unrelated input. Conversely, when the self-attention layer assigns too low a score to information in the context that you consider important, the model “forgets” this important information and generates output that doesn’t account for it.

Reset the model’s context between tasks to reduce the time it takes to perform inference tasks, and to ensure that the attention layer only operates on relevant input.

Applicability

You usually use Clean Slate in interactive sessions with coding assistants, when you determine that the model is “forgetting” or “ignoring” important information or focussing on irrelevant details, or when you want to change the task you work on with the assistant. Combine Extract Prompt or Record Prompt with Clean Slate to create sub-agents that work on isolated parts of an overall task with a separate context.

Clean Slate is hard to apply in unattended, “agentic” contexts, because you typically don’t watch the model generate output and only see the end result, and because the agent has a particular goal that you can’t change while it’s in progress. When You Review the output, you can use the agent’s transcript to identify whether attention problems lead to issues with the results you get, and Extract Prompt or divide the overall goal into Baby Steps that you design multiple agents to achieve together.

Consequences

Clean Slate offers the following benefits:

• Continue a session with a coding assistant across multiple tasks.
• Take control over the information in the model’s context.
• Decrease the time the model requires to generate results.
• Avoid inattention, or attention to irrelevant information.

Implementation

You can always get a Clean Slate by starting a new interaction with the model, optionally quitting the tool in which you have an existing interaction with its built-up context.

Additionally, some coding assistants and other tools have built-in facilities for resetting context. In IDEs, this is typically a “Start New Task” or “New Conversation” button. In command-line interfaces, you type a special command like /clear or /reset.

Sub-agents provide a specialized version of a Clean Slate, in tools that support them. You supply a separate system prompt for the sub-agent, and you describe your task in a user prompt to the “main” agent (the agent that you interact with through the tool’s UI). The main agent generates one or more user prompts that it uses to prompt the sub-agent.

In this interaction, both the main agent and the sub-agent gain the benefits of Clean Slate. The intermediate information generated in the sub-agent’s response to its prompt doesn’t populate the main agent’s context, and the sub-agent only receives its system prompt and the generated user prompt, it doesn’t have other information from the main task in its context.

Example

Fixing Errors in a File

In a long-running session with the Devstral 2 model in Mistral Vibe, the agent frequently made mistakes when the model generated search_replace tool use to change the content of the file. It repeated content, wrote its boundary markers to the file, or searched for content that didn’t exist in the file.

I reviewed its output and hypothesized that the model was using outdated versions of the file’s content from its context, and generating tool uses that didn’t directly work with the current state of the file.

I typed /reset to get a Clean Slate, and prompted the agent to read the file, check if the syntax was valid, and correct it. It returned the file to a consistent state on the next use of the search_replace tool.

Related Patterns

Combine Clean Slate with Extract Prompt or Record Prompt when you create sub-agent definitions.

Use Clean Slate between Baby Steps in a large task, or a process with multiple steps.

Stop and Plan, then create a Clean Slate when you move from planning to implementation.

You need a Clean Slate when you Start Over to ensure the model doesn’t pay attention to details from your earlier, failed attempt at your task.