AI Coding Guide

AI-Assisted Software Engineering

Context Management

By Richard Osborne, CTO at Visual Hive

Last updated:

TLDR

AI context is expensive and degrades with length. One task = one conversation. When in doubt, write a task doc and start fresh. Your documentation carries context across sessions — chat history doesn't need to.

How Context Works

Every AI conversation has a finite context window. Every message you send includes the entire conversation history — which means each message costs more than the last. A 100-message conversation is expensive. An 80-message conversation with stale debugging tangents is both expensive and produces worse output.

Your documentation is persistent. Chat history isn't. Design accordingly.

The Task-Conversation Rule

One task = one conversation. This isn't just about cost. It's about clarity. A conversation that starts fresh with a clear task spec produces better output than one carrying 40 messages of prior context, half of which is no longer relevant.

When to Start Fresh Mid-Task

Start a new conversation when:

  • You've been going in circles for 3+ exchanges on a bug or approach
  • AI keeps referencing something that's no longer true from earlier in the conversation
  • A side issue has taken over the conversation from the original task
  • The conversation is over 60 messages and you're not done yet
  • AI contradicts itself between messages

The Context Rescue Pattern

When stuck mid-task, before starting fresh:

We're going in circles. Before starting a fresh conversation,
can you write a task document capturing:
1. What we were trying to accomplish
2. What we've tried so far
3. What the current state of the code is
4. What the next steps should be

I'll use this to start a fresh conversation.

This preserves the useful context from the current conversation without carrying the noise. The task doc goes in your project docs and becomes the starting point for the fresh session.

What Your Docs Carry (Not Chat History)

  • Architecture decisions → ARCHITECTURE.md
  • Discoveries and gotchas → LEARNINGS.md
  • Quality rules → .clinerules
  • Current sprint state → Sprint plan (update task status)
  • Solutions to problems → LEARNINGS.md (add as you go)

If something is worth remembering across sessions, it goes in a doc. If it's not worth documenting, it's probably not worth carrying in context either.

Context Cost Control

Practical cost management:

  • Don't add entire files to context unless AI needs them for the task
  • Reference docs by path when AI doesn't need the full content
  • For large refactors, split into smaller tasks rather than one massive context
  • Use @ references in Cline to add specific files rather than the whole repo

The overhead of fresh conversations (re-reading docs) is typically 5–10k tokens. Carrying unnecessary context in a long conversation costs 30–80k+ tokens per exchange. Fresh conversations win on cost and quality.

Building something with AI?

Talk to Visual Hive →