Optimizing AI-Assisted Development: Lessons from 900 Sessions

Some engineers, including me, may assume that keeping a long AI coding session alive is more efficient – the model has all the context loaded, the cache is warm, and starting fresh would waste time rebuilding that context. After analyzing 7 months of my own usage data, I found that this intuition is wrong. Long sessions are dramatically more expensive, and the loaded context provides diminishing returns.

This article documents what I learned from 923 sessions across 82 active days using OpenCode , an open-source AI coding agent, processing 2.6 billion tokens. Every session was tracked by the opencode-metrics plugin, which I developed to support this research. It writes session metrics to a local SQLite database and is available as an open-source OpenCode plugin for anyone to use. I analyzed the data using a Grafana dashboard and direct SQL queries against the metrics database.

My goal is to share the patterns and anti-patterns I found so other engineers can make informed decisions about cloud vs local models, session management, and cost optimization – without having to accumulate months of data first.

How Cloud LLM Pricing Works

Before diving into the analysis, it helps to understand the pricing model that drives all the cost patterns in this article.

Token Types and Their Costs

Cloud LLM providers charge per token, but not all tokens cost the same. Here are the token types and their approximate pricing for Claude Opus (Anthropic), the model used in 97% of my sessions:

The critical insight is the 8x price difference between fresh input ($15/M) and cached input ($1.875/M). If the provider can serve tokens from cache instead of processing them fresh, the cost drops by 87.5%.

Note that some prices may change along the time and how they are consumed, so take this is a educated estimation.

Prompt Caching: The Hidden Efficiency

Prompt caching is managed entirely by the LLM provider (Anthropic, Google, OpenAI) – not by the coding agent. It works on prefix matching: if the beginning of your prompt is identical to what was sent in a recent request, the provider serves those tokens from cache instead of reprocessing them.

Here is how it works in a multi-turn session:

Turn 1:  [system prompt + tools + skills]  --> CACHE WRITE  (354K tokens written to cache)
         [user message]                    --> FRESH INPUT  (small)
         [assistant response]              --> OUTPUT       (always fresh)

Turn 2:  [system prompt + tools + skills]  --> CACHE READ   (354K tokens from cache, 90% cheaper)
         [turn 1 messages]                 --> CACHE READ   (extends cached prefix)
         [new user message]                --> FRESH INPUT  (small)
         [assistant response]              --> OUTPUT

Turn N:  [everything from turns 1..N-1]    --> CACHE READ   (grows with conversation)
         [latest user message]             --> FRESH INPUT

The cache has a TTL (about 5 minutes for Anthropic, extended on each hit). As long as you keep interacting with a session within this window, the cache stays warm.

My Cache Numbers

The 99.996% hit rate means virtually every input token after the first turn of each session was served from cache at roughly 1/8 of the full input price. Without caching, my input token costs would have been approximately 8x higher.

What makes sessions cache-efficient:

1. Stable system prompt prefix – the system prompt (instructions, tool definitions, skills, project conventions) is assembled once and stays identical across turns. This is the bulk of the cached prefix.
2. Monotonic conversation history – each turn appends to the conversation, never rewriting earlier messages. This extends the cached prefix on each turn.
3. Session continuity – interacting within the TTL window keeps the cache warm.

The coding agent does not control prompt caching. OpenCode simply sends the
full conversation history on each turn, and the provider recognizes that the
prefix matches a cached version. There is no configuration to enable or
disable it -- it is automatic and transparent.

Why High Cache Hit Rate Does Not Mean Cheap

Here is the counterintuitive part: even with 99.996% cache hits, sessions over 60 minutes cost roughly 30x more on average than sessions under 5 minutes. Cache makes input tokens cheap, but:

• Output tokens are never cached – they are generated fresh every time at full price. As conversation context grows, the model tends to produce more complex (and more expensive) responses.
• Cached tokens are cheap but not free – processing 354K cached tokens per turn still has a cost. Over 200 turns, the accumulated cache read cost for the system prompt prefix alone becomes significant.
• Context growth compounds – each turn adds to the conversation history. Turn 100 sends all 99 previous turns as cached input. The per-turn cache cost grows linearly with conversation length.

This is the fundamental tension: cache makes each token cheaper, but long sessions generate more tokens per turn.

Cloud vs Local Models: A Data-Driven Comparison

Given the costs above, should you run models locally instead? The answer depends on your workload, quality requirements, and infrastructure budget.

The Cost Comparison

For context, a team or individual using Claude Opus for daily coding work with high cache efficiency can expect monthly API costs in the range of a few hundred dollars per active engineer, depending on intensity and session management practices.

A local GPU setup capable of running models for coding tasks:

At the entry level, the amortized local cost (~$64/month) appears cheaper than typical cloud API costs. But this comparison only works if the local model performs equivalently.

The Quality Gap

This is where the comparison breaks down. My workload consists of multi-step architecture decisions, cross-repository analysis, iterative design refinement, and complex code generation across multiple files. These are tasks where model quality directly impacts productivity.

Current open-weight models vs Claude Opus for coding tasks:

The context window is the hard constraint. My system prompt alone is 354K tokens – larger than the effective context window of any consumer-deployable open model. Sessions with this prompt simply would not work on local models without fundamentally restructuring the workflow.

The Hidden Costs of Local

Beyond the quality gap, local inference has costs that API pricing does not:

• No cache optimization – the 90% discount from prompt caching disappears. Locally, you pay full compute for every token on every turn. In my case, caching reduced input token costs by roughly 8x – that entire saving vanishes with local inference.
• Maintenance – driver updates, CUDA compatibility, model weight management, quantization decisions, out-of-memory debugging.
• Availability – your workstation is offline during reboots, updates, and travel. The API is “always” on.
• Opportunity cost – GPU memory and power consumed by inference are unavailable for other tasks.

When Local Models Make Sense

Local models are not universally worse – they excel in specific scenarios:

• Privacy-sensitive code that cannot leave your machine
• Offline work in environments without reliable internet
• High-volume, low-complexity tasks like commit message generation, simple refactoring, or boilerplate code
• Experimentation where you want to try many prompts without cost concerns

The optimal approach is a hybrid setup: use local models (via Ollama ) for exploration and simple tasks, and cloud models for complex code generation. AI coding agents like OpenCode support agent routing – configuring different models for different agent roles (e.g., a small local model for exploration, Opus for code generation). This gives you the cost benefits of local where quality does not matter, and the quality of cloud where it does.

What the Metrics Reveal: Patterns and Anti-Patterns

The following analysis is based on 923 sessions tracked over 7 months (February to September 2026) using the opencode-metrics plugin. Each session records cost, token counts (input, output, cache read, cache write), duration, message count, classification, model, agent type, and project.

The Pareto Distribution

A tiny fraction of sessions drives most of the cost:

The top 3.5% of sessions (those costing more than $10 each) account for 74% of total spending. The bottom 88.7% of sessions (under $5 each) account for only 18.3%. Cost optimization should focus on the expensive tail, not the cheap majority.

Session Duration is the Strongest Cost Predictor

Sessions over 60 minutes account for 83% of all cost despite being only 16% of sessions. The average cost jumps 13x from the 5-60 minute bracket to the 60+ minute bracket. This is the compounding effect of growing conversation context: each additional turn sends all previous turns as input (cached but not free), and the model’s output complexity tends to grow with context length.

Note: most of these 60+ sessions are sessions that were paused and resumed during a period of multiple days. They are not necessarily continuous 60 minutes, but the final pattern is the same regardless of the pauses, except for less cache hits on each pause. See below.

Message Count Directly Correlates with Cost

The jump from 20 messages to 100+ increases average cost by nearly 50x. Sessions with 101-500 messages – the “long working sessions” – account for 60.8% of total spending with only 62 sessions. This is where session hygiene has the most impact.

Multi-Day Session Reuse is Expensive

Reusing a session across multiple days costs 54x more on average than completing work within a single day. This is the most actionable finding in the data. While session reuse avoids cache write costs on the system prompt, the growing conversation history far outweighs that saving. Multi-day sessions (83 sessions, 9% of total) account for 72% of total cost.

Classification Reveals Which Work Types Cost the Most

The openspec-workflow classification (structured spec-driven development sessions) accounts for 71.5% of all cost. These are the long-running sessions where I design, implement, and iterate on features using OpenSpec . They are also the most productive sessions – but their cost is driven by session length, not by the workflow itself.

In contrast, exploration and multi-agent sessions are the cheapest per session. These are short, focused sessions where a sub-agent investigates a specific question and returns an answer. The 35x cost difference between a typical openspec-workflow session and an exploration session shows the value of delegation.

I believe here lays the biggest opportunities to improve. This classification of sessions is pretty customizable in "opencode-metrics" plugin and allows us to bring visibility for sessions that needs more investigation in terms of efficiency. In short, we know where to focus!

Token Efficiency Varies by Work Type

Exploration sessions produce nearly 4x more output per dollar than openspec-workflow sessions. This is because short sessions have smaller conversation contexts, so more of the cost goes to productive output rather than reprocessing history.

Five Principles for Efficient AI-Assisted Development

Based on the metrics analysis, here are five principles that reduce cost without sacrificing quality. They are ordered by impact.

1. One Change, One Session

When you switch from one task or feature to another, start a new session. This prevents “kitchen sink” sessions that accumulate context from unrelated work. A session focused on a single change stays small and efficient.

The data supports this: classification-focused sessions (exploration, implementation, ad-hoc) cost 1-5x the baseline, while mixed long-running sessions cost 35x the baseline.

2. New Day, New Session

Do not resume yesterday’s session. Start fresh. The artifacts on disk – code, specs, designs, task lists – are the persistent context. A new session reads them from the filesystem and resumes with clean context and no history noise.

The data is unambiguous: multi-day sessions cost 28x more on average than same-day sessions. The one thing you need is a sentence of context in your first message:

"Continue implementing the auth-tokens change. Check openspec/changes/auth-tokens/tasks.md for current progress."

That single sentence, combined with files on disk, replaces hundreds of messages of conversation history. The new session reads the actual files (the source of truth) rather than relying on a long conversation that may contain outdated information, abandoned approaches, and debugging tangents.

3. Explore in Sub-Agents, Build in Main

When you need to investigate something – search code, check documentation, analyze a schema, review upstream issues – delegate it to a sub-agent rather than doing it inline in your main session.

The cost difference is dramatic: 284 exploration sub-agent sessions cost 35x less on average than long-running workflow sessions. Doing the same investigation inline in a long session adds to the growing context and increases the cost of every subsequent turn.

AI coding agents like OpenCode support this natively with agent routing. The explore agent can use a cheaper, smaller model while the main build agent uses the most capable model. The delegation is both cheaper (smaller model, shorter session) and higher quality (focused context, no noise from the main task).

4. Use Spec-Driven Development

Spec-driven development frameworks like OpenSpec persist structured artifacts to disk: proposals, designs, specifications, and task lists. These artifacts serve as compressed, structured context that any new session can consume.

This is the key enabler for principles 1-3. Without persisted artifacts, starting a new session means losing context. With them, sessions become disposable – the artifacts carry the intent, the code carries the implementation, and git carries the history. No single session needs to hold everything.

The artifacts are small (typically 2-5K tokens for a complete proposal + design + tasks) compared to the conversation history that produced them (often 100K+ tokens across hundreds of messages). A new session reads 5K tokens of structured context instead of replaying 100K tokens of unstructured conversation.

Spec-driven development scales well. At any given time, you have 1-3
active changes. Completed changes are archived, and their implementation
lives in the codebase. The AI reads the current code and the active
change artifacts -- not hundreds of archived changes.

5. Finish and Merge Today

If a change cannot be completed in a single day, it should be broken into smaller changes. This is both a productivity principle and a cost principle.

The data: sessions spanning 8+ days cost 54x more on average than same-day sessions. The cost compounds because multi-day sessions accumulate context, grow their message count into the hundreds, and lose focus as the scope drifts.

Breaking a 5-day feature into three 1-day changes produces better results:

Each smaller change is independently reviewable, mergeable, and testable. If you are interrupted, the completed changes are already merged – no half-finished mega-change lingering on a branch.

Methodology and Tooling

Data Collection: opencode-metrics

The opencode-metrics plugin is an OpenCode plugin that captures session metrics on every session.idle event. It writes to a local SQLite database with the following schema:

• sessions table: session ID, project, agent, model, classification, title, timestamps
• measurements table: one row per session per metric (cost, tokens_input, tokens_output, tokens_cache_read, tokens_cache_write, cache_hit_ratio, duration_seconds, files_changed, lines_added, lines_deleted, messages_total)
• measurement_deltas table: incremental changes per metric per idle event, enabling accurate daily cost attribution for multi-day sessions
• projects table: project metadata

The plugin also computes derived metrics like cache_hit_ratio = tokens_cache_read / (tokens_cache_read + tokens_input) and classifies sessions automatically based on configurable rules (e.g., sessions using the explore agent are classified as “exploration”).

Visualization: Grafana Dashboard

The metrics are visualized using a Grafana dashboard I built as part of this research and included in ansible-role-ai , a public Ansible role for managing AI development tooling locally. The dashboard runs as an ephemeral Podman container with the frser-sqlite-datasource plugin, connecting directly to the metrics database.

Key dashboard features:

• KPI row: Today’s Cost, This Week, This Month, This Year, Total – with color thresholds derived from a configurable monthly budget
• Cost Overview: Daily cost (from incremental deltas), cost by classification, model, project
• Token Efficiency: Token usage over time, distribution by type, cache hit ratio trends
• Session Analytics: Sessions by day, classification breakdown, duration distribution, agent usage
• Trends: Weekly cost trend, 7-day rolling average

A configurable monthly budget variable (ai_grafana_metrics_monthly_budget) drives all cost threshold colors automatically. Setting your budget (e.g., $100/month) and running the playbook updates every KPI panel’s green/yellow/red thresholds:

Incremental Cost Tracking

A key technical challenge was accurate daily cost attribution. The metrics plugin initially stored only cumulative session cost – the total cost of a session at the time of its last update. For sessions reused across multiple days, the entire cumulative cost was attributed to the day of the last update, not spread across the days the cost was actually incurred.

The solution was a measurement_deltas table that tracks incremental changes: on each idle event, the plugin computes delta = new_value - previous_value and records it with a timestamp. Grafana panels that need daily accuracy (Daily Cost, Weekly Cost Trend, 7-Day Rolling Average, Today’s Cost KPI) query SUM(delta) from the deltas table, while panels that need all-time totals (Cost by Classification, Total Cost, Avg Session Cost) continue using cumulative values from the measurements table.

What Can Be Automated

The five principles above are effective but require discipline. Some of them can be enforced through tooling, reducing the cognitive burden on engineers.

Already Automated

• Model routing: Agent configuration routes different task types to appropriate models. Exploration uses a cheaper model, code generation uses the most capable model. This feature in ansible-role-ai was inspired by Jay Flowers’ gist on per-agent model routing. It is a one-time configuration with no ongoing discipline required.
• Budget monitoring: The Grafana dashboard with budget-derived thresholds provides passive cost awareness. Engineers see red/yellow/green indicators without actively checking.
• Classification: The opencode-metrics plugin auto-classifies sessions based on agent type and workflow patterns, enabling the analytics breakdowns shown in this article.

Should Be Automated Next

These are opportunities for the AI coding agent ecosystem:

• Session length warning: A plugin or core feature that displays a gentle notification when a session exceeds a configurable message count (e.g., 100 messages). Not blocking – just a reminder: “This session has 100+ messages. Consider starting fresh for better performance and cost.”
• Stale session detection: When resuming a session that has been idle for more than N hours, suggest starting a new one: “This session has been idle for 14 hours. Start fresh? Artifacts are preserved on disk.”
• Live cost indicator with budget context: The agent’s status bar could show the current session cost with a color threshold based on the daily budget pace, making cost awareness continuous rather than dashboard-based.

The best engineering practices are the ones that do not require remembering a rule. Every principle that can be encoded as an automated check or default behavior removes a failure mode.

Conclusion

The single most important finding in this analysis is that session management drives cost more than model choice, caching, or any other factor. The 28x cost multiplier between same-day and multi-day sessions is a structural effect that no amount of prompt optimization can offset.

The five principles – one change per session, new day new session, explore in sub-agents, use spec-driven development, finish and merge today – are actionable immediately with zero tooling changes. They are grounded in 7 months of real usage data across 923 sessions, not theoretical optimization.

For the cloud vs local question: at current pricing with prompt caching, cloud models offer a compelling value proposition for quality-sensitive coding work. The combination of 99.996% cache efficiency, 200K+ context window, and frontier model quality is difficult to replicate locally at any consumer hardware price point. Local models remain valuable for privacy, offline use, and high-volume low-complexity tasks – the hybrid approach (local for exploration, cloud for code generation) captures both benefits.

Tools like opencode-metrics and the Grafana dashboard make cost awareness passive. You do not need to actively monitor spending – the dashboard’s budget-derived thresholds surface anomalies automatically. The data collection happens in the background via the OpenCode plugin system, and the visualization is a single opencode-grafana start command away.

The bottom line: write smaller changes, keep sessions short, delegate investigations, and let the artifacts on disk be your persistent memory. The conversation is ephemeral. The code and specs are permanent.

References