Coding agent benchmarks: what actually won?
A coding agent's tools and testing policy help determine its benchmark result. Understand what was compared before choosing a setup for your team.
Guides, news and explanations about AI and reliable systems.
A coding agent's tools and testing policy help determine its benchmark result. Understand what was compared before choosing a setup for your team.
Anthropic cut live internet access from internal evaluations after agents acted on real systems. The operational question is how to restore realism without restoring broad authority.
Claude Haiku 5.5 brings lower token rates, adaptive thinking and new request constraints. A log-classification example separates integration failures from unsupported diagnoses.
Anthropic’s October 7 credits add a Console-funded path for eligible subscribers. Subscription access remains, while scheduled agents still need task budgets spanning retries and verification.
Current Sonnet 5.5 docs list lower cache-read pricing. Review thinking and tool-call changes, then compare the full cost of an accepted task.
Anthropic reports lower Opus task costs and new preserved-thinking behavior. A runbook exception provides a concrete comparison of the model’s reasoning and integration requirements.
What prompt-cache hits save once retries and failed work are included in task cost, with a local calculator and ledger.
Claude’s on-demand compaction lets an application replace older conversation history with a signed summary. A conditional restart shows why the summary must preserve relationships between instructions.
Claude Enterprise’s Compliance API beta now includes Chrome-session transcripts. Tool exchanges help locate an attempted action; the destination’s records establish whether it took effect.
A rollback exception tests whether Opus 4.6 preserves decisive evidence across a long session, and whether the resulting answer arrives in time to help.