Find articles, news, tools and templates by subject or by the work you need to do.
91 results for “Evaluate an agent” · 49–60
Guide · Observability ·
Zero errors, no traffic and absent series produce different PromQL results. Synthetic tests show how ratios, fallback zeros and absence checks behave for a checkout service.
Read guide: Why a Prometheus error-rate query can return no dataGuide · AI in production ·
An unchanged prompt prefix can reuse model computation. A worked handoff comparison and Python calculator account for cache charges, retries and the reports reviewers accept.
Read guide: What prompt caching saves in an incident-handoff workflowGuide · AI in production ·
A rollback exception tests whether Opus 4.6 preserves decisive evidence across a long session, and whether the resulting answer arrives in time to help.
Read guide: Evaluating Claude Opus 4.6 for incident workGuide · AI in production ·
An approved summary task can take a new data path when a connector is added. Account controls, retention and output sharing apply at different points along that path.
Read guide: Where workplace ChatGPT data goesCommentary · Engineering leadership ·
A misrouted support ticket shows how automated decisions affect customers and what a usable correction path requires from the people responsible.
Read commentary: Who puts an AIOps mistake right?Guide · AI in production ·
Follow a weight through a training update, save a tiny model, and learn how checkpoints, fine-tuning, LoRA and held-out evaluation fit together.
Read guide: What are model weights, and how does training change them?Guide · Incident response ·
Build a local, synthetic citation check that exposes wrong-version evidence while keeping interpretation, source freshness and action permission separate.
Read guide: Checking AI incident-summary citations with PythonNews · AI in production ·
Separate distributed inference from a contained replication test, then match an edge-fleet containment claim to the evidence that can establish it.
Read news: What Anthropic actually tested in its AI self-replication studyNews · Platform engineering ·
Maia 300 joins chiplets, memory locality, networking, software, power and cooling. The operational test is accepted work under the same demand.
Read news: Microsoft Maia 300 makes inference efficiency a systems problemNews · Platform engineering ·
The October 12 OCP release spans rack designs, facility specifications, open-source operations software and planned optics. Operators should test each layer against workload evidence.
Read news: NVIDIA opens more of the Vera Rubin infrastructure stackExplainer · AI in production ·
Compare reference and speculative paths for token-level equivalence, then run a separate capacity trial so speed cannot conceal a serving-contract change.
Read explainer: Bit-exact decoding turns inference speedups into testable changesGuide · AI in production ·
Pin the model artifact, template, runtime and hardware, then exercise fixed protocol tests and preserve raw differences before approving a serving change.
Read guide: Build an inference compatibility suite before changing the runtime