Find articles, news, tools and templates by subject or by the work you need to do.
113 results for “Improve on-call workload” · 25–36
Guide · AI in production ·
What a model does when you ask a question, how we got here, and how to choose between local and hosted inference without confusing fast text with useful work.
Read guide: Model inference: what happens when you ask AI a question?Guide · AI in production ·
Connect logical tasks to every model and tool attempt, then measure cost and latency per accepted outcome without hiding failed or unknown work.
Read guide: Measuring retry amplification in AI agent workflowsNews · Platform engineering ·
Maia 300 joins chiplets, memory locality, networking, software, power and cooling. The operational test is accepted work under the same demand.
Read news: Microsoft Maia 300 makes inference efficiency a systems problemExplainer · AI in production ·
Compare reference and speculative paths for token-level equivalence, then run a separate capacity trial so speed cannot conceal a serving-contract change.
Read explainer: Bit-exact decoding turns inference speedups into testable changesExplainer · AI in production ·
Measure cached input against total input, locate the prefix miss boundary, and judge cache changes beside latency, task cost and correction work.
Read explainer: Long agent sessions need a prompt-cache SLIExplainer · Platform engineering ·
Basalt narrows local inference to Qwen3.8-Flash-Next on Blackwell. Its memory and concurrency controls matter more than copying the largest throughput number.
Read explainer: What Basalt changes in a Blackwell local inference stackGuide · Observability ·
Run a candidate log classifier beside the current route, preserve complete outcomes, and promote only the slices whose mistakes and review cost are understood.
Read guide: Shadow-testing an AI log classifier before migrationGuide · Observability ·
A client-side SSE probe separates first answer text from normal completion. Fixed arrivals and outcome counts show whether a serving change delivers more useful answers.
Read guide: Measuring streaming LLM latency from first text to completed workGuide · AI in production ·
Instinct brings personal tasks into connected services. A hypothetical booking and notification worker show how permission, provider confirmation and uncertain results affect what happens next.
Read guide: Instinct: from a text or call to a confirmed taskNews · AI in production ·
Google's new image model rejects familiar generation settings. Follow a thumbnail job through its outgoing request, retries and delivery to evaluate the upgrade.
Read news: What Nano Banana 2.1 changes in an image-generation serviceNews · Platform engineering ·
Google API Gateway’s streaming preview delivers response chunks while a model is still generating. Creation-time settings and request deadlines shape the migration.
Read news: Google API Gateway adds streaming for LLM responsesGuide · AI in production ·
OpenShell can stop an agent after Kubernetes has accepted its scaling request. Recovery then depends on the Deployment state, available replicas and customer-facing result.
Read guide: Recovering an agent task after OpenShell containment