Find articles, news, tools and templates by subject or by the work you need to do.
137 results for “Set a release policy” · 49–60
Explainer · AI in production ·
Compare reference and speculative paths for token-level equivalence, then run a separate capacity trial so speed cannot conceal a serving-contract change.
Read explainer: Bit-exact decoding turns inference speedups into testable changesGuide · AI in production ·
Pin the model artifact, template, runtime and hardware, then exercise fixed protocol tests and preserve raw differences before approving a serving change.
Read guide: Build an inference compatibility suite before changing the runtimeExplainer · AI in production ·
Measure cached input against total input, locate the prefix miss boundary, and judge cache changes beside latency, task cost and correction work.
Read explainer: Long agent sessions need a prompt-cache SLINews · AI in production ·
Claude Haiku 5.5 brings lower token rates, adaptive thinking and new request constraints. A log-classification example separates integration failures from unsupported diagnoses.
Read news: Haiku 5.5 lowers prices and changes what a log classifier must handleExplainer · Platform engineering ·
Infermeld makes a mixed-GPU llama.cpp launch inspectable. Its recorded failures show why separate device allocations and completed workload checks matter more than adding VRAM capacities.
Read explainer: How Infermeld runs one model across AMD and NVIDIA GPUsGuide · Observability ·
A client-side SSE probe separates first answer text from normal completion. Fixed arrivals and outcome counts show whether a serving change delivers more useful answers.
Read guide: Measuring streaming LLM latency from first text to completed workExplainer · AI in production ·
Kimi K3 can keep a larger incident record in view. Its practical value depends on evidence handling, agent integration and the cost of useful investigation.
Read explainer: What Kimi K3’s million-token context offers SRE teamsNews · AI in production ·
Search results from Ceramic.ai, Exa and Linkup can enter an agent workflow through one gateway, while source versions and provider terms still need inspection.
Read news: Cloudflare adds web search to AI GatewayGuide · Observability ·
Follow a slow AI request from Cloudflare to an instrumented origin, configure sampling, and verify that the trace preserves the evidence needed to investigate.
Read guide: Tracing an AI request through Cloudflare and its originGuide · Observability ·
Build a latency SLI from Prometheus histogram buckets, include failed requests, and compare classic and native queries with a worked search-service example.
Read guide: Measuring a latency SLO with Prometheus histogramsNews · Observability ·
Apigee separates model latency and token use from MCP tool traffic. Tool filters narrow a slow-assistant investigation, while proxy policies determine which charts populate.
Read news: Apigee adds model and MCP tool performance dashboardsGuide · Observability ·
Tail sampling selects recorded traces by error, latency and other evidence. A checkout timeout shows how routing, waiting time and memory affect the trace a responder can retrieve.
Read guide: OpenTelemetry tail sampling through a checkout failure