Tool Sla Quality Limits

An agent relies on a tool that advertises a specific accuracy figure for an ML-based capability — an entity-extraction API claiming 95% precision, a classification model claiming 90% F1 — and treats results as trustworthy at that advertised rate. In production, real-world accuracy often falls short of the marketed number because vendor benchmarks are measured on curated test sets that don't reflect the agent's actual input distribution. The agent, having no independent accuracy monitoring, keeps trusting outputs at the assumed rate and propagates a higher error rate downstream than anyone accounted for.

A tool vendor experiences an internal incident — an overloaded backend, a partial regional outage, a resource-constrained fallback mode — and quietly degrades service quality (higher latency, lower accuracy, reduced feature availability) to keep the system technically "up," without posting to a status page or notifying API consumers. The agent has no explicit signal that anything has changed; it just observes worse results and, absent any error or status indicator, has no basis to distinguish a genuine data or logic problem from a vendor-side degradation it should be working around.

A tool's documented latency SLA (e.g., "p99 under 500ms") is regularly exceeded in actual production traffic, and the agent's own timeout and retry logic — tuned to trust that documented figure — fires prematurely relative to the tool's real behavior, or conversely the agent's own downstream SLA commitment to its users gets breached because it inherited an unrealistic latency assumption from the upstream tool. Either way, the mismatch between advertised and actual latency propagates as a reliability problem that looks like the agent's own bug.

An agent depends on an ML-powered tool (a classifier, a recommendation engine, a scoring API) whose underlying model the vendor updates server-side — a retraining, a new model version rollout, a fine-tuning change. Because the API contract (request/response shape) usually stays the same across a model update, nothing about the integration breaks; the model simply starts producing systematically different, and sometimes measurably worse, predictions for the agent's specific use case, with no changelog entry, version number bump, or notification distinguishing "same API, different model behavior underneath."

A tool's actual uptime falls short of its advertised availability SLA (e.g., "99.9% uptime" translating to roughly 43 minutes of allowed downtime per month, but real outages exceeding that budget). The agent, built with no fallback path because the SLA implied outages would be rare and brief, treats every outage as an unexpected, unhandled condition — retrying blindly, failing the entire user-facing workflow, or queuing work indefinitely — rather than having a designed response for a scenario the SLA math said should barely ever happen but that occurs often enough in practice to matter.