Evals Prove Your Agent Can Work. Tracing Is How You Find Out Why It Didn't.

Evals answer a different question than the one you have in production

An eval suite tells you whether an agent can complete a task under conditions you anticipated. That's a necessary check and we've written about why "it feels smart" isn't a substitute for one. But once an agent is live, the question changes. It's no longer "does this pass our test set" — it's "why did this specific run, for this specific customer, at 2:14pm on a Tuesday, do the wrong thing." Your eval suite has no opinion on that run. It never saw it.

This is the gap a lot of teams discover the hard way: an agent clears its evals, ships, and then someone asks why it refunded the wrong order or looped three times calling the same tool before giving up. The honest answer, for most teams six months ago, was "we don't know — we'd have to reproduce it." That answer is what tracing exists to remove.

What a trace captures that a pass/fail score doesn't

A trace is the full, ordered record of what an agent actually did on one real run: every model call, every tool invocation, every retrieved document, every retry, with inputs, outputs, token counts, and latency attached to each step, nested in the order they happened. The difference between "the agent failed" and "the agent called the pricing tool with a stale currency parameter, got a plausible-looking wrong number, and never cross-checked it" is the difference between an eval score and a trace.

The practical failure mode teams run into without this: an APM dashboard built for web requests shows exactly one or two spans for an entire agent run, because the agent framework reports itself as a single HTTP call. Everything that happened inside — the planning step, the three tool calls, the retrieval, the self-correction loop — is invisible. You can see that a request took 11 seconds and returned something wrong. You cannot see which of the internal steps caused it. Teams instrumenting agents for the first time consistently describe this same surprise: the thing they thought they were monitoring was a black box with a timer on it.

The standardization fight you should know about before you pick a stack

As of this year there are two competing ways to describe what's happening inside an LLM call in telemetry data. OpenTelemetry's GenAI working group (active since April 2024) has been extending the standard gen_ai.* attribute namespace — things like gen_ai.usage.input_tokens and gen_ai.agent.name — to cover agent orchestration and tool calls, not just single completions. OpenInference is a separate, parallel convention that predates full agent support in OTel and is what a lot of existing tracing backends actually implemented first.

Neither has fully won. The attribute names don't match between them, which means a dashboard built against one won't correctly read spans emitted in the other, and some of the newer fields (evaluation scores attached to spans, cost attribution across cached vs. fresh tokens) are still being finalized rather than locked. If you're instrumenting an agent today, check which convention your framework and backend actually emit before you build dashboards on top — don't assume gen_ai.* just because it's the vendor-neutral option; plenty of production stacks are still on OpenInference.

Picking a backend, briefly

You don't need to solve observability theory to ship this. Three backends cover most real decisions:

• LangSmith is the default if you're already deep in LangChain or LangGraph — tightest integration, least setup. The tradeoff is real lock-in: it's built around that ecosystem's abstractions, and ripping it out later costs more than it looks like on day one.
• Langfuse is MIT-licensed, self-hostable, and deployment-agnostic — the right pick if you want your trace data on your own infrastructure or you're not committed to one agent framework. It was acquired by ClickHouse in early 2026; the open-source offering hasn't changed as a result, but it's worth confirming current terms before you commit.
• Arize Phoenix is OpenTelemetry-native and runs locally in a container, which makes it a good standards-based debugging tool. Its license is Elastic License 2.0, not a fully permissive open-source license, and production alerting requires their commercial product — fine for debugging, a real constraint if you want self-hosted alerting for free.

None of these replace evals. They answer different questions, and you need both: evals before you ship to decide if the agent is good enough to trust with real traffic, tracing after you ship to find out what actually happened when it wasn't.

What this actually requires you to build

Tracing isn't a dashboard you bolt on later — it has to be designed into the agent's architecture. Every tool call needs to emit its inputs and outputs, not just "tool X was called." Every retrieval step needs the actual documents returned, not just a hit count. Every retry or self-correction loop needs to be a visible step, not folded into the parent span's latency. If you build the agent first and try to add this after a production incident, you'll find the exact step you need to see was never recorded, because nobody thought to instrument it before it mattered.

The teams that handle this well treat trace instrumentation as part of the agent's spec from day one — the same way they'd treat logging for a payment service — not as an afterthought bolted on after the first bad incident makes the gap obvious.

Takeaway

An eval tells you an agent is good enough to ship. A trace tells you what it actually did on the run that just failed. If you only have the first, every production incident starts with "let me try to reproduce this" — and some of them you never will.