How to measure and improve instrumentation quality for better full-stack observability

Modern engineering teams instrument everything, with metrics, logs, traces, and profiles flowing from hundreds of services at once. But full-stack observability isn’t really about collecting more telemetry; it’s about having a single, unified picture of how your services connect to every layer beneath them, including their dependencies, the pods and nodes they run on, and the logs, traces, and profiles that explain their behavior. 

But there’s often a quiet problem hiding underneath all that data: not all instrumentation is created equal, and every gap silently breaks one of those connections.

One service, for example, might emit metrics but no logs, so when you pivot from “this is erroring” to “show me why,” you hit a dead end. Another might have logs but an invalid service.name or a missing k8s.pod.name that breaks correlation, dropping it out of the graph and away from its pods and nodes. A third service might look perfectly healthy right up until an incident, when you discover its traces were never wired up and the trail goes cold exactly when you need it most.

To fix this, Grafana Cloud’s Knowledge Graph now includes an instrumentation quality report: an automated, continuously updated assessment of how well each of your services is instrumented—and, in effect, how they plug into the full-stack picture.

In this post, we’ll walk through how to read the instrumentation quality report, how the scoring works, and how to use it to systematically raise the observability bar, keeping every layer of your stack joined up across every service you run.

What is instrumentation quality?

Instrumentation quality is a measure of how complete and correct the telemetry for a given service is, judged against a set of automated checks.

Each service is evaluated by a server-computed set of quality checks: small, focused rules that validate one specific thing about a service’s telemetry. 

A few examples:

• Does the service emit logs?

• Are service graph metrics present?

• Is the service name well-formed (no stray slashes, valid service.namespace)?

• Are the expected Kubernetes labels attached so the service can be correlated with its pods, nodes, and cluster?

Other checks cover span metrics, profiles, metric cardinality, and more, each one validating a different signal or connection.

It helps to read that list not as a compliance checklist, but as a web of connections. Service graph metrics link a service to its upstream and downstream dependencies. Span metrics and traces reveal the request path across the entire call chain. Kubernetes labels anchor it to the pods, nodes, and clusters it runs on. Logs and profiles tie it to its own error lines and hot code paths. And a well-formed service.name is the identity that holds all of it together in the graph. Miss one, and a whole layer of the full-stack picture quietly disappears.

Based on the status of these checks, each service gets a single quality score, mapped to a human-readable tier so you can tell at a glance whether a service is well-instrumented or needs attention.

A rising score, then, isn’t just a tidier telemetry checklist; it’s a measure of full-stack visibility, and how much of your stack you can actually traverse when it matters.

Why use Grafana Cloud to measure instrumentation quality: an example

Other tools tell you whether a service is emitting data. Through the instrumentation quality report, Grafana Cloud’s Knowledge Graph tells you whether that data is good enough to troubleshoot with and what to fix when it isn’t.

A day on call

Let’s say a checkout service starts breaching its latency SLO. Its instrumentation quality report is green: service graph metrics, Kubernetes labels, logs, and traces are all present, so the connections you need to investigate the issue are intact.

You follow the service graph to a slow payments dependency. From there, the Kubernetes labels connect the payments service to the node it’s running on, where you see that CPU is saturated. You can then jump to the service logs to find timeout errors and inspect a trace to confirm the root cause. Checkout → payments → node → logs → trace. Because the telemetry is properly instrumented and connected, you can follow the problem across the stack without hitting a dead end.

A sibling service that same week tells the opposite story. It shows up in metrics but nowhere in the graph: no dependencies, no neighbors, and the trail goes cold almost immediately. The instrumentation quality report explains why: a stray slash in its service.name has dropped it out of the graph, and the service isn’t emitting logs. With those two broken connections, the full-stack picture simply can’t be drawn. 

The only real difference between the two incidents isn’t how much telemetry each service produces; it’s whether the connecting signals are there. The quality report makes those missing connections visible, so you can fix them long before the next 3 a.m. page. 

Find and fix instrumentation gaps 

The instrumentation quality report turns a vague “improve observability” mandate into a prioritized to-do list. The Instrumentation quality tab scores every service and shows its failing checks, so you can quickly identify which services need the most attention. And every finding is actionable rather than just diagnostic: each failing check comes with details about the impact, a link to the exact setup docs, and, where applicable, an Explore query button to confirm the gap in seconds.