How to generate real-world load tests using Grafana Cloud k6 and production telemetry

For many development teams, a load test starts with a set of assumptions. 

You pick 100 virtual users because it sounds reasonable. You ramp for 30 seconds because that’s what the tutorial showed. You set a 500ms threshold because it feels like a good target. The test passes, you ship the release, and production falls over at 6 p.m. on a Tuesday because your synthetic load never resembled how real users interact with your application. 

The good news is that, if you’re running Grafana Cloud, you already have the data you need to run realistic load tests. Your dashboards capture how your users behave, including request rates, latency distributions, and traffic patterns over time. The signal is already there; you just need to connect it to your test configuration.

Using production telemetry to shape your tests makes the results far more meaningful. Instead of validating against a hypothetical workload, you’re testing against patterns your systems already experience, leading to more reliable baselines, better thresholds, and fewer surprises in production. 

This post walks through how to do exactly that: pull real telemetry signals from Grafana Cloud and translate them into a Grafana Cloud k6 testing scenario that reflects production environments (rather than assumptions). 

VU count isn’t the only place to start

Virtual user (VU) count is an important part of how Grafana Cloud k6, the fully managed performance testing platform powered by k6 OSS, runs tests. However, VU count doesn’t always need to be the first lever you pull. When your goal is to test how a service behaves under a known request rate, arrival-rate executors let you express that intent more directly.

What you may care about is the arrival rate, or how many requests per second your system handles. VU count is a byproduct of arrival rate and response time. If your p95 latency doubles, you need twice as many VUs to sustain the same throughput. Start from VUs and you’re tuning the wrong variable.

This distinction often comes down to API testing vs. website testing. There is typically a difference between trying to simulate a number of “real users,” including their actions and pauses and how those translate into backend capacity, vs. simulating a certain throughput on API endpoints. While the underlying test is ultimately similar, how you derive those traffic levels depends on the type of telemetry data you’re working with. For instance, analyzing and simulating full user sessions would align more with Grafana Cloud Frontend Observability, whereas the metrics described below are focused on deriving the traffic levels of your API endpoints.

k6 has two executor types that map directly to how real traffic works:

• constant-arrival-rate:  a fixed number of iterations per second, regardless of how long each one takes. Use this when you want to hold a steady request rate and observe what happens to latency under that pressure.

• ramping-arrival-rate:  arrival rate changes over time according to stages you define. Use this when your traffic has a shape: a morning ramp, an evening peak, a midday lull.

Both of these need real numbers to be useful. Here’s how to get them.

Step 1: Find your actual request rate

Open Grafana Cloud and run this query against your Prometheus or Mimir data source. Swap in your service name:

Look at this over a representative window. A full week is better than a single day. You’re looking for two things:

1. Your baseline rate during normal operation (requests per second)

2. Your peak rate, which is the highest sustained load you’ve actually seen

That peak number is your load test target. Not a guess or just 2x your average, but the actual peak your system has handled (or needs to handle).

If your baseline is 120 req/s and your peak is 340 req/s, you now have real numbers to work with.

Step 2: Get your latency baseline

This becomes your threshold. Run this query to get p95 latency under normal production load:

This value becomes the check in your Grafana Cloud k6 script, for example: http_req_duration: ["p(95)<280"].

Whatever this returns, such as 280ms, that’s your threshold. If your load test pushes p95 above 280ms, the test should fail. 

Also pull p99 while you’re here. A gap between p95 and p99 tells you something about tail latency behavior that will matter under load.