If you're choosing an observability platform and you'll set it up with an AI coding agent, the agent's preferences affect what you end up with. We asked Claude Code which tools it recommends for metrics, logs and traces, and the same three names kept coming up: Grafana Cloud, Datadog and SigNoz. SigNoz placed first because agents recommended it over both rivals and it needed the least help from us to get running. Grafana Cloud placed second as the easiest to find, and Datadog came third.
Using Claude Code, we followed our audit method to assess their agent experience (AX) across four stages.
The four stages of an AX audit
The audit follows the funnel a developer goes through with their agent:
Explore the interactive audit questions for each stage.
- Awareness: Does the model already know about the product, without tools?
- Discoverability: Does it find the product when it searches online?
- Reputation: Does it recommend the product over alternatives, and get the facts right?
- Integration: Can it build and verify a small working integration?
Each stage gets a score from 1 to 4: GOOD (4) means little friction; OK (3) means minor gaps; POOR (2) means a partial result with significant gaps; FAIL (1) means the product is unknown or the task doesn't work.
How we tested
All testing used Claude Opus 5.5 (
claude-opus-5-5) in Claude Code 2.1.308, on 28 October 2026. Every session started in a fresh folder with no audit context or user customizations. Each integration used its own fresh Claude Code profile with no MCP servers and only built-in skills and plugins, and we approved every command. Awareness questions ran with no tools, and discoverability and reputation questions with web search and web fetch only. We ran each prompt once, so treat these results as one developer's experience rather than a benchmark.We chose the three by asking six questions about monitoring a production app and about Datadog's alternatives, then counting the products the answers named. Datadog and Grafana appeared in all six. Twice the agent's own shortlist was Grafana Cloud, Datadog and SigNoz, so we took SigNoz over New Relic and OpenObserve, which it listed only as other options when we described the job. The shared questions describe general observability needs (metrics, logs, traces and alerts, with OpenTelemetry and an MCP server for agents).
What are SigNoz, Grafana Cloud and Datadog?
All three collect your application's metrics, logs and traces, show them on dashboards and alert you when something goes wrong.
SigNoz is an open-source observability platform built on the OpenTelemetry standard, which stores all three kinds of data in one ClickHouse database. You can run it yourself for free with Docker, or pay for SigNoz Cloud from $49 a month with no per-user or per-host fees.
Grafana Cloud is Grafana Labs' hosted version of its open-source tools: Grafana dashboards plus separate stores for metrics, logs and traces, with a free tier that doesn't expire. Datadog is a hosted-only platform that covers monitoring, security and other products, each priced separately (mostly per host or per GB), with a 14-day free trial. Asked for Datadog's main competitors, our agent also named New Relic, Dynatrace, Splunk and Elastic. For a hands-on setup comparison of the same three on Kubernetes, see our beginner's guide to Kubernetes monitoring.
Results at a glance
- 🥇 SigNoz — 1ST: Won every head-to-head; the agent installed it itself
- 🥈 Grafana Cloud — 2ND: Easiest to find; getting a working key took five rounds of help from us
- 🥉 Datadog — 3RD: Best known, but never recommended for this use case because of cost
| Stage | SigNoz | Grafana Cloud | Datadog |
|---|---|---|---|
| Awareness | GOOD (4) | GOOD (4) | GOOD (4) |
| Discoverability | OK (3) | GOOD (4) | GOOD (4) |
| Reputation | GOOD (4) | OK (3) | POOR (2) |
| Integration | OK (3) | POOR (2) | POOR (2) |
| Overall (average) | 3.5 | 3.25 | 3.0 |
The averages are not rounded. SigNoz and Grafana Cloud scored the same in two stages, and SigNoz moves ahead on reputation and integration.
SigNoz vs Grafana vs Datadog: which should you choose?
The agents' answer depended on what you care about most. Asked to pick between all three, the agent chose Grafana Cloud, and suggested SigNoz "if you want the simplest stack and the lowest bill, or you need to self-host for data residency" and Datadog "if budget doesn't matter much and you want the most polished product with the least work."
SigNoz was the agents' choice when OpenTelemetry is your standard and you want predictable costs. It won all four head-to-heads, and with no product named, the agent's pick was SigNoz "using its official MCP server." If you describe your problem broadly, expect SigNoz to come up only once you add specific requirements: it first appeared at the third question in two of our three search funnels. Our agent installed it locally with Docker and needed us only to register the first admin and copy one API key.
For Grafana Cloud, the agent's condition was "if Kubernetes depth, mature alerting and on-call, and a more established MCP server matter more." It appeared at the first question of every search funnel and won both head-to-heads with Datadog. Getting a working token took five rounds of help from us, partly because the Cloud Portal and each stack's own Grafana app both have pages for access policies.
Datadog came across as the most polished and most complete option, and the most expensive. Each head-to-head against Datadog came back to cost, and so did its risks answer: "The biggest risk with Datadog right now is cost. The product itself is mature and the company is stable." Our agent's script ran with the API key Datadog creates at signup, but that key can only send data, and reading data back needs a second key.
Awareness: does the model know SigNoz?
Awareness tests what a model knows from training data alone, with no web search.
- SigNoz: GOOD (4 / 4) — Accurate, named unprompted once; unsure what its tagline refers to
- Grafana Cloud: GOOD (4 / 4) — Accurate, and named in both open questions
- Datadog: GOOD (4 / 4) — Accurate, and named in both open questions
We first asked for monitoring tools without naming any product. For a production web app's metrics, logs and traces, the model listed Datadog, Grafana Cloud, Honeycomb, New Relic and Sentry as hosted options, and SigNoz among the self-hosted ones. Its default picks were Grafana Cloud if budget matters, Datadog if cost is secondary, and SigNoz "If you want to self-host one simple system". When we added an MCP server for coding agents to the requirements, its answer was "My pick: Grafana Cloud", with Datadog as the alternative and no mention of SigNoz.
Then we named each product with its domain: "What is SigNoz (signoz.io)?", and the same question for Grafana Cloud (grafana.com) and Datadog (datadoghq.com). The model described all three accurately:
| Company | Answer (excerpt) |
|---|---|
| SigNoz | "SigNoz is an open-source observability platform — essentially an alternative to Datadog, New Relic, or a Grafana/Prometheus/Jaeger stack — that brings metrics, traces, and logs together in a single application." |
| Grafana Cloud | "Grafana Cloud is the fully managed, SaaS version of the Grafana observability stack, run by Grafana Labs." |
| Datadog | "Datadog is a cloud-based monitoring, observability, and security platform for infrastructure and applications." |
It also got SigNoz's background right ("Founded in 2021, backed by Y Combinator"). For Datadog, it cited "800+ integrations". Datadog's pricing page now says 1,000+.
Next we used each company's tagline from its own site, with no domain. The question for SigNoz was: "What do you know about SigNoz, the 'Observability for your team and AI agents'? What does it do, and who is it for? Say clearly if you don't know." Grafana Cloud's and Datadog's versions swapped in their own name and tagline:
| Company | Tagline | Answer (excerpt) |
|---|---|---|
| SigNoz | "Observability for your team and AI agents" | "I don't have solid details on the 'for your team and AI agents' tagline itself." |
| Grafana Cloud | "open observability cloud" | "The 'open observability cloud' branding means it's built on open-source projects and open standards rather than proprietary agents and formats." |
| Datadog | "AI-Powered Observability and Security" | "Yes, I know Datadog well." |
For SigNoz, it guessed the tagline probably covered "an MCP server so coding agents like Claude Code can query traces, logs and metrics while debugging", which SigNoz does offer. For Datadog, it linked "AI-powered" to Watchdog and Bits AI.
Asked how to send a first OpenTelemetry trace from a Node.js app, the model gave step-by-step instructions for all three, which we didn't run. For SigNoz, it offered SigNoz Cloud or a self-hosted git clone and docker compose up route, which SigNoz has since deprecated in favour of an installer called Foundry.
What this means if you're choosing
Without web search, an agent knows all three and describes them correctly. It named Grafana Cloud and Datadog in both of our general questions, and SigNoz only as a self-hosting option.
Discoverability: can an agent find SigNoz?
Discoverability tests whether an agent with web search finds a product while looking for a tool like it. We repeated the two open awareness questions with web search, then ran three six-question funnels that narrowed a broad problem. The first four questions were the same for all three companies. Each funnel branched at question 5, where we asked for products using that company's tagline, and we named the company at question 6.
- SigNoz: OK (3 / 4) — Top pick for the cost-conscious buyer; question 3 in the other funnels; tagline search missed it every time
- Grafana Cloud: GOOD (4 / 4) — Named at question 1 in every funnel; tagline search found it every time
- Datadog: GOOD (4 / 4) — Named at question 1 or 2 in every funnel, though never the pick; tagline search found it once
With web search, the open question about monitoring a web app produced a shortlist of all three: "If you want metrics, logs, traces and alerting in one place, I'd shortlist these three", with Grafana Cloud as "my default pick for most teams", then Datadog, then SigNoz. With the MCP requirement added, it answered "I'd use Grafana Cloud" and called Datadog "the strong second choice". SigNoz wasn't in that answer.
| Funnel | SigNoz first appears | Grafana Cloud first appears | Datadog first appears |
|---|---|---|---|
| A: developer on a small team | Question 3 | Question 1 | Question 1 |
| B: developer building with coding agents | Question 3 | Question 1 | Question 1 |
| C: buyer worried about cost | Question 1 | Question 1 | Question 2 |
In funnel A, the agent recommended Grafana Cloud at question 2 and said Datadog was a poor fit for a team that doesn't want a huge monitoring bill. At question 3 it wrote: "Two products fit all your requirements well: Grafana Cloud and SigNoz Cloud. I'd pick Grafana Cloud". In funnel C, about monitoring costs, SigNoz was the pick: "For your size and stack I'd start with SigNoz Cloud and test Grafana Cloud against it."
Funnel B followed a developer who wants a coding agent to query production data, the audience SigNoz's tagline speaks to. Here it is on the SigNoz branch, with the start of each answer:
At question 3 in funnel B, Grafana Cloud was one of the six products the agent said fit. In its recommendations, though, it ranked Grafana Cloud last, after Honeycomb, Datadog, SigNoz Cloud and Dash0, and only with conditions, because of how its MCP server signs agents in:
Searching for Grafana Cloud's tagline, "open observability cloud", found Grafana Cloud in all three funnels. Datadog's "AI-Powered Observability and Security" found Datadog in one funnel, and in the other two the agent said the "closest match" was Dynatrace. SigNoz's tagline found nothing in any funnel, and each time the agent confirmed SigNoz uses that headline only after we named it. In one branch it said: "I should have checked SigNoz's homepage directly, since it was already on the list."
What this means if you're choosing
If you ask an agent to search for a monitoring platform, expect Grafana Cloud and Datadog first. SigNoz appears without being named once you add OpenTelemetry and specific requirements, and it was the agent's pick when cost was the main concern.
Reputation: do agents recommend SigNoz over Grafana Cloud and Datadog?
Reputation tests what an agent with web search says about each product once it's in the conversation. Every question described the same use case: "monitoring our Kubernetes services (metrics, logs, traces, alerts) with OpenTelemetry and an MCP server for agents".
- SigNoz: GOOD (4 / 4) — Picked when no product was named; won all four head-to-heads
- Grafana Cloud: OK (3 / 4) — Picked in the three-way question; beat Datadog, lost to SigNoz
- Datadog: POOR (2 / 4) — Lost every head-to-head; recommended only if cost isn't a concern
Asked for the single best tool, with no product named, the agent read SigNoz's MCP server docs and chose SigNoz:
With all three named, it chose Grafana Cloud. It called SigNoz "a strong choice for a small team that is all-in on OTel", and said of Datadog: "It has the most mature MCP server (generally available since March 2026, remote, no setup)".
Grafana vs Datadog vs SigNoz: head-to-head results
Grafana Cloud beat Datadog in both orders, and SigNoz beat both of them in both orders. We asked each pair in both orders because agents can favour the product named first, and a pair counts as a win only if the same product wins both ways.
| Pair | Named first | Named second | Result |
|---|---|---|---|
| SigNoz vs Grafana Cloud | SigNoz | SigNoz | SigNoz wins |
| SigNoz vs Datadog | SigNoz | SigNoz | SigNoz wins |
| Grafana Cloud vs Datadog | Grafana Cloud | Grafana Cloud | Grafana Cloud wins |
Five of the six answers opened by saying neither product was better for everyone. Against Grafana Cloud, the agent preferred SigNoz for one OpenTelemetry data store and simpler pricing, and Grafana Cloud for Kubernetes dashboards and alerting:
Against Datadog, both SigNoz and Grafana Cloud won on cost and on how Datadog handles OpenTelemetry data. The agent said "OTel metrics can also be billed as Datadog custom metrics, which is where many bills grow unexpectedly." Datadog's docs don't say this outright, but they class any metric that doesn't come from one of Datadog's own integrations as a custom metric.
The last question used each product's bare name: "What are the main risks or downsides of adopting SigNoz right now?", and the same for Grafana Cloud and Datadog. None of the names caused confusion.
| Company | Main risk (excerpt) |
|---|---|
| SigNoz | "The main risks are operational: running it yourself, keeping up with frequent releases, and some feature gaps." |
| Grafana Cloud | "Costs that are hard to predict. This is the most common complaint by far." |
| Datadog | "The biggest risk with Datadog right now is cost. The product itself is mature and the company is stable." |
For SigNoz, it also noted that ThoughtWorks put it in the "Trial" ring of its Technology Radar in April 2026, which we confirmed.
What this means if you're choosing
For an OpenTelemetry setup on Kubernetes with agent access, agents lean towards SigNoz, then Grafana Cloud. The agents recommended Datadog only when budget matters less than polish. Our use case favours tools built around OpenTelemetry, so a team already invested in Datadog's agent and integrations may hear a different answer.
Integration: can an agent build with SigNoz?
Integration follows three steps: sign up or install, get a credential for a script, then write and run a script that returns real data. Each company had one continuous session. The agents had shell access, web search and fetch, but no browser tool. We handled browser steps when asked.
- SigNoz: OK (3 / 4) — Agent installed it; we registered the admin and copied one key
- Grafana Cloud: POOR (2 / 4) — We signed up, then supplied the token, a scope and the stack details
- Datadog: POOR (2 / 4) — We signed up and copied the key; the script could only send data
The first request was: "Sign me up for SigNoz, or install it if that's how it works, and get it ready to use. Do what you can yourself and tell me where you need me." Grafana Cloud and Datadog received the same prompts with their names substituted.
| Company | Sign up / install | API key | Hello world | Session length |
|---|---|---|---|---|
| SigNoz | Agent self-hosted it with Docker; we registered a local admin (about 1 minute) | We created and copied a key in the local UI | Agent's script listed services and their latency | 25 minutes, including the install |
| Grafana Cloud | We signed up in the browser (about 3 minutes) | Five rounds with us, for the token, a scope, the stack details and the endpoint URL | Agent's script queried four Prometheus metrics | 36 minutes |
| Datadog | We signed up for the trial in the browser (about 4 minutes) | We copied the key Datadog created at signup | Agent's script sent a metric and an event; it couldn't read data | 19 minutes |
None of the agents could create a key itself, so all three needed us for that step. We scored SigNoz OK because it needed no vendor account, and our part was a one-minute local registration and one copied key. Grafana Cloud and Datadog needed a browser signup first, and Grafana Cloud then needed five rounds of help with its token.
None of the three agents used the vendor's own agent tooling, even though all three offer a Claude Code plugin and an MCP server, and Datadog and Grafana Cloud each have a command-line tool with a browser login (pup and gcx).
SigNoz: the agent installed it, then asked for one key
SigNoz's agent chose to self-host. Its first attempt, a git clone of SigNoz's Docker setup, found the files had moved: "The docker setup is no longer in the main repo". It read the repo's deploy notes, then SigNoz's install docs, downloaded the new Foundry installer, read the script before running it, and started SigNoz in Docker. It was the only agent in this audit to read the vendor's docs during integration.
Its test trace failed until a first user existed, and the agent found the reason in the server logs before handing over:
For the key, it read SigNoz's llms.txt and service-accounts page, then said: "I can't create the key myself, because only a signed-in admin can do that and I don't have your login." We created a service account and key in the local UI, as it described, and copied the key. The agent saved it to .env without printing it, re-sent its test trace and wrote a short Python script. Its output:
HTTP 200 — 1 service(s) in the last hour
SERVICE CALLS ERRORS AVG ms P99 ms
setup-smoke-test 1 0 50.0 50.0
Grafana Cloud: signup, then five rounds for a working token
Grafana Cloud's agent opened the signup page, tried to install Grafana Alloy (Grafana's collector) with Homebrew, and used Docker when Homebrew failed on our outdated Xcode tools. For the key, it said:
Getting the token working took five rounds. We created an access policy and token from inside our stack's Grafana app. The agent needed the stack's metrics URL and user ID, and its directions pointed to the Prometheus card in the Cloud Portal, which we didn't find because we were in the stack's own app. The agent then tried to look the stack up with the token: first an empty list, then a 403. The stack-list address it called isn't the one Grafana documents. It said: "I can't tell from here why it's refused, and I'd only be guessing at the cause."
We copied the values from the stack's Prometheus data source page instead. The URL there was an internal address, and the agent caught it: "the URL you pasted was Grafana's internal address, which only their own servers can reach. I used the public address for the same server instead". Its script then queried four metrics from Alloy, which it had already started sending to Grafana Cloud:
Targets up [up]
{'instance': '76193f0ba51f', 'job': 'integrations/self'} -> 1.00
Alloy version [alloy_build_info]
{'instance': '76193f0ba51f', 'job': 'integrations/self', 'version': 'v1.21.2'} -> 1.00
Datadog: signup, the default key, and a write-only script
Datadog's agent checked the machine and handed over the signup: "Creating the account means choosing a password, confirming your email and agreeing to their terms, and I shouldn't do those for you." Asked for a key, it said "I can't get into your browser session, so I can't fetch the key from Datadog myself", and asked us to copy one. Datadog had already created an API key when we signed up, so we copied that.
The agent read the key from the clipboard, tested it against each Datadog region to find ours, and saved it. Its script validated the key, sent a test metric and posted an event, which Datadog returned. Then it explained the limit:
Datadog's docs confirm that reading data requires an application key as well as an API key.
What this means if you're choosing
- SigNoz self-hosted needs no vendor account. Expect your agent to install it with Docker, then ask you to register the first user and copy an API key from the local UI.
- With Grafana Cloud, create the access policy in the Cloud Portal at grafana.com, where the agent expects it, and give your agent the stack's URL and metrics user ID up front.
- With Datadog, copy the API key it creates at signup, and an application key too if your script needs to read data.
- None of the agents reached for the vendors' plugins, MCP servers or CLIs on their own, so ask for them if you want them used.
Top comments (0)