My First 3 Hours With SigNoz

Mohammad Mudassir — Fri, 17 Jul 2026 20:34:42 +0000

It was 8:14 PM and I had one goal: run docker compose up, the command every tutorial swears by, and get SigNoz running. Three hours later I had live traces, a dashboard, and an alert going off because I made it go off. Along the way I hit a deprecated install path, a query builder that rejected my filter, and an alert that happily fired on completely healthy traffic.

Here's the one thing I'd tell you up front: a single broken request can show up as four "errors" in SigNoz. I didn't know that, and it quietly wrecked my first alert. Keep it in mind the whole way through.

My exact setup

A couple of the problems below only happen on specific machines, so:


Chip	Apple M3 (arm64)
OS	macOS 26.3
RAM	16 GB (7.65 GB given to Docker)
Docker	28.0.4, Compose v2.34
Python	3.10.14

Hour 1: the install that wasn't `docker compose up`

My first problem happened before I typed a single SigNoz command. docker info gave me "Cannot connect to the Docker daemon." Docker Desktop wasn't running. Started it, moved on.

Then the plan: clone the repo, run compose.

git clone --depth 1 -b main https://github.com/SigNoz/signoz.git

I went hunting for deploy/docker/docker-compose.yaml, the file basically every guide points you to. It wasn't there. The deploy/ folder had three files, and the README explained why:

"The install.sh script and the docker-compose manifests have been deprecated. SigNoz now installs and runs through Foundry."

This was the most useful thing I learned all night, and it's buried under a pile of outdated search results: as of 2026, docker compose up is no longer how you run SigNoz. The compose file still exists. A tool called Foundry just generates it for you.

Rather than guess, I followed the official docs at signoz.io/docs/install/docker. Foundry is an open source CLI (foundryctl) that "runs your self-hosted observability stack as code." Install it:

curl -fsSL https://signoz.io/foundry.sh | bash

I read the script first (worth doing with any curl | bash). It installs to ~/.local/bin, no sudo. Then a config file:

# casting.yaml
apiVersion: v1alpha1
kind: Installation
metadata:
  name: signoz
spec:
  deployment:
    flavor: compose
    mode: docker

I ran foundryctl forge -f casting.yaml first, just to see what it builds, and there was my compose file at pours/deployment/compose.yaml. Reading it was worth the detour, because the 2026 stack looks nothing like the old tutorials: the UI and query engine are one signoz/signoz container on port 8080, telemetry comes in through a signoz-otel-collector ingester on 4317/4318, storage is clickhouse-server:25.12.5, and coordination is now ClickHouse Keeper, not Zookeeper. If a guide mentions Zookeeper, it's out of date.

Then the real command:

foundryctl cast -f casting.yaml

First run pulls every image and went from nothing to a healthy stack in 3 minutes and 9 seconds. Then it tried to scare me: docker ps -a showed two containers as Exited (0). My first reaction was "something crashed." It didn't. The migrator and init containers are one-shot jobs that run once, set up the schema, and stop. Exit code 0 means it worked. Leave them alone.

curl localhost:8080 returned a 200. I made an admin account, and there it was.

"You're not sending any data yet." True. SigNoz is running, but it's got nothing to show until you feed it.

Hour 2: sending real data with OpenTelemetry

I wrote the smallest app that would still produce an interesting trace. A Flask /rolldice endpoint that calls a downstream /roll on itself over HTTP, with random latency and a deliberate 15% error rate:

# app.py (trimmed)
from opentelemetry import trace
tracer = trace.get_tracer("rolldice.manual")

@app.get("/roll")
def roll():
    time.sleep(random.uniform(0.01, 0.4))          # real latency
    if random.random() < 0.15:                      # real errors, ~15%
        raise RuntimeError("the die fell off the table")
    return jsonify(value=random.randint(1, 6))

@app.get("/rolldice")
def rolldice():
    with tracer.start_as_current_span("rolldice.handler") as span:
        span.set_attribute("player.name", random.choice(["alice","bob","carol"]))
        resp = requests.get(f"http://localhost:{PORT}/roll", timeout=2)
        resp.raise_for_status()
        span.set_attribute("dice.value", resp.json()["value"])
        return jsonify(value=resp.json()["value"])

For instrumentation I used OpenTelemetry's zero-code agent, with commands pulled straight from opentelemetry.io/docs/zero-code/python instead of memory:

pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install

The second command scans what you've installed and pulls matching instrumentation, in my case Flask and requests. Then you run the app through the agent:

OTEL_SERVICE_NAME=rolldice \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 \
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \
OTEL_TRACES_EXPORTER=otlp \
opentelemetry-instrument python app.py

This is where macOS got me. Flask defaults to port 5000, and on macOS that's quietly owned by AirPlay Receiver. My app looked like it started but nothing answered. I moved it to 5001 and it was fine.

I fired 40 requests. The logs showed no export errors, which is a good sign but not proof anything arrived, so I checked ClickHouse directly:

docker exec signoz-telemetrystore-clickhouse-0-0 clickhouse-client \
  -q "SELECT count() FROM signoz_traces.distributed_signoz_index_v3 WHERE serviceName='rolldice'"
# 161

161 spans from 40 requests, so about 4 spans each. Remember that, it's the whole point of this post.

Within seconds, rolldice showed up under Services with real APM numbers I never configured.

P99 of 411 ms and a 16.10% error rate, auto-computed from my spans. The 16% lines up with the ~15% I put in the code, which told me the pipeline was real.

The payoff: my first trace

I opened Traces, filtered to service.name = rolldice, and clicked into one request. This is where it finally made sense.

One request, four nested spans. GET /rolldice calls my rolldice.handler, which makes an outbound GET, which hits GET /roll. You can see exactly where the time goes.

Clicking the rolldice.handler span showed the attributes I set in code, now searchable in the UI.

dice.value: 1 and player.name: "bob", straight from my code.

My first dashboard, and the error that stopped me

I wanted a P99 latency chart. Time Series panel, data source Traces, p99(duration_nano), group by name. In the filter box I typed what felt obvious, service.name rolldice, and got two red errors:

invalid_input: Found 2 errors while parsing the search expression
full text search is not supported

I'd given it a key and a value but no operator between them. Without the =, it tried to treat rolldice as a free text search, which the trace query builder doesn't do. Type the key, pick it from the dropdown, choose =, then the value:

service.name = 'rolldice'

The y-axis is in nanoseconds because the field is duration_nano, so 400,000,000 is 400 ms. GET /roll sits on top, which is where my time.sleep lives.

Hour 3: my first alert, and why it screamed at healthy traffic

I built a trace based alert: service.name = 'rolldice', count(), notify when it goes above 10 in the last 10 minutes.

The preview was wrong immediately. The count line sat around 20, permanently above my threshold of 10. This alert would fire forever, on traffic that was fine. The reason: count() with no filter counts every span, successes included. I only cared about errors, so I added the flag (in my version it's has_error, snake case):

service.name = 'rolldice' AND has_error = 'true'

The baseline dropped under the line. Now to trip it on command, I hit the app with a burst:

seq 1 300 | xargs -P 25 -I{} curl -s -o /dev/null http://localhost:5001/rolldice
# 250 OK, 50 errors, in about 3 seconds

SigNoz evaluates on an interval, so a minute later the rule flipped.

Firing. My first alert, set off by my own burst.

Two bursts, two spikes well over the red line, calm baseline in between.

The thing that actually clicked: error spans are not failed requests

Something still bugged me. Even with the error filter, my "calm" baseline was 8 to 20 error spans a minute, far more than a low error rate should give. So I asked ClickHouse to break it down by span name:

SELECT name, count() FROM signoz_traces.distributed_signoz_index_v3
WHERE serviceName='rolldice' AND hasError=true
  AND timestamp > now() - INTERVAL 10 MINUTE
GROUP BY name

GET /rolldice     137
rolldice.handler  137
GET               137
GET /roll         137

There it was. One failed request produces four error spans, because the failure propagates up the whole trace: the downstream /roll throws, then the requests client span, then my handler, then the root. So 137 failed requests became 548 error spans. My alert's numbers were inflated four times over, which is exactly why the "calm" baseline kept bumping the threshold.

That "4 spans per request" number from earlier is why. The fix is to count what you actually mean, failed requests, by pinning the filter to the root span:

service.name = 'rolldice' AND has_error = 'true' AND name = 'GET /rolldice'

Now the chart is honest: a flat baseline of 2 to 4, and my two bursts as clean spikes.

Same alert, one extra filter. Now "count" means "failed requests," and the baseline sits where it should.

An alert is only as good as the query behind it. Counting every span in a broken trace tells you how deep your traces are, not how many requests failed.

What I'd tell myself before I started

docker compose up is not how you run SigNoz anymore. Use Foundry, or foundryctl forge if you still want the raw compose file.
Exited (0) is not a crash. The migrator and init containers run once and stop.
On macOS, skip port 5000. AirPlay owns it. Use 5001.
In the query builder, always put an operator between key and value.
Confirm your data landed, don't just assume it sent. One clickhouse-client count saved me a lot of guessing.
Count requests, not spans. One broken request can create several error spans, so filter to the root span when your metric means "requests."

Wrapping up

Three hours, one deprecated command I didn't expect, a few small mistakes, and one lesson about spans versus requests that I'll carry into every alert I write. SigNoz got me from an empty screen to a firing alert faster than I thought, and most of the friction was the useful kind that teaches you how the thing works underneath.

If you're just starting: use the official docs at signoz.io/docs/install/docker, check your OpenTelemetry setup against opentelemetry.io, and when something looks weird, go straight to ClickHouse. The answer is usually one query away.

If your errors are different from mine, good, that's the fun part.

DEV Community: Mohammad Mudassir