DEV Community

OnaEiuspkz
OnaEiuspkz

Posted on

An LLM observability platform stores prompts, and prompts are the application

#zoomeye #exposure #langfuse #attacksurface

An LLM observability platform stores prompts, and prompts are the application

A title query for Langfuse returns 346 matches in ZoomEye. The number is small and the contents are unusual. An observability tool for language models records the text that goes into them and the text that comes back, which makes a tracing store closer to a source repository than to a metrics backend.

Context and method

The query ran on 2026-10-01 and matched the title field for the literal string. ZoomEye title matching is inclusive, so a record returns when the string appears anywhere in the parsed title. One match equals one indexed record, captured whenever the scanner last reached the host.
The measurement therefore describes how many records present the name. It says nothing about whether the deployment holds production traffic or a developer's experiments, whether authentication is configured, or which of the platform's optional components are running.

What the platform records

Langfuse instruments an application that calls a language model. Each call produces a trace holding the prompt, the model parameters, the completion, token counts, latency and whatever metadata the developer attaches. It also manages prompt templates and runs evaluation jobs.
Two of those fields explain why this category deserves attention. The prompt is the application logic for a model-driven product. The completion is the output that logic produces with real inputs attached. Neither field is a metric, and neither is a log line in the ordinary sense.

Why traces are sensitive in a specific way

Traces hold whatever the application sends. Support tools forward user tickets to a model, and the trace contains the ticket. Internal assistants forward document excerpts, and the trace contains the excerpts. Retrieval pipelines forward retrieved passages, and the trace contains the corpus.
Secrets travel the same channel. A broken authentication flow sends credentials, a debugging session echoes configuration, and an agent that calls tools writes its tool arguments into the trace, including the API keys and access tokens the model was given.
The prompt template is a second asset. A managed template often encodes the system instructions, the guardrails and the structure of a proprietary workflow, and a competitor who reads it learns how the product is built.

Reading the count sensibly

A title match cannot tell whether an instance is a personal experiment, an internal tool behind a proxy, or a service that other teams submit traces to. It also cannot tell you whether a public demonstration instance is sharing a tenant with production data.
The claim that holds is small and specific. Hundreds of LLM tracing deployments are indexed, which is a modest population, and one of them can hold the prompts, the outputs and the embedded secrets of a production system.

Running the check honestly

List the instances your organisation runs and confirm which are reachable from outside the network. Confirm whether the interface and the ingestion API require authentication, and whether the keys that applications use to send traces can also read them back out. Confirm the retention configuration, because a trace store accumulates everything the application sends to it. Confirm whether prompt templates are stored there, and whether user input reaches the trace unredacted.
Where an instance is not meant to be public, remove the address rather than relying on an obscure path. Where traces must be kept, apply redaction at the application boundary and keep the retrieval keys separate from the ingestion keys.

Implications for defenders

Treat tracing infrastructure as a system of record for application text, and review it the way a code repository is reviewed. The useful question concerns what the largest trace in yours contains, not how many instances exist elsewhere.

References

Top comments (0)