DEV Community: Patrick Londa

Introduction to A2A and Agent Search

Patrick Londa — Tue, 26 May 2026 18:32:08 +0000

Authored by David Tracey

AI is rapidly evolving from simple tools to increasingly complex agents capable of reasoning and decision making. As agents are used for more tasks, the ability to use multiple co-operating agents will become increasingly important — particularly for tasks requiring specialised knowledge or access across domains.

Large-scale, flexible solutions will require a common means for agents to communicate capabilities to each other using a trusted means of collaboration. That's what the Agent2Agent (A2A) protocol is designed to provide.

This is the first post in a three-part series:

Part 1 (this post): Introduction to A2A concepts and a "Hello World" agent
Part 2: Using MCP to query multiple data sources including Bronto's logging platform
Part 3: Combining A2A and MCP with a "SuperAgent" orchestrating multiple agents

Why A2A?

The Agent2Agent protocol — originally developed by Google and now under the Linux Foundation as an open source project — gives agents an open protocol for interaction built on HTTP. It standardizes how agents exchange messages, requests, and data, allowing agents to:

Discover each other's capabilities without exposing internal state, memory, or implementation
Negotiate interaction details (text, forms, media, etc.)
Collaborate on running tasks securely

IBM's definition of agentic AI captures it well: "Agentic AI is an artificial intelligence system that can accomplish a specific goal with limited supervision", with each agent in a multi-agent system performing a specific subtask coordinated through AI orchestration.

A2A and MCP: Complementary Protocols

MCP and A2A serve different but complementary roles:

MCP exposes tools and structured data sources to an agent. It extends an agent's capabilities by standardizing how an agent accesses databases or product APIs. It focuses on a single agent accessing tools, and lacks built-in agent authentication.
A2A provides structured agent-to-agent communication — allowing multiple autonomous agents to collaborate, delegate tasks, and exchange information using capabilities exposed via Agent Cards. It enables multi-agent interactions with secure communication and capability discovery.

MCP Tools are generally basic operations with defined inputs/outputs. A2A is designed for autonomous agents that reason, use tools, and collaborate on complex problems. The two protocols are not mutually exclusive — agents can use MCP for specific tool access and A2A for broader agent collaboration.

Part 3 of this series will show an emerging pattern where A2A agents sit in front of MCP servers.

Key A2A Concepts

Agent Card — public metadata describing an agent's capabilities, skills, URL, and authentication requirements. Other agents retrieve this card to discover what an agent can do.

A2A Server — exposes an HTTP API endpoint implementing A2A methods and executes tasks on behalf of other agents.

A2A Client — an application or agent that sends requests to an A2A server to initiate tasks.

Task — initiated by a client sending an A2A message (role: user). Each task has a unique ID and states: submitted, working, input-required, completed.

Message — contains a role (user or agent), optional metadata, and an array of parts (TextPart, FilePart, or structured JSON data).

Communication flows:

Discovery — client fetches the Agent Card via GET /.well-known/agent-card.json
Initiation — client sends a request with a task ID to the server agent
Processing — server processes the task, streaming intermediate updates via Server-Sent Events if needed

Building a Hello World A2A Agent

Installation and Setup

These examples use the official A2A samples repository. Requirements:

Python 3.12 or higher
UV (recommended Python package manager)

Running the Sample

# Clone the repo
git clone https://github.com/a2aproject/a2a-samples.git
cd a2a-samples/samples/python/agents/helloworld

# Install UV and run the server
pip install uv
uv run .

This runs __main__.py in the helloworld directory, which defines an AgentCard, AgentSkill, and extended card, then calls uvicorn to run a server on localhost:9999.

In a separate terminal, run the client:

uv run test_client.py

The Agent Card Response

The server returns its public Agent Card:

{
  "capabilities": { "streaming": true },
  "defaultInputModes": ["text"],
  "defaultOutputModes": ["text"],
  "description": "Just a hello world agent",
  "name": "Hello World Agent",
  "preferredTransport": "JSONRPC",
  "protocolVersion": "0.3.0",
  "skills": [
    {
      "description": "just returns hello world",
      "examples": ["hi", "hello world"],
      "id": "hello_world",
      "name": "Returns hello world",
      "tags": ["hello world"]
    }
  ],
  "supportsAuthenticatedExtendedCard": true,
  "url": "http://localhost:9999/",
  "version": "1.0.0"
}

Authenticated clients additionally receive an extended card with a super_hello_world skill — demonstrating how A2A supports capability tiers based on authentication level.

What's Coming in Parts 2 and 3

Part 2 will show how to build simple MCP servers for Bronto's REST API and a SQLite database, then use Claude to query both simultaneously with a natural language prompt — no data movement required.

Part 3 will extend this into a full A2A scenario with a "SuperAgent" orchestrating two A2A agents (one per data store), demonstrating how A2A and MCP work together.

The Bigger Picture: Challenges A2A Still Needs to Solve

A2A is an evolving protocol. Current areas requiring further development include:

Multi-agent orchestration — standardized support for conflict handling and failure recovery across agents from different organizations
Shared vocabulary — no agreed-upon standard definitions for common items like invoices, policies, or receipts
Trust — establishing trust between agents across organizational boundaries is key to adoption
Security — authentication is provided, but richer methods are needed for privacy across organizational/national boundaries
Enterprise readiness — standardized usage management, SLOs, SLAs, and automated negotiation

The separate open source project AGNTCY, backed by Cisco, LangChain, Galileo and others, provides the Open Agent Schema Framework (OASF) as another approach to standardizing agent capability descriptions.

Explore Bronto's AI Features

Bronto for Fastly: Real-Time CDN Logging That Actually Scales

Patrick Londa — Tue, 26 May 2026 17:04:31 +0000

Authored by Mike Neville-O'Neill

A company processing tens of TBs of Fastly logs daily through traditional logging vendors faces an impossible choice: pay enterprise prices for comprehensive logging, or fly blind with minimal retention.

That's why most teams default to a 7-day retention window. Not because they don't need historical data — because the economics don't work.

That was the exact situation for Contentstack, a leading global platform for digital experiences. They now keep a full year of logs, run unlimited queries, and report 50% cost savings versus their previous solution.

The Real Cost of Restricted Logging

Processing 10TB of Fastly logs daily can cost ~$30,000/month for ingestion alone on traditional platforms. To reduce costs, teams resort to:

Abbreviating field names (EdgeResponseStatus → ers) to save bytes
Sampling 1 in 100 requests and hoping issues appear in that sample
Keeping only 7 days of data when seasonal patterns span months
Dropping entire log categories to keep storage costs down

When ingestion and storage are expensive, comprehensive logging becomes a luxury instead of standard practice. But what if the economics actually worked?

What Changes When You Can Keep Everything

Bronto's architectural efficiency — 90% compression, optimized storage, purpose-built for logs — means you pay 50–90% less to ingest the same data. This changes everything.

Consider tracking API performance across enterprise customers:

-- Monitoring performance across multiple API customers
SELECT count(*), avg(resTime)
WHERE (apiKey = 'customer_key_1' OR apiKey = 'customer_key_2' OR apiKey = 'customer_key_3')
  AND resTime > 500
  AND cache != 'MISS, MISS'
GROUP BY shield, cache
-- Time range: 30 days

When ingestion is affordable, you can actually keep the 30 days of high-volume logs this investigation requires. No sampling, no abbreviations, no compromises.

30-Day Performance Archaeology

A customer report of intermittent slowdowns triggered a deep investigation:

SELECT max(resTime), avg(resTime), count(*)
WHERE apiKey = 'customer_12345'
GROUP BY shield, cache
LIMIT 100
-- Analyzing response times week-over-week for the past month

Analyzing max and average response times over 30 days revealed certain endpoints were gradually slowing down — a pattern invisible in a 7-day window but clear with month-long data.

The team spotted week-over-week degradation that would have caused a critical outage within weeks. With 7-day retention, they never would have seen it coming.

Multi-Month Error Forensics

When 404 errors started appearing randomly across the CDN, the team needed to know: new issue or recurring pattern?

SELECT count(*)
WHERE url ILIKE '%/api/v2/content/%'
  AND response_status = '404'
GROUP BY cache, shield, pop
LIMIT 100
-- 90-day investigation of 404 errors for content API endpoints

Filtering 90 days of complete, unsampled logs revealed what looked like random errors was actually a cache invalidation race condition — one that only occurred under specific geographic and timing conditions. With 7-day windows and aggressive sampling, this would have remained an unsolved mystery.

The Intelligence Layer: From Logs to Insights

Long-term retention unlocks intelligence — but only if you can query it. Bronto's analytics engine lets you:

Run SQL queries across months of data in seconds
Set up change detection that automatically flags when cache hit rates drop
Build filterable dashboards to click into any spike or anomaly
Use natural language with the AI dashboard builder: "Show me API response times by customer and region for the last 90 days"
Use BrontoScope to interrogate logs with context, correlating events and surfacing root causes that would take hours to piece together manually

What Becomes Possible

API customer journey mapping — track the entire API consumer lifecycle:

Which endpoints do new users test first?
How does usage scale over months?
What usage patterns precede account downgrades?

Intelligent cache optimization — with a year of data you can finally answer:

How do cache hit ratios vary by season, not just by day?
Which content genuinely needs global distribution vs. regional?
How do different shield configurations perform under various traffic patterns?

Security pattern recognition — long-term retention enables sophisticated threat detection:

Bot networks that rotate through IP ranges over weeks
Credential stuffing campaigns that evolve over months
DDoS reconnaissance that occurs weeks before an attack

Before and After Bronto

Before	After
Field names abbreviated to save bytes	Full field names: readable and searchable
Aggressive sampling: keeping 1% of logs	100% coverage: every request, response, error
Short retention: 7 days max	365-day retention: full seasonal patterns visible
Blind spots: log categories dropped to save costs	Complete visibility: CDN and compute logs

Technical Foundation: Built for Fastly's Scale

Bronto integrates with Fastly through a built-in logging endpoint. Configure it once in your Fastly service settings and logs start streaming in minutes:

# Simple Fastly Integration
name: Bronto
format: JSON
url: https://logs.bronto.io/fastly
compression: zstd  # 10:1 automatic compression
method: POST
header_name: X-Bronto-Token
header_value: ${BRONTO_TOKEN}

Why it scales:

Elastic ingestion that grows with traffic
90% data compression
Time-series optimization for CDN access patterns
Intelligent tiering keeps recent data hot, historical data accessible
Column-oriented storage for efficient aggregations

Bronto Express: From Ingestion to Insights in Minutes

Setting up this level of insight shouldn't require weeks of work. Bronto Express for Fastly uses AI to take raw log streams and dynamically build Fastly-specific queries, monitors, and dashboards personalized for your organization.

It includes pre-built dashboards for real-time performance, geographic traffic, API usage, and security monitoring; a saved searches library for common investigations; and intelligent monitors for performance degradation and error rate thresholds.

Setup is four steps:

Create your Bronto instance (selecting the Fastly Express pack)
Add Bronto as a logging endpoint in Fastly (5-minute configuration)
Watch pre-built dashboards populate with your data
Customize searches and alerts for your specific needs

No agents. No forwarders. No complex pipelines.

The Platform Effect: When Every Team Has Access

When CDN logs become affordable and accessible, every team benefits:

Engineering — debugs issues with complete historical context
Security — investigates threats across meaningful timeframes
Product — understands real usage patterns, not samples
Sales — identifies expansion opportunities from actual usage data
Finance — accurately allocates costs to business units
Support — resolves tickets with full forensic capability

This isn't just about saving money on logging (though 50–90% savings helps). It's about removing the artificial constraints that prevent teams from fully understanding their infrastructure.

See Sub-Second Search Across Your Fastly Logs

Investigation Reports: When Monitors Get Smarter

Patrick Londa — Fri, 22 May 2026 18:04:01 +0000

Authored by Marco Aquilanti

When a monitor fires, there's a familiar sequence of checks required to find the root cause. The engineers who set up the monitor usually know these steps by heart — they know the dependencies, the error codes, what to check and where. But for the on-call responder, these steps aren't always obvious. Historically, the solution was to force engineering teams to document the checks in a playbook and hope the responder would read it under pressure.

Today, we can offload these checks to an LLM, shifting the responder's role from gathering evidence to reviewing a diagnosis — significantly reducing MTTR.

Our new Investigation Reports feature does exactly this: an LLM completes the investigation and delivers a detailed report before the human even acknowledges the alert.

Investigation Reports builds on BrontoScope, our first AI-powered investigation feature — and on the positive customer feedback it generated.

BrontoScope vs. Investigation Reports

Both BrontoScope and Investigation Reports perform automated investigations and provide reports, but they work differently.

BrontoScope starts with a user request to investigate a specific error event in the logs. The investigation follows a defined workflow aimed at establishing when and where an error is occurring. The LLM guides the process and summarizes findings synchronously — the user is waiting for a response and gets it in seconds.

Investigation Reports is triggered by a system event (a monitor firing), with no user waiting for a synchronous response. This gives the LLM more time — not seconds but minutes — to query data and analyze results. Investigating an alert is also a more generic task than BrontoScope's focused error investigation, making it harder to define a single fixed workflow that succeeds in every scenario.

For these reasons, Investigation Reports lets the LLM operate more freely — giving it tools and context rather than a coded workflow.

Tools Are Easy, Context Is Hard

The tools side is straightforward: the LLM can call Bronto's APIs to perform lightning-fast log search, query key-value dictionaries, check monitor history, retrieve precomputed metrics, and more.

Context is the harder problem.

LLMs make good logical decisions when provided with relevant, well-explained context. But the context window is limited — and it's been demonstrated repeatedly that longer context leads to worse answers and higher hallucination rates. This is known as "context rot" (see research from Chroma and this arXiv paper).

For an effective investigation, the LLM needs more than just the monitor that fired. It needs historical context and an understanding of the monitored system. But dumping thousands of tokens of documentation into the prompt backfires — it drastically increases hallucination risk and degrades report accuracy.

User-Defined Investigation Context

The precise knowledge needed for a good investigation is hard for an LLM to infer autonomously — but it can be provided by the engineer who owns the monitor.

In a dedicated "Investigation Prompt" text area, the user instructs the LLM on what to check and what to do when the monitor fires. Free-form text makes the feature highly flexible, effectively letting users define an ad-hoc workflow for each specific use case.

Engineers and SREs commonly include:

List of dependencies of the affected service
Related log datasets and how to correlate/query them
Relevant keys and metrics to check
What to include in the report — affected components, customers, or users

The screenshot below shows an example investigation prompt telling the LLM to check datasets in a collection named "booking system":

And here's the Investigation Report generated when that monitor fired — the LLM followed the instructions, ran multiple queries, and produced a report with a potential root cause, diagnosis, and timeline:

Investigation Reports Beyond Incident Response

Bronto's own customer support and sales teams found an unexpected use case. They set up monitors to be notified when new organizations are created or contracts are updated in the system — keeping the team up to date on new sign-ups and customer onboarding.

Investigation Reports automates the task of fetching context about each event. Details like contract type, retention plan, company size, and location are queried across multiple logs and assembled into a report that arrives within a minute of the monitor notification. This lets the team quickly identify relevant events among routine ones.

Below is the investigation prompt used by the customer support team, and an example of the automatically generated report:

Investigation Reports are a great illustration of what LLMs are genuinely good at: taking a well-framed task with relevant context and producing a structured, actionable summary faster than any human could. Every monitor notification now comes with relevant information to speed up resolution.

We'll be building further on this capability in the coming months — using AI alongside Bronto's logging platform to help teams reduce toil, resolve issues faster, and extract more value from their data.

Explore Bronto Labs

Using Dashboard Filtering to Get Customer Usage in Seconds from TBs of Data

Patrick Londa — Fri, 22 May 2026 13:53:30 +0000

Authored by Conall Heffernan

As the Customer Success lead at Bronto, I need fast, reliable insights into customer health and product usage — but I don't have time to constantly update indexes, schemas, or individual widgets just to answer new questions. I need to spot patterns, explore trends, and get answers in real time without manual overhead.

That's why Bronto's dashboards are so critical to my work. We recently added a new query filtering feature that lets me use SQL to look for any pattern across all widgets in a dashboard simultaneously — and update everything at lightning speed.

What Bronto Dashboards Provide

Rich visualisation options — time-series charts, geomaps, numeric value widgets (with units like bytes and time), top lists, treemaps, and log event lists for drilling into raw data
AI widget builder — describe what you want in natural language; an LLM builds the query and creates the widget without you needing to know the query language or which datasets to select (see the full post on this feature)
Full screen widget mode — compare different timeframes for the same query (e.g. vs. previous day, week, month)
Easy filtering — use the query builder with a searchable dropdown of top keys and values, or write SQL filters directly in the filter bar

The Power of Filtering at Scale

In many logging or observability tools, applying a filter means updating every single chart, table, or widget individually. Tedious. Time-consuming. It doesn't scale.

With Bronto, applying a filter in the main query bar instantly updates every single widget on the dashboard for your given timeframe. With a default retention period of one year, you don't need to worry about missing long-term trends — all your data is fully searchable and visualisable. Concerned about when an issue started? You can analyze trends over months, not days.

Our widgets use pre-computed log-based metrics (LBMs) for rapid responses, but the new dashboard filtering goes further — running raw log queries to filter the data in your dashboard in real time. Results come back in seconds, and you can drill down across all widgets simultaneously using SQL or by clicking from a dropdown of top keys and values.

One thing worth noting: there's no initial configuration of keys required. I can use any key I want in the filter, with no setup.

Log-Based Filtering vs. Log-Based Metrics

Log-based metrics (LBMs) deliver millisecond responses — perfect for real-time dashboards, trend analysis, and high-level views of system behavior.

When something unexpected happens or a new question comes in, we seamlessly switch to log-based filtering to investigate deeper, explore raw data, and uncover answers without being constrained by pre-defined fields or aggregations.

Log-based filtering excels because it lets you query and visualize raw logs immediately — no upfront configuration like index definitions or field extraction required. Bronto combines this with structured parsing and indexing, using the right approach for the right job.

The result: both fast, flexible investigation and high-performance queries on known fields, without forcing you to predefine every key or build parsing pipelines before you can search effectively.

How I Use Dashboard Filtering for Customer Usage

My primary use for this feature is gathering and presenting product usage data to our leadership team. Questions like:

"How much data did Org ID 54321 send over the last 6 months?"
"How much did company ACME search last month?"

Instead of building 10 custom dashboards (which doesn't scale as your customer base grows), I use dashboard filtering:

Navigate to our main Usage Dashboard
Enter the specific org_id in the main query filter (e.g. org_id: 54321)
Every widget updates instantly to reflect only that organization's data

That's it. A complex, multi-step data lookup becomes a quick and easy process.

The first time I tried filtering across the dashboard for an org_id, I thought something wasn't working right — the results were rendered so fast across terabytes of data. It was a genuine "wow" moment. As a customer support lead, it's great to see the under-the-hood changes we're building for customers also improving my own day-to-day.

How Bronto Dashboards Compare

Most dashboard tools are optimized for known questions: predefined fields, fixed widgets, metrics decided on ahead of time. That works for stable monitoring but breaks down when you need to explore new questions, investigate unexpected behavior, or quickly slice data in different ways.

Feature	Bronto Dashboards	Traditional Competitors
Query Scope	SQL filtering on any log value	Often restricted to predefined or indexed fields
Update Speed	Instant — all widgets update simultaneously	Manual, per-widget updates required
Performance	Terabytes rendered in seconds across all datasets	Latency issues with large datasets; often minutes to render
Setup	Schema-less — no upfront definitions or configuration	Parsing and indexing pipelines required first
Cost	Filtering included in monthly quota; faster MTTR	Slow dashboards can inflate costs by consuming more resources

Summary

Modern teams need dashboards that are fast, flexible, and easy to adapt as questions change. Bronto dashboards combine high-performance log-based metrics with instant, dashboard-wide filtering to help you explore usage, investigate issues, answer leadership questions, and quickly re-run reports as requirements evolve.

Questions like "what does this customer's usage look like over the last six months?" are answered in seconds — without upfront schemas, per-widget reconfiguration, or slow refresh cycles.

See Dashboard Filtering in Action

Your LLM Logs Deserve Better — Send Claude Code Events to Bronto

Patrick Londa — Thu, 21 May 2026 18:47:19 +0000

Authored by Conall Heffernan

In this post, I'll walk through how to integrate Anthropic's Claude Code monitoring with Bronto so you can get a clear, unified view of your AI tool usage — all in one place.

This project came out of our recent company AI sprint — a chance for me, as a technical support manager, to jump into an engineering sprint and brush up on my Python and Bash skills. After years of helping customers set up and troubleshoot their environments, it was exciting to get hands-on and demo my findings to the wider team over pizza and beers.

What You Can Send from Claude Code to Bronto

Claude Code has a built-in monitoring feature that provides a detailed view into how your teams use AI-assisted coding — from productivity metrics to cost visibility. When paired with Bronto, this data becomes even more useful.

Here's what you can do with the exported data:

Understand usage patterns — analyze how different users and teams engage with Claude Code over time
Measure productivity — track lines of code modified and models used
Monitor costs — get a clear picture of spend by team, user, or session
Audit prompts — see what users are actually asking, building, or automating; identify common use cases, detect misuse, and refine prompts based on real-world usage

Why Bronto specifically:

Centralise all event logging across your organisation for unified analysis
Visualise effortlessly — create dashboards using the AI widget creator without writing queries
Set up monitors to alert on sudden cost spikes, unusual token usage, or high session volumes from specific teams
Fast, scalable search — explore and investigate your LLM logs quickly
Handle high volume easily — Claude Code can generate massive, detailed logs; Bronto's storage and query engine handles this without breaking your budget
Long-term retention — 12 months of hot logs by default, which is increasingly important for support tracking, debugging non-deterministic LLM behavior, and EU AI Act compliance

Two Ways to Connect Claude Code to Bronto

Option 1: Direct (Simplest)

Claude Code has built-in OTel support, so it can send telemetry straight to Bronto. Just set these environment variables (full details in Claude Code's docs):

export CLAUDE_CODE_ENABLE_TELEMETRY=1
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=https://ingestion.eu.bronto.io
export OTEL_EXPORTER_OTLP_HEADERS="x-bronto-api-key=<api_key_here>,x-bronto-collection=AISprint,x-bronto-dataset=claude-code"
export OTEL_LOGS_EXPORT_INTERVAL=2000
export OTEL_LOG_USER_PROMPTS=1

With this setup, Claude Code sends structured logs directly to Bronto — no additional infrastructure to run or maintain.

Best for:

Running Claude Code locally or in a single environment
No existing centralised OTel collector
Lightweight, zero-maintenance integration

Option 2: Via an Existing OTel Collector

If your organisation already runs an OTel Collector, route Claude Code's logs through that same pipeline before forwarding to Bronto. This is the approach I used — I spun up a small AWS EC2 instance running the OTel collector, configured Claude Code to send logs to it, and had the collector forward those logs to Bronto.

Point Claude Code at your local collector:

export CLAUDE_CODE_ENABLE_TELEMETRY=1
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_LOG_USER_PROMPTS=1

Pro tip: Persist these environment variables — otherwise starting a new terminal session will clear them and you'll wonder why nothing is flowing. Ask me how I know.

Collector receiver config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317  # Default OTLP gRPC port
      http:
        endpoint: 0.0.0.0:4318  # Default OTLP HTTP port
        cors:
          allowed_origins:
            - "*"

Best for:

Aggregating logs from multiple sources before forwarding
Applying processors like filters
Keeping your Bronto exporter config in one place

Generating Some Log Data

To verify everything was working, I fired up Claude Code and asked it to create a simple Python script, then made a few tweaks — spin up a basic GUI, take an input value, save it locally. While I was doing this, Claude Code was generating log events in the background, flowing through my OTel collector and straight into Bronto.

Searching Claude Logs in Bronto

During testing, Claude was hitting errors because I'd been using an old revoked API key from a previous project. While the UI showed an error, I could also see the API errors flowing into Bronto. A simple error query surfaced the logs directly.

I also created a tree map to instantly visualize which models were used across the project — making it easy to see usage patterns and identify which models were most heavily used.

Token usage analysis — a multi group-by on $input_tokens and $output_tokens helps identify outliers and inefficient prompts, control API spend, and uncover latency trends tied to model size or context length.

Session duration — tracking how long each session stayed active helps identify inefficient workflows, idle connections, or hanging requests that inflate costs. It's also a signal for user engagement: shorter sessions suggest quick completions, longer ones can point to complex prompts or slow model responses.

Prompt auditing — prompts are redacted by default. To enable them, set OTEL_LOG_USER_PROMPTS=1 (already included in the config above).

Conclusion

From Claude Code to Bronto, all connected through OpenTelemetry — a few environment variables, an open source agent, and you have logs and insights flowing in. Once it's set up, you get full visibility into how your AI tooling is being used, cost metrics, and what's actually happening under the hood.

Appendix: Full OTel Collector Config

Receivers:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
        cors:
          allowed_origins:
            - "*"

Exporters:

exporters:
  otlphttp/brontobytes:
    logs_endpoint: "https://ingestion.eu.bronto.io/v1/logs"
    compression: gzip
    headers:
      x-bronto-api-key: "INGESTION_API_KEY_GOES_HERE"
    timeout: 30s
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 120s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 5000

Pipelines:

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, resource, batch, transform]
      exporters: [otlphttp/brontobytes]

Try Bronto Free — Send Your First Claude Logs

The 4L's of Logging — The New Standard for the AI Era

Patrick Londa — Thu, 21 May 2026 16:18:59 +0000

Authored by Noel Ruane & Trevor Parsons

Traditionally, the world of logging had 3L's — low cost, low latency, and large scale — and the accepted wisdom was that you'd have to compromise by choosing two of the three.

Until recently, the logging market bore this out. Solutions fell into one of three categories:

Expensive and fast — Datadog, ELK-type solutions
Slow and cheap(er) — index-free solutions like Grafana Loki (still relatively expensive when you actually search)
Tweeners — expensive, relatively slow, not great at scale — think CloudWatch, Google Cloud Logging

Anyone who's ever hired builders knows the iron rule: fast, cheap, or quality — pick two.

But the world has changed with the arrival of AI. As everyone builds and innovates around LLMs and agents, achieving all three L's has become more essential than ever. And AI has added a fourth requirement to the list: long-term retention.

3 Becomes 4: What Long-Term Retention Opens Up

With a short retention window — 3, 15, or 30 days, as most solutions provide — logs are accessible only for outages and system issues severe enough to warrant immediate attention.

Long-term retention opens up an entirely new set of use cases.

A common example: comparing customer API request latencies or conversion metrics month-on-month, looking at seasonality effects, or preparing data for a customer QBR. You might want to analyze customers in aggregate to spot trends, or drill into a specific customer's history.

Support issues can be recent or ongoing over months. Being able to look beyond a few days of data is often the difference between resolving a customer issue properly and leaving it half-answered.

In an AI world where agents can interrogate your data and solve problems at a completely different scale, long-term hot retention is essential — not the cumbersome, costly process of rehydrating logs from archives that most providers offer. The common workaround of dumping logs to S3 and spinning up Athena is another great example of the kind of toil that shouldn't exist in modern logging.

The importance of long-term hot data is borne out by our own customers: the longer the data retention, the more use cases open up, the longer the time periods searched, and the more value customers extract from their data. This value will only increase as agents become more powerful.

The 4L's Set a New Standard for the AI Era

Microsoft's Satya Nadella put it well: AI is at the intersection of where intelligence meets data. If you don't have the data, you can't point AI at it. The new requirements for data in the AI era map directly to the 4L's:

Large Scale — data that is always and easily accessible
Long-Term Retention — data retained so AI can analyze it and take action on it
Low Cost — storage and search must be cost-efficient as volumes see exponential growth
Low Latency — fast search and querying

As Bessemer Venture Partners notes in their Roadmap for AI Infrastructure, the volume of unstructured data is projected to reach 612 zettabytes by 2030, driven by ML/AI and synthetic data generation. Log data is one specific type where this applies acutely.

Here's why logging requirements are intensifying in the AI era:

Systems of action are replacing systems of record. As AI-native apps don't just store data but act on it, having log data accessible and hot means AI models can actually do something meaningful with it. The era of retaining logs for 30 days and locking them in cold storage is ending.

AI-led Root Cause Analysis. We're already seeing AIs perform RCA using log data to help teams solve problems faster. BrontoScope, for example, automatically runs an investigation that assesses how widespread an error is — figuring out what queries to run, assessing results, running further queries, and delivering an immediate assessment with recommendations. It can save anywhere from minutes to hours per incident. As MCP becomes standard, log data will be analyzable with additional context from other systems — replicating what engineers do manually today when piecing together evidence.

LLMs are non-deterministic. Given the unpredictable nature of LLMs, logs will play an even more important role in understanding what happened — whether from a support, debugging, audit, security, or compliance perspective. The same action can deliver different results, making logs essential for showing exactly what occurred. (See our guide to collecting Ollama logs to get started.)

Regulation. EU AI Act Article 19, for example, will require logs of certain higher-risk AI systems to be retained for at least 6 months from 2026.

A Purpose-Built Adaptive Technology for Log Data

How do you actually achieve all 4L's? The short answer: treat logs as a first-class citizen and build a purpose-built data store for them.

Until now, vendors have always repurposed existing datastores for logging:

Elasticsearch — at the core of most logging solutions for two decades. Super fast because it indexes everything, but super expensive due to data bloat and operational complexity.
ClickHouse — more recently vendors have turned to ClickHouse. Great for metrics, good for some logs — but only if you know the shape of your data in advance, which is a non-starter for a general-purpose logging platform. (Full write-up: Why ClickHouse fails as a general-purpose logging solution)
Snowflake — vendors like Observe have turned to Snowflake to address the cost problem, but customers regularly report search latency issues and burning through CPU credits when searching large data volumes.

The key learning from two decades in this space: not all log data should be treated equally. Don't just "index everything" (inverted indexes) or at the other extreme "index nothing" (index-free solutions). Instead, take a more thoughtful approach — let the system adapt and optimize based on the shape of your data and the queries you run.

These optimizations operate at every layer:

Indexing: We combine multiple patent-pending adaptive compact indexes. The platform automatically decides which to apply based on data and query patterns. Techniques include Bloom filters, summary indexes, partition indexing, automatic log-to-metric conversion, and additional patent-pending technologies.

Storage: Our storage tier routes data to the most appropriate structure based on its shape and how it will be searched — columnar store, specialized data structures, high compression, partial decoding, and storage tier coordination.

Search: Specialized algorithms applied depending on the query being run, highly parallelized to operate at any scale.

For deeper reading: Why Bronto is so fast at searching logs, or the Log Management Cost Trap series — Part 1: Ingestion, Part 2: Storage, Part 3: Search.

No Maintenance Required

One important point often overlooked: we see vendors trying to solve the cost problem by deploying within an organization's cloud (BYOC), using the customer's hardware to reduce costs. This requires deploying a logging platform into your environment and having the vendor manage it there. Unless you have tight data regulations, this feels like an overly complex backward step.

Bronto is built on a modern cloud multi-tenanted architecture. No setup, no maintenance, no management required by the customer. We take advantage of compute/storage separation and independently scaling services.

AI = the Intersection of Intelligence and Data

If your current provider is building AI capabilities on top of an old, repurposed, or expensive logging platform — one with 2 L's instead of 4 — you won't be able to take full advantage of the AI revolution.

The 4L's aren't a nice-to-have. In the AI era, they're the baseline.

See the 4L's in Action

BrontoScope: AI-Powered Error Investigations

Patrick Londa — Thu, 21 May 2026 14:37:22 +0000

Authored by Marco Aquilanti

Today we're introducing BrontoScope, one of the Bronto AI Labs initiatives aimed at reducing user toil, increasing team efficiency, and reducing MTTR.

The Problem with AI in Observability

Almost every software company is adding AI features to their products — often with mixed results. As a user, I'm frequently annoyed by the continuous stream of AI features popping up everywhere: messaging apps that want you to chat with an LLM while you're looking for your friends, search engines surfacing LLM answers first and leaving you wondering whether what you're reading is true or a hallucination.

The observability space is no exception. Many products are being "enriched" with AI features, but most are missing the point. Here's why.

Observability has always been hard. A production system can easily produce terabytes of logs, millions of traces, and millions of metrics every hour — too much for any human to easily inspect. LLMs should be the next pillar in observability, reducing burden and improving reliability. But only if focused on making the user's life simpler.

Most current AI features in observability actually make the user's life harder by:

Requiring a detailed prompt as input — users must invest significant time crafting prompts to get well-structured responses
Producing long, verbose text responses — even when the AI has nailed the request, the answer is often diluted across lines and lines of text
Taking too long — complex multi-step LLM workflows leave users waiting far too long for answers during an incident

The Bronto Approach

At Bronto, we're extending the logging platform with LLM capabilities focused on one goal: automating recurring work patterns to make the user's life simpler, not harder.

Our Bronto Labs initiative is built around three tools:

Auto-Parsing — using AI to automatically structure logs
AI Dashboard Creation — generating dashboards from natural language
BrontoScope — AI-powered incident investigation

The philosophy behind all of these: before adding any new feature, we make sure it will be genuinely useful to most users and won't slow down or hinder any of their existing tasks.

BrontoScope

Incidents don't wait for business hours. When an alert fires at 3am, one or a few on-call engineers need to move fast — often without access to the domain experts who know the affected system best.

The first steps of any incident are always the same:

Understand the scope of the incident
Estimate the impact on customers and the broader system
Assign a priority and decide how to tackle it

Staying calm, thinking clearly, and acting quickly are all required — even when you've just been woken up. But too much haste leads to incorrect diagnosis.

LLMs can help enormously in these scenarios — they can summarize large amounts of data in seconds and are not affected by panic, confusion, or a 3am wake-up call.

BrontoScope automates the incident investigation process with a single click on any error event in your logs. The LLM writes and runs tens of queries against your data, analyzes the results, generates a summary report, and delivers it to you in just a few seconds.

What the Report Includes

Scope — when the errors started appearing, and which users, customers, services, regions, or hosts are affected
Probable causes — resource exhaustion, network issues, software bugs, traffic spikes, etc.
Suggestions — how to stop the error occurring or how to continue the investigation
Supporting data — the query results and charts that led the LLM to its conclusions, so you can validate that the model isn't hallucinating

How It Works

The process works in stages: first, the LLM analyzes the error and its surrounding context to guide subsequent data retrieval. The search engine then queries the relevant data and presents all findings to the LLM in a single comprehensive prompt — essentially, an ad-hoc dashboard built around the error and composed of many charts. The final response is streamed to the user via Server-Sent Events, allowing them to read the output as it's generated in real time.

BrontoScope is powered by AWS Bedrock's most advanced AI models, ensuring all data is processed within the AWS ecosystem — prompts and responses are never stored or shared with model providers or third parties.

Why It Actually Makes Life Easier

No prompt required — just click on a log event. The LLM analyzes and understands the error, writes its own filter to find similar occurrences, and scans the data autonomously
Concise reports — goes straight to the point, with charts included to maximize the information density
Fast — in most cases the report is streamed to the user in under 10 seconds, even though tens of queries are run per investigation, thanks to the speed of Bronto's search engine

Availability

BrontoScope is currently available on request and is being used internally by the Bronto team as well as by a number of design partner customers in real-world situations. Improvements will be made in the coming months.

This is just one of the AI features being developed at Bronto — stay tuned for future posts, or join our AI initiative and help shape what we build next.

Join Bronto Labs

Log Parsing with AI at Bronto

Patrick Londa — Thu, 21 May 2026 01:06:23 +0000

Authored by Gary Nicholls

This post follows on from our AWS Nova log benchmarking article, where we explored how smaller LLMs perform on log analysis tasks. That earlier post highlighted that LLMs are surprisingly good at parsing logs. While that work focused on understanding logs, this post tackles an earlier step: automatically structuring logs using AI.

Logging Origins

Logs are one of the oldest — and still most valuable — forms of observability. Mainframes and early Unix systems were already using logs to record system activity, with tools like syslog dating back to the early 1980s.

Even as systems have become more distributed and complex, logs remain foundational, especially for investigating issues when things go wrong. Logs are typically written to local files before being shipped to modern observability platforms using agents like OpenTelemetry or Fluent Bit.

Why So Many Formats?

The OpenTelemetry (OTel) project is encouraging the adoption of structured JSON logs — and that's a good thing. Structured logs are easier to search, more human-readable, safer to manipulate, and more cloud-native.

But the reality isn't that simple. Many systems still generate unstructured or semi-structured logs where key=value pairs are embedded inside free-text messages. And even among structured formats, things vary wildly — timestamps alone appear in dozens of different formats.

Logs reflect the unique fingerprint of each tech stack:

syslog — still widely used, with quirks in its timestamp formatting
Apache — uses the Common Log Format
nginx — has its own custom variant
Java apps — use logback, log4j, or slf4j
AWS services — often emit structured JSON or a hybrid of text and JSON

With no single standard, Bronto set out to solve the problem in an innovative way — using AI to generate parsers automatically, reducing the toil and complexity that users typically face.

Automated Log Parsing

Parsing logs in real time is a performance-critical operation. When ingesting millions of events per second, every millisecond counts. Regex-based parsing can be complex and hard to maintain, requiring expertise in tools like Grok or Dissect — and can become a bottleneck at scale when applied indiscriminately.

At Bronto, we use a multi-layered approach that separates offline detection from online parsing. Online parsing happens in real time as part of the ingestion pipeline; offline detection occurs outside the pipeline with a short delay. This hybrid approach ensures speed without sacrificing flexibility, while reducing user toil.

Layer 1: Curated Java Parsers

We maintain a library of high-performance Java-based parsers, optimized for the most common formats seen at high volumes across multiple customers. These are purpose-built for speed and designed to fail fast if they encounter a log that doesn't match their expected format.

After applying a Java parser, we run additional lightweight processors to normalize key fields:

Timestamp parser — auto-detects and normalizes varied timestamp formats
Log level parser — maps diverse severity keywords into five standard levels
KVP parser — extracts key=value pairs from the message or body, even if only present in some events

Layer 2: Dissect and Grok Fallback

For less common but still important formats, we fall back to Dissect or Grok:

Dissect — fast and great for structured, delimiter-based logs
Grok — more flexible, supports regex-based parsing, but comes at a performance cost

Bronto maintains a large database of both dissect and grok patterns. Due to their runtime cost, we don't attempt to apply every pattern to every event online. Instead:

We sample log events offline
Match them against our full pattern library
If a match is found, we automatically assign a parser hint to the dataset
Future events in that dataset are parsed using the matched pattern
We gather metrics on parse quality per dataset and periodically revalidate hints

Layer 3: AI-Generated Parsing

When we encounter unknown or proprietary formats, other tools might require users to handcraft regexes through a UI. At Bronto, we let AI do the work.

When enabled, we send a sample of the dataset to an internal AI engine that analyzes the log structure and generates a custom dissect pattern. We test the pattern against a wider sample. If it matches a high percentage of events, we present the pattern and sample results to the user — they can tweak field names if desired — and once approved, the parser is saved and applied automatically to all future events in that dataset.

A Worked Example

Suppose your application logs look like this:

After analyzing hundreds of lines, the AI generates a dissect pattern:

The parsed result looks like this:

`app_name`	`timestamp`	`log_level`	`message`
APP01	2024-07-03 12:50:59	WARN	Invalid sessionId: sessionId=expired
APP01	2024-07-03 12:49:41	INFO	Authentication token issued
APP01	2024-07-03 12:48:27	INFO	User logout: userId=14141
APP01	2024-07-03 12:47:10	INFO	New login attempt
APP01	2024-07-03 12:45:37	INFO	Form submitted: formId=contact-us

The KVP parser then further extracts fields like sessionId, userId, and formId from the message value.

Under the Hood: AWS Bedrock

We use AWS Bedrock as a managed service to access LLMs (including Claude). Our infrastructure chooses the most appropriate model for each application and sends structured prompts — for example, instructing the LLM which patterns to avoid and how to handle keys like timestamps. The user doesn't have to worry about models or prompts; they just use the application.

Bedrock also provides important SaaS-grade guarantees:

Built-in safeguards to detect and filter harmful content
Never stores or uses our data to train models
All data remains within the AWS network
Works seamlessly with Lambda and S3 — no platform rearchitecting required

Looking Ahead

At Bronto we believe parsing should be fast, accurate, and hands-free. Today we generate dissect patterns using AI. Soon we'll be generating Grok patterns too — bringing AI to even more complex and less structured formats.

As OTel continues to push for JSON-based structured logging, the hope is that log parsing becomes a less painful problem over time. But until then, automated, adaptive parsing isn't just a convenience — it's a necessity.

Summary

Bronto combines curated Java parsers, flexible Dissect/Grok matching, and AI-powered pattern generation into a unified pipeline for parsing any log format, structured or otherwise.

If your logs are weird or messy — we've got you.

Explore Bronto's AI Features

Benchmarking AWS Nova on Log Data: How It Compares to ChatGPT-3.5

Patrick Londa — Wed, 20 May 2026 21:18:45 +0000

Authored by Benoit Gaudin

This post explores the use of large language models (LLMs) for analyzing log data. To do so, we reproduced part of the An Assessment of ChatGPT on Log Data benchmark, originally conducted in 2023 by Intel researchers Priyanka Mudgal and Rita Wouhaybi.

While that initial benchmark used ChatGPT-3, our study evaluates the AWS Nova Micro model. Our goal: assess whether more recent, smaller, and cheaper models can match — or exceed — the performance of ChatGPT-3 from a few years ago. The economics are particularly interesting: Nova Micro's cost per input token is 14 times lower than GPT-3.5-turbo was two years ago.

Benchmark Setup

The original benchmark evaluated GPT-3.5-turbo across ten research questions grouped into four categories:

Log Parsing & Analytics — Can the model parse logs and identify errors, root causes, security events, and anomalies? Can it identify frequently used APIs?
Prediction — Can it predict future log events based on past logs?
Summarization — Can it summarize single and multiple log messages?
General Capabilities — Can it handle bulk log data, and what message lengths can it process?

Experiments used datasets from the Loghub collection — 2,000 labeled log messages from various systems (Windows, Linux, mobile, distributed, etc.).

Our experiment reused the same methodology and the same 19 Loghub datasets, with these differences:

We evaluated AWS Nova Micro rather than GPT-3.5-turbo
We focused on the first three categories (7 questions) — the fourth category covers context window size, which is no longer a meaningful differentiator (GPT-3.5-turbo: 16,385 tokens; Nova Micro: 128,000 tokens)
Where the original benchmark tested multiple input sizes (e.g. 5, 10, 50 log entries), we used only the maximum (50), to give the model the most context
Results were manually evaluated by a human, using the same prompts as the original

Category	Question	Prompt	Description
Log Parsing	Q1	Extract the log template and variables from this log message.	How does the model perform on log parsing?
Log Analytics	Q2	Summarize the errors and warnings and identify the root cause.	Can it extract errors and root causes from raw logs?
Log Analytics	Q3	Show the APIs called most with count.	Can it perform advanced analytics tasks?
Log Analytics	Q4	Are there any malicious users, URLs, IPs, and connection status?	Can it extract security information?
Log Analytics	Q5	Detect the anomalies from the following log messages.	Can it detect anomalies?
Log Analytics	Q6	Predict the next 10 log events based on these log messages.	Can it predict future events?
Log Summarization	Q7	Summarize the log message.	Can it summarize a single log message?

Results: AWS Nova Micro's Performance

Prompt	Correct Answers	Remarks
Extract log template and variables	17/19 (89%)	Failed on HDFS logs; IDs not always categorized accurately
Summarize errors and identify root cause	10/19 (53%)	Erroneously reports warnings in Hadoop logs; confuses timestamps and error codes in HPC; over-reports issues in HealthApp and Mac logs
Show most-called APIs with count	4/19 (21%)	Counting is very challenging; many datasets lack API-related entries; model over-reports results that don't make sense
Detect malicious users, URLs, IPs	18/19 (95%)	High accuracy, but hard to conclude on the general case as no obvious security issues were present in the sampled logs
Detect anomalies	9/19 (47%)	Reports anomalies based on irrelevant criteria (e.g. entries that "occur towards the end of the sample" or are "repetitive")
Predict next 10 log events	0/19 (0%)	Even for extremely repetitive logs, IDs and timestamps are not predicted correctly
Summarize a single log message	16/19 (84%)	Good results overall; challenging for unfamiliar log formats without named fields

In summary, our evaluation confirms the findings of the original benchmark: similar to ChatGPT-3, Nova Micro performs well at parsing and summarizing log data. Other types of analysis — counting, anomaly detection, prediction — remain challenging for LLMs.

The malicious content detection result (95%) looks strong, but needs a caveat: the sampled datasets didn't contain clearly malicious entries. The model didn't produce false positives here, which is valuable in itself — especially compared to anomaly detection, where false positives were common.

This benchmark demonstrates that it is now possible to achieve parsing and summarising of log data in a much more cost-effective way.

Reflection on Datasets

The Loghub collection is invaluable for reproducible benchmarking — without it, meaningful cross-benchmark comparisons would be impossible. That said, the datasets have some limitations worth noting.

At Bronto, we work frequently with log types common in real-world production environments: CDN logs, web access logs, AWS CloudTrail audit logs, application logs. LLMs tend to have a strong understanding of these formats because they're widely documented and structured.

Structured logs change the picture significantly. When we ran Q2 and Q3 prompts against synthetic structured CDN log data (based on real examples), the model performed substantially better:

For Q2 (error identification), the model perfectly identified HTTP errors by associating status codes ≥ 400 with errors — even though the field name never used the word "error". It correctly categorized 400 (Client-Side), 404 (Not Found), 500 (Internal Server Error), and 503 (Service Unavailable).
For Q3 (most-called APIs), the model correctly identified the reqPath field as representing API endpoints and extracted the top results accurately.

Counting remains a consistent weakness across all dataset types. When Q3 requires providing a count of the most common API calls, the model's counts are frequently inaccurate regardless of dataset.

One additional observation: several Loghub datasets (HPC, HealthApp, BGL, Proxifier) appear to be uncommon enough that Nova Micro doesn't have a solid prior understanding of them. When asked to generate sample logs for these systems, the output doesn't resemble the actual Loghub data — suggesting the model is less reliable when operating outside familiar territory.

Conclusion

This benchmark reproduces the 2023 ChatGPT log analysis study using AWS Nova Micro. The results are strikingly similar — with one major difference: the cost per token is 14x lower.

Given that log data is notoriously voluminous, this cost difference matters enormously for any production use of LLMs in log analysis pipelines.

The Loghub datasets are also not fully representative of what most production logging systems generate. Real-world logs — web access, CDN, application, audit — tend to be more structured and more familiar to LLMs, which leads to better performance than the benchmark scores suggest.

We believe LLMs have genuine potential to improve production logging systems, particularly for analyzing the common, structured log formats that make up the majority of real-world observability data.

Explore Bronto's AI Features

Why ClickHouse Fails as a General-Purpose Logging Solution

Patrick Londa — Wed, 20 May 2026 21:18:18 +0000

Authored by Mike Neville-O'Neill

Let's talk about ClickHouse. It's fast, it's efficient, it's open source. But it's designed to be a general-purpose analytical database, which makes it challenging — in terms of time and resources — to use for logging at scale.

I've spent over a decade working on logging systems and watched countless organizations struggle with the same pattern: a team frustrated with expensive legacy log management turns to ClickHouse, excited by early performance wins on well-structured data. They rebuild critical logging infrastructure around it, roll it out, and everything seems great... until it doesn't.

Why Teams Try ClickHouse for Logs

The appeal is undeniable. When you first point ClickHouse at a clean, structured dataset, the performance is genuinely impressive:

Speed — Built for analytical queries, can scan billions of rows in seconds on modest hardware
Cost — Open source, so licensing costs are zero
Familiarity — If your team works with SQL, the learning curve is minimal
Flexibility — Complete control over your data, schema, and retention policies

Early benchmarks usually reinforce this perception. Testing on controlled, clean data with predictable patterns, ClickHouse shines. You might start with a specific subset of logs — API access logs or well-structured application events — and see impressive query speeds.

The problem isn't that ClickHouse doesn't work for logs. The problem is that it doesn't work for all logs, all the time, at scale, without significant engineering investment. And that's where organizations get trapped.

The Architectural Mismatch

Logs in the wild are messy, inconsistent, and unpredictable — the direct opposite of what ClickHouse was designed to handle.

The Columnar Conundrum

ClickHouse's columnar storage engine is optimized for queries that scan large portions of specific columns — brilliant for analytics workloads with well-defined dimensions. But logs are fundamentally different:

Unpredictable schemas — Log formats change; new fields appear and disappear without warning
Nested structures — Modern logs, especially from cloud environments, contain deeply nested JSON with inconsistent structures
High cardinality — Fields like session IDs and request IDs have enormous numbers of unique values, which presents challenges for ClickHouse
Mixed query patterns — Log analysis involves aggregation, filtering, full-text search, and pattern matching — not just the analytical queries ClickHouse excels at

Forcing logs into ClickHouse is like racing a Formula 1 car on an off-road trail — impressive engineering, wrong application.

Schema Rigidity vs. Log Reality

In the real world, logs evolve constantly:

Developers add new fields to debug specific issues
Third-party systems change their log formats without notice
Cloud providers modify their event structures
Microservices introduce different logging conventions across teams

With ClickHouse, schema changes require careful planning to avoid additional overhead. A purpose-built logging system accommodates this reality by adapting to your logs — not the other way around.

Indexing Limitations

ClickHouse's primary indexing mechanism — the sparse primary index — works by creating index marks every N rows (typically thousands). This excels for batch analytical queries but creates significant compromises for log search:

Limited selectivity — Works best when scanning large portions of the dataset, not when looking for specific events
Suboptimal for high-cardinality fields — User IDs, session IDs, trace IDs all struggle with this approach
Text search limitations — Full-text search is functional but not optimized for the complex pattern matching common in log analysis

Real-World Scaling Shatters the Illusion

The true challenges emerge as your ClickHouse logging solution scales. What worked smoothly in development with gigabytes becomes increasingly problematic at terabyte or petabyte scale.

Query Performance Degrades

Initially, queries return in seconds or less. As data grows, the same searches can take minutes. Without careful tuning and constant optimization, query performance becomes increasingly unpredictable — your team ends up spending more time optimizing queries than using them to solve problems.

Operational Overhead Becomes Crushing

Managing a ClickHouse cluster at scale is a specialized skill. You'll need to handle:

Rebalancing — Manually triggering operations as data distribution becomes uneven
Storage management — Careful tuning of merge settings, partitioning schemes, and TTL rules
Resource allocation — Preventing resource-intensive queries from impacting overall system performance
Schema evolution — Adding, removing, or modifying columns requires careful planning
Multi-tenancy — ClickHouse has limited built-in multi-tenancy, creating additional complexity

One large e-commerce team shared with us that they had six full-time engineers dedicated just to keeping their ClickHouse logging infrastructure operational — an enormous hidden cost that rarely factors into initial calculations.

Reliability Issues Emerge

Ingestion pipeline failures — As log volumes spike, ingestion often falls behind because ingestion and search aren't separated
Complex failure modes — Troubleshooting ClickHouse problems requires deep database expertise most DevOps teams don't have
Recovery complexity — Restoring from failures or data corruption is time-consuming and error-prone

The DIY Trap

When engineers suggest "just use ClickHouse for logging," they're often underestimating what that actually means. ClickHouse isn't a logging platform — it's a powerful but complex analytical database that requires substantial expertise to turn into a production-ready observability solution.

The ClickHouse Learning Curve

ClickHouse has unique concepts that take months to develop expertise in:

MergeTree engines — Understanding when to use ReplicatedMergeTree vs. ReplacingMergeTree vs. dozens of other variants
Partitioning and primary key design — Critical decisions that affect query performance and can't easily be changed later
Merge behavior tuning — Configuring background processes that determine how your data gets organized and compressed

Building the Missing Pieces

Even with ClickHouse expertise, you still need to build an entire logging platform around it:

Ingestion pipelines that handle backpressure, parse multiple log formats, and deal with schema evolution
Query interfaces — APIs and UIs that let users actually search and analyze their logs
Access control — multi-tenancy and permissions
Alerting and monitoring systems
Operational tooling — backup/recovery, cluster health monitoring, schema migrations

The Hidden Costs

What starts as "let's save money by using open source" often becomes:

Months of engineering time just to get basic functionality working
Ongoing operational overhead that scales with data volume
Opportunity cost of engineers focusing on infrastructure instead of product features
Risk of outages due to misconfiguration or lack of operational expertise

For most teams, the total cost of ownership of a DIY ClickHouse solution exceeds purpose-built logging platforms.

Built for Logs, Not Analytics

Instead of adapting an analytics database for logging, the right approach is to build a system specifically for the messy, unpredictable nature of log data:

Separation of compute and storage — Decoupling storage from compute so each scales independently, keeping searches fast regardless of data volume
Indexless architecture with Bloom filters — Sub-second search performance without the overhead and complexity of traditional indexing, particularly effective for high-cardinality fields
Schema-agnostic ingestion — New fields, changing structures, and varying formats handled automatically without schema migrations or pipeline adjustments
Optimized for search patterns — Full-text search, pattern matching, filtering, and aggregation all handled natively

The result:

Sub-second search on terabytes of data
Seconds on petabytes — no rehydration from cold storage
12-month retention by default
Zero operational overhead
Transparent, predictable pricing

Conclusion

ClickHouse is an impressive technology for what it was designed to do: structured data analytics at scale. But turning it into a general-purpose logging solution requires enormous engineering investment and still results in fundamental compromises.

When you can instantly search across years of data, logs shift from a compliance checkbox or last-resort troubleshooting tool to an active part of your operational toolkit.

Instead of spending years rebuilding what already exists, consider whether your team's time is better spent solving your core business problems. The logging layer should be a foundation that just works — not a perpetual engineering project.

See Bronto in Action

Breaking Logging's Flywheel of Compromises

Patrick Londa — Tue, 19 May 2026 18:26:09 +0000

Authored by Mike Neville-O'Neill

Let's face it — logging is broken. Not just a little broken, but fundamentally misaligned with the needs of modern engineering teams. At a recent AWS Summit talk in London, Benoit Gaudin (our Head of Infrastructure) and I shared Bronto's vision for fixing this mess once and for all.

The Problem We're All Living In

If you're running any significant infrastructure today, you're probably stuck in what we call the "3C flywheel of compromises":

Cost — Logging at scale has become ridiculously expensive
Coverage — So you cut corners, dropping those infra logs and long-tail workflows
Complexity — And end up with a Frankenstein's monster of 5–8 different systems duct-taped together

This isn't just inefficient — it's actively harmful. Engineers end up building parallel solutions just to get basic visibility because the main tool is too limited, too slow, or too expensive.

Logs Matter More Than Ever

Logs aren't just a compliance checkbox anymore. They're your operational ground truth in the AI era.

They feed your LLMs. They power your agents. They're your audit trail, your RAG source, your behavioral training set. And one log message from an LLM-based system might contain 50–100 nested events in a single payload.

Try scaling that with a solution built before the separation of compute and storage was even a thing.

How We're Breaking the Cycle

Bronto was built to tackle this head-on with three non-negotiable capabilities:

Subsecond search on all logs — whether they're two seconds or two years old
Petabyte-scale retention — no infrastructure for you to manage
Completely different pricing — think cents per GB, not dollars

The platform is built natively on AWS (S3, Lambda, DynamoDB), but engineered so you don't have to deal with pipelines, pre-processing, or glue code.

Bronto's Architectural Advantage

The ingestion layer accepts data from standard sources — OpenTelemetry Collector, FluentD, FluentBit — through HTTP endpoints, with AWS EC2 load balancers doing the heavy lifting. Data is buffered through Kafka (AWS MSK), but then things diverge from the standard playbook.

Instead of traditional approaches, data is processed from Kafka and written to S3 in a proprietary format that borrows techniques from data analytics: data partitioning, Bloom filtering, push predicates, compression, and columnar-based formats. Metadata lives in DynamoDB for speed.

The real magic happens at search time. When you query through the UI or API, Lambda functions launch in parallel and process data directly from S3. No overprovisioning for big queries — horizontal scaling on demand, paying only while functions run.

This architecture is what enables both the performance (subsecond on terabytes, seconds on petabytes) and the pricing model. No expensive clusters running 24/7 — just cloud resources used exactly when and where they're needed.

Real Teams, Real Results

API-First Content Platform

A team running a massive content delivery platform, serving APIs behind a global CDN for websites, mobile apps, and e-commerce systems. Every request hits their API with a unique key — they need to trace errors, group by status codes, and export logs to their own customers.

Before Bronto

40TB monthly ingestion cap
30+ minute query times (when they worked at all)
Dashboards that routinely failed
Constant budget pressure

After Bronto

Boosted ingestion to 60TB monthly
Cut their logging bill in half
Complex multi-day queries now return in subseconds
Built reliable log exports for their own customers

Their exact words? "Bronto changed our lives." A logging tool. Actually improving engineers' lives.

Global SaaS Project Management Platform

A company running a suite of SaaS tools across distributed cloud services and product lines.

Before Bronto

Graylog for live logs
S3 for long-term storage
HAProxy logs dumped into S3 with gnarly Athena queries
A mix of Athena, Superset, and QuickSight for analytics
Just 1–2 days of retention across most systems

After Bronto

Everything centralized — HAProxy, Kubernetes, application logs, audit trails
Extended to 90-day hot retention
Real dashboards tracking error spikes, traffic anomalies, and app version drift
Engineers focused on product, not maintaining logging infrastructure

They went from managing logs to actually using them.

Logs as Your Secret Weapon

Your log data is massively undervalued — not because it lacks signal, but because current tooling hides that signal behind cost barriers, friction, and compromises.

Logs used to be a liability. With the right approach, they can be your secret weapon.

We're building Bronto to be for logging what Dyson was for vacuum cleaners, what iPhone was for smartphones, and what Tesla was for electric cars — a complete reimagining of what's possible when you refuse to accept the status quo.

After all, when was the last time your logging tool made your life better instead of worse?

See Bronto in Action

The CDN Logging Crisis

Patrick Londa — Tue, 19 May 2026 17:39:56 +0000

Authored by Benoit Gaudin

Every second, your CDN is generating thousands of logs that tell a critical story about your application's performance, security, and user experience. For large enterprises, this can mean terabytes of log data every day — data that contains invaluable insights about your business.

But here's the uncomfortable truth: most organizations capture only a small fraction of their CDN logs, and retain that limited data for just days or weeks. This isn't because engineering teams don't understand the value. It's because the economics of traditional logging solutions make comprehensive CDN logging prohibitively expensive.

The result? Critical blind spots that can be extremely costly during outages, security breaches, or major events.

Welcome to the flywheel of compromises:

Cost — Traditional logging vendors charge egregious per-GB rates that make comprehensive CDN logging unaffordable
Coverage — Companies respond by severely limiting what logs they collect and how long they retain them
Complexity — To compensate for coverage gaps, teams cobble together 5–8 different logging solutions, creating a management nightmare

The Current State of CDN Logging

The observability sector today resembles markets before transformative innovation — vacuum cleaners before Dyson, mobile phones before iPhone, electric cars before Tesla. Existing solutions were designed for a completely different era: before the separation of compute and storage, before the explosion of log data volumes, and certainly before the demands of the AI era.

Consider how most logging vendors operate today:

Datadog charges around $2–5 per GB for log ingestion with 15-day retention. A company generating 10TB of CDN logs daily could pay upwards of $600,000 per month
Splunk forces customers into complex licensing schemes that effectively limit how much data they can realistically log
New Relic and other vendors offer marginally better pricing but still force unacceptable trade-offs between cost and coverage

What's most frustrating is that these pricing models persist despite dramatic changes in the underlying technology. The separation of compute and storage has revolutionized data economics across virtually every other category of software, yet logging vendors continue to operate on business models created 15 years ago.

A Hypothetical (But Entirely Plausible) Scenario

To illustrate the real-world impact of incomplete CDN logging, consider this:

A week before a major live streaming event, a provider's engineering team makes a routine CDN configuration change. Under normal traffic loads, the misconfiguration goes unnoticed — cache hit ratios remain stable and performance appears normal.

After a week, any trace of the configuration change disappears from their logs due to their 7-day retention policy. Capacity planning teams review infrastructure and assume current backend capacity can handle the anticipated load — after all, it worked fine during the last similar event. Unfortunately, the now-invisible change makes that assumption dangerously wrong.

During the live event, CDN cache efficiency plummets under heavy load. Backend servers get hit much harder than expected. Users experience buffering and connection problems, but the operations team struggles to diagnose the root cause.

By the time they identify the issue — tracing it back to the forgotten configuration change — the damage is done. Over a million viewers have abandoned the stream, social media is flooded with complaints, and the company's stock takes a hit.

With complete CDN logging and longer retention, they could have:

Identified when the degradation trend first appeared, correlating it to the configuration change
Maintained visibility throughout the planning period
Quickly correlated the performance issues with the earlier change during the incident

Limited logging coverage transformed a minor configuration error into a major business incident. The cost of their logging "savings"? Potentially millions in lost ad revenue and subscription cancellations.

The Three Horsemen of the Logging Apocalypse

Cost Explosion

Traditional logging vendors price their products based on data volume, charging premium rates for both ingestion and storage. This pricing model was created when storage was genuinely expensive. In 2025, with cloud storage costs continuing to plummet, this model serves primarily to protect vendor margins.

For CDN logs — which are high-volume by nature — this creates an impossible equation. When faced with estimates of $500,000+ monthly for complete CDN logging, even the most data-driven organizations are forced to compromise.

Coverage Sacrifice

The inevitable result of cost pressure is reduced coverage. Organizations typically:

Ingest only a sample of the data
Limit retention to days instead of months
Exclude high-volume CDNs or regions entirely
Drop detailed fields that would aid troubleshooting

These compromises create dangerous blind spots. Intermittent issues, security threats that develop over time, and regional performance problems remain invisible. When an incident occurs, teams often discover they're missing exactly the data they need.

Complexity Creep

To compensate for coverage limitations, organizations implement a patchwork of supplementary solutions:

Self-hosted ELK stacks for longer-term storage (with all the maintenance overhead)
Cloud provider-specific logging solutions (AWS CloudWatch, GCP Logging)
Custom scripts to archive logs to object storage with rehydration workflows
Open-source tools for log analysis and visualization

The result is a Frankenstein's monster of logging infrastructure that no one fully understands, requires constant maintenance, and still fails to provide comprehensive visibility.

CDN Logging for the AI Era

These challenges are escalating as we enter the AI era:

Exploding volumes — Microservices, containers, and edge computing are all contributing to the data deluge
AI-powered analysis — ML systems require comprehensive, long-term data to identify patterns and anomalies effectively
Agentic applications — Autonomous applications require complete historical data to make intelligent decisions

Legacy logging business models simply cannot accommodate these realities. They weren't designed for terabytes of daily log ingestion, years of retention, or a world where AI agents might need to analyze months of historical CDN patterns.

A Different Approach

Solving the CDN logging crisis requires rebuilding the logging stack from the ground up — not incremental improvements on broken foundations. Three core principles drive the right approach:

1. Economics Aligned with Modern Infrastructure

Leveraging the separation of compute and storage to deliver CDN logging at a fraction of traditional costs:

90% cost reduction compared to Datadog and similar vendors
12-month retention by default
No charges for search or compute resources

2. Lightning-Fast Search Across Petabytes

"Tracey's Law": the faster you make log search, the more valuable logging becomes to an organization.

Sub-second search across terabytes of CDN logs
Seconds-long queries across petabytes
No rehydration from cold storage, ever
Fast dashboards even across months of data

When queries return in seconds instead of minutes (or timing out entirely), teams use logging data proactively rather than as a last resort.

3. A Single Unified Logging Layer

Eliminating the patchwork by providing one comprehensive logging layer:

All CDN providers in one place
Drop-in replacement for existing solutions
Two-line configuration change for implementation
Automatic parsing and PII removal

Breaking Free from the Flywheel

The CDN logging crisis isn't just a technical problem — it's a business problem with real implications for reliability, security, and user experience. For too long, organizations have accepted a dysfunctional status quo because there seemed to be no alternative.

"Every single word about the logging crisis resonates. We were spending over $400,000 monthly on CDN logging with Datadog, and still only capturing about 20% of our logs. With Bronto, we now have 100% coverage, 12-month retention, and our bill is under $40,000."

This isn't an incremental improvement — it's a fundamental reinvention of how logging works. Just as Apple reinvented the smartphone, Dyson reinvented the vacuum cleaner, and Tesla reinvented the electric car, the logging industry is overdue for the same transformation.

Bronto is reinventing logging from the ground up for the AI era. The team brings 150+ years of collective logging domain expertise, with previous experience building and scaling logging platforms at IBM, Rapid7, and Logentries.

See What 100% CDN Log Coverage Looks Like