Getting Started with Bits Agent Builder: An Introduction and Five Real-World Use Cases

Hi, I'm @futahashi, a Principal Engineer at Nulab.

On August 10, 2026, I gave a talk at Japan Datadog User Group Meetup #20 (JDDUG #20) in Sapporo. The topic was "Getting Started with Bits Agent Builder and Real-World Use Cases." Bits Agent Builder is a feature that lets you build custom agents inside Datadog, the observability platform. This post is a digest of that talk.

On a personal note, the trip was a genuinely rewarding one: I got to return the favor to people who had traveled all the way from Hokkaido to speak in Fukuoka, and I finally met someone from Classmethod whose work I've long admired.

Getting Started with Bits Agent Builder: Real-World Use Cases - Speaker Deck

speakerdeck.com

Why do we need AI agents?

With the rise of generative AI, work like research, summarization, and writing text or code became something a human could delegate and speed up. Tasks too complex for traditional programming came within reach, and I've genuinely felt the amount of toil — repetitive, automatable manual work — shrink as a result.

The next thing that matters is expanding the space where AI works autonomously, without waiting for human instructions. That's where agents come in.

You give an agent a goal, tools, constraints, and trigger conditions. From there, it starts work on its own — not only when a human asks, but when an alert fires or a schedule comes due — making judgment calls as the situation demands, operating tools, and carrying the task through to completion. Chaining together multiple tool operations and processing steps no longer requires assembling a program. In the talk, I described this shift as "growing AI into a worker that acts on its own."

The work best suited to agents, I think, is work that's too complex for static automation but too repetitive for a person to handle every single time. Handing that work off means humans no longer carry every task themselves, and can spend their time on higher-value activities: prioritization, decision-making, communicating with people, and improving the systems around them. The goal isn't simply to make work faster — it's to free humans from the work itself.

What is Bits Agent Builder?

Bits Agent Builder is a custom AI agent that runs inside Datadog, and it became generally available in June 2026. I see three strengths as distinctive:

• Rich context: It can use Datadog's rich data — logs, metrics, traces — directly.
• Diverse tools: Roughly 3,000 actions, around 50 services, and 4 MCPs. It can operate AWS, Kubernetes, GitHub, Slack, and more.
• Flexible triggers: It can be started from around 30 sources, including Workflows, Monitors, and Incidents.

In one sentence: it's an AI agent that runs on Datadog and uses the power of a unified observability platform to move your organization's operations forward.

Unlike an AI agent you fire up ad hoc on your local machine, this one is hosted on Datadog — so the data collected on the platform, the integrations you've already configured, and the wide range of triggers are all shared and reused across the team. Because it runs without depending on any individual's environment, you can make it a durable part of how your organization operates.

Four building blocks for creating and improving an agent

There are three ways to create an agent: from a prompt, from a Blueprint, or from scratch. Once created, you configure and improve it using four building blocks:

• Instructions: What you tell the agent. Its role, its procedures, its constraints.
• Tools: What the agent can use. If you don't give it a tool, it can't use it.
• Automation: When it starts. Schedules, Monitors, Incidents, and so on.
• Chat: The conversational interface. Run it, debug it, ask it questions.

Beyond these, you can also choose which model the agent uses, depending on the task.

Instructions aren't something you write once and walk away from. You watch how runs turn out, write down what you learn, and iteratively refine the agent's behavior over time.

Use cases

Bits Agent Builder hasn't been GA for long, but here are several agents I've actually run and found effective. I'm only showing the Slack reports here, but Notebook output is also worth trying — it renders as a clean, rich view.

Use case 1: Cross-organization monitoring of AWS Commitments, with purchase recommendations

AWS Commitments (Reserved Instances, Savings Plans, and the like) are AWS's mechanism for granting discounts in exchange for committing to usage up front — a handful of large purchases each year. In an organization running many AWS accounts, the types of commitments, when they were purchased, and when they expire end up scattered across accounts and services.

If you leave that management to individual teams, the frequency of review and the criteria for renewal decisions vary from team to team. Knowledge of how commitments work and when they apply also varies, so sometimes they aren't even recognized as an option and the opportunity is missed entirely. And when someone transfers to another team or changes roles, the history behind past decisions can disappear with them. Rather than relying on each team's knowledge and attention, you need shared policies and decision criteria, plus a mechanism that manages this continuously across the organization.

AWS's built-in features do surface expiration dates, utilization rates, and purchase recommendations. But organization-wide visibility lives in the management account, and a member account can generally only see its own information. Cross-account investigation requires the right permissions, and tracing actual usage across accounts means bouncing between multiple screens. Pulling all that together, layering your own company's policies on top, deciding what to renew and what to let expire, and getting that decision to the right person — doing all of that end to end with AWS features alone is not straightforward.

So I built an agent that monitors commitments across the entire organization against the same criteria. If all you want is a list, a script is enough. What I delegated to the agent was the judgment: taking usage and organizational policy into account, separating what it recommends renewing from what it doesn't, and reporting to Slack along with the reasoning behind each call. The value here is that finding these opportunities and continuing to review and decide becomes a shared organizational process, rather than something that depends on having a commitments expert or on any one team's effort.

Use case 2: Cross-organization cost trend analysis (Datadog and AWS spend)

Datadog and AWS costs span multiple accounts, services, and teams. If each team only reviews its own slice, the comparison windows and the thresholds for "this looks abnormal" won't line up, and changes that cross team boundaries can slip through. Rather than leaving continuous monitoring to individual teams, it's important to watch across the whole organization with the same lens and the same cadence, as shared work.

So when an anomaly appears in Datadog or AWS cost trends, I have an agent analyze it. Seeing that a bill went up or down is something an invoice or dashboard can tell you; the hard part is judging why it changed and whether it's a change that needs action. When this agent finds an abnormal cost trend, it doesn't stop there — it follows the related data and investigates the cause.

For Datadog cost analysis, it compares daily usage against the prior period to distinguish a temporary fluctuation from a real increase. For AWS cost analysis, it correlates telemetry, events, and logs alongside billing data, pinpointing the resources driving the increase down to the ID or ARN level. I find real value in getting insights that billing data alone could never produce.

What comes back from the agent isn't just the trend, but the insights from its investigation in a consistent format: likely reasons for the increase, the services and resources involved, and the evidence behind its conclusions. Because cross-organization monitoring and initial investigation are guaranteed by the system, whether an investigation happens at all — and how good it is — no longer depends on which team happens to have spare capacity. The people running cross-cutting operations, and the teams that need to act, can skip chasing data point by point and focus on verifying the identified cause and deciding what to do.

Use case 3: A scribe for incident response

Ever been asked "so, where are we right now?" in the middle of an incident? This agent takes the incident response conversation and records a summary, a timeline, action items, and responders in our company's own format. Pulling "what happened, and when did we do what" out of an unstructured conversation that's also full of side chatter is exactly the kind of judgment-heavy work you couldn't write as a program. Communication during response becomes structured, and responders can focus on the substance of fixing the incident.

As an aside, Datadog offers Incident Management, which handles timeline management, postmortem assistance, and more. We haven't adopted it yet, so I used Bits Agent Builder to replicate part of it. I'll skip the details here, but if it sounds relevant to you, it's well worth a look.

We also use Bits Investigation (formerly Bits AI SRE) in our incident response. By having multiple agents take over investigation and reporting, we're building an environment where humans can concentrate on directing the response and making decisions.

For more on how we use Bits Investigation and what's changed with it, see my earlier talk, "The Evolution of Bits AI SRE, Me, and My Organization."

The_Evolution_of_Bits_AI_SRE.pdf - Speaker Deck

speakerdeck.com

Use case 4: Security Signal triage

Plenty of security alerts turn out, after investigation, to need no action at all. But you can't know that until you investigate — so every one of them becomes a burden on a human.

When a Security Signal (Datadog's security event detection) fires, the agent immediately investigates the related information and produces an initial verdict. For example, a signal generated by a burst of AccessDenied errors caused by insufficient permissions was judged as benign — the detection itself was correct, but the activity wasn't malicious — and auto-archived with the reasoning and supporting information attached. Things that can't be confidently declared benign, such as a path traversal detection, get escalated to a human via a Security Case.

As an aside, Datadog also offers Bits Security Analyst. There are some signals it can't handle yet, though, so we're using Bits Agent Builder partly as a point of comparison. I'll leave the details for another time.

Use case 5: WAF log analysis and continuous improvement

A WAF isn't something you deploy, configure once, and forget. Writing a perfect rule set that blocks every attack from day one while never affecting legitimate requests is essentially impossible. You have to analyze real logs, confirm your rules are actually working, and improve them continuously. But finding anomalies in a massive volume of logs — while carefully assessing the blast radius so you don't accidentally block legitimate traffic — is far from easy.

This agent extracts anomalies from WAF logs and prioritizes them, evaluates how effective the existing rules are as defenses, and compiles improvement proposals into a report. The final rule changes are still a human call, made after reviewing the impact on the service; the agent handles the ongoing analysis and proposals. Down the road, this could extend into responding to attacks in real time.

Common pitfalls

Here are some mistakes people commonly hit when building agents. To make real failures easy to grasp, I deliberately exaggerated them in the slides.

Pitfall 1: Vague instructions produce the wrong procedure

This one comes from the cost analysis agent in use case 2. I assumed it would look at the cost metrics — instead, it tried to compute the numbers itself from usage volume and unit prices. The lesson: give the agent explicit instructions, and check the evidence behind its output so you can correct errors.

Pitfall 2: Context overflow

There's a limit to how much information AI can handle at once, and cramming too much processing into a single run leads to unintended behavior or outright errors. The fix is to split the task up and narrow the scope of the output.

What we got out of it

Outcome 1: Work starts moving proactively

Previously, a task would arise, a human would notice it, instruct the AI, and check the result. In other words, work sat still until a human noticed it and picked it up. Agents change that ordering. The task itself triggers the agent, which moves first, and what reaches the human is the investigation results and the points that need a decision. When a security signal fires, the initial investigation is already done before I even notice it.

The value isn't only in reduced hands-on time. Work that had been pushed aside as "we should do this, but nobody has the bandwidth" now starts moving the moment it arises. High-impact decisions stay with humans, but humans no longer carry every first response — they can concentrate on review and decision-making.

Outcome 2: Operations become articulated

Writing Instructions means putting your procedures into words. Ways of working that only existed in someone's head get written down. Once they're in words, the team can review them, improve them, and share the same decision criteria. Operational knowledge that lived with one person becomes an asset the team owns.

And when the agent doesn't behave as expected, that itself reveals where your own procedures and criteria were ambiguous.

Outcome 3: You can start small and grow big

You don't have to hand over all the judgment and execution from day one. Start with a narrow scope and delegate only investigation and recommendations. Humans review the results, and once you're satisfied with the quality, you can gradually expand its permissions and its scope.

The foundation is the three strengths I mentioned at the start. Because you can reuse the context already collected in Datadog, the integrations already connected, and the flexible triggers, you don't need to stand up a new execution environment every time — you can start small and grow both deeper and wider.

Managing usage and cost

In the talk I briefly covered how AI Credits work, and I also got a question from the audience about Bits Agent Builder pricing. Running agents continuously requires visibility into cost, not just capability.

Bits Agent Builder consumes AI Credits. According to Datadog's official pricing page, a single run averages about 3 credits — but note that actual consumption varies with task complexity and the volume of context the model processes.

The Agent Console (Preview) gives you a single place to review usage and cost across Datadog's AI agents (including Bits Agent Builder) and coding agents like Claude Code and Cursor. You can analyze usage over time by user, team, and agent, which makes it easy to pinpoint where consumption is running hot.

AI Credit Limits lets you set monthly caps at both the organization and per-user level. Once a cap is reached, new runs stop — a useful guardrail against unintended charges. Note, though, that these limits are shared across every Datadog AI feature that uses AI Credits, not just Bits Agent Builder, so you'll want to set them with your other usage in mind.

Wrapping up

The value of Bits Agent Builder is that it combines the rich context collected in Datadog, your already-connected tools, and a wide range of triggers — and turns complex, judgment-heavy work that humans used to carry into team operations that keep running on their own.

Three things I hoped to get across in this post:

• Agents are a good fit for work that's too complex for static automation and too repetitive for a person to handle every time.
• Bits Agent Builder leverages a unified observability platform to make investigation, judgment, and execution part of how your organization operates.
• Don't aim for perfection from the start — try something small and feed what you learn back into the Instructions, and you can grow it safely.

That said, handing work to AI doesn't make operational expertise unnecessary. Deciding what evidence to base a judgment on, how far execution should be allowed to go, and under what conditions to hand off to a human — those are human decisions. Checking the evidence and quality of the output and continuously reflecting it in the Instructions is what raises an agent's quality.

I'd love for you to try building a custom agent with Bits Agent Builder to make your own work more efficient — and to share what you learn at a user group. JDDUG is the community for Datadog users, and I help run the Fukuoka side of it. Event announcements go up on the Japan Datadog User Group connpass, so please join us if you're interested.

Last but not least: thank you to everyone who organized JDDUG #20, everyone who came and talked with me at the venue, the Classmethod team for hosting us, and everyone at Datadog!