If you've ever SSH'd into a box at 2am, typed docker ps -a, squinted at fifteen container names, then run docker logs --tail 200 three times because you kept guessing the wrong one — this post is for you.
I built a small AI agent that does exactly that investigation loop, except you just ask it in plain English: "why did the nginx container stop?" — and it goes and checks for you.
This post walks through what I built, how it works under the hood, and a few real bugs I hit along the way that are worth learning from if you're building something similar.
The problem with "just add an LLM"
The naive approach to this is: paste your container logs into ChatGPT and ask what's wrong. That works exactly once, for exactly one container, at exactly one point in time. It doesn't scale, and worse — the model has no way to actually check anything. It's guessing based on what you pasted, not investigating.
What you actually want is an LLM that can call real tools — run docker ps, run docker logs, look at the actual output — and reason over that, not over what you remembered to paste in. That's the whole idea behind this project.
Architecture: MCP + LangChain + Ollama
You (terminal)
│
▼
LangChain agent (Ollama LLM + tool-calling loop)
│
▼
MCP client ──stdio──▶ docker_mcp_server.py (FastMCP)
│
▼
subprocess → docker CLI
│
▼
Docker daemon
Three pieces:
An MCP server: I used FastMCP to expose a handful of Docker operations as "tools" the LLM can call. MCP (Model Context Protocol) is Anthropic's open standard for this — it's basically a clean, structured way for an LLM to discover "here are the functions you can call, here's what each one does, here's what arguments it needs."
A local LLM via Ollama: I'm running this fully locally with gemma3 through Ollama, no API calls to a hosted model required. This matters if you're troubleshooting inside an environment where you don't want container logs (which can contain sensitive data) leaving your machine.
A LangChain agent: this is the orchestration layer that actually runs the reasoning loop: read the user's question → decide which tool(s) to call → call them → read the result → decide if more tool calls are needed → respond in plain English.
The MCP server
from fastmcp import FastMCP
import subprocess
mcp = FastMCP("Docker MCP Server")
@mcp.tool
def show_running_containers() -> str:
"""Show currently running Docker containers."""
result = subprocess.run(["docker", "ps"], capture_output=True, text=True)
return result.stdout
@mcp.tool
def show_all_containers() -> str:
"""Show all Docker containers, including stopped ones."""
result = subprocess.run(["docker", "ps", "-a"], capture_output=True, text=True)
return result.stdout
@mcp.tool
def show_container_logs_by_name(container_name: str) -> str:
"""
Show logs for a specific Docker container by its name or ID.
Use this when the user asks to see logs, debug an issue, investigate
errors, or perform root cause analysis (RCA) for a particular container.
"""
result = subprocess.run(
["docker", "logs", "--tail", "200", container_name.strip()],
capture_output=True, text=True
)
if result.returncode != 0:
return f"Error fetching logs for '{container_name}': {result.stderr.strip()}"
output = result.stdout.strip()
return output if output else f"No logs found for container '{container_name}'."
A deliberate design choice here: **each tool is narrow and does one specific thing**, rather than one generic "run any docker command" tool. This matters for two reasons:
The LLM makes better decisions when the options are clear and specific — "show logs for a named container" is unambiguous; "run arbitrary docker command" invites the model to construct commands from scratch, which is where mistakes creep in.
It's a security boundary. A generic "run any command" tool means anything the LLM decides to type gets executed. A closed set of specific tools means the worst it can do is call docker logs on the wrong container name — not run something destructive it wasn't supposed to.
Notice the docstrings are doing real work here, not just documentation. MCP tool schemas are generated from the function signature and the docstring — the line "Use this when the user asks to see logs, debug an issue, investigate errors, or perform root cause analysis" is directly what helps the model pick the right tool when someone asks a vague question like "why is this crashing?" rather than a literal one like "show me logs for X."
The agent loop
async def main():
client = MultiServerMCPClient({
"docker-mcp": {
"transport": "stdio",
"command": "python",
"args": ["docker_mcp_server.py"]
}
})
tools = await client.get_tools()
llm = ChatOllama(model="gemma3:27b", temperature=0.5)
agent = create_agent(llm, tools)
conversation = []
while True:
user_input = input("You: ").strip()
if user_input.lower() in ("exit", "quit"):
break
conversation.append({"role": "user", "content": user_input})
response = await agent.ainvoke({"messages": conversation})
print(f"\nAssistant: {response['messages'][-1].content}\n")
conversation = response["messages"] # keep full history for follow-ups
The important detail here is transport: "stdio". The MCP server isn't a network service — it's spawned as a local subprocess, and the agent talks to it over stdin/stdout. That means no ports to open, no auth to configure for a local dev setup, and the whole thing runs entirely on one machine.
The other detail worth calling out: conversation = response["messages"] at the end of the loop. This keeps the entire running message history — including all the tool calls and their results — as context for the next turn. That's what makes a follow-up question like "and what caused that?" actually work; the model still has the log output from the previous turn in its context, it doesn't need you to repeat yourself.
Why I kept it read-only
Every tool in this project only reads — docker ps, docker ps -a, docker logs. There's no docker kill, docker rm, or docker restart tool exposed to the model.
This is deliberate. There's a growing body of evidence from real production "AI SRE" deployments that LLMs are meaningfully better at diagnosing _ a problem than at _deciding the correct fix. Research on LLM-driven incident recovery has found root-cause diagnosis accuracy in the 90%+ range, while the validity of recovery/remediation actions chosen by the same systems drops to roughly 40-60%. In other words: trust the agent to tell you what's wrong, be a lot more careful about letting it act on that conclusion autonomously.
If I ever add write capability (restart a container, prune a stopped one), it'll go through an explicit propose → confirm → execute pattern, never a direct action from a single LLM decision.
What's next
This was just the warm-up. A Kubernetes version of this agent is on the way — same MCP + LangChain pattern. Follow along if you don’t want to miss it.
The code
docker-ai-troubleshooting-agent
If you build something similar or spot an improvement, I'd genuinely love to hear about it in the comments.
Top comments (1)
Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support