DEV Community

shashank ms
shashank ms

Posted on

Kimi K2 Thinking Model Explained

#learnai #oxlo #ai

I recently built an internal root-cause analysis agent that feeds raw production logs to Kimi K2 Thinking and returns a step-by-step incident breakdown. In this tutorial, I will walk you through the exact version I deployed on Oxlo.ai so you can adapt it for your own on-call stack.

What you'll need

Before we start, grab the following:

Oxlo.ai uses flat per-request pricing (see https://oxlo.ai/pricing), so dumping hundreds of log lines into a single request does not inflate your cost. That makes it a natural fit for this workload.

Step 1: Initialize the Oxlo.ai client

I use the OpenAI SDK as a drop-in client. The only changes are the base URL and the model name.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 2: Write the system prompt

Kimi K2 Thinking excels at extended chain-of-thought reasoning. I explicitly ask it to show its work inside <thinking> tags before emitting the final verdict. This lets us audit its logic and catch bad assumptions.

SYSTEM_PROMPT = """You are a senior site-reliability engineer performing root-cause analysis.

Instructions:
1. Inside <thinking> tags, reason step by step. List hypotheses, test them against the evidence, and eliminate contradictions.
2. After closing </thinking>, output a JSON object with exactly these keys: root_cause, confidence (high, medium, or low), affected_services, and recommended_fix.
3. If the evidence is ambiguous, state what additional data you need.

Do not hallucinate metrics. Base conclusions only on the provided evidence."""

Step 3: Format the incident data

I keep the input formatter simple. It concatenates alerts and log lines into a plain-text block. Because Oxlo.ai charges per request rather than per token, I do not have to truncate logs to save money. I can paste the full context and let the model reason over it.

def build_incident_message(timeframe, alerts, logs):
    alert_lines = "\n".join(f"- {a}" for a in alerts)
    log_lines = "\n".join(f"{ts}: {msg}" for ts, msg in logs)
    return (
        f"Incident window: {timeframe}\n\n"
        f"Alerts:\n{alert_lines}\n\n"
        f"Logs:\n{log_lines}"
    )

Step 4: Call Kimi K2 Thinking

Now I wire the formatter to the chat completion endpoint. The response will contain the model's raw reasoning followed by the structured report.

def analyze_incident(timeframe, alerts, logs):
    user_message = build_incident_message(timeframe, alerts, logs)

    response = client.chat.completions.create(
        model="kimi-k2-thinking",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )

    return response.choices[0].message.content

Run it

Here is a realistic incident scenario. I run the script locally and pipe the output to a file for the post-mortem.

if __name__ == "__main__":
    alerts = [
        "High latency on /api/v1/checkout",
        "DB connection pool exhausted",
    ]
    logs = [
        ("14:02:01", "POST /api/v1/checkout 200 OK in 4200ms"),
        ("14:02:03", "DB pool timeout after 3000ms"),
        ("14:02:04", "Retry attempt 1 failed"),
        ("14:02:05", "Circuit breaker opened for payment-service"),
    ]

    result = analyze_incident("14:00-14:05 UTC", alerts, logs)
    print(result)

Example output:

<thinking>
1. Hypothesis: The checkout endpoint is slow because of a downstream payment issue.
   - Evidence: The checkout latency is 4200ms, but the HTTP status is still 200. This suggests the request eventually succeeds, but waits on something.
   - However, the DB pool timeout occurs at 14:02:03, which is after the checkout request started.

2. Hypothesis: Database connection pool exhaustion is the primary cause.
   - Evidence: The DB pool timeout is logged immediately after the slow checkout. If the pool is exhausted, new checkout requests block until timeout.
   - The circuit breaker for payment-service opens at 14:02:05, which is a secondary effect. The payment service likely depends on the same DB pool.

3. Elimination: The alerts list "DB connection pool exhausted" as a separate symptom, but the log sequence shows it is the root blocker. The checkout latency and payment-service circuit breaker are downstream effects.

Conclusion: The root cause is database connection pool exhaustion, likely caused by a connection leak or a spike in sustained connections that are not being released. The payment service fails because it shares that pool.
</thinking>

{
  "root_cause": "Database connection pool exhaustion causing cascading latency and circuit breaker trips",
  "confidence": "high",
  "affected_services": ["checkout-api", "payment-service"],
  "recommended_fix": "Increase pool size temporarily, identify and fix the connection leak, and add pool monitoring alerts."
}

Wrap-up

You now have a working root-cause analysis agent that leverages Kimi K2 Thinking's chain-of-thought reasoning through Oxlo.ai. Two concrete next steps: wire this into your PagerDuty or Opsgenie webhook so it auto-generates a first-pass analysis when an alert fires, or extend the formatter to include Grafana snapshot URLs so the model can cross-reference metrics with logs.

Top comments (0)