Monday, 2:41 AM. A build fails on line 1,214 of a 9,000-line log. The error message is a missing gem, buried under three screens of deprecation warnings. I did what many of us do: copied the last screenful into a chat, got a confident answer, and pasted it into the Gemfile. That fix broke three other tests. I had no idea how many tokens that mistake cost me.
Over the next 48 hours, I ran a different kind of experiment. I pointed a small script at MonkeyCode's free AI server and asked it to diagnose a set of real CI logs. Then I recorded the token usage from every API response. I wanted to see how much context a free model actually needs to find a root cause, and where the token budget leaks.
MonkeyCode is an open-source coding assistant with a free server tier. The allowance is generous — 10 million tokens — and the server itself costs nothing to spin up. Disclosure: This article was prepared as part of MonkeyCode's product outreach. But "free" never means "unlimited." It means you have a budget, and the budget disappears faster than you think if you send the whole log.
Here is the core of what I built. A Python function trims a log to its first 20 lines and last 180, sends that to an OpenAI-compatible endpoint, and returns the model's verdict plus the token counts.
import argparse
import json
import os
from pathlib import Path
from openai import OpenAI
def trim_log(log_text: str, max_lines: int = 200) -> str:
"""Keep the head and tail of a log; CI errors usually live in the tail."""
lines = log_text.splitlines()
if len(lines) <= max_lines:
return log_text
head = "\n".join(lines[:20])
tail = "\n".join(lines[-max_lines + 20 :])
return f"# First 20 lines\n{head}\n# Last {max_lines - 20} lines\n{tail}"
def analyze_failure(client: OpenAI, log_text: str, model: str) -> dict:
prompt = """You are a CI diagnostician. Read the log excerpt and return JSON with keys:
- root_cause (string)
- confidence (0-1)
- suggested_command (string or null)
Do not explain anything else."""
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "You analyze build logs concisely."},
{"role": "user", "content": f"Log excerpt:\n{trim_log(log_text)}\n\nReturn JSON."},
],
temperature=0.1,
response_format={"type": "json_object"},
)
return {
"output": response.choices[0].message.content,
"prompt_tokens": response.usage.prompt_tokens,
"completion_tokens": response.usage.completion_tokens,
"total_tokens": response.usage.total_tokens,
}
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--log", required=True, type=Path)
parser.add_argument("--model", required=True, help="Model alias provided by your server")
parser.add_argument("--api-base", default=os.getenv("MC_BASE_URL"))
parser.add_argument("--api-key", default=os.getenv("MC_API_KEY"))
args = parser.parse_args()
if not args.api_base or not args.api_key:
raise SystemExit("Set MC_BASE_URL and MC_API_KEY from your server credentials.")
client = OpenAI(base_url=args.api_base, api_key=args.api_key)
result = analyze_failure(client, args.log.read_text(), args.model)
print(json.dumps(result, indent=2))
The script is deliberately small. It does not guess. Every response carries a JSON object with prompt tokens, completion tokens, and total tokens. That is the metric that mattered.
What broke first? I sent an entire 9,000-line log. The prompt token count went to something absurd, and the model's answer buried the actual error under generic advice. Trimming to head and tail fixed that. But then I noticed a second leak: the system prompt is part of every request. My system prompt was one long sentence about being a CI diagnostician. It repeated on every call, so a 30-call session paid for that sentence 30 times. Shrink it, and you save real tokens across a month of runs.
Rate limits hit too. After a burst of rapid calls, the server started returning 429s. I added a two-second sleep between calls. That is a constraint, not a bug; a free server that never throttled would die. The sleep also forced me to batch logs instead of firing them blindly.
The model also refused to return clean JSON on some attempts. My parser failed because the answer included
json markers. I added a small fallback that strips backticks. That is the kind of thing you discover only when you run the loop over real logs, and it taught me to expect small inconsistencies from every AI endpoint.
What would I repeat? The habit of reading the `usage` field before reading the answer. Every call teaches you how expensive your prompt really is. I would also keep the system prompt under fifty tokens, and I would never let an AI read more than the head and tail of a log, because the middle is where the token budget goes to die.
What would I not repeat? Sending secrets. Logs often contain environment variables or internal endpoints. A free server is a third party, so redact before you send, or add a filter that replaces anything that looks like a key. I spent one annoying evening scrubbing a log that had a live API token in it. Do not learn that lesson the same way.
Who should not use this approach? Teams that need real-time CI alerts, because the request latency is seconds, not milliseconds. Anyone with logs larger than a few thousand lines, because memory and token costs grow together. And anyone who expects a free tier to behave like a dedicated GPU. The free server is a starting point, not a production SLA.
If you want to try the same loop, MonkeyCode's free server is a quick way to start. Point the script at it, use the model alias from its setup page, and watch your token counter move. The real lesson from 48 hours is not about any specific vendor. It is about the discipline of token accounting. A free allowance of 10 million tokens sounds infinite until you realize each log analysis can consume thousands. The moment you treat tokens as a meter, you stop pasting whole files and start asking better questions. That skill survives any model, any server, and any budget.
Top comments (1)
Your approach to optimizing token usage by trimming CI logs is a smart trade-off, especially in balancing context with cost. I’ve faced similar challenges, and your observation about the repeated system prompt is a great reminder to consider every part of the API call for efficiency. For further optimization, you might explore caching common responses for repeated prompts to enhance performance without inflating token consumption. If you're looking for additional engineering support on MonkeyCode, I’d be open to discussing a paid collaboration to help enhance its capabilities!