Before Adding Gemma 4 to MonkeyCode, Run a Model Capability Contract

Gemma 4 is arriving in model catalogs with an unusually broad capability surface. Google's official overview, last updated July 8, 2026, lists five sizes, text and image input across the family, audio on selected variants, native system-role support, and 128K or 256K context windows depending on model size.

Those are model-family properties. They do not prove that a particular hosted endpoint, quantization, API adapter, or AI coding platform exposes every property correctly.

That distinction matters when adding a model to MonkeyCode, an AGPL-3.0 open-source AI development platform for teams. Its public repository describes managed server-side development environments, AI model and task management, project requirements, online use, native mobile clients, and private deployment.

This article reviews MonkeyCode at commit c58bcd4 and builds the missing second gate: a reusable model capability contract.

It is a source review and test harness, not a Gemma 4 benchmark or a claim that MonkeyCode currently ships a verified Gemma 4 integration. I validated the harness against a local OpenAI-compatible fixture, not a live Gemma 4 endpoint.

What MonkeyCode already validates

MonkeyCode is relevant here because model choice is a first-class platform concern rather than a hard-coded SDK call.

At the reviewed commit, its model configuration contract records:

• provider, base URL, model ID, and API key;
• OpenAI Chat, OpenAI Responses, or Anthropic interface type;
• context and output limits;
• thinking and image-support flags.

The add-model flow then performs a health check before saving. That is the right first gate: reject a bad URL, credential, model ID, or protocol choice early.

The important limitation is explicit in the health-check source: it checks API reachability, authentication, and model existence, not answer content. For an OpenAI Chat endpoint, it sends hi with max_tokens: 1 and accepts a non-error response.

That proves transport health. A coding agent needs more.

Define the contract before enabling the model

I would separate readiness into four gates:

Gate	Question	Failure response
Transport	Can the platform authenticate and reach the exact model ID?	Do not save or route traffic
Protocol	Do system roles, streaming, tool calls, cancellation, and errors match the selected API contract?	Keep the model disabled
Capability	Does this deployed variant actually support the modalities and limits you advertise?	Remove the capability flag or change the route
Task quality	Does it pass your repository-specific coding evaluation within latency and cost budgets?	Do not make it a default

MonkeyCode's current check covers the first gate. The next script exercises three protocol behaviors that a one-token response cannot establish.

A small OpenAI Chat capability probe

Save this as probe-openai-model.mjs. It requires Node.js 20 or newer and never prints the API key.

const required = ["MODEL_BASE_URL", "MODEL_API_KEY", "MODEL_ID"];
for (const name of required) {
  if (!process.env[name]) throw new Error(`Missing ${name}`);
}

const baseUrl = process.env.MODEL_BASE_URL.replace(/\/$/, "");
const endpoint = `${baseUrl}/chat/completions`;

async function chat(body) {
  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      authorization: `Bearer ${process.env.MODEL_API_KEY}`,
      "content-type": "application/json",
    },
    body: JSON.stringify({ model: process.env.MODEL_ID, ...body }),
    signal: AbortSignal.timeout(30_000),
  });
  const text = await response.text();
  if (!response.ok) throw new Error(`HTTP ${response.status}: ${text.slice(0, 300)}`);
  return { response, text };
}

const textResult = await chat({
  messages: [
    { role: "system", content: "Reply with exactly SYS_OK and nothing else." },
    { role: "user", content: "Follow the system instruction." },
  ],
  temperature: 0,
  max_tokens: 16,
});
const textBody = JSON.parse(textResult.text);
const content = textBody.choices?.[0]?.message?.content?.trim();
if (content !== "SYS_OK") {
  throw new Error(`system-role contract failed: ${JSON.stringify(content)}`);
}
console.log(`PASS system role — usage reported: ${Boolean(textBody.usage)}`);

const toolResult = await chat({
  messages: [{ role: "user", content: "Look up the weather for Paris." }],
  tools: [{
    type: "function",
    function: {
      name: "lookup_weather",
      description: "Look up weather by city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
        additionalProperties: false,
      },
    },
  }],
  tool_choice: "required",
  max_tokens: 128,
});
const toolBody = JSON.parse(toolResult.text);
const call = toolBody.choices?.[0]?.message?.tool_calls?.[0];
if (call?.function?.name !== "lookup_weather") {
  throw new Error(`tool-call contract failed: ${toolResult.text.slice(0, 300)}`);
}
const args = JSON.parse(call.function.arguments);
if (args.city !== "Paris") throw new Error(`unexpected tool arguments: ${call.function.arguments}`);
console.log(`PASS tool call — ${call.function.name}`);

const streamResult = await chat({
  messages: [{ role: "user", content: "Reply with STREAM_OK." }],
  stream: true,
  temperature: 0,
  max_tokens: 16,
});
const contentType = streamResult.response.headers.get("content-type") || "";
if (!contentType.includes("text/event-stream")) {
  throw new Error(`stream contract failed: content-type was ${contentType || "missing"}`);
}
if (!streamResult.text.includes("data:") || !streamResult.text.includes("[DONE]")) {
  throw new Error(`stream contract failed: ${streamResult.text.slice(0, 300)}`);
}
console.log(`PASS SSE stream — ${streamResult.text.length} bytes`);

Run it against the exact endpoint and model ID you plan to configure:

MODEL_BASE_URL="https://provider.example/v1" \
MODEL_API_KEY="your-test-key" \
MODEL_ID="the-exact-provider-model-id" \
node probe-openai-model.mjs

The expected shape is:

PASS system role — usage reported: true
PASS tool call — lookup_weather
PASS SSE stream — 184 bytes

Treat that byte count as an example, not a benchmark. A provider may also omit usage from the response; the script reports that fact without failing the protocol gate.

Do not turn model-card limits into configuration guesses

The official Gemma 4 overview says small models have 128K context while medium models support 256K. It also warns that context length adds KV-cache memory beyond the static model weights.

So context_limit: 256000 should not be copied into MonkeyCode merely because the family documentation contains that number. Record a smaller verified operational envelope for the exact serving stack:

model_id: exact-provider-model-id
interface: openai_chat
system_role: pass
tool_calls: pass
streaming: pass
image_input: not_tested
audio_input: not_exposed_by_this_contract
declared_context_tokens: 256000
tested_context_tokens: 32000
tested_output_tokens: 4096
concurrent_requests: 4
timeout_seconds: 30
tested_at: 2026-07-14

This is deliberately conservative. A declared limit is documentation; a tested limit is evidence.

For a real coding rollout, add repository tasks after the protocol probe:

1. edit a file under an explicit scope;
2. run the project's actual test command;
3. recover from one failing test without touching unrelated files;
4. call a tool with quoted paths and Unicode input;
5. survive a timeout or malformed tool result;
6. report the patch, tests, latency, token usage, and failure class.

Run the same set against the current default model. Promote Gemma 4 only when a named variant, serving stack, and configuration meet predeclared thresholds. Do not infer coding quality from parameter count, context length, or one successful prompt.

Where this leaves MonkeyCode

MonkeyCode already provides useful control points for this workflow: central model configuration, explicit interface selection, capability flags, managed development environments, team task workflows, and private deployment. Because the project is open source, the exact health-check boundary can be inspected rather than guessed.

The practical improvement is to preserve that fast health check and add a versioned capability result beside it. Then a team can distinguish:

• reachable from protocol-compatible;
• protocol-compatible from task-qualified;
• documented limit from tested operating limit.

That is a safer way to adopt a fast-moving model family without slowing experimentation to a halt.

Disclosure: I contribute to the MonkeyCode project. The MonkeyCode observations above are based on the linked public repository at the specified commit. The probe was validated against a local fixture, not a live Gemma 4 deployment, and this article does not claim completed Gemma 4 compatibility or benchmark results.

If your team is evaluating model routing or private deployment, the MonkeyCode Discord is the direct place to compare endpoint contracts and ask about currently supported configurations.