Using CloudWeGo Eino With a Custom OpenAI-Compatible API in Go: BaseURL, Zero Retries and a 12-Step Agent Limit

Eino is CloudWeGo's Go framework for LLM applications: chat models, tools, graphs and a ReAct agent, in the style of LangChain but written for Go. Its OpenAI component (github.com/cloudwego/eino-ext/components/model/openai) is the one most people use with self-hosted or hosted OpenAI-compatible servers such as vLLM, Ollama's /v1 endpoint, LiteLLM or a gateway.

The setup is three fields. The behaviour around them is less obvious, so I tested eino v0.7.13 and the eino-ext OpenAI component v0.1.13 on Go 1.27.2 against a small local mock server. The mock logged the path, headers and JSON body of every request and could return 401, 400, 429, 500 or 503 on demand, or keep asking for a tool call forever. Everything below is what went over the wire. The examples use APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions with that in mind. Its base URL is https://apiclaw.biz/v1. Replace YOUR_MODEL_ID with an exact ID from your provider's model list rather than guessing one.

Install, and one build error you may hit

go get github.com/cloudwego/eino@latest
go get github.com/cloudwego/eino-ext/components/model/openai@latest

On Go 1.27 my first go build failed inside github.com/bytedance/sonic/internal/rt with undefined: GoMapIterator. Eino pulled in sonic v1.14.1, which does not build on that toolchain. Upgrading sonic fixed it:

go get github.com/bytedance/sonic@latest github.com/bytedance/sonic/loader@latest
go mod tidy

That moved me to sonic v1.15.4 and everything compiled. If you are on an older Go release you may never see this.

The minimal config

import (
    "context"
    "os"
    "time"

    "github.com/cloudwego/eino-ext/components/model/openai"
    "github.com/cloudwego/eino/schema"
)

ctx := context.Background()
cm, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
    BaseURL: "https://apiclaw.biz/v1",
    APIKey:  os.Getenv("APICLAW_API_KEY"),
    Model:   "YOUR_MODEL_ID",
    Timeout: 60 * time.Second,
})
if err != nil {
    return err
}
out, err := cm.Generate(ctx, []*schema.Message{
    schema.SystemMessage("Be brief."),
    schema.UserMessage("Say hi"),
})

The struct comments say BaseURL is for Azure, but it is used for every non-Azure client too. Eino does not read OPENAI_API_KEY or OPENAI_BASE_URL from the environment, so pass both explicitly.

Three things I checked:

• The path is BaseURL + /chat/completions, nothing more. https://host/v1 and https://host/v1/ both produced POST /v1/chat/completions. A bare https://host produced POST /chat/completions, which most gateways answer with a 404. Keep the /v1.
• The body is minimal. With only the fields above, the request was {"messages":[...],"model":"YOUR_MODEL_ID"}. No temperature, no max_tokens, no stream:false. Headers were Authorization: Bearer <key> and Go's default User-Agent: Go-http-client/1.1.
• Timeout defaults to none. The struct comment says so, and nothing else in the chain adds one. If you leave it unset and the upstream stalls, Generate waits as long as your context allows. If you pass your own HTTPClient, Timeout is ignored and the client's timeout applies.

Sampling options are pointers, so zero really is sent

Temperature, MaxTokens, TopP and friends are pointer fields. Setting Temperature: &zero put "temperature":0 in the body, which some Python wrappers drop because they treat zero as "unset". Per-call options override the config:

cm.Generate(ctx, msgs, model.WithTemperature(0.7), model.WithModel("OTHER_MODEL_ID"))

That second call sent "temperature":0.7 and "model":"OTHER_MODEL_ID" while keeping the configured max_tokens. model.WithModel is a cheap way to try a second model on the same gateway without building another client.

Streaming asks for usage automatically

cm.Stream(ctx, msgs) sent "stream":true and "stream_options":{"include_usage":true} without being asked. After concatenating the chunks with schema.ConcatMessages, ResponseMeta.Usage held the prompt, completion and total token counts from the final usage chunk.

If your server rejects unknown fields like stream_options, that is the first thing to check when streaming fails but Generate works.

For reasoning models, a reasoning_content field in the response landed in out.ReasoningContent. Inline <think>...</think> text stayed inside out.Content; Eino does not strip it, so do that yourself if your model emits it.

There are no retries

This surprised me most. Every error status got exactly one request and came back in a millisecond or two:

Server response	Requests sent	Returned error
401	1	status code: 401 ... message: ...
400	1	same shape
429 with Retry-After: 2	1	same shape
500 / 503	1	same shape

Retry-After is ignored because nothing retries. That is fine for a CLI. For a service sitting behind a gateway that rate-limits per minute, one 429 becomes a user-visible failure.

The error can be inspected with errors.As against the eino-ext type, not the underlying go-openai fork's type. I tried both. The fork's *APIError did not match, because the component converts it.

var apiErr *openai.APIError // github.com/cloudwego/eino-ext/components/model/openai
if errors.As(err, &apiErr) && apiErr.HTTPStatusCode == 429 {
    // back off
}

That still matched when the 429 happened inside a ReAct agent run, where the outer error text starts with [NodeRunError] and names the node path.

Adding retries with an http.RoundTripper

Since ChatModelConfig accepts an *http.Client, the cleanest fix is a transport that replays the request body:

type retryTransport struct {
    base     http.RoundTripper
    attempts int
}

func (t retryTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    var body []byte
    if req.Body != nil {
        body, _ = io.ReadAll(req.Body)
        req.Body.Close()
    }
    var resp *http.Response
    var err error
    for i := 0; i < t.attempts; i++ {
        req.Body = io.NopCloser(bytes.NewReader(body))
        resp, err = t.base.RoundTrip(req)
        if err != nil || (resp.StatusCode != 429 && resp.StatusCode < 500) || i == t.attempts-1 {
            return resp, err
        }
        wait := time.Duration(1<<i) * time.Second
        if s, e := strconv.Atoi(resp.Header.Get("Retry-After")); e == nil {
            wait = time.Duration(s) * time.Second
        }
        resp.Body.Close()
        select {
        case <-time.After(wait):
        case <-req.Context().Done():
            return nil, req.Context().Err()
        }
    }
    return resp, err
}

cm, _ := openai.NewChatModel(ctx, &openai.ChatModelConfig{
    BaseURL: "https://apiclaw.biz/v1",
    APIKey:  os.Getenv("APICLAW_API_KEY"),
    Model:   "YOUR_MODEL_ID",
    HTTPClient: &http.Client{
        Timeout:   60 * time.Second,
        Transport: retryTransport{base: http.DefaultTransport, attempts: 3},
    },
})

Against the mock, a server that returned 429 twice and then succeeded produced three requests about two seconds apart in total and a normal reply. A permanent 503 produced three requests and then the usual error. A 401 still produced one request, which is what you want. Note that http.Client.Timeout covers all attempts together, so size it for the retries. Retrying a streaming call this way only helps when the error arrives before the first byte, which is where 429s usually happen.

The ReAct agent stops at MaxStep, and steps are not tool calls

agent, err := react.NewAgent(ctx, &react.AgentConfig{
    ToolCallingModel: cm,
    ToolsConfig:      compose.ToolsNodeConfig{Tools: []tool.BaseTool{getTime}},
    MaxStep:          20,
})

With a tool built by utils.InferTool, the tool schema went out with "additionalProperties":false and a required list, plus "tool_choice":"auto". There was no strict flag. The assistant message that carried the tool call was sent back with no content key at all, followed by a role: tool message with the matching tool_call_id.

To see the limit, I made the mock request a tool call on every turn:

• With MaxStep unset (default 12), the run made 7 model requests and 6 tool calls, then returned [GraphRunError] exceeds max steps.
• With MaxStep: 40, it made 20 model requests and 20 tool calls before the same error.

MaxStep counts graph node executions, and each round is a model node plus a tools node, so you get roughly MaxStep / 2 tool calls, not MaxStep. Each request also re-sends the whole history, so the last request in the 40-step run carried 39 messages. Check for the limit with:

if errors.Is(err, compose.ErrExceedMaxSteps) {
    // the model kept calling tools; surface a partial answer or ask the user
}

When the mock returned a tool call once and then plain text, the agent finished normally after two requests.

Checklist

• Set BaseURL with /v1; Eino only appends /chat/completions.
• Pass APIKey explicitly; environment variables are not read.
• Set Timeout or your own HTTPClient; the default is no timeout.
• Add retries yourself with a transport, and match errors with the eino-ext *openai.APIError.
• Expect stream_options.include_usage on every stream.
• Treat MaxStep as about twice the number of tool rounds you want to allow.

Disclosure: I build APIClaw, the OpenAI-compatible gateway used in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.