stop_sequences takes an array of strings, and Cohere documents a maximum of five. The two things worth knowing beyond that are whether the matched text is included in the response, and how you distinguish a stop from an answer that ran out of room.
The documented limit
Cohere’s Chat API reference documents stop_sequences as a list of up to five strings, on both /v1/chat and /v2/chat. Five is the same ceiling OpenAI applies to its stop parameter, and low enough that a system generating stop sequences programmatically — one per tool name, say — will hit it and get a validation error rather than a silent truncation of the list.
{
"model": "command-r-plus-08-2024",
"message": "List the three warehouse regions, one per line, then write END.",
"stop_sequences": ["END", "\n\n\n"]
}
Cohere has revised parameter limits between API versions before. Five is the documented cap at the time of writing; the reference above is authoritative if the two ever disagree.
Where the cut lands
The matched sequence is excluded from the returned text. Given the request above, the model generates the three lines and then the token sequence spelling END, and what comes back is:
{
"text": "Amsterdam\nUtrecht\nRotterdam\n",
"finish_reason": "STOP_SEQUENCE",
"meta": {"billed_units": {"input_tokens": 24, "output_tokens": 11}}
}
Two details in that response are load-bearing. The trailing newline before END survives — a stop sequence cuts at the sequence, not at the last thing you would have wanted to keep, so any whitespace preceding it is yours to trim. And output_tokens includes the tokens the model spent generating the stop sequence itself. You are billed for text you never receive. This is small per request and entirely real at volume, and it is why an eight-character stop sequence is a worse choice than a two-character one when both would work.
Telling a stop from a truncation
Three different endings produce three different values of finish_reason, and they demand different responses from your code:
-
COMPLETE— the model emitted its end-of-turn token. The answer is finished. -
STOP_SEQUENCE— one of your sequences matched. The answer is finished at a boundary you chose, which is a success. -
MAX_TOKENS— the output ceiling was reached. The answer is cut mid-thought and is not safe to render as final. See what actually bounds Command’s output length.
Treating all three as success is the standard bug. It is worth writing the check as an explicit allowlist — accept COMPLETE and STOP_SEQUENCE, treat everything else as a partial result — because the other values Cohere can return, including error variants, then fail closed instead of silently passing through as content.
Why a stop sequence can be missed
Stop sequences are matched against generated text, but generation happens in tokens, and the two do not line up. A sequence only stops generation when the text the model actually produced contains it. The failure modes follow directly:
- Whitespace differences. Asking to stop at
"Answer:"does not match"Answer :"or"answer:". Matching is literal and case-sensitive. - Markdown decoration. The model writes
**END**because it is writing a formatted list, and"END"is still present as a substring — so it does stop, but the asterisks around it end up split across the boundary. Pick a sequence the model has no reason to decorate. - The model never emits it. A stop sequence is not a constraint on generation; it is a condition checked during it. If the model does not write the string, generation runs to
MAX_TOKENS. If a stop sequence is load-bearing, the preamble should ask for it explicitly, and a lower temperature makes the model more likely to follow that instruction.
Stop sequences under streaming
Streaming and stop sequences interact in a way that is invisible in a non-streamed request and obvious once you see it: a stop sequence can span token boundaries, and tokens are what get streamed. If your stop sequence is "END" and the model emits it as "EN" + "D", the first fragment has already been sent to the client before anything knows a stop is coming.
In practice this means a UI that renders every delta the instant it arrives can briefly display the leading characters of a sequence the reader was never meant to see. The mitigation is a small holdback buffer: keep the last few characters of the stream unrendered until either a further delta proves they are not the start of a stop sequence, or the stream ends. The buffer needs to be at least as long as your longest stop sequence minus one character, which is a further reason to prefer short sequences.
The same reasoning applies to any post-processing that runs on partial text. Splitting a streamed answer on a delimiter, or parsing it incrementally, has to tolerate a delimiter arriving in pieces. Code that checks chunk.includes(delimiter) per chunk rather than against an accumulated buffer will miss a delimiter roughly whenever the tokenizer happens to split it — which is data-dependent, intermittent, and therefore the sort of bug that gets closed as unreproducible.
Note also that finish_reason only arrives at the end of a stream, so “did a stop sequence fire?” is a question you cannot answer until the last event. See the streaming event types for where it lands in each API version.
What they are actually good for
Stop sequences solve a narrower set of problems than they used to, and two of the three classic uses have better answers now.
Cutting off a runaway list or a self-continuing dialogue is still their job, and so is enforcing a boundary in a completion-style prompt where you supplied the structure. But truncating JSON at a closing brace is obsolete — response_format constrains the output properly and a stop sequence on "}" breaks on the first nested object. And capping cost is better done with max_tokens, which bounds generation directly instead of hoping a string appears.
They also interact badly with tool use, which is a case worth stating explicitly because it is not obvious. A stop sequence is matched against generated text, and in a tool-calling turn much of what the model generates is the plan and the call arguments rather than prose. A stop sequence that happens to appear inside a JSON argument value — a quote character, a brace, a newline pair — cuts the generation mid-call, and what reaches your executor is an incomplete arguments string. If you use stop sequences at all in an agent loop, choose ones that cannot occur in your tools’ argument space, and be aware that a rare, data-dependent parse failure in a tool call is a plausible symptom of this and a thoroughly unpleasant one to track down.
The one use with no substitute: making the model’s stopping point machine-detectable when you are parsing a multi-part answer out of one response. A rare delimiter you asked for in the preamble, matched here, gives you an unambiguous cut that no amount of prose parsing matches for reliability.
Top comments (0)