A model switch looks harmless when an application handles one request at a time. Set MODEL, call the API, change MODEL, and call again. Under concurrency, that pattern becomes a race: Agent A may prepare its prompt, Agent B changes the global model, and Agent A is sent to the wrong model.
This article solves one verifiable problem: bind each Agent task to an immutable model value before concurrent requests begin. Python, JavaScript, and PowerShell examples send two tasks at the same time without modifying process-wide configuration.
Disclosure: VectorNode is the gateway used in the examples. Per-request model binding is an application responsibility and works with other compatible endpoints.
1. The developer problem
The unsafe pattern is global mutation:
set MODEL = primary
start task A
set MODEL = reviewer
start task B
If task A reads MODEL after task B changes it, both requests may use the reviewer model. Logs can become even more confusing if they read the environment again after the response arrives.
The failure may not produce an HTTP error. Both calls can return 200, while cost, latency, and output quality silently differ from the intended routing decision.
2. Why the race happens
Environment variables and global client settings belong to the process, not to one request. Async functions, threads, jobs, and runspaces can interleave between a write and a later read.
The official OpenAI Chat Completions reference defines model as a field in each request body. The gateway's current interface and parameter details are available at https://o1kqetbjmk.apifox.cn.
Use that request boundary. Read configuration once at startup, then pass the selected model into the function that constructs the payload. Never select a model by changing MODEL immediately before a call.
3. Configure named model roles once
Keep secrets and stable startup configuration outside source code:
API_KEY=replace-with-a-real-key-locally
BASE_URL=https://www.vectronode.com/v1
MODEL_PRIMARY=replace-with-a-current-model-id
MODEL_REVIEW=replace-with-another-current-model-id
The names describe application roles, not provider promises. Copy model IDs currently visible to the account. Add .env to .gitignore, and use a secret manager in production.
Install the small dependencies:
py -m venv .venv
.\.venv\Scripts\python.exe -m pip install requests python-dotenv
npm install dotenv
4. Minimal concurrent examples
Each task carries task_id, model, and a synthetic prompt. The response record preserves both requested_model and the model reported by the API. The key is never included in task data or output.
Python
import os
from concurrent.futures import ThreadPoolExecutor
import requests
from dotenv import load_dotenv
load_dotenv()
required = ("API_KEY", "BASE_URL", "MODEL_PRIMARY", "MODEL_REVIEW")
missing = [name for name in required if not os.getenv(name)]
if missing:
raise RuntimeError(f"Missing environment variables: {', '.join(missing)}")
API_KEY = os.environ["API_KEY"]
BASE_URL = os.environ["BASE_URL"].rstrip("/")
def call_agent(task_id: str, model: str, prompt: str) -> dict:
expected = f"TASK:{task_id}"
response = requests.post(
f"{BASE_URL}/chat/completions",
headers={"Authorization": f"Bearer {API_KEY}"},
json={
"model": model,
"messages": [
{
"role": "system",
"content": f"Return exactly {expected}.",
},
{"role": "user", "content": prompt},
],
},
timeout=60,
)
request_id = response.headers.get("x-request-id", "unavailable")
if not response.ok:
return {
"task_id": task_id,
"requested_model": model,
"http_status": response.status_code,
"request_id": request_id,
"ok": False,
}
data = response.json()
choice = data["choices"][0]
content = choice["message"].get("content")
text = content.strip() if isinstance(content, str) else ""
return {
"task_id": task_id,
"requested_model": model,
"reported_model": data.get("model"),
"finish_reason": choice.get("finish_reason"),
"request_id": request_id,
"text": text,
"ok": text == expected,
}
tasks = [
("draft", os.environ["MODEL_PRIMARY"], "Run the draft probe."),
("review", os.environ["MODEL_REVIEW"], "Run the review probe."),
]
with ThreadPoolExecutor(max_workers=2) as pool:
results = list(pool.map(lambda item: call_agent(*item), tasks))
for result in results:
print(result)
if not all(result["ok"] for result in results):
raise SystemExit("At least one Agent routing probe failed.")
JavaScript
import "dotenv/config";
const required = ["API_KEY", "BASE_URL", "MODEL_PRIMARY", "MODEL_REVIEW"];
const missing = required.filter((name) => !process.env[name]);
if (missing.length) {
throw new Error(`Missing environment variables: ${missing.join(", ")}`);
}
const config = Object.freeze({
apiKey: process.env.API_KEY,
baseUrl: process.env.BASE_URL.replace(/\/$/, ""),
primaryModel: process.env.MODEL_PRIMARY,
reviewModel: process.env.MODEL_REVIEW,
});
async function callAgent({ taskId, model, prompt }) {
const expected = `TASK:${taskId}`;
const response = await fetch(`${config.baseUrl}/chat/completions`, {
method: "POST",
headers: {
Authorization: `Bearer ${config.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
messages: [
{ role: "system", content: `Return exactly ${expected}.` },
{ role: "user", content: prompt },
],
}),
});
const requestId = response.headers.get("x-request-id") ?? "unavailable";
if (!response.ok) {
return {
task_id: taskId,
requested_model: model,
http_status: response.status,
request_id: requestId,
ok: false,
};
}
const data = await response.json();
const choice = data.choices?.[0];
const content = choice?.message?.content;
const text = typeof content === "string" ? content.trim() : "";
return {
task_id: taskId,
requested_model: model,
reported_model: data.model ?? null,
finish_reason: choice?.finish_reason ?? null,
request_id: requestId,
text,
ok: text === expected,
};
}
const tasks = [
{ taskId: "draft", model: config.primaryModel, prompt: "Run the draft probe." },
{ taskId: "review", model: config.reviewModel, prompt: "Run the review probe." },
];
const results = await Promise.all(tasks.map(callAgent));
console.log(JSON.stringify(results, null, 2));
if (!results.every((result) => result.ok)) process.exitCode = 1;
PowerShell 7
$required = "API_KEY", "BASE_URL", "MODEL_PRIMARY", "MODEL_REVIEW"
$missing = @(
$required | Where-Object {
-not [Environment]::GetEnvironmentVariable($_, "Process")
}
)
if ($missing.Count -gt 0) {
throw "Missing environment variables: $($missing -join ', ')"
}
$apiKey = $env:API_KEY
$baseUrl = $env:BASE_URL.TrimEnd('/')
$tasks = @(
[pscustomobject]@{
task_id = "draft"
model = $env:MODEL_PRIMARY
prompt = "Run the draft probe."
}
[pscustomobject]@{
task_id = "review"
model = $env:MODEL_REVIEW
prompt = "Run the review probe."
}
)
$results = $tasks | ForEach-Object -Parallel {
$task = $_
$key = $using:apiKey
$apiBase = $using:baseUrl
$expected = "TASK:$($task.task_id)"
$headers = @{ Authorization = "Bearer $key" }
$payload = [ordered]@{
model = $task.model
messages = @(
[ordered]@{ role = "system"; content = "Return exactly $expected." }
[ordered]@{ role = "user"; content = $task.prompt }
)
} | ConvertTo-Json -Depth 6 -Compress
try {
$response = Invoke-RestMethod `
-Uri "$apiBase/chat/completions" `
-Method Post `
-Headers $headers `
-ContentType "application/json" `
-Body $payload `
-TimeoutSec 60
} catch {
[pscustomobject]@{
task_id = $task.task_id
requested_model = $task.model
error = $_.Exception.GetType().Name
ok = $false
}
return
}
$choice = $response.choices[0]
$text = ([string]$choice.message.content).Trim()
[pscustomobject]@{
task_id = $task.task_id
requested_model = $task.model
reported_model = $response.model
finish_reason = $choice.finish_reason
text = $text
ok = ($text -eq $expected)
}
} -ThrottleLimit 2
$results | ConvertTo-Json -Depth 4
if ($results.ok -contains $false) {
exit 1
}
5. Verify the routing result
A successful run produces one record per task:
[
{
"task_id": "draft",
"requested_model": "primary-model-id",
"reported_model": "primary-model-id",
"text": "TASK:draft",
"ok": true
},
{
"task_id": "review",
"requested_model": "review-model-id",
"reported_model": "review-model-id",
"text": "TASK:review",
"ok": true
}
]
Do not require reported_model to equal the requested string unless the gateway contract guarantees that behavior; an alias may be reported as a canonical ID. The essential evidence is that each request logged its immutable selection and returned the correct task marker.
6. Common errors
| Symptom | Check |
|---|---|
| Both tasks use the same model | Confirm each task object contains its own model before concurrency starts |
model_not_found for one task |
Replace only that role's model ID with one available to the account |
401 or 403 for both tasks |
Check the shared key without printing it |
429 under parallel execution |
Lower the worker or throttle count; do not add an unlimited retry loop |
| Correct models but wrong task markers | Check task IDs, prompt construction, and response-to-task association |
| Logs disagree with requests | Log the function argument, not a later read from the environment |
7. Security and scope
Read keys and model-role configuration at startup, then treat them as immutable. Never include the API key in task objects, result records, exceptions, screenshots, or public issues. Use synthetic prompts for concurrency tests.
Per-request binding prevents a local race; it does not prove model availability, equal capabilities, or equal cost. Validate each role separately before increasing concurrency, and keep authorization decisions independent from model selection.
Try the example
After creating an account and selecting current model IDs, use the tested Python, JavaScript, and PowerShell examples in the VectorNode API Quickstart:
https://github.com/Vector-Compute-Engine/api-quickstart
Never publish your complete API key in source code, screenshots, videos, or public issues.
Top comments (0)