Your Model Call Is a Job, Not a Request: An Outbox Queue for Free Endpoints

A synchronous model call is a bet. You bet the endpoint stays up. You bet it answers fast. Free endpoints lose that bet daily.

An outbox queue changes the bet. Requests persist first. A worker sends them later. The caller gets a job ID, not a blocked socket.

This tutorial builds a durable outbox for free model access. Plain Node. SQLite for persistence. It runs on a free server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why synchronous fails

Free endpoints have three failure modes: 429s, timeouts, and crashes. A synchronous caller feels all three. The user waits. The request dies. The work is lost.

An outbox decouples arrival from execution. The HTTP layer only writes a row. The worker owns the call. A crash loses nothing.

Step 1: Define the job table

Use SQLite. One file. No server. Perfect for a free server.

CREATE TABLE jobs (
  id TEXT PRIMARY KEY,
  prompt TEXT NOT NULL,
  status TEXT NOT NULL DEFAULT 'pending',
  attempts INTEGER NOT NULL DEFAULT 0,
  max_attempts INTEGER NOT NULL DEFAULT 5,
  next_attempt_at INTEGER NOT NULL,
  result TEXT,
  error TEXT,
  created_at INTEGER NOT NULL
);

Statuses: pending, processing, done, dead. Attempts track retries. next_attempt_at drives the schedule.

Verify: create the database and table.

sqlite3 jobs.db "CREATE TABLE jobs (...);"
sqlite3 jobs.db ".schema"

Step 2: Enqueue a job

The API accepts a prompt. It writes a row. It returns immediately.

app.post('/jobs', async (req, res) => {
  const { prompt } = req.body;
  const id = crypto.randomUUID();
  await db.run(
    'INSERT INTO jobs (id, prompt, next_attempt_at, created_at) VALUES (?, ?, ?, ?)',
    [id, prompt, Date.now(), Date.now()]
  );
  res.json({ id, status: 'pending' });
});

No model call here. The response is instant. The caller stores the ID.

Verify: post a job. Confirm the row exists.

curl -s -X POST http://localhost:3000/jobs \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"Explain outbox pattern"}'
sqlite3 jobs.db "SELECT id, status FROM jobs;"

Step 3: The worker loop

The worker polls for due jobs. It marks them processing. It calls the model. It stores the result.

async function worker() {
  const job = await db.get(
    'SELECT * FROM jobs WHERE status = ? AND next_attempt_at <= ? ORDER BY created_at LIMIT 1',
    ['pending', Date.now()]
  );
  if (!job) return;

  await db.run('UPDATE jobs SET status = ? WHERE id = ?', ['processing', job.id]);
  try {
    const result = await callModel(job.prompt);
    await db.run(
      'UPDATE jobs SET status = ?, result = ? WHERE id = ?',
      ['done', JSON.stringify(result), job.id]
    );
  } catch (err) {
    await handleFailure(job, err);
  }
}

The worker runs on an interval. One job at a time. Simple and safe.

Verify: run the worker manually. Check the job status flips to done.

Step 4: Retry with backoff

Failures are expected. Each failure increments attempts. The next attempt moves into the future.

function backoffMs(attempt) {
  return Math.min(1000 * 2 ** attempt, 30_000);
}

async function handleFailure(job, err) {
  const attempts = job.attempts + 1;
  if (attempts >= job.max_attempts) {
    await db.run(
      'UPDATE jobs SET status = ?, error = ? WHERE id = ?',
      ['dead', err.message, job.id]
    );
    return;
  }
  await db.run(
    'UPDATE jobs SET status = ?, attempts = ?, next_attempt_at = ?, error = ? WHERE id = ?',
    ['pending', attempts, Date.now() + backoffMs(attempts), err.message, job.id]
  );
}

Exponential backoff respects the endpoint. A 429 gets a pause. A timeout gets a longer pause.

Verify: set max_attempts to 2. Point the endpoint at a dead URL. Confirm the job lands in dead.

Step 5: Idempotency keys

A crash can interrupt a processing job. The worker may retry it. The model may have already answered. Idempotency keys prevent double work.

Add a column: idempotency_key TEXT UNIQUE. The caller supplies it. The insert fails on duplicates.

app.post('/jobs', async (req, res) => {
  const { prompt, idempotency_key } = req.body;
  try {
    await db.run(
      'INSERT INTO jobs (id, prompt, idempotency_key, next_attempt_at, created_at) VALUES (?, ?, ?, ?, ?)',
      [crypto.randomUUID(), prompt, idempotency_key, Date.now(), Date.now()]
    );
    res.json({ accepted: true });
  } catch (e) {
    if (e.code === 'SQLITE_CONSTRAINT_UNIQUE') {
      res.json({ accepted: false, duplicate: true });
    } else {
      throw e;
    }
  }
});

The caller retries safely. The queue stores one job. No duplicate model calls.

Verify: send the same idempotency key twice. The second response says duplicate.

Step 6: Poll for results

The caller checks the job status. No webhooks needed.

app.get('/jobs/:id', async (req, res) => {
  const job = await db.get('SELECT * FROM jobs WHERE id = ?', [req.params.id]);
  if (!job) return res.status(404).json({ error: 'not found' });
  res.json({
    id: job.id,
    status: job.status,
    result: job.result ? JSON.parse(job.result) : null,
    error: job.error
  });
});

Verify: poll until status is done. Then read the result.

Step 7: Deploy on the free server

MonkeyCode's free server option runs this stack. Install SQLite. Set the database path. Start the worker and the HTTP server.

export DATABASE_PATH=/data/jobs.db
export FREE_MODEL_URL="https://your-endpoint.example/v1/complete"
export FREE_MODEL_KEY="your-key"
node server.js

The free model access supplies the endpoint. The free server supplies the uptime. The outbox supplies the durability.

Verify: restart the server mid-job. Confirm the job resumes from pending. No lost work.

Limitations

The outbox is not real-time. A job waits for the next poll. Polling adds latency. Use it for batch work, not chat.

Storage grows. Every prompt and result lives in SQLite. Add a retention job to delete old rows.

The worker is single-threaded. One slow call blocks the queue. Scale workers with care, or accept the bottleneck.

Idempotency only works if callers cooperate. A caller without a key can still double-submit.

Who should skip this

Skip this for interactive apps. Skip it for streaming responses. Skip it if your model endpoint is reliable and fast.

Use it for code review batches, documentation generation, test-case synthesis, or any workload where a minute of delay is fine.

The takeaway

A model call is a job. A job belongs in a queue. A queue survives crashes.

The outbox pattern turns a flaky free endpoint into a dependable worker. Requests persist first. Retries happen later. The caller moves on. That is the whole trick.