DEV Community

Ana Julia Bittencourt
Ana Julia Bittencourt

Posted on • Originally published at blog.memoclaw.com

Batch storing memories: when MemoClaw's bulk API saves 87 percent

Batch storing memories: when MemoClaw's bulk API saves 87 percent

Every OpenClaw agent starts with a few manual stores. Then a migration shows up and you suddenly need to stuff thousands of facts into MemoClaw. Calling memoclaw store one record at a time feels safe, but it torches both time and budget. The batch endpoint was built for these moments. It writes up to one hundred memories per call for $0.04, keeps tags and importance intact, and gives you atomic feedback so you know which payloads succeeded. If you plan around that endpoint from the start, onboarding a new customer or backfilling a year of tickets becomes a short background job instead of an all-nighter.

Our first big pilot with a design studio forced this lesson. The lead dev tried to import 4,800 markdown snippets with a while loop and individual store calls, then pinged me at 3 a.m. asking why the wallet balance evaporated. We rewrote the job with batch uploads, re-ran it in twelve minutes, and spent $1.92 instead of $24. That became the policy: no more manual drips when a replay is on the calendar.

TL;DR

  • Single store calls cost $0.005 each after the free tier. Batch store writes up to 100 records for $0.04, which is an 87 percent discount once you batch more than eight items.
  • Batch shines on user onboarding, migrating MEMORY.md files, nightly session summaries, and any workflow that emits structured observations.
  • Design payloads with consistent schemas: text body, tags array, namespace, importance. Validate locally before shipping them to MemoClaw.
  • Batch imports land outside the prompt window, so you add thousands of memories with zero token tax while every OpenClaw sub-agent on the same wallet gains access instantly.
  • Pair the batch endpoint with OpenClaw cron jobs so long imports happen off the critical path. Capture partial failures, retry idempotently, and log the returned IDs for audits.

The cost math

MemoClaw only charges for operations that touch embeddings or GPT. That still adds up when you are loading historic data.

Records Singles cost Batch cost Savings
10 $0.05 $0.04 20%
50 $0.25 $0.04 84%
100 $0.50 $0.04 92%

Those numbers mean you should default to batch for any replay job over eight items. Singles still make sense for real time logging during an agent session, but scheduled jobs should always bundle writes.

When batch storage wins

Customer onboarding

A new enterprise wallet shows up with a PDF of preferences, network maps, runbooks, and gotchas. Parsing that once and batching the output saves two things: money and opportunity cost. Instead of waiting half an hour for a loop of single stores, you drop the entire kit for $0.04 and move on.

MEMORY.md migrations

Thousands of OpenClaw users still keep long MEMORY.md files. When they move to MemoClaw they have to import years of notes. Split the file into paragraphs, assign tags like source:memory-md plus the original folder path, then batch. The importer can even keep the original timestamp in a custom tag so future recalls filter by timeframe.

Session summaries

Agents that run 24 hours keep end of shift summaries. Rather than storing each bullet by itself, buffer them for five minutes and write one batch. You keep semantic granularity while slashing calls.

Cron driven data sync

If you need to sync answers from an external system every hour, there is no reason to use single stores. Fetch the data, build the payload, and send it through the batch endpoint so you stay inside budget.

Designing a clean payload

A good batch payload is boring and predictable. Each memory in the array should carry the same fields so downstream recalls stay reliable.

[
  {
    "text": "Customer SolarFlux deploys every Wednesday and freezes changes during launches.",
    "namespace": "customer-ops",
    "tags": ["customer:solarflux", "cadence:weekly", "risk:launch"],
    "importance": 0.74
  },
  {
    "text": "Customer SolarFlux keeps emergency contacts at ops@solarflux.dev.",
    "namespace": "customer-ops",
    "tags": ["customer:solarflux", "contact:ops"],
    "importance": 0.63
  }
]
Enter fullscreen mode Exit fullscreen mode

Tips:

  • Keep texts under 8192 characters. That is the per memory cap.
  • Normalize tags. Lowercase, no spaces, predictable prefixes like customer: or source:.
  • Carry timestamps. If the source data has dates, put them in a tag like captured:2026-03-18 so recall filters can reason about freshness.
  • Validate locally. Run a JSON schema check before calling the API so you fail fast.

Wiring batch storage into OpenClaw

The easiest place to run batch jobs is a cron agent. Here is a starter flow for migrating a MEMORY.md file every night until the backlog is empty.

#!/bin/bash
set -euo pipefail

SOURCE=~/openclaw/workspace/memory/backlog/*.md
for file in $SOURCE; do
  jq -Rs 'split("\n\n") | map(select(length > 0)) |
    each as $paragraph |
    {text: $paragraph, namespace: "memory-backfill", tags: ["source:memory-md", "file:'"$file"'"], importance: 0.55}' \
    "$file" > /tmp/batch.json
  memoclaw store --batch < /tmp/batch.json
  mv "$file" ~/openclaw/workspace/memory/imported/
done
Enter fullscreen mode Exit fullscreen mode

You do not have to write bash. The CLI accepts JSON files or stdin, and the SDKs expose the same structure if you prefer Python or TypeScript. The pattern is what matters: gather items, shape them, batch once.

Handling partial failures

The batch endpoint returns success per memory. You still have to act on it.

  • Check response codes. A 200 means the API processed the batch. Inside the body you will find per item status plus IDs for the successful ones.
  • Retry idempotently. Keep a hash of text + namespace + tags. If a retry payload matches an earlier write you can skip storing it again or accept the duplicate and clean it later with consolidate.
  • Log IDs. Write every returned memory ID into a spreadsheet or datastore. That log is what you will use later for deletes or audits.
  • Alert on repeated failures. If more than 5 percent of items fail, pause the job and surface the errors. They usually indicate a malformed payload or a tag naming bug.

Observability and budgets

Batch jobs are invisible unless you record what happened.

  • Use memoclaw stats --namespace <name> --since <date>. It shows how many stores hit the namespace each day. Spikes reveal failed dedupe or a runaway job.
  • Export after each batch. memoclaw export --namespace memory-backfill --since 2026-03-18 gives you the exact payloads that landed. Store the file in object storage.
  • Track spend per namespace. Run memoclaw stats --json --namespace memory-backfill | jq -r '.spend.usd' (or hit /v1/stats?namespace=memory-backfill with x-wallet-auth) and feed that number into whatever dashboard Finance watches so they see the real batch costs.
  • Compare before vs after recall latency. Once the batch job completes, measure how long recalls take. If latency rises, add more tags or split namespaces to keep result sets small.

Real example: onboarding a pilot customer

A studio signs up for a pilot and hands you a trove of context. Here is how the flow looks:

  1. Normalize the data. Parse their Google Doc into discrete facts: toolchain, release cadence, escalation contacts, risk items.
  2. Score importance. Anything tied to production stability gets 0.8+. Preferences land around 0.6.
  3. Assign namespaces. Put operational details into customer-ops. Put product wishes into customer-product. That split makes later recalls sharper.
  4. Batch store. Build the JSON payload and run cat payload.json | memoclaw store --batch.
  5. Record IDs. Log the output to customer-solarflux-import.log with timestamp, ID, and source paragraph.
  6. Trigger a recall. Have the support agent call memoclaw recall --namespace customer-ops --tags customer:solarflux --limit 5 during its first plan step. The context shows up immediately for every agent that shares the wallet, no prompt restuffing required.

Total time: under fifteen minutes including scripting. Without batch you would have spent half an hour waiting on sequential writes and would have paid five times as much.

Checklist for safe imports

  1. Confirm the wallet has enough balance for the entire batch. 100 records cost $0.04, so free tier wallets can cover two batches before paying.
  2. Dry run by slicing the first ten items and uploading them alone. Inspect the returned IDs and tags.
  3. Record the git commit or document that generated the data so you can trace the source later.
  4. After the import, run memoclaw list --namespace <name> --limit 20 --json to spot check content.
  5. Schedule a follow up recall script that ensures the new data actually influences agent plans. Storage without recall is wasted effort.

Advanced batching patterns

  • Rolling window batches: When agents generate dozens of observations per hour, drop them into a queue and flush every five minutes or every 40 items, whichever comes first. That keeps latency low while still unlocking the discount.
  • Hybrid writes: Mix high priority singles with low priority batches. For example, store a critical outage note immediately, but buffer the supporting details for the next batch.
  • Namespace mirroring: Write the same batch to two namespaces when you need different recall strategies. Support might recall by customer, while product recalls by feature.
  • Importance stratification: Sort the payload before uploading so high importance items sit first. If a batch fails halfway through you at least persisted the critical pieces.
  • Cross wallet mirroring: Some teams keep a staging wallet for QA. Run the exact same batch against staging before production. When staging passes you can trust the production upload.

TypeScript batch runner for OpenClaw subagents

import { readFileSync } from "fs";
import { MemoClaw } from "@memoclaw/sdk";

const client = new MemoClaw({ walletKey: process.env.WALLET_KEY! });

async function runBatch(path: string) {
  const payload = JSON.parse(readFileSync(path, "utf8"));
  const response = await client.storeBatch(payload);
  response.items.forEach(item => {
    if (item.status === "ok") {
      console.log(`stored ${item.id} in ${item.namespace}`);
    } else {
      console.error(`failed item: ${item.error}`);
    }
  });
}

runBatch(process.argv[2]).catch(err => {
  console.error(err);
  process.exit(1);
});
Enter fullscreen mode Exit fullscreen mode

Drop that script into an OpenClaw automation and have the agent generate the payload file, then call node batch-runner.js payload.json. It keeps retries simple and gives you structured logs per item.

Post-batch recall strategy

Uploading is half the job. You also have to confirm that future recalls actually use the new data. Build a short checklist for every import:

  1. Run targeted recalls such as memoclaw recall --namespace customer-ops --tags customer:solarflux --limit 3. Read the output and make sure it reflects the payload.
  2. Add automated tests that spin up a throwaway OpenClaw agent, call the recall command, and assert that the returned JSON includes the expected tags.
  3. Monitor token usage. Once the agent stops re asking humans for the same context, you should see prompt size drop. If it does not, you forgot to wire the recall into your planning loop.
  4. Teach humans how to ask for the new memories. If sales engineers do not know tags exist, they will never filter by customer.

Governance and cleanup

Batches can rot if you never revisit them. Schedule maintenance as part of the same project:

  • Age off stale imports. If an onboarding batch is older than six months, export it, archive it, then delete it so recalls stay lean.
  • Tag drift audits. Once a month run memoclaw list --namespace customer-ops --limit 100 --json | jq '.[].tags' | sort | uniq -c to inspect which tags dominate. If new prefixes appear, decide whether they are valid or need cleanup.
  • Immutable guardrails. For compliance memories, pass --immutable when storing. During audits, verify the flag is still present so nobody silently edits those facts.
  • Cross namespace checks. If two namespaces keep referencing the same source data, merge them or add better tags. Overlapping namespaces confuse recalls and double your spend.

FAQ

What happens if one memory in the batch is too large? That item fails with a descriptive error. The rest of the batch still writes. Fix the oversized entry by trimming the text or splitting it into two records, then retry only that item.

Can I mix namespaces inside one batch? Yes. Each item carries its own namespace, tags, and importance score. That makes it easy to import a mix of support notes and engineering summaries without extra calls.

How do I keep batch jobs from clashing with real time stores? Use namespaces to isolate historical imports. Real time agents can keep using default or support-live while your cron agent writes to support-backfill. Later you can consolidate the two spaces.

Does batching affect recall relevance? No. MemoClaw indexes each record individually. As long as your tags and importance scores are honest, the search results stay sharp.

Do I still get 100 free calls? Yes. The free tier counts API calls, not records. A wallet can run 100 batch calls for free, which translates to up to 10,000 memories before you pay.

Conclusion

Batch storage is the cheapest lever you have for loading context into MemoClaw. It turns migrations, onboarding, and nightly flushes into predictable jobs with predictable bills. Plan the payload, validate tags, watch the stats, and treat the returned IDs like receipts. Once you make batching a default, you will stop telling customers to wait while an agent backfills context one record at a time.

Top comments (0)