A week of reader feedback: what changed in my four AI-for-commerce tools

#rag #mcp #ai #opensource

Last week I published four posts about open-source tools I built for AI in e-commerce. The best thing that came out of it was the feedback. Several comments pointed at real gaps, and a few turned into fixes within a day. Here's what changed.

woocommerce-mcp: who confirmed, not just what

Original post: Safe write tools for agents

The server already bound a confirm to the exact arguments and the store state the preview was computed from. A commenter pointed out the missing half: the server couldn't tell a human-read confirm from the agent chaining preview and confirm in one turn.

v0.6.0 adds opt-in signed approvals. Set WOOCOMMERCE_APPROVAL_SECRET and every confirmed write needs a short-lived, single-use HMAC-SHA256 token bound to the previewId, tool and store. A person creates it out of band with woocommerce-mcp approve. Without the secret, behavior is unchanged. (commit)

Another thread was about honesty in the docs. stale_preview re-reads the records at confirm time and refuses if they changed, but it's check-then-write, not a lock. WooCommerce's REST API has no conditional writes, so a change that lands between the re-read and the write can't be caught. The docs now say so plainly. (commit)

persian-rag: one query, one vote

Original post: Why RAG breaks on Persian text · Released in v0.5.0

The original query got weight 2.0 in reciprocal rank fusion, and every spelling or half-space variant got a flat 1.0. A reader noticed that with up to six variants, correlated rewrites could together outvote the original for a competing chunk. The number of variants had become a hidden ranking parameter.

Now the original keeps its weight and all variants share one fixed weight, split evenly. Each result also records how much each variant contributed (metadata["query_contributions"]), so you can see why a chunk ranked where it did. (commit)

On the dense side, the default embedding model is now multilingual-e5-small with the correct E5 prefixes (#9), and multilingual-e5-base is available as a more accurate, heavier option via short names e5-small / e5-base (#10).

llm-search-eval: the judge changes under you

Original post: Is your new search actually better? · Released in v0.5.0

When you ask an OpenAI-compatible API for a model, the response often names a dated snapshot. If that snapshot changes mid-project, last month's scores and this month's aren't on the same scale, and a better judge can look like a regression in your system.

Every judgment now stores judge_model_version (per vote for panels), and report.json lists the versions seen. (commit)
If a model returned more than one version in a run, the report sets mixed_model_versions and every report format prints a warning. (commit)

commerce-agent-skills: a bug the tests couldn't see

Original post: Treat agent skills like code · Released in v0.3.3

The allocate-stock skill checks whether one location can fill an order. It checked each line on its own, so two lines of 3 for the same SKU both fit a location with 4 free units, and the order took 6. Quantities are now summed per SKU first.

Worse, removing the stock decrement entirely left the test suite green. The fix adds a small reserve() helper, a race test for the last unit, and a control test that runs the same race against a naive read-then-write reserve and expects it to oversell, proving the test can fail. (commit)

Upgrading

All four fixes are released:

# woocommerce-mcp 0.6.0
npm install -g @sadrmanesh/woocommerce-mcp@latest

# persian-rag 0.5.0 (Docker: ghcr.io/sadrmanesh/persian-rag:0.5.0)
pip install -U persian-rag

# llm-search-eval 0.5.0 (Docker: sadrmanesh/llm-search-eval:0.5.0)
pip install -U llm-search-eval

# commerce-agent-skills 0.3.3: re-run the installer to refresh your skills
npx @sadrmanesh/commerce-agent-skills@latest init

Takeaways

Write down what a safety check doesn't do. "Not a lock" is one sentence and saves someone a bad assumption.
Hidden parameters hide in counts. Number of variants, number of votes: anything that scales with a count can quietly dominate.
Record the version of everything that scores you. Including the judge.
Make sure your tests can fail. A test suite that passes with the core logic deleted is decoration.

Thanks to everyone who commented. Keep the hard questions coming.

woocommerce-mcp · persian-rag · llm-search-eval · commerce-agent-skills

DEV Community