DEV Community: Jono

Four surprises from writing a BigCommerce to Sanity catalog sync

Jono — Thu, 13 Aug 2026 14:57:16 +0000

TL;DR: We built turbo-start-bigcommerce, where a BigCommerce catalog syncs into Sanity and page-builder blocks reference the synced documents. Four things caught us out: webhooks that structurally can't do the job, an API that lies about page size, deletes needing their own program, and ids that belong to the other system. All four are in the open-source package if you'd rather read the code.

Every integration where content in one system references entities in another has the same shape, and it's always more work than it looks.

Ours is turbo-start-bigcommerce: the catalog lives in BigCommerce, the editorial content lives in Sanity, and page-builder blocks hold references across the gap. The sync is packages/sanity-sync, it's open source, and these are the four things we didn't see coming.

1. Webhooks structurally can't do the job

The plan is always webhooks. Entity changes over there, you get told, you update over here.

Here's the comment we ended up writing at the top of reconcile.ts:

packages/sanity-sync/src/reconcile.ts

/**
 * The sweep is the primary sync mechanism, not a fallback. BigCommerce has no
 * CRUD webhooks for variants and none for brands, and most product image
 * changes — including changing the thumbnail — fire no update event at all.
 * Webhook payloads are id-only, unordered, and can duplicate. A webhook-only
 * sync is therefore structurally incomplete; webhooks are at best a latency
 * optimisation layered on top of this sweep.
 */

The mechanism everyone budgets for turns out to be the one that can't do the job alone. That isn't a knock on BigCommerce. Partial webhook coverage is normal across catalog and asset APIs, and the gaps are never in the documentation. You discover them by noticing a thumbnail that didn't change.

Which means you write the sweep anyway. And once the sweep exists, the webhook becomes an optimisation you might not bother shipping.

2. The API lies about page size

packages/sanity-sync/src/reconcile.ts

/**
 * Admin REST caps a catalog page at 50 — but silently drops it to 10 the moment
 * `options` or `modifiers` are included. Verified against the sandbox:
 * `?limit=50&include=variants,options,images` comes back `per_page: 10`.
 */
const PAGE_SIZE = 50;
const PAGE_SIZE_WITH_OPTIONS = 10;

Ask for 50, get 10, no warning. Drive your pagination loop off the limit you requested rather than the total_pages the server reports and you'll sync a fifth of the catalog while your logs report success.

Categories have a different quirk. That resource has no date_modified:min filter at all, and asking for one returns 422 The filter(s): date_modified:min are not valid filter parameter(s), so categories get swept whole every time regardless of whether you asked for an incremental run.

Two unrelated quirks, one vendor, on precisely the two resources we needed. Neither is difficult to handle once you know. Finding out is the expensive part, and finding out is not on anybody's estimate.

3. Deletes are a separate program

An upsert sweep tells you what exists. It says nothing about what stopped existing, so there's a second pass:

packages/sanity-sync/src/reconcile.ts

const live = await client.fetch<string[]>(
  "*[_type in $types && store.isDeleted != true]._id",
  { types: ["bigcommerceProduct", "bigcommerceProductVariant", "bigcommerceCategory"] }
);

const stale = staleMutations(live, sweep.seen);

Everything Sanity still holds, minus everything the catalog just returned, is stale. We soft-delete rather than hard-delete, because content may still reference those documents and a tombstone beats a hole.

The pass can only run on a full sweep. Run it after an incremental one and every unmodified entity looks deleted, because the sweep never saw it in the first place. So it's guarded:

packages/sanity-sync/src/reconcile.ts

if (options.since) {
  logger.info("Incremental sweep — skipping the soft-delete pass.");
} else {
  softDeleted = await sweepDeletes(sweep, client);
}

One if, standing between you and soft-deleting your entire catalog on a cron.

4. The ids aren't yours

This is the one that generalises hardest, and the one that bit us in production.

Synced documents get deterministic ids: bigcommerceProduct-{entityId}, where entityId is whatever BigCommerce minted when the product was created. Every store counts from its own starting point. The crewneck that's 181 on the store we captured content from is some other number on yours, so a committed reference to bigcommerceProduct-181 is correct on exactly one store on earth and dangling everywhere else.

And a dangling weak reference, as our seed docs now say in as plain a form as we could manage:

A dangling weak reference renders as nothing — no error, no gap in the log, just an empty navbar.

No stack trace, no alert. A homepage quietly rendering four fewer products than it should, until someone scrolls past it on a Tuesday.

So the seed data contains no ids. It contains placeholders named by slug, bigcommerceProduct-bramley-wool-crewneck, and a separate command resolves them against whatever the sync actually wrote:

pnpm sync:bigcommerce    # catalog out of BigCommerce, into Sanity
pnpm seed:refs --write   # repoint the content at the ids this store minted

An entire extra build step, whose only reason to exist is that the identifier belongs to the other system. It's idempotent, since it only rewrites references whose tail isn't already numeric, and it's all-or-nothing, because half a remap leaves a dataset that is neither the old state nor the new one.

It also carries a trap we had to write down: no slug may be entirely numeric. Something like bigcommerceProduct-2024 looks like an id that has already been resolved, so the remap skips it, and it stays dangling. Empty navbar, no error, eighteen months later nobody remembers why.

Why any of this matters beyond commerce

Swap products for images and every item above survives the translation. That's the case against bolting a separate asset manager onto your CMS, and I've written that argument up properly with the numbers attached: the DAM cost procurement forgets.

The general rule we took away: before you agree to an integration, work out whether a reference from your content to the other system's entity will be a real reference your CMS understands, or a string. If it's a string, everything above is on your roadmap whether it's on the estimate or not.

The whole package is MIT and the tests are in there too. Take it, that's what a starter is for.

Building commerce on Sanity and Next.js? Structured content modelling, catalog integrations, and migrations off legacy stacks. See our Sanity service. Expect some of it to come back as a recommendation to spend nothing.

Frequently asked questions

Can I use the sync package outside the starter?
It's written against this repo's schema, but the shape is portable: a paginated sweep, an upsert path shared with the single-entity sync, a soft-delete pass, and an id remap. Most of the value is in the structure rather than the BigCommerce specifics.

Why soft-delete instead of removing the document?
Because content may still reference it. A soft-deleted document lets the front end render a tombstone or skip the item deliberately, rather than resolving a reference to nothing and rendering an empty space nobody notices.

Do you run the sweep on a schedule?
Nothing invokes it in the starter, deliberately. You run it by hand or wire it to whatever scheduler you use. The single-entity path exists so you can reproduce a webhook delivery from a terminal before the route exists.

I'm Jono. I run Roboto Studio. Next.js and Sanity mostly, plus a lot of time spent unpicking integrations that seemed reasonable at the time. If you've written a sync like this, I'd like to hear which surprise got you.

The DAM cost procurement forgets, in code

Jono — Thu, 13 Aug 2026 14:34:29 +0000

TL;DR: DAM quotes land in the mid five figures a year, and that figure doesn't include the integration. The moment your site depends on assets living in someone else's system, you own a sync pipeline: partial webhook coverage, a reconciliation sweep, a delete pass, and ids you didn't mint. We've written that pipeline. It's more work than the demo call implies, and it's the strongest argument for keeping assets in your content graph.

There's a meeting on every enterprise headless build where someone asks where the DAM goes. We wrote up the commercial answer to that question in Mux + Sanity: do you really need a DAM?, and the short version is that most teams already own a DAM through Sanity's Media Library, and that dedicated licences run to $30k–70k a year for a system that still can't stream video.

One line in that post's FAQ gets a single sentence and deserves a whole article:

That second cost is the one procurement forgets: a DAM your site depends on needs a sync pipeline, webhook handling, and someone who owns it.

This is that article. Not the licence, the pipeline.

The distinction that decides everything

An asset in Sanity's Media Library is a document. Your front end queries it like any other document, and a reference to it from a page is a real reference the CMS understands.

An asset in a bolted-on DAM is, from your CMS's point of view, a string. A URL or an external id, sitting in a field, pointing at a row in someone else's database. As the original post puts it, it's "a file behind someone else's API that you mirror into your CMS and hope stays in sync."

That word hope is doing a lot of work, and closing the gap it describes is an engineering project. Here's what's in it.

Webhooks won't cover it

The plan is always webhooks. The asset changes over there, you get told, you update over here.

We build commerce integrations with exactly this shape, where a catalog lives in one system and the content referencing it lives in Sanity. Here's the comment we ended up writing at the top of our sync package, after finding out how much the events actually cover:

packages/sanity-sync/src/reconcile.ts

/**
 * The sweep is the primary sync mechanism, not a fallback. BigCommerce has no
 * CRUD webhooks for variants and none for brands, and most product image
 * changes — including changing the thumbnail — fire no update event at all.
 * Webhook payloads are id-only, unordered, and can duplicate. A webhook-only
 * sync is therefore structurally incomplete; webhooks are at best a latency
 * optimisation layered on top of this sweep.
 */

That's BigCommerce, not a DAM, but the shape is the vendor-agnostic one. Webhook coverage is partial, the gaps aren't in the documentation, and you find them by noticing a thumbnail that didn't change. Which means you write a scheduled sweep that walks the whole library and compares it against what you're holding. Once the sweep exists, the webhook is a latency optimisation you may not bother shipping.

Nobody scopes the sweep on the demo call. It's the actual deliverable.

Then the parts that only show up later

Deletes are a second program. An upsert sweep tells you what exists and says nothing about what stopped existing. So you need a subtraction pass: everything you hold, minus everything the library just returned, is stale. Ours soft-deletes rather than hard-deletes, because content may still reference those documents and a tombstone beats a hole. It also has to be guarded so it only ever runs on a full sweep, because after an incremental one every unmodified asset looks deleted. That guard is one if standing between you and wiping the library on a cron.

The ids aren't yours. Assets get their identity from the DAM. Anything you seed, commit, or move between environments is carrying an identifier minted by a system you don't control, so you need a resolution step that repoints content at whatever ids this environment actually has. We ship one as a separate build command. It exists purely because the identifier belongs to someone else.

Alt text has to be copied. You're not making a third-party HTTP call during a server render, so asset metadata gets denormalised into your CMS. Now you hold two copies and a job whose entire purpose is keeping them equal, and every field marketing adds over there is a schema change over here.

Nothing tells you when it breaks. This is the one that stings. A reference pointing at an id that no longer resolves doesn't throw. In our stack it renders as nothing at all, which we wrote into the seed docs in as plain a form as we could manage:

A dangling weak reference renders as nothing — no error, no gap in the log, just an empty navbar.

No stack trace, no alert. A homepage quietly rendering four fewer things than it should, until somebody scrolls past it on a Tuesday.

The whole pipeline is open source if you want to read it rather than take our word for it, and I've written up the four things that caught us out building it: Four surprises from writing a BigCommerce to Sanity catalog sync.

What this is worth in the DAM decision

Put the two costs next to each other. The licence is $30k–70k a year. The pipeline above is a build, plus an owner, plus an on-call story, indefinitely. Procurement prices the first and rarely the second, which is how a DAM ends up looking cheaper than it is.

None of it exists if assets are documents in the same content graph as your content. There's no webhook because there's no second system, no reconciliation because there's nothing to reconcile, and referential integrity is just referential integrity, because the CMS knows what uses what.

Usual disclosure: Sanity is the stack we build on and sell, so weigh our enthusiasm accordingly. The architecture question survives whatever CMS you're on. Ask it of Contentful, Payload or Strapi and the answer will tell you what you're signing up for.

Video is the honest exception, and it splits the other way. Storing a video and delivering one are unrelated problems, and no DAM solves the second, so a specialist service earns its place there whatever you do about images. We wrote up the player side separately, lazy-loaded facade and all: a really nice Mux video component.

Modelling content before you buy tooling. Structured content modelling, migrations off legacy stacks, and the unglamorous business of deciding what should be a reference and what shouldn't. See our Sanity service. Expect some of it to come back as a recommendation to spend nothing.

The question to ask in the meeting

Not "what features does it have". This:

Will a reference from a page to an asset be a real reference the CMS understands, or a string?

If the answer is a string, ask who owns the sweep, who's on call for it, and what happens when the two systems disagree. Those questions all have answers. They're just expensive, and they're much cheaper to hear before the contract than after.

The genuine exception holds up. If several disconnected projects share one asset operation and most of the consumers aren't on your CMS, a central DAM is doing work nothing else is doing, and the build cost really can climb past the licence. That's a smaller share of teams than the sales cycle implies.

Frequently asked questions

Isn't a sync pipeline just the cost of any integration?
Mostly it's the cost of stateful integrations. Plenty of services are request-response: you call them, you get an answer, nothing needs reconciling afterwards. Asset systems are different because durable content holds references to their records, so the two systems have to agree over time rather than at one moment.

Our DAM ships a CMS plugin. Doesn't that cover it?
It covers the picker, usually the metadata copy, sometimes the sweep. It can't give you referential integrity, because that would need your CMS to own the asset record, which is the thing the DAM exists to prevent. Read the plugin's source and check what it does on a delete.

What if the DAM is already bought?
Count the consumers. If everything downstream is a single CMS, you're paying a licence plus the engineering tax above to keep two systems agreeing. If genuinely disconnected systems consume those assets, it's earning its keep and this post isn't about you.

I'm Jono. I run Roboto Studio. Next.js and Sanity mostly, plus a lot of time spent unpicking integrations that seemed reasonable at the time. If you've written the sync pipeline, I'd like to hear which part caught you out.

Agentic eCommerce with Shopify and Sanity

Jono — Wed, 17 Jun 2026 21:04:48 +0000

Turbo Start Aisle is our agentic e-commerce starter on Shopify and Sanity. Instead of filters and category pages, you chat with an agent that builds the shopping UI as you go, narrowing by brand, size, and colour, and answering questions like the return policy in plain language. In the video I shop for tea and let the agent do the filtering.

FAQs

What is agentic e-commerce?

It's a way for customers to interact with your store by talking to an AI agent instead of clicking through filters and category menus. You ask for what you want in plain language, whether that's a product, a size, a colour, a brand, or a policy question, and the agent answers in real time.

How do agents generate UI in e-commerce?

The agent reads your request and renders the matching products and options on the spot, rather than sending you to a pre-built category page. You see only what fits what you asked for.

Can I ask agents about product specifics like size and color?

Yes, that's most of what the demo does. Ask whether a product comes in a given size, colour, or brand and the agent checks inventory and shows what's in stock. It narrows things down through conversation instead of stacked filters.

What else can agentic e-commerce agents help with besides finding products?

More than product search. You can ask about returns, shipping, or warranties and get a plain-language answer, or ask anything else about how the store works.

How does this compare to traditional e-commerce filtering?

Traditional e-commerce makes you navigate categories and set the filters yourself. With an agent you describe what you need and it does the searching and filtering for you in the chat.

Is agentic e-commerce only for small websites?

We demoed it on a small store, but it suits large catalogues just as well. The more products and categories you have, the more an agent saves customers from hunting through menus.

How do I try agentic e-commerce?

We built Turbo Start Aisle on Shopify and Sanity as a working example. Open the chat, describe what you want, and the agent helps you narrow it down.

Transcript

Hey, how's it going? Today I'm going to talk about Agentic e-commerce, what it means, and how it's going to completely shape the way users interact with your e-commerce website. In today's example, I will show you how the agents will work to generate UI on the fly and it would be completely the opposite of the traditional way of using filters and categories in your e-commerce website. So let's have a look.

Okay, so I'm going to try to find a tea for myself. And I'm at Turbo at Isle, which is Robboto's e-commerce website built with Shopify and Sanity. So let's go to the chat. And I'm going to first of all find out what brands they have. Oh, great. So they have Robboto. Perfect. That's the top end of my budget. 50 quid tea is still quite expensive. Oh, thank God they have three options. Great. Do you have any in small? Oh, amazing. Do you have in small and pink? Pink, not whatever I typed. Oh, great. So let's have a look. Okay, so they have Tazmin in pink and in my size. So this is a great example of just showing how you would interact with agents in an e-commerce website.

You can also probably find out what the policy is like, return policy. So, um, explain written policy to me like I am a two-year-old. Okay, so that's pretty straightforward. So this is one example of how you would interact with agents in your e-commerce website. Obviously you can do it in many different ways and Turbo at Isle is a very, you know, small website with some products and categories. But if it's a large e-commerce, the world is your oyster. Go crazy with it. Hope you enjoyed today's video. Thank you so much for watching.

Sanity SEO done right: Open Graph, JSON-LD, fallbacks

Jono — Tue, 19 May 2026 20:25:57 +0000

Every Sanity project we ship starts with the same SEO baseline: a title and description on the document, an SEO tab that overrides them when needed, and Open Graph fields one layer deeper. Fallbacks all the way down, so the page always renders something sensible even when an editor forgets a field. Here's the exact pattern we use, and where to find it in Turbostart Sanity if you want to copy it.

FAQs

Why does Sanity feel so unopinionated about SEO?

Because Sanity has to model anything — a blog, an e-commerce catalogue, a screen that shows live train times. The trade-off for that range is that the platform won't pick an SEO pattern for you. You bring the opinion; Sanity gives you the schema.

What fields should every page document have as an SEO baseline?

Title and description at the top level, used as both the page heading/blurb and the default meta. A fallback SEO image. Then a separate SEO tab with seoTitle, seoDescription, seoImage, seoNoIndex, and seoHideFromLists for overrides. Open Graph fields sit a layer deeper for the cases where you want social cards to differ from search snippets.

Why have an SEO tab if the title and description already exist on the document?

Consistency. Editors get one place to write the page copy and one place to override the meta when SEO needs something different — without leaving fields blank or duplicating data. The top-level fields always act as the fallback, so nothing renders empty.

Should JSON-LD live in the CMS or be generated automatically?

Generate it. If the author, title, description, and publish date are already structured in Sanity, map them straight into JSON-LD in your Next.js page. Set and forget. Editors never touch it, and the structured data stays in lockstep with the content.

What does `seoHideFromLists` actually do?

It's a flag that lets editors keep a page indexable but exclude it from listing pages. Useful for landing pages tied to paid campaigns, or one-off blog posts you don't want surfacing on the blog index. Filter on it in your GROQ query and the page disappears from lists without needing noIndex.

Transcript

Hey bud, how you doing? So, first of all, I wanted to put this on YouTube so everybody else can actually see the answer to this question because I get asked this so many times on a daily basis, but I thought it might be helpful for you.

Uh, so in short, and this is a very very common thing that we get: has anybody done SEO work on Sanity before? It doesn't really go too much into what's actually built into the model, but I hear this time and time again. Um, and the reason why you have this happen so much is because Sanity is very very unopinionated. And the reason why it's very very unopinionated is because it can create models that can do anything — genuinely anything when it comes to just writing a blog, or it can be more complex as to running an entire, like, e-commerce system, or it can be as simple as creating, like, a terminal that tells you the latest train times at a train station. It's really really um scalable.

Anyway, so what I'll talk about is a little bit of this. So this is an old learn article that we wrote with Sanity, which is really helpful. It tells you about how you should opinionate your actual SEO, and it's all good and fine if you want to take the time out to go look through this, that's great. You've got a lot more time than I have, so that's fantastic.

Now, I'm going to show you something totally different. This is actually the Turbo Art, and this is the way that we do it. It's the same thing that's in the Sanity learn documentation. Um, but I know how low people's attention spans are today, so very, very quickly, I'll show you how this works. This is how we opinionate SEO and we do the exact same on every single one of our projects.

So, when we build a page for a page builder or for a blog or any other document type, it has a title. Okay? And you can do validation. You can do things like, oh, it has to be x number of characters. You can do that. That's really cool if you think it would help the client. But for the simplicity of this, we don't have, like, that little number underneath um because we're trying to be unopinionated.

Um, and description, which is going to be effectively used as your meta description on the page. This can also double up. So if, for example, it's a blog, um, the title of the blog could also be the title that sits inside of here, and the description could be the little blurb that you get just before you click into the blog on a card. That's one thing to know. So this becomes our baseline. The same with images. So you may see an image in here as well. Um, which I believe is an SEO — so ours is a SEO image override. Um, however, sometimes we actually include the image in here as well. It could be that it should be in here. Nope. Okay. Um, that might get tweaked on the Turbo Art Sanity, but essentially I like to have an image inside of here as well as part of the pages as a fallback image.

However, we have this secondary tab which is SEO. This is your SEO, your meta title. So your title tags at the top, um, and the meta description which would be in the meta description section. Um, and these will override the title and the description. And you might ask, hold up, if we've got the title and description, then what's going to happen if we override them? Then there surely there's no reason to have these in the first place. Well, the reason we have these is because they're always a fallback. So it means that you have a consistent way of editing your pages, whether that's a page with page builder blocks or a blog which has, like, the blog title, blog description. So it may not appear on all pages.

Now we've got our SEO. So our metas, what about our open graph? Well, makes perfect sense. Let's do open graph title, open graph description. And you can go a step further and also do, I think there's a separate open graph image. Uh, you can separate it out down to as granular level as here's what we want to have on Twitter, here's what we want to have on Facebook, and I think LinkedIn does it as well, just off the top of my head. Um, this is kind of like our de facto baseline. So I would highly, highly recommend doing it this way. Um, however, you can go a lot more granular. You can put JSON LD in there. You can do all sorts of things. However, a note on JSON LD, and I've said this to a lot of different people: I much, much prefer set and forget JSON LD. So if you're already putting all this data, like titles and descriptions and things like that, in there, and authors — if you're on a blog page — then if you're on a blog page, let me show you a quick example of this. Um, I'm wondering if we have authors on here. I hope we do. If we don't, I'm going to tell you how to do it. Uh, we do. Great. So if we've got all of these authors that are inside of here, why don't we just pull those authors into JSON LD by default, which is set and forget? So you don't have to think about it. You just enter the data inside of Sanity, and a like-for-like pair comes up with JSON LD. And it's not so complicated these days with AI. So you can even do this relatively quickly. Um, but yeah, this is what I'm talking about when I talk about a de facto default SEO experience. What you should use as a baseline for every single Sanity website that is going to be a website. This obviously differs if it's going to be a different function. But if you're looking to start this straight away, or in your case, I would just literally go to the Sanity exchange, go find Turbo Art Sanity. You can also find this on GitHub. Um, go view the repo and tell the developer to just go look inside of here. Um, and pay special attention to two files. One of them is query. So query shows you how we handle our GROQ queries. And GROQ is the query language from Sanity. And then the second thing you want to pay attention to inside of here, um, is — let me see if I can find it for you. Let me just press up top any second now — is the SEO. I think it's SEO fields, which you can see. You can reuse this. This is what we use on ours where it's the SEO title, SEO description. Makes perfect sense. SEO image, SEO no index, and SEO hide from lists. And the reason we have these two left is because obviously you want to have a no index. Sometimes maybe it's a pay-per-click content page that you very specifically want to, like, tunnel down um a specific person that's coming to your website. And the second one is SEO hide from lists because, for example, you have a blog post. It may be that you don't want that blog post to show up on certain lists. It may be that you have a no index and you also don't want it to show up on certain lists, or vice versa. That's the reason why we do this because if you have SEO hide from lists inside of your GROQ, you can say give me all the blogs except for the ones that are getting hidden from lists, and it's really nice and easy to work with.

Uh, there's some very basic validation here, but I hope this helps with you trying to solve this SEO problem. If it continues, just give us a shout. We're more than happy to try and help and give you pointers in the right direction. Thanks a lot. Have a great day. Take care. Bye-bye.

DEV Community: Jono

Four surprises from writing a BigCommerce to Sanity catalog sync

1. Webhooks structurally can't do the job

2. The API lies about page size

3. Deletes are a separate program

4. The ids aren't yours

Why any of this matters beyond commerce

Frequently asked questions

The DAM cost procurement forgets, in code

The distinction that decides everything

Webhooks won't cover it

Then the parts that only show up later

What this is worth in the DAM decision

The question to ask in the meeting

Frequently asked questions

Agentic eCommerce with Shopify and Sanity

FAQs

What is agentic e-commerce?

How do agents generate UI in e-commerce?

Can I ask agents about product specifics like size and color?

What else can agentic e-commerce agents help with besides finding products?

How does this compare to traditional e-commerce filtering?

Is agentic e-commerce only for small websites?

How do I try agentic e-commerce?

Transcript

Sanity SEO done right: Open Graph, JSON-LD, fallbacks

FAQs

Why does Sanity feel so unopinionated about SEO?

What fields should every page document have as an SEO baseline?

Why have an SEO tab if the title and description already exist on the document?

Should JSON-LD live in the CMS or be generated automatically?

What does seoHideFromLists actually do?

Transcript

What does `seoHideFromLists` actually do?