At the end of last November, the quality manager of a medical-device manufacturer — I'll call her Renata — sent us a screenshot instead of a ticket. It showed her company's batch-release table, filtered to the current month. Eleven of the rows were hers. She had built them three weeks before go-live while learning the approval flow, and had typed the word TEST into the batch number field so that she would know which ones to clean up afterwards.
The go-live went fine. The audit did not. Her auditor wanted to know why eleven released batches had no corresponding production order, and "those are mine, I was practising" is not a sentence you can say to an auditor, because by then the platform had stamped each row with her name, a timestamp, and a completed approval chain.
Eleven rows. No code involved, no deployment, no migration. That is the part I keep coming back to.
Copying code is a solved problem. Copying a business is not.
Every other kind of software has a well-worn answer here: you branch, you spin up a staging environment, you point it at a copy of the database with the names scrambled. The reason it works is that the application and its state are separable. You can ship the code without the customers.
In a low-code platform that separation quietly disappears. What a customer builds is not a program; it is a configuration plus reference data plus files plus credentials plus the history those three have already accumulated. The "application" is a thin interpreter over the business's own tables. So a test environment cannot be a copy of the code, because the code is empty. It has to be a copy of the business, and nobody wants a copy of their business sitting on a server for three weeks going stale.
What teams do instead is copy the definition and leave the data behind. That environment loads, the forms render, and the first automation fails because the customer master it joins to has four rows instead of forty thousand. A test environment that cannot reproduce the data cannot reproduce the bug, so people go back to the only place where the data is real.
Our transaction ends when the automation ends
We are proud of our atomicity story, and it is true as far as it goes: one automation runs in one transaction, and if it throws or is told to stop, everything it wrote to its own tables rolls back.
That guarantee is scoped to a unit of execution, not to a unit of intent. A test is a session: a person, a form, a submit, a notification, a second person approving, a dashboard refreshing. Only the first fragment is inside a transaction. By the time Renata had clicked through four screens, three of those clicks had already committed, and each was a decision made by a different part of the platform that had no idea it was participating in a rehearsal.
The rollback we offer is a rollback of a step. The rollback a tester needs is a rollback of an afternoon.
The leak is not the data. It is the message.
Then there is the detail that turns a training mistake into an incident. The approval flow had a notification step. When Renata submitted her eleventh test batch, a message went out through the company's messaging client to the plant director, the QA lead, and — because the escalation rule was still at its default — the general manager.
Nobody could recall that message. There is no draft state for a notification that has already been read on someone's phone at 9:40 on a Tuesday night.
We do have a suppression mechanism: a thread-level flag that a script can set to stop notifications during a batch run. It is a real feature, built for imports and bulk updates. But it lives in the script runtime, so the person who can use it is the person who writes JavaScript. The person building the approval flow in the designer — the one this product exists for — cannot reach it. The "test mode" exists, and it is available to exactly the users who need it least.
A test record has no colour
Every system I worked on before this one had some way of marking synthetic data: a flag column, a tenant prefix, a partitioned schema, at minimum a naming convention the reporting layer knew to ignore.
Ours has none. A test record is a record. It counts toward the row quota. It appears in the dashboard that the CFO opens on Monday. It is included in the export that goes to the auditor. It shows up in the change log with a real user's name, which is worse than showing up as anonymous, because a log with a name on it reads like testimony.
Renata's team did the reasonable thing and wrote a cleanup script. It ran at the end of the pilot and deleted the rows, and the deletion itself became an event: eleven deletions by a quality manager in a table that is supposed to be append-only, each one recorded. The audit trail of the cleanup was longer than the cleanup.
The scheduler does not have a tomorrow
The hardest case is the one nobody demonstrates. A nightly job that closes overdue work orders, an escalation rule that fires after four hours, a monthly statement. How do you test something whose trigger is the passage of time?
You can wait. You can also invoke the same automation over HTTP by hand, which is what competent teams do — and which tests the logic while skipping the trigger, so the schedule itself stays untested until its first real night.
What you cannot do is ask the platform what this job will do at 2 a.m. on the last day of the quarter, against data that will exist then. There is no dry run, no as-of date, no shadow execution. And because only one scheduled job can execute at a time, a manual test run at the wrong minute can consume the slot the production run was waiting for — the test does not merely fail to prove anything, it occasionally displaces the real work.
What a real sandbox would cost
I have costed it. The number is not the reason we have not built it. The reason is that a sandbox forces a decision nobody wants to make in front of a customer: which side effects are real.
Database writes can be isolated; that is a copy of a database. Notifications, messaging, email: do we swallow them, route them to a test inbox, or let them out with a banner? Every option is visible to end users, and every option is wrong for somebody. Credentials: the sandbox needs the ERP's test endpoint, and half our customers do not have one, so it either reaches production systems or cannot run the integration at all.
Then the part that actually kills it. A sandbox needs data, and a copy of production data in a lower environment is a security incident waiting to be scheduled. Subsetting and masking sound like a weekend of work and are an ongoing programme, owned by someone, reviewed annually, with its own audit findings.
A sandbox is a second product, priced like an environment and staffed like a compliance function.
The uncomfortable conclusion
Our documentation says the platform supports automated testing and continuous integration. I wrote parts of that page. At the time I meant it: the sentence is about our own regression suite, and the customer reads it as a promise about theirs.
The uncomfortable part is not that we lack a sandbox. Most low-code platforms do, for the same economic reasons, and their customers have adapted with prefixes and test accounts and prayer. The uncomfortable part is that the absence is invisible in the demo. A demo never needs a second environment, because a demo has no history to protect. The cost arrives later, in an audit, in a message that cannot be recalled, in eleven rows that are true statements about a false event.
We sold the removal of the deployment step as a feature. We did not mention that the deployment step was also the thing that separated trying from doing.
Renata's eleven rows are still there. They are correct now, in a way — annotated, explained, accepted by the auditor with a finding attached.
The platform did exactly what she told it to. It recorded a batch release, because she asked for one, and it had no idea she was only asking.
Top comments (1)
The point that a realistic test environment means copying the business, not the config, matches what I've seen. The alternative to masked production copies that has worked for me is generating the business state from a declared schema with a fixed seed, so a tester can throw away the afternoon and get the exact same starting state back in seconds. It doesn't solve notifications or scheduled jobs, but it removes the pressure to copy real records at all. I build Misata, a groud truth artificial data generator for this, so I'm biased, but even a seeded script per tenant gets you most of the way.