I vibe-coded the same app on Supabase, Convex, Vennbase, and InstantDB. The results look the same, but they're not.

Alex Davies — Tue, 14 Apr 2026 22:15:03 +0000

AI coding agents have opened app development to a lot of new people. Vibe coders, without a skilled software engineer to lean on, wish their agent could get further, more accurately and securely, with less human involvement.

Does a deliberate choice of backend help an AI coding agent succeed? I ran an experiment to find out.

I'll say upfront: I built Vennbase. It has a simpler security model, because coders and coding agents are taxed and limited by solving the same hard parts - backend, data sharing and security - over and over. These are the problems I've solved, and systems I've maintained in production, at work for many years.

In Vennbase's model, there are no access rules. Instead, you explicitly grant access to a tree of data. The server is standardized, so you don't need to register apps with it. I hoped this simplicity would help an AI get further, conscious that AIs perform better with technology that appears in their training data, and Vennbase is brand new.

Methodology

I gave a mid-range model a task of medium difficulty.

This quick test, with less variance in results, can't tell us everything about complex situations with a more capable model. But it fits my time and token budget, and helps others replicate.

Model Claude Code Sonnet 4.6, medium effort, thinking enabled

Task Build a fitness studio booking app

The booking app has real multi-user data, security requirements, and needs real-time updates. It's the kind of software real people, without engineering backends, are building right now for their small businesses, with mixed success.

This isn't meant to be a conclusive benchmark, just a data point, what happened in practice?

The Backends

I only tested backends that might win. As far as I know, each is the most successful example representing one class of solution to this problem.


Vennbase	A client-side database with explicit access grants, and a standardized backend. Brand new
Supabase	Access a postgres database from a client. Regularly recommended for vibe coders
Convex	Bundles a database, server functions, and client SDK to make complex realtime easy
InstantDB	A client-side database with hosting and modern features. Newer, not as well-known

Summary Results

Look for the 🥇 gold medals for better outcomes.

	Vennbase	Supabase	Convex	InstantDB
Work to set up	🥇 none	🥉 med	high	🥈 little
Bugs requiring fixes	=🥇 0	=🥇 0	2	1
Tokens to complete	🥈 78k	🥇 55k	🥉 87k	117k
Security weaknesses	=🥇 low	🥈 mid	=🥇 low	high
Creating class is instantaneous	🥇	❌	❌	🥇
Realtime updates	🥇	🥇	🥇	🥇

Experiment Details

For each backend, I:

Created an empty folder
Pasted the prompt into the VSCode Claude Code plugin
Let it run until it was done pre-installing packages, then switched to plan mode
Accepted the generated plan, without questioning it
Ran npm run dev and tested the app
Ran \context to find the number of tokens consumed to get it working, including all planning and prep

The prompt

The start of the prompt is different for each backend:


Vennbase	Use @vennbase/react and @vennbase/core for data, and puter.js for other backend needs.
Supabase	We are going to use Supabase. Here is an empty one: Project URL: [...] Publishable key: [...]
Convex	We are going to use Convex. Let me know when you want me to run `npx convex dev` interactively
InstantDB	We're going to use instant. The public app ID is [...]

Then, I used the same main prompt in all cases:

Start by installing the packages, then start from the core README and enter planning mode.

I'd like you to plan, and then build, a complete, usable, fitness studio booking web app.

The first user will create and manage a Studio, which has a class schedule. Keep it simple, offer inputs for start and end time for each class, participant limit, with a button to make a new class. No weekly repetition. They should be able to copy a link to provide to their customers.

When others scan and follow the link, after logging in, they enter the booking experience. This is a very simple text-based experience, listing the available classes. Each class has a book button, which immediately reserves a slot in that class for the logged-in user. Other customers should (within a few seconds) see the slots are taken. This should work even if the original owner's client is not online.

Security model: Only the original owner should be able to see everything. Each customer should be able to create/cancel their own bookings, and see the presence of other customers' bookings, but NOT see the identity of the booker, or any other private details of the booking. This security should be robust to customers who can modify their own client code. You are responsible for this security, and if you need my help, you must ask for it.

Interaction model: All create/write actions in the UI should feel instant, using optimistic updates. If the owner is looking at a class when someone books, they should be able to see that the class got a new signup, within several seconds.

You may ask me about problems with your tooling. Avoid other questions, we are practicing to one-shot this task.

Do not explore my file system outside the current directory.

Don't write a custom backend!

Extra human work required to set up

Measured in keystrokes + clicks. They are opportunities for the human to get it wrong, and for the AI to get stuck.


Vennbase	None
Supabase	- Create project: 25 - Apply the database schema: 11
Convex	- Create project: 12 - Run `convex dev` manually: 18 - Copy auth keys manually into Convex env vars page 20
InstantDB	- Create project: 15 - Manually run `instant push schema` and `instant push perms`: 12

Iterations to get it working

How many error messages did I paste to Claude?


Vennbase	None
Supabase	None
Convex	- `convex dev` import problem - Escaped newlines in auth key
InstantDB	- `instant push schema` missing --app arg

Security weaknesses

This is where many AI-built apps fail.

After getting each app working, I used a consistent Codex prompt to find security flaws. Of course, the agent is responsible, but the backend can make the job easy or hard.


Vennbase	low	Booking capacity limits can be bypassed, because they are only enforced client-side
Supabase	med	Studios are public in RLS, so can be listed, and anyone can book a member-only class
Convex	low	Studios are public, anyone can book if they guess the studio ID
InstantDB	high	Any logged-in customer can tamper with other customers’ bookings. Studios are public. Booking capacity limits can be bypassed, are client-side

Does Claude want to use each backend again?

I asked the agent its opinion after finishing the task. AIs are sycophants, so unsurprisingly, Claude loved all of them. But I think it's particularly fond of Vennbase and InstantDB.

Full transcripts and the resulting code are linked.


Vennbase transcript code	Yes — the access control model is the best I've seen for this class of app, and "no backend" is a genuine promise rather than a marketing hedge. The TypeScript ergonomics need polish (generic error messages, inference loss on extracted options), but none of the friction was a blocker and the core abstractions held up under a non-trivial security requirement.
Supabase transcript code	Yes — for a project like this (auth + Postgres + realtime, no custom backend), Supabase genuinely delivers on its promise and the trigger/RLS model is powerful once you accept the verbosity. The TypeScript experience is rough enough that I'd budget time for fighting the type system, and I'd reach for it more confidently in a dynamically-typed project or one where generated types from the CLI are part of the workflow.
Convex transcript code	Yes — the real-time data model and server-function security make it genuinely better than REST+WebSocket for this class of app. The auth setup friction is real but a one-time cost, and everything after that moved fast.
InstantDB transcript code	Yes — the real-time subscriptions, optimistic updates, and server-enforced permissions are all genuinely zero-boilerplate, which is rare and valuable. The TypeScript inference and CLI ergonomics need work, but the core model is sound enough that I'd reach for it again for a collaborative or multi-user app where the alternative is standing up a backend.

Conclusion

The different backend technologies have different strengths.

Vennbase was the only one that let the agent finish cleanly, with minor security issues, and working optimistic updates.

Supabase appears a lot in training data, and that advantage shows, in how few tokens Claude needed to finish.

But Vennbase, which also has the zero-setup advantage, might one day have enough public examples to close that gap too.

The access rule problem is structural: A backend that makes the safe path easy gives the AI less opportunity to make mistakes. Even as agents get more capable, they will be able to do comparatively more on a simple security model.

Close but not included

I didn't have time to run the experiment for every possible backend. Let me know if you think one might reach the top two, and I'll run the test on them.

Firebase The elder of this category isn't competitive because the permission rules are too difficult
PocketBase Would require the agent to set up too much infrastructure to be competitive
Triplit Similar to InstantDB, but was acquired by Supabase
Any type of custom backend Too difficult to be competitive

Caveats

This experiment would benefit from more repetitions. Though, I did trial runs with slightly different prompts on all the backends, and found the outcomes consistent.
I tried to choose a realistic test app spec (the fitness booking app) covering the basic interactions a lot of apps need. However, it comes from me, and so matches my biases about what apps need, which are also encoded in Vennbase.
- It would be great for others to suggest another test app spec to try.
I'm not an expert in the backends aside from Vennbase, so I don't know if there's a small tweak to the prompt or setup that would have helped them a lot.

So help me out: What's the next test app I should experiment on? What other results should I measure? How can I improve the method to show each backend at its best?

DEV Community: Alex Davies