DEV Community: Zerops Devs

"Read the failure. Ship the fix." Abdul Talha on building DeployDoctor on Zerops

Zerops Devs — Fri, 28 Aug 2026 05:00:00 +0000

A DevOps technical writer built a tool that reads your deployment logs and tells you what broke. The interesting decision is that it checks its own rules before it calls the AI.

Over the weekend of August 8 and 9, Zerops ran The Zerops Challenge in collaboration with WeMakeDevs. Forty eight hours, solo entries only, and one hard rule: the finished thing had to be a real application deployed on Zerops with a live URL, still running when the judges reached it. Not a local demo, not a screen recording.

Over 1,000 developers signed up, more than 10,000 build and deployment pipelines fired, and 250+ valid projects came out the other end. Seven of them won. One was DeployDoctor, by Abdul Talha.

Here are his answers, close to verbatim.

First, introduce yourself.

I am Abdul Talha, a DevOps technical writer focused on cloud infrastructure, application deployment, self-hosting, and open-source technologies.

I work with tools like Docker, Docker Compose, Linux, cloud platforms, reverse proxies, databases, and observability tools. When I write a technical guide, I prefer to deploy and test the application myself so I can document the actual process, issues, and solutions rather than just explain the theory.

I also follow a docs-as-code workflow using Git and Markdown. Along with technical writing, I build projects to learn and experiment with different DevOps and cloud technologies. DeployDoctor was one of those projects, which I built as part of the Zerops Challenge.

What did you build, and where did the idea come from?

I built DeployDoctor, a tool that helps developers troubleshoot deployment issues.

The idea came from my own experience. While deploying different applications, I faced many issues where finding the actual cause and solution took a lot of time. So I thought of building a tool that could identify frequently occurring deployment problems and provide their solutions in one place.

DeployDoctor analyzes deployment logs, identifies known error patterns, and provides the likely cause and recommended fix. For issues that the built-in analyzer cannot identify with enough confidence, it uses Gemini for further analysis.

The example in the screenshot is one every developer has met at least once: the app is connecting to PostgreSQL via localhost instead of the database service hostname.

Walk us through your architecture.

DeployDoctor has three main components: a React frontend, a Fastify API, and a PostgreSQL database.

The frontend is responsible for the user interface, while the API handles the deployment log analysis and application logic. PostgreSQL stores the incidents and analysis results.

For log analysis, I used two layers. First, the API checks the logs against known deployment error patterns. If it finds a match with enough confidence, it returns the result directly. If it cannot identify the issue confidently, it sends the logs to Gemini for further analysis.

On Zerops, I deployed the frontend and API as separate services. This kept the application components separated and also helped me understand how services are configured and connected on Zerops.

One part I found interesting was using the rule-based analyzer before Gemini. It makes the analysis more predictable for known issues and avoids using AI for every request.

What did Zerops get wrong most often, and how did you catch it?

Most of the issues I faced were related to deployment configuration rather than the application itself.

I had to make a few changes to the Zerops configuration, including the Prisma setup and the API URL used by the frontend. I caught these issues by checking the build and deployment logs and testing the application after each deployment.

Whenever something failed, I checked the logs, identified the configuration issue, made the change, and deployed it again.

What did you not understand about Zerops when you first opened it?

When I first opened Zerops, I wasn't immediately familiar with how the platform structured applications and services.

I was used to deploying applications with Docker Compose or directly on a VPS, so the Zerops service-based approach was different from what I was used to. I initially needed some time to understand how the frontend, API, database, environment variables, and deployment configuration fit together.

Once I understood that structure, the deployment process became much easier to follow.

Any advice for people who are just starting their deployment journey?

Start with a simple application and understand how it works before trying to deploy it.

Learn the basics of the application stack first, especially things like ports, environment variables, databases, and how different services communicate with each other. When something fails, don't immediately change multiple things at once. Check the build and application logs, identify the actual error, and then fix it step by step.

Also, don't be afraid of deployment errors. They are a good way to understand how applications actually work in production. The more applications you deploy and troubleshoot, the easier the process becomes.

What we are taking from this

Two things stand out, and neither is the AI.

The first is the design decision Abdul flags himself: the rule-based analyzer runs before Gemini, not after, and not instead. A known failure signature is matched deterministically and returned immediately. Only when confidence is too low does the log get handed to a model. That means the common cases are instant, free, and give the same answer every time, and the model is reserved for the cases that actually need judgement.

That is a better instinct than most AI tooling shows right now, and it is the same instinct behind good infrastructure. Reach for the deterministic thing first. Keep the expensive, non-repeatable thing for the problems that earn it. It is not a coincidence that this came from someone whose day job is deploying software carefully enough to write down exactly what happened.

The second is his answer about the first hour. He came from Docker Compose and from putting things on a VPS directly, and the thing that slowed him down was not a command or an error message. It was the mental model. How the frontend, the API, the database, the environment variables and the deployment config relate to each other on Zerops is genuinely a different shape from a compose file, and until that shape is in your head, nothing else lands.

We think about this a lot. Zerops asks you to think in services on a private network rather than in containers on a host, and that model is the reason the platform can do the things it does. It is also, reliably, the thing that costs people their first hour. Abdul's account is useful precisely because he is not complaining about it. He needed time, he got it, and then, in his words, the deployment process became much easier to follow.

The gap between those two states is the whole problem worth solving, and it belongs in onboarding rather than in a hackathon weekend.

Congratulations, Abdul. Well earned.

DeployDoctor

Live demo: https://frontend-2bdf.prg1.zerops.app/
Source: https://github.com/abdultalha0862/deploydoctor
Project page: https://appstage-290f.prg1.zerops.app/projects/deploydoctor-ai-powered-deployment-incident-diagnosis

The live demo ships with example logs you can click, including CrashLoopBackOff, Node OOM, DNS ENOTFOUND and a missing env var, so you can try it without having to break something first.

The Zerops Challenge

Hackathon page: https://www.wemakedevs.org/hackathons/zerops
Kickoff livestream with Kunal Kushwaha and Francesco Ciulla: https://www.youtube.com/live/1bpt0iuXuNM

Connect with Abdul Talha

X: https://x.com/AbdulTa62185243
LinkedIn: https://www.linkedin.com/in/abdul-talha/
GitHub: https://github.com/abdultalha0862

More winner interviews from The Zerops Challenge are on the way.

Wasp on Zerops: from recipe to a real challenge dashboard

Zerops Devs — Thu, 27 Aug 2026 10:24:18 +0000

How we shipped a full-stack Wasp app on Zerops in minutes — and used the same path to publish Octoverse-style stats for 270+ WeMakeDevs × Zerops Challenge submissions.

Wasp already gives you the full stack in one repo: React client, Node.js API, Prisma, and auth. Zerops already gives you managed PostgreSQL, static hosting, Node runtimes, and a deploy model that treats “client + API + DB” as ordinary services.

Put them together and you stop fighting glue. You write the app. A single zerops.yaml tells the platform how to build the SPA, bundle the API, wire DATABASE_URL, and run migrations on deploy. That pattern lives in the official Wasp Hello World recipe — and it is exactly what we used to ship zerops-zcp-challenge-summary.

Live showcase: https://appstage-290f.prg1.zerops.app

One-click recipe: Deploy Wasp on Zerops

Deploy on Zerops

Why Wasp and Zerops fit

Wasp concern	Zerops answer
React SPA	static service (Nginx) from wasp build + Vite output
Node API	nodejs@24 service serving the Wasp server bundle
PostgreSQL + Prisma	Managed postgresql@16 with ${db_*} → DATABASE_URL
Auth / cookies / CORS	Project-level APP_URL + API_URL mapped into Wasp env keys
Migrations	zsc execOnce on deploy so schema lands once per version

Wasp wants two public URLs (web + API) and a Postgres connection string. Zerops speaks that language natively: two services, one database, cross-service references in YAML — no hand-rolled Docker Compose, no “how do I expose port 3001?” side quest.

Start from the recipe (really)

The recipe is not a toy dump. It is a production-shaped topology:

Import — import.yaml creates app (static), api (Node), and db (PostgreSQL), plus project env for public URLs.
Build — zerops.yaml defines three setups from one repo: prod-client, prod-api, and dev.
Run — Client is static files; API starts the Wasp bundle on port 3001; init commands migrate (and optionally seed).

You can click Deploy on Zerops, get a running hello-world with demo login, then replace pages and entities with your product. That is the path we took.

What we built on top of it

zerops-zcp-challenge-summary is a public field report for the WeMakeDevs × Zerops Challenge — Octoverse-style, not a private admin tool. Build with ZCP.

It answers questions like:

How many teams submitted? How many shipped a live Zerops URL, a repo, a social post?
Which frameworks showed up most?
Which official Zerops recipes did teams map to?
Which Zerops services carried the load?
Who won — and can the community browse and rank projects?

Product surface

Route	Purpose
/	Overview KPIs, completeness, rankings, timeline
/projects	Browse and filter 270+ submissions
/projects/:slug	Project detail (repo, live, social, tags)
/ranked	Community ranking via email-gated likes
/winners	Headline, social, and credit winners

Stack stayed the recipe stack: Wasp ^0.25, React, Prisma, PostgreSQL on Zerops, client on static, API on Node 24 (Ubuntu) so Prisma’s native engine has OpenSSL without Alpine workarounds.

From hello-world to challenge summary

Roughly the loop we ran:

Adopt / deploy the Wasp recipe topology — static app, Node API, Postgres.
Keep zerops.yaml structure — build commands, Ubuntu for API, readiness check, URL mapping.
Replace the domain model — Submission + Vote, seed from challenge export JSON.
Ship product pages — overview charts, filters, detail, ranking, winners.
Promote through stages — appdev / apidev for iteration, appstage / apistage for the public link above.

We did not rewrite how Wasp deploys. We reused the recipe contract and spent the time on the product.

Try it yourself

Open the Wasp Hello World recipe.
Deploy the small-production environment (or start from the recipe app repo).
Confirm client + API + DB come up with the demo user.
Swap in your entities, pages, and seeds — leave the Zerops wiring alone until you need a second database or cache.

When you want a concrete example of “recipe → real app,” open the challenge summary:

zerops-zcp-challenge-summary on Zerops

270+ submissions, live rankings, winners — full-stack Wasp, managed Postgres, static + API on Zerops, no custom PaaS fairy dust.

Useful Links

Recipe: https://app.zerops.io/recipes/wasp-hello-world
Recipe app repo: https://github.com/zerops-recipe-apps/wasp-hello-world-app
Wasp: https://wasp.sh
Zerops: https://zerops.io
Showcase: https://appstage-290f.prg1.zerops.app

Zerops was the only platform where this project could exist as designed

Zerops Devs — Tue, 25 Aug 2026 06:45:45 +0000

Danish Ali Siddiqui built Doorbell, a webhook tunnel that keeps accepting requests while your laptop is shut. Along the way he hit four rough edges on Zerops, let's discuss these!

In three days the challenge brought Zerops over 1,000 new sign-ups, triggered more than 10,000 build and deployment pipelines, and produced 250+ valid projects. Two tracks each had a winner, and a $5,000 credits pool was split across the standout projects (overall 7 big winners). One of them was Doorbell, by Danish, and it is the entry that gave us the most uncomfortable and most useful feedback of the whole weekend. We are publishing his answers close to verbatim, including the parts that do not flatter us.

First, introduce yourself. I'm Danish, an AI/ML engineer working with a Riyadh-based company in Saudi Arabia. I build enterprise AI solutions, backend systems, workflow automations, and full-stack applications. I enjoy working on practical engineering problems where reliability matters as much as functionality.

What did you build, and where did the idea come from? I built Doorbell, a self-hosted webhook tunnel that keeps accepting requests even when your laptop is offline.

The idea came from a common frustration in webhook development. Services like Stripe or GitHub need a public endpoint, but when you close your laptop or lose connection, a normal tunnel returns an error and the request can be lost. A third-party tunnel provider that wanted to hold those payloads for you would be holding your signatures and tokens on its own infrastructure.

Doorbell solves this by keeping the database under your control. It writes each request to Postgres before responding with HTTP 202. When the local client reconnects, the queued requests are delivered in the same order they arrived.

Walk us through the architecture. The CLI maintains a single outbound TCP connection to the gateway's raw public port. Each incoming request becomes a yamux stream over that connection, and Go's httputil.ReverseProxy forwards the traffic through it. This supports chunked request bodies, keep-alive connections, and WebSocket upgrades without requiring a custom framing protocol.

The deployment contains three services defined in one YAML file: a Go gateway, Postgres for data that must survive restarts, such as reserved names and stored request bodies, and Valkey for temporary shared state, such as identifying which container currently owns a tunnel connection.

The mailbox is the most important part of the architecture. A request is stored before the sender receives a response, preventing data loss. An atomic lease prevents concurrent drains from delivering the same queued request twice. Sensitive signing headers are redacted before storage by matching the shape of the header name rather than naming each vendor, so the next provider is covered before anyone has heard of it. The trade-off is documented clearly: a queued webhook is delivered without its original signing header, so a signature check on a held request will fail.

What was the easiest part? The import YAML. One pasted file gave me a Go service, Postgres, and Valkey on a private network in ninety seconds. Service-to-service environment interpolation, ${db_connectionString}, meant I did not have to write any networking configuration.

And what did Zerops get wrong? How did you catch it? The difficult issues were not obvious failures. They were silent platform behaviours that initially looked like problems in my own code.

The one my entire project depends on is the raw public TCP port. It cannot be declared in the import YAML, so after the "one pasted file" setup, the port still has to be opened manually in the GUI. A shared IPv4 cannot carry one at all, and the API rejects it with publicIpTypeNotSupported. The options are IPv6 or a dedicated IPv4, and during the weekend I could not work out whether the dedicated address could be paid for from credit. I stayed on IPv6. The HTTP side of the gateway is unaffected, because that goes through the shared balancer: webhooks, the dashboard and the live URL all work fine over IPv4. What IPv6-only costs is the one step that shows the whole idea, which is opening a tunnel of your own with the CLI.

Two other issues cost me about an hour each. The load balancer defaults send_timeout to two seconds, so my SSE dashboard connection was terminated whenever the stream went quiet for that long, until I added a heartbeat frame. The balancer also replaces the response body for any 5xx error, which prevented Doorbell's own error message, nothing is listening on 127.0.0.1:3000, from reaching the user. What arrives instead is the balancer's own advice to check the port, minus the port it actually tried. I discovered that one by following my own README from the perspective of a new user.

What helped most was replacing assumptions with direct verification. I wrote small scripts to test the live API and kept them in the repository under tools/verify.

Did the docs help? They helped on the happy path. What cost me time was mostly documented, just not where I ran into it. The shared IPv4 restriction is in the Public Access reference, and the send_timeout default is a row in the balancer configuration table. Neither shows up in the port routing screen or in the import file, which is where you are standing when it matters. The 5xx response-body replacement I could not find written down anywhere. So I verified behaviour directly against the live API using throwaway scripts. They are still in my repository under tools/verify, which says a lot.

What did you not understand when you first opened Zerops? I initially assumed the *.zerops.app subdomain was the public entry point for every service in the project, including raw TCP ports. It is actually a shared HTTP balancer, while a raw port is exposed through the project's own public IP. I spent a while testing the wrong address and assuming the fault was in my implementation.

Once I understood that distinction, several issues that had appeared unrelated became much easier to diagnose.

What do you wish existed?

Public port routing in the import YAML. The docs say routing capabilities are on the roadmap. Until they land, the "one pasted file" experience stops one step short for any project that needs a public raw port.
A plain statement of whether the dedicated IPv4 add-on can be paid for from credit. I lost time trying to determine that during the weekend and never settled it.
The raw TCP facts gathered where you meet them: that a shared IPv4 will not carry a raw port, that the *.zerops.app subdomain will not either, and what IPv6-only access means for the people using your service. It exists in the networking reference. It does not exist in the port routing screen.
send_timeout explained in a streaming context rather than only as a row in a configuration table. Two seconds is the default, it is raisable, and nothing connects those facts to SSE or WebSocket work.

Did you use ZCP? I did not, so I cannot fairly judge it. I was building a Go service that maintains a long-lived TCP socket, so I needed a tight local development loop with the race detector and integration tests against real Postgres and Valkey instances. Deployment through zerops-import.yml and zcli push was painless.

Any advice for people just starting out their deployment journey? Verify platform behaviour before making it part of your architecture. If a capability is essential to the design, test it directly instead of relying on assumptions.

Deploy on the first day, not the final night. Infrastructure constraints often affect the design itself, so discovering them early gives you time to adapt properly.

Finally, follow your own README exactly as a new user would. Running every command in order helped me find two broken commands and one promise the deployed application did not keep.

What we are taking from this

Danish also wrote something we want to quote, because it is the clearest statement of the thing we are usually trying to explain at much greater length:

I chose Zerops because it was the only platform where this project could exist as designed: a raw public TCP port, a continuously running process, and private managed Postgres and Valkey together. Serverless was structurally unsuitable. That combination, deployable from one file, is a strong differentiator.

That is the argument. A function stops when it answers, and nothing is left holding the socket. Doorbell needs something holding the socket, and that is not a preference, it is the feature.

Which makes the rest of his feedback sharper rather than softer. The ninety second path from one pasted file to three networked services is exactly the experience we want. The problem is what happens the moment a project needs a raw public port: the config file stops being sufficient, you drop into the GUI, and then you meet an IP type constraint that is written down in the networking reference and nowhere near the screen where you hit it. The gap between "this platform is the only one that can express my design" and "I cannot find out how to expose it" is the worst possible place to lose someone, and it is entirely our fault, not theirs.

His four asks are all reasonable and all specific. Two of them are placement problems rather than missing pages: the raw TCP and IP type facts and the send_timeout default are both documented, and both are absent from the moment a developer needs them. The send_timeout default in particular silently breaks a common pattern. It is two seconds, it is raisable to as much as five minutes in the balancer configuration, and a developer running SSE should not have to reverse engineer "the platform terminated your stream" from connection behaviour before finding that out. The 5xx body replacement is the one we genuinely never wrote down, and Danish is right that swallowing an application's own error message is worse than saying nothing.

And the IPv6 detail deserves to be said plainly rather than buried. The reachable, working parts of Doorbell were reachable over IPv4 the whole time, because the HTTP balancer handles them. The one action that demonstrates the entire premise, opening your own tunnel, needed IPv6, because a raw port cannot go through that balancer and a shared IPv4 will not carry one.

Congratulations, Danish. Well earned!

Doorbell

Live gateway: https://gw-2ad0-3000.prg1.zerops.app
Source: https://github.com/BigAchiever/doorbell
Four minute demo: https://www.youtube.com/watch?v=KIYEptODPeo

The README has a thirty second live verification you can run without installing or signing up for anything. Send a webhook to an offline tunnel, get 202 instead of 502, and watch the counter go up.

The Zerops Challenge

Hackathon page: https://www.wemakedevs.org/hackathons/zerops
Kickoff livestream with Kunal Kushwaha and Francesco Ciulla: https://www.youtube.com/live/1bpt0iuXuNM

Connect with Danish

X: https://x.com/big_achieverx
GitHub: https://github.com/BigAchiever
LinkedIn: https://www.linkedin.com/in/big-achiever

More winner interviews from The Zerops Challenge are on the way.

Hardik Khanduja built ZeroArch in 48 hours in Zerops: a canvas you draw your infrastructure on, and an AI that audits it before it writes the YAML

Zerops Devs — Mon, 24 Aug 2026 07:29:16 +0000

"I'm a visual thinker, and a lot of developers are too" - Hardik Khanduja

Over the weekend of August 8 and 9, Zerops ran The Zerops Challenge in collaboration with WeMakeDevs. The format was deliberately unforgiving. Forty-eight hours, solo only, no planning phase and no slide decks. The only hard requirement was that the finished thing had to be a real deployed application on Zerops with a live URL, still running when the judges got to it. Not a local demo, not a screen recording.

Over 1,000 developers signed up, more than 10,000 build and deployment pipelines fired, and 250+ valid projects came out the other end. Seven of them won. One of the winners was Hardik Khanduja, who took a share of the $5,000 credits pool for ZeroArch, a visual infrastructure designer that turns a drag-and-drop diagram into a production ready zerops.yml.

We sent him some questions. His answers turned out to be one of the most useful pieces of platform feedback we got out of the whole weekend, so we are publishing them close to verbatim.

First, introduce yourself. Hi, I am Hardik Khanduja. I am a full-stack developer who loves building in public and experimenting with new tech. I've always been interested in finding ways to make developer workflows smoother, especially when it comes to the gap between writing code and actually managing the infrastructure it runs on.

What did you build, and where did the idea come from? I built ZeroArch. It's an interactive, drag-and-drop canvas where you can visually map out your cloud architecture, like connecting your frontend to an API and a database. Once you design it, Google Gemini AI audits the setup for security issues and generates a production-ready zerops.yml config file for you.

The idea came from a personal frustration: writing YAML files by hand is annoying and error-prone. One wrong space or an exposed port, and your app breaks or becomes vulnerable. I'm a visual thinker, and I realized a lot of developers are too. I wanted a tool where you could visually sketch what you want, and let the code write itself.

The canvas. Components come out of the left rail, and connections between them are the architecture.

Walk us through the architecture. The stack is Next.js 15 using the App Router, React Flow for the interactive canvas, PostgreSQL for storing saved blueprints, and Prisma as the ORM.

The most interesting part is how the AI compiler works. React Flow stores the canvas state as a JSON of nodes and connections. When you hit Generate, our server converts that graph into an AST, an abstract syntax tree, and feeds it to Gemini. The prompt is designed to act like a strict DevOps engineer. It doesn't just write the YAML, it actively warns you if you do something insecure, like trying to connect a frontend directly to a database without an API layer in between.

The audit pass on a heavier design. Note what it is actually saying: the ingestion layer is correctly decoupled through RabbitMQ, a connection to the results database is missing, and only the web client and upload API should have public routing enabled. The queue, the worker and the database stay inside the private project network.

That last screenshot is worth sitting with for a second, because it is the argument ZeroArch is quietly making. The interesting part of an architecture is not the list of services. It is which of them can be reached from the outside, and which ones can only be reached by their neighbours. Drawn on a canvas, that becomes obvious. Written by hand in YAML at two in the morning, it does not.

What did the AI get wrong most often, and how did you catch it? And on the platform side, what fought you hardest? When Gemini was generating the zerops.yml files, it would occasionally hallucinate Alpine-specific packages or get the YAML indentation wrong. To catch this, I had to refine the system prompt with strict schema constraints, forcing the AI to output structured data that we could parse safely.

On the platform deployment side, the trickiest thing was Alpine's lack of legacy OpenSSL 1.1 libraries, which Prisma was expecting. Prisma would crash on boot because it couldn't find the shared library. I had to explicitly configure Prisma's binaryTargets to support OpenSSL 3.0, and add a prepareCommands step in zerops.yml to install openssl via apk using sudo, since the setup runs as a non-root user.

What did you not understand about Zerops when you first opened it? The database connection binding. Coming from platforms that auto-inject the database URL directly into the app, I didn't realize at first that Zerops gives you the base connection string, pointing to /postgres, but you have to manually suffix it with /db, or whatever your specific database name is, in the secret variables to let Prisma write to it. Once I realized PostgreSQL 15 has strict permissions on the default system database, it clicked, but it definitely had me scratching my head for a bit.

Any advice for people just starting out their deployment journey? Don't ignore the logs, and pay attention to your environment's OS. A lot of developers write code on macOS or Windows and forget that their app is going to run in a lightweight Linux container, like Alpine, in production. If you rely on database engines or native node modules, test your container configurations early. And most importantly, build in public. The feedback and support from the community are what keep you going when deployments get frustrating.

What we took from this

The Alpine and OpenSSL one is the flip side of running real Linux containers rather than an abstracted app runtime. You get prepareCommands, apk, and control over the image, which is exactly what let him fix it in one step. You also get to meet the problem in the first place. That trade is one we would make again, but it is the sort of thing worth being upfront about.

And his advice is the correct advice. The gap between a laptop and a production container is where most first deployments die, and it is why we would rather you meet that container early than have it hidden from you until something breaks.

Congratulations, Hardik. Well earned.

ZeroArch

Live app: https://app-1eb-3000.ny1.zerops.app/
Source: https://github.com/Hardikkhanduja/ZeroArch
Hardik's original build post: https://www.linkedin.com/posts/hardik-khanduja-60a9a6287_buildinpublic-zeropshackathon-wemakedevs-activity-7491519883357286400-JiO2

The Zerops Challenge

Hackathon page: https://www.wemakedevs.org/hackathons/zerops
Kickoff livestream with Kunal Kushwaha and Francesco Ciulla: https://www.youtube.com/live/1bpt0iuXuNM
ZCP quickstart, the route most builders took from prompt to live URL: https://docs.zerops.io/zcp/quickstart

Connect with Hardik

X: https://x.com/HardikKhanduja
GitHub: https://github.com/Hardikkhanduja
LinkedIn: https://www.linkedin.com/in/hardik-khanduja-60a9a6287/

More winner interviews from The Zerops Challenge are on the way.

Our journey integrating Qdrant in Zerops

Zerops Devs — Thu, 22 Aug 2024 21:09:13 +0000

Vector databases are revolutionizing how data is managed and stored for AI applications. At Zerops, we recognized the growing importance of vector databases, leading us to integrate Qdrant, one of the most popular options available. While it might seem straightforward to spin up a Qdrant instance using a Docker container, the reality of managing a production-ready vector database is far more complex. In this article, we'll explore the intricacies of Qdrant implementation and how we've addressed the challenges at Zerops.

Understanding Data in Qdrant

In Qdrant, data is organized into collections, which are essentially named sets of vectors. Each collection contains vectors that share the same dimensionality and are compared using a specific metric. Collections in Qdrant are further divided into shards—independent stores of vectors that can handle all the operations provided by collections.

This architecture allows for distributing the workload across multiple Qdrant nodes in a cluster, enhancing performance and reliability. To ensure high availability (HA), shards can be replicated across nodes. Understanding this structure is crucial for grasping the complexities of deployment and management.

Deployment Options: Balancing Simplicity and Reliability

At Zerops, we offer support for both HA and non-HA modes of Qdrant, giving you the flexibility to choose the configuration that best suits your needs.

Non-HA Mode: Simplicity for Non-Production Projects

The Non-HA mode is perfect for non-production projects where data persistence isn't critical. This setup involves a single Qdrant node, making it easy to install and manage. In Zerops, deploying Qdrant in Non-HA mode is straightforward:

We run the Qdrant binary in a Ubuntu container.
Port 6333 is opened for communication.

Even if a data outage occurs, our backup solution ensures minimal disruption.

HA Cluster: Reliability for Production Environments

For production environments where reliability and data availability are crucial, the HA cluster mode is essential. Setting up an HA cluster can be complex, but we've streamlined the process in Zerops:

Cluster Configuration: We enable cluster mode by adding enabled: true to the configuration file and configure peer-to-peer communication on port 6335:

   cluster:
     enabled: true
     p2p:
       port: 6335

Node Setup: Building a cluster requires careful configuration of each node:
- The first node runs with a --uri <uri> flag to let other peers know how to be reached.
- We make the address of the first peer node1.db.<service_name>.zerops available in local DNS.
- All other peers start with the --bootstrap <cluster_uri> flag to locate the rest of the cluster.
High Availability: Qdrant uses the RAFT protocol, which requires more than 50% of the nodes to be functional. We automatically set up 3 nodes per cluster to meet this requirement.
Automatic Replication: By default, we automatically create replicas of any collection across all nodes. This safeguards against data loss if a node fails and serves as a safety net for incorrect replication_factor configurations. This approach ensures data safety even if the replication_factor is incorrectly configured by the user. If desired, this feature can be disabled by setting the automaticClusterReplication parameter to false.
Failure Recovery: If a node fails, we automatically start a new node, connect it to the cluster, and create replicas of each collection and shard on the new node.
Node Cleanup: We streamline this process by utilizing the Qdrant API to identify the current cluster Leader node, which then efficiently handles the removal of failed nodes.

Overcoming Technical Challenges

During the integration of Qdrant with Zerops, we encountered and solved several complex issues. Fortunately, Qdrant offers robust tools to monitor and manage clusters through its well-documented REST and gRPC APIs.

One issue we faced was that when a new node was added to the cluster, it appeared fully operational but couldn't receive new replicas. This is often due to ongoing data transfers between nodes, which can take considerable time. To address this, Qdrant provides the POST /cluster/recover endpoint, which can be triggered on any non-leader node. This endpoint sends a request to the current leader to create a snapshot. The leader then sends this snapshot back to the requesting node for application. This snapshot captures the cluster's agreed-upon state at a specific point in time, allowing the cluster to recover and synchronize.

Data Backup: Safeguarding Your Qdrant Data

At Zerops, we prioritize the safety and security of your data by providing:

Daily, automatic backups at no extra cost
Backups in the form of encrypted disk snapshots for each collection
Secure upload to our S3-compatible backup storage
Retention of up to 100 backups per stack for a maximum of one month
Current maximum storage size per project of 25 GiB

You also have the flexibility to choose between different backup options, including one-time backups, regular backups with a customizable frequency, or even disabling backups entirely.

Our backup retention policy is designed to provide comprehensive coverage. For example, if you back up every hour, you'll have up to 4 days of backups available. This ensures that you have access to recent versions of your data in case of any issues.

The Value of Managed Vector Databases

While it's possible to set up and manage Qdrant yourself, doing so properly requires significant expertise and resources. Our managed Qdrant solution allows you to leverage the power of vector databases without the operational complexities. You can focus on developing AI features, while we ensure your vector database is running optimally, is highly available, and is protected against data loss.

Ready to harness the power of vector databases without the operational headaches? Give Zerops a try