DEV Community: Aldorax

The Trader's Paradox: Why Being Good at Crypto Means You Can't Prove It

Aldorax — Tue, 10 Mar 2026 17:25:24 +0000

There's an ugly secret hiding beneath every profitable trader's dashboard.

The moment you prove you're good at this... you lose the very thing that makes you good.

Let me walk through the logic, because it's brutal.

To prove you're consistently profitable, you have to show everything. Your entries. Your exits. How you sized each position. When you got in. When you got out. The exact strategy that's been printing money.

And those details? That's exactly what creates your edge in the first place.

So here's the trap most traders find themselves in:

Reveal your strategy and lose your advantage. Or keep it private and stay unverified.

Meanwhile, the people trying to follow traders face their own nightmare.

They see screenshots of massive wins. Leaderboards showing insane returns. Dashboard after dashboard claiming 10x performance.

But none of it answers the one question that actually matters:

Can I trust any of this?

Usually? No.

Performance data gets cherry-picked. Losers conveniently disappear from screenshots. Returns get fabricated. In an industry built on "transparency," the signals people actually use to find good traders are shockingly opaque.

This is where Obscura comes in.

What If You Could Prove You're Good Without Revealing Anything?

That's the question the team behind Obscura asked themselves. And instead of trying to force traders to choose between privacy and credibility, they built something that lets you have both.

Here's how it works in plain English:

A trader can cryptographically prove their performance metrics — total profit, win rate, how many trades they've taken — without revealing a single trade.

Followers can independently verify those claims.

The shift is subtle but massive.

We move from "trust me, bro" to "here's the proof, verify it yourself."

The Missing Piece DeFi Never Built

Over the last few years, crypto has rebuilt almost everything from traditional finance.

We've got decentralized exchanges. Lending platforms. Derivatives. Automated market makers that run 24/7 without banks.

But one critical thing never got built:

Reputation.

Right now, every participant in DeFi is treated as equally risky. The guy who's been trading profitably for three years gets the same treatment as someone opening their first wallet today.

Protocols have no way to evaluate trading history. So they protect themselves with blunt instruments — massive over-collateralization, conservative risk models, one-size-fits-all treatment.

The consequences hit everyone:

Skilled traders can't prove their track record without exposing their strategies
Followers can't tell genuine performance from cherry-picked screenshots
Protocols can't price risk accurately or allocate capital intelligently

Obscura exists to fix this by building something that should have existed years ago: a privacy-preserving reputation layer for trading.

How "Trade With Proof" Actually Works

That phrase — Trade With Proof, Not Trust — sounds like marketing. But it's weirdly literal.

Instead of asking you to believe a trader's claims, Obscura generates cryptographic proofs that those claims are mathematically correct.

A trader can prove things like:

Total profit and loss over any period
Win rate percentage
How many trades they've taken
Performance on specific assets
Results within any time window (30 days, 90 days, since they started)

These stats get computed from actual trading activity and encoded into something called a zero-knowledge proof.

The proof confirms: "Yes, these metrics were calculated correctly from real trades."

But the trades themselves? Completely hidden.

Once generated, that proof gets submitted to smart contracts where anyone can verify it independently.

The result? Verified outcomes with private strategies.

Let Me Paint You a Picture of How This Plays Out

Imagine you're a trader who's been consistently profitable for two years. You've found a strategy that works. You've refined it. It's your edge.

Under the old model, if you wanted to attract followers or get better terms from protocols, you'd have to show your work. Every trade. Every entry. Every exit.

Your edge evaporates overnight.

Under Obscura's model, you generate a proof that says: "Over two years, I've taken 847 trades. My total return is 143%. My win rate is 68%."

Followers can verify that proof instantly. They know you're legit.

But they have no idea which trades you took. Which assets. At what times. With what size.

Your edge stays yours.

The Tech Stack That Makes This Possible

I'm going to get slightly technical here, but I promise to keep it human.

Obscura combines three technologies to pull this off:

1. Zero-knowledge proofs — These allow someone to prove a statement is true without revealing the information behind it. Think of it like proving you're over 21 without showing your driver's license with your exact birthdate and address.

2. Decentralized oracle networks — These bring trade data in from exchanges in a way that can't be manipulated. Multiple independent nodes verify each trade. They have to agree before anything gets accepted.

3. Confidential computing — This handles the sensitive stuff like API credentials and private trade data securely.

The Hardware Problem That Needed Solving

Here's something interesting that happened during development.

Earlier versions of Obscura used something called Trusted Execution Environments (TEEs). These are hardware-enforced secure areas inside computer processors. Think of them as vaults inside the chip itself.

They work. They're used everywhere.

But they come with trade-offs.

You have to trust the hardware manufacturer. You have to trust the cloud provider hosting that hardware. And specialized infrastructure gets expensive fast.

For a protocol trying to build verifiable reputation across decentralized markets, relying on centralized hardware felt wrong.

So the team shifted direction.

Enter Nillion: Private Computation Without Hardware Vaults

The newer architecture integrates infrastructure from something called Nillion, which handles private data differently.

Two pieces matter here.

First: Secret-shared storage.

Instead of storing sensitive data in one encrypted location, it gets split into multiple fragments called "shares."

Each share individually reveals nothing. It's like giving ten people one piece of a jigsaw puzzle each — no single piece shows the full picture.

Only when enough shares come together (a "threshold") can the original information be reconstructed.

For Obscura, this means:

API credentials get split into fragments
Each fragment lives on a different node
No single node ever holds the complete secret

If someone compromised one node? They'd have a meaningless fragment. Not enough to reconstruct anything useful.

Second: Multi-party computation.

Protecting data while it's stored is one thing. But the system also needs to use that data — signing instructions, verifying credentials, performing calculations.

Multi-party computation lets multiple nodes jointly compute something while each holds only a fragment of the input.

The inputs stay private throughout. The secrets never get reconstructed in one place. But the computation still happens correctly.

Why Zero-Knowledge Proofs Still Run the Show

Here's the way to think about how these pieces fit together:

Nillion's infrastructure protects private inputs.

Zero-knowledge proofs verify correct public outputs.

Confidential computing keeps secrets secret. Zero-knowledge proofs prove that the claims coming out of that system are mathematically correct.

The performance metrics get calculated using circuits written in a language called Noir. These circuits process normalized trade data and produce cryptographic proofs of correctness.

Multiple proofs can be combined using something called recursive proof composition — imagine compressing a library's worth of verification into a single page.

That compressed proof gets submitted to an on-chain verifier where anyone can check it with minimal computational cost.

What This Actually Means for Traders

Let me translate all the technical stuff into practical reality.

If you're a trader building a public track record, here's what changes:

Your strategy stays private. Nobody sees your timing, execution details, or positions.

Your credibility becomes verifiable. Performance claims come with cryptographic receipts, not just dashboard screenshots.

Your reputation travels with you. Activity across multiple exchanges can feed into one reputation profile. You're not starting from zero every time you try a new platform.

For the first time, you can prove long-term consistency without revealing the mechanisms that generate your edge.

What This Means for Followers

If you're someone trying to find traders worth following, the shift is equally significant.

Instead of squinting at screenshots wondering if they're real, you can verify that reported statistics actually came from authentic trading activity.

You can see risk signals alongside returns — drawdown, trade frequency, consistency metrics.

The goal isn't just highlighting high performers. It's providing verifiable signals that help you make better decisions.

This Goes Way Beyond Trading

Here's where it gets interesting.

Obscura starts with trading reputation. But a portable, verifiable reputation layer could eventually change how DeFi works at a fundamental level.

Imagine:

Lending protocols evaluating borrower credibility based on verifiable history rather than just collateral
Insurance markets pricing risk based on actual behavior
Governance systems weighting participation by expertise rather than token holdings alone

Reputation becomes something you build over time and carry with you. Not a platform-specific score. Actual infrastructure that travels wherever you go.

The Long Game

The objective is straightforward, even if the technology is complex:

Build a reputation layer that's private by default and verifiable by anyone.

There's still work to do. Making proof generation faster. Normalizing data consistently across different exchanges. Figuring out exactly how reputation should be modeled.

But the principle is solid.

In decentralized markets, trust shouldn't depend on screenshots, platform dashboards, or hoping some intermediary is honest.

It should depend on verifiable computation.

Trade With Proof, Not Trust

If systems like Obscura succeed, something fundamental shifts in crypto trading.

Traders no longer have to reveal everything to prove anything.

They prove what matters — and keep the rest private.

In that world, reputation becomes something that's earned, verified, and portable.

And trust moves from belief to mathematics.

No screenshots required.

Follow Obscura's development here:

Website: https://portal.obscura.finance
Twitter/X: https://x.com/UseObscura

Building QuantaCode: Our Bolt Hackathon Saga (and the AI Brainrot is Real!)

Aldorax — Tue, 01 Jul 2025 13:48:25 +0000

Alright, fellow code warriors and digital adventurers! Buckle up, because I'm about to take you on a wild ride – our journey building QuantaCode for the Bolt Hackathon. Was it smooth sailing? Absolutely not. Was it epic? You bet your git commit -m "final final final" it was. And yes, there might be some slight exaggerations, but I promise, the core truth is always just a semicolon away.

🚀 The Spark of Genius (or, How We Avoided Total Brain Drain)

The idea for QuantaCode hit us like a perfectly formatted, lint-free commit: developers spend half their lives wrestling with messy code, debugging nightmares, and trying to figure out if their codebase is more "pristine garden" or "dumpster fire after a coding convention argument." We thought, "What if we could get quantum-level precision in code analysis?" (Okay, maybe the "quantum-level" part was a slight exaggeration, but we definitely aimed for deep insights!) We dreamed of an AI that didn't just tell you what was wrong, but why, and even talked to you about it. Voice-powered code analysis? Madness! But the good kind of madness. This vision pushed us to explore the bleeding edge of AI integration in developer tooling, aiming for something truly transformative.

🎯 What We Cooked Up (Spoiler: It's Pretty Cool)

So, what exactly is this mythical beast, QuantaCode? It's an AI-powered codebase health management platform. Think of it as your code's personal, highly intelligent, and slightly judgmental doctor, always ready to give you a check-up and prescribe improvements.

Core Features (The Good Stuff):

Quantum AI Analysis: We fed OpenAI GPT-4 all the code it could eat, and it spat out insights on quality, security, and maintainability. It's like having a super-senior architect review your entire repo in seconds. (Okay, maybe not seconds, but definitely faster than a human, and without the passive-aggressive comments!) Our AI delves into code patterns, potential anti-patterns, and even documentation gaps to provide a holistic view.

Voice-Powered Insights: This is where the magic (and a bit of ElevenLabs wizardry) happens. You can literally talk to your codebase and get audio summaries. Imagine asking, "Hey QuantaCode, what's the biggest issue in my auth module?" and getting a coherent, spoken answer. Free users get a charmingly robotic voice; premium users get a voice so natural, you'll wonder if your code is flirting with you. This feature was designed to make complex analysis accessible and interactive.

Real-time Monitoring: Because nobody likes surprises, especially when it's a critical bug lurking in production. QuantaCode keeps a vigilant eye on your repository, alerting you to changes in health metrics as they happen.
Premium Subscriptions: We gotta eat, right? Stripe handles the billing, even with fancy promo codes. This allows us to offer advanced features and sustain development.

Admin Analytics: For us to stalk... I mean, understand user behavior and platform performance. This dashboard gives us insights into feature adoption, common issues, and overall system health, enabling continuous improvement.
Responsive Design: Because even your phone deserves to see beautiful code analysis. We meticulously crafted the UI to ensure a seamless experience across desktops, tablets, and mobile devices.

Our Tech Stack (The Guts & Glory):

Frontend: Next.js 13, React, TypeScript, Tailwind CSS (the usual suspects, dressed to impress, and chosen for their robust ecosystem and developer experience).
Backend: Next.js API Routes, Prisma ORM (our database savior, after some initial wrestling).
Database: PostgreSQL (production-ready!), SQLite (for quick dev cycles, though with some Bolt-specific quirks).
AI: OpenAI GPT-4 (the brains behind the operation), ElevenLabs (the voice of reason, and sometimes, humor).
Payments: Stripe (our financial guardian, diligently handling subscriptions).
Deployment: Netlify (because who has time for complex deploys when you're in a hackathon?).
Authentication: NextAuth.js (keeping things secure and user sessions managed).

🛠️ The Grand Build-Out (or, How Bolt Saved My Sanity... Mostly)

Building QuantaCode was a multi-phase saga, and honestly, Bolt, the AI generator, was my co-pilot for a significant chunk of it. It's like having an incredibly fast, sometimes opinionated, but ultimately indispensable team member.

Phase 1: Laying the Foundation
We started with Next.js 13, structuring everything like a perfectly organized closet. Clean, modular, scalable. (Truth: It was probably a messy pile of clothes at first, but we cleaned it up, honest, usually with a quick prettier --write and a prayer!) This foundational work was crucial for ensuring the project could grow without collapsing under its own weight.

Phase 2: Database & Authentication - The Great Pivot (and Bolt's Preferences)
Oh, Drizzle ORM. You promised so much, but delivered... headaches. Our complex schema for subscriptions and analytics just wasn't playing nice. The real kicker here wasn't Drizzle itself, but that Bolt seemed to have a strong preference for Prisma, making it challenging to get Drizzle to work seamlessly within the Bolt environment. This forced our hand. So, we did what any sane developer does in a hackathon: we pivoted to Prisma! It meant rewriting our entire data layer (cue dramatic music), but Prisma's TypeScript integration and robust relationship handling were a godsend. Seriously, Bolt, if you're listening, some native engine support for Prisma, especially allowing for the latest versions, would be a game-changer. We also hit a snag with testing: Bolt didn't readily allow us to spin up a local SQLite database for quick testing; we needed an actual PostgreSQL link even for development. This added an unexpected layer of setup complexity.

Phase 3: AI Integration - Bolt's Brainpower
This is where Bolt truly shined. Crafting those "quantum-level" prompts for OpenAI GPT-4? That's where Bolt was instrumental. It helped us generate the sophisticated queries that made the AI understand documentation, dependencies, code quality, and security. It was like having a super-fast, tireless prompt engineer on demand, allowing us to iterate on AI analysis capabilities at an incredible pace.

Phase 4: Voice Features - Making Code Talk
Integrating ElevenLabs was pure fun. To get started, you basically sign up, navigate to your profile settings, and grab your API key – it’s usually under an "API" or "Developers" tab. Then, you just plug that key into your backend environment variables, and voila! Your code starts chatting. We built out the tiered system: free users get a charmingly monotone robot, while premium users get voices so smooth, they could read a phone book and make it sound interesting. This feature added a unique layer of accessibility and engagement to our analysis.

Phase 5: Payment Integration - The Stripe Tango
Stripe for subscriptions, naturally. This was less "tango" and more "salsa with a blindfold." Setting up webhooks is a delicate dance. You register an endpoint on Stripe, get your webhook secret, and then in your Next.js API route, you verify the signature to ensure the request is legit. (Truth: We spent an embarrassing amount of time debugging why payments weren't updating user statuses. Turns out, a single missing await can ruin your whole day, leading to silent failures and frustrated users. We had to meticulously trace the webhook payload and update our user models.)

🎢 Challenges We Faced (The Uncensored Director's Cut)

Every hackathon has its demons, and QuantaCode was no exception. These were the moments that tested our resolve, fueled by caffeine, and occasionally, mild panic.

Database Migration Complexity & Bolt's Preferences: Drizzle was like that friend who promises to help you move, then shows up with a single box and an "Oops, my car broke down." We ended up doing most of the heavy lifting ourselves by migrating to Prisma. The core issue wasn't Drizzle's capabilities, but Bolt's compatibility. We also faced hurdles with SWC (Speedy Web Compiler), which, while fast, sometimes threw unexpected errors with certain Next.js 13 features or dependency configurations, forcing us to spend time on workarounds. (Truth: It was a necessary pain that paid off, but better ORM and compiler support from Bolt would have saved precious hours.)
Stripe Webhook Reliability: Payments were like ghosts – sometimes they'd appear, sometimes they'd wouldn't. Users would pay, and then... crickets. (Truth: Webhooks are tricky beasts. We learned the hard way about needing robust error handling, comprehensive logging, and multiple user lookup strategies. We even wrote specific debugging scripts like fix-stripe-subscriptions.js to bring order to the chaos, ensuring that even if a webhook failed, we had mechanisms to reconcile user subscription statuses.)
GitHub API Rate Limiting: Trying to analyze a massive repo felt like asking GitHub for too many favors. It would just shut us down. (Truth: We were hitting rate limits like a drum solo. Our fix involved intelligent request batching, prioritizing key files like package.json for initial analysis, setting a strict 15-minute analysis timeout to prevent endless loops, and running analysis in the background so users wouldn't get stuck staring at a spinner. We also implemented exponential backoff for retries.)
Real-time Analysis Updates: Imagine watching paint dry, but the paint might also disappear if you blink. That was our real-time analysis. (Truth: We needed background processing to keep analysis alive even if users navigated away, coupled with aggressive 5-second polling and slick visual progress indicators. This involved setting up a robust state management system to reflect the analysis progress accurately.)
Voice Feature Complexity: Getting the AI to "understand" the codebase context for voice chat was like teaching a parrot quantum physics. (Truth: It required a sophisticated context system, feeding the AI project metadata, file structure, code snippets, and conversation history to make it sound smart and relevant. This involved careful prompt engineering and managing token limits.)
Production Performance: Our analysis times were initially so slow, you could bake a cake while waiting. (Truth: We optimized by limiting file analysis to the 30 most important files, batching GitHub API calls, and adding more robust error handling. This significantly reduced the load and improved response times.)
The GSAP Gauntlet: Oh, GSAP. You beautiful, powerful, infuriating beast. Bolt, you're amazing at so many things, but when it came to animations, it felt like we were speaking different languages. (Truth: My experience with GSAP was very poor. It wasn't Bolt's fault directly, as it's a complex library, but prompting for intricate animations proved incredibly frustrating. We faced issues with conflicting timelines, unexpected element behavior in React's lifecycle, and debugging GSAP animations felt like trying to find a needle in a haystack made of JavaScript. We eventually got some cool animations, but it was a battle of wills, often requiring manual tweaking after Bolt's initial output.)
The AI Brainrot Chronicles: Let's be honest. When you have an AI generator like Bolt, there's a constant, insidious temptation: "Why read this error message? Just ask Bolt to fix it!" (Truth: I definitely faced a bit of "AI brainrot." The urge to just hand off debugging was strong. But I quickly learned that while Bolt is incredible for generating code and concepts, the human element – logical thinking, understanding the why behind an issue, and meticulously reading documentation – is absolutely non-negotiable. It's a powerful collaboration, but the human still needs to be the conductor, guiding the AI and verifying its output.)

🏆 Accomplishments That Made Us Fist-Pump the Air

Despite the rollercoaster, we emerged victorious (and slightly sleep-deprived) with some truly awesome accomplishments:

First Voice-Powered Codebase Interaction: We actually made code talk! This innovative feature sets QuantaCode apart and makes code analysis more engaging than ever before.
Quantum-Level AI Insights: Thanks to Bolt's prompting prowess and our logical thinking, the AI delivers deep, actionable analysis. We're proud of the sophistication of the insights, moving beyond simple linting to provide genuine value.
Real-time Background Processing: Analysis runs in the background, so you can keep coding without interruptions. This was a critical UX improvement that ensures a smooth workflow.
Comprehensive Admin Analytics: Because data is power, even for hackathon projects. Our admin dashboard provides a clear overview of platform usage and performance.
Sophisticated Subscription Management: Payments actually work! (Mostly!) Implementing a robust Stripe integration with webhooks was a significant technical achievement.
Production-Ready Architecture: It's not just a prototype; it's built to scale. We focused on a clean, modular codebase that can handle future growth and features.
Mobile Excellence: Looks good on tiny screens too. Our commitment to responsive design ensures a consistent and pleasant user experience across all devices.

📚 What We Learned (The Wisdom Gained)

This hackathon was a masterclass in rapid development and problem-solving, teaching us invaluable lessons that will shape our future projects:

Database Choice Matters: Seriously, pick your ORM wisely. Prisma saved us after Drizzle proved challenging within the Bolt environment. We also learned that Bolt's engine support for Prisma could be improved, specifically regarding the inability to upgrade Prisma to its latest versions, which limited our access to newer features and optimizations.
Webhook Reliability is Critical: Treat webhooks like precious, fragile babies. They require meticulous handling, comprehensive logging, and robust retry mechanisms to ensure data consistency.
API Rate Limiting Strategy: Be nice to external APIs, or they'll shut you down. Intelligent request batching and strategic prioritization are key to working within API constraints.
Background Processing: Essential for long-running tasks and happy users. It's a fundamental pattern for maintaining a responsive UI during intensive operations.
Error Handling is Everything: Assume everything will break, then build for it. Comprehensive error logging, user feedback, and recovery mechanisms are vital for production-ready applications.
Animation Library Nuances: GSAP is powerful but has a steep learning curve. Don't underestimate it. We found that our experience with GSAP was quite poor, struggling with prompting for complex animations and debugging the resulting code. This highlights a potential area for improvement in AI-generated animation code.
AI-Human Collaboration: Bolt is a phenomenal co-pilot, but human logical thinking, detailed documentation, and the willingness to get your hands dirty are still paramount. The AI accelerates development, but human oversight ensures quality and correctness.
UI Component Library Selection: While Shadcn UI is great, we learned that sometimes a "softer" UI aesthetic might be a better fit for certain projects. It's all about matching the vibe and user preferences, and exploring alternatives beyond the default.
AI Model Training Data: For future AI projects, training on structured sample code (like https://www.google.com/search?q=dycomps.oimmi.com) and real-world market projects would be invaluable for better AI understanding and output. This would provide more realistic and diverse examples, leading to more robust and accurate code generation and analysis.

🔮 What's Next for QuantaCode (The Future is Bright... and Quantum)

QuantaCode isn't just a hackathon project; it's a glimpse into the future of AI-powered development. We showed that quantum-level precision can meet intuitive user experience, even with the challenges of a rapid development cycle.

The journey taught us that true innovation comes from embracing challenges, learning from every bug, and constantly iterating. Every line of code, every optimization, every polished feature brought us closer to our vision of quantum-level code intelligence.

We're excited to continue enhancing our AI models, expanding our analysis capabilities, and exploring new intuitive ways for developers to interact with their code health insights. We envision a future where QuantaCode becomes an indispensable tool in every developer's arsenal, making code healthier, more secure, and easier to understand.

QuantaCode isn't just a hackathon project - it's a glimpse into the future of AI-powered development tools, where quantum-level precision meets exceptional user experience.

Built with ❤️ (and a lot of coffee, and Bolt's help!) for the Bolt Hackathon 2024

My Trial by Fire: A Complete Chronicle of Setting Up OSO as a PLDG Cohort-3 Newcomer

Aldorax — Sun, 29 Jun 2025 12:31:25 +0000

Prologue: Joining the PLDG Family

As I received my acceptance into Protocol Labs Developer Guild's Cohort-3, a mix of excitement and nervous energy coursed through me. Having chosen Open Source Observer (OSO) as one of my focus projects, I was eager to dive in. Little did I know that simply setting up the development environment would become an epic odyssey filled with technical challenges, small victories, and valuable lessons.

Chapter 1: The Installation Wars

The UV Dilemma

The documentation casually mentioned:

"Install UV: The modern Python package manager OSO uses"

First Attempt:

pip install uv

Result: The command ran without errors, but something felt off. When I tried using UV, I encountered strange permission errors and version conflicts.

The Investigation:

After two hours (just kidding—I actually just clicked the link in the OSO documentation for UV), I discovered the official UV installation guide, which revealed the proper method:

curl -LsSf https://astral.sh/uv/install.sh | sh

The Moment of Truth: The installation script scrolled through my terminal, each line loading with anticipation. Would it work this time? The final "Installation complete" message brought immense relief.

Why This Matters:

UV is a relatively new tool (launched in 2024) that's rapidly becoming the standard for Python projects. Its speed and reliability improvements over traditional pip make it worth the initial setup hassle.

The DuckDB Installation Saga

The next prerequisite was DuckDB, described as "SQLite for analytics." The installation seemed straightforward:

brew install duckdb  # For macOS

But on my Linux machine:

sudo apt-get install duckdb

Lesson Learned: Always check the official DuckDB installation docs for your specific OS—especially when dealing with tools you've never used before.

Chapter 2: The Missing Setup Revelation

Now I had to run:

uv sync
source .venv/bin/activate

It worked! —At least, that's what I thought until I ran:

uv run oso local sqlmesh-test --duckdb plan dev --start '1 week' --end now

The Crash:

ModuleNotFoundError: No module named 'metrics_service'

The Panic:

I spent hours (okay, less than one hour):

Checking Python versions
Verifying virtual environments
Re-reading the setup guide
Scouring GitHub issues for similar problems

The Breakthrough:

While examining the project structure, I noticed the pyproject.toml file. Aha! The project needed to be installed in development mode:

uv pip install -e .

The Magic: The -e flag stands for "editable" mode, meaning:

Changes to source files are immediately available
All dependencies are properly linked
The package is recognized system-wide

Why This Wasn't Obvious:

This step is second nature to experienced Python developers but rarely explained in project setup guides. For newcomers, it's a critical missing piece.

Chapter 3: The GCP Initiation—Credit Cards and Courage

When I first signed up for Google Cloud Platform (because OSO needed it), I approached it with the reckless abandon of a college student signing up for their first credit card. The process went something like this:

The Innocent Beginning: "Oh look, I can just use my Google account! How convenient!"
The Moment of Truth: "Add a payment method to continue" stared back at me like a bouncer at an exclusive club.
The Rationalization: "Well, they're offering $300 in free credit... that's basically free money, right?"

At this point, I had two options:

Option A: Carefully research GCP pricing, set up budget alerts, and monitor usage like a hawk
Option B: YOLO it and see what happens

Guess which path I chose?

My Actual Thought Process:

"Ha! Let's see how far I can stretch this $300! If I accidentally burn through it all in one glorious inferno of cloud computing, that'll make a great story! (Note to readers: Please don't actually do this—we'll get to the responsible part later.)"

I clicked through the signup with the enthusiasm of someone who's never received an unexpected cloud bill, added my card (while mentally calculating how many months of ramen $300 could buy me as a backup plan), and was immediately rewarded with my shiny new GCP playground.

The CLI Tool Tango

With my new GCP account ready, I turned to the documentation like a pirate consulting a treasure map. The official gcloud CLI docs became my bible as I ran commands with the confidence of someone who absolutely knew what they were doing (I didn't).

Installing gcloud CLI

The first command failed spectacularly:

gcloud init
zsh: command not found: gcloud

The Installation Process:

Downloaded the SDK:

curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-x86_64.tar.gz

Extracted and installed:

tar -xzf google-cloud-cli-linux-x86_64.tar.gz
./google-cloud-sdk/install.sh --quiet

Updated my shell configuration:

echo '# Google Cloud SDK' >> ~/.zshrc
echo 'source /path/to/google-cloud-sdk/path.zsh.inc' >> ~/.zshrc
echo 'source /path/to/google-cloud-sdk/completion.zsh.inc' >> ~/.zshrc
source ~/.zshrc

The Authentication Maze

Running gcloud init launched an interactive wizard:

Account Selection:
- Opened a browser window for signing in
- Asked if I trusted gcloud—I said yes, and boom! Done, back to the terminal!
Project Configuration:

The same command prompted me with:

Pick cloud project to use:  
 [1] *****  
 [2] ****  
 [3] Enter a project ID  
 [4] Create a new project  
Please enter numeric choice or text value (must exactly match list item):  4

I entered aldo-pldg-oso, and then:

Your current project has been set to: [aldo-pldg-oso].  

Not setting default zone/region (this feature makes it easier to use  
[gcloud compute] by setting an appropriate default value for the  
--zone and --region flag).  
See https://cloud.google.com/compute/docs/gcloud-compute section on how to set  
default compute region and zone manually. If you would like [gcloud init] to be  
able to do this for you the next time you run it, make sure the  
Compute Engine API is enabled for your project on the  
https://console.developers.google.com/apis page.  

Created a default .boto configuration file at [/home/aldy/.boto]. See this file and  
[https://cloud.google.com/storage/docs/gsutil/commands/config] for more  
information about configuring Google Cloud Storage.  
The Google Cloud CLI is configured and ready to use!

Region Selection: Now I had to make sure the project was set:

gcloud config get-value project  # Should return "aldo-pldg-oso"

It returned:

aldo-pldg-oso

Next, I needed to set the project ID in my shell for easy access:

export GOOGLE_PROJECT_ID="aldo-pldg-oso"

Critical Step:

gcloud auth application-default login

This created credentials at ~/.config/gcloud/application_default_credentials.json—the golden ticket for local development.

I also exported it for good measure:

export GOOGLE_APPLICATION_CREDENTIALS="$HOME/.config/gcloud/application_default_credentials.json"

Chapter 4: Service Account Security Theater

Creating the Service Account

gcloud iam service-accounts create oso-local-dev \
  --display-name="OSO Local Development Account"

Permission Assignment:

gcloud projects add-iam-policy-binding aldo-pldg-oso \
  --member="serviceAccount:oso-local-dev@aldo-pldg-oso.iam.gserviceaccount.com" \
  --role="roles/editor"

Generating the Key File

gcloud iam service-accounts keys create ~/oso-service-account-key.json \
  --iam-account=oso-local-dev@aldo-pldg-oso.iam.gserviceaccount.com

Handling the Key:

Set the environment variable:

export GOOGLE_APPLICATION_CREDENTIALS="$HOME/oso-service-account-key.json"

Chapter 5: The BigQuery Showdown

Enabling the API

After all this, I tried running OSO again but got this frustrating error:

403 BigQuery API has not been used in project ***** before or it is disabled.

The Fix:

gcloud services enable bigquery.googleapis.com

Verification:

gcloud services list --enabled | grep bigquery

Which returned:

bigquery.googleapis.com             BigQuery API  
bigqueryconnection.googleapis.com   BigQuery Connection API  
bigquerydatapolicy.googleapis.com   BigQuery Data Policy API  
bigquerymigration.googleapis.com    BigQuery Migration API  
bigqueryreservation.googleapis.com  BigQuery Reservation API  
bigquerystorage.googleapis.com      BigQuery Storage API

Testing the Connection

bq ls

First Attempt: Success!

The empty result was beautiful—it meant everything was working.

Chapter 6: Final Victory

With all pieces in place, the moment of truth:

oso local initialize

The output scrolled by:

2025-06-29T12:23:51 - INFO - Loading opensource-observer.farcaster.profiles...  
2025-06-29T12:23:53 - INFO - Initialization complete!

Celebration: I may have done a little victory dance at this point.

Epilogue: Lessons for Future Cohort Members

Documentation is a Starting Point: Expect to supplement official guides with your own research.
Cloud Setup is Complex: Budget significant time for authentication and service configuration.
Error Messages are Gold: Learn to read them carefully—they often contain the solution.
Ask Early: The PLDG community is incredibly supportive—don't struggle alone.
Document Your Journey: Your struggles will help the next cohort member.

Resources That Saved Me

UV Documentation - Essential for understanding this new package manager.
DuckDB Installation Guide - OS-specific instructions.
Google Cloud Quickstart - Great for understanding core concepts.
Python Packaging Guide - Explains editable installs and more.

Final Thoughts

This setup journey, while challenging, taught me more about modern development practices than any tutorial could have. The combination of new tools (UV), specialized databases (DuckDB), and cloud services (GCP) represents exactly the kind of real-world complexity we need to master as open-source contributors.

To my fellow cohort members: embrace the struggle. Each error resolved is a lesson learned, and every working setup is a victory worth celebrating. See you in the commits!

Why We Built FlowSpec: A Better Way to Orchestrate AI Workflows

Aldorax — Sat, 28 Jun 2025 21:47:34 +0000

By Emmanuel Appah, CTO of Auvra AI

At Auvra AI, we build sophisticated automation systems for trading bots, content generation, and AI reasoning. Like many developers, we initially used platforms like n8n—but we quickly hit limitations that forced us to rethink workflow orchestration from the ground up.

That’s why my teammate Emmanuel Akanji and I created FlowSpec, a next-generation workflow language designed specifically for AI and high-performance automation.

Here’s why it’s fundamentally better than JSON-based systems like n8n’s.

The Problem with JSON-Based Workflows

Platforms like n8n rely on JSON to define workflows. While this works for simple tasks, we encountered three critical limitations when building advanced AI pipelines:

1. Spaghetti Connections

JSON forces you to define every connection manually. For example, branching a workflow to 3 destinations requires repetitive edge definitions:

"connections": {
  "Node A": {
    "main": [
      [{"node": "Node B", "type": "main", "index": 0}],
      [{"node": "Node C", "type": "main", "index": 0}],
      [{"node": "Node D", "type": "main", "index": 0}]
    ]
  }
}

This becomes unmanageable at scale.

2. No Native Support for AI Patterns

Dynamic ports (e.g., generating inputs/outputs based on data) require hacky workarounds.
Loops and retries must be manually implemented.
No type safety—runtime errors from mismatched data are common.

3. Poor Performance at Scale

Parsing large JSON workflows is slow, and execution is linear by default—no built-in parallelism.

FlowSpec: A Language Built for AI Workflows

FlowSpec solves these problems with a YAML-based DSL optimized for AI/media automation.

1. Clean, Declarative Syntax

Compare n8n’s JSON to FlowSpec for the same branching logic:

n8n (JSON)

"connections": { /* 10+ lines of nested arrays */ }

FlowSpec (YAML)

connections:
  branch_example:
    from: node_a.output
    to: [node_b.input, node_c.input, node_d.input]

70% less code
Human-readable

2. Native AI/Media Features

✅ Dynamic Ports

audio_mixer:
  type: multi_speaker
  dynamic_inputs:
    speakers: 
      from: script.speaker_count  # Auto-create ports
      port_template: speaker_{index}

✅ Built-in Loops & Retries

connections:
  process_retry:
    from: transcriber.output
    to: validator.input
    retry: 
      max_attempts: 3
      delay: 5s

✅ Type Safety

nodes:
  gpt_agent:
    inputs:
      prompt: 
        type: string 
        format: markdown
        min_length: 50  # Validation

3. Performance Optimizations

Parallel execution by default
Priority controls for critical paths
5x faster parsing than JSON

Real-World Impact at Auvra AI

Since adopting FlowSpec:

🚀 40% faster development of new workflows

🔧 90% fewer runtime errors

📈 3x more scalable (100+ node workflows)

Example: Our trading bot’s workflow shrank from 45 lines of n8n JSON to 20 lines of FlowSpec with better error handling.

Metric	n8n (JSON)	FlowSpec (YAML)
Lines of Code	45+	20
Branching Logic	Manual edges	Native syntax
Execution Speed	~500ms/node	~100ms/node

Try FlowSpec Today

We’re open-sourcing FlowSpec soon. Follow Emmanuel Akanji for updates or try it at auvraai.net.

For developers battling workflow complexity: Sometimes the solution isn’t another platform—it’s a better language.

— Emmanuel Appah, CTO of Auvra AI

Key Takeaways

✔ JSON-based tools like n8n fail at scale for AI workflows.
✔ FlowSpec combines YAML’s readability with AI-specific features.
✔ Results: Faster dev, fewer errors, and enterprise scalability.

What workflow challenges are you facing? Let’s discuss below!

How I Fixed GitHub's Large File Warning with Git LFS for Auvra AI

Aldorax — Sat, 28 Jun 2025 21:33:37 +0000

The Warning That Started It All

When pushing updates to my Auvra AI project, I encountered this alarming message:

remote: warning: File public/hero.mp4 is 50.96 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.

Despite the warning, GitHub actually accepted my 50.96MB file (slightly over the limit), but I knew this wasn't a sustainable solution as I'd soon need to add even larger files.

The False Start: Git LFS Not Installed

My first attempt to use Git LFS failed spectacularly:

git lfs install
git: 'lfs' is not a git command.

This taught me an important lesson: Git LFS is a separate tool that needs to be installed, not something that comes bundled with Git.

Installing Git LFS on Ubuntu

After some research, I found the most reliable installation method for my Ubuntu system:

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt update && sudo apt install git-lfs

The installation process:

Added the Git LFS repository to my package sources
Updated my package lists
Installed the git-lfs package (which was about 3.9MB)

Setting Up Git LFS for Video Files

With Git LFS installed, I configured it to handle my large MP4 files:

git lfs track "*.mp4"

This command:

Created a .gitattributes file (if it didn't exist)
Added a rule to handle all MP4 files with LFS

I then committed this configuration change:

git add .gitattributes
git commit -m "Configure Git LFS for MP4 files"

The Magic of LFS in Action

When I added and committed my hero.mp4 video:

git add public/hero.mp4
git commit -m "Add hero video"

The push was now handled differently:

Uploading LFS objects: 100% (1/1), 53 MB | 218 KB/s, done.

Notice how:

The file was uploaded separately via LFS
The main Git push only transferred 527 bytes (just the pointer file)
No more size warnings from GitHub

Why This Solution Works

Git LFS replaces large files with text pointers in your Git history. For my hero.mp4:

version https://git-lfs.github.com/spec/v1
oid sha256:4d7a...f4c5
size 53456789

The actual file content is stored in LFS storage and downloaded only when needed.

Lessons Learned

Git has limits: While it pushed my 50.96MB file, this was cutting it close
LFS needs explicit installation: It's not part of standard Git
Configuration is simple: Just track file patterns and commit normally
Better for repository health: Keeps the main Git history lightweight

Personal Reflection: Git LFS Saved My Project, But It's Not Ideal

Let me be honest - while Git LFS solved my immediate problem with pushing large files to GitHub, the whole experience left me with mixed feelings. Here's my real take:

The Relief Was Temporary

That moment when I saw "Uploading LFS objects: 100% (1/1), 53 MB" felt like a victory. But later, I realized I'd just kicked the can down the road. Now I have to:

Monitor my LFS storage quota (GitHub gives you just 1GB free)
Ensure every team member has LFS installed
Deal with slower clones because of additional LFS fetching

The Hidden Costs Nobody Talks About

What they don't tell you about Git LFS:

It's not truly version controlled - LFS stores single versions of files, not full history
Collaboration headaches - New contributors often forget to install LFS
Backup complications - Your important assets now live in a separate storage system

"There Has to Be a Better Way" Moments

While wrestling with LFS, I kept thinking:

Maybe I shouldn't version control binary assets at all
Perhaps cloud storage (S3, GCS) with versioning would be better
Maybe my 50MB hero video should be compressed or hosted elsewhere

The Reality Check

For Auvra AI, Git LFS was the pragmatic choice today, but I'm not convinced it's the right architectural decision long-term. The fact that I had to use it feels like a symptom of:

Poor asset pipeline planning on my part
Trying to force Git to do something it wasn't designed for
Not properly separating code from media assets

Moving Forward

My personal takeaway? Git LFS is a band-aid, not a cure. For my next project, I'll:

Design my asset pipeline first
Consider dedicated asset hosting for large files
Only use Git for what it's best at - versioning code

Have you faced similar dilemmas? I'd love to hear how others balance convenience with proper architecture when dealing with large files.

✨ Supercharging Your FastAPI Documentation with Scalar

Aldorax — Sat, 26 Apr 2025 02:11:40 +0000

FastAPI is famous for its blazing speed and auto-generated documentation.

Out of the box, it gives you:

Swagger UI at /docs
ReDoc UI at /redoc

But here’s the thing:

If you want your API to feel polished, developer-friendly, and production-ready,

you need better documentation than just the basics.

Today, I’ll show you how to use scalar-fastapi to create beautiful, customizable API docs — and why great documentation is as important as great code.

🛠 The Problem with Default Swagger Docs

Swagger is great for getting started, but:

It's plain and developer playground-like
It doesn't scale visually when you have 50+ endpoints
It lacks branding — it doesn't feel like your product
First impressions matter — messy docs = perceived messy product

🚀 Meet `scalar-fastapi`

scalar-fastapi is a sweet package that upgrades your FastAPI documentation experience instantly.

It turns your OpenAPI schema into a:

Beautifully designed API reference
Modern, responsive UI
Developer-first experience

And the best part?

It takes just a few lines of code to integrate.

📦 How To Set It Up

First, install Scalar:

pip install scalar-fastapi

Now update your main.py:

from fastapi import FastAPI
from app.core.config import settings
from app.api.v1 import router as v1_router
from scalar_fastapi import get_scalar_api_reference

app = FastAPI(title=settings.PROJECT_NAME)

app.include_router(v1_router, prefix=settings.API_V1_STR)

@app.get("/")
def read_root():
    return {"message": "Welcome to the FastAPI backend!"}

@app.get("/scalar", include_in_schema=False)
async def scalar_html():
    return get_scalar_api_reference(
        openapi_url=app.openapi_url,
        title=app.title,
    )

✅ That’s it — now visiting /scalar will show you the upgraded docs!

🎨 Before and After — Visual Impact

Swagger UI (default)	Scalar API UI (better)
Messy when large	Clean and scalable
Limited customization	Full branding support
Feels like a demo project	Feels like a real product

A small upgrade that changes the entire experience.

🛠 Full Configuration Options (Explained)

Scalar gives you serious control over how your docs look and behave:

Option	Default	What It Controls
`layout`	`Layout.MODERN`	Layout style (`MODERN`, `CLASSIC`, etc.)
`show_sidebar`	`True`	Whether the left sidebar is visible
`hide_download_button`	`False`	Hides the "Download OpenAPI Spec" button
`hide_models`	`False`	Hides the models/schemas section
`dark_mode`	`True`	Dark mode by default
`search_hot_key`	`SearchHotKey.K`	Keyboard shortcut for search
`hidden_clients`	`[]`	Hide specific SDK clients
`servers`	`[]`	Define multiple server environments
`default_open_all_tags`	`False`	Expand all tags by default
`authentication`	`{}`	Pre-configure API authentication

🔥 Example: Fully Customized Scalar UI

from scalar_fastapi import get_scalar_api_reference, Layout, SearchHotKey

@app.get("/scalar", include_in_schema=False)
async def scalar_html():
    return get_scalar_api_reference(
        openapi_url=app.openapi_url,
        title="My Awesome API",
        layout=Layout.CLASSIC,
        show_sidebar=True,
        hide_download_button=False,
        hide_models=False,
        dark_mode=True,
        search_hot_key=SearchHotKey.K,
        servers=[
            {"url": "https://api.production.example.com"},
            {"url": "https://api.staging.example.com"}
        ],
        default_open_all_tags=True,
        authentication={"bearerAuth": "your-static-token-if-needed"},
    )

Boom — your docs now:

Match your brand
Show your production & staging servers
Look polished out of the box

🌍 Bonus: Deploying Your Public Docs

After setting up Scalar, you might want to expose your API docs nicely.

Here are two easy ways:

Docker Deploy: Serve your FastAPI app inside a Docker container behind nginx.
Cloud Deploy: Push it to AWS, Koyeb or DigitalOcean for HTTPS and autoscaling.

FastAPI + Scalar scales to real production traffic easily.

📣 Why Beautiful API Docs Matter

Good docs aren't just about looking nice:

✅ They save onboarding time for new devs

✅ They reduce mistakes using the API

✅ They build trust in your platform

✅ They differentiate you from lazy competitors

In real startups and production teams, beautiful docs = better adoption.

That's why companies like Stripe, Twilio, and Shopify invest heavily into theirs.

🏁 Final Thoughts

✅ If you're serious about building professional APIs

✅ If you care about other developers using your platform

✅ If you want to stand out

Then upgrading your FastAPI docs with Scalar is one of the easiest, most impactful things you can do today.

It takes 5 minutes, but leaves a permanent pro impression.

🚀 What's Next?

In the next part of this series, I’ll show how to:

Tag your endpoints for even cleaner documentation
Add authentication flows (JWT, OAuth) to your docs
Customize OpenAPI metadata (terms, license, support contacts)

Stay tuned — we’re building real production-ready APIs! 🎯

✍️ Final CTA

"If you enjoyed this guide, connect with me on LinkedIn for more full-stack engineering, and developer experience tips!" 🚀

🚀 How to Build a Scalable FastAPI Backend with Docker and Versioned APIs — The Right Way (Opinionated)

Aldorax — Sat, 26 Apr 2025 02:00:05 +0000

FastAPI makes it easy to get started with APIs.

But when you're thinking about building real projects — projects that might one day be used by thousands of users — structure and scalability matter from day one.

In this post, I'll show you:

How to structure your FastAPI project cleanly
How to organize versioned routes (so you can evolve your API over time)
Why using Docker even for small apps saves you headaches later
Clear explanations for every design decision (not just code)

📦 Why Structure Matters

When you first start, it’s tempting to put everything in a main.py.

And that’s fine — at first.

But as soon as your app grows:

It becomes hard to maintain
You can’t version your API easily
Adding new features becomes a mess

That’s why we’ll set up a modular, versioned, dockerized FastAPI app — from the start.

🏗 Project Structure (Professional Layout)

fastapi-app/
├── app/
│   ├── api/
│   │   ├── v1/
│   │   │   ├── endpoints/
│   │   │   │   ├── users.py
│   │   │   │   └── items.py
│   │   │   └── __init__.py
│   │   └── __init__.py
│   ├── core/
│   │   ├── config.py
│   │   └── __init__.py
│   ├── main.py
│   └── __init__.py
├── Dockerfile
├── requirements.txt
└── README.md

Why this structure?

app/api/v1/: all your routes go here, grouped by version (easy to maintain APIs even if your app lives 5+ years)
endpoints/: split your functionality logically — users.py for user routes, items.py for item routes, and so on.
core/: keep your app-wide configuration (env vars, settings) separate.
main.py: the real entry point — it brings everything together cleanly.

Good structure = easier scaling, faster onboarding of new developers, and fewer bugs over time.

🐍 Building the FastAPI App

`requirements.txt`

fastapi
uvicorn

We keep dependencies lightweight to start. Later, you’ll add things like SQLAlchemy, Alembic, etc.

`app/core/config.py`

class Settings:
    PROJECT_NAME: str = "FastAPI App"
    API_V1_STR: str = "/api/v1"

settings = Settings()

👉 Why?

Instead of hardcoding paths like /api/v1/ everywhere, we centralize configuration in one place.

Later, you can add database URLs, secrets, CORS configs here too.

`app/api/v1/endpoints/users.py`

from fastapi import APIRouter

router = APIRouter()

@router.get("/users", tags=["users"])
def get_users():
    return [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]

👉 Why use an APIRouter?

Routers keep related endpoints grouped together
Easier to version, test, and maintain in the long run
You can apply middlewares, auth, and permissions per-router later

`app/api/v1/endpoints/items.py`

from fastapi import APIRouter

router = APIRouter()

@router.get("/items", tags=["items"])
def get_items():
    return [{"id": "book", "price": 12}, {"id": "laptop", "price": 999}]

`app/api/v1/init.py`

from fastapi import APIRouter
from app.api.v1.endpoints import users, items

router = APIRouter()
router.include_router(users.router)
router.include_router(items.router)

👉 Why a central router?

Instead of cluttering main.py, we import and group all v1 routes here neatly.

`app/main.py`

from fastapi import FastAPI
from app.core.config import settings
from app.api.v1 import router as v1_router

app = FastAPI(title=settings.PROJECT_NAME)

app.include_router(v1_router, prefix=settings.API_V1_STR)

@app.get("/")
def read_root():
    return {"message": "Welcome to the FastAPI backend!"}

👉 Key points:

The root path gives a basic welcome message.
We attach all versioned routes under /api/v1, easily expandable to /api/v2 someday without chaos.

🐳 Why Docker?

Even if you're just starting, Docker brings 3 huge benefits:

Benefit	Why It Matters
Consistency	Same environment everywhere (local, staging, production)
Isolation	No \"works on my machine\" problems
Deployment Ready	You're always 1 `docker build` away from deploying

📄 Dockerfile

FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy app
COPY app ./app

# Run app
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]

Why this Dockerfile?

Python 3.11 slim image = fast and minimal
Workdir /app = standard for small backend services
No-cache pip install = smaller image sizes
Uvicorn reload mode = fast development cycles

🏃‍♂️ Running the App

Local dev:

pip install -r requirements.txt
uvicorn app.main:app --reload

Using Docker:

docker build -t fastapi-app .
docker run -p 8000:8000 fastapi-app

Access your app:

http://localhost:8000/
http://localhost:8000/docs (beautiful auto Swagger UI)

🧠 Why Versioning Early?

Imagine:

/api/v1/ is live with 10,000 users
You want to change your API behavior — without breaking old clients

With versioning:

You simply create /api/v2/
Clients who want the new features switch to v2
Older apps can keep using v1 safely

This is how real companies (Stripe, Twilio, etc.) handle their APIs.

🏁 Final Thoughts

✅ Structured project = easier scaling

✅ Versioned APIs = future-proofing your backend

✅ Docker = production readiness from day one

FastAPI gives you the speed of development — but your structure is what keeps your project alive long-term.

🚀 What's Next?

If you enjoyed this setup, in the next post I'll show you how to:

Add JWT Authentication
Connect to a real PostgreSQL Database
Write tests to make your app production-grade

Stay tuned! 🎯

🚀 Scaling Client-Side Search: 100,000 Users, Skills, and Real-Time Filtering in React

Aldorax — Sat, 26 Apr 2025 01:30:18 +0000

Today, I came across a post on dev.to from Fatemeh Paghar on using useTransition and react-window. I decided to push a typical React search/filter UI to the extreme by scaling it up to handle 100,000 users — not a typo — one hundred thousand records client-side.

Instead of simple emails, each user now has:

A name
A skillset (e.g., React, Python, AWS)

To make it even more interesting, I added a second search field — but with a twist:

You must first search by name before the skill search field becomes active.

🛠 How It Works

First input: Search by name (e.g., “Alice”, “Bob”)
Second input: Search by skill (e.g., “React”, “Rust”), but it remains disabled until a name is entered.

Once a name is typed, the skill search input unlocks and users can further refine their search.

⚡ First: The Basic Naive Version (No Optimization)

Let's look at a simple implementation first — without react-window and without useMemo.

"use client"
import React, { useState, useTransition } from "react";

const generateUsers = (count: number) => {
  const names = ["Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Helen"];
  const skills = ["React", "Python", "Node.js", "Django", "Rust", "Go", "Flutter", "AWS"];

  return Array.from({ length: count }, (_, i) => ({
    id: i,
    name: `${names[i % names.length]} ${i}`,
    skill: skills[i % skills.length],
  }));
};

const usersData = generateUsers(100000);

export default function BasicMassiveSearch() {
  const [nameQuery, setNameQuery] = useState("");
  const [skillQuery, setSkillQuery] = useState("");
  const [isPending, startTransition] = useTransition();

  const handleNameSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setNameQuery(value));
  };

  const handleSkillSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setSkillQuery(value));
  };

  const filteredUsers = usersData.filter((user) => {
    const matchesName = user.name.toLowerCase().includes(nameQuery.toLowerCase());
    if (!matchesName) return false;
    if (skillQuery) {
      return user.skill.toLowerCase().includes(skillQuery.toLowerCase());
    }
    return true;
  });

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-4">Simple Skill Search</h1>

      <input
        type="text"
        value={nameQuery}
        onChange={handleNameSearch}
        placeholder="Search by name..."
        className="border p-2 mb-4 block w-full"
      />

      <input
        type="text"
        value={skillQuery}
        onChange={handleSkillSearch}
        disabled={!nameQuery}
        placeholder="Search by skill..."
        className="border p-2 mb-4 block w-full"
      />

      {isPending && <p className="italic text-sm">Filtering...</p>}

      <div className="mt-4">
        {filteredUsers.length === 0 ? (
          <p className="italic text-gray-500">No users found.</p>
        ) : (
          filteredUsers.map((user) => (
            <div key={user.id} className="p-2 border-b">
              <p className="font-bold">{user.name}</p>
              <p className="text-sm text-gray-600">{user.skill}</p>
            </div>
          ))
        )}
      </div>
    </div>
  );
}

😵 Problems With This Naive Version

No Virtualization: Every single result (up to 100,000 DOM elements) renders at once. ➔ Browser lags, page freezes, and scrolling becomes painful.
No Memoization: Every re-render recomputes .filter() unnecessarily. ➔ Causes typing delays as the dataset grows.
Performance collapse: Mobile users and low-end laptops will suffer first.

✅ Now: The Optimized Version (Virtualized + Memoized)

"use client"
import React, { useState, useMemo, useTransition } from "react";
import { FixedSizeList as List } from "react-window";

const generateUsers = (count: number) => {
  const names = ["Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Helen"];
  const skills = ["React", "Python", "Node.js", "Django", "Rust", "Go", "Flutter", "AWS"];

  return Array.from({ length: count }, (_, i) => ({
    id: i,
    name: `${names[i % names.length]} ${i}`,
    skill: skills[i % skills.length],
  }));
};

const usersData = generateUsers(100000);

export default function MassiveSkillSearch() {
  const [nameQuery, setNameQuery] = useState("");
  const [skillQuery, setSkillQuery] = useState("");
  const [isPending, startTransition] = useTransition();
  const [searchDuration, setSearchDuration] = useState<number>(0);

  const filteredUsers = useMemo(() => {
    const start = performance.now();

    const lowerName = nameQuery.toLowerCase();
    const lowerSkill = skillQuery.toLowerCase();

    const result = usersData.filter((user) => {
      const matchesName = user.name.toLowerCase().includes(lowerName);
      if (!matchesName) return false;
      if (lowerSkill) {
        return user.skill.toLowerCase().includes(lowerSkill);
      }
      return true;
    });

    const end = performance.now();
    setSearchDuration(end - start);
    return result;
  }, [nameQuery, skillQuery]);

  const handleNameSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setNameQuery(value));
  };

  const handleSkillSearch = (e: React.ChangeEvent<HTMLInputElement>) => {
    const value = e.target.value;
    startTransition(() => setSkillQuery(value));
  };

  const Row = ({ index, style }: { index: number; style: React.CSSProperties }) => {
    const user = filteredUsers[index];
    return (
      <div style={style} key={user.id} className="p-3 border-b border-green-100 hover:bg-green-50">
        <p className="font-medium text-gray-800">{user.name}</p>
        <p className="text-sm text-gray-500">{user.skill}</p>
      </div>
    );
  };

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-4">🧠 Skillset Search (Virtualized)</h1>

      <input
        type="text"
        value={nameQuery}
        onChange={handleNameSearch}
        placeholder="Search by name..."
        className="border p-2 mb-4 block w-full"
      />

      <input
        type="text"
        value={skillQuery}
        onChange={handleSkillSearch}
        disabled={!nameQuery}
        placeholder="Search by skill..."
        className="border p-2 mb-4 block w-full"
      />

      {isPending && <p className="italic text-sm">Filtering...</p>}

      {filteredUsers.length === 0 ? (
        <p className="italic text-gray-500">No users found.</p>
      ) : (
        <List height={500} itemCount={filteredUsers.length} itemSize={70} width="100%">
          {Row}
        </List>
      )}

      <div className="mt-6 text-sm text-gray-600 text-center">
        <p>Total users: {usersData.length}</p>
        <p>Filtered users: {filteredUsers.length}</p>
        <p>Last search took: {searchDuration.toFixed(2)} ms</p>
        <p>{isPending ? "Searching..." : "Idle"}</p>
      </div>
    </div>
  );
}

🧠 What Changed — Why It Matters

Feature	Basic Version	Optimized Version
Rendering	100,000 real DOM nodes	Only ~20 visible DOM nodes
Filtering	Recalculates `.filter()` every render	Memorized with `useMemo`, recalculates only when query changes
Scrolling	Laggy and crash-prone	Smooth and buttery
Typing	Freezes momentarily on big datasets	Remains fluid
CPU Usage	Very high	Low and efficient

🏁 Final Verdict

✅ If you are just prototyping or handling small datasets (few hundred items),

you can survive without virtualization and memoization.

❗ But the moment you deal with tens of thousands of records (or more),

you must:

Virtualize large lists
Use useMemo smartly
Possibly move search server-side

Otherwise, your app will become unusable — fast.

🎯 Closing Note

React gives you amazing flexibility, but scale tests your architecture.

Always think about memory, CPU, and user experience as you grow datasets!

DEV Community: Aldorax

The Trader's Paradox: Why Being Good at Crypto Means You Can't Prove It

What If You Could Prove You're Good Without Revealing Anything?

The Missing Piece DeFi Never Built

How "Trade With Proof" Actually Works

Let Me Paint You a Picture of How This Plays Out

The Tech Stack That Makes This Possible

The Hardware Problem That Needed Solving

Enter Nillion: Private Computation Without Hardware Vaults

Why Zero-Knowledge Proofs Still Run the Show

What This Actually Means for Traders

What This Means for Followers

This Goes Way Beyond Trading

The Long Game

Trade With Proof, Not Trust

Building QuantaCode: Our Bolt Hackathon Saga (and the AI Brainrot is Real!)

🚀 The Spark of Genius (or, How We Avoided Total Brain Drain)

🎯 What We Cooked Up (Spoiler: It's Pretty Cool)

🛠️ The Grand Build-Out (or, How Bolt Saved My Sanity... Mostly)

🎢 Challenges We Faced (The Uncensored Director's Cut)

🏆 Accomplishments That Made Us Fist-Pump the Air

📚 What We Learned (The Wisdom Gained)

🔮 What's Next for QuantaCode (The Future is Bright... and Quantum)

My Trial by Fire: A Complete Chronicle of Setting Up OSO as a PLDG Cohort-3 Newcomer

Prologue: Joining the PLDG Family

Chapter 1: The Installation Wars

The UV Dilemma

The DuckDB Installation Saga

Chapter 2: The Missing Setup Revelation

Chapter 3: The GCP Initiation—Credit Cards and Courage

My Actual Thought Process:

The CLI Tool Tango

Installing gcloud CLI

The Authentication Maze

Chapter 4: Service Account Security Theater

Creating the Service Account

Generating the Key File

Chapter 5: The BigQuery Showdown

Enabling the API

Testing the Connection

Chapter 6: Final Victory

Epilogue: Lessons for Future Cohort Members

Resources That Saved Me

Final Thoughts

Why We Built FlowSpec: A Better Way to Orchestrate AI Workflows

The Problem with JSON-Based Workflows

1. Spaghetti Connections

2. No Native Support for AI Patterns

3. Poor Performance at Scale

FlowSpec: A Language Built for AI Workflows

1. Clean, Declarative Syntax

2. Native AI/Media Features

3. Performance Optimizations

Real-World Impact at Auvra AI

Try FlowSpec Today

For developers battling workflow complexity: Sometimes the solution isn’t another platform—it’s a better language.

Key Takeaways

How I Fixed GitHub's Large File Warning with Git LFS for Auvra AI

The Warning That Started It All

The False Start: Git LFS Not Installed

Installing Git LFS on Ubuntu

Setting Up Git LFS for Video Files

The Magic of LFS in Action

Why This Solution Works

Lessons Learned

Personal Reflection: Git LFS Saved My Project, But It's Not Ideal

The Relief Was Temporary

The Hidden Costs Nobody Talks About

"There Has to Be a Better Way" Moments

The Reality Check

Moving Forward

✨ Supercharging Your FastAPI Documentation with Scalar

🛠 The Problem with Default Swagger Docs

🚀 Meet scalar-fastapi

📦 How To Set It Up

🎨 Before and After — Visual Impact

🛠 Full Configuration Options (Explained)

🔥 Example: Fully Customized Scalar UI

🌍 Bonus: Deploying Your Public Docs

📣 Why Beautiful API Docs Matter

🚀 Meet `scalar-fastapi`

`requirements.txt`

`app/core/config.py`

`app/api/v1/endpoints/users.py`

`app/api/v1/endpoints/items.py`

`app/api/v1/init.py`

`app/main.py`