DEV Community: Saksham Solanki

Python Is Overrated? The REAL Best Language for Web Scraping

Saksham Solanki — Tue, 30 Dec 2025 13:49:54 +0000

“Is there any library which won’t suck when I’m deploying?”
That’s a question I used to ask for at least 90% of my projects.

Setting up undetected-chromedriver on macOS used to be pure pain for me. And since many clients didn’t really understand how all of this worked, they wanted a GUI to interact with. Of course, Python has libraries to build GUIs… but most of them look like they’re straight out of the early 2000s.

So I started looking for alternatives and eventually landed on JavaScript. It had better scraping libraries like Puppeteer, deployment felt smoother, and it’s already the primary language for web applications, so things just fit together better.

But then came the other limitations.

Data cleaning wasn’t great.
There were weird quirks, runtime oddities, and overall I just didn’t enjoy writing code in it as much. Scaling in js was also difficult, definitely doable but harder to optimize.

That’s when I discovered Golang.

It's community is small and is still growing, but it absolutely wins when it comes to speed, reliability, concurrency, and deployment. Goroutines and channels make concurrent workloads way easier as compared to threads or async-heavy code in other ecosystems.

Create a single binary and that's it.
No virtual environments, no dependency hell. Just copy it to whatever platform you want and run it.

That was a gold find for me.

Go is extremely good for networking, has solid support for web automation through libraries like chromedp and rod, and when it comes to scaling, Go performs far better than Python specially in CPU-bound and high-concurrency workloads. It has a predictable memory footprint too also the startup times are faster, and it handles thousands of concurrent connections without chugging resources unnecessarily.

You can argue with me that Python's got more libs, and that TypeScript makes JavaScript more reliable. And sure, I agree. But the raw performance, simplicity of deployment, and consistency Go provides are hard to match. You can even build IPC-based communication layers and easily integrate Go services with other languages for frontends or specialized tooling.

Yes, Go has comparatively smaller ecosystem.
Yes, there are fewer libraries right now.

But if it’s supported well and adopted by more developers, Go can very much challenge Python’s dominance in web scraping, especially for large-scale, production-grade systems where performance, stability, and deployment actually matter.

Dig at Berghain Coding Challenge

Saksham Solanki — Wed, 03 Sep 2025 17:34:29 +0000

Ever wondered what it's like to be the bouncer at the world's most exclusive nightclub? Listen Labs has created a fascinating algorithmic challenge that lets you experience just that! Minus the music and questionable fashion choices. The Berghain Challenge is a brilliant constraint satisfaction problem disguised as a nightclub simulation, and I've got to hand it to the creators, this is one addictive optimization puzzle.

The Challenge: More Than Just a Velvet Rope

The Berghain Challenge presents you with a simple task: fill a venue with exactly 1000 people while meeting specific demographic quotas. People arrive sequentially with binary attributes (young/old, well-dressed/casual, techno-lover/not, etc.), and you must make immediate accept/reject decisions. No take-backs.

You're scored on how many people you reject before filling the venue. Less rejections = better score.

First Impressions: The Naive Approach

When I first encountered this challenge, my instinct was to be that friendly bouncer who lets everyone in. Spoiler alert: that doesn't work when you need 600 young people out of 1000 spots as well as 600 well dressed, but only 32.25% of the crowd is actually young. The math doesn't lie - you need to be selective.
My initial attempts were... educational. Accept everyone who meets any constraint? Venue fills up with the wrong mix. Be too picky? Hit 2000-3000 rejections before filling the venue. There's a sweet spot, and finding it requires strategy.

The Strategy: Proportional Filling with a Safety Buffer
After several iterations, I developed what I call the "Proportional Filling with Buffer" strategy.
Here's the core insight:

pythonvenue_filled_percent = admitted_count / 1000 target_progress = min(venue_filled_percent + 0.10, 1.0)

The idea is elegant: maintain each attribute at your current venue fill percentage plus a 10% buffer. If you're 40% full, you want each constraint to be at least 50% complete. This buffer is crucial - it prevents the desperate scramble when you're at 950 admissions and still need 50 more young people.

The Implementation: Three-Phase Execution

My bot operates in three distinct phases:
Phase 1: Early Aggression (0-300 admissions)
# Aggressively accept valuable individuals if person_has_multiple_attributes: return ACCEPT # These people are gold

Early on, we can afford to be somewhat liberal with acceptances, especially for people with multiple desired attributes. They're rare (only ~6.8% have both young AND well-dressed in Scenario 1) and incredibly valuable.

Phase 2: Strategic Balance (300-800 admissions)
# Check if person helps maintain proportions for attribute in constraints: current_progress = attribute_counts[attribute] / constraints[attribute] if person_has_attribute and current_progress < target_progress: return ACCEPT

This is where the magic happens. We're constantly recalculating whether accepting someone keeps us on track. The 10% buffer means we're never in crisis mode - we're always slightly ahead of where we need to be.

Phase 3: Endgame Flexibility (900+ admissions)
# Constraints met? Fill remaining spots if all_constraints_satisfied: return ACCEPT # Take anyone at this point

Once we've hit our quotas, it's party time - everyone gets in until we hit capacity.

The Secret Sauce: Strategic Rejection
Here's what separates good strategies from great ones: knowing who to reject and when. In Scenario 1, about 41.7% of people have neither desired attribute. These are your strategic rejection candidates:
# Strategic rejection of "neither" category if has_no_attributes and min_progress > venue_filled + threshold: return REJECT

But here's the kicker - you can't reject them all immediately. You need some flexibility for the endgame. My strategy uses a dynamic threshold that adjusts based on rejection count and progress differential.

Results

With this approach, I consistently achieve:

Scenario 1: 900-950 rejections (compared to the optimal ~770-800)
Acceptance Rate: 51-53%
Success Rate: 100% constraint satisfaction
Total Decisions: ~1950

Is it the absolute best? No. I've since developed strategies that can go below 750 rejections. But this initial approach taught me valuable lessons about online decision-making, constraint satisfaction, and the importance of safety buffers in optimization problems.

Technical Insights

What makes this challenge particularly interesting from an algorithmic perspective is that it's an online constraint satisfaction problem with:

No lookahead: You can't see the queue, making this a true online algorithm challenge
Stochastic input: The attribute distribution is random but follows known probabilities
Multiple constraints: Balancing multiple quotas simultaneously requires careful tracking
Hard capacity limit: Exactly 1000 spots, no more, no less

The optimal solution likely involves more sophisticated techniques like:

Dynamic programming for threshold calculation
Bayesian updating of attribute probabilities
Linear programming-inspired dual variable pricing

Final Thoughts

The Berghain Challenge is a masterclass in gamified algorithm design. It's accessible enough that you can get started with basic if-then logic, but deep enough that optimizing your solution requires genuine algorithmic thinking. Huge props to the Listen Labs team for creating this engaging challenge - it's problems like these that make algorithm design feel less like homework and more like solving a really satisfying puzzle.

Whether you're aiming for the leaderboard top spot or just trying to beat your personal best, this challenge offers a perfect playground for exploring optimization strategies.

Challenge link: https://berghain.challenges.listenlabs.ai/

Have you tackled the Berghain Challenge? What strategies worked for you? Drop a comment below - I'd love to compare notes on optimization approaches!

I built a simple todo app that actually gets out of your way

Saksham Solanki — Tue, 02 Sep 2025 13:26:15 +0000

You download a productivity app to track simple tasks, then spend 20 minutes setting up projects, categories, and integrations just to add "buy milk" to a list.

I got fed up and built Zenith - a task manager that strips away everything except what you actually need.

The Problem with Modern Todo Apps

Most task managers suffer from feature creep. They want to be project management systems, team platforms, and life coaches all in one. The result? Apps that take longer to set up than your actual tasks.

What frustrated me most:

Complex onboarding and account creation
Feature overload (subtasks, dependencies, custom fields)
Web-based limitations requiring constant internet
Too many ways to organize one simple task

What I Built Instead

Zenith is a desktop app with one philosophy: task management should be fast and simple.

Core Features

Visual calendar - Click any date to add tasks
Drag & drop - Move tasks between dates easily
One-click completion - No ceremony, just check it off
Color coding - Six colors for visual organization
Analytics dashboard - Simple productivity trends

What I Left Out

No user accounts or cloud sync
No team collaboration
No complex project hierarchies
No integrations with dozens of services
No subscription pricing

Technical Stack

I kept it as simple as the UX:

Frontend: Vanilla JavaScript, HTML5, CSS3
Backend: Express.js with REST API
Database: SQLite (single file)
Desktop: Electron for cross-platform

The entire database is one table with 12 fields. The API is 5 endpoints. That's it.

Design Philosophy

Every feature decision came down to: "Does this help someone quickly add, complete, or reschedule a task?"

If no, it didn't make it in. This led to interesting constraints:

No subtasks (break complex tasks into separate ones)
Same-day deadlines only (focus on today)
Limited time estimates (30 min to 8 hours)

Why It Works

It's fast. Adding a task takes 5 seconds vs navigating menus and forms.

It stays hidden. No upgrade prompts, no feature tips, no social elements.

Constraints help. Fewer options make you more decisive about priorities.

What's Next

Probably nothing major. The whole point is that it's done. It does what it needs to do.
I might fix bugs, but I won't add AI assistants or any bs. Zenith will remain a tool that gets out of your way.

Try Zenith: https://github.com/sardanioss/zenith

Built for people who believe task management should be simple, not complicated.