DEV Community

Vishwas Batra
Vishwas Batra

Posted on

The End of ScrapeForge

This post was supposed to be:
Day 6 of building ScrapeForge 🛠️

But unfortunately I have to stop here.

I am genuinely sorry to everyone who followed this journey and supported the project. I can no longer continue building ScrapeForge at this time.

Why I am stopping?
The reason is simple and honest. Lack of funds.

I know this sounds like a common excuse but it is the truth.

Quick recap:
Yesterday I completed around 80% of Instagram profile scraper. After that I started researching deployment options.

Until now I had only deployed static or frontend focused apps and had no real experience deploying a production grade backend.

During this research I realized I missed an important detail during the planning phase on Day 2.

ScrapeForge is a complex backend. It is long running stateful and resource intensive.

This kind of system cannot run reliably on platforms like Netlify or Vercel. It needs proper cloud infrastructure.

After estimating the costs I realized running ScrapeForge on cloud infrastructure would cost around $150 to $220 per month. With a current budget of $0 this is simply not possible.

At this point I could ask for preorders or investments to fund the project. But that does not feel right to me.

So I have decided to pause ScrapeForge entirely. There will be no further dev logs after this one.

What is next?
For now I am focusing on freelancing to collect funds.

I will be building websites and projects and using the skills I gained while building ScrapeForge such as web scraping reverse engineering APIs and building crawlers.

Once I have enough funds I will return to ScrapeForge and continue full time.

If you want ScrapeForge to come back sooner please share this post with anyone who needs a developer or wants data scraped from websites. This would genuinely help me move faster.

Thank you to everyone who followed supported and believed in this journey.

Top comments (0)