<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: shrey vijayvargiya</title>
    <description>The latest articles on DEV Community by shrey vijayvargiya (@shreyvijayvargiya).</description>
    <link>https://dev.to/shreyvijayvargiya</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F584582%2F70be12eb-0227-48f0-b401-7a01e86cf0e2.jpeg</url>
      <title>DEV Community: shrey vijayvargiya</title>
      <link>https://dev.to/shreyvijayvargiya</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/shreyvijayvargiya"/>
    <language>en</language>
    <item>
      <title>Building and Running the Content Platform</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Mon, 13 Jul 2026 16:21:18 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/building-and-running-the-content-platform-3i6a</link>
      <guid>https://dev.to/shreyvijayvargiya/building-and-running-the-content-platform-3i6a</guid>
      <description>&lt;p&gt;Welcome to the new blog&lt;/p&gt;

&lt;p&gt;If you are new here, I am shrey, a software developer and creator of multiple products online. &lt;br&gt;
I have been running iHateReading for the past 4 years, and we have crossed 10k users signed up and are operating at 1k monthly active users.&lt;/p&gt;

&lt;p&gt;I am glad that readers are revisiting the website; the churn rate has been low for the past few months, and people are subscribing to our newsletter more than ever.&lt;/p&gt;

&lt;p&gt;This year, in 2026, in the first 6 months, I've improved the landing page and added other ƒeatures on the website as explained below&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Explore github trending repositories directly on the web home page&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://ihatereading.in/saas-directories" rel="noopener noreferrer"&gt;New 300+ SaaS directories page&lt;/a&gt; (our 2026 most visited page)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Nothing new has been done apart from consistent writing on the platform.&lt;/p&gt;

&lt;p&gt;iHateReading is FREE to read, not forced to log in, and no compulsory subscription to read.&lt;/p&gt;

&lt;p&gt;Each blog is handcrafted with zero AI involvement. I've been doing this for the past 5 years, which makes it easy for me to craft a 500-word article in 10-20 minutes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Building iHateReading
&lt;/h2&gt;

&lt;p&gt;I've never covered details about how I am running this platform, how I am earning, etc&lt;/p&gt;

&lt;p&gt;iHateReading is built entirely using Nextjs and Firebase, and the  detailed tech-stack is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Nextjs&lt;/li&gt;
&lt;li&gt;React&lt;/li&gt;
&lt;li&gt;Tailwind&lt;/li&gt;
&lt;li&gt;Lucide &lt;/li&gt;
&lt;li&gt;Firebase&lt;/li&gt;
&lt;li&gt;Redux&lt;/li&gt;
&lt;li&gt;Tiptap editor&lt;/li&gt;
&lt;li&gt;Vercel&lt;/li&gt;
&lt;li&gt;Server functions &lt;/li&gt;
&lt;li&gt;Resend&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The deployment is on Vercel, and the database is Firebase. We do use the paid version of Firebase with the Pay-as-you-go feature, and the reason is that I've made iHateReading 5 years ago and migrating to a new database is not easy when you have 10k total users.&lt;/p&gt;

&lt;h2&gt;
  
  
  Promoting the website
&lt;/h2&gt;

&lt;p&gt;Most of the time, I've got the same question to address: How to market my website, how to get visitors and users or traffic?&lt;/p&gt;

&lt;p&gt;First and foremost, it's not easy, and it won't happen on day-1&lt;br&gt;
You need 3-6 months in total to grab a good user base.&lt;/p&gt;

&lt;p&gt;Marketing on multiple other platforms, such as X, LinkedIn, TikTok, IG, Youtube, Medium, Devto, Hashnode, Dailydev, Cold Emailing, Reddit, all these platforms are needed to promote the website.&lt;/p&gt;

&lt;p&gt;One can say pay attention to one. Yes, if YouTube, then prefer YouTube, if good on LinkedIn, then prefer LinkedIn and so on. Choice is yours, but choose one platform and become very good at it to gain an audience.&lt;/p&gt;

&lt;p&gt;My mistake 4 years ago was choosing multiple platforms at the same time and promoting my content on multiple platforms. The strategy was good, but too much promotion was creating chaos in my work schedule and making it difficult to manage. I prefer choosing one platform and becoming good at it.&lt;/p&gt;

&lt;p&gt;But you might have to prompt or distribute the backlinks, hyperlink of your blog to the other trusted high-ranking domains, such as X, YouTube, LinkedIn, Hackerrank, or Devto, this increase the trust, and an AI agent like Chatgpt can recommend your website very well.&lt;/p&gt;

&lt;p&gt;I was contacted by one of the people whose SAAS business is to bring your blog content platform or website on chatgpt recommendations, his SAAS idea was to help people promote their website on chatgpt and AI agent suggestions/recommendations, increasing the trust, domain rating and traffic.&lt;/p&gt;

&lt;p&gt;Another good solution is to submit the website to Google Search Console with sitemaps and have the page indexed, preferably by buying the .com domain or other trusted domains; do not go beyond fancy, untrusted domains, otherwise your emails will be spammed in the inbox.&lt;/p&gt;

&lt;p&gt;Make a list of distribution channels for your website content, for me it's Medium, Hackernoon, Hashnode, Devto and LinkedIn. Do not spam similar content across platforms; leverage each platform's audience, understand what they want, and only promote that type of content.&lt;/p&gt;

&lt;h2&gt;
  
  
  Admin Panel
&lt;/h2&gt;

&lt;p&gt;I've been writing for years and have used multiple writing tools and CMS, but it was never easy to write content and push to the database and then render on the website with all SEO tags and an updated sitemap.&lt;/p&gt;

&lt;p&gt;I've learned this admin panel and its benefits in my previous company, and using a similar technique, I have an admin panel for our iHateReading to write blogs, edit blogs, manage users, create email newsletters on a Tiptap notion-like editor, send emails to our subscribers directly from the admin panel, store blogs directly into the database, etc &lt;/p&gt;

&lt;p&gt;All these features are handled within the admin panel; it's more like iHateReading CMS.&lt;/p&gt;

&lt;p&gt;I am a big fan of the Admin panel, and we have built the most advanced Admin Panel, which we reuse across all of our existing products and future SAAS products.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://demo-app.buildsaas.dev/admin" rel="noopener noreferrer"&gt;Check Admin Panel&lt;/a&gt;, just to provide an overview of the admin panel&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx9bosuuqpqke9hx7cikc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx9bosuuqpqke9hx7cikc.png" alt=" " width="799" height="399"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Earning sources
&lt;/h2&gt;

&lt;p&gt;Initially, iHateReading's target was to reach to the exact audiences and then increase the subscriber base. As a platform, iHateReading wasn't a pain killer its a vitamin so we can't depend on users to come and subscribe. &lt;/p&gt;

&lt;p&gt;The first target was to bring more subscribers, and for that, in 2023, I started crafting bi-weekly email updates on programming, tech and AI. &lt;/p&gt;

&lt;p&gt;Within the first 8 months, we have crossed 1k subscribers because of the traffic, but nobody told you how to reach to that point. &lt;/p&gt;

&lt;p&gt;It took me almost 1 to 2 years, with consistent writing of almost 500 blogs, to promote the website to 1k active monthly visitors. &lt;/p&gt;

&lt;p&gt;Promoting the website has a compounding effect, and it works slowly, but certainly it does work.&lt;/p&gt;

&lt;p&gt;Once the subscriber base was increasing, I was confident enough to show something to our early sponsor. Cold emailing for sponsors is easy-peasy money, but it's not easy to get those sponsors.&lt;/p&gt;

&lt;p&gt;We got our first sponsor for iHateReading in January 2025, Anima AI. The company reached out to me to create blog content and promote the content on iHateReading via newsletter, cold emailing and through the blog on the website. &lt;/p&gt;

&lt;p&gt;iHateReading has provided 300+ monthly visitors with 0.6% of conversion ratio for Anima, and these were their words, not mine. 300 users were not bad for any small startup, and so many solo indie developers needed that many visitors.&lt;/p&gt;

&lt;p&gt;Other ways to provide sponsorship are the website banner position, navbar sponsors, footer and other placements on the website, job posting, email marketing via cold emailing to all subscribers, all these are the ways to earn.&lt;/p&gt;

&lt;p&gt;One easy way is to contact the companies or SAAS founders, asking them to provide a banner, blog and one/2/3 cold email package promoting their website/tool/product. &lt;/p&gt;

&lt;p&gt;The package works every time. &lt;br&gt;
Instead of selling one sponsored blog, I prefer providing one blog for one month, 2/3 cold emailing marketing to our subscribers, and one banner or footer placement for advertisement and promoting a number at the end of the month, such as 1k visitors in total for the next 30-40 days.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://ihatereading.in/advertise-with-us" rel="noopener noreferrer"&gt;Check our Advertisement page for a better understanding&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you can't find the client for sponsorship, here as few ways to do so&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Find manually on LinkedIn and contact each of them, DM &lt;/li&gt;
&lt;li&gt;Find each on X or Reddit and DM them promoting your website&lt;/li&gt;
&lt;li&gt;Use third-party client finding solutions, such as Apollo for lead generation and other AI LinkedIn or X lead generation tools&lt;/li&gt;
&lt;li&gt;Social media tweets, posts asking for sponsors and replying to all comments sections&lt;/li&gt;
&lt;li&gt;Posting a small YouTube short for the same&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Manually reaching out to them works well, since you only need 2-3 sponsors each month, and manually doing it will save time and increase the conversion or success rate.&lt;/p&gt;

&lt;p&gt;That would be enough for today, see you in the next one&lt;br&gt;
Cheers&lt;br&gt;
Shrey&lt;br&gt;
&lt;a href="//ihatereading.in"&gt;iHateReading&lt;/a&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>ai</category>
      <category>programming</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Build a directory listing website that earns online</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Fri, 10 Jul 2026 05:44:29 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/build-a-directory-listing-website-that-earns-online-4o78</link>
      <guid>https://dev.to/shreyvijayvargiya/build-a-directory-listing-website-that-earns-online-4o78</guid>
      <description>&lt;p&gt;A starter kit to quickly make your own directory listing website using Nextjs, Supabase and Stripe&lt;/p&gt;

&lt;p&gt;This blog tells the story about &lt;a href="https://tooldir.buildsaas.dev/" rel="noopener ugc nofollow noreferrer"&gt;ToolDir&lt;/a&gt;, the directory listing website template built using Nextjs, Supabase and Stripe, a full-stack template to quickly build your directory listing website platform that can earn online.&lt;/p&gt;

&lt;p&gt;Directory listing websites are boring, but in 2026, we got tonnes of SaaS submission websites, AI tool listing websites, prompt directories, Design md directories, and even React frontend UI directories.&lt;/p&gt;

&lt;p&gt;Each directory has almost the same pattern of building, and every time we do the same process as defined&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Create a frontend landing page&lt;/li&gt;
&lt;li&gt;Each product page and all product listings pages with filters and search&lt;/li&gt;
&lt;li&gt;Login page for authentication&lt;/li&gt;
&lt;li&gt;Submit page for submitting tools/prompts/designs based on the directory website&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Below is our ToolDir LLM.txt for the directory template. Copy the text and paste it into Claude to get more details about our listing directory template, its features, and how to create one on your own.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# ToolDir — LLM context file&lt;/span&gt;
&lt;span class="gh"&gt;# Purpose: help coding agents understand this repo quickly without exploratory scanning.&lt;/span&gt;
&lt;span class="gh"&gt;# Last cleaned for launch template readiness.&lt;/span&gt;

&lt;span class="gu"&gt;## Product&lt;/span&gt;

ToolDir is a Next.js 15 (Pages Router) AI tool directory boilerplate.
Buyers launch a public directory site, accept tool submissions, sell launch slots,
publish a blog, and manage everything from &lt;span class="sb"&gt;`/admin`&lt;/span&gt;.

This is NOT a generic multi-tenant SaaS CMS. Older SaaS modules (email campaigns,
waitlist, kanban, products catalog, forms builder, cron, codegen) were removed.

&lt;span class="gu"&gt;## Tech stack&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; Next.js 15 Pages Router (&lt;span class="sb"&gt;`pages/`&lt;/span&gt;), React 18, Tailwind, shadcn/ui
&lt;span class="p"&gt;-&lt;/span&gt; Supabase (Auth + Postgres + Storage)
&lt;span class="p"&gt;-&lt;/span&gt; TanStack React Query for server state
&lt;span class="p"&gt;-&lt;/span&gt; Redux Toolkit only for subscription/customer client state
&lt;span class="p"&gt;-&lt;/span&gt; TipTap for blog CMS
&lt;span class="p"&gt;-&lt;/span&gt; Stripe &lt;span class="gs"&gt;**one-time**&lt;/span&gt; Checkout for launch slots (server builds &lt;span class="sb"&gt;`price_data`&lt;/span&gt;; no Price ID catalog required)
&lt;span class="p"&gt;-&lt;/span&gt; Set &lt;span class="sb"&gt;`STRIPE_TEST_ZERO=true`&lt;/span&gt; (+ &lt;span class="sb"&gt;`NEXT_PUBLIC_STRIPE_TEST_ZERO`&lt;/span&gt;) to force $0 checkout while wiring webhooks/admin
&lt;span class="p"&gt;-&lt;/span&gt; Resend for optional slot-payment receipt (&lt;span class="sb"&gt;`sendSlotPaymentEmail`&lt;/span&gt;)
&lt;span class="p"&gt;-&lt;/span&gt; Framer Motion for marketing motion

&lt;span class="gu"&gt;## Critical architecture note&lt;/span&gt;

&lt;span class="sb"&gt;`app/`&lt;/span&gt; is NOT the App Router.
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`pages/`&lt;/span&gt; = real Next.js routes and API handlers
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`app/admin/`&lt;/span&gt; = admin UI components
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`app/frontend/`&lt;/span&gt; = marketing site components (Navbar, Footer, directory UI)
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`components/ui/`&lt;/span&gt; = shadcn primitives
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`lib/`&lt;/span&gt; = APIs, hooks, config, data seeds, store, utils

&lt;span class="gu"&gt;## Public routes (keep)&lt;/span&gt;

| Route | File | Role |
|-------|------|------|
| &lt;span class="sb"&gt;`/`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/index.js`&lt;/span&gt; | Directory home |
| &lt;span class="sb"&gt;`/tool/[slug]`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/tool/[slug].js`&lt;/span&gt; | Tool detail |
| &lt;span class="sb"&gt;`/categories/[slug]`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/categories/[slug].js`&lt;/span&gt; | Category filter |
| &lt;span class="sb"&gt;`/submit`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/submit.js`&lt;/span&gt; | Multi-step submission |
| &lt;span class="sb"&gt;`/pricing`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/pricing.js`&lt;/span&gt; | Launch slot calendar |
| &lt;span class="sb"&gt;`/dashboard`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/dashboard.js`&lt;/span&gt; | User submissions |
| &lt;span class="sb"&gt;`/blog`&lt;/span&gt;, &lt;span class="sb"&gt;`/blog/[slug]`&lt;/span&gt; | &lt;span class="sb"&gt;`pages/blog*.js`&lt;/span&gt; | Public blog |
| &lt;span class="sb"&gt;`/contact`&lt;/span&gt;, &lt;span class="sb"&gt;`/privacy`&lt;/span&gt;, &lt;span class="sb"&gt;`/terms-and-conditions`&lt;/span&gt; | legal |
| &lt;span class="sb"&gt;`/auth/callback`&lt;/span&gt; | Supabase OAuth |
| &lt;span class="sb"&gt;`/robots.txt`&lt;/span&gt;, &lt;span class="sb"&gt;`/sitemap.xml`&lt;/span&gt; | SEO |

&lt;span class="gu"&gt;## Admin routes (keep)&lt;/span&gt;

| Route | Role |
|-------|------|
| &lt;span class="sb"&gt;`/admin`&lt;/span&gt; | Shell + tabs |
| &lt;span class="sb"&gt;`/admin/blogs/new`&lt;/span&gt;, &lt;span class="sb"&gt;`/admin/blogs/[id]`&lt;/span&gt; | Nested blog editor |
| &lt;span class="sb"&gt;`/admin/editor/blog`&lt;/span&gt; | Legacy redirect into nested editor |

Admin tabs (sidebar): analytics, users, submissions, payments, invoices, pricing-slots, blogs.
Tab id &lt;span class="sb"&gt;`forms`&lt;/span&gt; was renamed to &lt;span class="sb"&gt;`submissions`&lt;/span&gt; (legacy &lt;span class="sb"&gt;`forms`&lt;/span&gt; still opens SubmissionsTab).

&lt;span class="gu"&gt;## API routes (keep)&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="sb"&gt;`/api/directory-tools/publish`&lt;/span&gt; — approve submission → listing
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/directory-tools/seed`&lt;/span&gt; — seed from &lt;span class="sb"&gt;`lib/data/tools.js`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/tool-submissions/create`&lt;/span&gt; — public submit
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/pricing-slots/upsert`&lt;/span&gt; — admin slot CRUD
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/storage/upload-image`&lt;/span&gt; — Supabase Storage uploads
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/messages/create`&lt;/span&gt; — contact form
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/stripe/checkout`&lt;/span&gt; — one-time Checkout for launch slot (&lt;span class="sb"&gt;`price_data`&lt;/span&gt;, optional &lt;span class="sb"&gt;`$0`&lt;/span&gt; via &lt;span class="sb"&gt;`STRIPE_TEST_ZERO`&lt;/span&gt;)
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`/api/stripe/webhook`&lt;/span&gt; — marks submission paid, upserts &lt;span class="sb"&gt;`payments`&lt;/span&gt; / &lt;span class="sb"&gt;`customers`&lt;/span&gt; / &lt;span class="sb"&gt;`users`&lt;/span&gt;, sends Resend receipt

Removed: &lt;span class="sb"&gt;`/api/stripe/cancel-subscription`&lt;/span&gt; (no SaaS subscriptions).

&lt;span class="gu"&gt;## Core modules (do not delete)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;lib/api/directoryTools.js&lt;br&gt;
lib/api/toolSubmissions.js&lt;br&gt;
lib/api/pricingSlots.js&lt;br&gt;
lib/api/blog.js&lt;br&gt;
lib/api/storage.js&lt;br&gt;
lib/api/auth.js&lt;br&gt;
lib/hooks/useDirectoryTools.js&lt;br&gt;
lib/hooks/useBlogs.js&lt;br&gt;
lib/hooks/usePricingSlots.js&lt;br&gt;
lib/data/tools.js&lt;br&gt;
lib/data/pricing-slots.js&lt;br&gt;
lib/data/testimonials.js&lt;br&gt;
app/frontend/marketing/*&lt;br&gt;
app/admin/components/SubmissionsTab.jsx&lt;br&gt;
app/admin/components/PricingSlotsTab.jsx&lt;br&gt;
app/admin/components/BlogTab.jsx&lt;br&gt;
app/admin/components/BlogTiptapEditor.jsx&lt;br&gt;
app/admin/views/BlogEditorView.jsx&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;
&lt;span class="gu"&gt;## Data model (Supabase)&lt;/span&gt;

Primary tables: &lt;span class="sb"&gt;`directory_tools`&lt;/span&gt;, &lt;span class="sb"&gt;`tool_submissions`&lt;/span&gt;, &lt;span class="sb"&gt;`pricing_slots`&lt;/span&gt;, &lt;span class="sb"&gt;`blogs`&lt;/span&gt;
Billing/admin: &lt;span class="sb"&gt;`users`&lt;/span&gt;, &lt;span class="sb"&gt;`customers`&lt;/span&gt;, &lt;span class="sb"&gt;`payments`&lt;/span&gt;, &lt;span class="sb"&gt;`checkouts`&lt;/span&gt;, &lt;span class="sb"&gt;`invoices`&lt;/span&gt;, &lt;span class="sb"&gt;`messages`&lt;/span&gt;, &lt;span class="sb"&gt;`app_analytics`&lt;/span&gt;
Storage bucket: &lt;span class="sb"&gt;`saas-starter-storage`&lt;/span&gt; (name kept for compatibility)

See &lt;span class="sb"&gt;`supabase-migrations.sql`&lt;/span&gt; (single file: tables + payment columns + Storage bucket).

When Supabase is empty / unset, directory listing can fall back to mock data in &lt;span class="sb"&gt;`lib/data/tools.js`&lt;/span&gt;.

&lt;span class="gu"&gt;## Supabase Storage&lt;/span&gt;

Bucket name: &lt;span class="gs"&gt;**`saas-starter-storage`**&lt;/span&gt; (constant &lt;span class="sb"&gt;`STORAGE_BUCKET`&lt;/span&gt; in &lt;span class="sb"&gt;`lib/api/storage.js`&lt;/span&gt;).
TipTap editors call &lt;span class="sb"&gt;`uploadBlogEditorImage`&lt;/span&gt; → direct Storage upload, fallback &lt;span class="sb"&gt;`POST /api/storage/upload-image`&lt;/span&gt;.
Paths: &lt;span class="sb"&gt;`blogs/content/`&lt;/span&gt;, &lt;span class="sb"&gt;`blogs/{id}/content/`&lt;/span&gt;, &lt;span class="sb"&gt;`blogs/banners/`&lt;/span&gt;, &lt;span class="sb"&gt;`blogs/{id}/banner-*`&lt;/span&gt;.
SQL: run &lt;span class="sb"&gt;`supabase-migrations.sql`&lt;/span&gt; once (includes Storage bucket &lt;span class="sb"&gt;`saas-starter-storage`&lt;/span&gt;).

&lt;span class="gu"&gt;## Environment variables&lt;/span&gt;

Required for core:
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_SITE_URL`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_SUPABASE_URL`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_SUPABASE_KEY`&lt;/span&gt;

Payments:
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`STRIPE_SECRET_KEY`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`STRIPE_WEBHOOK_SECRET`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY`&lt;/span&gt; (optional if you only start checkout server-side)
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`STRIPE_TEST_ZERO=true`&lt;/span&gt; — charge $0 in Checkout for local testing
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_STRIPE_TEST_ZERO=true`&lt;/span&gt; — show $0 copy on submit UI

Optional:
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`RESEND_API_KEY`&lt;/span&gt;, &lt;span class="sb"&gt;`RESEND_FROM_EMAIL`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_POSTHOG_KEY`&lt;/span&gt;, &lt;span class="sb"&gt;`NEXT_PUBLIC_POSTHOG_HOST`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`NEXT_PUBLIC_APP_URL`&lt;/span&gt; (legacy Stripe fallback; prefer SITE_URL)

Use &lt;span class="sb"&gt;`.env.local`&lt;/span&gt; locally. &lt;span class="sb"&gt;`.env*`&lt;/span&gt; is gitignored.

&lt;span class="gu"&gt;## Intentional dual editors&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="sb"&gt;`BlogTiptapEditor.jsx`&lt;/span&gt; — canonical nested route editor (&lt;span class="sb"&gt;`/admin/blogs/*`&lt;/span&gt;)
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="sb"&gt;`TiptapEditor.jsx`&lt;/span&gt; — still used by BlogTab modal list workflow

Do not delete either without consolidating BlogTab first.

&lt;span class="gu"&gt;## What was removed for launch&lt;/span&gt;

Do not reintroduce these without a product need:
&lt;span class="p"&gt;-&lt;/span&gt; Forms builder (&lt;span class="sb"&gt;`FormsTab`&lt;/span&gt;, &lt;span class="sb"&gt;`/api/forms/*`&lt;/span&gt;, &lt;span class="sb"&gt;`forms`&lt;/span&gt; / &lt;span class="sb"&gt;`form_submissions`&lt;/span&gt; tables)
&lt;span class="p"&gt;-&lt;/span&gt; Email campaigns, subscribers, waitlist, kanban/tasks, idea DB, changelog, cron, assets CMS
&lt;span class="p"&gt;-&lt;/span&gt; Builder zip/AST APIs, roles-config codegen (&lt;span class="sb"&gt;`modules-config`&lt;/span&gt;, generate-&lt;span class="err"&gt;*&lt;/span&gt;)
&lt;span class="p"&gt;-&lt;/span&gt; Unused deps: octokit, archiver, simple-git, nginx, react-toastify, @vercel/analytics, @stripe/stripe-js, unused Radix accordion/checkbox/avatar/tabs/tooltip/switch/scroll-area

&lt;span class="gu"&gt;## Conventions for agents&lt;/span&gt;
&lt;span class="p"&gt;
1.&lt;/span&gt; Prefer Pages Router for new pages under &lt;span class="sb"&gt;`pages/`&lt;/span&gt;.
&lt;span class="p"&gt;2.&lt;/span&gt; Put reusable marketing UI in &lt;span class="sb"&gt;`app/frontend/marketing/`&lt;/span&gt;.
&lt;span class="p"&gt;3.&lt;/span&gt; Put Supabase access in &lt;span class="sb"&gt;`lib/api/*`&lt;/span&gt;; use React Query hooks in &lt;span class="sb"&gt;`lib/hooks/*`&lt;/span&gt;.
&lt;span class="p"&gt;4.&lt;/span&gt; Do not change visual theme / CSS / layout unless the task is design work.
&lt;span class="p"&gt;5.&lt;/span&gt; Do not commit &lt;span class="sb"&gt;`.env`&lt;/span&gt; files.
&lt;span class="p"&gt;6.&lt;/span&gt; Keep packaging lean — delete orphan modules instead of leaving stubs.
&lt;span class="p"&gt;7.&lt;/span&gt; Brand copy is ToolDir, not "SAAS Starter".

&lt;span class="gu"&gt;## Local commands&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
bash&lt;br&gt;
npm install&lt;br&gt;
npm run dev     # &lt;a href="http://localhost:3002" rel="noopener noreferrer"&gt;http://localhost:3002&lt;/a&gt;&lt;br&gt;
npm run build&lt;br&gt;
npm run lint&lt;br&gt;
npm run pack    # clean buyer zip → dist/tooldir.zip&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When I was building this boilerplate, I wrote the LLM text to help everyone ask questions related to the boilerplate on Claude or chatgpt.&lt;/p&gt;

&lt;p&gt;Once done, &lt;a href="https://tooldir.buildsaas.dev/" rel="noopener ugc nofollow noreferrer"&gt;check the demo&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Once checked, &lt;a href="https://shreyvijayvargiya.gumroad.com/l/nextjs-supabase-directory-boilerplate?layout=profile" rel="noopener ugc nofollow noreferrer"&gt;buy the template 😃&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Let me explain what this template contains&lt;/p&gt;

&lt;h2&gt;
  
  
  How to build your Directory website in seconds?
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;You will get the zip file&lt;/li&gt;
&lt;li&gt;Unzip it and run some commands (All details are in the Readme.md file in the zip)&lt;/li&gt;
&lt;li&gt;Once run, add the supabase migration SQL in supabase SQL editor to create the database tables&lt;/li&gt;
&lt;li&gt;For authentication, simply enable supabase Google and Email as a sign-in provider and add Google Cloud Console credentials, such as Client ID and Client secret, along with the OAuth callback URL (Details are in the readme file)&lt;/li&gt;
&lt;li&gt;Add the Stripe secret key from the Stripe dashboard to enable payment for the early submission booking slot&lt;/li&gt;
&lt;li&gt;Change and customise the pricing, landing page and blogs as per your project needs&lt;/li&gt;
&lt;li&gt;Deploy the app and put the env keys containing supabase key and the Stripe key on the deployment server&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;7 steps and your directory website is ready with database, authentication and payment integrated.&lt;/p&gt;

&lt;h2&gt;
  
  
  Admin Panel to Manage Directory Website Business
&lt;/h2&gt;

&lt;p&gt;That is not enough&lt;/p&gt;

&lt;p&gt;Inside the zip folder, you can find app/admin folder, this contains the admin panel to write blog, review submissions, check payments and website analytics and update pricing for each slot, all reflects directly on the website within the admin panel.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://tooldir.buildsaas.dev/admin" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://tooldir.buildsaas.dev/admin" rel="noopener noreferrer"&gt;https://tooldir.buildsaas.dev/admin&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This makes it easy for you to run the entire directory website content, manage users and view analytics without touching code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tech Stack
&lt;/h2&gt;

&lt;p&gt;The template is built using the latest technology stack&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Nextjs&lt;/li&gt;
&lt;li&gt;Shadcn for UI components&lt;/li&gt;
&lt;li&gt;Tailwind for styling&lt;/li&gt;
&lt;li&gt;framer motion for animation&lt;/li&gt;
&lt;li&gt;Lucide React for icons&lt;/li&gt;
&lt;li&gt;Supabase for the database&lt;/li&gt;
&lt;li&gt;Supabase storage for images&lt;/li&gt;
&lt;li&gt;Stripe for payment&lt;/li&gt;
&lt;li&gt;Resend for emailing&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;All tech stacks are universally accepted and open-source, making it robust by nature. &lt;/p&gt;

&lt;h1&gt;
  
  
  Conclusion
&lt;/h1&gt;

&lt;p&gt;Directory listing websites are in trend, and making one is a good way for you to earn money online. ToolDir ease the pain in making the entire website and saved almost $2k as a developer's costs; it works well with AI agents like Cursor, v0 or Lovable or Bolt.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://tooldir.buildsaas.dev/" rel="noopener ugc nofollow noreferrer"&gt;Check the demo&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://tooldir.buildsaas.dev/" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://shreyvijayvargiya.gumroad.com/l/nextjs-supabase-directory-boilerplate?layout=profile" rel="noopener ugc nofollow noreferrer"&gt;Get the template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Cheers&lt;/p&gt;

&lt;p&gt;Shrey&lt;/p&gt;

&lt;p&gt;If needed, custom template, shoot me an email at &lt;a href="http://shreyvijayvargiya26@gmail.com/" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="mailto:shreyvijayvargiya26@gmail.com"&gt;shreyvijayvargiya26@gmail.com&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>programming</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Making of Aantraa</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Fri, 26 Jun 2026 15:20:30 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/making-of-aantraa-3475</link>
      <guid>https://dev.to/shreyvijayvargiya/making-of-aantraa-3475</guid>
      <description>&lt;h1&gt;
  
  
  Making of Aantraa
&lt;/h1&gt;

&lt;p&gt;&lt;strong&gt;aantraa.site&lt;/strong&gt; — AI audio &amp;amp; video translation, caption generator, and viral shorts cutter.&lt;/p&gt;




&lt;h2&gt;
  
  
  Under the Hood
&lt;/h2&gt;

&lt;p&gt;I run a small YouTube channel. I'm not a full-time content creator, but YouTube is a solid platform to gain traffic for your online work, business, project, or idea.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Aantraa&lt;/strong&gt; is what I built in a week. The main concept is simple:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Video translation&lt;/strong&gt; into multiple languages&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audio translation&lt;/strong&gt; — including text-to-audio, with MP3 output for Premiere Pro&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Long-form to shorts&lt;/strong&gt; — convert YouTube long-form video into short clips&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;At that time, only three features were needed, so website development wasn't the heavy lift. The real work was building APIs, backend infrastructure to integrate AI into video, and dealing with heavy storage.&lt;/p&gt;




&lt;h2&gt;
  
  
  Breaking the execution into steps: How I made Aantraa
&lt;/h2&gt;

&lt;h3&gt;
  
  
  AI LLM layering and provider
&lt;/h3&gt;

&lt;p&gt;Aantraa is heavily dependent on AI APIs — we need reliable infrastructure for LLM providers.&lt;/p&gt;

&lt;p&gt;OpenRouter, Portkey, Vercel AI SDK labs, and individual APIs for Anthropic, Deepseek, and OpenAI are solid options.&lt;/p&gt;

&lt;p&gt;I prefer &lt;strong&gt;OpenRouter&lt;/strong&gt; for Aantraa for one reason: &lt;strong&gt;multiple model support&lt;/strong&gt; — it's easy to pick the cheapest capable model for each job. Easy to integrate, strong community support, free model access, and more.&lt;/p&gt;

&lt;p&gt;AI LLM APIs are needed at almost every stage in the backend:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Understanding video context and creating a script&lt;/li&gt;
&lt;li&gt;Translating the script into target languages&lt;/li&gt;
&lt;li&gt;Recording the script into MP3 or WAV format&lt;/li&gt;
&lt;li&gt;Summarising the video&lt;/li&gt;
&lt;li&gt;Generating captions&lt;/li&gt;
&lt;li&gt;Cutting videos into shorts&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Building APIs and servers
&lt;/h3&gt;

&lt;p&gt;Each layer needs heavy AI context and prompt engineering. &lt;strong&gt;Loop engineering&lt;/strong&gt; is the trend here — and it's required for aantraa.&lt;/p&gt;

&lt;p&gt;For example, &lt;strong&gt;video translation&lt;/strong&gt; works in multiple connected steps:&lt;/p&gt;

&lt;h4&gt;
  
  
  Video translation API breakdown
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;AI understands the video, fed into the LLM via the &lt;strong&gt;ffmpeg&lt;/strong&gt; module&lt;/li&gt;
&lt;li&gt;AI generates a script/caption from the video&lt;/li&gt;
&lt;li&gt;AI translates the script into the desired language&lt;/li&gt;
&lt;li&gt;AI generates audio (MP3 or WAV) of the new translation&lt;/li&gt;
&lt;li&gt;AI glues audio and video together using ffmpeg&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each step depends on the previous one, which makes production debugging hard when something breaks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Solution:&lt;/strong&gt; Track each process — usage tokens, estimated time, errors, and response metadata.&lt;/p&gt;

&lt;p&gt;The same pattern applies to audio translation, viral clip cutter, and caption generator.&lt;/p&gt;

&lt;h3&gt;
  
  
  Infrastructure and servers
&lt;/h3&gt;

&lt;p&gt;Local API development is manageable until you ship to production.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Layer&lt;/th&gt;
&lt;th&gt;Choice&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Framework&lt;/td&gt;
&lt;td&gt;Hono.js&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Backend hosting&lt;/td&gt;
&lt;td&gt;Vercel Edge or Fly.io&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DevOps&lt;/td&gt;
&lt;td&gt;Docker, simple Git CI/CD&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Database&lt;/td&gt;
&lt;td&gt;Firebase / Supabase&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Storage&lt;/td&gt;
&lt;td&gt;UploadThing&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;I found &lt;strong&gt;UploadThing&lt;/strong&gt; as a practical alternative to AWS S3 and Firebase/Supabase storage for file uploads. It provides client and server SDKs to upload files quickly (5 MB per chunk on the free plan).&lt;/p&gt;

&lt;p&gt;We need storage heavily because every AI layer doesn't keep its own memory — every generated audio/video file must land in storage.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;FFmpeg&lt;/strong&gt; is essential for video and audio work, but it has limitations on serverless functions and Vercel Edge. That pushed us toward &lt;strong&gt;Fly.io&lt;/strong&gt;, Railway, or Render for heavier media workloads.&lt;/p&gt;




&lt;h2&gt;
  
  
  Video translation into 90+ languages
&lt;/h2&gt;

&lt;p&gt;Aantraa supports &lt;strong&gt;90+ languages&lt;/strong&gt; for video and audio translation.&lt;/p&gt;

&lt;p&gt;AI translates scripts, text, on-screen text, and video context well — it needs to understand the video through the script and each frame. &lt;strong&gt;FFmpeg&lt;/strong&gt; helps with that pipeline.&lt;/p&gt;

&lt;p&gt;The flow:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;AI generates a script from the source&lt;/li&gt;
&lt;li&gt;AI translates into the target language&lt;/li&gt;
&lt;li&gt;AI creates dubbed audio in that language&lt;/li&gt;
&lt;li&gt;FFmpeg merges audio and video into a new translated file&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each step needs debugging, prompt engineering, and FFmpeg integration.&lt;/p&gt;

&lt;p&gt;Finally, the output uploads to storage and returns a URL to the client for download and playback.&lt;/p&gt;




&lt;h2&gt;
  
  
  Audio to MP3 in multiple languages
&lt;/h2&gt;

&lt;p&gt;Aantraa isn't only video translation. As a creator, I also wanted to turn blog posts or text into audio — podcast-style listening.&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;audio translation&lt;/strong&gt; tool covers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Text to audio&lt;/strong&gt; — download MP3 or WAV&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;90+ languages&lt;/strong&gt; for text and video sources&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Video to audio&lt;/strong&gt; extraction and translation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multiple target languages&lt;/strong&gt; in parallel&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That makes aantraa a supporting platform: one recording → 90+ language MP3s, ready for one-click sharing.&lt;/p&gt;




&lt;h2&gt;
  
  
  YouTube videos to shorts
&lt;/h2&gt;

&lt;p&gt;The viral shorts feature converts long-form YouTube video into short clips you can upload directly to your channel.&lt;/p&gt;

&lt;h4&gt;
  
  
  API breakdown
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;AI understands the full video context — summary and script&lt;/li&gt;
&lt;li&gt;AI breaks the script by timestamp into the desired number of shorts&lt;/li&gt;
&lt;li&gt;FFmpeg cuts each clip; APIs upload to storage&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;It sounds simple, but production needs FFmpeg tuning, AI context limits, and file-size guardrails — videos over ~10 MB cost more time and money to process.&lt;/p&gt;




&lt;h2&gt;
  
  
  Video translation examples
&lt;/h2&gt;

&lt;p&gt;See &lt;strong&gt;10+ translated videos&lt;/strong&gt; in Spanish, Hindi, Bengali, Gujarati, Marathi, Tamil, French, English, Japanese, Chinese, and more on our &lt;a href="https://aantraa.site/examples" rel="noopener noreferrer"&gt;examples page&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;90+ languages make aantraa a universal, global platform.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;The &lt;strong&gt;first version is live&lt;/strong&gt;. We also offer &lt;strong&gt;APIs for business agencies and startup teams&lt;/strong&gt; — reach out via &lt;a href="https://aantraa.site/contact" rel="noopener noreferrer"&gt;contact&lt;/a&gt; if you're interested.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Links&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Website: &lt;a href="https://aantraa.site" rel="noopener noreferrer"&gt;aantraa.site&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Blog: &lt;a href="https://aantraa.site/blog" rel="noopener noreferrer"&gt;aantraa.site/blog&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Examples: &lt;a href="https://aantraa.site/examples" rel="noopener noreferrer"&gt;aantraa.site/examples&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Pricing: &lt;a href="https://aantraa.site/#pricing" rel="noopener noreferrer"&gt;aantraa.site/#pricing&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Try the product and share your feedback — early signups get &lt;strong&gt;1 free minute&lt;/strong&gt; of AI translation credit.&lt;/p&gt;

&lt;p&gt;Cheers,&lt;br&gt;&lt;br&gt;
&lt;strong&gt;Shrey&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Aantraa · &lt;a href="https://aantraa.site" rel="noopener noreferrer"&gt;aantraa.site&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>I built a CRM Template Nobody Asked For Until 10+ Businesses Bought It</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Wed, 17 Jun 2026 13:58:27 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/i-built-a-crm-template-nobody-asked-for-until-10-businesses-bought-it-33p8</link>
      <guid>https://dev.to/shreyvijayvargiya/i-built-a-crm-template-nobody-asked-for-until-10-businesses-bought-it-33p8</guid>
      <description>&lt;h1&gt;
  
  
  I Built a CRM Template Nobody Asked For — Then 10+ Businesses Bought It
&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;How shipping something "boring" taught me more about product-market fit than any hot SaaS idea ever did.&lt;/em&gt;&lt;/p&gt;




&lt;p&gt;I almost didn't build this.&lt;/p&gt;

&lt;p&gt;CRMs are not sexy. Nobody tweets about their CRM. No one posts "just shipped a killer role-based access control system" and goes viral. It's the kind of product you build because you have to, not because you want to.&lt;/p&gt;

&lt;p&gt;But that's exactly why I built it — and why it worked.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Problem Nobody Talks About
&lt;/h2&gt;

&lt;p&gt;When I started working on a CRM project, I quickly realised something: &lt;strong&gt;businesses don't just need a CRM, they need a CRM that fits their workflow.&lt;/strong&gt; Off-the-shelf tools are either too generic or too expensive to customise. And building one from scratch? That's a rabbit hole most teams don't have the time or budget to fall into.&lt;/p&gt;

&lt;p&gt;CRMs are quite complex:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;20+ pages of UI that all need to communicate with each other&lt;/li&gt;
&lt;li&gt;Role-based access control for different team members&lt;/li&gt;
&lt;li&gt;API integrations with third-party tools&lt;/li&gt;
&lt;li&gt;Version control for data schemas&lt;/li&gt;
&lt;li&gt;Authentication, permissions, audit trails&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  It's not just a CRUD app. It's a whole product surface.
&lt;/h2&gt;

&lt;h2&gt;
  
  
  Version 1: Ship Ugly, Ship Fast
&lt;/h2&gt;

&lt;p&gt;The first version was not impressive. Here's what I launched:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;10+ pages&lt;/strong&gt; of core CRM UI (contacts, deals, pipeline, tasks, settings)&lt;/li&gt;
&lt;li&gt;Basic &lt;strong&gt;authentication flow&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;API database schemas&lt;/strong&gt; for the most common use cases&lt;/li&gt;
&lt;li&gt;Frontend UI components built with React and Tailwind&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That's it. No documentation. No Figma files. No localization.&lt;/p&gt;

&lt;p&gt;I put it on Gumroad, shared it with a small developer community I'd been building, and waited.&lt;/p&gt;




&lt;h2&gt;
  
  
  The First Sale
&lt;/h2&gt;

&lt;p&gt;One month later, someone bought it.&lt;/p&gt;

&lt;p&gt;Not a big company. Not a VC-backed startup. Just a small team that needed a CRM and didn't want to spend six months building one from scratch. They paid, they used it, and then — this is the part that changes everything — &lt;strong&gt;they gave me feedback.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That feedback was worth more than the sale price.&lt;/p&gt;

&lt;p&gt;They told me what was missing. What didn't work the way they expected. What they wished was there. And I built it.&lt;/p&gt;




&lt;h2&gt;
  
  
  Three Months of Compounding Improvements
&lt;/h2&gt;

&lt;p&gt;Here's what the template looked like three months after the first sale:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;More pages, production-grade quality&lt;/strong&gt;&lt;br&gt;
From 10 to &lt;strong&gt;22 fully functional pages&lt;/strong&gt; — including analytics dashboards, activity logs, user management, reporting views, and more. Not just wireframes. Actual working pages with real data flow.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Proper schema support&lt;/strong&gt;&lt;br&gt;
Developers have different preferences and constraints. I added schema definitions for &lt;strong&gt;Zod, Supabase, Firebase, and Postgres&lt;/strong&gt; so the template could slot into almost any backend stack without a full rewrite.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Documentation and multi-language support&lt;/strong&gt;&lt;br&gt;
The first version had zero documentation. That's fine for an MVP, but not for a product you're selling to businesses. I wrote proper docs, added setup guides, and built in multi-language support for international teams.&lt;/p&gt;


&lt;h2&gt;
  
  
  What 10+ Businesses Taught Me
&lt;/h2&gt;

&lt;p&gt;By month four, the template had helped &lt;strong&gt;10+ businesses worldwide&lt;/strong&gt; — from solo founders to small agencies to companies building internal tools.&lt;/p&gt;

&lt;p&gt;Each sale brought a new perspective. The feedback loop looked like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Sale → Feedback → Improvement → Better product → Next sale
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few things surprised me:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The "boring" parts mattered most.&lt;/strong&gt; Nobody cared about the UI animations. They cared about whether the schema worked with their database, whether the auth was secure, whether their developers could understand the codebase in an afternoon.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Customization was the real ask.&lt;/strong&gt; Several clients didn't just want the template — they wanted us to customize it for their existing business. The template became a starting point for consulting work. One product, two revenue streams.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Figma files were a latent need.&lt;/strong&gt; One client specifically requested Figma files. This wasn't something I'd planned for. But it made sense — designers need to adapt the UI before developers touch the code. Now it's part of the offering.&lt;/p&gt;




&lt;h2&gt;
  
  
  What's in the Template Today
&lt;/h2&gt;

&lt;p&gt;For developers and teams evaluating this, here's the current state:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Feature&lt;/th&gt;
&lt;th&gt;Details&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Pages&lt;/td&gt;
&lt;td&gt;22 production-grade pages&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Auth&lt;/td&gt;
&lt;td&gt;Full authentication with role-based access&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Schemas&lt;/td&gt;
&lt;td&gt;Zod, Supabase, Firebase, Postgres&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Frontend&lt;/td&gt;
&lt;td&gt;React + Tailwind CSS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Documentation&lt;/td&gt;
&lt;td&gt;Full setup guide included&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Language support&lt;/td&gt;
&lt;td&gt;Multi-language ready&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Figma files&lt;/td&gt;
&lt;td&gt;Available on request&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The goal was always to give teams a &lt;strong&gt;90% head start&lt;/strong&gt; so they could spend their engineering time on what makes their business unique — not reinventing contact management for the hundredth time.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Boring products solve real problems.&lt;/p&gt;

&lt;p&gt;If you're a developer thinking about what to build next, consider: &lt;strong&gt;what problem is so unsexy that nobody is building a great solution for it?&lt;/strong&gt; That gap is usually where the money is.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;If you're building a CRM, need a CRM for your business, or want a starting point for a custom internal tool:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Live demo:&lt;/strong&gt; &lt;a href="http://crm-saas.buildsaas.dev" rel="noopener noreferrer"&gt;crm-saas.buildsaas.dev&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Documentation:&lt;/strong&gt; &lt;a href="http://crm-saas.buildsaas.dev/about" rel="noopener noreferrer"&gt;crm-saas.buildsaas.dev/about&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Buy (USD):&lt;/strong&gt; &lt;a href="https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template" rel="noopener noreferrer"&gt;Gumroad&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Buy (INR):&lt;/strong&gt; &lt;a href="https://checkout.dodopayments.com/buy/pdt_0NdFsoIah9Nc7oQ5SZScQ" rel="noopener noreferrer"&gt;Dodo Payments&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;DM me on &lt;a href="https://x.com/treyvijay" rel="noopener noreferrer"&gt;@treyvijay&lt;/a&gt; or mention in the comment section.&lt;/p&gt;

&lt;p&gt;Cheers&lt;br&gt;
Shrey&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>learning</category>
      <category>showdev</category>
    </item>
    <item>
      <title>Aantraa: AI Audio Video Translation Tool</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Wed, 10 Jun 2026 17:10:16 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/aantraa-ai-audio-video-translation-tool-4mgl</link>
      <guid>https://dev.to/shreyvijayvargiya/aantraa-ai-audio-video-translation-tool-4mgl</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo0hkjimqj2amrwflehy2.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fo0hkjimqj2amrwflehy2.png" width="700" height="350"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Under the Hood&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Global internet consumers will reach 8 billion users by 2030.&lt;/p&gt;

&lt;p&gt;The Internet is flooded with videos and texts from people all over the world.&lt;/p&gt;

&lt;p&gt;In future, every 4 in one person will be an internet creator in some or the other ways. Not every creator is a YouTuber; some are on X, IG, TikTok, Pinterest, Reddit, Substack and LinkedIn.&lt;/p&gt;

&lt;p&gt;In India, we have billions of consumers, in total 1.5, each one in his/her naqtive language consumes the internet.&lt;/p&gt;

&lt;p&gt;Distributing the content needs a start of creating content in the native language. AI models are certainly a good choice to create content in native languages or, more precisely, to translate content.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Introducing Aantraa&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Aantraa is an AI audio-video translation platform.&lt;/p&gt;

&lt;p&gt;Aantraa provides 190+ language translations for your next video or audio.&lt;/p&gt;

&lt;p&gt;Dub one video into 90+ languages with cloned voices.&lt;/p&gt;

&lt;p&gt;Aantraa turns video, audio, or text into localised versions in minutes.&lt;/p&gt;

&lt;p&gt;No studio. No back-and-forth. No one-language-at-a-time workflow.&lt;/p&gt;

&lt;p&gt;If your content travels, your voice should too.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fymdd1bf66ylfnd4gnb0y.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fymdd1bf66ylfnd4gnb0y.png" width="700" height="349"&gt;&lt;/a&gt;&lt;a href="http://aantraa.site/" rel="noopener noreferrer"&gt;Aantraa video/audio translation form website screenshot&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I built aantraa to curb the need and demand of content creators across the globe to translate the content from Russian, German, Chinese, and English to Hindi for 1.5 billion people and vice versa.&lt;/p&gt;

&lt;p&gt;It's the simplest tool that takes a URL or uploads the video and selects the language of translation, and within minutes, aantraa translates the video.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Aantraa by Examples&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Video posted in English on &lt;a href="http://x.com" rel="noopener noreferrer"&gt;x.com&lt;/a&gt;, and below is the translated video URL in Hindi.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://x.com/elonmusk/status/2053230433299771580?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E2053230433299771580%7Ctwgr%5E%7Ctwcon%5Es1_&amp;amp;ref_url=" rel="noopener noreferrer"&gt;https://x.com/elonmusk/status/2053230433299771580?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E2053230433299771580%7Ctwgr%5E%7Ctwcon%5Es1_&amp;amp;ref_url=&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://b4fcijccdw.ufs.sh/f/mVUSE925dTRYdk7jrjutB0mAHE68ivjy7GN4WZpIoQk2Ydcg" rel="noopener noreferrer"&gt;https://b4fcijccdw.ufs.sh/f/mVUSE925dTRYdk7jrjutB0mAHE68ivjy7GN4WZpIoQk2Ydcg&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Aantraa works with 190+ languages for translation, as shown below on the website.&lt;/p&gt;

&lt;p&gt;&lt;a href="http://aantraa.site/" rel="noopener noreferrer"&gt;Check more examples on the website&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;How it Works&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;This is what 3 minutes buys you:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Upload a video.&lt;/li&gt;
&lt;li&gt;Pick 5 languages.&lt;/li&gt;
&lt;li&gt;Get voice-cloned dubbed exports in one queue.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most teams still localise like it’s 2014.&lt;/p&gt;

&lt;p&gt;We built aantraa so one file can become a whole market.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Dubbing voice-over with 20+ male/female voices&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Aantraa provides 190+ languages and over 20 male/female voice-overs to select the dubbed voice.&lt;/p&gt;

&lt;p&gt;Press enter or click to view image in full size&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmty3vxntb1v99giaotee.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmty3vxntb1v99giaotee.png" width="700" height="349"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Why Aantraa&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;We kept hearing the same complaint from creators and teams:&lt;/p&gt;

&lt;p&gt;“We don’t need a studio. We need multilingual output by Friday.”&lt;/p&gt;

&lt;p&gt;So we built aantraa around speed, parallel jobs, and private uploads.&lt;/p&gt;

&lt;p&gt;Less process. More published content.&lt;/p&gt;

&lt;p&gt;Stop dubbing one language at a time.&lt;/p&gt;

&lt;p&gt;aantraa lets you run parallel jobs for Spanish, Hindi, French, and more from the same source file.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Who can use Aantraa&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Built for creators, agencies, and teams that need localised content without the production overhead.&lt;/p&gt;

&lt;p&gt;Cheaper than a full dubbing workflow.&lt;/p&gt;

&lt;p&gt;That’s the best.&lt;/p&gt;

&lt;p&gt;If you’re a creator, agency, or content team, paying per minute for fast self-serve dubbing is easier than coordinating translators, voice talent, and editors.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;strong&gt;Conclusion&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;There are already strong tools in the dubbing space.&lt;/p&gt;

&lt;p&gt;That’s exactly why we built &lt;a href="http://aantraa.site/" rel="noopener noreferrer"&gt;aantraa&lt;/a&gt; with a different angle.&lt;/p&gt;

&lt;p&gt;Not a bloated media suite.\&lt;br&gt;
Not a consulting-heavy service.\&lt;br&gt;
Not a workflow that assumes you have a production team.&lt;/p&gt;

&lt;p&gt;Just a fast, self-serve way to turn one video, audio file, or script into localised voice output in 90+ languages.&lt;/p&gt;

&lt;p&gt;The core idea is boring in the best way: Make the first translation easy, then make the next four translations basically free.&lt;/p&gt;

&lt;p&gt;That matters for creators, agencies, and content teams that need to ship fast and keep margins sane.&lt;/p&gt;

&lt;p&gt;We’ll keep improving quality, timing, and export workflow.\&lt;br&gt;
But the north star stays the same: help one recording do the work of many.&lt;/p&gt;

&lt;p&gt;More in the next blog!!&lt;/p&gt;

&lt;p&gt;Go ahead, try the product, early users get the FREE one-minute credit!!&lt;/p&gt;

&lt;p&gt;Cheers\&lt;br&gt;
Shrey\&lt;br&gt;
&lt;a href="http://aantraa.site/" rel="noopener noreferrer"&gt;Aantraa&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>programming</category>
      <category>saas</category>
    </item>
    <item>
      <title>Scrapefast: Smallest and fastest FREE web scraping API</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Mon, 11 May 2026 14:26:15 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/scrapefast-smallest-and-fastest-free-web-scraping-api-57le</link>
      <guid>https://dev.to/shreyvijayvargiya/scrapefast-smallest-and-fastest-free-web-scraping-api-57le</guid>
      <description>&lt;p&gt;Hello folks!!&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/shreyvijayvargiya/scrapefast/" rel="noopener noreferrer"&gt;https://github.com/shreyvijayvargiya/scrapefast/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I’ve built the smallest and fastest web scraping free open-source Node. js-based endpoint.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Handles concurrent multiple URL scraping&lt;/li&gt;
&lt;li&gt;Manage memory allocation, cache cleaning&lt;/li&gt;
&lt;li&gt;API endpoints for scraping, screenshot for a single URL and multiple URL
Most of the AI LLMs need internet access and web scraping for reading content from a website/URL, and a web scraping endpoint will help in the same way&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Wrote a detailed blog about it, read blog: &lt;a href="https://ihatereading.in/t/One-honojs-file-for-entire-web-scraping-API" rel="noopener noreferrer"&gt;https://ihatereading.in/t/One-honojs-file-for-entire-web-scraping-API&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Feel free to contribute&lt;br&gt;
Cheers&lt;br&gt;
Shrey&lt;/p&gt;

</description>
      <category>github</category>
      <category>opensource</category>
      <category>webdev</category>
      <category>javascript</category>
    </item>
    <item>
      <title>Build any SAAS website in just 299Rs/$49</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Thu, 30 Apr 2026 16:16:12 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/build-any-saas-website-in-just-299rs49-117a</link>
      <guid>https://dev.to/shreyvijayvargiya/build-any-saas-website-in-just-299rs49-117a</guid>
      <description>&lt;p&gt;Hello folks!!&lt;/p&gt;

&lt;p&gt;I am glad to announce that buildsaas.dev now supports INR pricing and UPI&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Buildsaas?
&lt;/h2&gt;

&lt;p&gt;Most of the vibe coders and indie hackers build SaaS products, add payments, authentication, and databases, and finally launch in months, only to realise that the idea is not working.&lt;/p&gt;

&lt;p&gt;Instead, save some time by using boilerplate or an already existing code repository or template.&lt;/p&gt;

&lt;p&gt;But we often get confused about which template to use, and most of the templates are too costly stating from $99.&lt;/p&gt;

&lt;p&gt;Buildsaas solves this problem. We have already used buildsaas for existing products in the market, such as ihatereading.in, gettemplate. website, inkgest.com, kixi.app and more&lt;/p&gt;

&lt;p&gt;All of our existing products or new upcoming ones are and will be built using buildsaas&lt;/p&gt;

&lt;p&gt;It's a starter kit that has payments, authentication, an admin dashboard to write blogs, manage users, visitors, customers, create invoices and record payments and a lot more. In fact, everything can be done on one admin dashboard, and this all-in-one starter kit is for just 299Rs (Offer is valid only for the next 10 users)&lt;/p&gt;

&lt;p&gt;Buildsaas will save 150+ development or vibe-coding hours, almost $100 in total for those AI coding tools and building such applications will cost around $1k and more&lt;/p&gt;

&lt;h2&gt;
  
  
  Get the Buildsaas
&lt;/h2&gt;

&lt;p&gt;Do check the demo: &lt;a href="https://demo-app.buildsaas.dev/" rel="noopener noreferrer"&gt;https://demo-app.buildsaas.dev/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Visit the admin panel you will get once you buy the buildsaas: &lt;a href="https://demo-app.buildsaas.dev/admin" rel="noopener noreferrer"&gt;https://demo-app.buildsaas.dev/admin&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Purchase link: &lt;a href="https://checkout.dodopayments.com/buy/pdt_0NV5cXHH4abz83r7XX5dv?session=sess_NrwOKiVN6y" rel="noopener noreferrer"&gt;https://checkout.dodopayments.com/buy/pdt_0NV5cXHH4abz83r7XX5dv?session=sess_NrwOKiVN6y&lt;/a&gt; &lt;/p&gt;

&lt;p&gt;Buy in USD: &lt;a href="https://shreyvijayvargiya.gumroad.com/l/build-saas-starter-boilerplate?layout=profile" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/build-saas-starter-boilerplate?layout=profile&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Once the payment is made, you get all types of zip files of the starter kit as explained below&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Nextjs, Firebase, Stripe&lt;/li&gt;
&lt;li&gt;Nextjs, Firebase, Polar&lt;/li&gt;
&lt;li&gt;Nextjs Supabase, Stripe&lt;/li&gt;
&lt;li&gt;Nextjs Supabase Polar&lt;/li&gt;
&lt;li&gt;Blog starter kit template&lt;/li&gt;
&lt;li&gt;Newsletter starterkit template&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Cheers&lt;br&gt;
DM for anything you need&lt;br&gt;
Shrey&lt;/p&gt;

&lt;p&gt;Buildsaas.dev&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Free Open-Source Documentation Creator Tool</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Wed, 29 Apr 2026 16:22:10 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/free-open-source-documentation-creator-tool-3204</link>
      <guid>https://dev.to/shreyvijayvargiya/free-open-source-documentation-creator-tool-3204</guid>
      <description>&lt;p&gt;Hey folks!!&lt;/p&gt;

&lt;p&gt;Welcome to the new blog&lt;/p&gt;

&lt;p&gt;I was building a product, buildsaas.dev and writing documentation for it. I tried to find alternatives on the internet, find Fumadocs, Docusaurus and Mintlify. &lt;/p&gt;

&lt;p&gt;But they have a loophole, each of them not serving the way I want it. I am required to create a simple Nextjs React repository that helps me to create documentation for any SAAS product in future.&lt;/p&gt;

&lt;p&gt;This repository should also help with the following features, as defined&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Easy to download, install the repository, and packages&lt;/li&gt;
&lt;li&gt;Markdown support and static content rendering for SEO&lt;/li&gt;
&lt;li&gt;AI chatbot and language support integration&lt;/li&gt;
&lt;li&gt;Dark/light theme&lt;/li&gt;
&lt;li&gt;Latest tech stack&lt;/li&gt;
&lt;li&gt;Works well with AI code editors&lt;/li&gt;
&lt;li&gt;One-click deploy on Github&lt;/li&gt;
&lt;li&gt;Responsive webpage with an editor that renders all UI types of components&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Most of the tools I've found are either not free or do not provide a code repository&lt;/p&gt;

&lt;p&gt;That is why I created this documentation creator website; it's free and open-source.&lt;/p&gt;

&lt;p&gt;All instructions on installations are in the repository; no third-party APIs are needed, only frontend packages&lt;/p&gt;

&lt;h1&gt;
  
  
  How to use
&lt;/h1&gt;

&lt;ul&gt;
&lt;li&gt;Clone the repository&lt;/li&gt;
&lt;li&gt;Edit the markdown content .md files or add new ones&lt;/li&gt;
&lt;li&gt;Edit the sidebar if adding a new route, page, or content&lt;/li&gt;
&lt;li&gt;Push to Github&lt;/li&gt;
&lt;li&gt;Deploy to vercel&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It's literally that simple, FREE and easy to customise.&lt;/p&gt;

&lt;p&gt;Github: &lt;a href="https://github.com/shreyvijayvargiya/docs-template" rel="noopener noreferrer"&gt;https://github.com/shreyvijayvargiya/docs-template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Demo: &lt;a href="https://docs-template-henna.vercel.app/" rel="noopener noreferrer"&gt;https://docs-template-henna.vercel.app/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Cheers&lt;br&gt;
Shrey&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>opensource</category>
      <category>github</category>
      <category>documentation</category>
    </item>
    <item>
      <title>20 Stunning Websites for Design Engineers</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Fri, 24 Apr 2026 11:53:10 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/20-stunning-websites-for-design-engineers-3hio</link>
      <guid>https://dev.to/shreyvijayvargiya/20-stunning-websites-for-design-engineers-3hio</guid>
      <description>&lt;p&gt;Hey there!!&lt;/p&gt;

&lt;p&gt;Welcome to the new blog.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;20 good websites for design engineers&lt;/strong&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  🔥 Websites for Landing Page Inspiration
&lt;/h1&gt;

&lt;h3&gt;
  
  
  Tigran Z
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://tigranz.com/" rel="noopener noreferrer" class="c-link"&gt;
            Tigran Azatyan | Design &amp;amp; Motion
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Motion designer working across brand, product, interaction, and advertising—bringing ideas to life through expressive, purposeful movement.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FncYusKOUYX8RA9erL7i1RkFBWs.png" width="568" height="555"&gt;
          tigranz.com
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Skale
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://skale.solutions/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fassets%2F1dJ1fydf7AAhIa2id5vxb1SIbx0.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://skale.solutions/" rel="noopener noreferrer" class="c-link"&gt;
            Skale - Home
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            We'll create a websites that fulfills your business goals. High-converting design solutions for SaaS, Startups, Agencies, and Digital Creators
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FIwncHZSdHBWaIoCWRaiKx8KsTG0.png" width="64" height="64"&gt;
          skale.solutions
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Supa Power
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://supa-power.framer.website/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fassets%2FwKObYhpYiNW2SLCqjPgDarcn2I0.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://supa-power.framer.website/" rel="noopener noreferrer" class="c-link"&gt;
            Supapower Framer Template
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Supapower Framer Template. Made for web and mobile app startups.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2Ft3qdQCHBtCGO7FgFRHSKnJFveyQ.svg" width="64" height="64"&gt;
          supa-power.framer.website
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Menoa Template
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://menoa-template.framer.website/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fassets%2F9GC652224eLAjWWcJMJLmOLoK5E.jpg" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://menoa-template.framer.website/" rel="noopener noreferrer" class="c-link"&gt;
            Menoa – Framer Portfolio Template
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Menoa is a sleek and modern portfolio template built for freelance designers, creatives, and small agencies. It's clean, fast to set up, and crafted to help you look professional and win better clients.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FByItZ81Ct2XRi1rvCUBnGcIoRQ.png" width="64" height="64"&gt;
          menoa-template.framer.website
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  CMD Supply
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.cmd.supply/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FJb3iJ1JmyVShdPqNi9bAmfxzg.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.cmd.supply/" rel="noopener noreferrer" class="c-link"&gt;
            CMD Supply | Premium Framer Templates
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Build and launch beautiful, responsive websites in minutes with our easy-to-use templates. Built for Framer, no code needed.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2Fw4cuzTn9eCPaL3DQnWylmwn0kVs.png" width="64" height="64"&gt;
          cmd.supply
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Budarina Studio
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://budarina.studio/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fassets%2F3JpfnTf7ymBRZdHQTlRDzKUINc.png" height="421" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://budarina.studio/" rel="noopener noreferrer" class="c-link"&gt;
            Budarina Studio - premium design solutions for busy founders.
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            High-converting solutions for tech and AI companies to build revenue. 
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FCQZxg9idcWIGsdJeNKsEgbzGKLM.png" width="80" height="80"&gt;
          budarina.studio
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Viral SMA
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://viral-sma.framer.website/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FKfEvJD0tcOXIe2Plh2HpSUHIGmw.jpg" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://viral-sma.framer.website/" rel="noopener noreferrer" class="c-link"&gt;
            Viral - Social Media Agency Website Template for Framer
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Viral is a premium Framer template built for social media agencies that want to look credible, results-driven, and conversion-focused from day one.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FTQzw53CcJfN5rlajBHEdbvKlis.png" width="320" height="320"&gt;
          viral-sma.framer.website
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Slush
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://slush.app/" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;slush.app&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Buildkits
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.buildkits.dev/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.buildkits.dev%2Fcommon%2Fopengraph.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.buildkits.dev/" rel="noopener noreferrer" class="c-link"&gt;
            SaaS Boilerplates, starter kits, and design systems | Buildkits
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Whether you're a start-up trying to validate ideas quickly or an established company 
        looking for scalable solutions, you can find the perfect tools to meet your needs right here!
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.buildkits.dev%2Ffavicon-32x32.png" width="32" height="32"&gt;
          buildkits.dev
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Just Fucking Use Astro
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://justfuckinguseastro.com/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjustfuckinguseastro.com%2Fopengraph%2Ffacebook.png" height="315" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://justfuckinguseastro.com/" rel="noopener noreferrer" class="c-link"&gt;
            Just Fucking Use Astro
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            A public service announcement for developers still building websites like it's 2012 or shipping full SPAs for marketing sites. Most sites are content plus a few interactive bits. Just fucking use Astro.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjustfuckinguseastro.com%2Ffavicon.ico" width="800" height="400"&gt;
          justfuckinguseastro.com
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Napkin AI Blog
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.napkin.ai/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.napkin.ai%2Fassets%2Fog-image-v5.png%3Fv%3D2" height="630" class="m-0" width="1199"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.napkin.ai/" rel="noopener noreferrer" class="c-link"&gt;
            Napkin AI - The visual AI for business storytelling
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Just type, copy-paste or generate your text and Napkin will instantly transform it into insightful visuals. Make your communication more effective with Napkin.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.napkin.ai%2Ffavicon-32x32.png%3Fv%3D2" width="97" height="97"&gt;
          napkin.ai
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  World Monitor
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.worldmonitor.app/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.worldmonitor.app%2Ffavico%2Fog-image.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.worldmonitor.app/" rel="noopener noreferrer" class="c-link"&gt;
            World Monitor - Real-Time Global Intelligence Dashboard
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Real-time global intelligence dashboard with live news, markets, military tracking, infrastructure monitoring, and geopolitical data. OSINT in one view.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.worldmonitor.app%2Ffavico%2Ffavicon.ico" width="16" height="16"&gt;
          worldmonitor.app
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Planetono
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.planetono.space/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fa.storyblok.com%2Ff%2F287985154612015%2F1200x630%2Fbe575015cd%2Fog.png%3Fcv%3D1777031240766" height="630" class="m-0" width="1200"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.planetono.space/" rel="noopener noreferrer" class="c-link"&gt;
            Planetoño
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            We tried making this real. NASA said No. So for now, it's just a delicious concept by Tubik.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fa.storyblok.com%2Ff%2F287985154612015%2F96x96%2F2f0831a892%2Ffavicon-96x96.png" width="96" height="96"&gt;
          planetono.space
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Context.dev
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://www.context.dev/" rel="noopener noreferrer"&gt;https://www.context.dev/&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Keytomic
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://keytomic.com/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fassets%2Ff83kJjm7y60ImMlWSgUj7UPTRc.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://keytomic.com/" rel="noopener noreferrer" class="c-link"&gt;
            Keytomic – Rank on LLMs and Search Engines on Autopilot
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Keytomic is an all-in-one AI SEO and content tool that helps brands rank on LLMs and search engines without manual efforts or guesswork.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2F6sjio8w7vtmvievJPhYd8ZXtvo.png" width="470" height="470"&gt;
          keytomic.com
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Outrank
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.outrank.so/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.outrank.so%2Fopengraph-image.png%3F31d914fdfd6b5383" height="1200" class="m-0" width="2400"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.outrank.so/" rel="noopener noreferrer" class="c-link"&gt;
            Outrank - Grow Organic Traffic on Auto-Pilot
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Get traffic and outrank competitors with Backlinks &amp;amp; SEO-optimized content while you sleep.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.outrank.so%2Ficon.png%3Fb2cf51a3eea7f48a" width="192" height="192"&gt;
          outrank.so
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Sixtyfour AI
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.sixtyfour.ai/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FKhyIWSULnwxK818vjwAjRBurE.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.sixtyfour.ai/" rel="noopener noreferrer" class="c-link"&gt;
            Sixtyfour | Intelligence Infrastructure on People and Entities
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            The Enterprise Data Platform to deploy AI agents that unify social, contact, and proprietary data into decision-ready profiles.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FrHOgjs6iNmyrbTX4ZBczbc6H7WQ.png" width="800" height="800"&gt;
          sixtyfour.ai
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Faces Profile
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://faces.app/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Ffaces.app%2Fopengraph-image.png%3Fopengraph-image.0.35pr0nnnga..png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://faces.app/" rel="noopener noreferrer" class="c-link"&gt;
            Faces: Interactive Presentations
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Create interactive, explorable decks that truly deliver. Made effortlessly with AI.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Ffaces.app%2Ffavicon.ico%3Fv%3D2" width="48" height="48"&gt;
          faces.app
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Creem
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://www.creem.io/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.creem.io%2Fbanner.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://www.creem.io/" rel="noopener noreferrer" class="c-link"&gt;
            Creem
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            Smooth payments for SaaS and Indie Hackers that don't break the bank
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fwww.creem.io%2Ffavicon.ico" width="256" height="256"&gt;
          creem.io
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;h3&gt;
  
  
  Henna
&lt;/h3&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://henna.framer.website/" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FHGiFNKbPKT1En7BngsTOQW2pDE.png" height="420" class="m-0" width="800"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://henna.framer.website/" rel="noopener noreferrer" class="c-link"&gt;
            Henna - Designer &amp;amp; Photographer Portfolio
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            A modern minimal portfolio template for designers and photographers, built with clean layouts, smooth micro-interactions, and fully customizable sections to showcase work with clarity and impact.
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fframerusercontent.com%2Fimages%2FjTv0ZfCAA9XZlrn5rO8N7IpQ.png" width="64" height="64"&gt;
          henna.framer.website
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;Originally published on &lt;a href="//ihatereading.in"&gt;iHateReading&lt;/a&gt;&lt;/p&gt;

</description>
      <category>html</category>
      <category>webdev</category>
      <category>javascript</category>
      <category>frontend</category>
    </item>
    <item>
      <title>One honojs file for entire web scraping API</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Mon, 13 Apr 2026 17:00:31 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/one-honojs-file-for-entire-web-scraping-api-1ni0</link>
      <guid>https://dev.to/shreyvijayvargiya/one-honojs-file-for-entire-web-scraping-api-1ni0</guid>
      <description>&lt;p&gt;Building scrape endpoints using one honojs file&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tags:&lt;/strong&gt; Web Scraping, Github Repository, Scraping, Backend, Honojs&lt;/p&gt;

&lt;p&gt;Hey there!!&lt;/p&gt;

&lt;p&gt;Welcome to the new blog&lt;/p&gt;

&lt;p&gt;Web scraping is always in demand and will be in demand. I've checked tonnes of APIs, including FREE and Paid APIs for web scraping.&lt;/p&gt;

&lt;p&gt;SERP API, Google API, and ScrapingBee all platforms provide APIs for web scraping, but in today's story, we will be building one of our own web scraping APIs&lt;/p&gt;

&lt;p&gt;I'll also provide github repository in the end, which you can use easily to launch your own scraping APIs on vercel, Cloudflare, netlify or, &lt;a href="http://fly.io" rel="noopener noreferrer"&gt;fly.io&lt;/a&gt; or even on a Docker container&lt;/p&gt;

&lt;h2&gt;
  
  
  How does it start?
&lt;/h2&gt;

&lt;p&gt;I was working on &lt;a href="https://inkgest.com" rel="external noopener noreferrer"&gt;inkgest.com&lt;/a&gt;, and we need scraping APIs to scrape the website content legitimately. The above provides options that have a few edge cases, each costs a couple of $$ for just 100 requests per month, and that is not a feasible option&lt;/p&gt;

&lt;p&gt;Inkgest needs to make tonnes of API requests to scrape or do web scraping, and for that, relying on third-party wasn't a good solution, forcing us to make our own scraping APIs&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Why Honojs?&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;We have a couple of options for simple open-source web scraping APIs as a framework to choose from: FastAPI or NestJS. Anything works well, but I want to choose something fast, small in size and easy to deploy anywhere using Docker, so I prefer honojs with bun, which is a new tech stack but a faster one.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;What is Web Scraping?&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;To newcomers, web scraping is a simple API that simulates a browser, opens a website URL or link, and scrapes the content.&lt;/p&gt;

&lt;p&gt;It does more, or can do more, because every scraping algorithm basically uses the Chrome browser, headless or head, except if you are using the LightPanda browser SDK, which does this without using headless Chrome, making it faster than ever before.&lt;/p&gt;

&lt;p&gt;Web scraping is nothing more than an API endpoint whose task is to create a browser instance, load the URL or website, scrape the content or read the DOM elements, and finally parse the content in a human-readable format.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Packages for Web Scraping?&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;A few packages or SDKs that are open-sourced are used in web scraping, as listed below&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Puppeteer&lt;/li&gt;
&lt;li&gt;Playwright&lt;/li&gt;
&lt;li&gt;Cheerio&lt;/li&gt;
&lt;li&gt;Crawlee&lt;/li&gt;
&lt;li&gt;Lightpanda browser&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A lot more options we have, one needs to do a good Google search&lt;/p&gt;

&lt;p&gt;&lt;b&gt;How does a Web Scraping API work?&lt;/b&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Loads the browser instance&amp;nbsp;&lt;/li&gt;
&lt;li&gt;Go to the URL page or web link&lt;/li&gt;
&lt;li&gt;Read the DOM elements (Wait for elements/webpage to load)&lt;/li&gt;
&lt;li&gt;Parse the DOM elements, read the content&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;b&gt;Why Scraping is Hard or Illegal?&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Web scraping is not illegal, but websites have the authority to block the scrapers scraping the website, including Reddit, X, Substack or Medium. &lt;/p&gt;

&lt;p&gt;Scraping is not allowed for most of the websites; each website has a robots.txt file providing instructions for the web scraper, and each web scraper needs to follow those instructions. &lt;/p&gt;

&lt;p&gt;We can't simply scrape every content, every time, from any webpage; we will hit the rate limit, scraper bot detectors, or get IP blocked.&lt;/p&gt;

&lt;p&gt;That is why web scraping is not an easy one; it needs an art to do the following&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Rotate IP addresses&amp;nbsp;&lt;/li&gt;
&lt;li&gt;Modify the user agents in our scraping API for the browser instance&amp;nbsp;&lt;/li&gt;
&lt;li&gt;Handling cookies, overlays or modals for the website before scraping&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A lot to be done for scraping a website's content and handling those API endpoints on your own won't cost you much, but it will costs the time&lt;/p&gt;

&lt;h2&gt;
  
  
  Simple Scraping API
&lt;/h2&gt;

&lt;p&gt;Below is the code for a simple web scraping API&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="cm"&gt;/**
 * Minimal Hono API: POST /scrape + POST /screenshot
 * Pattern aligned with ihatereading-api: BrowserPool → puppeteer-core →
 * request interception, stealth-ish evaluateOnNewDocument, JSDOM cleanup, markdown.
 *
 * Env:
 *   BROWSER_POOL_SIZE=2
 *   CHROME_PATH=/path/to/chrome   (local dev; optional if @sparticuz/chromium works)
 */&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;serve&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;@hono/node-server&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;Hono&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;hono&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;JSDOM&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;jsdom&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;TurndownService&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;turndown&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;POOL_SIZE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;parseInt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;BROWSER_POOL_SIZE&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;2&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;CHROME_ARGS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--no-sandbox&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-setuid-sandbox&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-dev-shm-usage&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-gpu&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-web-security&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--no-zygote&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--single-process&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;];&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;isValidHttpUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;u&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;());&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;u&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;protocol&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;http:&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;u&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;protocol&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;https:&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="cm"&gt;/** In-memory sliding-window rate limit (same idea as index.js). */&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;rateLimitMap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Map&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;rateLimit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;windowMs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;now&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;record&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;rateLimitMap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;record&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;now&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;resetTime&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;rateLimitMap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;count&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;resetTime&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;now&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;windowMs&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;allowed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;remaining&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;limit&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="na"&gt;allowed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;retryAfter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ceil&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;resetTime&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;now&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="na"&gt;remaining&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;};&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="nx"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;allowed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;remaining&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;limit&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nf"&gt;setInterval&lt;/span&gt;&lt;span class="p"&gt;(()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;now&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;for &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;rateLimitMap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;entries&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;now&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;resetTime&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;rateLimitMap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;delete&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;BrowserPool&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;constructor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_size&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;size&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_ready&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_loading&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nf"&gt;_launch&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;puppeteer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;puppeteer-core&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;chromium&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;@sparticuz/chromium&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;chromePath&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;CHROME_PATH&lt;/span&gt;&lt;span class="p"&gt;?.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;chromePath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;puppeteer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;launch&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
                &lt;span class="na"&gt;headless&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="na"&gt;executablePath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;chromePath&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="na"&gt;args&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;CHROME_ARGS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;executablePath&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;chromium&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;executablePath&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;puppeteer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;launch&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
                &lt;span class="na"&gt;headless&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="nx"&gt;executablePath&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="na"&gt;args&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[...&lt;/span&gt;&lt;span class="nx"&gt;chromium&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;args&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-web-security&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
                &lt;span class="na"&gt;ignoreDefaultArgs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;--disable-extensions&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
            &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Could not launch Chrome. Set CHROME_PATH to your Chrome/Chromium binary.&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nf"&gt;initialise&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_ready&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_loading&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_loading&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;browsers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;length&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_size&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_launch&lt;/span&gt;&lt;span class="p"&gt;()),&lt;/span&gt;
        &lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;browsers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;browser&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;index&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="nx"&gt;browser&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;index&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;}));&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_ready&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_loading&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="nf"&gt;_acquire&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;free&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;free&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;free&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;free&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="nf"&gt;_release&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;entry&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;entry&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;next&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;shift&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;next&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;f&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
            &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                &lt;span class="nf"&gt;next&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;unshift&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;next&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="nf"&gt;withPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_ready&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;initialise&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;entry&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_acquire&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;page&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;entry&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;browser&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;newPage&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
                &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;
            &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_release&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;entry&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="kd"&gt;get&lt;/span&gt; &lt;span class="nf"&gt;stats&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;free&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;queued&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;busy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="na"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;free&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;busy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;queued&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;_queue&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;};&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;browserPool&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;BrowserPool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;POOL_SIZE&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;turndown&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;TurndownService&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
    &lt;span class="na"&gt;headingStyle&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;atx&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;codeBlockStyle&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;fenced&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;NOISE_SELECTORS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;script&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;style&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;noscript&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;header&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;footer&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;nav&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;aside&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;.ad&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;.ads&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;[class*='cookie' i]&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;[id*='cookie' i]&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;, &lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;stripDomNoise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelectorAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;NOISE_SELECTORS&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;el&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;el&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;remove&lt;/span&gt;&lt;span class="p"&gt;());&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;setupPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;viewport&lt;/span&gt; &lt;span class="p"&gt;})&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setViewport&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;viewport&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setUserAgent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setExtraHTTPHeaders&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
        &lt;span class="na"&gt;Accept&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Accept-Language&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;en-US,en;q=0.9&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;

    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;evaluateOnNewDocument&lt;/span&gt;&lt;span class="p"&gt;(()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nb"&gt;Object&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;defineProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;navigator&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;webdriver&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;get&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;undefined&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="nb"&gt;Object&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;defineProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;navigator&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;plugins&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;get&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="nb"&gt;Object&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;defineProperty&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;navigator&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;languages&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;get&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;en-US&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;en&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;

    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setRequestInterception&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;on&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;request&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resourceType&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;url&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;toLowerCase&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;image&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;font&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;media&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abort&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;type&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;stylesheet&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;respond&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;status&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;contentType&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;text/css&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;body&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;""&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;cloudflare&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
            &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;challenge&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
            &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;bot-detection&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abort&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;

    &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setDefaultNavigationTimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setDefaultTimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="cm"&gt;/**
 * Core scrape — simplified from scrapeSingleUrlWithPuppeteer (no Firestore, proxy, Reddit, G2, AI).
 */&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;scrapeUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;options&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{})&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;waitForSelector&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;timeout&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;includeSemanticContent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;includeImages&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;includeLinks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;extractMetadata&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;options&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;viewport&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1366&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;height&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;768&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;deviceScaleFactor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;browserPool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;withPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;setupPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;viewport&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;

        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;goto&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;waitUntil&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;domcontentloaded&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;timeout&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;

        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;scrapedData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;includeSemanticContent&lt;/span&gt;
            &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;evaluate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;opts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                            &lt;span class="na"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;location&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;href&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                            &lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;title&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                            &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{},&lt;/span&gt;
                            &lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{},&lt;/span&gt;
                            &lt;span class="na"&gt;links&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[],&lt;/span&gt;
                            &lt;span class="na"&gt;images&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[],&lt;/span&gt;
                        &lt;span class="p"&gt;};&lt;/span&gt;
                        &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h2&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h3&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h4&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h5&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;h6&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;tag&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                            &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;content&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;tag&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                                &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelectorAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;tag&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
                            &lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;h&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;h&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textContent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;());&lt;/span&gt;
                        &lt;span class="p"&gt;});&lt;/span&gt;
                        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;opts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;extractMetadata&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                            &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelectorAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;meta&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                                &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
                                    &lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getAttribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;name&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getAttribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;property&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
                                &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;content&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getAttribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;content&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
                                &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;content&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;metadata&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;content&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                            &lt;span class="p"&gt;});&lt;/span&gt;
                        &lt;span class="p"&gt;}&lt;/span&gt;
                        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;opts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includeLinks&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                            &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;host&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;location&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;hostname&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                            &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;seen&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
                            &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;links&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelectorAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;a[href]&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
                                &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt;
                                    &lt;span class="na"&gt;text&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textContent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                                    &lt;span class="na"&gt;href&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;href&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                                    &lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getAttribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;title&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                                &lt;span class="p"&gt;}))&lt;/span&gt;
                                &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                                    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                                        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;href&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nx"&gt;hostname&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nx"&gt;host&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                                        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                    &lt;span class="p"&gt;}&lt;/span&gt;
                                    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;text&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;title&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;k&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;text&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;|&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;l&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;href&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;k&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                    &lt;span class="nx"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;k&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
                                    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                                &lt;span class="p"&gt;});&lt;/span&gt;
                        &lt;span class="p"&gt;}&lt;/span&gt;
                        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;opts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includeImages&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                            &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;images&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelectorAll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;img[src]&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
                                &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;img&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;src&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;startsWith&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;data:&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
                                &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;img&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;({&lt;/span&gt;
                                    &lt;span class="na"&gt;src&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;src&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                                    &lt;span class="na"&gt;alt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;alt&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;""&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                                &lt;span class="p"&gt;}));&lt;/span&gt;
                        &lt;span class="p"&gt;}&lt;/span&gt;
                        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
                    &lt;span class="p"&gt;},&lt;/span&gt;
                    &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;extractMetadata&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;includeLinks&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;includeImages&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
                &lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;title&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="na"&gt;content&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{},&lt;/span&gt; &lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{},&lt;/span&gt; &lt;span class="na"&gt;links&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[],&lt;/span&gt; &lt;span class="na"&gt;images&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;

        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;html&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;content&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;dom&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;JSDOM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;html&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;doc&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;dom&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="nf"&gt;stripDomNoise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;doc&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;markdown&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;turndown&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;turndown&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;doc&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="na"&gt;success&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;data&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;scrapedData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;markdown&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;};&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;screenshotUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;options&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{})&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;timeout&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;fullPage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="nx"&gt;waitForSelector&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;options&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;viewport&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;width&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1366&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;height&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;768&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;deviceScaleFactor&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;browserPool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;withPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;setupPage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;viewport&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;goto&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;waitUntil&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;domcontentloaded&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;timeout&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;buf&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;screenshot&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;png&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;fullPage&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;encoding&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;binary&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;from&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;buf&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;toString&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;base64&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Hono&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;/&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt;
    &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
        &lt;span class="na"&gt;ok&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="na"&gt;hint&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;POST /scrape or /screenshot with JSON { url, ... }&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}),&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;/scrape&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-forwarded-for&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)?.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]?.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-real-ip&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
        &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;unknown&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;rl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;rateLimit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;rl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;allowed&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Retry-After&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;rl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;retryAfter&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;rate_limited&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;retryAfter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;rl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;retryAfter&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{};&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;invalid_json&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nf"&gt;isValidHttpUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;valid https url required&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;scrapeUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="na"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;includeSemanticContent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includeSemanticContent&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;includeImages&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includeImages&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;includeLinks&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includeLinks&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;extractMetadata&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;extractMetadata&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="p"&gt;...&lt;/span&gt;&lt;span class="nx"&gt;result&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;timestamp&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;toISOString&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="na"&gt;poolStats&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;browserPool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;stats&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;success&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;?.&lt;/span&gt;&lt;span class="nx"&gt;message&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;scrape_failed&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;post&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;/screenshot&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-forwarded-for&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)?.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]?.&lt;/span&gt;&lt;span class="nf"&gt;trim&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;x-real-ip&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt;
        &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;unknown&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;rl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;rateLimit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;rl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;allowed&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;header&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;Retry-After&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;rl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;retryAfter&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;rate_limited&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{};&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;req&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;invalid_json&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nf"&gt;isValidHttpUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;valid https url required&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;b64&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;screenshotUrl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="nx"&gt;_000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;fullPage&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fullPage&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;waitForSelector&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;waitForSelector&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="na"&gt;success&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;image_base64&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;b64&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;mime&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;image/png&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="na"&gt;timestamp&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;toISOString&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="na"&gt;poolStats&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;browserPool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;stats&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;success&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;error&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;e&lt;/span&gt;&lt;span class="p"&gt;?.&lt;/span&gt;&lt;span class="nx"&gt;message&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;screenshot_failed&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;port&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Number&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;PORT&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="mi"&gt;3000&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`Listening on http://localhost:&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;port&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nf"&gt;serve&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;port&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A few things to note down&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;We are using a small version of Playwright, &lt;a class="mentioned-user" href="https://dev.to/sparticuz"&gt;@sparticuz&lt;/a&gt;/chromium, as it works on Vercel Edge as well&lt;/li&gt;
&lt;li&gt;For making API work on Cloudflare, use Playwright or Puppeteer&lt;/li&gt;
&lt;li&gt;Turndown npm is used to parse the content into markdown&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Now using one simple method as an API, our web scraping API is ready&lt;/p&gt;

&lt;h2&gt;
  
  
  Deploying in Production
&lt;/h2&gt;

&lt;p&gt;Let's be honest, this API works for personal use cases but not in production for mass users. So our next task is to make it production-ready&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Create a pool for the browser instances, which works well for multiple URL web scraping, using BrowserPool&lt;/li&gt;
&lt;li&gt;Add rate limiting to the API endpoint&lt;/li&gt;
&lt;li&gt;URL validation layer&lt;/li&gt;
&lt;li&gt;Caching for already scraped URL content, no need to rerun the API, save costs and time&lt;/li&gt;
&lt;li&gt;Lastly, add some noise selectors to avoid while scraping the content, which helps the API to avoid loading unwanted scripts such as Google Analytics or ads&lt;/li&gt;
&lt;li&gt;IP rotations and user agent proxies to avoid getting banned while scraping too much&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The above points are added to the code, making the API a robust production-grade level scraping API.&lt;/p&gt;

&lt;h2&gt;
  
  
  Take Screenshot &amp;amp; AI Summary
&lt;/h2&gt;

&lt;p&gt;The same API helps to take a screenshot if the &lt;b&gt;takeScreenshot&lt;/b&gt; parameter is passed as true in the API&lt;/p&gt;

&lt;p&gt;For a screenshot, we load the webpage and wait for a few seconds, and then use the playwright page.screenshot method to take the screenshot, quite an easy one.&lt;/p&gt;

&lt;p&gt;Another thing to mention is aiSummary, using an AI LLM model, we can generate an AI summary for every webpage by passing the scraped content into an AI LLM to generate the summary. This is quite interesting and useful&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/shreyvijayvargiya/scrapefast/" rel="external noopener noreferrer"&gt;Check the Github repository&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That would be enough for today. If you need anything else, feel free to email me.&lt;/p&gt;

&lt;p&gt;Cheers&lt;/p&gt;

&lt;p&gt;Shrey&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Why I Keep Building CRM Templates (Even Though They’re “Boring”)</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Tue, 31 Mar 2026 14:59:19 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/why-i-keep-building-crm-templates-even-though-theyre-boring-37nj</link>
      <guid>https://dev.to/shreyvijayvargiya/why-i-keep-building-crm-templates-even-though-theyre-boring-37nj</guid>
      <description>&lt;p&gt;&lt;strong&gt;Hey there,&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I’ve been asked this a few times:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;“Why do you keep making CRM templates?”&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And honestly, the answer is pretty simple:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Because the market desperately needs them.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  The Truth Nobody Talks About
&lt;/h2&gt;

&lt;p&gt;Let’s be real for a second.&lt;/p&gt;

&lt;p&gt;CRM systems are not exciting.&lt;/p&gt;

&lt;p&gt;They don’t go viral.\&lt;br&gt;
They don’t look flashy on Twitter.\&lt;br&gt;
And they definitely don’t give you that “wow” feeling like a new AI tool does.&lt;/p&gt;

&lt;p&gt;But they &lt;em&gt;matter&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;A lot.&lt;/p&gt;

&lt;p&gt;Because behind every growing business, there’s usually a messy backend that needs structure.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Building a CRM From Scratch Is Painful
&lt;/h2&gt;

&lt;p&gt;If you’ve ever tried building a CRM, you already know:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It’s &lt;strong&gt;not beginner-friendly&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;It requires &lt;strong&gt;deep domain understanding&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;You deal with &lt;strong&gt;business-critical data&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;You need &lt;strong&gt;complex API + database integrations&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And suddenly, what looked like “just a dashboard” becomes:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;a full-blown system that runs an entire company.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That’s heavy.&lt;/p&gt;




&lt;h2&gt;
  
  
  So I Decided to Simplify It
&lt;/h2&gt;

&lt;p&gt;Instead of letting every developer or business struggle through this from scratch…&lt;/p&gt;

&lt;p&gt;I started building &lt;strong&gt;CRM templates&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Not just UI kits—but &lt;strong&gt;real, usable systems&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Frontend templates (React + Tailwind)&lt;/li&gt;
&lt;li&gt;Full-stack-ready structures&lt;/li&gt;
&lt;li&gt;Pre-built forms, widgets, and flows&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The goal?&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Help people skip the boring, repetitive setup—and focus on building their actual business.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Why CRM (and Admin Templates) Are Always in Demand
&lt;/h2&gt;

&lt;p&gt;There’s a category I like to call:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;The “Admin Layer” of the Internet&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This includes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CRM (Customer Management)&lt;/li&gt;
&lt;li&gt;CMS (Content Management)&lt;/li&gt;
&lt;li&gt;Documentation systems&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And here’s the thing:&lt;/p&gt;

&lt;p&gt;Every business needs this.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Startups&lt;/li&gt;
&lt;li&gt;Freelancers&lt;/li&gt;
&lt;li&gt;SMEs &amp;amp; MSMEs&lt;/li&gt;
&lt;li&gt;Enterprises&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Everyone eventually reaches a point where they need to:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Manage users&lt;/li&gt;
&lt;li&gt;Track sales&lt;/li&gt;
&lt;li&gt;Organize data&lt;/li&gt;
&lt;li&gt;Monitor performance&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In short:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Run their business from a single source of truth&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  A Real Example: My Own Workflow
&lt;/h2&gt;

&lt;p&gt;I run a platform called &lt;a href="http://ihatereading.in" rel="noopener noreferrer"&gt;&lt;strong&gt;ihatereading.in&lt;/strong&gt;&lt;/a&gt;, where I publish software content.&lt;/p&gt;

&lt;p&gt;Behind the scenes, everything runs through an internal admin panel:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Writing and managing blogs&lt;/li&gt;
&lt;li&gt;Sending emails&lt;/li&gt;
&lt;li&gt;Tracking performance&lt;/li&gt;
&lt;li&gt;Planning content&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This system helps me:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Improve SEO&lt;/li&gt;
&lt;li&gt;Increase domain rating&lt;/li&gt;
&lt;li&gt;Scale content production&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Today, the platform serves &lt;strong&gt;15K+ monthly users worldwide&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;And without a proper admin system?&lt;/p&gt;

&lt;p&gt;That would be chaos.&lt;/p&gt;




&lt;h2&gt;
  
  
  That’s When It Clicked
&lt;/h2&gt;

&lt;p&gt;I realized:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;If I need this… every growing business probably does too.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That’s what pushed me to go deeper into building:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CRM templates&lt;/li&gt;
&lt;li&gt;CMS systems&lt;/li&gt;
&lt;li&gt;Documentation dashboards&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;I created a dedicated collection:&lt;/p&gt;

&lt;p&gt;👉 &lt;a href="http://gettemplate.website" rel="noopener noreferrer"&gt;&lt;strong&gt;gettemplate.website&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It includes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;100+ React + Tailwind templates&lt;/li&gt;
&lt;li&gt;100+ forms&lt;/li&gt;
&lt;li&gt;100+ reusable widgets&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;All designed to help you:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Go from idea → working system in hours, not weeks.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Some CRM Templates You Can Explore
&lt;/h2&gt;

&lt;p&gt;Here are a few I’ve built:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;CRM Template for Developers&lt;/strong&gt;\
&lt;a href="https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template?layout=profile" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template?layout=profile&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Freelancer CRM Template&lt;/strong&gt;\
&lt;a href="https://shreyvijayvargiya.gumroad.com/l/freelancer-crm-template?layout=profile" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/freelancer-crm-template?layout=profile&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Real Estate CRM (Next.js 14)&lt;/strong&gt;\
&lt;a href="https://www.gettemplate.website/premium-templates/real-estate-crm-next-js-14-template" rel="noopener noreferrer"&gt;https://www.gettemplate.website/premium-templates/real-estate-crm-next-js-14-template&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Indie Hacker SaaS CRM Template&lt;/strong&gt;\
&lt;a href="https://shreyvijayvargiya.gumroad.com/l/inkgest-saas-admin-template?layout=profile" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/inkgest-saas-admin-template?layout=profile&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Final Thought
&lt;/h2&gt;

&lt;p&gt;CRM templates might seem boring.&lt;/p&gt;

&lt;p&gt;But they solve a very real problem:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Turning chaos into structure.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And in business, that’s everything.&lt;/p&gt;




&lt;p&gt;If you’re building something and need a head start, feel free to explore or reach out:&lt;/p&gt;

&lt;p&gt;📩 &lt;a href="mailto:shreyvijayvargiya26@gmail.com"&gt;shreyvijayvargiya26@gmail.com&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Cheers,&lt;/strong&gt;\&lt;br&gt;
Shrey&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>ai</category>
      <category>javascript</category>
      <category>programming</category>
    </item>
    <item>
      <title>Few treding github repositories</title>
      <dc:creator>shrey vijayvargiya</dc:creator>
      <pubDate>Sat, 21 Mar 2026 13:02:21 +0000</pubDate>
      <link>https://dev.to/shreyvijayvargiya/few-treding-github-repositories-31m3</link>
      <guid>https://dev.to/shreyvijayvargiya/few-treding-github-repositories-31m3</guid>
      <description>&lt;h1&gt;
  
  
  One person AI organisation employee team
&lt;/h1&gt;

&lt;p&gt;Few more trending github repositories&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Tags:&lt;/strong&gt; Github, Trending Repositories, Paperclip, Liteparse, LLMs&lt;/p&gt;

&lt;p&gt;Hey there!!&lt;/p&gt;

&lt;p&gt;Welcome to the new blog&lt;/p&gt;

&lt;p&gt;This blog is about some cool and trending open-source projects that really grab my attention, and a few of them I would love to share below&lt;/p&gt;

&lt;p&gt;&lt;a href="https://paperclip.ing/?ref=ihatereading.in" rel="external noopener noreferrer"&gt;Paperclip&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The AI employee agents repository that helps to create your entire AI employee company, assign, schedule and manage tasks and let the AI agent do the job for you.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://lightpanda.io/docs?ref=ihatereading.in" rel="external noopener noreferrer"&gt;Lightpanda&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Lightpanda is the AI-native web browser built using Zig for AI agents, the alternative to Puppeteer and Playwright, but it works with them as well, with 10x less memory and 10x faster than the Chrome browser.&lt;/p&gt;

&lt;p&gt;For our new product, &lt;a href="http://inkgest.com" rel="noopener noreferrer"&gt;inkgest.com&lt;/a&gt;, we are using Puppeteer for web scraping to create our own scraping infrastructure using Chrome browser to scrape content from a URL, but Lightpanda seems to be 10x cheaper and faster and AI-native, so it might be a good choice for your next scraping tasks in 2026.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://alibaba.github.io/page-agent?ref=ihatereading.in" rel="external noopener noreferrer"&gt;PageAgent&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Alibaba's new open-source project is promising, enabling AI to operate your web page in plain language. PageAgent scrapes the DOM from the web page and uses AI to perform tasks or operate the webpage.&lt;/p&gt;

&lt;p&gt;It's quite useful to fill the form, scrapes the content, for example or URLs, images, external links, hyperlinks, etc., and does all boring tasks using plain english.&lt;/p&gt;

&lt;p&gt;I use a lot of Chrome extension tools to scrape and download content from a webpage in Chrome, but using PageAgent, one can do the tasks using plain english.&lt;/p&gt;

&lt;p&gt;PageAgent works without the Chrome browser, headless Chrome, and works much faster with 10+ LLMs and providers.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/run-llama/liteparse?ref=ihatereading.in" rel="external noopener noreferrer"&gt;Liteparse&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Open-source liteparse module that helps parse documents.&lt;/p&gt;

&lt;p&gt;LiteParse is a standalone OSS PDF parsing tool focused exclusively on fast and light parsing. It provides high-quality spatial text parsing with bounding boxes, without proprietary LLM features or cloud dependencies. Everything runs locally on your machine.&lt;/p&gt;

&lt;p&gt;5 CRM templates for your next CRM projects. Last week, I launched 5 CRM templates for Businesses in the SaaS, Health, Medical, and Energy sectors/domains to help them launch their CRM systems and run their businesses. Each template is built using Nextjs, React and Tailwind CSS, helping you save 50+ hours of development time.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Check Templates 👇&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;SaaS CRM Template: &lt;a href="https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/saas-crm-react-template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Freelancer CRM Template: &lt;a href="https://shreyvijayvargiya.gumroad.com/l/freelancer-crm-template" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://shreyvijayvargiya.gumroad.com/l/freelancer-crm-template" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/freelancer-crm-template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Real Estate CRM Template: &lt;a href="https://shreyvijayvargiya.gumroad.com/l/estate-os-real-estate-crm-template" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://shreyvijayvargiya.gumroad.com/l/estate-os-real-estate-crm-template" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/estate-os-real-estate-crm-template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Non-Profit CRM Template: &lt;a href="https://shreyvijayvargiya.gumroad.com/l/non-charity-crm-template" rel="noopener ugc nofollow noreferrer"&gt;&lt;/a&gt;&lt;a href="https://shreyvijayvargiya.gumroad.com/l/non-charity-crm-template" rel="noopener noreferrer"&gt;https://shreyvijayvargiya.gumroad.com/l/non-charity-crm-template&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That would be enough for today&lt;/p&gt;

&lt;p&gt;See you in the next one, originally published on &lt;a href="https://ihatereading.in/t/One-person-AI-organisation-employee-team" rel="noopener noreferrer"&gt;iHateReading&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Shrey&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
      <category>ai</category>
      <category>javascript</category>
    </item>
  </channel>
</rss>
