DEV Community: Vikash Rathee

How to scrape a website using Chrome extension?

Vikash Rathee — Thu, 15 Oct 2026 09:44:46 +0000

Most scraping jobs I see start the same way. Someone needs to scrape data from product list, a directory of companies, or a page of search results in a spreadsheet, and writing a Python script with selectors and pagination logic feels like too much for a one-off task.

For that kind of job, a point-and-click extension is faster. In this post I'll show how to scrape a list page with the Agenty's Scraping Extension and export the result to CSV, Excel, or JSON.

I'll use Scraping Sandbox, a free practice site with 500+ sample e-commerce products. It's explicitly open for scraping practice, so you can follow along without worrying about anyone's terms of service.

Install the extension

Install Agenty from the Chrome Web Store and pin it to your toolbar. Installing it and running scrapers in your browser is free.

Pick a Scraping mode

Open the page you want to scrape, then click the Agenty icon to open the side panel. You'll see four modes:

List: for repeating items like product grids, search results, and directories
Page details: for fields on a single page, which you can then reuse across many URLs
Images: collects images on the page so you can preview and download them as a ZIP
Emails: finds email addresses in page text, mailto: links, and attributes

There are also ready-made templates for 50+ popular sites. If your target is one of them, start there and skip the selector work.

The Scraping Sandbox home page shows a grid of 24 product cards per page, so choose List.

Click the data you want

Click the name of the first product. Agenty highlights the other product names that match and generates the selector for you. You don't need to open DevTools or write anything by hand.

Repeat for the other fields you need. Each card has a vendor, rating, price, original price (on discounted items), category, SKU, stock status, and a link to the product page. Then clean up the field list:

Rename fields to something you'll recognize in the spreadsheet (name, price, sku, url)
Remove anything it picked up that you don't need
Add a custom CSS selector when the auto-generated one grabs too much or too little

Discounted products are a good example. They show two prices, the current one and the crossed-out original. If you only want the current price, use .product-card .price as a custom selector. The original price has its own class, .compare-at-price, so you can add it as a separate field if you want both.

Handle pagination

A single page is rarely enough. Agenty supports a few ways to move through pages:

Current page only
Auto-scroll, for infinite-scroll pages that load more items as you scroll
A "Next" button or link
A URL pattern, such as a numbered range of page URLs
A list of URLs you paste in

Scraping Sandbox splits the products across 21 pages with numbered buttons and a next arrow at the bottom. The URL doesn't change when you switch pages, so a URL pattern won't work here. Use the "Next" button option and point it at the next arrow.

On sites where pages do have predictable URLs (like ?page=2), a URL pattern is usually more reliable than clicking a button. If you want to try auto-scroll, the sandbox has a separate infinite scroll page for that.

You can also adjust speed, delays, timeouts, page limits, and scroll behavior. Keep the delays reasonable. Hammering a site with fast requests is the quickest way to get blocked, and it's not fair to the site owner.

Run it and export

Run the scraper. It works inside your browser, so you can watch it move through pages. When it finishes, review the results table and check a few rows against the site.

Export to XLSX, CSV, or JSON.

Save the scraper if you'll need the data again. Saved scrapers rerun with one click, and the run history lets you reopen past results.

Scraping detail pages

The List mode gets you the overview. If you need fields that only appear on each item's own page, like the description, review count, or tags, use Page details:

Open one product page, for example /product/1, and choose Page details.
Click the fields you want, the same way as before.
Point the scraper at the product URLs. You can paste the url column from your list run as a URL list, or use a URL pattern, since product pages follow /product/1, /product/2, and so on.

The fields you picked once are applied to every page.

Each product page on the sandbox also shows a "Scraped JSON Data" block with the expected values for that product. Compare a few of your rows against it to check that your selectors picked up the right data.

I built an URL shortener with Cloudflare KV

Vikash Rathee — Sat, 09 Dec 2023 10:49:55 +0000

I am building an open source, highly scalable, and low-cost URL shortener with Cloudflare KV

By low-cost I mean, just $5 per month on Cloudflare worker and a free plan for basic requirement -

Idea

At DaySchedule, we had many customer upvoted for a URL shortener feature with auto-expirable links to book an appointment instead sharing their original URL.

So, I decided to try Cloudflare for this project -

By leveraging Cloudflare KV's capabilities, the goal was to develop a solution that offered not only URL shortening but also customization options, analytics, and scalability, all while maintaining a significantly lower cost compared to our established services on AWS.

Features

Expirable links using cacheTtl option on KV
Custom domain using Cloudflare for SaaS
Analytics using Worker analytics

API

I used hono.dev by @yusukebe for API routing, and it's blazing fast and offers Cloudflare pages template to build Edge projects quickly.

Here is an example of server.ts from their docs -

import { Hono } from 'hono'
const app = new Hono()

app.get('/', (c) => c.text('Hono!'))

export default app

Controller

The /links API is all I need to create, update and manage the short links.

const links = new Hono<{ Bindings: Bindings }>();

links.post(
  '/',
  validator('json', (value, c) => {
    const parsed = linkSchema.safeParse(value);
    if (!parsed.success) {
      return c.json(parsed, 400);
    }
    return parsed.data as ShortLink;
  }),
  async (ctx) => {
    const data = ctx.req.valid('json');
    // Set expire_at if given, or fallback to 1 month expiry
    const expireAt = data.expire_at
      ? dayjs(data.expire_at).unix()
      : dayjs().add(1, 'month').unix();
    if (expireAt <= dayjs().unix()) {
      return ctx.json(
        { message: 'The expire_at must be greater then current date time' },
        400
      );
    }

    let key = data.key || nanoid(7);
    let exists = await ctx.env.SHORTLINKS.get(key);
    while (exists) {
      key = nanoid(7);
      exists = await ctx.env.SHORTLINKS.get(key);
    }
    await ctx.env.SHORTLINKS.put(key, JSON.stringify(data), {
      expiration: expireAt,
    });
    return ctx.json(
      { ...data, key: key, short_url: `https://idm.in/${key}` },
      200
    );
  }
);

Code explanations -

This code is POST request to create short links. Here's a breakdown of the code to explain what all I am doing:

const links = new Hono<{ Bindings: Bindings }>(): It creates an instance of the Hono object.
Validation with zod - The validator('json', (value, c) => {...}): function is used as a middleware for validating the incoming JSON payload defined in linkSchema.
Set expiration: Checks if an expiry date is provided in the payload. If not, it sets an expiry date one month from the current date and time.
Generates a unique key for the link using nanoid(7) or uses the provided key from the payload if available.
Stores the link data (converted to a JSON string) in the Cloudflare KV (SHORTLINKS) with an expiration time based on the calculated expiry date.

Demo

The demo is available on IDM, it's free to use and open-sourced on Github to build your own custom URL shortener.

You can clone the repo to deploy on your Cloudflare account.

Star the repository on Github to show your support :-)

How to embed appointment scheduling widget on your website?

Vikash Rathee — Mon, 03 Apr 2023 12:02:57 +0000

Embedding a Dayschedule appointment scheduling widget on your website is a great way to simplify the appointment scheduling process for your clients. Here are the steps to follow:

Choose a appointment scheduling widget

There are many scheduling widgets available online, such as Calendly, Doodle, Acuity Scheduling and DaySchedule. Research different options and choose one that meets your business needs.

In this post, I am going to show you how to embed the dayschedule widget

Sign up for an account

Login to Dayschedule (or create a new account, it's free for basic use),
Create a resource
Set availability, questions etc.

Customize your scheduling widget

Customize your scheduling widget to fit the design and branding of your website. You can set color, upload logo etc.

Generate an embed code

Once your scheduling widget is customized, generate an embed code. This code is what you will use to add the widget to your website.

Add the embed code to your website

Install the package from NPM

npm i dayschedule-widget

Add the code to open appointment booking popup on click.

<button type="button" 
    onClick="bookAppointment()">
    Book an appointment
</button>

function bookAppointment(){
  daySchedule.initPopupWidget({
    "url": "https://demo.dayschedule.com",
    "type": "popup",
    "color": {
            "primary": "#0f0980",
        "secondary": "#afeefe"
    }
  })
}

See examples - https://stackblitz.com/@vickyRathee/collections/dayschedule-appointment-bookings

And paste the embed code into the HTML of your website where you want the scheduling widget to appear. This could be on a contact page, a services page, or a dedicated scheduling page.

Test your scheduling widget

Once you have added the scheduling widget to your website, test it out to make sure it is functioning properly. Schedule a test appointment to make sure the process works as expected.

Promote your scheduling widget

Now that your scheduling widget is embedded on your website, promote it to your clients and customers. Include a link to the scheduling widget in email signatures, social media posts, and other marketing materials.

Introducing Agenty "Agents for Machine Intelligence"

Vikash Rathee — Tue, 02 Apr 2019 15:33:06 +0000

Hi, Dev Community!

We have been working on Agenty from a long time, and today I am so excited to show it to the Dev community.

We are launching 4 agents today :

Scraping agents : Point-and-Click web scraping agent to extract data from websites of choices. Convert websites into api in lass then 2 minutes (see my video https://www.agenty.com/docs/video-tutorials.aspx ) or install the extension from Chrome store - https://chrome.google.com/webstore/detail/agenty-advanced-web-scrap/gpolcofcjjiooogejfbaamdgmgfehgff
Textract agents : Extract text and metadata from almost any documents in batch
OCR agents : Extract text from images and pdfs.
Bar code recognition agent - Recognize the bar code number and it's type from products photo captured by mobile or cameras.

All agents are API ready, see the documentation - https://www.agenty.com/docs/api

Go to our website and signup to get 100 pages credit free to try any agent you like. And you can go PRO anytime if you like it, the pricing plans just starts from $29 per month (10x lower then import.io for web scraping whose plans starts from $299)

I’ll keeps checking this page today to answer any comments, feedback you have, really looking forward to it :)