DEV Community: Nidhi

How to find which Indian government schemes you qualify for (and which documents to get ready first)

Nidhi — Tue, 27 Oct 2026 10:43:53 +0000

India has thousands of central and state welfare schemes, from crop insurance to scholarships to pensions. Most people who qualify for one never apply. That is usually not because the scheme is hard. It's because finding the right scheme, reading its fine print and collecting the papers takes longer than anyone has. Here's the process I'd follow, plus a small free tool I built to speed up the first step.

Step 1: Start from the official catalogue

The Government of India runs myScheme (https://www.myscheme.gov.in), a single portal that lists central and state schemes with their eligibility, benefits, documents and application process. State departments also run their own portals, such as MahaDBT in Maharashtra or the SSP portal in Rajasthan.

Whatever site you use to discover a scheme, the official page is where you decide. Check that the address ends in .gov.in or .nic.in before you type in Aadhaar or bank details.

Step 2: Write down the facts that decide eligibility

Nearly every scheme filters on the same handful of facts. Write yours down once:

State of residence. Many schemes are state-only.
Age, in completed years.
Gender. Some schemes are only for women or for transgender persons.
Annual family income, usually from all sources, and usually backed by an income certificate.
Occupation: farmer, student, self-employed, salaried, unemployed.
Social category: General, OBC, SC, ST, PVTG or DNT.
Disability, and its percentage if you have a certificate.
Rural or urban residence.
Special groups: ex-serviceman, freedom fighter or dependent, orphan, widow, ward of police or CAPF personnel.
Lists you're on: a ration card (AAY or PHH), or an SECC or below-poverty-line list.

Step 3: Read the fine print the filters miss

Filters get you a shortlist, not an answer. These conditions usually sit only in the eligibility text:

Land in your name. PM-KISAN, for example, is for landholding farmer families with cultivable land in their own names.
Being on an existing list. Ayushman Bharat PM-JAY uses the SECC 2011 deprivation criteria. Free food grain under PM-GKAY goes to Antyodaya and Priority Household ration-card families.
Being a current student at a recognised institution, often with an attendance or marks condition. This applies to most scholarships.
A life event. Some schemes are for a marriage, a first pregnancy, a death in the family, or a serious illness.
Not already receiving a similar benefit. Many pensions and scholarships forbid holding two at once.
Not paying income tax. Atal Pension Yojana, for example, has been closed to income-tax payers since October 2022.
Open and close dates. Some schemes run in yearly windows, and some have ended. If a scheme shows a close date, confirm it is still running before you spend time on it.

Step 4: Prepare the documents most schemes ask for

You'll save the most time here, because the same papers come up again and again:

Aadhaar card, with your current mobile number linked to it
A bank account in your own name, linked to Aadhaar, since most benefits are paid by direct transfer
Income certificate from the tehsil or revenue office
Caste or category certificate, if you're applying under a reserved category
Domicile or residence certificate for state schemes
Ration card, which matters for food and many health schemes
Land records for farmer schemes
Disability certificate or UDID card
Student documents: previous mark sheets, an admission or bonafide certificate, and fee receipts
Recent passport-size photographs

Certificates such as income and caste can take days or weeks, so apply for them before you start the scheme application.

Step 5: Apply only through official channels

Use the scheme's official portal or your nearest Common Service Centre (CSC), where a village-level entrepreneur can file the application for you. Be wary of anyone promising guaranteed approval for a fee. Keep the acknowledgement number, and check your status on the same portal.

A shortcut for step 1: YojanaMatch

I built YojanaMatch (https://nidhi-bhai.github.io/yojana-match/) to make the shortlist step faster. You answer the ten questions from Step 2, and it checks them against 239 central and state schemes copied from myScheme. It splits the results in two:

Matches. A scheme counts as a match only when every condition in its official eligibility text maps to one of your answers. That bar is strict, and only a handful of schemes clear it, mostly pensions such as old-age and widow pensions, and MGNREGA. Most people get zero to two firm matches.
Possibly eligible, check conditions. This is where most schemes land: the official text has a condition the form can't settle, such as land in your name, a marriage, a first pregnancy, or not paying income tax. It quotes that condition so you know what to check.

Schemes whose close date on myScheme has passed are marked Closed and are never shown as matches. Each scheme shows its eligibility text, the documents myScheme lists, the official source page, and the apply link where myScheme gives one. Every scheme also has its own page you can share. It works in English, Hindi and Telugu. The Telugu scheme text is a machine translation, marked beta, so check the English.

To be clear about its limits:

It is unofficial. It's not a government website, and a match isn't a promise of eligibility. Read the eligibility text and confirm on the official page before you apply.
It covers a subset. 239 schemes is far from every scheme in India. Some states and UTs, including Goa, Uttarakhand, Himachal Pradesh and Puducherry, have no state schemes in it yet, and the form labels them "central schemes only".
The data is a copy. Each record says "Copied from myScheme on 2026-10-27". Rules change, so the official page wins.
It's free. There's an optional $2 PDF (in English) of your firm matches, but it's the same checklist the page shows for free. Payment is through PayPal, which can't take payments from buyers in India, so for most people the free page is the whole product.

If you help other people apply, as a CSC operator, at an NGO or just as the family member who handles the paperwork, I'd like to hear which schemes or states you'd want added.

Unofficial guide. Scheme rules change, so always verify on the official site (myscheme.gov.in or the scheme's own portal) before applying.

How to find which countries actually import your product, using free UN Comtrade data

Nidhi — Tue, 27 Oct 2026 09:40:49 +0000

Most first-time exporters start with a list of buyers' emails. A better place to start is one question: which countries actually paid for this product last year, and how much? You can answer it for free in about fifteen minutes. Here is how, with real 2024 numbers.

1. Get the right HS code (this is where most people go wrong)

Customs and trade statistics are organised by the Harmonized System (HS), a six-digit product code that is shared worldwide. Countries add their own digits after the first six. India's ITC-HS uses eight, for example. International comparisons only work at six digits.

Pick the most specific six-digit code that honestly describes what you ship:

Green (unroasted) coffee is 0901.11, "Coffee; not roasted or decaffeinated". That is a clean match.
Turmeric is 0910.30, which covers turmeric (curcuma) in every form. That is close enough, though it isn't one commercial grade.
Moringa leaf powder has no code of its own. It usually falls under 1211.90, a basket of plants used in pharmacy and perfumery. Any number you pull for it covers far more than moringa.

Write down how specific your code is. It decides how much you can trust everything that follows.

2. Pull imports from the world

Open Comtrade Plus (comtradeplus.un.org), or call the public preview API directly. No key is needed for small queries:

https://comtradeapi.un.org/public/v1/preview/C/A/HS?flowCode=M&period=2024&partnerCode=0&cmdCode=090111

This query asks for annual (A) imports (flowCode=M) of HS 090111 in 2024 from all partners combined (partnerCode=0, "World"), across every reporting country. Each row has a primaryValue in US dollars and, usually, a quantity in kilograms.

For green coffee, the response I pulled on 27 Oct 2026 had 147 reporting countries. The top three were:

Reporter	2024 imports, HS 090111 (USD)
USA	6,319,859,239
Germany	4,706,705,407
Italy	2,753,529,337

3. Read the numbers correctly

Imports are usually valued CIF. That means the value includes freight and insurance to the buyer's border, so it is not comparable to an FOB export quote.
Value divided by quantity gives a rough unit value. Use it as a sanity check on price bands, not as a price.
Not every country has reported yet. A missing country may simply be late, so check the reporter count.
Big producers can be big importers. In 2024, India reported $42.1M of turmeric imports (HS 091030) and ranked second after the USA ($56.2M). That is easy to miss if you assume a producer never imports. It is also a lead: someone in India is buying turmeric from abroad.

4. Go from country to company

A country can't sign a purchase order. Once you have three to five target countries, narrow down to actual buyers:

Trade show exhibitor and visitor lists. Exhibitor lists for fairs like Anuga, Gulfood, BioFach or Seafood Expo Global are often public and name real companies in your category.
Industry associations. Member directories (coffee, tea, herbal products, valves…) list firms that already buy in the category.
Company types, not just names. Ask which kind of business buys this as an input: botanical manufacturers, spice grinders, food wholesalers. Then search for that type in the target country.
Job titles. Your email should reach the purchasing manager or the category buyer, not info@.

Then write to them with the specification they would file: grade, moisture, certificates and MOQ. Don't lead with a market-size slide.

Where BuyerRadar fits

I got tired of redoing steps 1–4 by hand, so I built BuyerRadar (https://nidhi-bhai.github.io/buyer-radar/). For 48 curated products (40 goods and 8 software categories), it:

ranks importing countries from a stored UN Comtrade 2024 snapshot, using the exact API values and saying when the HS heading is broader than the product, as it is for moringa;
scores buyer company types, trade shows, associations, marketplaces and job titles from 1 to 5 on Money, Authority and Need, with a written reason and a source link for each score;
shows the top 5 channels for free, and sells the full ranked report for $19.

Its limits:

It covers only those 48 products, and it doesn't name individual importers.
Every score except the Comtrade figures is an editorial judgment, not a measurement.
The company types and job titles use US classifications (NAICS and O*NET).
The software categories have no trade data at all.

Treat it as a shortlist to check, not as a buyer list.

The method above works for any product, whether or not it is on that list. The free data is good enough to stop you from cold-emailing the wrong country.

If you want it in one file, the full ranking with an outreach angle for each channel is a $19 Markdown/PDF report. Every source in it is public and linked, so you're paying for the assembly, not secret data.

How to track RBI, DGFT and GST changes that affect your export business

Nidhi — Tue, 27 Oct 2026 09:35:06 +0000

If you export from India, the rules that move your margin rarely arrive as news. They come out as a DGFT notification, an RBI circular addressed to banks, a CBIC notification or a Gazette entry, often late in the evening, and on sites that weren't built for skimming. This is the routine I use to keep up, plus a free tool I built to make it lighter.

1. Know which publisher owns which risk

Start by mapping what you care about to who publishes it.

DGFT covers export and import policy, licences, export incentives and restricted items. It publishes three kinds of documents, and they mean different things:
- Notifications amend the Foreign Trade Policy or the ITC(HS) import and export schedules. When an item moves from "free" to "restricted", this is usually where it shows up.
- Public notices amend the Handbook of Procedures and its appendices: procedures, forms and documentation.
- Trade notices are clarifications, advisories and process updates. They are often operational, but they are not always minor.
RBI covers foreign-exchange rules (FEMA), export proceeds, banking and credit. For exporters the key series is the A.P. (DIR Series) circulars. They are addressed to Authorised Dealer banks, which means your bank will apply them to you whether or not you have read them.
CBIC / GST covers GST rates, refunds, filing changes and portal notices.
The Gazette of India is where notifications are formally published. If someone tells you "the amendment is in force", the Gazette entry is what to check.
SEBI and PIB matter less for day-to-day exporting. PIB is still useful as an early, plain-English heads-up that a ministry has decided something.

2. Read the instrument, not the headline

Headlines tell you very little. "Amendment in import policy of raw sugar classified under Exim Code 170114..." tells you the item and the direction, but not the effective date, any exceptions or any transition period. Before you change a price, a shipment or a filing, open the PDF and check three things:

Effective date. Is it immediate, from a future date, or retrospective?
Scope. Which HS codes, which ports, which class of exporter?
Transition. Are shipments already in the pipeline protected, for example those with a letter of credit opened before the date?

If you can't answer all three from the document, ask your CA or customs broker. Don't guess.

3. Batch it once a day

Checking six sites ad hoc means you either miss things or check too often. A better pattern:

One fixed slot a day, for example 9 a.m., to scan everything published since yesterday.
Filter by your exposure. Look up your HS chapters and your instruments (incentive schemes, advance authorisation, and so on). Skip everything else.
Log what you acted on. Note the date, the notice number, what changed and what you did.

4. Use RSS where it exists

RBI and SEBI publish RSS feeds, and PIB has a press-release RSS. Any feed reader can pull them into one place. DGFT, CBIC and the Gazette are harder because their pages aren't built for feeds, which is exactly where people fall behind.

5. The tool I built for this: PolicyPulse

I got tired of doing step 4 by hand, so I built PolicyPulse. It's a static site. A daily GitHub Actions job fetches the public pages of PIB, RBI, SEBI, the CBIC GST portal, DGFT and the Gazette, and stores each notice with its date and official link. It then files each notice on sector pages: exports, agri, MSME, banking, GST, pharma and startups. The archive has 654 notices, including a 2023-26 backfill labelled with each document's own date, and there's an RSS feed.

What I deliberately didn't do:

It doesn't summarise the law for you. Each card shows the official headline and source type, and tells you to open the source. An impact note appears only where the notice itself states one. Lines about amendments, restrictions or duties appear only when those words are in the official text. Any optional AI note is dropped if it contains a number that isn't in the source.
Sector filing is keyword matching, so it will sometimes mis-file a notice. A company with "Exports" in its name can land on the exports page, for example. Treat the sector pages as a first filter, not a verdict.
Sources can fail. The sources page shows the result of the last fetch for each publisher, so you can see when one didn't come through and check that site directly.

The daily feed, the sector pages and the RSS feed are free. Pro is a convenience rather than extra data: $9 once, via PayPal, for a link to the unlisted weekly Pro brief page, updated every Monday; no renewal, no account. It sorts that week's notices by sector with official links, and it funds the project.

A ten-minute daily routine

Open your sector page (or your RSS reader) at a fixed time.
Open the official PDF for anything that touches your HS codes, your bank or your GST filing.
Check the effective date, the scope and the transition.
Log it, and loop in your CA or broker if it changes anything you do.

Disclaimer: This article and PolicyPulse are for education and information only. They are not legal, tax or investment advice, and PolicyPulse is not a government website. Always read the official notice and consult a qualified professional before acting.

How to move a PowerPoint deck into a brand template without retyping it

Nidhi — Tue, 27 Oct 2026 09:26:55 +0000

Someone hands you a finished deck. Marketing hands you the new company template. You're told to "just put it in the template". An hour later you're retyping titles into placeholders. Here is why that happens and how to avoid most of it.

What a template actually is

A .pptx file is a zip of XML files. Your words, pictures and tables live on the slides. The look lives in three other places:

the slide master, which holds the logo, background, footer and default text styles;
the layouts, named arrangements of placeholders such as Title Slide, Title and Content, Section Header and Picture with Caption;
the theme, which holds the font pair and the colour palette.

"Applying a template" means getting your content into the template's placeholders. If the old deck was built from loose text boxes, PowerPoint has nothing to map, which is why "Design → Browse for themes" so often leaves slides looking half-converted.

Step 0: check both files first

Open the template's Slide Master view (View → Slide Master). You want real title and body placeholders. If the "template" is a full-slide picture, there is nothing to reflow into, so ask for the proper file.
Check the aspect ratio. Moving from 4:3 to 16:9 will always need some nudging.
List what's hard to move. Charts, SmartArt, embedded videos and animations need manual work whatever method you use. Knowing how many you have tells you how long this will really take.

Method 1: Reuse Slides (manual, built in)

Start a new file from the template, then go to Home → New Slide → Reuse Slides and pick the old deck. The slides arrive keeping their source formatting. Select them, choose a layout from the Layout gallery, and click Reset.

It works, and it's free. But Reset often puts text into the wrong placeholder when the source used text boxes, and on a 30-slide deck you'll be fixing slides one at a time.

Method 2: Copy and paste with "Use Destination Theme"

Paste slides into the template deck and choose Use Destination Theme. It's quick for decks built on standard layouts. For decks made of free-floating shapes it mostly recolours them and leaves the layout alone.

Method 3: Rebuild the slides automatically (SlideFit)

I built SlideFit for the case where Methods 1 and 2 fall over. You give it two files, the deck and the template, and it:

reads each slide and pulls out the title, subtitle, bullets (with their indent level), pictures and table cell text;
classifies the slide: a first slide with no bullets becomes a Title Slide, a short lone line becomes a Section Header, a picture with text becomes Picture with Caption, a table goes in the body of Title and Content, and everything else becomes Title and Content;
writes a new .pptx on the template's master, layouts, fonts and colours. Your original file isn't touched.

It runs entirely in the browser tab using JSZip. There's no account and no server, and the deck isn't uploaded. The code is public at github.com/Nidhi-bhai/slide-fit, and you can confirm it in your browser's network panel.

The limits, plainly:

Charts and SmartArt are not rebuilt. The slide keeps its title and the preview marks it "Skipped: chart or diagram", so you can paste the chart back in by hand.
Speaker notes and animations are not rebuilt.
Inline formatting (bold, colour, and links) is not kept; the words come across as plain text.
Custom layouts become standard ones. A hand-built collage turns into a title, a text block and pictures. That's usually what a brand team wants, but it isn't a pixel copy.
Overflow happens. Long bullets or a 4:3 deck on a widescreen master may need nudging after you download.

The free plan keeps the first 5 slides, then adds a SlideFit watermark slide, which is enough to check a template. After that there are two one-time PayPal payments, and neither auto-renews: $5 for 24 hours of conversions in that browser or $12 for a 30-day pass covering every deck in that browser.

A 15-minute workflow

Convert 5 slides first to check that the template maps the way you expect.
Convert the whole deck.
Go through the slides marked "Skipped" and paste those charts back from the original.
Check for overflow, then re-add any emphasis (bold words) that matters.
Re-add speaker notes if you present from them.

Whichever method you pick, the rule is the same: get the content into placeholders, and the template does the rest.

Pull Indian tech and AI-startup job data from employer career pages, in one API call

Nidhi — Tue, 27 Oct 2026 08:21:43 +0000

If you build a job board, run recruiting outreach or track hiring as a market signal, sooner or later you need the same thing: a clean, structured list of open roles. The usual approach is to scrape LinkedIn or Naukri, and that means logins, proxies, CAPTCHAs and reposted listings.

There is a simpler source. Most product companies host their careers page on an applicant tracking system (ATS) such as Greenhouse, Lever, Ashby or SmartRecruiters. Those vendors publish public JSON job-board feeds, so that companies can embed their openings on any site. No login is required.

I built two Apify actors on top of those feeds:

India Tech Jobs Scraper: https://apify.com/prospect_antechamber/india-tech-jobs-scraper
AI & Startup Jobs Scraper: https://apify.com/prospect_antechamber/ai-startup-jobs-scraper

This post shows how to call them, what you get back, what it costs and where they fall short.

What each actor covers

India Tech Jobs Scraper has a built-in directory of 57 employers that hire in India. The Store page lists names such as Stripe, Databricks, Okta, MongoDB, Zscaler, Swiggy, CRED, Meesho, Freshworks, Groww, Sarvam AI and InMobi. Every job is tagged isIndia and indiaCity, and the actor normalises spellings like Bangalore/Bengaluru and Gurgaon/Gurugram. In my test run on 27 Oct 2026 it returned 1,431 live India jobs from 61 company boards in 51 seconds.

AI & Startup Jobs Scraper has a directory of 144 companies, each tagged by sector: ai, fintech, devtools, security, crypto, health, autonomy_space_defense and consumer_gaming. It includes OpenAI, Anthropic, Perplexity, ElevenLabs, Cursor (Anysphere), Stripe, Databricks and SpaceX. A full run on 27 Oct 2026 returned 23,192 jobs from 144 boards (Greenhouse 16,881, Ashby 5,506, Lever 805), and 4,303 of them were remote.

Both actors also accept any other company that runs on one of the four supported ATSs. You can pass a careers URL (https://jobs.lever.co/meesho), an ats:slug pair (greenhouse:stripe) or a bare slug.

Example 1: data and ML roles in Bengaluru, posted this week

This input uses fields from the India actor's input schema:

{
  "useIndiaDirectory": true,
  "companies": ["https://jobs.lever.co/meesho", "greenhouse:stripe"],
  "indiaOnly": true,
  "includeGlobalRemote": false,
  "cities": ["Bengaluru", "Bangalore"],
  "keywords": ["data", "machine learning"],
  "excludeKeywords": ["intern"],
  "remoteOnly": false,
  "postedWithinDays": 7,
  "includeDescription": false,
  "maxItems": 500
}

Here is what the fields do:

keywords and excludeKeywords match against the title, department and team.
postedWithinDays uses the date each board publishes or updates.
maxItems caps the run. You pay only for jobs returned, so this is also your cost ceiling.
Set indiaOnly: false to get every location from the boards.

Example 2: AI-lab engineering roles in San Francisco or remote

This input uses fields from the AI & Startup actor's schema:

{
  "useDirectory": true,
  "sectors": ["ai"],
  "companies": [],
  "locations": ["San Francisco", "Remote"],
  "keywords": ["engineer"],
  "excludeKeywords": ["intern", "manager"],
  "remoteOnly": false,
  "postedWithinDays": 14,
  "includeDescription": false,
  "maxItems": 2000
}

If you leave sectors empty, the actor scans all 144 directory companies. locations keeps any job whose location string contains one of your values, such as "London", "United States" or "India".

Calling it from Python

pip install apify-client   # snippet follows the Python example on the actor's Store page

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "useDirectory": True,
    "sectors": ["ai"],
    "locations": ["San Francisco", "Remote"],
    "keywords": ["engineer"],
    "postedWithinDays": 14,
    "maxItems": 2000,
}

run = client.actor("prospect_antechamber/ai-startup-jobs-scraper").call(run_input=run_input)

for job in client.dataset(run.default_dataset_id).iterate_items():
    print(job["company"], "|", job["title"], "|", job["location"], "|", job["jobUrl"])

To call the India actor instead, swap in prospect_antechamber/india-tech-jobs-scraper and its input. The same calls work from JavaScript (apify-client on npm), from the Apify CLI (apify call ... --output-dataset), or as a tool for an AI agent through the Apify MCP server (https://mcp.apify.com/?tools=prospect_antechamber/ai-startup-jobs-scraper).

What one row looks like

Both actors return the same flat schema, one row per job:

{
  "ats": "ashby",
  "companySlug": "atlan",
  "company": "Atlan",
  "jobId": "20fe5046-b287-40f6-990f-842ad75526e5",
  "title": "Staff Software Engineer - Infrastructure (AI Native)",
  "location": "India",
  "allLocations": ["India"],
  "department": "Engineering",
  "team": "Engineering",
  "employmentType": "FullTime",
  "workplaceType": "Remote",
  "postedAt": "2026-10-23T14:44:45.243000+00:00",
  "updatedAt": null,
  "jobUrl": "https://jobs.ashbyhq.com/atlan/20fe5046-b287-40f6-990f-842ad75526e5",
  "applyUrl": "https://jobs.ashbyhq.com/atlan/20fe5046-b287-40f6-990f-842ad75526e5/application",
  "salary": null,
  "descriptionText": null,
  "isIndia": true,
  "indiaCity": null,
  "isRemote": true,
  "scrapedAt": "2026-10-27T07:26:28.940092+00:00"
}

The AI & Startup actor adds a sectors array to each row. You can export results as JSON, CSV or Excel from the Apify console, or send them on to Google Sheets, Zapier, Make or a webhook.

A pattern that works: a daily "new roles" feed

Schedule the actor once a day in Apify with postedWithinDays: 1.
Upsert rows into your database, using ats + companySlug + jobId as the key.
Mark rows that disappear from a later run as closed.

That gives you a fresh feed for a job board or newsletter. It also gives you a time series of hiring by company, team and city, which is useful for hiring-intent signals or for analyst work.

Pricing

Both actors use Apify's pay-per-event model:

$1.50 per 1,000 jobs returned ($0.0015 per job), plus Apify platform usage.
Filters are applied before charging, so jobs you filter out cost nothing.
For example, 500 jobs cost $0.75 plus usage. A full India directory scan of about 1,400 jobs is roughly $2.15 plus usage.
Set maxItems or an Apify spending limit to cap the cost.

Limitations

Four ATSs only. The actors cover Greenhouse, Lever, Ashby and SmartRecruiters. They do not cover LinkedIn, Naukri, Indeed, or companies that run their own careers system or Workday.
The directories are curated. They hold 57 and 144 companies, not "every company in India" or "every startup". Anything else has to go in companies.
Salary is often null. It appears only when the employer publishes it.
The data is only as clean as the board. Departments, teams and employment types are whatever each company enters. Some boards leave them empty.
Location matching is string-based. A job listed only as "Remote" with no country is dropped when indiaOnly is on, unless you set includeGlobalRemote.
Descriptions slow runs down. includeDescription makes SmartRecruiters boards slower in particular.
These are new actors. They have very few users so far and no Store ratings yet, so test on a small maxItems first.

Is it allowed?

The actors read only the public job-board endpoints that the ATS vendors provide for publishing jobs. They use no logins or cookies, and they don't collect candidate data. You are still responsible for how you use the data in your own jurisdiction, and for each source's terms.

If a company you need is missing from a directory, add its careers URL to companies, or tell me and I'll consider adding it.

Disclosure: I built and maintain both actors, and I earn the per-job fee above. (Nidhi, @RealNidhibhai, GitHub Nidhi-bhai)

What an Indian agri export price actually is, and how to read one

Nidhi — Tue, 27 Oct 2026 08:01:23 +0000

Education only. This is not trading or investment advice, and nothing here is an offer to sell.

If you import spices, herbs, millets or oils from India, the first question is usually "what does this cost right now?" Public answers do exist, but they are spread across UN Comtrade, the Government of India's Agmarknet mandi dashboards, and seller listings. Each of those sources measures something different.

The India Agri Export Price Index (https://nidhi-bhai.github.io/india-agri-export-prices/) is a free page I publish that brings those sources together each week for 18 products, priced in USD/kg. This article explains what the numbers mean and how an importer, trader or analyst can use them without over-reading them.

The 18 products

The index covers:

Herbs and leaf powders: moringa leaf powder, ashwagandha root powder, neem leaf powder, curry leaf powder, indigo leaf powder and wheatgrass powder.
Spices: turmeric (finger and powder).
Sugar: jaggery powder.
Millets: ragi, bajra, foxtail, the small millets (kodo, little, proso, barnyard and browntop), and khapli (emmer) wheat and atta.
Oils: cold-pressed groundnut oil, cold-pressed coconut oil, karanja (pongamia) oil and moringa seed oil.
Roots: dried vetiver (khus) root.

For wheatgrass powder, the page says plainly that there is "no reliable public price yet". It leaves the cell empty rather than guessing.

Three kinds of number, and why they differ

Every figure on the page is labelled with its type, source and date.

1. Official export unit values (UN Comtrade). India reports its exports each month by HS customs code. Dividing the FOB value in USD by the net weight in kg gives a unit value: the average price of everything shipped under that code in that month.

These figures come with a lag of about 2–4 months. The 27 Oct 2026 snapshot shows July 2026 as the latest month.
Many products share one code. HS 121190, for example, covers moringa, ashwagandha, neem, curry leaf, vetiver and hundreds of other herbs. The index marks such series as pooled. A pooled number tells you about the market, not the price of the exact product you want.

2. Mandi prices (Agmarknet). These are daily all-India average wholesale prices from regulated markets, published in INR per quintal (100 kg). The index converts them to USD/kg at the ECB reference rate.

They are raw, farm-gate commodity prices, recorded before cleaning, grading, grinding, packing, testing and port costs. So they sit well below export prices.
The page also shows each day's arrivals. Low arrivals mean a thin, noisy price. In this snapshot, ragi had only 0.1 t of arrivals on the reported day and coconut oil 0.25 t.

3. Estimates. For powders and other niche items there is no official price series. For these the index compiles ranges from public seller offers, rate cards and trade reports, and links each source.

These are asking prices, not transaction prices.
Some estimates rest on very little evidence, and the page says so. The khapli figure (2.58 USD/kg) comes from one emmer-flour shipment, which the data file calls "a single data point, not a market price". Several millet ranges are cited from July 2026 and are marked "older".

A worked example: turmeric

Turmeric (HS 091030) is one of the cleaner series because it has its own code. From the 27 Oct 2026 snapshot:

Export unit value, Jul 2026: 1.853 USD/kg FOB. India shipped 13,489 t worth USD 24.99 million that month.
12-month range of monthly unit values (Aug 2025 – Jul 2026): 1.70–2.29 USD/kg. The high was December 2025.
Mandi price, 25 Oct 2026: INR 19,500 per quintal, about 2.05 USD/kg. Arrivals that day were only 2.0 t, so treat that figure as thin.
Destinations in 2025: unit values ranged from about 1.30 USD/kg (Bangladesh) to 3.56 USD/kg (USA). That spread most likely reflects differences in grade, form and buyer requirements, not one "India price".

The takeaway is that a single headline number hides a wide range. The destination table is often more useful than the average, because it shows what buyers in markets like yours have paid on average.

How importers can use it

Sanity-check a quote. If an offer sits far outside both the Comtrade range and the public-offer range, ask why. The answer may be grade, certification, packing or freight terms. Note that the index covers conventional grade only.
Watch direction, not just level. Comtrade's monthly series shows whether average export unit values are rising or falling, even with the lag.
Separate raw material from finished product. The gap between the mandi price and the export price is roughly the processing, compliance and logistics cost.
Download and build your own view. Everything is available as latest.csv, latest.json and an append-only history.csv, under CC BY 4.0. The history file started with the 27 Oct 2026 snapshot, so it holds one week of data for now and grows by one snapshot a week.

Caveats worth repeating

Pooled HS codes are context, not product prices. On the home page, neem leaf powder (4.056) and indigo leaf powder (3.803) show pooled-code values. Karanja oil and moringa seed oil both show 6.227 USD/kg because they share HS 151590.
Comtrade lags by months. Mandi data is daily but thin for niche items.
Estimates are seller asks compiled from a handful of public sources, and some of them are old.
The cold-pressed coconut oil estimate (2.70 USD/kg) comes from a general coconut-oil FOB Cochin index for August 2026. It is not specific to cold-pressed oil.
USD/INR conversions use the ECB reference rate for one day.

Who publishes it

The index is published by Nidhi Farms, a sourcing firm (a trader, not a grower) in Andhra Pradesh, India, supplying conventional grade only. To be transparent: Nidhi Farms also takes quote requests, and the page links to them. The reference data itself is free and open, and corrections are welcome at nidhi69n@gmail.com. The methodology is at https://nidhi-bhai.github.io/india-agri-export-prices/methodology.html.

Building a $0, no-backend portfolio backtester and the data-licensing rabbit hole

Nidhi — Sun, 18 Oct 2026 03:18:39 +0000

I built BacktestCard, a static-site backtester for Indian investors who measure in USD. It has no framework, no backend and no dependencies. Three things were more interesting than I expected.

1. Data licensing decides your architecture

Yahoo, Stooq and NSE all forbid redistribution.
FRED was the surprise. Its ToS bars storing or caching FRED content, even for public-domain series.
So the build script fetches from the publishers themselves:
- Federal Reserve DDP for H.10 and H.15
- BLS flat files
- the OECD SDMX API
- the World Bank Pink Sheet (CC BY 4.0)
The result is a 108 KB JSON file, refreshed monthly by a GitHub Action.
Licensed indices such as NIFTY come only through a user CSV upload. It's parsed with FileReader and never sent anywhere.

2. Test your maths with a second implementation

The engine is about 200 lines of plain JS. The tests include a separate pandas implementation that must match every statistic to 1e-9. On day one it disagreed on bond returns.

Bond returns are modelled from yields. Each month you buy a 10-year par bond and reprice it at the new yield. I had added coupon/12 on top of a dirty price that already included accrued interest, which added 1.4%/yr of phantom return to a 60/40 mix. The fix uses a closed form:

// remaining flows at j/2 - 1/12 years => dirty = (1+y/2)^(1/6) * P_20(y)
return Math.pow(1 + y / 2, 1 / 6) * price20 - 1;

3. Share state belongs in the URL

The whole portfolio state is in the hash, e.g. #p=us_eq:60,in_eq:40&s=2000-01&c=USD.
It's whitelisted on decode, so __proto__:5 gets dropped.
The share card is a 1200×630 canvas exported with toDataURL.
?embed=1 gives a compact iframe with a badge, which is the growth loop.

Code (MIT): https://github.com/Nidhi-bhai/backtestcard. It's educational only, not investment advice.

CGPA to GPA Conversion: How to Convert a 10-Point CGPA to a 4.0 GPA (and When Not To)

Nidhi — Sat, 17 Oct 2026 17:25:51 +0000

If you studied in India or anywhere else that uses a 10-point CGPA, you've probably been asked for your "GPA on a 4.0 scale" when applying for a master's abroad, a job, or a scholarship. The internet is full of one-line formulas that give contradictory answers. This guide explains the common methods, where each one comes from, and which number you should actually put on an application.

The short version: there is no single universal formula. If an application asks for a converted GPA, check whether the institution wants a credential evaluation (WES, ECE, or their own). If they ask you to self-report, many tell you exactly how they want the number. Use what they specify, and include your original CGPA and scale wherever possible.

Why there's no single formula

A 10-point CGPA and a US 4.0 GPA aren't just the same thing at a different scale. They're different grading systems:

Grading in many Indian universities is stricter at the top. An 8.5 CGPA can be a very strong result, while a US 3.4 (85% of 4.0) is decent but not outstanding.
Different universities map marks to grade points differently.
US GPAs are built from letter grades per course, so a proper conversion works course by course, not from the final number.

That's why a straight linear formula and a professional evaluation can give quite different answers.

Method 1: Simple linear scaling

GPA = (CGPA ÷ 10) × 4

Example: CGPA 8.0 → 3.2. CGPA 9.0 → 3.6.

This is easy and widely quoted, but it often undervalues strong Indian CGPAs because it ignores how differently the grades are distributed. Treat it as a rough first estimate only.

Method 2: Via percentage

Some people convert CGPA to a percentage first and then map the percentage to a letter grade. For CBSE, the official rule of thumb is:

Percentage ≈ CGPA × 9.5

So a CGPA of 8.0 ≈ 76%. Important: this ×9.5 rule comes from CBSE and was meant for CBSE school results. Many universities publish their own formula instead (for example ×10, or (CGPA − 0.5) × 10, or (CGPA − 0.75) × 10). Your university's official conversion, usually printed on the transcript or in the academic regulations, always comes first.

After that, people map the percentage to a US grade (for example 60%+ treated as roughly an A in some evaluators' tables for Indian first-class results). These tables vary by evaluator, which is why we haven't printed one as if it were official.

Method 3: Course-by-course (what evaluators do)

Credential evaluators like WES generally look at each course grade on your transcript, convert it to a US grade equivalent using their own tables, and then calculate a credit-weighted GPA. This is the most accurate method, and it's often the one universities trust. It's also why the evaluator's number may not match any formula you use at home. WES publishes a free "iGPA calculator" preview on its website that you can try with your own grades.

If an application requires a WES or ECE evaluation, use the GPA from that report. Don't try to calculate your own.

Which number should you report?

If the application has a field for your original scale, enter your CGPA out of 10. Admissions teams that see many Indian applicants are used to reading these directly.
If they give a formula or table, use exactly that one.
If they require a credential evaluation, use the evaluator's GPA.
If they ask you to self-convert and give no method, use the linear estimate, label it as approximate, and give the original as well: "8.4/10 CGPA (≈3.36/4.0 by linear scaling)." Being transparent never hurts you. A number that looks inflated can.

Never round up or pick the most flattering formula without saying so. Your transcript will be checked.

Calculating your CGPA correctly first

Before you convert anything, make sure the CGPA itself is right. CGPA is the credit-weighted average of your semester SGPAs:

CGPA = Σ(SGPA × semester credits) ÷ Σ(semester credits)

A plain average of SGPAs is only correct if every semester carried the same credits, and that's often not the case.

Worked example:

Semester	SGPA	Credits	SGPA × Credits
1	7.8	20	156
2	8.2	22	180.4
3	8.6	24	206.4
4	8.9	18	160.2
Total		84	703

CGPA = 703 ÷ 84 = 8.37. A simple average of the four SGPAs would give 8.375, which is close here, but the gap gets bigger when credit loads differ a lot.

The arithmetic itself is quick: multiply each semester's SGPA by its credits, add them up, and divide by total credits (any spreadsheet or AI chat can do it in seconds). If you want somewhere to keep your semester grades organised over time, the free Study OS 2027 Lite planner template includes a grade tracker alongside a weekly planner.

Quick reference: linear 10-point to 4.0 estimates

These are linear estimates only, not official equivalents:

CGPA (10)	Linear GPA (4.0)	CBSE ×9.5 %
6.0	2.40	57.0%
6.5	2.60	61.8%
7.0	2.80	66.5%
7.5	3.00	71.3%
8.0	3.20	76.0%
8.5	3.40	80.8%
9.0	3.60	85.5%
9.5	3.80	90.3%
10.0	4.00	95.0%

Expect a course-by-course evaluation to come out higher than the linear column for many strong Indian transcripts, but that isn't guaranteed. The only way to know is to run the evaluation.

Common mistakes to avoid

Using a plain average of SGPAs when credit loads differ between semesters.
Using the ×9.5 rule for a university result when the university publishes its own formula.
Reporting only a converted number without the original CGPA and scale.
Picking whichever online calculator gives the highest number. Evaluators will recalculate it anyway, and a big gap looks bad.
Leaving it to the last week. Credential evaluations can take several weeks, so request them early.

FAQ

Is 8 CGPA good for a US master's?
It's generally considered solid, but admissions decisions depend on the program, your university's reputation, test scores, and your profile. A GPA number alone doesn't decide it.

Can I just multiply my CGPA by 0.4?
That's the same as the linear method. Fine as a rough estimate, but label it as one.

Does WES accept my self-calculated GPA?
No. WES calculates its own from your official transcripts.

My university uses a 7-point or 5-point scale. Does this apply?
The same logic holds: GPA = (your grade ÷ your scale maximum) × 4 for a linear estimate. The same caveats apply too.

Keep tracking while you're still studying

If you're still in university, the easiest way to avoid conversion stress later is to keep track of every semester's SGPA and credits as you go. The free Study OS 2027 Lite spreadsheet includes a grade tracker for this, plus a semester planner. Download it free. Any spreadsheet works, though. What matters is recording credits along with grades.

To sum up: get your CGPA right, use whatever conversion the institution asks for, always show the original, and never present a formula estimate as if it were official.

How to Plan Your Spring 2027 Semester (A Week-by-Week System That Actually Survives February)

Nidhi — Sat, 17 Oct 2026 17:25:20 +0000

Most semester plans die in week three. The real reason usually isn't laziness. The plan was built for an imaginary student with unlimited energy and no surprises. This guide walks you through a spring 2027 plan built the other way around: you start with the hard dates you already know, you protect a small amount of study time every week, and you leave room for the weeks that go badly.

You'll need a syllabus for every course, a calendar, and about 90 minutes. If you'd rather work from a ready-made sheet, there's a free template linked at the end. Everything below also works on paper.

Step 1: Collect every hard date in one place (30 minutes)

Open each syllabus and copy out every date that carries marks or can't be moved:

Exams and midterms
Assignment and project deadlines
Quizzes, labs and presentations
Add/drop and withdrawal deadlines (people forget these, and they matter most when a course is going badly)
Holidays and spring break

Put all of them in one list, sorted by date. Don't keep a separate list per course. The point is to see the collisions. If three deadlines land in the week of March 8, you want to know that in January, while you can still start one of them early.

For each item, write down its weight (the share of the final grade). A 5% quiz and a 30% project should never get the same amount of worry.

Step 2: Mark your crunch weeks

Count the weighted deadlines per week. Any week where the deadlines add up to more than about 20% of a course grade, or where two or more courses have major deliverables, is a crunch week. Most spring semesters have two or three: around midterms, just before spring break, and the last two weeks.

For every crunch week, pick one deliverable and move its personal deadline one week earlier. You aren't changing the real due date. You're just making sure the crunch week holds two big things instead of four.

Step 3: Build a minimum weekly study block

Here's where most plans go wrong. Students schedule 6 hours a night, miss it by day four, and then drop the plan entirely. Instead, decide on a minimum you could hit even in a bad week. That's usually 45–90 minutes per course per week of active review, plus the time assignments take.

A workable default for a 5-course load:

Day	Block 1 (45 min)	Block 2 (45 min)
Mon	Course A review	Course B problem set
Tue	Course C review	Course D reading
Wed	Course E review	Catch-up / flex
Thu	Course A practice questions	Course B review
Fri	Light: flashcards only	—
Sat	Assignment work	Assignment work
Sun	Weekly review (20 min)	Course C/D practice

The "flex" block matters. It's what keeps one bad day from wrecking the whole week.

Step 4: Use spaced review instead of rereading

Rereading notes feels productive but does surprisingly little for exam performance. Research on learning (Dunlosky et al., 2013, Psychological Science in the Public Interest) rates practice testing and distributed (spaced) practice as the two most useful techniques of the ten it reviewed. Rereading and highlighting score low.

In practice: after each lecture, write 3–5 questions on the material. Answer them again 1 day later, 3 days later, 7 days later and about 2–3 weeks later. Each round takes minutes. There's a full schedule for this in our spaced repetition study schedule guide.

Step 5: Track your grade as you go, not at the end

Most students only find out where they stand when final grades come out. Don't do that. Every time a mark comes back, log it with its weight. That lets you answer the most useful question of the semester: what do I need on the final to get the grade I want?

You can work that out in seconds: required final = (target − current × (1 − final weight)) ÷ final weight. If your current grade is 82% and the final is worth 25%, you need 114% to reach a 90 overall. That target is impossible, and it's far better to learn that in March than in May. Then you can put effort where it can still change the outcome.

To keep your grades, deadlines and weekly plan in one place all semester, you can use the free Study OS 2027 Lite planner template (disclosure: I made it).

Step 6: The 20-minute Sunday review

Once a week, answer four questions:

What's due in the next 14 days?
Which course am I furthest behind in?
Did I hit my minimum study blocks? If not, was the plan unrealistic, or did the week just go badly?
What's the one thing that would make next week easier?

Change the plan based on the answers. If you keep missing a block, shrink it. A 30-minute block you actually do beats a 2-hour block you skip.

Step 7: Plan for bad weeks on purpose

You'll get sick, or a family thing will come up, or you'll just lose a week to burnout. Decide ahead of time what "survival mode" looks like: deadlines only, plus 15 minutes of flashcards a day. When it happens, switch to survival mode without guilt, then come back to the normal plan the next week. Students who plan for bad weeks recover from them. Students who don't usually abandon the whole system.

A sample spring 2027 timeline

Dates vary by school, so treat this as a shape, not a calendar:

January, weeks 1–2: Collect dates, mark crunch weeks, set your minimum blocks. Low workload, so this is the cheapest time to get ahead.
Weeks 3–6: First quizzes and labs. Start spaced review from week 1's material.
Weeks 7–8 (midterms): First crunch. Your spaced review means you're revising, not learning from scratch.
Spring break: Rest for most of it. One or two short sessions to get ahead on the biggest late-semester project.
Weeks 10–13: Project-heavy. Run the required-final math to see which courses still have room to move.
Weeks 14–16 (finals): Turn your question bank into practice exams. Put the most time into the courses where the final carries the most weight and your target is still reachable.

Common mistakes

Planning by course instead of by date. Collisions stay hidden until it's too late.
Treating every assignment equally. Look at the weight.
Building a perfect-week plan. It breaks on the first imperfect week.
Only checking your grade at the end. You lose every chance to change it.
Too many tools. One calendar and one tracker is enough. Five apps means none of them gets updated.

A free template if you want one

I made Study OS 2027 Lite, a free Google Sheets/Excel planner with a deadline list sorted by date, a GPA tracker, and a study schedule laid out like the one above. Download Study OS 2027 Lite (free). There's also a paid full version ($12) with extra features, but you don't need it to use this method. A paper notebook works too.

The system matters more than the tool. Put every date in one list, keep your weekly minimum small, review on a schedule, track your grade as it comes in, and plan for the bad weeks. Do those five things and spring 2027 will feel a lot calmer than last semester.

The Spaced Repetition Study Schedule: A Practical Day-by-Day Plan for Students

Nidhi — Sat, 17 Oct 2026 17:25:12 +0000

Spaced repetition has a lot of evidence behind it, and most students still never use it, mostly because nobody shows them what it looks like on an ordinary Tuesday. This guide gives you a concrete schedule, explains why it works without overselling it, and shows how to fit it around a normal course load.

What spaced repetition is

Spaced repetition means reviewing material at gradually longer intervals instead of cramming it all at once. The basic finding goes back to Hermann Ebbinghaus in the 1880s: we forget quickly at first and then more slowly, and each well-timed review slows the forgetting further.

Modern research backs this up. A large review by Dunlosky and colleagues (2013) rated distributed practice and practice testing as the two highest-utility study techniques of the ten they examined. Cepeda et al. (2008) found that the best gap between reviews gets longer the longer you need to remember something, which is why intervals should expand over time.

It's not magic. It works best for material you need to recall: definitions, formulas, mechanisms, dates, vocabulary, problem types. It won't make up for not understanding a concept in the first place. But it is one of the best-supported ways to keep what you've learned.

The two ingredients: spacing + testing

Spacing on its own isn't enough. Rereading your notes at day 1, 3 and 7 is better than rereading them once, but the bigger gain comes from retrieval: making yourself recall the answer before you look at it. So each review session should be a mini-test, not a reread.

Ways to test yourself:

Flashcards (paper or an app like Anki)
Writing questions in the margin of your notes and covering the answers
Blank-page recall: write down everything you remember about a topic, then check
Past exam questions and end-of-chapter problems

The schedule

Here's a simple schedule that works for most courses. Day 0 is the day you first learn the material (the lecture or reading).

Review	When	Time needed
1	Day 1 (next day)	10–15 min
2	Day 3	10 min
3	Day 7	10 min
4	Day 14–21	5–10 min
5	Before the exam	5 min

If you get something wrong in a review, it goes back to a shorter interval. If you get it right easily, the next gap can be longer. Apps like Anki do this for you. On paper, a "Box 1 / Box 2 / Box 3" system (the Leitner system) does the same job.

What a real week looks like

Say you have lectures Monday, Wednesday and Friday in Biology. Your reviews pile up like this:

Tuesday: Review Monday's lecture (review 1).
Thursday: Review Wednesday's lecture (review 1) + Monday's lecture (review 2).
Saturday: Review Friday's lecture (review 1) + Wednesday's (review 2).
Next Monday: Review last Monday's lecture (review 3) + Friday's (review 2).

In any given week you're doing roughly 20–40 minutes of review per course, spread across several days. That's far less than a cram session, and you remember much more of it.

The catch is that the reviews pile up. By week 6 you're reviewing material from weeks 1, 3, 4 and 5 all at once. That's why each review has to be short and focused on testing yourself, and why a tracker helps.

Scaling to 5 courses

With five courses, schedule review as a fixed daily habit rather than per lecture:

Daily: 30–45 minutes of "due" reviews across all courses. Do whatever's due today, oldest first.
Weekly: one longer session (60 min) of mixed practice problems from older topics. Mixing topics (interleaving) also has decent evidence behind it, especially for maths and science problem-solving.
Before exams: turn your question bank into a timed practice test.

If a day gets missed, don't try to do two days' reviews tomorrow. Do today's, and add the missed ones over the next few days. Overdue reviews are still useful, only a little less so.

How to write good review questions

Bad questions make spaced repetition useless. A few rules:

One fact per question. "Explain the Krebs cycle" is too big. "What is the net ATP produced directly by one turn of the Krebs cycle?" is a good one.
Ask why, not just what. "Why does increasing temperature eventually decrease enzyme activity?" builds understanding as well as recall.
Use your own words. Copying textbook sentences gets you recognition, not recall.
Include worked problems. For quantitative subjects, the "question" is a problem you have to solve from scratch.
Delete questions you've mastered. If you've got one right four times over a month, retire it.

Tools (free options)

Anki: free on desktop and Android, and it schedules intervals automatically. It takes some learning.
Paper flashcards plus Leitner boxes: zero setup, and surprisingly effective.
A spreadsheet: list each topic, the date you learned it, and the next review date. Sort by due date each day.

The free Study OS 2027 Lite template has a review tracker that works like the spreadsheet option. You log a topic and it shows the next review dates. Get it free here. Anki works just as well if you prefer an app. Use whichever one you'll actually open every day.

Linking review to your grades

Spaced repetition helps most when you aim it well. Before an exam block, check where your grades stand. If the final in one course is worth 40% and your target is still reachable, that course should get the most review time. If a target is already secured or can't be reached, move that time to another course.

You can check quickly: work out what score you need on the final to hit your target, then plan your reviews around it. If you'd like a ready-made place to track grades and your review schedule, there's a free Study OS 2027 Lite planner template (disclosure: I made it).

Common mistakes

Rereading instead of recalling. It feels good and doesn't do much.
Cards that are too big. You'll mark them "kind of right" and fool yourself.
Starting the week before the exam. Spacing needs time. Start in week 1.
Skipping hard cards. The ones you get wrong are exactly the ones you need.
Never deleting anything. Your daily load grows until you quit.

How long until it pays off?

Be realistic. In the first week, spaced review feels slower than cramming, because you're spending time on material you won't be tested on for a month. The payoff shows up around midterms, when you find you're revising things you already know instead of learning them again from scratch. Most students who stick with it for four to six weeks say exam prep feels noticeably less frantic. The ones who quit usually stop in week two, before the reviews have had time to build up. If you can get through the first fortnight, the habit mostly runs itself.

A starter plan for this week

After your next lecture, write 5 questions on the material.
Tomorrow, answer them without notes (10 minutes).
Put reminders in your calendar for day 3 and day 7.
Repeat for one more course.
After two weeks, add the rest of your courses.

Start small. A 15-minute daily habit you keep for a whole semester will beat a perfect system you drop after a week, every time.

Feed a whole YouTube playlist into Claude or ChatGPT as one file (+ MCP setup)

Nidhi — Mon, 12 Oct 2026 03:08:31 +0000

Paste a YouTube link into Claude and ask for a summary, and one of three things happens: it tells you it can't open the link, it searches the web and summarizes what other people wrote about the video, or it quietly guesses from the title. None of those is the video.

What every assistant can do is read text. So the reliable workflow is:

Get the captions as text.
For a playlist, put all of them in one file in order, with a heading per video.
Upload that file, or connect a transcript tool over MCP so the assistant fetches it on its own.

This post covers both, with the free DIY route first.

How much fits? (do this maths before uploading)

Speech runs at roughly 130–160 words a minute, so one hour of video is about 8,000–10,000 words, or roughly 11,000–13,000 tokens for English. Rough guide:

Playlist	Approx. words	Approx. tokens
10 × 15-min videos	20k	27k
20 × 1-hour lectures	180k	240k
100 × 1-hour lectures	900k	1.2M

Context limits differ by model and plan and change often, so check yours. Two practical rules:

Under your model's context limit: upload the single file and ask questions across the whole playlist.
Over it: upload to a Claude Project or a ChatGPT Project / custom GPT as knowledge files. Those retrieve the relevant parts instead of reading everything at once, so split the playlist into a few files (e.g. one per module) with clear headings.

Step 1 (free): playlist → one file with yt-dlp

pip install yt-dlp
mkdir subs && cd subs
yt-dlp --skip-download --write-subs --write-auto-subs \
  --sub-langs "en.*" --sub-format vtt \
  -o "%(playlist_index)03d - %(title)s.%(ext)s" \
  "https://www.youtube.com/playlist?list=YOUR_PLAYLIST_ID"
cd ..

This downloads only subtitle files (no video), creator captions first, auto-captions otherwise, numbered in playlist order. Run it from a home connection; YouTube often blocks cloud and VPN IPs with a "confirm you're not a bot" error.

Then merge them. Raw .vtt auto-captions repeat each line two or three times, so dedupe while you merge:

import re
from pathlib import Path

TAG = re.compile(r"<[^>]+>")
out = ["# Playlist transcripts\n"]
for f in sorted(Path("subs").glob("*.vtt")):
    lines, last = [], None
    for raw in f.read_text(encoding="utf-8", errors="ignore").splitlines():
        s = TAG.sub("", raw).strip()
        if not s or s == "WEBVTT" or "-->" in s or s.startswith(("Kind:", "Language:")):
            continue
        if s != last:
            lines.append(s)
        last = s
    name = re.sub(r"\.[\w-]+\.vtt$", "", f.name)  # drop ".en.vtt"
    body = " ".join(lines)
    out.append(f"\n## {name}\n\n{body}\n")
Path("playlist.md").write_text("".join(out), encoding="utf-8")
print(len("".join(out).split()), "words")

Step 2: prompt it like a long document, not a video

Headings per video are what make this work. The model can cite "## 007 - Backpropagation" instead of vaguely saying "in one of the videos".

Prompts that use the whole playlist:

"This file has one section per video. For each section, give one line: the main claim and the best example used."
"Where do videos 3 and 11 disagree? Quote both."
"Build a study plan: what to rewatch, in what order, for someone who already knows X."
"Turn the whole series into a checklist I can follow, with the video number next to each step."

Ask it to quote when it matters. Quotes from a transcript are checkable; paraphrases can drift.

Step 3 (optional): let the assistant fetch transcripts itself with MCP

MCP (Model Context Protocol) lets Claude, ChatGPT, Cursor and other clients call external tools. With a YouTube transcript MCP server connected, you can say "get the transcripts for this playlist and compare videos 2 and 5", and the assistant fetches the text itself.

You can self-host an open-source transcript MCP server (several exist on GitHub, most built on youtube-transcript-api). They work well locally, but the same IP blocking applies when you deploy them to a cloud server.

Or use a hosted one. I built YT Transcriber, which has a remote MCP server (disclosure: I'm the maker, a student). The setup is the same pattern for any remote MCP server:

Get a key. Sign in, then Settings → API keys → Create, and copy it once. Then Settings → AI & Automations gives you the MCP server URL with the key included. Treat that URL like a password.
Claude: Customize → Connectors → Add custom connector. Name it, paste the MCP URL, and choose no extra sign-in, since the URL carries the key.
ChatGPT: Settings → Apps → Create MCP app. Paste the MCP URL, authentication: none.
Cursor: Settings → MCP → New MCP server, preferably with the key in a header instead of the URL:

{
  "mcpServers": {
    "yt-transcriber": {
      "url": "https://13-234-185-132.sslip.io/mcp",
      "headers": { "Authorization": "Bearer ytk_YOUR_KEY" }
    }
  }
}

The server exposes five tools: submit_transcripts (video, playlist or channel URLs), list_jobs, get_transcript, export_transcript (txt, md, json, toon, srt, pdf) and check_quota. In the web app, a finished playlist can also be downloaded as one file for this exact workflow, auto-split under 450,000 words between videos.

Honest limits: it reads existing captions (no speech-to-text), so caption-less videos fail. The free tier is 3 videos a day after sign-in, and you can try one video on the homepage without signing in. Whole playlists are $9/month.

Summary

Don't trust a model that says it "watched" a YouTube link. Give it the transcript.
One file per playlist, a heading per video, in playlist order.
Check the size against your model's context; use Projects or knowledge files when it's bigger.
MCP turns "go get the transcript" into something the assistant can do on its own.

How to get around NotebookLM's 50-source limit with YouTube playlists

Nidhi — Mon, 12 Oct 2026 03:08:30 +0000

NotebookLM (Google renamed it Gemini Notebook in July 2026, same product) is one of the best free ways to study from long video courses. You add a YouTube link, it reads the video's captions, and you can ask questions, make flashcards or generate an audio overview.

The problem shows up with real courses. Google's help pages list these limits for the free plan:

50 sources per notebook
500,000 words per source (or 200 MB per uploaded file)

Each YouTube link is one source. A semester course with 120 lectures needs three notebooks, and then you can't ask one question across the whole course.

The fix is in the second number. 50 sources is small, but 500,000 words per source is big. One hour of lecture speech is roughly 8,000–10,000 words of captions, so one source can hold around 45–55 hours of transcript. Turn the playlist into one text file and the course becomes one source, leaving 49 free slots for slides, papers and notes.

Below is the free, do-it-yourself way, then a shortcut if you don't want to touch a terminal.

What you need

Python 3
yt-dlp (pip install yt-dlp)
A playlist whose videos have captions (manual or auto-generated). This method reads captions; it does not transcribe audio, so a video without captions gives you nothing.

Step 1: Download only the captions for the whole playlist

mkdir subs && cd subs
yt-dlp --skip-download --write-subs --write-auto-subs \
  --sub-langs "en.*" --sub-format vtt \
  -o "%(playlist_index)03d - %(title)s.%(ext)s" \
  "https://www.youtube.com/playlist?list=YOUR_PLAYLIST_ID"
cd ..

What the flags do:

--skip-download fetches no video or audio, only subtitle files.
--write-subs --write-auto-subs takes the creator's captions if they exist, otherwise YouTube's auto-captions.
--sub-langs "en.*" picks English tracks. Change en to your language code. If you get the wrong language, run yt-dlp --list-subs URL on one video to see the exact track names.
%(playlist_index)03d puts a 001, 002, ... prefix on each file so the lectures stay in order.

If you get "Sign in to confirm you're not a bot" errors, you're probably on a cloud server or VPN. YouTube blocks many datacenter IPs. Run it from your normal home connection, and add --sleep-requests 1 for big playlists so you don't hammer YouTube.

Step 2: Combine the subtitle files into one clean Markdown file

Raw .vtt files are full of timing lines, formatting tags and, for auto-captions, every line repeated two or three times as the captions "roll". This script strips all of that, keeps the lectures in order under a ## heading each, and starts a new part file only if you go past 450,000 words (a safety margin under the 500k cap). It never cuts a lecture in half.

Save it as combine_subs.py:

#!/usr/bin/env python3
"""Combine every .vtt subtitle file in a folder into one Markdown file per
~450,000 words (under Gemini Notebook / NotebookLM's 500,000-words-per-source cap).
Usage: python3 combine_subs.py ./subs "My Course"
"""
import re, sys
from pathlib import Path

WORD_CAP = 450_000
TAG = re.compile(r"<[^>]+>")            # <00:00:01.234><c> word timing tags
CUE_TIME = re.compile(r"-->")           # 00:00:01.000 --> 00:00:03.000 lines

def vtt_to_text(path: Path) -> str:
    lines, last = [], None
    for raw in path.read_text(encoding="utf-8", errors="ignore").splitlines():
        line = TAG.sub("", raw).strip()
        if (not line or line == "WEBVTT" or CUE_TIME.search(line)
                or line.startswith(("Kind:", "Language:", "NOTE")) or line.isdigit()):
            continue
        if line == last:                # auto-captions repeat each line as they roll
            continue
        lines.append(line)
        last = line
    return " ".join(lines)

def main(folder: str, title: str) -> None:
    files = sorted(Path(folder).glob("*.vtt"))
    if not files:
        sys.exit(f"No .vtt files in {folder}")
    parts, cur, words = [], [f"# {title}\n"], 0
    for f in files:
        name = re.sub(r"\.[\w-]+\.vtt$", "", f.name)   # drop ".en.vtt"
        body = vtt_to_text(f)
        n = len(body.split())
        if words and words + n > WORD_CAP:              # start a new part between videos
            parts.append(cur)
            cur, words = [f"# {title} (continued)\n"], 0
        cur.append(f"\n## {name}\n\n{body}\n")
        words += n
    parts.append(cur)
    for i, p in enumerate(parts, 1):
        out = Path(f"{title} - part {i}.md" if len(parts) > 1 else f"{title}.md")
        out.write_text("".join(p), encoding="utf-8")
        print(f"wrote {out} ({sum(len(x.split()) for x in p):,} words)")

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else "Playlist")

Run it:

python3 combine_subs.py ./subs "Intro to Algorithms"
# example output: wrote Intro to Algorithms.md (182,000 words)

Big courses come out as ... - part 1.md, ... - part 2.md, each under the cap.

Step 3: Upload it to NotebookLM as one source

Open your notebook, click Add sources, and upload the .md file. If the upload dialog won't take .md on your account, rename it to .txt; it's plain text either way.
Because every lecture sits under its own ## 001 - Title heading, citations point you to the right lecture.
Use the 49 free slots for the course slides, papers and your own notes.

Prompts that work well on a whole-course source

"Make a one-page outline of the course, one line per lecture, using the lecture headings."
"Which lectures explain dynamic programming? Quote the key sentence from each."
"Write 20 exam-style questions that need ideas from at least two different lectures."

Limits to know (so you're not surprised)

Captions only. If a lecturer writes a formula on the board without saying it, it isn't in the transcript. Add the slides as a separate source.
Auto-captions make mistakes with names and technical terms. Spot-check anything you'll quote.
Timestamps are dropped by this script to save words. If you need to jump back to the exact moment, keep the original .vtt files, or export SRT instead.
Private, members-only and caption-less videos won't work with any caption-based method.

The shortcut (if you'd rather not run scripts)

I built a web tool, YT Transcriber, that does Steps 1 and 2 in the browser: paste a playlist or channel URL and download it as one file for NotebookLM, ChatGPT or Claude, auto-split under 450,000 words between videos. You can try one video on the homepage with no sign-in. With a Google sign-in it's 3 videos a day free, and $9/month for whole playlists. Disclosure: I'm the maker, a student. It reads captions like the script above, so the same limits apply.

The free script works fine too. The point of this post is the trick: one big file beats 50 small links.