Many developers try to extract data from websites by scraping HTML.
That approach usually involves:
- fetching the page
- parsing the DOM
- extracting elements using selectors
But modern web applications rarely render their data directly in HTML.
Instead, most sites load data through internal APIs that the frontend calls.
If you can find those APIs, you can often access clean structured data instead of parsing HTML.
The Trick: Use DevTools Network Tab
You can discover these APIs directly from the browser.
Step 1 — Open Chrome DevTools
Press: F12 or Right Click → Inspect
Step 2 — Go to the Network Tab
Open the Network tab inside DevTools.
This panel shows every request the page makes.
Step 3 — Filter API Requests
Click: Fetch/XHR
This filters the requests to show only API calls.
Now reload the page. Ctrl + R
Step 4 — Inspect JSON Responses
Look for requests returning JSON data.
Examples might look like:
/graphql
/api/posts
/api/content
/api/search
Click any request and open the Response tab.
You may see something like:
{
"title": "Example Article",
"author": "Jane Doe",
"content": "...",
"publishedAt": "2026-03-17"
}
This means you just discovered the API powering the page.
Why This Is Useful
Instead of scraping HTML, using the API often provides:
- structured JSON data
- faster requests
- more reliable results
- easier integrations
Many APIs require authentication or have usage restrictions.
Final Thoughts
Modern websites are essentially frontends built on top of APIs.
Once you know how to inspect network requests, you can quickly understand how data flows inside an application.
And sometimes, the easiest way to get data from a website is simply to use the same API the frontend already uses.
Top comments (0)