eBay is one of the world's largest online marketplaces, with over 1.7 billion live listings at any given time. For developers, researchers, and business analysts, eBay represents a goldmine of product data — pricing trends, seller behavior, market demand signals, and competitive intelligence.
In this comprehensive guide, you'll learn how to scrape eBay product listings, extract pricing and seller data, navigate pagination efficiently, and scale your scraping operation using cloud-based tools like Apify.
Why Scrape eBay?
eBay data is incredibly valuable for several use cases:
- Price tracking: Monitor product prices over time to spot deals or understand market trends
- Competitive intelligence: Understand what sellers are charging and how they position products
- Market research: Identify trending products, underserved niches, and demand patterns
- Arbitrage opportunities: Find price differences between eBay and other marketplaces
- Academic research: Study auction behavior, pricing dynamics, and marketplace economics
- Inventory monitoring: Track stock levels and availability of specific products
As always, respect eBay's Terms of Service and use data responsibly. Don't overload their servers, and don't use data for prohibited purposes.
Understanding eBay's Site Structure
eBay has a complex but well-organized site structure. Understanding it is the first step to building an effective scraper.
Search Results Pages
https://www.ebay.com/sch/i.html?_nkw=iphone+15&_pgn=1
Search result pages display listings in either grid or list view. Each listing card includes:
- Product title
- Price (Buy It Now or current bid)
- Shipping cost
- Item condition (New, Used, Refurbished)
- Seller name and rating
- Number of bids (for auctions)
- Time remaining (for auctions)
- Product image thumbnail
- Free returns badge
Product Detail Pages
https://www.ebay.com/itm/123456789012
Product pages are the richest data source and contain:
- Full product title and subtitle
- All product images (full gallery)
- Price details (Buy It Now price, auction information, best offer option)
- Item specifics (brand, model, color, size, UPC, etc.)
- Item condition and detailed condition description
- Seller information (username, feedback score, positive feedback percentage)
- Shipping options and costs
- Return policy details
- Item location
- Watchers count
- Number of units sold
Seller Profile Pages
https://www.ebay.com/usr/seller_username
Seller profiles provide:
- Total feedback score and positive percentage
- Member since date
- Location
- Recent feedback comments and ratings
- Active listings count
Navigating eBay Pagination
eBay search results pagination is one of the trickier aspects to handle correctly. Here's what you need to know.
URL Parameters
eBay uses query parameters to control search behavior. Understanding these lets you build precise search URLs programmatically:
const buildSearchUrl = (query, page = 1, options = {}) => {
const params = new URLSearchParams({
_nkw: query, // Search keywords
_pgn: page, // Page number
_ipg: 240, // Items per page (60, 120, or 240)
_sop: options.sort || 12, // Sort order
LH_BIN: options.buyItNow ? 1 : 0, // Buy It Now only
LH_Free: options.freeShipping ? 1 : 0, // Free shipping
_udlo: options.minPrice || '', // Min price
_udhi: options.maxPrice || '', // Max price
});
return `https://www.ebay.com/sch/i.html?${params}`;
};
Available Sort Options
| Value | Sort Order |
|---|---|
| 12 | Best Match (default) |
| 1 | Time: ending soonest |
| 10 | Time: newly listed |
| 15 | Price + Shipping: lowest first |
| 16 | Price + Shipping: highest first |
Pagination Limits and Workarounds
eBay typically limits search results to about 10,000 items — roughly 42 pages at 240 items per page. To get more results, split your queries:
- Use category filters to narrow the result set
- Apply price range filters to create smaller, non-overlapping result sets
- Sort by newly listed and scrape incrementally over time
// Strategy: Split by price ranges to bypass the 10K limit
async function scrapeFullCategory(query) {
const priceRanges = [
{ min: 0, max: 25 },
{ min: 25, max: 50 },
{ min: 50, max: 100 },
{ min: 100, max: 250 },
{ min: 250, max: 500 },
{ min: 500, max: null } // No upper limit
];
const allResults = [];
for (const range of priceRanges) {
const url = buildSearchUrl(query, 1, {
minPrice: range.min,
maxPrice: range.max
});
const results = await scrapeAllPages(url);
allResults.push(...results);
console.log(
`Price $${range.min}-$${range.max || '∞'}: ${results.length} items`
);
}
// Deduplicate by item ID
const unique = [...new Map(
allResults.map(item => [item.itemId, item])
).values()];
return unique;
}
Scraping Search Results
Here's a complete working example for extracting listing data from eBay search results across multiple pages:
const { chromium } = require('playwright');
async function scrapeEbaySearch(query, maxPages = 5) {
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext({
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ' +
'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
viewport: { width: 1440, height: 900 }
});
const page = await context.newPage();
const allItems = [];
for (let pageNum = 1; pageNum <= maxPages; pageNum++) {
const url = `https://www.ebay.com/sch/i.html?_nkw=${
encodeURIComponent(query)
}&_pgn=${pageNum}&_ipg=240`;
console.log(`Scraping page ${pageNum}...`);
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.srp-results .s-item', {
timeout: 10000
});
const items = await page.evaluate(() => {
const listings = document.querySelectorAll(
'.srp-results .s-item'
);
return Array.from(listings).map(item => {
const titleEl = item.querySelector('.s-item__title span');
const priceEl = item.querySelector('.s-item__price');
const linkEl = item.querySelector('.s-item__link');
const shippingEl = item.querySelector('.s-item__shipping');
const conditionEl = item.querySelector('.SECONDARY_INFO');
const sellerEl = item.querySelector(
'.s-item__seller-info-text'
);
const bidEl = item.querySelector('.s-item__bids');
const timeLeftEl = item.querySelector('.s-item__time-left');
return {
title: titleEl?.textContent?.trim(),
price: priceEl?.textContent?.trim(),
url: linkEl?.href?.split('?')[0],
shipping: shippingEl?.textContent?.trim(),
condition: conditionEl?.textContent?.trim(),
seller: sellerEl?.textContent?.trim(),
bids: bidEl?.textContent?.trim() || null,
timeLeft: timeLeftEl?.textContent?.trim() || null,
isAuction: !!bidEl
};
}).filter(item =>
item.title && item.title !== 'Shop on eBay'
);
});
allItems.push(...items);
console.log(` Found ${items.length} items`);
// Check if there's a next page
const hasNext = await page.$(
'.pagination__next:not(.pagination__next--disabled)'
);
if (!hasNext) break;
// Respectful delay between page loads
await new Promise(r =>
setTimeout(r, 2000 + Math.random() * 3000)
);
}
await browser.close();
return allItems;
}
Extracting Detailed Product Data
For richer data, you need to scrape individual product detail pages. This is slower but gives you the full picture:
async function scrapeEbayProduct(page, url) {
await page.goto(url, { waitUntil: 'domcontentloaded' });
const data = await page.evaluate(() => {
// Title
const title = document.querySelector(
'h1.x-item-title__mainTitle span'
)?.textContent?.trim();
// Price information
const priceEl = document.querySelector('.x-price-primary span');
const price = priceEl?.textContent?.trim();
const originalPriceEl = document.querySelector(
'.x-price-was span'
);
const originalPrice = originalPriceEl?.textContent?.trim();
// Condition
const condition = document.querySelector(
'.x-item-condition-text span'
)?.textContent?.trim();
// All product images
const images = Array.from(
document.querySelectorAll('.ux-image-carousel img')
).map(img => img.src || img.dataset.src).filter(Boolean);
// Item specifics (brand, model, color, etc.)
const specifics = {};
document.querySelectorAll(
'.ux-labels-values__labels-content'
).forEach((label, i) => {
const value = document.querySelectorAll(
'.ux-labels-values__values-content'
)[i];
if (label && value) {
specifics[label.textContent.trim()] =
value.textContent.trim();
}
});
// Seller information
const sellerName = document.querySelector(
'.x-sellercard-atf__info__about-seller a'
)?.textContent?.trim();
const sellerFeedback = document.querySelector(
'.x-sellercard-atf__about-seller span'
)?.textContent?.trim();
// Shipping details
const shippingCost = document.querySelector(
'.ux-labels-values--shipping .ux-textspans'
)?.textContent?.trim();
const deliveryDate = document.querySelector(
'.ux-labels-values--deliverto .ux-textspans--BOLD'
)?.textContent?.trim();
// Location
const itemLocation = document.querySelector(
'.ux-labels-values--itemLocation .ux-textspans'
)?.textContent?.trim();
// Social proof: watchers and sold count
const watchersEl = document.querySelector(
'.x-watch-count span'
);
const soldEl = document.querySelector(
'.x-quantity__availability span'
);
return {
title, price, originalPrice, condition,
images, specifics,
seller: { name: sellerName, feedback: sellerFeedback },
shipping: {
cost: shippingCost,
estimatedDelivery: deliveryDate
},
itemLocation,
watchers: watchersEl?.textContent?.trim(),
soldInfo: soldEl?.textContent?.trim()
};
});
return { ...data, url };
}
Extracting Seller Data
Understanding seller behavior and reputation is crucial for competitive analysis. Here's how to scrape seller profile data:
async function scrapeSellerProfile(page, username) {
const url = `https://www.ebay.com/usr/${username}`;
await page.goto(url, { waitUntil: 'domcontentloaded' });
const sellerData = await page.evaluate(() => {
const feedbackScore = document.querySelector(
'.str-seller-card__stats-content b'
)?.textContent;
const positivePercent = document.querySelector(
'.str-seller-card__stats-content span'
)?.textContent;
const memberSince = document.querySelector(
'.str-seller-card__member-since'
)?.textContent?.trim();
const location = document.querySelector(
'.str-seller-card__store-info span'
)?.textContent?.trim();
// Recent feedback entries
const feedbackItems = Array.from(
document.querySelectorAll('.fdbk-detail-list .card')
).map(card => ({
rating: card.querySelector('.fdbk-detail-list__icon')
?.getAttribute('aria-label'),
comment: card.querySelector('.fdbk-detail-list__comment')
?.textContent?.trim(),
item: card.querySelector('.fdbk-detail-list__item a')
?.textContent?.trim(),
date: card.querySelector('.fdbk-detail-list__date')
?.textContent?.trim()
}));
return {
feedbackScore,
positivePercent,
memberSince,
location,
recentFeedback: feedbackItems
};
});
return { username, ...sellerData };
}
Handling eBay's Anti-Scraping Measures
eBay has robust anti-bot protections. Here are the key techniques for handling them.
1. Session Management
eBay tracks sessions closely. Maintain consistent browser sessions rather than creating new ones for each request:
const context = await browser.newContext({
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ' +
'AppleWebKit/537.36',
locale: 'en-US',
timezoneId: 'America/New_York',
// Persist cookies across requests
storageState: './ebay-session.json'
});
2. Proxy Rotation
Rotating proxies is essential for large-scale eBay scraping to avoid IP-based blocking:
const proxyList = [
'http://proxy1:port',
'http://proxy2:port',
'http://proxy3:port'
];
async function getRandomProxy() {
return proxyList[Math.floor(Math.random() * proxyList.length)];
}
const browser = await chromium.launch({
proxy: { server: await getRandomProxy() }
});
3. Human-Like Browsing Behavior
Add realistic interactions — scrolling, mouse movements, and variable timing — to avoid triggering bot detection:
async function humanLikeBrowse(page) {
// Scroll down naturally
await page.evaluate(() => {
window.scrollBy(0, Math.random() * 500 + 200);
});
await new Promise(r =>
setTimeout(r, 1000 + Math.random() * 2000)
);
// Move mouse to a random position
await page.mouse.move(
Math.random() * 1200 + 100,
Math.random() * 600 + 100
);
}
4. Retry Logic with Exponential Backoff
Always implement retry logic to handle transient failures gracefully:
async function scrapeWithRetry(fn, maxRetries = 3) {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await fn();
} catch (error) {
console.log(`Attempt ${attempt} failed: ${error.message}`);
if (attempt === maxRetries) throw error;
const delay = Math.pow(2, attempt) * 1000
+ Math.random() * 1000;
await new Promise(r => setTimeout(r, delay));
}
}
}
Scaling with Apify
When you need to scrape thousands or millions of eBay listings, managing your own browser infrastructure becomes impractical. Apify provides a cloud-based platform specifically designed for web scraping at scale.
Using Apify for eBay Scraping
The Apify Store offers pre-built eBay scraper actors that handle all the hard parts — proxy rotation, browser management, anti-bot evasion, retry logic, and structured data storage.
const { ApifyClient } = require('apify-client');
const client = new ApifyClient({
token: 'YOUR_APIFY_TOKEN',
});
async function scrapeEbayAtScale() {
const run = await client.actor('ACTOR_ID').call({
searchQueries: [
'iphone 15 pro max',
'macbook pro m3',
'sony ps5'
],
maxItems: 1000,
scrapeProductPages: true,
proxyConfiguration: {
useApifyProxy: true,
apifyProxyGroups: ['RESIDENTIAL']
}
});
const { items } = await client.dataset(
run.defaultDatasetId
).listItems();
console.log(`Scraped ${items.length} eBay listings`);
// Download as CSV
const csvUrl = `https://api.apify.com/v2/datasets/${
run.defaultDatasetId
}/items?format=csv`;
console.log(`Download CSV: ${csvUrl}`);
return items;
}
Scheduling Regular Scrapes
For ongoing price monitoring, set up automated scheduled runs:
const schedule = await client.schedules().create({
name: 'ebay-daily-price-check',
cronExpression: '0 9 * * *', // Every day at 9 AM
actions: [{
type: 'RUN_ACTOR',
actorId: 'ACTOR_ID',
runInput: {
searchQueries: ['gaming laptop'],
maxItems: 500
}
}]
});
Webhook Integration for Data Pipelines
Get notified when your scrape completes so you can trigger downstream processing:
const run = await client.actor('ACTOR_ID').call(input, {
webhooks: [{
eventTypes: ['ACTOR.RUN.SUCCEEDED'],
requestUrl: 'https://your-app.com/api/ebay-data-ready',
payloadTemplate: JSON.stringify({
datasetId: '{{resource.defaultDatasetId}}',
itemCount: '{{resource.stats.itemsScraped}}'
})
}]
});
Search Pages vs. Product Pages: When to Use Each
Choosing the right scraping strategy depends on your data needs:
| Data Need | Search Pages | Product Pages |
|---|---|---|
| Basic price comparison | Sufficient | Overkill |
| Seller identification | Yes | Yes |
| Full item specifics | No | Yes |
| Image gallery | Thumbnail only | Full gallery |
| Shipping details | Basic | Detailed |
| Item condition details | Basic label | Full description |
| Auction bid history | Bid count only | Full history |
| Speed | Fast (240 items/page) | Slow (1 item/page) |
| Proxy cost | Low | Higher |
Best practice: Start with search pages to identify items of interest, then selectively scrape product pages only for items that match your specific criteria. This two-pass approach is far more efficient than scraping every product page.
Extracting Completed/Sold Listings
One of eBay's most valuable datasets for market research is completed listings — items that actually sold at a real price:
function buildSoldListingsUrl(query) {
const params = new URLSearchParams({
_nkw: query,
LH_Complete: 1, // Completed listings
LH_Sold: 1, // Sold items only
_sop: 13, // Sort: recent first
_ipg: 240
});
return `https://www.ebay.com/sch/i.html?${params}`;
}
Completed listings show you what people actually paid, not what sellers hope to get. This is incredibly valuable for pricing research and market analysis.
Data Analysis Patterns
Once you've collected the data, here are practical analysis patterns.
Price Distribution
function analyzePrices(items) {
const prices = items
.map(item => parseFloat(
item.price?.replace(/[^0-9.]/g, '')
))
.filter(p => !isNaN(p));
prices.sort((a, b) => a - b);
return {
count: prices.length,
min: prices[0],
max: prices[prices.length - 1],
median: prices[Math.floor(prices.length / 2)],
average: (
prices.reduce((a, b) => a + b, 0) / prices.length
).toFixed(2),
p25: prices[Math.floor(prices.length * 0.25)],
p75: prices[Math.floor(prices.length * 0.75)]
};
}
Seller Competition Analysis
function analyzeCompetition(items) {
const sellerMap = new Map();
items.forEach(item => {
if (!item.seller) return;
if (!sellerMap.has(item.seller)) {
sellerMap.set(item.seller, {
count: 0, totalValue: 0
});
}
const seller = sellerMap.get(item.seller);
seller.count++;
seller.totalValue += parseFloat(
item.price?.replace(/[^0-9.]/g, '')
) || 0;
});
return Array.from(sellerMap.entries())
.map(([name, data]) => ({
seller: name,
listingCount: data.count,
totalValue: data.totalValue.toFixed(2),
avgPrice: (data.totalValue / data.count).toFixed(2)
}))
.sort((a, b) => b.listingCount - a.listingCount)
.slice(0, 20);
}
Best Practices and Tips
Start with search pages: Extract basic data from search results before hitting individual product pages to minimize requests.
Use item IDs for deduplication: eBay item numbers are globally unique — use them to avoid storing duplicate listings.
Handle auction vs. Buy It Now: These listing types have different data structures and require different extraction logic. Always check which type you're dealing with.
Watch for regional differences: eBay has country-specific domains (ebay.co.uk, ebay.de, ebay.com.au) with different HTML layouts and price formats.
Monitor completed listings: Add
LH_Complete=1&LH_Sold=1to get actual sold prices rather than aspirational listing prices.Respect rate limits: Keep requests under 20 per minute per IP to avoid blocks. Use random delays.
Cache aggressively: Product pages change slowly. Cache data and refresh on a schedule rather than re-scraping everything.
Validate extracted prices: eBay shows prices in many formats — "$29.99", "$25.00 to $35.00", "GBP 19.99". Build robust parsing logic.
Handle "Best Offer" listings: Many listings accept offers below the listed price. The listed price may not reflect actual transaction values.
Use structured data when available: Like Etsy, eBay embeds JSON-LD structured data in some pages. Parse it for more reliable extraction.
Conclusion
eBay scraping opens up a world of market intelligence — from real-time price tracking to competitive analysis and trend detection. The key challenges are handling eBay's anti-bot protections, managing pagination efficiently, and correctly processing the different listing formats (auctions, Buy It Now, variations, and Best Offer).
For small-scale projects and learning, a custom Playwright scraper gives you full control and deep understanding. For production workloads where reliability and scale matter, platforms like Apify with their pre-built eBay actors, managed proxy infrastructure, and scheduling capabilities can save you significant development and ongoing maintenance effort.
Browse the Apify Store for eBay-focused actors that handle the complexity of proxy rotation, browser management, and anti-detection measures out of the box — letting you focus on analyzing the data rather than collecting it.
Start with a specific use case, build a focused scraper, validate your data quality, and expand from there. Happy scraping!
Need production-ready eBay scraping? Check out the Apify Store for pre-built actors that handle all the infrastructure complexity so you can focus on your data.
Top comments (0)