DEV Community: Ahmed Jasarevic

How to Get Reliable Options Chain Data in Python (When yfinance Keeps Failing)

Ahmed Jasarevic — Mon, 21 Sep 2026 14:57:30 +0000

TL;DR — Getting stock prices in Python is easy. Getting options chain data (strikes, expirations, implied volatility, open interest, volume) reliably and at scale is a different problem. This guide shows you how to pull clean, structured options data into a pandas DataFrame and build an options screener and IV dashboard on top of it — without fighting rate limits or rewriting your data layer every few weeks.

The problem nobody warns you about

You've probably built this pipeline before:

import yfinance as yf

aapl = yf.Ticker("AAPL")
chain = aapl.option_chain("2026-10-17")
calls = chain.calls
puts = chain.puts

It works beautifully in a notebook. Then you ship it.

And then, three weeks later, it returns empty DataFrames. Or throws a 429. Or works for 40 tickers and silently fails for the other 460. Or your nightly job just… doesn't have data this morning.

If you're building anything that depends on options data — a screener, a volatility dashboard, an alert bot, a backtest — the data layer is where projects die. Not the strategy. Not the model. The data layer.

This article is about fixing that layer once, so you can get back to building the interesting part.

Why options data is much harder than stock prices

Stock quotes are one number. Options chains are hundreds of contracts per ticker that each carry their own:

Strike price and expiration date
Contract type (call / put)
Implied volatility (IV)
Open interest and volume
Bid / ask / last price
In-the-money status
The underlying stock price at the moment of the snapshot

That's a lot of surface area. And unlike a simple price feed, options data is:

Large — a single liquid ticker can have thousands of contracts across all expirations.
Time-sensitive — IV and open interest shift throughout the trading day.
Painful to fetch at scale — you need hundreds of requests for a realistic watchlist.

Most free options data sources were never designed for this. They're fine for a demo with three tickers. They fall over the moment you point them at a real universe.

Why the free route breaks in production

If you've used yfinance or yahoo_fin, you already know the pattern. These libraries are excellent for exploration and research, and I genuinely recommend them for that. But they share a few characteristics that make production usage painful:

What you need	The free/DIY route
Predictable results at scale	Rate limiting appears as you scale up
Consistent schema across tickers	Fields shift; empty responses happen
Hundreds of tickers per run	You manage retries, throttling and pacing yourself
Fresh data on a schedule	Silent failures — you only notice when data is missing
Zero maintenance	Breaks when upstream changes

The core issue isn't that these tools are bad. It's that they're not a data pipeline. They're a convenience wrapper. When you build on top of them, you inherit 100% of the reliability burden — retries, backoff, session handling, proxy rotation, schema normalization, and monitoring.

That's a full-time engineering project. And it has nothing to do with the options strategy you actually want to build.

What "reliable" options data actually looks like

Before choosing a tool, get clear on the contract. You want each row to look like this — one row per option contract:

Field	Meaning
`ticker`	Underlying symbol (e.g. `AAPL`)
`symbol`	Full options contract symbol
`type`	`call` or `put`
`strike`	Strike price
`expiration`	Expiration date
`price`	Option contract price
`stockPrice`	Underlying stock price at scrape time
`iv`	Implied volatility
`volume`	Contracts traded
`openInterest`	Open contracts outstanding
`itm`	Whether the contract is in-the-money
`scrapedAt`	Timestamp of the snapshot

That's a clean, flattened schema — one row per contract, ready for a DataFrame, ready for CSV, ready for a database, ready for a chart. No nested JSON to unpack, no per-ticker field drift.

The goal is simple: you ask for tickers, you get back a tidy table.

Getting the data (in about 15 lines)

Instead of maintaining a scraper, you can call a managed data source that returns exactly the schema above. Here's the whole integration — a plain HTTP call, so it works from Python, Node, a cron job, or an agent:

import requests
import pandas as pd

APIFY_TOKEN = "YOUR_APIFY_TOKEN"
ACTOR = "ahmed_jasarevic~yahoo-finance-options"

resp = requests.post(
    f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items",
    params={"token": APIFY_TOKEN},
    json={
        "tickers": ["AAPL", "TSLA", "NVDA"],
        "limitPerTicker": 50,
    },
    timeout=300,
)
resp.raise_for_status()

df = pd.DataFrame(resp.json())
print(df.head())
print(f"{len(df)} contracts across {df['ticker'].nunique()} tickers")

That's it. No retry logic, no session management, no proxy setup, no per-ticker error handling.

Input options:

Field	Type	Required	Default	Notes
`tickers`	array	✅	—	Symbols to fetch, e.g. `["AAPL", "TSLA"]`
`limitPerTicker`	integer	❌	`20`	Number of call + put contracts per ticker
`proxyConfiguration`	object	❌	Apify Proxy	Residential proxy recommended for larger runs

A couple of things worth knowing before you build on it:

limitPerTicker controls cost and speed. Start at 20–50 while developing, raise it when you go live.
The output is one row per contract, so 50 tickers × 50 contracts = 2,500 rows. Size your DataFrame accordingly.
Pricing is per result (roughly $1.20 per 1,000 contracts at the free tier, and lower as your Apify plan tier goes up). A 10-ticker daily screener is fractions of a cent per run.

You can also run it interactively from the Store page if you just want to eyeball the output first:
👉 Yahoo Options — Chains, IV, Live

Use case 1 — Build an options screener

Now the fun part. With a clean DataFrame, a screener is genuinely a few lines.

Let's find liquid calls — high open interest, real volume, and an IV band that filters out both dead contracts and lottery-ticket noise:

import pandas as pd

# Normalise types
df["iv"] = pd.to_numeric(
    df["iv"].astype(str).str.replace("%", "", regex=False),
    errors="coerce",
)
df["openInterest"] = pd.to_numeric(df["openInterest"], errors="coerce")
df["volume"] = pd.to_numeric(df["volume"], errors="coerce")

calls = df[df["type"].str.lower() == "call"].copy()

screened = calls[
    (calls["openInterest"] > 500) &      # real liquidity
    (calls["volume"] > 100) &            # actually trading today
    (calls["iv"].between(15, 60)) &      # avoid dead + lottery contracts
    (calls["itm"] == False)              # out-of-the-money only
].sort_values("openInterest", ascending=False)

print(screened[["ticker", "strike", "expiration", "iv", "openInterest", "volume"]].head(10))

That table is the beginning of a real tool. From here you can add your own filters: distance from spot, days-to-expiration windows, spread width, IV rank against history — whatever your strategy needs.

Because the schema is stable, your screener logic never has to change when the data source does. That's the whole point.

Use case 2 — Track implied volatility over time

IV is the single most-watched number in options. But IV only becomes useful when you have history — today's IV means nothing without yesterday's to compare it to.

The pattern is: snapshot on a schedule → store → chart.

# Append each run to a CSV (or swap in SQLite / Postgres / DuckDB)
df["scrapedAt"] = pd.to_datetime(df["scrapedAt"], errors="coerce")
df.to_csv("iv_history.csv", mode="a", header=False, index=False)

Then compute a simple average IV per ticker over time:

history = pd.read_csv("iv_history.csv")
history["scrapedAt"] = pd.to_datetime(history["scrapedAt"])

trend = (
    history.groupby(["ticker", "scrapedAt"])["iv"]
    .mean()
    .reset_index()
    .sort_values("scrapedAt")
)

# A 30-point IV jump on a single name is worth looking at
latest = trend.groupby("ticker")["iv"].last()
previous = trend.groupby("ticker")["iv"].nth(-2)
spikes = (latest - previous).sort_values(ascending=False)
print(spikes.head())

You've now got the foundation of a volatility alert system — the thing that actually tells you when something interesting is happening in the market, instead of you refreshing a page all day.

Use case 3 — Automate it and stop thinking about it

The real value kicks in when this runs without you. Two options:

Scheduling: Apify supports cron-style schedules, so you can run the same configuration every market day at a fixed time — pre-market, at the open, or at the close.

Piping the output anywhere: Because the output is clean JSON, you can push it straight into Google Sheets, a webhook, Make/Zapier, your own database, or an AI agent that summarises what changed.

A practical setup for a watchlist monitor:

Schedule a run each trading morning with your ticker list.
Append results to your storage.
Diff against yesterday's snapshot.
Alert when IV, open interest or volume moves beyond a threshold you care about.

That's a monitoring product you'd otherwise spend a month building. It's now a config file and a schedule.

How this compares to a DIY pipeline

Capability	DIY free library	This managed actor
Get options chains	✅	✅
Flattened, stable schema	⚠️ varies	✅ one row per contract
Scale to hundreds of tickers	⚠️ rate limits	✅ built for it
Retries / throttling handled	❌ you build it	✅
Runs unattended on a schedule	⚠️ fragile	✅
Maintenance when upstream changes	❌ ongoing	✅ none
Cost per 1,000 contracts	"free" + your time	~$1.20

The honest framing: free libraries are great for learning and one-off research. A managed data source is what you use when the data needs to be there every single time — because you're shipping a product on top of it.

Frequently asked questions

Is there a free Yahoo Finance options API?

Yahoo shut down its official public API in 2017. Everything you see today — including yfinance and similar libraries — relies on unofficial endpoints, which is exactly why reliability varies over time. For production use, most developers move to a managed data source rather than maintaining an unofficial one themselves.

Why does yfinance return empty options data?

It usually comes down to rate limiting, throttling, or upstream changes. When you request many tickers quickly, you can hit limits that return empty or partial results instead of a clear error — which is the worst kind of failure, because your pipeline looks healthy while quietly producing nothing.

How do I get options chain data in Python?

Two routes: use a library that wraps an unofficial source (fast to start, fragile in production), or call a managed API that returns structured JSON you load straight into a DataFrame. The second route removes the maintenance burden entirely — the snippet above is the whole integration.

Can I get implied volatility and open interest, not just prices?

Yes — and those are usually the fields that matter most. Implied volatility tells you what the market expects, and open interest tells you where the real money is positioned. Any useful options tool is built on those two numbers far more than on raw contract price.

How much does it cost to pull options data at scale?

The actor is billed per result, roughly $1.20 per 1,000 contracts at the free tier and cheaper at higher plan tiers. A daily 10-ticker screener at 50 contracts each is 500 contracts — well under a cent per run.

Can I use this with an AI agent or LLM?

Yes. Because the output is structured JSON and the integration is a single HTTP call, it works cleanly as a tool for an LLM agent — useful for things like "summarise the biggest IV moves in my watchlist today."

Does it work for any ticker?

It's built for US-listed tickers with options chains. Set tickers to your watchlist and run it.

The takeaway

Options data doesn't have to be the fragile part of your project.

The pattern that works:

Get a stable, flattened schema — one row per contract.
Build your logic on top of it — screeners, IV tracking, alerts.
Schedule it so it runs without you.
Never maintain a scraper again.

Your edge is in what you do with the data. Everything upstream of that should be boring, predictable and boring again.

👉 Get started: Yahoo Options — Chains, IV, Live on Apify

Run it once in the UI to see the output, then drop the 15-line snippet into your project and get back to building.

If this saved you an afternoon of rate-limit debugging, drop a reaction — and tell me in the comments what you're building with options data. Screener, alert bot, or something weirder?

How I Built a Linktree, Beacons & Bio Email Scraper with Apify

Ahmed Jasarevic — Sun, 20 Sep 2026 20:42:11 +0000

1. Introduction

Collecting emails from Linktree, Beacons, and other bio links is often a tedious and time-consuming task. As a developer and freelancer working with lead generation projects, I faced this problem firsthand: manually navigating hundreds of profiles just to extract a few emails is inefficient and error-prone.
To solve this, I built a Linktree, Beacons & Bio Email Scraper Actor using Apify. It automates the process, reliably collects emails from multiple platforms, and provides structured outputs for marketing, data engineering, and business intelligence purposes.
This article details the journey: the challenges I faced, the design of the Actor, lessons learned, and tips for anyone looking to build production-grade scraping solutions.

2. Problem Context & Motivation

Manual email collection from bio links presents several challenges:

Dynamic pages: Platforms like Linktree often use JavaScript to load content asynchronously. Simple HTML scraping fails here.
Anti-scraping measures: Repeated requests can trigger rate limits or captchas.
Scale: For real lead-generation projects, you often need hundreds or thousands of emails daily—manual collection is not an option. I needed a solution that was fast, scalable, and reliable, and that could handle these technical challenges without breaking.

3. Actor Overview

Architecture and Workflow
The Actor is built on Apify and Crawlee, combining headless browser automation with structured data extraction. Here’s a high-level workflow:

Input: A list of Linktree, Beacons, or other bio URLs.
Navigation: Actor launches a headless browser to visit each URL.
Email Extraction: Uses DOM selectors and regex patterns to identify valid email addresses.
Anti-blocking: Rotates proxies and applies request throttling to avoid detection.
Output: Saves results in JSON or CSV with details: URL, email, timestamp.

Input, Output, and Configuration Options

Input: List of profile URLs (CSV, JSON, or manually typed).
Output: JSON, CSV, or Google Sheets integration.
Configurable Options: Maximum pages per run Timeout per request Proxy rotation (on/off) Output file format This flexibility allows the Actor to handle both small projects and large-scale campaigns.

4. Implementation Details

Handling Dynamic Content
Linktree and Beacons often load content via JavaScript. Initially, I tried simple HTTP requests, but emails were missing in the HTML response. Switching to Crawlee with Playwright solved the issue:

import { PlaywrightCrawler } from 'crawlee';

await crawler.run(['https://linktr.ee/example']);

This approach ensures the Actor captures all visible emails, even on dynamic pages.
Anti-Blocking Strategies
During early tests, some accounts triggered rate limits. To fix this, I added:
Proxy rotation
Randomized delays between requests
Error retries for failed pages
These measures increased the success rate to 99.6% over thousands of URLs.

5. Metrics & Results

To evaluate the Actor in a real-world scenario, I ran it against a dataset of 1,800+ URLs. The results showed strong reliability while keeping the cost of each run predictable:

99.6% success rate
1,816 emails collected
$14.47 total cost per run
$23.76 profit generated per run from the collected leads

These results demonstrated that the Actor could process large batches of URLs reliably while remaining cost-effective for lead-generation workflows.

This shows the Actor is not only technically reliable but also economically valuable for lead-generation workflows.

6. Lessons Learned

Building this Actor taught me several important lessons:
JavaScript rendering matters: Always test pages in a headless browser when dealing with dynamic content.
Anti-blocking is critical: Even simple rotation and throttling drastically improve success rates.
First-person debugging insights: Logging actual page content during development helped identify hidden issues.
Scalable design: Structuring input/output for batch processing makes the Actor production-ready.

7. Who Can Benefit

This Actor is useful for:

Developers looking to automate repetitive data collection tasks
Marketers and sales teams needing up-to-date email lists
Data engineers building pipelines that integrate multiple data sources
The architecture can also be adapted for other bio link platforms or email collection projects with similar challenges.

8. Conclusion

Automating email extraction from Linktree, Beacons, and bio links is no longer a manual nightmare. By using Apify and Crawlee, I built a reliable, scalable, and cost-effective Actor that delivers real-world results.
If you plan to build your own Actor, remember: focus on handling dynamic content, preventing blocks, and structuring your data pipeline. Sharing these lessons ensures other developers can build production-grade automation with confidence.

9. Next Steps

Test the Actor on new bio platforms
Add AI-powered validation to filter incorrect emails
Explore integrations with CRMs and email marketing tools