DEV Community

PyFlowMaster
PyFlowMaster

Posted on

How to Build a Real-Time Taiwan Stock Scraper in 50 Lines of Python

#python #webscraping #automation #beginners

Finding reliable, real-time stock data for the Taiwan Stock Exchange (TWSE) can be a pain. The official API is clunky, and paid services are expensive.

But what if I told you that you can build your own scraper in less than 5 minutes using Python?

In this tutorial, I'll show you how to extract real-time price, percentage change, and stock names directly from Yahoo Finance TW using requests and BeautifulSoup.

Prerequisites

You'll need Python installed and two libraries:

bash
pip install requests beautifulsoup4
Enter fullscreen mode Exit fullscreen mode

The Code

Here is the complete, working script. No fluff.

import requests
from bs4 import BeautifulSoup
import time

def get_stock_price(symbol):
    # Target URL: Yahoo Finance TW
    url = f"https://tw.stock.yahoo.com/quote/%7Bsymbol%7D.TW"
We need to look like a real browser
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
    }

    try:
        response = requests.get(url, headers=headers)
        soup = BeautifulSoup(response.text, "html.parser")

Yahoo TW uses meta tags for SEO
        price = soup.find("meta", {"itemprop": "price"})["content"]
        name = soup.find("h1").text.strip()
        change = soup.find("meta", {"itemprop": "priceChange"})["content"]

        return {
            "Symbol": symbol,
"Name": name,
            "Price": price,
            "Change": change
        }
    except Exception as e:
        return {"Error": str(e)}

if name == "main":
    # Example: TSMC (2330)
    target = "2330"
    print(f"Fetching data for {target}...")
    data = get_stock_price(target)
    print(data)
Enter fullscreen mode Exit fullscreen mode

Why This Works

  1. Fake User-Agent: We send a browser-like header so Yahoo doesn't block our script.
  2. Meta Tags: Instead of relying on fragile CSS classes (which Yahoo changes often), we scrape the itemprop meta tags. These are much more stable.
  3. JSON Output: The function returns a clean dictionary, ready to be saved to Excel or sent to a database.

Taking It Further

This script is great for a quick check, but for a production-grade bot, you'll need:

  • Proxy Rotation: To avoid getting IP banned after 100 requests.
  • AsyncIO: To scrape 1000+ stocks in seconds, not minutes.
  • Database Integration: To store historical data for analysis.

Need a Custom Solution?

I build robust, high-performance web scrapers and automation bots for a living.

If you need a custom tool—like a Line Notification Bot that alerts you when TSMC hits a certain price, or a Daily PTT Sentiment Analyzer—I can build it for you.
👉 Hire me on Fiverr (PyFlowMaster)

Let's automate your workflow today!

Top comments (1)

Collapse
 
martijn_assie_12a2d3b1833 profile image
Martijn Assie •

Nice breakdown, straight to the point and no hand waving!
Using Yahoo’s meta tags instead of brittle CSS selectors is the real trick here...
For beginners this is actually readable and runnable, which is rare!!
The example code does have a few rough edges, but the core idea is solid and practical.
Good reminder that “real time” scraping is often just smart HTML parsing, not magic :)