import requests
from bs4 import BeautifulSoup
import csv
# Fetch Hacker News job listings
response = requests.get('https://news.ycombinator.com/jobs')
html = response.text
soup = BeautifulSoup(html, 'html.parser')
# Extract job data
jobs = []
for item in soup.find_all('tr', class_='athing'):
title = item.find('span', class_='titleline').text.strip()
link = item.find('a')['href']
company = item.find('span', class_='subline').text.strip()
jobs.append({'title': title, 'link': link, 'company': company})
# Export to CSV
with open('hn_jobs.csv', 'w', newline='', encoding='utf-8') as file:
writer = csv.DictWriter(file, fieldnames=['title', 'link', 'company'])
writer.writeheader()
writer.writerows(jobs)
Tracking remote developer jobs manually is tedious. HN Job Scraper Pro automates this by fetching real-time data from Hacker News and exporting it to CSV. This tool saves hours of manual scraping and provides clean, structured data for hiring or career tracking.
The script uses Python's requests and BeautifulSoup libraries to scrape job listings from Hacker News. It extracts titles, links, and company names, then exports them to a CSV file. This makes it easy to analyze trends or monitor job postings without manual effort.
from datetime import datetime
import os
date = datetime.now().strftime('%Y-%m-%d')
output_path = f'hn_jobs_{date}.csv'
if not os.path.exists('exports'):
os.makedirs('exports')
with open(f'exports/{output_path}', 'w', newline='', encoding='utf-8') as file:
writer = csv.DictWriter(file, fieldnames=['title', 'link', 'company'])
writer.writeheader()
writer.writerows(jobs)
HN Job Scraper Pro includes a feature to automatically save daily job summaries to CSV files, organized by date. This helps users track hiring trends over time and maintain a historical record of job postings. The script also allows filtering by location, job type, and date range, making it flexible for different use cases.
The tool is designed for remote developers and recruiters who need real-time access to Hacker News job data. It eliminates the need for manual scraping, which can take hours each week. Instead, it provides clean, structured data that can be used for analysis or integration with other tools.
To get started, you can download the script and run it with Python 3.7+. The included requirements.txt file ensures all dependencies are installed with a single pip install -r requirements.txt command. The README.txt file provides step-by-step instructions for setup and usage.
For developers looking to automate job tracking or hiring processes, HN Job Scraper Pro is a powerful tool. It combines real-time data fetching with easy data export, making it ideal for those who want to stay ahead of the curve in the remote hiring landscape.
You can find the full script and documentation at HN Job Scraper Pro.
Top comments (0)