DEV Community

IntelliTools
IntelliTools

Posted on

HN Job Exporter: Automate Real-Time Job Tracking with Python

#python #automation #tutorial #productivity 
import requests
from bs4 import BeautifulSoup

def fetch_hn_jobs(page=0):
    url = f'https://news.ycombinator.com/jobs?show=story&score=100&sort=desc&desc=score&points=100&page={page}'
    headers = {'User-Agent': 'Mozilla/5.0'}
    response = requests.get(url, headers=headers)
    soup = BeautifulSoup(response.text, 'html.parser')
    jobs = []

    for item in soup.select('tr'):
        if 'hnuser' in item.get('class', []):
            title = item.select_one('span.title a').text.strip()
            link = item.select_one('span.title a')['href']
            company = item.select_one('span.com').text.strip()
            location = item.select_one('span.loc').text.strip()
            jobs.append({'company': company, 'role': title, 'location': location, 'link': link})
    return jobs
Enter fullscreen mode Exit fullscreen mode

The HN Job Exporter is a Python tool that automates the process of tracking Hacker News job postings. Unlike manual scraping, it handles pagination and network errors automatically, saving you hours of work each week. It exports clean CSV data with company, role, location, and link, making it ideal for remote developers and job seekers who need real-time updates.

import csv
from datetime import datetime

def export_jobs(jobs, filename='hn_jobs.csv'):
    with open(filename, 'w', newline='', encoding='utf-8') as f:
        writer = csv.DictWriter(f, fieldnames=['company', 'role', 'location', 'link'])
        writer.writeheader()
        writer.writerows(jobs)
    print(f'Exported {len(jobs)} jobs to {filename}')
Enter fullscreen mode Exit fullscreen mode

The tool includes a ready-to-run Python script that you can execute with minimal setup. It comes with a requirements.txt file, allowing you to install all dependencies with a single pip install command. The script is designed to be simple and efficient, with no unnecessary complexity.

if __name__ == '__main__':
    jobs = fetch_hn_jobs(page=0)
    export_jobs(jobs)
Enter fullscreen mode Exit fullscreen mode

This script fetches the first page of HN jobs and exports them to a CSV file. You can modify the page parameter to fetch more results or automate the process to run at regular intervals. The script is also robust, handling network errors gracefully and ensuring you don't miss any job postings.

The HN Job Exporter is a practical solution for developers who want to stay updated with real-time job postings without the hassle of manual scraping. It eliminates the need for copy-paste or manual data entry, making your job search more efficient. If you're looking for a reliable way to automate HN job tracking, this tool is a great choice.

For more information, visit HN Job Exporter to download the script and get started.

Top comments (0)