Filter High-Quality Tech Jobs from HackerNews in Seconds with Python

HackerNews is a goldmine of tech job listings, but manually sifting through hundreds of posts to find relevant opportunities is time-consuming. In this article, we’ll build a Python script that automates the filtering of high-quality tech jobs from HackerNews, saving you hours of manual screening. The script uses the HackerNews API to fetch job listings, applies a scoring system to prioritize quality, and exports clean results.

Why Automate HackerNews Job Filtering

Manually scanning HackerNews for job listings is inefficient. A typical developer might spend 2 hours per week sifting through posts, only to miss out on meaningful opportunities. By automating this process, we can focus on what matters: evaluating and applying to relevant jobs.

This script leverages the HackerNews API to fetch job posts, then applies a scoring system based on technical depth and relevance. The output is a clean list of jobs that can be exported for your team to use.

Building the Script from Scratch

Let’s start by importing the necessary libraries and setting up the API request. We’ll use the requests library to fetch data from the HackerNews API.

import requests

# HackerNews API endpoint for job posts
url = "https://hacker-news.firebaseio.com/v0/jobs.json"

# Fetch job data
response = requests.get(url)
jobs = response.json()

# Filter out non-job items
job_listings = [job for job in jobs if job.get('type') == 'job']
print(f"Found {len(job_listings)} job listings")

This script fetches all job listings from the HackerNews API and filters out any non-job items. The result is a list of job dictionaries that we can process further.

Scoring and Filtering Jobs

To prioritize high-quality jobs, we can implement a simple scoring system. We’ll look for keywords like "Python," "JavaScript," or "DevOps" and assign points based on the relevance and technical depth of the job description.

def score_job(job):
    score = 0
    description = job.get('description', '')
    title = job.get('title', '')

    # Basic keyword scoring
    if 'python' in description.lower():
        score += 10
    if 'javascript' in description.lower():
        score += 10
    if 'devops' in description.lower():
        score += 10
    if 'cloud' in description.lower():
        score += 10
    if 'machine learning' in description.lower():
        score += 20

    # Title-based scoring
    if 'python' in title.lower():
        score += 5
    if 'javascript' in title.lower():
        score += 5
    if 'devops' in title.lower():
        score += 5
    if 'cloud' in title.lower():
        score += 5
    if 'machine learning' in title.lower():
        score += 10

    return score

# Apply scoring to job listings
scored_jobs = [(job, score_job(job)) for job in job_listings]
scored_jobs.sort(key=lambda x: x[1], reverse=True)

This script adds a scoring system to each job listing, prioritizing those with more technical keywords. The result is a sorted list of jobs that are most relevant to Python developers and tech recruiters.

Exporting Clean Job Lists

Once we have our scored jobs, we can export them to a file for easy use. We’ll format the output to include the job title, description, and score.

with open("high_quality_jobs.txt", "w") as f:
    for job, score in scored_jobs[:20]:  # Top 20 jobs
        f.write(f"Title: {job.get('title')}\n")
        f.write(f"Description: {job.get('description')}\n")
        f.write(f"Score: {score}\n")
        f.write("-" * 40 + "\n")

This script writes the top 20 scored jobs to a text file, making it easy to share with your team or use in a recruitment process.

Final Thoughts

By automating the filtering of HackerNews job listings, you can save valuable time and focus on meaningful opportunities. The script we’ve built is a practical example of how Python can be used to enhance productivity in job screening processes.

If you're looking for a ready-to-use solution, you can find the HackerNews Job Auditor at https://intellitools.gumroad.com/l/hackernews-job-auditor. It includes all the code we've discussed, along with setup instructions and sample outputs.