Introduction
If you’re just getting started with Python, you’re probably wondering which libraries are essential and what problems they solve. I recently began my Python journey and compiled this list of must-know Python packages. Whether you’re into web development, data science, automation, or building APIs, these tools will come in handy.
1. Web Frameworks
- FastAPI — A modern web framework for building APIs with automatic Swagger documentation. Its fast, easy to learn and simple to use.
To install:
pip install "fastapi[standard]"
Example:
# main.py
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def home():
return {"Hello": "World"}
To run it, you would need to install Uvicorn
uvicorn main:app --reload
- Flask — A lightweight web framework for building web applications and APIs as it does not include built-in features like database abstraction layers, form validation, or extensive authentication systems. Instead, it focuses on providing the core functionalities for URL routing and page rendering.
To install:
pip install flask
Example:
from flask import Flask
app = Flask( __name__ )
@app.route("/")
def hello_world():
return "<p>Hello, World!</p>"
- Django — A high-level web framework that follows the Model-View-Template (MVT) pattern, a variation of the Model-View-Controller(MVC) pattern. It is a free and open-source, Python-based web framework designed for rapid development of interactive websites. It includes everything you need — no need to choose separate libraries for common features.
To install:
pip install django
To setup:
# Create project
django-admin startproject myblog
cd myblog
# Create app
python manage.py startapp blog
Example:
# Create a blog
# models.py - Define your data
from django.db import models
class Post(models.Model):
title = models.CharField(max_length=200)
content = models.TextField()
created_at = models.DateTimeField(auto_now_add=True)
def __str__ (self):
return self.title
# views.py - Handle requests
from django.shortcuts import render, redirect
from django.http import HttpResponse
from .models import Post
def home(request):
posts = Post.objects.all()
return render(request, 'home.html', {'posts': posts})
def create_post(request):
if request.method == 'POST':
title = request.POST.get('title')
content = request.POST.get('content')
if title and content:
Post.objects.create(title=title, content=content)
return redirect('home')
return render(request, 'create_post.html')
# urls.py - Define routes
from django.urls import path
from . import views
urlpatterns = [
path('', views.home, name='home'),
path('create/', views.create_post, name='create_post'),
]
# templates/home.html - Display data
<html>
<body>
<h1>My Blog</h1>
{% for post in posts %}
<div>
<h2>{{ post.title }}</h2>
<p>{{ post.content }}</p>
<small>{{ post.created_at }}</small>
</div>
{% endfor %}
<a href="/create/">Create New Post</a>
</body>
</html>
# templates/create_post.html - Create post-
<!DOCTYPE html>
<html>
<head>
<title>Add Blog</title>
</head>
<body>
<h1>Add new blog</h1>
<form method="post">
{% csrf_token %}
<input type="text" name="title" placeholder="Title" required><br>
<input type="text" name="content" placeholder="Content" required><br>
<button type="submit">Add</button>
</form>
<a href="/">Back to home</a>
</body>
</html>
To run:
python manage.py runserver
2. ASGI/WSGI Servers
ASGI and WSGI are server interface standards in Python for running web applications. They define the handling of requests and the interaction between your server and your code. WSGI serves as the conventional standard for synchronous Python web applications, whereas ASGI is its successor, tailored for asynchronous applications and able to accommodate both synchronous and asynchronous code
- Uvicorn — An ASGI server for running FastAPI and other async frameworks. When you install fastapi[standard] or fastapi[all]
- Uvicorn is automatically installed, unless you want a specific version.
# To install
pip install "fastapi[standard]"
# To run
uvicorn main:app --reload
- Gunicorn — A WSGI server for running Flask/Django applications in production. Use the WSGIs server like gunicorn
if you’re running Flask or Django (unless you’re adding async support to Django).
# To install
pip install gunicorn
# To run
gunicorn myapp:app
3. Data & Machine Learning
NumPy is short for Numerical Python, is an open-source library in Python for scientific computing. It supports large, multi-dimensional arrays and offers powerful tools for numerical computing.
pip install numpy
Example:
# To get the mean of a list
import numpy as np
arr = np.array([1, 2, 3])
print(arr.mean())
Pandas — A powerful library for data manipulation and analysis. It makes working with spreadsheet-like data (CSV files) easy to clean, analyze and manipulate it.
To install:
pip install pandas
Example:
import pandas as pd
df = pd.DataFrame({"name": ["Alice", "Bob"], "age": [25, 30]})
print(df.head())
Matplotlib & Seaborn — A plotting library for creating graphs and visualizations. Seaborn is built used for statistical data visualization.
To install:
pip install matplotlib seaborn
Example:
import seaborn as sns
import matplotlib.pyplot as plt
sns.set_theme()
sns.histplot([1, 2, 2, 3, 3, 3])
plt.show()
Scikit-learn — A machine learning library for tasks like classification, regression or clustering like predicting prices, classifying emails, or finding patterns in data. It comes with many built-in algorithms and datasets.
To install:
pip install scikit-learn
Example:
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestClassifier
import numpy as np
# Example 1: Predict house prices
# Data: [size, bedrooms] -> price
X = [[1000, 2], [1500, 3], [2000, 4], [2500, 4]] # features
y = [200000, 300000, 400000, 500000] # prices
# Train model
model = LinearRegression()
model.fit(X, y)
# Predict new house price
new_house = [[1800, 3]]
predicted_price = model.predict(new_house)
print(f"Predicted price: ${predicted_price[0]:,.0f}")
classifier = RandomForestClassifier()
TensorFlow — A deep learning framework used for building neural networks for image recognition, natural language processing, or complex pattern recognition.
To install:
pip install tensorflow
Example:
import tensorflow as tf
# Load dataset mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Normalize pixel values to [0, 1]
x_train, x_test = x_train / 255.0, x_test / 255.0
# Build model
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)), # Flatten image
tf.keras.layers.Dense(128, activation='relu'), # Hidden layer
tf.keras.layers.Dense(10, activation='softmax') # Output (10 classes)
])
# Compile and train
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3)
# Evaluate
loss, acc = model.evaluate(x_test, y_test)
print("Accuracy:", acc)
4.Databases & ORMs(Object Relational Mappers)
SQLAlchemy — A SQL toolkit and ORM for working with relational databases(PostgreSQL, MySQL, SQLite) and want to write Python instead of raw SQL. It provides both high-level ORM for easy database operations and low-level SQL toolkit for complex queries.
To install:
pip install sqlalchemy
Example:
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
email = Column(String(100))
# Setup
engine = create_engine('sqlite:///app.db')
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
# Create user
user = User(name="John", email="[email protected]")
session.add(user)
session.commit()
# Query users
users = session.query(User).filter(User.name == "John").all()
Pydantic- It is a library for data validation and parsing, and especially useful in FastAPI for defining request/response models. It has automatic validation with clear error messages, type conversion, and seamless integration with FastAPI. It comes with FastAPI when you install it.
from pydantic import BaseModel, EmailStr
from typing import Optional
class User(BaseModel):
name: str
email: EmailStr
age: int
is_active: Optional[bool] = True
# Valid data
user = User(name="John", email="[email protected]", age=25)
print(user.name) # "John"
# Invalid data - raises ValidationError
try:
User(name="John", email="not-an-email", age="not-a-number")
except ValidationError as e:
print("Validation failed!")
Psycopg2 — A database adapter for connecting Python with the PostgresQL database. It allows for direct access to the database with full control over SQL commands.
To install:
pip install psycopg2-binary
Example:
import psycopg2
# Connect
conn = psycopg2.connect(
host="localhost",
database="myapp",
user="postgres",
password="password"
)
cursor = conn.cursor()
# Execute SQL
cursor.execute("""
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name VARCHAR(50),
email VARCHAR(100)
)
""")
# Insert data
cursor.execute(
"INSERT INTO users (name, email) VALUES (%s, %s)",
("John", "[email protected]")
)
# Query data
cursor.execute("SELECT * FROM users WHERE name = %s", ("John",))
users = cursor.fetchall()
conn.commit()
cursor.close()
PyMongo — A MongoDB driver for Python applications. It provides direct interface to MongoDB with Pythonic API, perfect for unstructured or semi-structured data.
To install:
pip install pymongo
Example:
from pymongo import MongoClient
# Connect
client = MongoClient('mongodb://localhost:27017/')
db = client['myapp']
users = db['users']
# Insert document (any structure)
user = {
"name": "John",
"email": "[email protected]",
"preferences": {"theme": "dark", "lang": "en"}
}
users.insert_one(user)
# Find documents
john = users.find_one({"name": "John"})
dark_users = users.find({"preferences.theme": "dark"})
5. Networking & APIs
Requests — A simple library for making HTTP requests, download files and interact with web services. It is simple, clear syntax for HTTP requests.
To install:
pip install requests
Example:
import requests
# GET request
response = requests.get('https://api.github.com/users/octocat')
user_data = response.json()
print(user_data['name'])
HTTPX — An async alternative to Requests, and useful when build applications with FastAPI. The async/await supports allow for better performance.
To install:
pip install httpx
Example:
import httpx
import asyncio
# Synchronous (same as requests)
response = httpx.get('https://api.github.com/users/octocat')
print(response.json())
6. Testing & Debugging
Pytest — A framework for writing and running tests in Python.
To install:
pip install pytest
Example:
def add(x, y): return x + y
def test_add():
assert add(2, 3) == 5
7. Task Automation
Celery — A distributed task queue for handling background jobs. When you have long-running tasks that would block your web app, need distributed task processing across multiple servers, or require complex scheduling use Celery. Celery is battle-tested, supports multiple brokers, has advanced features like task routing, retries, and monitoring. Celery is enterprise ready, has a larger ecosystem and more features.
To install:
pip install -U Celery
Example:
# celery_app.py
from celery import Celery
# Create Celery app with Redis as broker
app = Celery('tasks', broker='redis://localhost:6379/0')
@app.task
def send_email(email, subject, body):
# This runs in the background
import time
time.sleep(5) # Simulate email sending
print(f"Email sent to {email}")
return f"Email sent successfully to {email}"
Celery use cases:
- E-commerce: Processing payments, sending order confirmations.
- Social media: Resizing uploaded images, generating thumbnails
- Analytics: Running reports, data processing pipelines.
Dramatiq — A simpler alternative to Celery for background task execution or building simpler applications. Its has cleaner API, better error handling out of the box, and easier to set up and maintain.
To install:
pip install -U 'dramatiq[all]'
Example:
# tasks.py
import dramatiq
import requests
from dramatiq.brokers.redis import RedisBroker
# Setup
redis_broker = RedisBroker(host="localhost", port=6379, db=0)
dramatiq.set_broker(redis_broker)
@dramatiq.actor
def fetch_user_data(user_id):
"""Fetch user data from external API"""
response = requests.get(f"https://api.example.com/users/{user_id}")
# Process and save data
return response.json()
Redis — A key-value store used for caching and message brokering commonly used with Celery. It shines when you need fast caching, session storage, real-time features, or a message broker for background tasks. Redis is extremely fast (in-memory), supports various data structures, and has built-in pub/sub capabilities.
To install:
pip install redis
Example:
import redis
import json
from datetime import timedelta
# Connect to Redis
r = redis.Redis(host='localhost', port=6379, db=0)
# 1. CACHING - Speed up database queries
def get_user_profile(user_id):
# Check cache first
cached = r.get(f"user:{user_id}")
if cached:
return json.loads(cached)
# Not in cache, fetch from database
user_data = fetch_from_database(user_id) # Slow DB query
# Cache for 1 hour
r.setex(f"user:{user_id}", timedelta(hours=1), json.dumps(user_data))
return user_data
8. Security & Authentication
Passlib — A password hashing library for when you need to securely store user passwords in your application. It handles password hashing complexities, supports multiple algorithms, and includes security best practices by default.
To install:
pip install passlib[bcrypt]
Example:
from passlib.context import CryptContext
# Create password context
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
# Hash a password
hashed = pwd_context.hash("my_secret_password")
# Verify a password
is_valid = pwd_context.verify("my_secret_password", hashed)
print(is_valid) # True
PyJWT — It is a Python library used when working with JSON Web Tokens (JWT) especially when building APIs that need stateless authentication or implementing single sign-on (SSO). It enables secure, compact token-based authentication without server-side session storage.
To install:
pip install pyjwt
Example:
import jwt
from datetime import datetime, timedelta
# Create a JWT token
payload = {
"user_id": 123,
"exp": datetime.utcnow() + timedelta(hours=24)
}
token = jwt.encode(payload, "secret_key", algorithm="HS256")
# Decode and verify token
try:
decoded = jwt.decode(token, "secret_key", algorithms=["HS256"])
print(f"User ID: {decoded['user_id']}")
except jwt.ExpiredSignatureError:
print("Token has expired")
9. Web Scraping & Parsing
Selenium — A browser automation tool often used for testing and web scraping. It controls a real browser so it works with dynamic content that Requests/ BeautifulSoup can’t handle.
To install:
pip install selenium
Example:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
# Setup browser (downloads driver automatically)
driver = webdriver.Chrome()
# Navigate to a page
driver.get('https://google.com')
# Find search box and type
search_box = driver.find_element(By.NAME, 'q')
search_box.send_keys('Python programming')
search_box.send_keys(Keys.RETURN)
# Wait for results to load
time.sleep(2)
# Get search results
results = driver.find_elements(By.CSS_SELECTOR, 'h3')
for result in results[:5]: # First 5 results
print(result.text)
# Take screenshot
driver.save_screenshot('page.png')
# Close browser
driver.quit()
BeautifulSoup — A library for parsing HTML and XML documents, mainly used for web scraping. It makes it easy to navigate and search HTML documents like a tree.
- Install:
pip install beautifulsoup4
- Example:
from bs4 import BeautifulSoup
import requests
# Scrape a webpage
response = requests.get('https://example.com/news')
soup = BeautifulSoup(response.content, 'html.parser')
# Find elements
title = soup.find('title').text
print(f"Page title: {title}")
10. Miscellaneous Utilities
Python-dotenv — This loads environment variables from a .env file. It manages environment variables, API keys, or configuration settings securely. It keeps sensitive data out of your code and makes configuration management clean and secure.
To install:
pip install python-dotenv
Example:
# .env file
DATABASE_URL=postgresql://user:pass@localhost/db
SECRET_KEY=your-secret-key-here
DEBUG=True
# Python code
from dotenv import load_dotenv
import os
load_dotenv()
database_url = os.getenv("DATABASE_URL")
secret_key = os.getenv("SECRET_KEY")
debug_mode = os.getenv("DEBUG") == "True"
Conclusion
These libraries form the foundation of most real-world Python projects. Whether you’re building APIs, working with data, or automating tasks, learning these tools early will boost your productivity and confidence.
Did I miss any essential package? Let me know!
Have a great one!!!
Author: Jane Nkwor
Thank you for being a part of the community
Before you go:
Whenever you’re ready
There are 4 ways we can help you become a great backend engineer:
- The MB Platform: Join thousands of backend engineers learning backend engineering. Build real-world backend projects, learn from expert-vetted courses and roadmaps, track your learnings and set schedules, and solve backend engineering tasks, exercises, and challenges.
- The MB Academy: The “MB Academy” is a 6-month intensive Advanced Backend Engineering Boot Camp to produce great backend engineers.
- Join Backend Weekly: If you like posts like this, you will absolutely enjoy our exclusive weekly newsletter, sharing exclusive backend engineering resources to help you become a great Backend Engineer.
- Get Backend Jobs: Find over 2,000+ Tailored International Remote Backend Jobs or Reach 50,000+ backend engineers on the #1 Backend Engineering Job Board.
Originally published at https://masteringbackend.com on July 30, 2025.
Top comments (0)