DEV Community: MarKSmaN98

Top 8 Python Libraries for Web Developers

MarKSmaN98 — Thu, 06 Apr 2023 13:34:18 +0000

Python libraries extend the functionality of an already very multifunctional language. Here are the top 8 for backend developers!

Pillow

The Python Pillow library is a powerful image processing library that provides extensive support for opening, manipulating, and saving many different image file formats. It provides various functions and classes to perform image processing tasks, such as resizing, cropping, rotating, filtering, and more. Pillow is a fork of the Python Imaging Library (PIL) and has become the de-facto standard for working with images in Python. It is widely used for tasks such as creating thumbnails, processing images for computer vision applications, and generating visualizations. Overall, the Pillow library is a versatile and useful tool for any Python developer who needs to work with images.

Bcrypt

The Python bcrypt library is a popular library used for password hashing and verification. It is a lightweight library that uses the bcrypt algorithm, which is a strong and secure hashing algorithm that is designed to be slow and difficult to brute-force. The bcrypt algorithm also incorporates a salt value, which helps to prevent against attacks such as rainbow table attacks. The bcrypt library provides a simple and easy-to-use API for hashing passwords and verifying them. It also allows for customization of the hashing process, such as specifying the number of iterations and the salt value used. The bcrypt library is widely used in web applications and is a recommended library for securely storing user passwords.

Pyinstaller

The Python PyInstaller library is a popular tool used for packaging Python scripts into standalone executables. It allows developers to create distributable packages of their Python applications that can be run on other machines without the need to install Python or any third-party libraries. PyInstaller supports various operating systems, including Windows, Linux, and macOS, and can package applications into various formats, such as EXE, MSI, and DMG.

PyInstaller works by analyzing the dependencies of a Python script and creating a self-contained package that includes the Python interpreter, the necessary libraries, and the application code. It also provides options for customizing the packaging process, such as specifying additional data files and including third-party libraries that are not automatically detected.

Overall, PyInstaller is a powerful tool for Python developers who want to create standalone applications that can be easily distributed and run on other machines. It simplifies the process of packaging Python applications and provides a way to distribute Python applications to users who may not have Python or the necessary libraries installed.

Scrapy

The Python Scrapy library is a popular open-source web crawling framework that is used for extracting data from websites. It provides a powerful set of tools for scraping and parsing data from websites, and is designed to handle large volumes of data with efficiency and speed.

Scrapy allows developers to define the structure of the data they want to extract using a set of rules and selectors, and provides various features for handling common web crawling tasks, such as handling cookies and session management, following links, and managing user-agent strings.

Scrapy is built on top of the Twisted networking engine, which allows it to handle asynchronous I/O operations and handle large volumes of requests simultaneously. It also supports various output formats, such as JSON and CSV, and can be integrated with other Python libraries and databases for further processing of the scraped data.

Overall, Scrapy is a powerful tool for web scraping and is widely used in a variety of applications, such as data mining, information retrieval, and machine learning.

Faker

The Python Faker library is a popular open-source library used for generating realistic and randomized test data. It provides a set of functions and classes that can be used to generate data for a variety of fields, such as names, addresses, phone numbers, dates, and more.

Faker is designed to generate data that looks realistic and is suitable for use in testing and development environments. It can generate data in various languages and locales, and provides options for customizing the generated data, such as specifying gender or country-specific data.

Faker is easy to use and provides a simple and consistent API for generating test data. It is widely used in test automation, data analysis, and other applications where realistic and randomized data is needed.

Overall, the Python Faker library is a useful tool for developers who need to generate realistic and randomized data for testing and development purposes, and can save a lot of time and effort in generating test data manually.

SQLAlchemy

The Python SQLAlchemy library is a popular open-source library used for working with relational databases. It provides a high-level and powerful set of tools for interacting with databases using an Object-Relational Mapping (ORM) approach, which allows developers to work with databases using Python objects and classes.

SQLAlchemy supports various database systems, including PostgreSQL, MySQL, SQLite, and Oracle, and provides a consistent and flexible API for working with different databases. It also provides support for transactions, data integrity, and concurrency control, and includes various features for querying databases, such as filters, joins, and aggregates.

SQLAlchemy can be used in various scenarios, such as web applications, data analysis, and reporting, and can be integrated with other Python libraries and frameworks. It also provides options for customizing the database schema and defining relationships between tables using Python classes and metadata.

Overall, the Python SQLAlchemy library is a powerful and versatile tool for working with relational databases in Python, and is widely used in many industries and applications.

Flask

The Python Flask library is a popular open-source micro web framework used for developing web applications. It provides a simple and flexible set of tools for building web applications and APIs, and is designed to be lightweight and easy to use.

Flask is built on top of the Werkzeug toolkit and the Jinja2 template engine, and provides various features for handling web requests and responses, such as routing, request handling, and response rendering. It also includes support for various extensions, such as SQLAlchemy for database access, WTForms for form handling, and Flask-RESTful for building RESTful APIs.

Flask is highly customizable and provides options for defining application configurations, error handling, and security. It can also be integrated with other Python libraries and frameworks, such as Numpy and Scikit-Learn, for data analysis and machine learning applications.

Overall, the Python Flask library is a popular and widely used web framework that provides a simple and flexible way to build web applications and APIs in Python. It is well-suited for small to medium-sized projects and is a great choice for developers who want a lightweight and easy-to-use framework for building web applications.

Textual

The Python Textual library is an open-source library used for building interactive command-line interfaces (CLIs) in Python. It provides a set of tools and abstractions for creating modern, feature-rich, and user-friendly CLI applications.

Textual is built on top of the Python Prompt Toolkit library and provides various features for handling user input and rendering output in the terminal, such as syntax highlighting, auto-completion, and pagination. It also includes support for various UI elements, such as dialogs, tables, and progress bars, and provides options for customizing the appearance and behavior of the CLI application.

Textual is easy to use and provides a consistent and intuitive API for building CLI applications. It also includes various utilities for handling common CLI tasks, such as argument parsing, configuration management, and logging.

Overall, the Python Textual library is a powerful and modern tool for building interactive CLI applications in Python, and is suitable for a wide range of use cases, such as system administration, data processing, and automation. It can save a lot of time and effort in building complex CLI applications and can provide a better user experience for CLI users.

Thank you for reading! These are some of my favorite libraries so far. Please leave a comment of your favorite library!

Python and SQL for beginners

MarKSmaN98 — Thu, 30 Mar 2023 16:01:43 +0000

Python Introduction

Python is a high-level, interpreted programming language that was first released in 1991. It is known for its simplicity, readability, and flexibility, and is widely used in a variety of applications, including web development, scientific computing, data analysis, and artificial intelligence. Python's syntax is designed to be easy to read and write, with a focus on code readability and minimalism. Its extensive standard library and large community of third-party packages make it a popular choice for developers of all skill levels. Python's popularity has also been driven by the rise of data science and machine learning, where it has become one of the primary languages used for these fields. Overall, Python is a versatile language that is well-suited to a wide range of applications and is a great choice for both beginner and experienced programmers. Note: I will be talking about Python3 for this blog. Some syntax may be different between Python2 and 3, and some commands may be different or not even exist!

SQL Introduction

SQL (Structured Query Language) is a programming language designed for managing and manipulating data stored in relational databases. SQL is widely used in many industries, including finance, healthcare, and e-commerce, to store, retrieve, and manage large amounts of data. SQL provides a standard set of commands for creating, modifying, and deleting database structures and data, as well as querying and manipulating data within those structures. SQL syntax is designed to be simple and easy to learn, with a focus on readability and expressiveness. The popularity of SQL has led to the development of many different database management systems, each with its own set of extensions and variations on the SQL language. Despite these variations, the core principles of SQL remain the same, making it a powerful and widely-used tool for managing data.

Why not combine two great things?

Python is knows for simplicity, readability, and amazing flexibility. The language is popular in fields from web development, game dev, AI, to data science and robotics. Python is an easy to grasp language for beginners and a near endless toolbox for experienced programmers. I'm sure, dear reader, that you can imagine the need and usefulness of storing your program's data for future use, even after exiting the program. Thankfully we can do this with databases. While we do have a few options for saving our data such as writing to a txt file or creating a json file, using a database file will prepare us for tackling bigger, potentially professional jobs later on.

Starting with sqlite3

Here I will walk through the installation of sqlite3, a lightweight and embedded relational database management system (RDBMS) that is implemented in the C programming language, and therefore able to be used in python!

A quick note from personal experience: sqlite3 has been included in Python since 2.5.x, but if you've not configured your Python install correctly you might find you can't use it. There are many tutorials on the web about how to uninstall, configure, and reinstall Python to get your packages working.

The first thing we want to do to get started using SQL and sqlite3 is to import the module:

import sqlite3

if this is your first time using this module you may want to run your file immediately to see if the program executes successfully or if you get an error stating sqlite3 can't be found.

at this point we want to create the heart of sqlite3, the connect and cursor.
The connect establishes a link between your .py file and the database. You can specify an existing db file, let connect create one for you, or use ':memory:' to create a temporary database stored in RAM.
cursor is the method you will use throughout your program to create, read, update, or delete tables and their data.

CONN = sqlite3.connect('my_db.db')
CURSOR = CONN.cursor()

From here we can use the cursor to create a table.

CURSOR.execute('''CREATE TABLE example (id INTEGER PRIMARY KEY, data TEXT)''')

To add data to our new table:

CURSOR.execute("INSERT INTO ex (data) VALUES ('Hello World')")

and to query the table's data:

res = CURSOR.execute('SELECT * FROM ex')
print(res.fetchall())
# -> [(1, 'Hello World')]

Sqlite3 uses SQL syntax and commands, so we can do anything we can do in a terminal in python. Create multiple tables, join them, migrate data, etc.

Making our lives easier

As you can see we need a bit of patience to write all those sql commands. while not the worst, it certainly isn't the best at avoiding needless repetition when trying to do nearly any operation.

We can make our lives easier by implementing a program called SQLAlchemy. This program allows us to use SQL by only executing python, and can drastically cut down on repetition.

Starting with SQLAlchemy

Unlike sqlite3, SQLAlchemy is not included in python by default. We will need to use pip to install it.

$ pip install sqlalchemy

now we can import the module into our Python code:

import sqlalchemy

The heart of sqlalchemy is the engine. This is a stop gap between your code and the database that interprets python and generates SQL queries for you. The session is your gateway between your program and the database. To get started we will type:

from sqlalchemy import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()
engine = sql.create_engine('sqlite:///your_db_here')
Session = sessionmaker(bind=engine)
session = Session()

From here we can do any SQL operation we wish. Some examples are:

Create a table:

base.metadata.create_all(engine)
class Example(Base):
    __tablename__='example_table'
    id = Column(Integer(), primary_key=True)
    data= Column(String())

^^ So just for presentation, I'm pretty sure that when you instantiate an instance of a class sqlalchemy is interpreting that class, grabbing the table name dunder method, your column declarations, and adding a table to the database. Could def be wrong here tho...

Insert into a table:

ex = Example(data="Hello World")
session.add(ex)
session.commit()

Insert multiple using bulk_save_objects(list)

ex1 = Example(data = "Hello")
ex2 = Example(data = "World")
session.bulk_save_objects([ex1, ex2])

We have many more options such as

session.query(Example)
session.query(Example).filter(condition, condition2, condition3, ...)
session.query(Example).filter(condition).first()
session.query(Example).filter(condition).many()
session.query(Example).filter(condition).delete()
session.query(Example).order_by(column)
session.query(Example).order_by(desc(column))
session.query(Example).filter(condition).update({column: new_value})

Overall, SQLAlchemy is a powerful and flexible library that makes it easy for developers to work with relational databases using Python. Its ORM layer and powerful query API make it easy to manipulate data, and its support for transactions and connection pooling make it a good choice for high-concurrency applications. If you're building a Python application that needs to work with a relational database, SQLAlchemy is definitely worth considering.

Thanks for reading, and as always, I can't fit an entire library's doc into a blog, so I haven't explained or used even a small fraction of the above libraries functionality. I strongly encourage anyone interested to go ahead and read up on the docs, or just play around until you get the hang of the programs.

Python Decorators

MarKSmaN98 — Fri, 17 Mar 2023 13:49:38 +0000

Python is a high-level programming language that offers a lot of flexibility to programmers. One of the features that make Python such a versatile language is decorators. Decorators are functions that modify other functions or classes, and they provide a way to add functionality to your code without changing the original code.

In this blog post, we will explore what Python decorators are, how they work, and some common use cases.

What are Python decorators?

In Python, a decorator is a function or class that takes a function as its argument and returns a new function or class, respectively. The new function or class is usually a modified version of the original function that adds some additional functionality. For the rest of the blog I will only be referring to function decorators, however please note that class decorators exist, can be called in the same way, and can extend class functionality without modifying the class. The decorator function is called before the original function and can modify the inputs, outputs, or behavior of the original function.

The syntax for a decorator is as follows:

@decorator_function
def my_function():
    pass

The @decorator_function syntax is known as the decorator syntax. It tells Python to pass the function my_function as an argument to the decorator_function.

Types of Python decorators

There are two types of decorators in Python: function decorators and class decorators.

1. Function decorators

Function decorators are the most common type of decorators in Python. They are used to modify the behavior of a function.

def my_decorator(func):
    def wrapper():
        print("Before function is called.")
        func()
        print("After function is called.")
    return wrapper

@my_decorator
def my_function():
    print("Hello, World!")

my_function()

In this example, my_decorator is a function that takes another function func as an argument and returns a new function wrapper. wrapper adds some additional functionality before and after func is called. The @my_decorator syntax tells Python to pass my_function as an argument to my_decorator and use the resulting function as the new definition of my_function.

2. Class decorators

Class decorators are used to modify the behavior of a class.

def my_class_decorator(cls):
    class NewClass:
        def __init__(self, *args, **kwargs):
            self.obj = cls(*args, **kwargs)

        def __getattr__(self, name):
            return getattr(self.obj, name)

        def __setattr__(self, name, value):
            setattr(self.obj, name, value)

    return NewClass

@my_class_decorator
class MyClass:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def my_method(self):
        return self.x + self.y

obj = MyClass(3, 4)
print(obj.my_method())

In this example, my_class_decorator is a function that takes a class cls as an argument and returns a new class NewClass. NewClass modifies the behavior of cls by intercepting attribute access and delegation to the original cls object. The @my_class_decorator syntax tells Python to use my_class_decorator to modify the behavior of MyClass.

Uses of Python decorators

Python decorators can be used for a wide range of purposes, including:

1. Logging
Decorators can be used to log the input and output of a function.

def log(func):
    def wrapper(*args, **kwargs):
        print(f"Calling function {func.__name__} with arguments {args} and keyword arguments {kwargs}")
        result = func(*args, **kwargs)
        print(f"Function {func.__name__} returned {result}")
        return result
    return wrapper

@log
def add(x, y):
    return x + y

add(3, 4)

In this example, the log decorator is used to log the input and output of the add method. The log decorator takes a function func as an argument and returns a new function wrapper that logs the input and output of func. The wrapper takes any number of positional and keyword arguments by using *args and **kwargs respectively. To use the log decorator simply add '@log' on the line before your function declaration.

2. Timing
Decorators can also be used to time the execution of a function.

import time

def timer(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"Function {func.__name__} took {end_time - start_time:.2f} seconds to execute.")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(2)

slow_function()

In this example, the timer decorator is used to time the execution of the slow_function. The decorator measures the time taken to execute the function and prints the elapsed time.

3. Authorization
Decorators can be used to restrict access to certain functions or classes based on authorization rules.

def authorized(func):
    def wrapper(*args, **kwargs):
        if is_authorized():
            return func(*args, **kwargs)
        else:
            raise Exception("You are not authorized to access this function.")
    return wrapper

@authorized
def secure_function():
    pass

secure_function()

In this example, the authorized decorator is used to restrict access to the secure_function. The decorator checks if the user is authorized to access the function and raises an exception if the user is not authorized.

4. Caching
Decorators can be used to cache the results of expensive function calls.

def cache(func):
    results = {}
    def wrapper(*args):
        if args in results:
            print(f"Returning cached result for {args}")
            return results[args]
        result = func(*args)
        results[args] = result
        print(f"Caching result for {args}")
        return result
    return wrapper

In this example, cache uses a wrapper to capture and store the results of func. The wrapper function first checks if the function has already been called with the same arguments by checking if the args are already in the results dictionary. If the args are in the results dictionary, the wrapper function returns the cached result and logs a message indicating that the result is being returned from the cache.

This is only a short list of what can be done with python decorators! The sky is truly the limit when you can call a function to modify another one. Some other useful implementations of decorators might include retry, validation, debugging, etc. If you're curious to dive into a deep but incredibly useful topic maybe try starting your journey here.

Using Fetch requests with api.weather.gov

MarKSmaN98 — Mon, 06 Mar 2023 05:26:49 +0000

An easy beginning project might include fetch requests to an API. I recently worked with some fellow students to create a mock weather app, and to get the weather data we decided to use Weather.gov's free API.

Weather.gov's api, located at api.weather.gov, uses a gridpoint system to give you weather data at a specified point. In order to get their proprietary gridpoint data for a location, however, you must first fetch to '/points/{latitude, longitude}' with the latitude and longitude of that point. An important detail here is that the API will return an error if your latitude or longitude have more than 4 decimal places. This first fetch will return a few different API urls with the correct gridpoint location already inputted for you, where applicable.

An example from the project:

function getPoints(location) {
    let correctedLocation = correctInput(location);
    let url = `https://api.weather.gov/points/${correctedLocation}`;
    fetch(url)
        .then(res => {
            console.log(res.status);
            return res.json();
        })

where 'correctInput' is

    let temp = input.split(',');
    let x = (parseFloat(temp[0]).toFixed(4)).toString();
    let y = (parseFloat(temp[1]).toFixed(4)).toString();
    return (x + ',' + y);

Luckily toFixed() already will take out any spaces in the original inputted string, so we don't have to worry about that.

From here we should get an output with some key features
.properties:

fireWeatherZone : "https://api.weather.gov/zones/fire/WAZ654"
forecast : "https://api.weather.gov/gridpoints/SEW/124,67/forecast"
forecastGridData : "https://api.weather.gov/gridpoints/SEW/124,67"
forecastHourly : "https://api.weather.gov/gridpoints/SEW/124,67/forecast/hourly"
forecastOffice : "https://api.weather.gov/offices/SEW"
forecastZone : "https://api.weather.gov/zones/forecast/WAZ558"

and
.relativeLocation.properties:

city : "Seattle"
state : "WA"

As an added bonus we also have a 'county' key in our object with a url value, and when fetched to you can access it's .properties to get the name of the county!

"properties": {
....
"type": "county",
"name": "King",
....
}

As you can see we have the gridpoints already put in and therefore a complete url for our next fetch. Now we simply need to fetch to the next url and, if desired, save our city and state to display on page.

Our next fetch should be done automatically after the original fetch's promise has resolved. We can use a .then to start the new fetch.

function getPoints(location) {
    let correctedLocation = correctInput(location);
    let url = `https://api.weather.gov/points/${correctedLocation}`;
    fetch(url)
        .then(res => {
            console.log(res.status);
            return res.json();
        })
        .then(dataObj => {
            fetch(dataObj.properties.forecast)
                .then(res => res.json())
                .then(data => { 
                    renderCards(data.properties.periods);
                });
            });

}

As you can see we decided to use the forecast url. This returns an array containing 14 weather data objects for day and night forecasts for the next week. Also included is an icon you can use in your website, even including a rain percentage if the chance is over 20%!
an example of the weather data we got looks something like

{
number: 1,
name: 'Tonight',
startTime: '2023-03-05T20:00:00-08:00',
endTime: '2023-03-06T06:00:00-08:00',
isDaytime: false, …}
detailedForecast: "A chance of rain showers before 3am. Mostly cloudy. Low around 34, with temperatures rising to around 36 overnight. South wind 3 to 10 mph. Chance of precipitation is 40%. New rainfall amounts less than a tenth of an inch possible."
dewpoint: {unitCode: 'wmoUnit:degC', value: 2.7777777777777777}
endTime: "2023-03-06T06:00:00-08:00"
icon: "https://api.weather.gov/icons/land/night/rain_showers,40/rain_showers,30?size=medium"
isDaytime: false
name: "Tonight"
number: 1
probabilityOfPrecipitation: {unitCode: 'wmoUnit:percent', value:40}
relativeHumidity: {unitCode: 'wmoUnit:percent', value: 94}
shortForecast: "Chance Rain Showers"
startTime: "2023-03-05T20:00:00-08:00"
temperature: 34
temperatureTrend: "rising"
temperatureUnit: "F"
windDirection: "S"
windSpeed: "3 to 10 mph"

so now we have the location from the first request (state, city, county), and we have 14 objects containing 12 hour periods for the next week. Of course, if your project needs something like the hourly forecast you're going to be using a different second fetch request, you're going to have more or less returned objects in the array, and your object's data may look a bit different.

The weather.gov API has a ton of data, and many methods to access it. This is just scratching the surface, I only included some of the urls from the first request, and even the first fetch is only one of the API's many urls you can fetch to. The full documentation is located HERE.

Thank you for reading! This is my first post and I will be making more in the future, both as a student at Flatiron and in my career!