DEV Community: Ezeana Micheal

LegalCheck: Built for freelancers and remote workers after an issue I ran into.

Ezeana Micheal — Sat, 28 Feb 2026 18:28:27 +0000

This is a submission for the DEV Weekend Challenge: Community

The Community

I built LegalCheck for freelancers, startup founders, indie builders, and non-legal tech practitioners who sign contracts without fully understanding what they’re agreeing to.

In places like Nigeria, legal support isn’t always accessible or affordable. A lot of people copy templates from the internet, sign client agreements quickly, or accept vendor contracts under pressure without truly knowing the risks(like me once before).

LegalCheck is for that community. The builders. The side-hustlers. The agency owners. The developers. The designers. The people shipping fast but still needing protection.

I’m built(and am still scaling) this for people like me.

What I Built

LegalCheck is an AI-powered contract analysis tool.

You upload a contract.
It reads it.
It highlights risky clauses.
It explains them in plain English.

No legal jargon. No intimidation.

What it does:

Analyzes contracts (freelance agreements, NDAs, service contracts, etc.)
Flags risky or one-sided clauses
Explains complex legal language in simple terms
Highlights missing protections
Breaks down what each key clause actually means for you

It doesn’t replace a lawyer.
It gives you clarity before you sign.

And that alone can save people from very expensive mistakes.

Demo

legalcheck.buildswithmike.com

(It takes some time to scan the contract about 5-10 mins, but it'll be worth the wait)

Code

Just-Mike4 / legal-check

AI-Legal Contract Analyzer

LegalCheck - AI-Powered Legal Document Analysis

LegalCheck is an intelligent legal document analysis platform that extracts, identifies, and explains specific clauses in legal documents. It translates complex legal language into plain English and provides visual representations where helpful.

Features

Instant Clause Detection: Automatically identify all key clauses in legal documents
Plain English Explanations: Complex legalese translated to simple language
Risk Assessment: Identify potentially risky clauses with severity levels
Visual Representations: Timelines and flowcharts for complex terms
Document Comparison: Compare clauses across similar documents
Q&A System: Ask questions about specific clauses

Tech Stack

Backend: Flask, SQLAlchemy
Authentication: Google OAuth (Authlib)
AI/ML: Google Gemini AI for clause analysis
Document Processing: pypdf, python-docx, pytesseract (OCR)
Frontend: Tailwind CSS, Vanilla JavaScript
Database: SQLite (development), PostgreSQL (production)

Setup Instructions

Prerequisites

Python 3.9 or higher
Google OAuth credentials
Gemini API key
Tesseract OCR (for…

View on GitHub

How I Built It

I built LegalCheck using Flask for the backend, Jinja2 for templating, Tailwind CSS for UI, and Google GenAI (Gemini 2.5) for the contract analysis.

Thanks for Reading!.

Custom QuerySets in Django: Writing Cleaner, Reusable Queries

Ezeana Micheal — Sat, 20 Sep 2025 07:38:57 +0000

Custom QuerySets in Django: Writing Cleaner, Reusable Queries

When building Django applications, especially as they scale when creating views and similar components, it's easy to find yourself using the same queries repeatedly in different logical steps. The repetition makes the code hard to maintain and can cause some confusion.

This is where querysets come in. Custom querysets allow us to write cleaner code, keeping some business logic close to your models while avoiding duplication.

But first, what is a queryset?

A queryset is simply a collection of database queries in Django, it allows us to make queries through Django’s ORM to the database. By default, objects is Django’s built-in Manager that returns a QuerySet. But what if you always need the query? Rewriting the filter everywhere does not follow the DRY (Don’t Repeat Yourself) principle.

from blog.models import Post

# Default QuerySets  
all_posts = Post.objects.all()  
published_posts = Post.objects.filter(status="published")

Now, let's get into the programming aspect. How do you define a custom queryset? We use the models imported from Django’s DB and inherit from the Queryset. Here’s an example, Suppose you have a Post model and you want to make some repeated queries.

from django.db import models

class PostQuerySet(models.QuerySet):  
    def published(self):  
        return self.filter(status="published")

    def drafts(self):  
        return self.filter(status="draft")

    def by_author(self, author):  
        return self.filter(author=author)

Let's now attach this to our model.

class Post(models.Model):  
    STATUS_CHOICES = (  
        ("draft", "Draft"),  
        ("published", "Published"),  
    )

    title = models.CharField(max_length=200)  
    content = models.TextField()  
    status = models.CharField(max_length=10, choices=STATUS_CHOICES)  
    author = models.ForeignKey("auth.User", on_delete=models.CASCADE)

    # Attach the custom QuerySet  
    objects = PostQuerySet.as_manager()

Now we can make queries like

# All published posts  
Post.objects.published()

# All drafts by a specific author  
Post.objects.drafts().by_author(user)

We can even chain them,

# Get published posts by a specific author  
Post.objects.published().by_author(user)

In a way, querysets and managers are similar and different. How would you know when to use either?

Use Custom QuerySets when:

You want reusable filters (like .published(), .active()).
You need chainable queries.

Use Custom Managers when:

You want to override get_queryset() itself.
You need queries that return something other than a QuerySet (like creating objects or aggregations).

In this article, We’ve seen how qureysets helps us in keeping the DRY principle and writing cleaner code, in the next one we’ll consider when and how the 3 powerhouses in our models.py can be used together in Custom Model Methods vs. Managers vs. QuerySets: When to Use Each.

Understanding Django Managers: The Gateway to Your Data

Ezeana Micheal — Sat, 13 Sep 2025 12:53:06 +0000

When working with Django models, you’re not just defining tables, you’re also defining how to interact with them. In this article, we’re going to use one of Django’s powerful tools when making queries, which is Django managers. Django managers define the interface for the interaction between your code and the database. The default one most are used to is the “objects” as in “students.objects.all()”.

Examples;

class Student(models.Model):  
    name = models.CharField(max_length=100)  
    age = models.PositiveIntegerField()

# Using the default manager  
all_students = Student.objects.all()  
adults = Student.objects.filter(age__gte=18)

Creating a Custom Manager

Alongside this default manager, we can also create custom managers. But why would we need to do so?

While objects is useful when performing basic queries, when dealing with larger projects, custom managers give user-defined reusable logic in code. Let's look at a practical example of this by using Django’s models. We then define a class that inherits from it and overwrites the get_queryset method in the student manager.

class ActiveStudentManager(models.Manager):  
    def get_queryset(self):  
        return super().get_queryset().filter(is_active=True)

class Student(models.Model):  
    name = models.CharField(max_length=100)  
    is_active = models.BooleanField(default=True)

    objects = models.Manager()        # Default manager  
    active = ActiveStudentManager()   # Custom manager

How then do you use this custom manager? Look below,

Student.objects.all()   # returns all students  
Student.active.all()    # returns only active students

Adding Custom Methods to Managers

You can also add methods to model managers to perform specific actions or queries. An example is sorting students by age group.

class StudentManager(models.Manager):  
    def teenagers(self):  
        return self.filter(age__gte=13, age__lte=19)

    def adults(self):  
        return self.filter(age__gte=18)

class Student(models.Model):  
    name = models.CharField(max_length=100)  
    age = models.PositiveIntegerField()

    objects = StudentManager()

Then using,

Student.objects.teenagers()  # all students aged 13–19  
Student.objects.adults()     # all students aged 18+

Multiple Managers in a Model

As seen in the first example, you can add more than 1 manager to a model to ease and streamline usage.

class ActiveStudentManager(models.Manager):  
    def get_queryset(self):  
        return super().get_queryset().filter(is_active=True)

class Student(models.Model):  
    name = models.CharField(max_length=100)  
    is_active = models.BooleanField(default=True)

    objects = models.Manager()        # Default manager  
    active = ActiveStudentManager()   # Custom manager

Managers vs. QuerySets

Managers are not qureysets. Managers are entry points to the database while QuerySets represent the actual collection of records. In the next article, we’ll see how we can use Querysets.

Multiple Managers in a Model

A model can have multiple managers, but they have one default manager, which is the objects; this can be subject to change, though.

class Student(models.Model):  
    name = models.CharField(max_length=100)  
    is_active = models.BooleanField(default=True)

    objects = models.Manager()       # default  
    active = ActiveStudentManager()  # custom

NOTE: The default manager is used in admin and migrations, so always keep objects = models.Manager() unless you know what you’re doing.

We have seen how Django managers can be used in this article, and how they help in holding some business logic and reusable queries. In the next article, we will explore querysets in Django and how they are used alongside this.

ForeignKey vs ManyToMany vs OneToOne: When to Use Each in Django

Ezeana Micheal — Sat, 06 Sep 2025 07:21:35 +0000

In common Relational Database concepts, relationships between tables are nothing new; these help to represent connections between tables and link them together. In this article, I’ll explore some common DB relationships and how they are used in Django. These include One-to-One, One-to-Many (foreign key), and Many-to-Many Relationships.

One to One

In a one-to-one relationship, one record in a model corresponds exactly to one record in another model. For example, a user’s relationship to his profile, using Django’s built-in user model, we can extend by creating a user profile:

from django.contrib.auth.models import User

class Profile(models.Model):  
    user = models.OneToOneField(User, on_delete=models.CASCADE)  
    bio = models.TextField()  
    birth_date = models.DateField(null=True, blank=True)

One to Many

In a one-to-many relationship, one record in a model corresponds to many records in another model. This means a record in a table can be linked to multiple records in another table, and this is done using a foreign key. A practical example of this is of an author having many books.

from django.db import models

class Author(models.Model):  
    name = models.CharField(max_length=100)

class Book(models.Model):  
    title = models.CharField(max_length=200)  
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

Many to Many

In a many-to-many relationship, a table is linked to another table where multiple records in one table are related to various records in the other. An example is a relationship between students and courses. A student can enroll in many courses, and a course can have many students.

class Student(models.Model):  
    name = models.CharField(max_length=100)

class Course(models.Model):  
    title = models.CharField(max_length=200)  
    students = models.ManyToManyField(Student)

In each of these types of relationships in Django, there are some common parameters and their meaning.

On_delete: In a foreign key and the one-to-one field, there’s the on_delete parameter.

The common options are:

models.CASCADE: delete related objects too.
models.SET_NULL: set field to NULL (requires null=True).
models.PROTECT: prevent deletion if related objects exist.
models.SET_DEFAULT: set field to default value.
Related_name: This defines the name used to access the reverse relationship; without this, Django automatically adds _set, i.e, (model_name)_set.

   class Course(models.Model):  
       title = models.CharField(max_length=200)  
       students = models.ManyToManyField(  
           "Student",   
           related_name="enrolled_courses"  
       )  

   #referenced here using  

   student = Student.objects.first()  
   student.enrolled_courses.all()  # instead of student.course_set.all()

blank and null: Blank and Null are boolean parameters; null=True means the database can store NULL values, while blank=True means validation allows empty values.

   class Profile(models.Model):  
       bio = models.TextField(blank=True, null=True)

Through: This is unique for the many-to-many field, it lets you define a custom intermediate model for many-to-many relationships.

   class Enrollment(models.Model):  
       student = models.ForeignKey("Student", on_delete=models.CASCADE)  
       course = models.ForeignKey("Course", on_delete=models.CASCADE)  
       enrolled_at = models.DateTimeField(auto_now_add=True)  
       grade = models.CharField(max_length=2, blank=True)  

   class Course(models.Model):  
       title = models.CharField(max_length=200)  
       students = models.ManyToManyField("Student", through="Enrollment")  

   # usage below  
   student = Student.objects.first()  
   course = Course.objects.first()  
   Enrollment.objects.create(student=student, course=course, grade="A")

From this article, we’ve been able to break down relationships in Django, their parameters, and how they’re referenced. Next, we’ll move into understanding Django managers.

Designing Better Django Models: Tips for Scalability and Clarity

Ezeana Micheal — Sat, 30 Aug 2025 10:05:17 +0000

I’ve been using Django for a while now, and I’ve seen countless guides on how to write better code, structured code, and the list goes on. I decided to write mine to cover one major aspect, which is simplicity, for this guide, and following ones, I'll make use of simple wordings and easy to understand python code that can guide you (or anyone) in writing simple and powerful django applications, and to start with, the models, which are the backbone of traditional backend applications, a poorly written model design will cause a lot of problems in your application like optimization and scaling difficulty, lets explore practices and use cases, for this article ill make use of a blog model to explain the process. To begin, we start with defining a clear data model.

Start with a Clear Data Model

Before you start writing code, it is essential to understand what you want the code to do first. It's always best to get a clear picture of what you want your database to look like through design, including the relationships. Tools like Draw.io can help with that. (This article won't cover how to design your model; it's more on writing code using Django)

After designing your database and relationships, we're ready to hop into the code. Keep 2 things in mind: naming matters, and keep models simple. For example, here's our post model.

from django.db import models  
class Post(models.Model):  
    title = models.CharField(max_length=255)  
    content = models.TextField()  
    published_at = models.DateTimeField(auto_now_add=True)

It's always best to use meaningful, singular model names like Post, not Posts, and we limit it to 3 fields: title, content, and published date. Next, let's look at choosing the right type of fields.

Choosing the Right Field Types

In Django’s models, there are several field types like CharField, TextField, and DateTimeField used in the above, but aside from this, there are others like URLField, EmailField. Let's take a look at the user profile table below:

class UserProfile(models.Model):  
    email = models.EmailField(unique=True)     
    website = models.URLField(blank=True)  
    bio = models.TextField(blank=True)  
    reputation = models.DecimalField(max_digits=6, decimal_places=2, default=0.00)

It's always best to use a specific field for some of the following reasons:

Django provides built-in validation
Helps Django’s retrieval from the database
It saves space and optimizes in the database (e.g, using Charfield for fields like name with a max length set, rather than a text field, which is better suited for description)

Now, let's consider primary keys and identifiers.

Primary Keys & Identifiers

When talking of primary keys, we consider the IDs of these tables since these are what link them up. It also brings up the question, when it is best to use Django’s default ID and a UUID.

By default, Django creates an ID this way:

id = models.BigAutoField(primary_key=True)

It auto-increments, and it's useful for small to medium-scale projects. The downside of this is that it's predictable, and it can expose user count or allow enumeration attacks in APIs.

UUIDs, on the other hand, are especially useful in distributed systems or microservices, where you can’t rely on a central database to generate sequential IDs.

import uuid  
class UserProfile(models.Model):  
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)  
    email = models.EmailField(unique=True)

UUIDs are harder to guess and good for public-facing URLs, but they require larger storage and are slower to index. Let's take a step into relationships.

Relationships Done Right

In Django, there are 3 major types of relationships: one-to-one, one-to-many (Foreign key), and many-to-many. Below are examples of such relationships.

class Post(models.Model):  
    author = models.ForeignKey("Author", on_delete=models.CASCADE, related_name="posts")  
    title = models.CharField(max_length=255)  
    content = models.TextField()

class Comment(models.Model):  
    post = models.ForeignKey("Post", on_delete=models.CASCADE, related_name="comments")  
    author = models.CharField(max_length=100)  
    body = models.TextField()

class Tag(models.Model):  
    name = models.CharField(max_length=50, unique=True)  
    posts = models.ManyToManyField("Post", related_name="tags")

models.Foreignkey, models.ManyToMany and models.OneToOne.

You’ll notice related name parameter placed in these relationship, it is the reverse relation name Django will use when looking up related objects.

For example:

post = Post.objects.first()  
post.comments.all()   # instead of post.comment_set.all()

The on_delete argument tells Django what to do with related objects when the referenced object is deleted. In our case models.CASCADE means delete the dependent rows too. This is important so it doesnt retain dependent roles in the database.

Model Methods & Business Logic

When writing business logic in the model.py file, we must ensure that this is not where all our methods should be; basically, just simple and commonly used logic can be implemented here, for example, using sample model methods.

class Post(models.Model):  
    title = models.CharField(max_length=255)  
    content = models.TextField()  
    published_at = models.DateTimeField(auto_now_add=True)

    def is_published_recently(self):  
        from django.utils import timezone  
        return self.published_at >= timezone.now() - timezone.timedelta(days=7)

The post.is_published_recently() can now be called and works as intended from here.

Performance Considerations

Finally, for some performance considerations, here are some tips: use indexes for frequent lookups; they query faster.

class BlogPost(models.Model):  
    slug = models.SlugField(unique=True, db_index=True)

Keeping Models Maintainable

Utilizing some built-in functions in Django models to help readability and reusability is mostly advised, adding str for readability, adding a class meta for verbose name, and ordering.

class Post(models.Model):  
    title = models.CharField(max_length=255)  
    body = models.TextField()  
    created_at = models.DateTimeField(auto_now_add=True)

    Class Meta:  
        ordering = ["-created_at"]  
        verbose_name = "Blog Post"

    def __str__(self):  
        return self.title

From this article, we’ve seen the need for simplicity in creating Django models and utilizing functionalities to optimize performance. In the next article, we’ll consider in-depth the use of foreign keys, one-to-one and many-to-many.

Distributed Systems: Designing Scalable Python Backends

Ezeana Micheal — Mon, 27 Jan 2025 07:08:27 +0000

Almost all systems today connected through the World Wide Web are distributed systems. Distributed systems are a group of multiple computers or servers that work together and functionalize optimally. This allows multiple users to utilize such software or services without facing slow loading times and poor performance. For example, imagine you build a website and host it on a single-user server, this would perform well until user traffic increases, demanding more resources and speed. Distributed systems aid performance and flexibility by splitting the application into individual services on separate servers that interact with each other. This would seem like a simple software or application to the user, but on the backend, it's multiple interconnected nodes talking to each other.

Python programming language is one of the slowest, but one of the most useful languages today, ever since the advent of artificial intelligence, machine learning, and Large Language Models, python has been the go-to language for these, but no one wants a chatbot or an ml service that takes a long amount of processing time to work, distributed systems is a key to optimizing such on an application. In this article, we’ll consider key features of distributed systems, why you should use distributed systems, and how we can scale a distributed system with Python.

Key Features Of Distributed Systems

The following are key features of distributed systems that make them work optimally:

Nodes: Individual computers or processes that work together as part of the system; each node performs certain tasks and connects with others to ensure the system functions properly.
Communication Protocols: Nodes can communicate and share information thanks to protocols like HTTP, gRPC, or TCP/IP, which guarantee dependable communication between components even when they are on different networks.
Shared Resources: Distributed systems frequently rely on resources like databases, file systems, or message queues; proper management enables consistent and efficient access across all nodes.
Fault Tolerance: Even if a node fails, distributed systems continue running, eliminating a single point of failure. Redundancy and replication techniques ensure reliability and high availability.
Scalability: The ability to handle increased load by adding more nodes (horizontal scaling) or enhancing the capacity of existing nodes (vertical scaling). Scalability ensures the system remains responsive under high demand.

Why Scalability Matters?

Scalability as mentioned earlier is a system’s ability to handle increased load by adding resources. This ensures the system is always at optimal performance during traffic spikes. There are 2 major types of scaling:

Horizontal scaling: This involves using more machines and servers for the application to work smoothly.
Vertical scaling: This involves increasing the system's RAM, storage, and capacity.

How to Design Scalable Python Backends

Knowledge of the right tools is required to design scalable Python backends to allow the system to grow and remain efficient. Below are some key tools and strategies for building scalable Python backends.

APIs: Use lightweight frameworks like Flask or Fast API to build scalable backend APIs. They are relatively easy to use when creating rest APIs. Fast API is best for performance and its support for asynchronous programming.
Asynchronous processing: To support the main application, it is wise to offload some background tasks(like emails or data processing) using Cerely with Redis as the message broker.
Load Balancing: To balance the load or traffic on the application, distribute incoming requests evenly across backend servers like Nginx or HAProxy in the distributed system.

Example: A task queue with Celery and Redis.

# tasks.py
from celery import Celery

app = Celery('tasks', broker='redis://localhost:6379/0')

@app.task
def process_order(order_id):
    print(f"Processing order {order_id}")

# Adding a task to the queue
process_order.delay(123)

Data Management in Distributed Systems

Maintaining the CAP theorem or properties when managing data in distributed systems.

Consistency: All nodes in the distributed system see the same data. If data is updated in a node, all nodes should reflect the updated value immediately.
Availability: The system responds even during a node failure. The system should always be operational.
Partition Tolerance: It should work despite network failures between nodes.

Useful databases are:

SQL Databases like PostgreSQL for transactional consistency.
NoSQL Databases like MongoDB for scalable, flexible schemas.

An example is a case setting up a distributed MongoDB cluster to store and retrieve user data across multiple nodes, ensuring high availability and fault tolerance.

Tools for Deployment and Scaling

Deployment and scaling are when tools like docker and Kubernetes come in.

Docker: Docker is used to containerize Python backend applications for consistent environments.
Kubernetes: This helps automate the deployment, scaling, and management of the containerized application.

Example: Below is a simple/basic example of deploying a Python backend application using docker and Kubernetes.

Dockerfile:

 FROM python:3.10
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]

Kubernetes Deployment:

 apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-backend
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flask-backend
  template:
    metadata:
      labels:
        app: flask-backend
    spec:
      containers:
      - name: flask-backend
        image: flask-app:latest
        ports:
        - containerPort: 5000

Monitoring and Maintenance

In distributed systems, it is important to monitor and maintain nodes around the system as they interact and function as one. This will help to identify and fix possible faults.

Examples of tools that do this are Prometheus and Grafana.

Prometheus: Prometheus helps to collect metrics on API performance, database latency, etc.
Grafana: Grafana visualizes metrics with customizable dashboards.

Case Study: Scalable Ecommerce Backend

Before I conclude this article, let's look at how to apply distributed systems during the development of an e-commerce system.

FastAPI can be used for the backend system and handle API’s for order processing.
Celery with Redis is used to handle asynchronous background processing of tasks like payments or inventory updates.
The application is deployed on docker and Kubernetes to ensure the scaling of the system.
The application is monitored using tools like Prometheus.

Conclusion

By using Python tools like Flask, FastAPI, Celery, Docker, and Kubernetes, developers can build robust and scalable systems. In this article, we’ve covered the general terms related to distributed systems and how they can help, as well as some basic examples with Python. You can do advanced research on these tools and how they can work together and help. Start experimenting with these tools and create backends that can handle the challenges of real-world traffic and growth. Happy hacking.

Advanced Database Query Optimization Techniques: A Practical Approach with Django

Ezeana Micheal — Mon, 20 Jan 2025 08:00:00 +0000

Quick information retrieval is necessary in today’s fast-paced world as it affects productivity and efficiency. This is also true about applications and databases. Many applications developed work hand in hand with databases through the backend interface. Understanding query optimization is essential for maintaining scalability, lowering latency, and assuring lower expenses. This article will reveal advanced techniques for optimizing database queries, specifically on Django, and their impact on query performance.

What is query optimization?

Query optimization improves database speed and effectiveness by selecting the most efficient way to perform a given query. Let us understand this in a problem-solving context. Naturally, there are many different ways of solving a problem, but the most efficient way will save more time and energy, improving performance. Query optimization is just like that, improving the quality of our queries will increase database performance.

Why optimize query?

Query optimization is important because it:

Improves application speed.
Reduces server load.
Enhances user experience.
Reduces operating expenses by using fewer resources.

Key Query Optimization Techniques in Django

The following are, but not limited to, optimization techniques in Django:

1. Use Database Indexes:

Making queries on unindexed fields can cause the database to scan the entire table looking for the query, leading to slower performance. On the other hand indexed queries are faster, especially for larger datasets.

With Indexed field

# Querying without an index
class Book(models.Model):
    title = models.CharField(max_length=200)
    #other fields
books = Book.objects.filter(title="Django Optimization")

Without Indexed field

class Book(models.Model):
    title = models.CharField(max_length=200, db_index=True) 
     #other fields
books = Book.objects.filter(title="Django Optimization")

2. Use Select Related and Prefetch Related:

Select Related and Prefetch Related are database optimization techniques for querying related objects. They help avoid the N+1 Query problem.

Selected Related: This method retrieves related data via a single SQL JOIN query. It is great for single-valued connections such as ForeignKey or OneToOneField. It returns the actual instance of the linked data without using several requests.
Prefetched Related:
Runs separate queries for related objects (for multi-valued relationships like ManyToManyField or reverse ForeignKey), but Django caches and connects the related data to prevent repeating the searches.

Without select related

# N+1 queries: Fetches each related Author object for every Book
books = Book.objects.all()
for book in books:
    print(book.author.name)

With select related

# Optimized: Single query to fetch books and their related authors
books = Book.objects.select_related('author')
for book in books:
    print(book.author.name)

3. Avoid N+1 Query Problem:

The N+1 problem occurs when queries that can be performed once are done repeatedly. For example, when fetching a list of items in an object, another set of queries is executed to get a list of related entities for each item.

N+1 problem example

# Inefficient: Queries executed inside a loop
books = Book.objects.all()
for book in books:
    reviews = book.review_set.all()  # Separate query for each book's reviews
    print(reviews)

Solution

# Optimized: Prefetch all reviews in a single query
books = Book.objects.prefetch_related('review_set')
for book in books:
    print(book.review_set.all())

4. Filter Early, Fetch Less Data:

This principle is to guide in filtering or querying only needed data rather than all. Performance is improved when we query only the data we need in some instances rather than querying all before filtering.

Without optimization

books = Book.objects.all()  # Loads all records into memory
filtered_books = [b for b in books if b.published_year >= 2020]

With optimization

filtered_books = Book.objects.filter(published_year__gte=2020)  # Query fetches only relevant data

5. Use Queryset Defer and Only:

Using Defer and Only helps us load only necessary fields from the database to our application.

Defer: Does not fetch the input fields in the query.
Only: Retrieves only the set of fields while deferring the rest.

Without optimization

# Fetches all fields, including a large 'content' field
articles = Article.objects.all()
for article in articles:
    print(article.title)  # Only the 'title' is used, but all fields are fetched

With optimization

# Excludes the 'content' field from the query
articles = Article.objects.defer('content')
for article in articles:
    print(article.title)  # Fetches only the 'title' field

6. Paginate Large Datasets:

Fetching and processing large data in a database increases memory usage thereby limiting performance. Use pagination to break down into smaller chunks, this reduces memory usage and speeds up response time.

Without Pagination

books = Book.objects.all()  # Loads all records at once

With pagination

from django.core.paginator import Paginator
paginator = Paginator(Book.objects.all(), 10)  # 10 items per page
page1 = paginator.get_page(1)  # Fetches only the first 10 records

7. Cache Frequent Queries:

Cache queries that are used frequently. This prevents recurring queries and reduces database load.

Without cache

books = Book.objects.all()  # Query hits the database each time

With cache

from django.core.cache import cache
books = cache.get_or_set('all_books', Book.objects.all(), timeout=3600)  # Cached for 1 hour

8. Optimize Aggregations:

Django provides powerful aggregation functions for querying aggregated data directly from the database. Database computations are faster than in Python, this improves speed.

Without Aggregations

products = Product.objects.all()
total_price = sum(product.price for product in products)  # Aggregation in Python
print(f"Total Price: {total_price}")

With Aggregations

from django.db.models import Sum

total_price = Product.objects.aggregate(total=Sum('price'))
print(f"Total Price: {total_price['total']}")

9. Monitor and Profile Queries:

To optimize database queries it's important to know how to monitor queries. This can be done using Django’s connection method. This helps to identify what slows down the database and resolve it.

Unmonitored query

# Blind execution without monitoring
books = Book.objects.all()

Monitored query

from django.db import connection
books = Book.objects.all()
print(connection.queries)  # Shows executed queries

10. Use Q objects for complex queries:

Rather than executing multiple filters during a certain query, it is better to utilize the Q object for better readability and efficiency.

Without Q

books = Book.objects.filter(title__icontains='Django').filter(author__name__icontains='Smith')

With Q

from django.db.models import Q
books = Book.objects.filter(Q(title__icontains='Django') | Q(author__name__icontains='Smith'))

Conclusion

Optimizing database queries is vital for keeping your Django application operating smoothly as it scales. Key optimization techniques, including indexing, caching, avoiding the N+1 problem, and monitoring the database regularly with tools like Django’s connection method or utilizing the Django-debug-toolbar, can ensure a faster and more efficient web application.

Implementing Linear Regression Algorithm from scratch.

Ezeana Micheal — Sat, 08 Jul 2023 13:26:41 +0000

The basis of many machine learning models used to make predictions today is statistics, from statistical analysis brings forth their implementation in the form of algorithms. In this article, we will look into one of sklearn’s supervised machine learning linear algorithms for predicting regression values, Linear Regression.

Understanding Linear Regression

Linear regression is based on the concept:

Y=C+M*X

Where C is the intercept of the plotted graph, it determines where the line touches the y-axis, Y is the target or predicted value and X represents the features, sometimes in the dataset X may vary in the sense that there may be more than one feature. Which leads to an equation.

Y=C+M1*X1+M2*X2….+Mn*Xn

The C and M are regarded as the bias in Machine Learning because it is added to the offset of all predictions that will be made.

To get the values of M.

Where Xi are the different input values of X used to predict Y. and Mean are the average values of the number of X inputs.

And to derive the value of the intercept c.

Creating the Model from Scratch

So we now have a brief review of how the model works, now to implement it.
We start by importing the numpy library as we would make use of it for some numerical calculations. We create the class LinearRegression and set some initialized parameters.


import numpy as np
class LinearRegression:
    def __init__(self, fit_intercept=True):
        self.fit_intercept = fit_intercept
        self.coef_ = None
        self.intercept_ = None

The class takes in the fit_intercept parameter which determines whether an intercept term would be included in the model. Its default value would be true. The coefficients and the intercept would be set as none for default values which would later be changed when the model is trained with data.

def _add_intercept(self, X):
        return np.concatenate((np.ones((X.shape[0], 1)), X), axis=1)

Next, we create a function ‘_add_intercept’ so that the model takes in the values of X and It creates an array of ones with the shape (X.shape[0], 1), representing the intercept term, and concatenates it with X along the y-axis.

 def fit(self, X, y):
        if self.fit_intercept:
            X = self._add_intercept(X)
        self.coef_ = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)
        if self.fit_intercept:
            self.intercept_ = self.coef_[0]
            self.coef_ = self.coef_[1:]

Next, we create a function in the class that fits the data into the model and performs some algorithmic calculations. It uses a closed-form solution that involves the use of matrix operations. It first takes the parameters X and Y and if fit_intercept is set to True it passes X through the _add_intercept function to add an intercept term. Next is the algorithm to compute the values of the coefficients by getting the inverse of the dot products of X and the dot product of the transpose of X and get the dot product of those with y. Then if the fit_intercept function is True, it takes the first value in self.coef as the intercept, and the rest are the coefficients.

def predict(self, X):
        if self.fit_intercept:
            X = self._add_intercept(X)
        return X.dot(np.concatenate(([self.intercept_], self.coef_)))

Next, we create predict function which takes in the value of X and if ‘fit_intercept’ is true it passes the values of X into the _add_intercept function and returns the dot product of X and the concatenated array of the intercept and coefficients.

Here’s the full code below:

import numpy as np
 class LinearRegression:
    def __init__(self, fit_intercept=True):
        self.fit_intercept = fit_intercept
        self.coef_ = None
        self.intercept_ = None
    def _add_intercept(self, X):
        return np.concatenate((np.ones((X.shape[0], 1)), X), axis=1)
    def fit(self, X, y):
        if self.fit_intercept:
            X = self._add_intercept(X)
        self.coef_ = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)
        if self.fit_intercept:
            self.intercept_ = self.coef_[0]
            self.coef_ = self.coef_[1:]
    def predict(self, X):
        if self.fit_intercept:
            X = self._add_intercept(X)
        return X.dot(np.concatenate(([self.intercept_], self.coef_)))

Thanks for reading. Like, and comment your thoughts below.

How to build a simple Machine Learning Clustering Model.

Ezeana Micheal — Sat, 01 Jul 2023 05:42:31 +0000

Clustering is an unsupervised machine learning concept which groups or rows of data based on their features or properties in a dataset.
In Unsupervised machine learning, there is no target feature, which means unlike supervised machine learning models that predicts a value ,clustering don’t, instead you only have the dataset to classify into separate clusters.Clustering techniques are applied to target segmentation, social network analysis, image segmentation and so on. In this article, we’ll explore one of sklearn's unsupervised machine learning algorithms which is KMeans.The link for the dataset is here

Training the model.

Import useful libraries.

import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns from sklearn.cluster import KMeans

We start by importing useful libraries:
Numpy: A python numerical library for statistical calculations.
Pandas:A python library for reading and manipulation of the dataset in csv format.
Seaborn:A python visualization library.
Sklearn:A python library for manipulation of data and supervised and unsupervised machine learning concepts. In this article, we’ll consider KMeans classifier, to segment a group of data into different classes.
Reading the data.
After importing the necessary modules, we read the data.

data=pd.read_csv('/Users/user/Downloads/covid_worldwide.csv')

We can check the contents of our data by using the pandas head command.

data.head()

Output:

Expository Data Analysis

We find out more about the data here.To get data and understand the columns we use pandas info method, to get the datatypes and number of data in each column.

data.info()

Output:

From the output, we notice some significant issues with the dataset.

There are some missing entries or values in some columns.
The columns are of datatype ‘object’, so we need to convert them to integers.

Data Cleaning And Preparation

In order to make our data more readable and to use it to train a clustering model, we need to solve those issues stated above.
To handle the missing values, we can drop the missing roles(This method is not advisable in some cases as it results in data loss, in future articles, we’ll see other ways we can handle missing values).
To drop the missing columns, Execute:

data.dropna(inplace=True)
Next, we want to remove the commas(,) present in the numbers in order to convert them to integers. We do so with the simple commas which takes all commas in the dataset.

data.replace(",","",regex=True , inplace=True)

We then collect the useful numerical columns into a variable named X.

X=data.drop(['Serial Number','Country'], axis=1)

So we convert the remaining column’s datatype to floats using the code below.

def tofloat(X,col): X[col]=X[col].astype(float) return X[col] for col in X.columns: X[col]=tofloat(X,col)
Then we use the pandas info() method to check our data again.
Output:

We can also use the pandas describe function to see the statistical inferences of the data.

X.describe()

Output:

Model Building

To train the model, we first load the model.

kmeans=KMeans(n_clusters=2,n_init=10)

The n_clusters parameter is to specify the number of clusters or number of groups you want to create from the dataset. The n_init parameter states the number of times Kmeans would run from different starting points in the dataset.

The next few lines of code, we use the fit predict function to both fit the dataset into the model and collect the prediction as a new column in X as a categorical datatype.We use the total cases and total deaths of covid cases to classify them.

X["Cluster"] = kmeans.fit_predict(X[['Total Cases','Total Deaths']]) X["Cluster"] = X["Cluster"].astype("category")

We can visualize the cluster in a simple way with the code.

sns.relplot( x="Total Cases", y="Total Deaths", hue="Cluster", data=X, height=6, );

Output:

Hope you enjoyed reading this article as i did writing it, like and feel free to comment your thoughts below.

How to build a simple Machine Learning Classification Model.

Ezeana Micheal — Sat, 24 Jun 2023 06:08:52 +0000

When you hear the word classify, what comes to mind is a group of things based on their differences. The same is so when it comes to machine learning, based on features and data gathered, the machine learning model can learn to distinguish between different classes through different patterns found by the machine learning model. In this article, you’ll learn how to build a supervised classification machine-learning model.
The dataset that would be used is a loaded dataset obtained from kaggle. The model we train in this article would be able to predict the possibility of a person repaying his/her loan based on their circumstances, Y is for loan approval and N is for loan declined.

Training the model.

Import useful libraries.

import numpy as np import pandas as pd import matplotlib as plt from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import ConfusionMatrixDisplay, confusion_matrix, classification_report import seaborn as sns
We import Numpy, a Python based library used for numerical calculations.
Pandas: To read the dataset and store it in data frame format in order to perform analysis, cleaning, and some calculations in the data frame.
Sklearn: Also known as Scikit-learn, which is one of the most popular libraries which contains several classes and some functions for analysis and machine learning models, One of which we’ll use in the article is the Random Forest Classifier. The Random forest classifier model is a very powerful classification model which can achieve high accuracy with little or no hyperparameter tuning.

Reading the data.

Using the pandas' library which has been imported as ‘pd’, we import the dataset using the datasets file path.
df=pd.read_csv('/Users/user/Documents/loan_sanction_train.csv')
To check if you’ve imported the file correctly type:
df.head()
Output:

Expository Data Analysis

Let's find out more about our data before we start training the model.
df.info()
Output:

From above we deduce the presence of 614 rows and 13 columns. We can also see some values are missing in them, to see the number of values missing in each column use,
df.isnull().sum()
Output:

We’ll discuss how to handle missing values in later articles, for now, let's drop all rows with missing values.
df=df.dropna()
We can get the summary of statistical values which include, mean, minimum value, maximum value, standard deviation, 25th percentile, 50th percentile(mid), 75th percentile, and number of the numerical columns in the distribution.
df.describe()
Output:

Data Cleaning And Preparation

Machine learning models only accept numerical and boolean (true or false) values, therefore in our features or columns, we’ll need to convert all strings or objects to integers, floats, or boolean values.
We can check the various categories in the feature with the code.
Example:
df['Property_Area'].value_counts()
Output:

We can then convert all the features that aren’t numerical or boolean by creating a function, calling the df.astype(“category”).cat.codes, and passing the dataframe and columns through it.
def category_val(df,col): df[col]=df[col].astype('category') df[col]=df[col].cat.codes return df[col] df['Gender']=category_val(df,'Gender') df['Married']=category_val(df,'Married') df['Education']=category_val(df,'Education') df['Property_Area']=category_val(df,'Property_Area') df['Self_Employed']=category_val(df,'Self_Employed') df['Dependents']=category_val(df,'Dependents')
You can confirm by either the value_counts function to see your new categorical values.
df['Property_Area'].value_counts()
Output:

Model Building

Now that we’ve cleaned our data a bit, let’s split the features and target variable, lets's say x are our features and y is our target.

x= df.drop(['Loan_ID','Loan_Status'], axis =1) y=df['Loan_Status']
We use the dataframe.drop function to remove the feature that won’t be used and the target feature, then we collect the target feature into y.

Next, we would split our data into a training set and a test set. To check how our model would perform on some real-world data, we use a test split, we divide our dataset into a training set and test set. We train our model with the training set and evaluate how it performs on the test set.
To split our model, we use the train_test_split function setting our test size to 30%(generally it's advisable to use 20%-30% of our dataset for the test set, so as not to lose too much data when training the model).

x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.3,random_state=True)

Next, we load our classification model to be used, which in this case, is the random forest classifier.
The random forest classifier is built off multiple decision trees, which is a model that branches to a particular result based on certain parameters.

The random forest model is then made of multiple decision trees, which makes thousands of branches on features and evaluates outcomes.
rf=RandomForestClassifier()
After loading our model, we fit the training set into our model.
rf.fit(x_train,y_train)

Model Evaluation

There are several ways to evaluate our machine learning in this article, we’ll consider one.
Accuracy
Accuracy determines how well your model performed when it predicts the test target data compared with the actual target results.
We can check our model accuracy after fitting the model with the simple code.
rf.score(x_test,y_test)
This gives the score of 0.743 which when converted is 74.3%.

That's it, you’ve successfully trained your first machine-learning model. In future articles, we’ll consider how to train unsupervised machine learning models, how to deploy them, and other ways of evaluating machine learning models.
Give a like and feel free to comment your thoughts down below.

How to build a simple Machine Learning Regression Model.

Ezeana Micheal — Sat, 17 Jun 2023 06:02:14 +0000

In this article, I'll provide a step to step method of building a regression model using sklearn's linear regression.
A regression model is a supervised machine learning model which predicts numerical values based on numeric or boolean inputs and data provided., for example house pricing prediction.
In this article, we'll be using the dataset obtained from kaggle.

Training the model.

Import useful libraries.

First, you import the useful functions we'll make use of and read the CSV file using pandas.
import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error

Numpy is a numerical library in python which contains faster access to some statistical calculations.
Pandas is a python library which is used for accessing and manipulating the dataset.

Matplotlib and seaborn are visualization libraries which we would use in later articles.

Sklearn is a python library that contains several machine learning models and tools for model evaluation. Today we’ll be using one of the libraries which is the linear regression model.

Reading the data.

First you can read the data by using pandas read csv method.
data=pd.read_csv(‘/Users/user/loan_sanction_train.csv’)

You can find out more about the data by checking the first 5 rows using the data.head() method.
data.head()
Output:

Expository Data Analysis

To get more information about the data we can check more by using the pandas info method.

data.info()

Output:

From there, we read about the number of columns and rows present in the dataset as well as their respective datatypes.
To get statistical inference from the dataset, we use the pandas describe method.

data.describe()

Output:

From the output seen, we can see the overall statistics in the dataset, which contains the mean, median,25th percentile, 75th percentile, standard deviation, minimum and maximum value.
I'll provide more explanation of these as well as useful visualizations in a future article on EDA(Expository data analysis).

Data Cleaning and Preparation

To train your model you need to convert some of the categorical values into numerical variables. You can then transform each column using pandas value counts to see each unique values in a column.
Example:
data[‘sex’].value_counts()
Output:

You can create a function that takes the data and column and then gets an assigned number to the particular category to convert all categorical variables into numeric datatypes.

def category_val(df,col): df[col]=df[col].astype('category') df[col]=df[col].cat.codes return df[col] data['sex']=category_val(data,'sex') data['smoker']=category_val(data,'smoker') data['region']=category_val(data,'region')

Model Building.

Next, You separate your data into x and y where y is the target variable.
X takes the rest of the features since we're using all the features to predict the target variable y which will only contain the target column.
x=data.drop('charges', axis=1) y=data['charges']

Next, we split our values of x and y into the training set and test set, the training set is the set used to train the model while the test set is used to evaluate the model based on what it's learnt in the training set. It is advisable to use between 20% to 30% of our data for the test set and the remaining 70% to 80% for our training set.in this article, we’ll use 30% for our test set.

x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.3, random_state=42)

We then load the linear regression model and fit our training set data into it

model=LinearRegression() model.fit(x_train,y_train)

We’ve successfully trained our machine learning model. Now how do we evaluate our data with the test set, there are several tools we can use to do this, more would be provided in future articles, we’ll use the mean squared error in this article.

Model Evaluation.

To evaluate the model, we’ll need to compare what the model predicts as the y_test values with the actual y_test results.
We create a variable named y_pred and use the model to predict the x_test values.

y_pred=model.predict(x_test)

Finally, we evaluate the values predicted using mean squared error.
mean_squared_error(y_test,y_pred)

Which gives a result of
33805466.898688614

there you have it, you just trained your first regression model, feels great?
Here's what happens underneath.

Linear regression uses an algorithm similar to the equation of a line
Y=mx+c
C is the intercept while x is a feature, m is the gradient and y is the target.

We can find how these relate using model.coef_ and model.intercept_

The intercept is given as -12364.39
And the coefficients were given in form of an array
Its easier to then represent the equation generated as this

charges=-12364.39+(age*261.63)+(sex*109.65)+(bmi*344.54)+(children*424.37)+(smoker*23620.80)+(region*-326.46)

Hope you enjoyed this article, like, share, and comment your thoughts thanks.

Machine learning overview

Ezeana Micheal — Sat, 10 Jun 2023 05:06:05 +0000

After the gathering, cleaning, and preprocessing of data to understand and recover insight from it, one way of implementing these is through the development of machine learning models.

Machine learning can be said to be a subfield in the data science field as well as in AI(artificial intelligence), as the name implies a computer can learn and make predictions based on what its data learned. This is important as it has led to the creation of important models like weather forecasts, fraud detection, and so on.
There are three types of machine learning

Supervised machine learning
Unsupervised machine learning
Reinforcement machine learning

Supervised machine learning: This is the type of machine learning whereby it has a target value. This consists of data used to make the prediction (independent variables) and a target value to be predicted (dependent variables).
There are two types of supervised machine learning:

Regression
Classification

Regression: Regression is the aspect of supervised machine learning that deals with the prediction of quantitative or numerical data. In this type of supervised machine learning, all values are numeric and it plots them(the features of the data to be used) on a graph and then draws a line over them, this line can be called the line of best fit. This prediction algorithm can be in the form y=mx+c (equation of a line) where x is the feature(provided only one feature was used for the prediction), and m and c are constants obtained from the findings of the computer. For example, y=26x+7 takes the feature value in new data x=15 and then uses the formula to apply it y=26*15+7, y=397
When the number of features is increased it can be written as y=c+mx, mx2, mx3...
This can be used when predicting house price values or other numerical operations. A model that does this is Sk-learn's Linear Regression model.

Classification: Unlike regression, the classification uses qualitative values as target values. The data consists of values that are numerical or boolean while the target values are the classified value names to be predicted, each feature is evaluated and separated by some complex algorithm by the computer so when it evaluates each of the features of the test data it will fit it into one of the categories listed. This is done in various real-life scenarios like a fraud detection model or an image classification model. There are several models for classification like a Decision tree classifier, logistic regression(it says regression but it is used to build classification models), K neighbors classifier, Random forest classifier, and so on.

Unsupervised machine learning: This is a type of machine learning algorithm where there is no target value, but the computer, finding matches between them, classifies or groups the data in clusters based on their values. As mentioned above there is no target value but the aim of the computer is to find a connection between each value, this is used in algorithms like movie recommendation systems, based on the movies and get as you watch you might like. This modelling technique is called clustering. One of Sk-learn's clustering algorithms is KMeans clustering.

Reinforcement machine learning: this is an aspect of machine learning where the computer learns to interact with the environment, examples of these are games like chess, in which the computer doesn't know the game but learns how to play it with time according to the rules of the game, another example is self-driving cars which have motion sensors in them which help the computer learn how to move under certain scenarios.

The four main steps or lines in a model building in machine learning after the visualisation and preprocessing are:

Separate the processed data into the train set and test set.
Load or import the model to be used.
Use the training set to train the model.
Evaluate the model using the appropriate method with the test set.

From simple modelling like the prediction of house pricing to complex ones like image recognition, machine learning has automated a lot of tasks and improved Information Technology in a lot of ways.
Thanks for reading this article, like, share, and comment on what you think thanks.