DEV Community: Nik Tomazic

Find and Fix N+1 Queries in Django Using AppSignal

Nik Tomazic — Wed, 18 Dec 2024 13:48:22 +0000

In this article, you'll learn about N+1 queries, how to detect them with AppSignal, and how to fix them to speed up your Django apps significantly.

We'll start with the theoretical aspects and then move on to practical examples. The practical examples will mirror scenarios you might encounter in a production environment.

Let's get started!

What Are N+1 Queries?

The N+1 query problem is a prevalent performance issue in web applications that interact with a database. These queries can cause significant bottlenecks, which intensify as your database grows.

The problem occurs when you retrieve a collection of objects and then access the related objects for each item in the collection. For instance, fetching a list of books requires a single query (1 query), but accessing the author for each book triggers an additional query for every item (N queries).

N+1 problems can also occur when creating or updating data in a database. For example, iterating through a loop to create or update objects individually, rather than using methods like bulk_create() or bulk_update(), can result in excessive queries.

N+1 queries are highly inefficient because executing numerous small queries is significantly slower and more resource-intensive than consolidating operations into fewer, larger queries.

Django's default QuerySet behavior can inadvertently lead to N+1 issues, especially if you're unaware of how QuerySets work. Querysets in Django are lazy, meaning no database queries are executed until the QuerySet is evaluated.

Prerequisites

Ensure you have:

Python 3.9+ and Git installed on your local machine
An AppSignal-supported operating system
An AppSignal account

Note: The source code for this project can be found in the appsignal-django-n-plus-one GitHub repository.

Project Setup

We'll work with a book management web app. The web app is built to demonstrate the N+1 query problem and how to resolve it.

Start by cloning the base branch of the GitHub repo:

$ git clone git@github.com:duplxey/appsignal-django-n-plus-one.git \
    --single-branch --branch base && cd appsignal-django-n-plus-one

Next, create and activate a virtual environment:

$ python3 -m venv venv && source venv/bin/activate

Install the requirements:

(venv)$ pip install -r requirements.txt

Migrate and populate the database:

(venv)$ python manage.py migrate
(venv)$ python manage.py populate_db

Lastly, start the development server:

(venv)$ python manage.py runserver

Open your favorite web browser and navigate to http://localhost:8000/books. The web app should return a JSON list of 500 books from the database.

The Django admin site is accessible at http://localhost:8000/admin. The admin credentials are:

user: username
pass: password

Install AppSignal for Django

To install AppSignal on your Django project, follow the official docs:

Ensure everything works by restarting the development server:

(venv)$ python manage.py runserver

Your app should automatically send a demo error to AppSignal. From this point forward, all your errors will be sent to AppSignal. Additionally, AppSignal will monitor your app's performance and detect any issues.

Web App Logic

The prerequisite to fixing N+1 queries is understanding your app's database schema. Pay close attention to your models' relationships: they can help you pinpoint potential N+1 problems.

Models

The web app has two models — Author and Book — which share a one-to-many (1:M) relationship. This means each book is associated with a single author, while an author can be linked to multiple books.

Both models have a to_dict() method for serializing model instances to JSON. On top of that, the Book model uses deep serialization (serializing the book as well as the book's author).

The models are defined in books/models.py:

# books/models.py
class Author(models.Model):
    first_name = models.CharField(max_length=64)
    last_name = models.CharField(max_length=64)
    birth_date = models.DateField()

    def full_name(self):
        return f"{self.first_name} {self.last_name}"

    def to_dict(self):
        return {
            "id": self.id,
            "first_name": self.first_name,
            "last_name": self.last_name,
            "birth_date": self.birth_date,
        }

    def __str__(self):
        return f"{self.first_name} {self.last_name}"


class Book(models.Model):
    title = models.CharField(max_length=128)
    author = models.ForeignKey(
        to=Author,
        related_name="books",
        on_delete=models.CASCADE,
    )
    summary = models.TextField(max_length=512, blank=True, null=True)
    isbn = models.CharField(max_length=13, unique=True, help_text="ISBN-13")
    published_at = models.DateField()

    def to_dict(self):
        return {
            "id": self.id,
            "title": self.title,
            "author": self.author.to_dict(),
            "summary": self.summary,
            "isbn": self.isbn,
            "published_at": self.published_at,
        }

    def __str__(self):
        return f"{self.author}: {self.title}"

They are then registered for the Django admin site in books/admin.py, like so:

# books/admin.py
class BookInline(admin.TabularInline):
    model = Book
    extra = 0


class AuthorAdmin(admin.ModelAdmin):
    list_display = ["full_name", "birth_date"]
    inlines = [BookInline]


class BookAdmin(admin.ModelAdmin):
    list_display = ["title", "author", "published_at"]


admin.site.register(Author, AuthorAdmin)
admin.site.register(Book, BookAdmin)

Notice that AuthorAdmin uses BookInline to display the author's books within the author's admin page.

Views

The web app provides the following endpoints:

/books/ returns the list of books
/books/<book_id>/ returns a specific book
/books/by-authors/ returns a list of books grouped by authors
/books/authors/ returns the list of authors
/books/authors/<author_id>/ returns a specific author

The links above are clickable if you have the development web server running.

And they're defined in books/views.py like so:

# books/views.py
def book_list_view(request):
    books = Book.objects.all()
    return JsonResponse(
        {
            "count": books.count(),
            "results": [book.to_dict() for book in books],
        }
    )


def book_details_view(request, book_id):
    try:
        book = Book.objects.get(id=book_id)
        return JsonResponse(book.to_dict())
    except Book.DoesNotExist:
        return JsonResponse({"error": "Book not found"}, status=404)


def book_by_author_list_view(request):
    try:
        authors = Author.objects.all()
        return JsonResponse(
            {
                "count": authors.count(),
                "results": [
                    {
                        "author": author.to_dict(),
                        "books": [book.to_dict() for book in author.books.all()],
                    }
                    for author in authors
                ],
            }
        )
    except Author.DoesNotExist:
        return JsonResponse({"error": "Author not found"}, status=404)


def author_list_view(request):
    authors = Author.objects.all()
    return JsonResponse(
        {
            "count": authors.count(),
            "results": [author.to_dict() for author in authors],
        }
    )


def author_details_view(request, author_id):
    try:
        author = Author.objects.get(id=author_id)
        return JsonResponse(author.to_dict())
    except Author.DoesNotExist:
        return JsonResponse({"error": "Author not found"}, status=404)

Great, you now know how the web app works!

In the next section, we'll benchmark our app to detect N+1 queries with AppSignal and then modify the code to eliminate them.

Detect N+1 Queries in Your Django App with AppSignal

Detecting performance issues with AppSignal is easy. All you have to do is use/test the app as you normally would (for example, perform end-user testing by visiting all the endpoints and validating the responses).

When an endpoint is hit, AppSignal will create a performance report for it and group all related visits together. Each visit will be recorded as a sample in the endpoint's report.

Detect N+1 Queries in Views

Firstly, visit all your app's endpoints to generate the performance reports:

Next, let's use the AppSignal dashboard to analyze slow endpoints.

Example 1: One-To-One Relationship (`select_related()`)

Navigate to your AppSignal app and select Performance > Issue list on the sidebar. Then click Mean to sort the issues by descending mean response time.

Click on the slowest endpoint (books/) to view its details.

Looking at the latest sample, we can see that this endpoint returns a response in 1090 milliseconds. The group breakdown shows that SQLite takes 651 milliseconds while Django takes 439.

This indicates a problem because an endpoint as simple as this shouldn't take as long.

To get more details on what happened, select Samples in the sidebar and then the latest sample.

Scroll down to the Event Timeline to see what SQL queries got executed.

Hovering over the query.sql text displays the actual SQL query.

More than 1000 queries were executed:

SELECT * FROM "books_book";
SELECT * FROM "books_author" WHERE "books_author"."id" = 3; -- 3= 1st book's author id
SELECT * FROM "books_author" WHERE "books_author"."id" = 6; -- 6= 2nd book's author id
...
SELECT * FROM "books_author" WHERE "books_author"."id" = n; -- n= n-th book's author id

These are a clear sign of N+1 queries. The first query fetched a book (1), and each subsequent query fetched the book's author's details (N).

To fix it, navigate to books/views.py and modify book_list_view() like so:

# books/views.py
def book_list_view(request):
    books = Book.objects.all().select_related("author")  # modified
    return JsonResponse(
        {
            "count": books.count(),
            "results": [book.to_dict() for book in books],
        }
    )

By utilizing Django's select_related() method, we select the additional related object data (i.e., author) in the initial query. The ORM will now leverage a SQL join, and the final query will look something like this:

SELECT * FROM "books_book"
    INNER JOIN "books_author" ON ("books_book"."author_id" = "books_author"."id")

Wait for the development server to restart and retest the affected endpoint.

After benchmarking again, the response time goes from 1090 to 45, and the number of queries lowers from 1024 to 2. This is a 24x and 512x improvement, respectively.

Example 2: Many-To-One Relationship (`prefetch_related()`)

Next, let's look at the second slowest endpoint (books/by-authors/).

Use the dashboard as we did in the previous step to inspect the endpoint's SQL queries. You'll notice a similar but less severe N+1 pattern with this endpoint.

This endpoint's performance is less severe because Django is smart enough to cache the frequently executed SQL queries, i.e., repeatedly fetching the author of a book. Check out the official docs to learn more about Django caching.

Let's utilize prefetch_related() in books/views.py to speed up the endpoint:

# books/views.py
def book_by_author_list_view(request):
    try:
        authors = Author.objects.all().prefetch_related("books")  # modified
        return JsonResponse(
            {
                "count": authors.count(),
                "results": [
                    {
                        "author": author.to_dict(),
                        "books": [book.to_dict() for book in author.books.all()],
                    }
                    for author in authors
                ],
            }
        )
    except Author.DoesNotExist:
        return JsonResponse({"error": "Author not found"}, status=404)

In the previous section, we used the select_related() method to handle a one-to-one relationship (each book has a single author). However, in this case, we're handling a one-to-many relationship (an author can have multiple books), so we must use prefetch_related().

The difference between these two methods is that select_related() works on the SQL level, while prefetch_related() optimizes on the Python level. The latter method can also be used for many-to-many relationships.

For more information, check out Django's official docs on prefetch_related().

After benchmarking, the response time goes from 90 to 44 milliseconds, and the number of queries lowers from 32 to 4.

Detect N+1 Queries in Django Admin

Discovering N+1 queries in the Django admin site works similarly.

First, log in to your admin site and generate performance reports (for example, create a few authors or books, update, and delete them).

Next, navigate to your AppSignal app dashboard, this time filtering the issues by admin:

In my case, the two slowest endpoints are:

/admin/login
/admin/books/author/<object_id>

We can't do much about /admin/login, since it's entirely handled by Django, so let's focus on the second slowest endpoint. Inspecting it will reveal an N+1 query problem. The author is fetched separately for each book.

To fix this, override get_queryset() in BookInline to fetch author details in the initial query:

# books/admin.py
class BookInline(admin.TabularInline):
    model = Book
    extra = 0

    def get_queryset(self, request):
        queryset = super().get_queryset(request)
        return queryset.select_related("author")

Benchmark once again and verify that the number of queries has decreased.

Wrapping Up

In this post, we've discussed detecting and fixing N+1 queries in Django using AppSignal.

Leveraging what you've learned here can help you significantly speed up your Django web apps.

The two most essential methods to keep in mind are select_related() and prefetch_related(). The first is used for one-to-one relationships, and the second is for one-to-many and many-to-many relationships.

Happy coding!

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

Track Errors in Your Python Flask Application with AppSignal

Nik Tomazic — Wed, 12 Jun 2024 10:13:07 +0000

In this article, we'll look at how to track errors in a Flask application using AppSignal.

We'll first bootstrap a Flask project, and install and configure AppSignal. Then, we'll introduce some faulty code and demonstrate how to track and resolve errors using AppSignal's Errors dashboard.

Let's get started!

Prerequisites

Before diving into the article, ensure you have:

Python 3.8+ installed on your local machine
An AppSignal-supported operating system
An AppSignal account (you can start a free 30-day trial)
Fundamental Flask knowledge

Project Setup

To demonstrate how AppSignal error tracking works, we'll create a simple TODO app. The app will provide a RESTful API that supports CRUD operations. Initially, it will contain some faulty code, which we'll address later.

I recommend you first follow along with this exact project since the article is tailored to it. After the article, you'll, of course, be able to integrate AppSignal into your own Flask projects.

Start by bootstrapping a Flask project:

Create and activate a virtual environment
Use pip to install the latest version of Flask
Start the development server

If you get stuck, refer to the Flask Installation guide.

Note: The source code for this project can be found in the appsignal-flask-error-tracking GitHub repo.

Install AppSignal for Flask

To add AppSignal to your Flask project, follow the AppSignal documentation:

Ensure everything works by starting the development server:

(venv)$ flask run

Your app should automatically send a demo error to AppSignal. From now on, all your app errors will be forwarded to AppSignal.

If you get an error saying Failed to find Flask application, you most likely imported Flask before starting the AppSignal client. As mentioned in the docs, AppSignal has to be imported and started at the top of app.py.

Flask for Python App Logic

Moving along, let's implement the web app logic.

Flask-SQLAlchemy for the Database

We'll use the Flask-SQLAlchemy package to manage the database. This package provides SQLAlchemy support to Flask projects. That includes the Python SQL toolkit and the ORM.

First, install it via pip:

(venv)$ pip install Flask-SQLAlchemy

Then initialize the database and Flask:

# app.py

db = SQLAlchemy()
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///default.db"
db.init_app(app)

Don't forget about the import:

from flask_sqlalchemy import SQLAlchemy

Next, create the Task database model:

# app.py

class Task(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(128), nullable=False)
    description = db.Column(db.Text(512), nullable=True)

    created_at = db.Column(db.DateTime, default=db.func.now())
    updated_at = db.Column(db.DateTime, default=db.func.now(), onupdate=db.func.now())

    is_done = db.Column(db.Boolean, default=False)

    def as_dict(self):
        return {c.name: getattr(self, c.name) for c in self.__table__.columns}

    def __repr__(self):
        return f"<Task {self.id}>"

Each Task will have a name, an optional description, an is_done field, and some administrative data. To serialize the Task, we'll use its as_dict() method.

Since Flask-SQLAlchemy doesn't automatically create the database and its structure, we must do it ourselves. To handle that, we'll create a simple Python script.

Create an init_db.py file in the project root with the following content:

# init_db.py

from app import Task
from app import db, app

with app.app_context():
    db.create_all()

    if Task.query.count() == 0:
        tasks = [
            Task(
                name="Deploy App",
                description="Deploy the Flask app to the cloud.",
                is_done=False,
            ),
            Task(
                name="Optimize DB",
                description="Optimize the database access layer.",
                is_done=False,
            ),
            Task(
                name="Install AppSignal",
                description="Install AppSignal to track errors.",
                is_done=False,
            ),
        ]

        for task in tasks:
            db.session.add(task)

        db.session.commit()

What's Happening Here?

This script performs the following:

Fetches Flask's app instance.
Creates the database and its structure via db.create_all().
Populates the database with three sample tasks.
Commits all the changes to the database via db.session.commit().

Defining Views

Define the views in app.py like so:

# app.py

@app.route("/")
def list_view():
    tasks = Task.query.all()
    return jsonify([task.as_dict() for task in tasks])


@app.route("/<int:task_id>", methods=["GET"])
def detail_view(task_id):
    task = db.get_or_404(Task, task_id)
    return jsonify(task.as_dict())


@app.route("/create", methods=["POST"])
def create_view():
    name = request.form.get("name", type=str)
    description = request.form.get("description", type=str)

    task = Task(name=name, description=description)
    db.session.add(task)
    db.session.commit()

    return jsonify(task.as_dict()), 201


@app.route("/toggle-done/<int:task_id>", methods=["PATCH"])
def toggle_done_view(task_id):
    task = db.get_or_404(Task, task_id)

    task.is_done = not task.is_done
    db.session.commit()

    return jsonify(task.as_dict())


@app.route("/delete/<int:task_id>", methods=["DELETE"])
def delete_view(task_id):
    task = db.get_or_404(Task, task_id)

    db.session.delete(task)
    db.session.commit()

    return jsonify({}), 204


@app.route("/statistics", methods=["GET"])
def statistics_view():
    done_tasks_count = Task.query.filter_by(is_done=True).count()
    undone_tasks_count = Task.query.filter_by(is_done=False).count()
    done_percentage = done_tasks_count / (done_tasks_count + undone_tasks_count) * 100

    return jsonify({
        "done_tasks_count": done_tasks_count,
        "undone_tasks_count": undone_tasks_count,
        "done_percentage": done_percentage,
    })

Don't forget about the import:

from flask import jsonify, request

What's Happening Here?

We define six API endpoints.
The list_view() fetches all the tasks, serializes and returns them.
The detail_view() fetches a specific task, serializes and returns it.
The create_view() creates a new task from the provided data.
toggle_done_view() toggles the task's is_done property.
The delete_view() deletes a specific task.
The statistics_view() calculates general app statistics.

Great, we've successfully created a simple TODO web app!

Test Your Python Flask App's Errors with AppSignal

During the development of our web app, we intentionally left in some faulty code. We'll now trigger these bugs to see what happens when an error occurs.

Before proceeding, ensure your Flask development server is running:

(venv)$ flask run --debug

Your API should be accessible at http://localhost:5000/.

AppSignal should, of course, be employed when your application is in production rather than during development, as shown in this article.

Error 1: `OperationalError`

To trigger the first error, request the task list:

$ curl --location 'localhost:5000/'

This will return an Internal Server Error. Let's use AppSignal to figure out what went wrong.

Open your favorite web browser and navigate to your AppSignal dashboard. Select your organization and then your application. Lastly, choose "Errors > Issue list" on the sidebar:

You'll see that an OperationalError was reported. Click on it to inspect it:

The error detail page will display the error message, backtrace, state, trends, and so on.

We can figure out what went wrong just by looking at the error message. no such table: task tells us that we forgot to initialize the database.

To fix that, run the previously created script:

(venv)$ python init_db.py

Retest the app and mark the issue as "Closed" once you've verified everything works.

Error 2: `IntegrityError`

Let's trigger the next error by trying to create a task without a name:

$ curl --location 'localhost:5000/create' \
       --form 'description="Test the web application."'

Open the AppSignal dashboard and navigate to the IntegrityError's details.

Now instead of just checking the error message, select "Samples" in the navigation:

A sample refers to a recorded instance of a specific error. Select the first sample.

By checking the backtrace, we can see exactly what line caused the error. As you can see, the error happened in app.py on line 53 when we tried saving the task to the database.

To fix it, provide a default when assigning the name variable:

# app.py

@app.route("/create", methods=["POST"])
def create_view():
    name = request.form.get("name", type=str, default="Unnamed Task")  # new
    description = request.form.get("description", type=str, default="")

    task = Task(name=name, description=description)
    db.session.add(task)
    db.session.commit()

    return jsonify(task.as_dict()), 201

Error 3: `ZeroDivisionError`

Now we'll delete all tasks and then calculate the statistics by running the following commands:

$ curl --location --request DELETE 'localhost:5000/delete/1'
$ curl --location --request DELETE 'localhost:5000/delete/2'
$ curl --location --request DELETE 'localhost:5000/delete/3'
$ curl --location --request DELETE 'localhost:5000/delete/4'
$ curl --location --request DELETE 'localhost:5000/delete/5'
$ curl --location 'localhost:5000/statistics'

As expected, a ZeroDivisonError is raised. To track the error, follow the same approach as described in the previous section.

To fix it, add a zero check to the statistics_view() endpoint like so:

# app.py

@app.route("/statistics", methods=["GET"])
def statistics_view():
    done_tasks_count = Task.query.filter_by(is_done=True).count()
    undone_tasks_count = Task.query.filter_by(is_done=False).count()

    # new
    if done_tasks_count + undone_tasks_count == 0:
        done_percentage = 0
    else:
        done_percentage = done_tasks_count / \
                          (done_tasks_count + undone_tasks_count) * 100

    return jsonify({
        "done_tasks_count": done_tasks_count,
        "undone_tasks_count": undone_tasks_count,
        "done_percentage": done_percentage,
    })

Retest the endpoint and mark it as "Closed" once you've verified the issue has been resolved.

Manual Tracking

By default, errors are only reported to AppSignal when exceptions are left unhandled. However, in some cases, you may want handled exceptions to be reported.

To accomplish this, you can utilize AppSignal's helper methods:

Wrapping Up

In this article, we've covered how to monitor errors in a Flask app using AppSignal.

We explored two error reporting methods: automatic tracking and manual tracking (using helper methods). With this knowledge, you can easily incorporate AppSignal into your Flask projects.

Happy coding!

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

Track Errors in Your Python Django Application with AppSignal

Nik Tomazic — Wed, 13 Mar 2024 12:12:17 +0000

In this post, we will specifically look at using AppSignal to track errors in a Django application.

We'll first create a Django project, install AppSignal, introduce some faulty code, and then use the AppSignal Errors dashboard to debug and resolve errors.

Let's get started!

Prerequisites

To follow along, you'll need:

Python 3.8+ installed on your local machine
An AppSignal-supported operating system
An AppSignal account
Basic Django knowledge

Project Setup

We'll be working on a movie review web app. The app will allow us to manage movies and reviews via a RESTful API. To build it, we'll utilize Django.

I recommend you follow along with the movie review web app first. After you grasp the basic concepts, using AppSignal with your projects will be easy.

Begin by creating and activating a new virtual environment, installing Django, and bootstrapping a new Django project.

If you need help, refer to the Quick install guide from the Django docs.

Note: The source code for this project can be found on the appsignal-django-error-tracking GitHub repo.

Install AppSignal for Django

To add AppSignal to a Django-based project, follow the AppSignal documentation:

To ensure everything works, start the development server. Your Django project will automatically send a demo error to AppSignal. In addition, you should receive an email notification.

From now on, all your app errors will be reported to AppSignal.

App Logic

Moving along, let's implement the movie review app logic.

You might notice that some code snippets contain faulty code. The defective code is placed there on purpose to later demonstrate how AppSignal error tracking works.

First, create a dedicated app for movies and reviews:

(venv)$ python manage.py startapp movies

Add the newly-created app to INSTALLED_APPS in settings.py:

# core/settings.py

INSTALLED_APPS = [
    # ...
    'movies.apps.MoviesConfig',
]

Define the app's database models in models.py:

# movies/models.py

from django.db import models


class Movie(models.Model):
    title = models.CharField(max_length=128)
    description = models.TextField(max_length=512)
    release_year = models.IntegerField()

    def get_average_rating(self):
        reviews = MovieReview.objects.filter(movie=self)
        return sum(review.rating for review in reviews) / len(reviews)

    def serialize_to_json(self):
        return {
            'id': self.id,
            'title': self.title,
            'description': self.description,
            'release_year': self.release_year,
        }

    def __str__(self):
        return f'{self.title} ({self.release_year})'


class MovieReview(models.Model):
    movie = models.ForeignKey(Movie, on_delete=models.CASCADE)
    rating = models.IntegerField()

    def save(self, force_insert=False, force_update=False, using=None, update=None):
        if self.rating < 1 or self.rating > 5:
            raise ValueError('Rating must be between 1 and 5.')

        super().save(force_insert, force_update, using, update)

    def serialize_to_json(self):
        return {
            'id': self.id,
            'movie': self.movie.serialize_to_json(),
            'rating': self.rating,
        }

    def __str__(self):
        return f'{self.movie.title} - {self.rating}'

What's Happening Here?

We define two models, Movie and MovieReview. A Movie can have multiple MovieReviews. Both models include a serialize_to_json() method for serializing the object to JSON.
The Movie model includes get_average_rating() for calculating a movie's average rating.
MovieReview's rating is locked in a [1, 5] interval via the modified save() method.

Now, make migrations and migrate the database:

(venv)$ python manage.py makemigrations
(venv)$ python manage.py migrate

Define the Views in Python

Next, define the views in movies/views.py:

# movies/views.py

from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods

from movies.models import Movie, MovieReview


def index_view(request):
    queryset = Movie.objects.all()
    data = [movie.serialize_to_json() for movie in queryset]

    return JsonResponse(data, safe=False)


def statistics_view(request):
    queryset = Movie.objects.all()
    data = []

    for movie in queryset:
        data.append({
            'id': movie.id,
            'title': movie.title,
            'average_rating': movie.get_average_rating(),
        })

    return JsonResponse(data, safe=False)


@csrf_exempt
@require_http_methods(["POST"])
def review_view(request):
    movie_id = request.POST.get('movie')
    rating = request.POST.get('rating')

    if (movie_id is None or rating is None) or \
            (not movie_id.isdigit() or not rating.isdigit()):
        return JsonResponse({
            'detail': 'Please provide a `movie` (int) and `rating` (int).',
        }, status=400)

    movie_id = int(movie_id)
    rating = int(rating)

    try:
        movie = Movie.objects.get(id=movie_id)
        MovieReview.objects.create(
            movie=movie,
            rating=rating,
        )

        return JsonResponse({
            'detail': 'A review has been successfully posted',
        })

    except Movie.DoesNotExist:
        return JsonResponse({
            'detail': 'Movie does not exist.',
        }, status=400)

What's Happening Here?

We define three views: index_view(), statistics_view(), and review_view().
The index_view() fetches all the movies, serializes them, and returns them.
The statistics_view() calculates and returns the average rating of every movie.
The review_view() allows users to create a movie review by providing the movie ID and a rating.

Register the URLs

The last thing we must do is take care of the URLs.

Create a urls.py file within the movies app with the following content:

# movies/urls.py

from django.urls import path

from movies import views

urlpatterns = [
    path('statistics/', views.statistics_view, name='movies-statistics'),
    path('review/', views.review_view, name='movies-review'),
    path('', views.index_view, name='movies-index'),
]

Then register the app URLs globally:

# core/urls.py

from django.contrib import admin
from django.urls import path, include  # new import

urlpatterns = [
    path('movies/', include('movies.urls')),  # new
    path('admin/', admin.site.urls),
]

Using Fixtures

To get some test data to work with, I've prepared two fixtures.

First, download the fixtures, create a fixtures folder in the project root, and place them in there:

django-error-tracking/
+-- fixtures/
    +-- Movie.json
    +-- MovieReview.json

After that, run the following two commands to load them:

(venv)$ python manage.py loaddata fixtures/Movie.json --app app.Movie
(venv)$ python manage.py loaddata fixtures/MovieReview.json --app app.MovieReview

We now have a functional API with some sample data to work with. In the next section, we'll test it.

Test Your Django App's Errors with AppSignal

During the development of our web app, we left intentional bugs in the code. We will now deliberately trigger these bugs to see what happens when an error occurs.

Before proceeding, ensure your Django development server is running:

(venv)$ python manage.py runserver

Your API should be accessible at http://localhost:8000/movies.

AppSignal should, of course, be employed when your application is in production rather than during development, as shown in this article.

Error 1: `ZeroDivisionError`

Start by visiting http://localhost:8000/movies/statistics in your favorite browser. This endpoint is supposed to calculate and return average movie ratings, but it results in a ZeroDivisonError.

Let's use AppSignal to figure out what went wrong.

First, navigate to your AppSignal dashboard and select your application. On the sidebar, you'll see several categories. Select "Errors > Issue list":

You'll see that a ZeroDivisionError has been reported. Click on it to inspect it.

The error detail page displays the error's summary, trends, state, severity, etc. But we are interested in the samples. Select the "Samples" menu item in the navigation bar to access them.

A sample refers to a recorded instance of a specific error. Select the first sample.

The sample's detail page shows the error message, what device triggered the error, the backtrace, and more. We can look at the backtrace to determine what line of code caused the error.

In the backtrace, we can see that the error occurred in movies/models.py on line 16. Looking at the code, the error is obvious: if a movie has no reviews, this results in a division by zero.

To fix this error, all you have to do is slightly modify the Movie's get_average_rating() method:

# movies/models.py

class Movie(models.Model):
    # ...

    def get_average_rating(self):
        reviews = MovieReview.objects.filter(movie=self)

        # new condition
        if len(reviews) == 0:
            return 0

        return sum(review.rating for review in reviews) / len(reviews)

    # ...

Reload the development server, test the functionality again, and set the error's state to "Closed" if everything works as expected.

Error 2: `ValueError`

We can trigger another error by submitting a review with a rating outside the [1, 5] interval. To try it out, open your terminal and run the following command:

$ curl --location --request POST 'http://localhost:8000/movies/review/' \
     --form 'movie=2' \
     --form 'rating=6'

As expected, a ValueError is reported to AppSignal. To track the error, follow the same procedure as in the previous section.

The error can be easily fixed by adding a check to review_view() in views.py:

# movies/views.py

from django.core.validators import MinValueValidator, MaxValueValidator

# ...

@csrf_exempt
@require_http_methods(["POST"])
def review_view(request):
    # ...

    movie_id = int(movie_id)
    rating = int(rating)

    if rating < 1 or rating > 5:
        return JsonResponse({
            'detail': 'Rating must be between 1 and 5.',
        }, status=400)

    try:
        movie = Movie.objects.get(id=movie_id)
        MovieReview.objects.create(
            movie=movie,
            rating=rating,
        )
        # ...

    # ...

After that, test the functionality again and tag the error as "Resolved" if everything works as expected.

Wrapping Up

In this article, you've learned how to use AppSignal to track errors in a Django application.

To get the most out of AppSignal for Python, I suggest you review the following two resources:

Happy coding!

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

DEV Community: Nik Tomazic

Find and Fix N+1 Queries in Django Using AppSignal

What Are N+1 Queries?

Prerequisites

Project Setup

Install AppSignal for Django

Web App Logic

Models

Views

Detect N+1 Queries in Your Django App with AppSignal

Detect N+1 Queries in Views

Example 1: One-To-One Relationship (select_related())

Example 2: Many-To-One Relationship (prefetch_related())

Detect N+1 Queries in Django Admin

Wrapping Up

Track Errors in Your Python Flask Application with AppSignal

Prerequisites

Project Setup

Install AppSignal for Flask

Flask for Python App Logic

Flask-SQLAlchemy for the Database

What's Happening Here?

Defining Views

What's Happening Here?

Test Your Python Flask App's Errors with AppSignal

Error 1: OperationalError

Error 2: IntegrityError

Error 3: ZeroDivisionError

Manual Tracking

Wrapping Up

Track Errors in Your Python Django Application with AppSignal

Prerequisites

Project Setup

Install AppSignal for Django

App Logic

What's Happening Here?

Define the Views in Python

What's Happening Here?

Register the URLs

Using Fixtures

Test Your Django App's Errors with AppSignal

Error 1: ZeroDivisionError

Error 2: ValueError

Wrapping Up

Example 1: One-To-One Relationship (`select_related()`)

Example 2: Many-To-One Relationship (`prefetch_related()`)

Error 1: `OperationalError`

Error 2: `IntegrityError`

Error 3: `ZeroDivisionError`

Error 1: `ZeroDivisionError`

Error 2: `ValueError`