DEV Community: AMIR TADRISI

Advanced Open edX Monitoring with AppSignal for Python

AMIR TADRISI — Wed, 20 Nov 2024 09:53:11 +0000

In the first part of this series, we explored how AppSignal can significantly enhance the robustness of Open edX platforms. We saw the challenges that Open edX faces as it scales and how AppSignal's features — including real-time performance monitoring and automated error tracking — provide essential tools for DevOps teams. Our walkthrough covered the initial setup and integration of AppSignal with Open edX, highlighting the immediate benefits of this powerful observability framework.

In this second post, we'll dive deeper into the advanced monitoring capabilities that AppSignal offers. This includes streaming logs from Open edX to AppSignal, monitoring background workers with Celery, and tracking Redis queries. We will demonstrate how these features can be leveraged to address specific operational challenges, ensuring that our learning platform remains fail-safe under varying circumstances.

By the end of this article, you will know how to utilize AppSignal to its full potential in maintaining and improving the performance and reliability of your Open edX platform.

Streaming Logs to AppSignal

One of AppSignal's strongest features is centralized log management.

Commonly at Open edX, the support team reports an issue with the site, and an engineer can SSH into the server right away to check for Nginx, Mongo, MySQL, and Open edX Application logs.

A centralized storage place that houses logs without the need for you to SSH into the server is a really powerful feature. We can also set up notifications based on an issue's severity.

Now let's see how we can stream our logs from Open edX to AppSignal.

Create a Source

Under the Logging section, click on Manage sources and create a new source, with HTTP as the platform and JSON as the format. After creating the source, AppSignal provides an endpoint and API KEY that we can POST our logs to.

To have more control over log transmission, we can write a simple Python script that reads logs from our local Open edX, pre-processes them, and moves the important ones to AppSignal. For example, I wrote the following script to move only ERROR logs to AppSignal (skipping INFO and WARNING logs):

import requests
import json
from datetime import datetime
import logging

# Setup logging configuration
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

# File to keep track of the last processed line
log_pointer_file = '/root/.local/share/tutor/data/lms/logs/processed.log'
log_file = '/root/.local/share/tutor/data/lms/logs/all.log'

# APpSignal API KEY
api_key = "MY-API-KEY"  # Replace with your actual API key
# URL to post the logs
url = f'https://appsignal-endpoint.net/logs?api_key={api_key}'

def read_last_processed():
    try:
        with open(log_pointer_file, 'r') as file:
            content = file.read().strip()
            last_processed = int(content) if content else 0
            logging.info(f"Last processed line number read: {last_processed}")
            return last_processed
    except (FileNotFoundError, ValueError) as e:
        logging.error(f"Could not read from log pointer file: {e}")
        return 0

def update_last_processed(line_number):
    try:
        with open(log_pointer_file, 'w') as file:
            file.write(str(line_number))
            logging.info(f"Updated last processed to line number: {line_number}")
    except Exception as e:
        logging.error(f"Could not update log pointer file: {e}")

def parse_log_line(line):
    if 'ERROR' in line:
        parts = line.split('ERROR', 1)
        timestamp = parts[0].strip()
        message_parts = parts[1].strip().split(' - ', 1)
        message = message_parts[1] if len(message_parts) > 1 else ''
        attributes_part = message_parts[0].strip('[]').split('] [')
        # Flatten attributes into a dictionary with string keys and values
        attributes = {}
        for attr in attributes_part:
            key_value = attr.split(None, 1)
            if len(key_value) == 2:
                key, value = key_value
                key = key.rstrip(']:').replace(' ', '_').replace('.', '_')  # Replace spaces and dots in keys
                if len(key) <= 50:
                    attributes[key] = value
        # Format the timestamp
        formatted_timestamp = datetime.strptime(timestamp, '%Y-%m-%d %H:%M:%S,%f').isoformat()[:-3] + 'Z'
        # Add the message and attributes to the log structure
        json_data = {
            "timestamp": formatted_timestamp,
            "group": "openedx",
            "severity": "error",
            "hostname": "tutor",
            "message": message,
        }
        json_data.update(attributes)  # Add the attributes directly to the json_data dictionary
        return json_data

def post_logs(json_data):
    headers = {'Content-Type': 'application/json'}
    response = requests.post(url, json=json_data, headers=headers)
    logging.info(f"Posted log to server; HTTP status code: {response.status_code}, Response: {response.content}")
    return response.status_code

def process_logs():
    last_processed = read_last_processed()
    with open(log_file, 'r') as file:
        for i, line in enumerate(file, 1):
            if i > last_processed:
                json_data = parse_log_line(line)
                if json_data:
                    response_code = post_logs(json_data)
                    if response_code == 200:
                        update_last_processed(i)
                    else:
                        logging.warning(f"Failed to post log, HTTP status code: {response_code}")

if __name__ == '__main__':
    logging.info("Starting log processing script.")
    process_logs()
    logging.info("Finished log processing.")

Here's how the script works:

Log File Management: Tutor saves all of the logs in the /root/.local/share/tutor/data/lms/logs/all.log file. This file contains MySQL, LMS, CMS, Caddy, Celery, and other services. The script uses a pointer /root/.local/share/tutor/data/lms/logs/processed.log file that tracks the last processed line. This ensures that each log is processed only once.
Error Filtering: As mentioned, we only send ERROR logs to AppSignal.
Data Parsing and Formatting: Each error log is parsed to extract key pieces of information, such as the timestamp and error message. The script formats this data into a JSON structure suitable for transmission.
Log Transmission: The formatted log data is sent to AppSignal using an HTTP POST request.

Important: Please make sure you don't send any personally identifiable information to the endpoint.

Now run this script and it should move ERROR logs to AppSignal:

You can also create a new trigger to notify you as soon as a specific event like ERROR happens:

Monitor Celery and Redis using AppSignal

Celery (a distributed task queue) is a vital component of Open edX, responsible for managing background tasks such as grading, certificate generation, and bulk email dispatch. Redis often acts as the broker for Celery, managing task queues. Both systems are essential for asynchronous processing and can become bottlenecks during periods of high usage. Monitoring these services with AppSignal provides valuable insights into task execution and queue health, helping you preemptively address potential issues. Let's see how we can monitor Celery and Redis.

First, install the necessary packages. Add the following to the OPENEDX_EXTRA_PIP_REQUIREMENTS variable in the .local/share/tutor/config.yml file:

- opentelemetry-instrumentation-celery==0.45b0
- opentelemetry-instrumentation-redis==0.45b0

It should look like the following:

OPENEDX_EXTRA_PIP_REQUIREMENTS:
  - appsignal==1.3.0
  - opentelemetry-instrumentation-django==0.45b0
  - opentelemetry-instrumentation-celery==0.45b0
  - opentelemetry-instrumentation-redis==0.45b0

As you can see, we are installing opentelemetry packages for Celery and Redis.

Now, we can instrument Celery with worker_process_init to report its metrics to AppSignal.

Heading back to our dashboard in AppSignal, we should see Celery and Redis reports in the Performance section, with background as the namespace.

For Redis queries, you can click on Slow queries:

Practical Monitoring: Enhancing Open edX with AppSignal

In this section, we'll revisit the initial issues outlined in part one of this series and apply practical AppSignal monitoring solutions to ensure our Open edX platform stays robust and reliable. Here’s a breakdown.

Site Performance Improvement

Let's begin by assessing overall site performance. In the Performance section, under the Issue list, we can see key metrics for all visited URLs:

Response Time: Directly reflects user experience by measuring the time taken to process and respond to requests. Factors influencing this include database queries and middleware operations.
Throughput: Indicates the number of requests handled within a given timeframe.
Mean Response Time: Provides an average response time across all requests to a specific endpoint. Any mean response time over 1 second is a potential concern and highlights areas that need optimization.
90th Percentile Response Time: For example, a 90th percentile response time of 7 ms for GET store/ suggests that 90% of requests complete in 7 ms or less.

Now let's order all the actions based on the mean. Any item higher than 1 second should be considered a red flag:

As we see, Celery tasks to rescore and reset student attempts, LMS requests to show course content, and some APIs are taking more than 1 second. Also, we should note that this is only for one active user. If we have more concurrent users, this response time will go up. Our first solution is to add more resources to the server (CPU and memory) and do another performance test.

After identifying actions with mean response times exceeding 1 second, consider performance optimization strategies such as:

Minimizing JavaScript execution
Using CDNs for static content
Implementing caching techniques.

Server Resource Monitoring

We talked about anomaly detection and host monitoring in the previous article. Let's add triggers for the following items:

CPU usage
Disk usage
Memory usage
Network traffic
Error rate

Custom Metrics

Two really important metrics for our platform are our number of active users and enrollments. Let's see how we can measure these metrics using AppSignal.

First, add increment_counter to common/djangoapps/student/views/management.py and openedx/core/djangoapps/user_authn/views/login.py to track and increment the number of logins and enrollments when there is a new event.

Now let's log in to Open edX and enroll in a course. Next, let's head to our dashboard in AppSignal. Click on Add dashboard, then Create dashboard, and give it a name and description.

Click on Add graph, enter Active Users as the title, select Add Metric and use login_count:

Your dashboard should look like the following:

You can follow the same steps to add a graph for enrollments using an enrollment_count metric.

Ensuring Consistent Styling

To make sure our site's styling stays consistent, let's add a new uptime check for static/tailwind/css/lms-main-v1.css and get notified when a URL is broken:

Email Delivery and Error Handling

In the Error section of the dashboard, we can view all errors, set up notifications for them, and work on fixes as soon as possible to prevent users from being negatively impacted.

Background Job Efficiency for Grading

In the Monitor Celery and Redis section of this article, we saw how to instrument Celery and Redis using AppSignal. Let's follow the same steps to enable AppSignal so we can see graded tasks. In the lms/djangoapps/grades/tasks.py file, add the following lines:

We should now see a couple of items to grade under Performance -> Issue list.

As you can see, recalculate_subsection_grade_v3 (our main grading Celery task) takes 212 milliseconds. For regrading, lms.djangoapps.instructor_task.tasks.reset_problem_attempts and lms.djangoapps.instructor_task.tasks.rescore_problem take 1.77 seconds.

Wrapping Up

In this two-part series, we integrated AppSignal with Open edX to fortify its monitoring capabilities. We started with the basics — setting up and understanding the fundamental offerings of AppSignal, including error tracking and performance monitoring.

In this article, we tackled how to efficiently stream logs from various Open edX services to AppSignal, ensuring all relevant information was centralized and readily accessible. We also monitored crucial asynchronous tasks handled by Celery and Redis.

Finally, we addressed some real-world challenges, such as slow site responses, resource bottlenecks during high enrollment periods, and unexpected issues like broken styling.

By now, you should have a comprehensive understanding of how to leverage AppSignal to not just monitor, but also significantly improve, the performance and reliability of your Open edX platform.

If you have any questions about Open edX or need further assistance, feel free to visit cubite.io or reach out to me directly at amir@cubite.io.

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

Integrating Open edX with AppSignal

AMIR TADRISI — Wed, 16 Oct 2024 12:29:44 +0000

Imagine stepping into the role of a DevOps engineer at an online learning company that utilizes Open edX as its core Learning Management System (LMS). As the platform scales to accommodate more learners, a myriad of challenges begin to surface:

Frequent reports of sluggish platform responses
Issues with password reset emails
Discrepancies in grading reflected on progress pages

These are just the tip of the iceberg. It's pivotal that you provide timely reports on site performance and error tracking in real time, and fix any issues before they affect a significant user base.

In this article (part one of a two-part series), we will see how AppSignal can be harnessed to enhance the robustness of Open edX. Leveraging AppSignal's comprehensive suite of features, we aim to not only monitor — but also significantly improve — platform performance.

What is Open edX?

Open edX is at the forefront of the e-learning landscape, powering Learning Management Systems for millions worldwide. With a high demand for online education, maintaining a high-quality learning experience becomes increasingly crucial.

Why AppSignal for Open edX?

In such a dynamic environment, robust monitoring and error tracking are indispensable. This is where AppSignal shines, offering a powerful toolset for early issue detection, thereby mitigating potential impacts on your learners. By integrating AppSignal with Open edX, you unlock essential features for real-time performance monitoring, automated error tracking, and actionable insights that drive informed decisions for your platform’s evolution.

Throughout this article, we’ll explore practical strategies to leverage AppSignal to build a resilient, fail-safe learning platform that consistently delivers outstanding educational experiences.

Open edX Challenges

Over the last decade of working with Open edX, I've frequently heard the following:

"The site is slow."
"We expect a high spike in enrollments and need more server resources."
"Users apply for password resets but never receive the emails."
"Grading delays are affecting the student progress page."
"The site styling breaks intermittently."
"Students encounter 500 errors without any clear explanation."

These issues underscore the critical need for a robust monitoring platform. Without it, functions like course enrollments could malfunction, overwhelming the support team or deteriorating user experiences. This may potentially cause learners to abandon courses that educators have heavily invested in developing.

The Need for Dedicated Monitoring

Open edX lacks a default system for real-time error tracking, performance monitoring, and user behavior analysis. This gap necessitates a third-party solution that can integrate seamlessly and enhance platform reliability.

Concurrent Users and Performance Issues: Handling large numbers of concurrent users during busy periods is vital for ensuring a smooth, uninterrupted user experience.
Uptime Monitoring: Monitoring downtimes and uptimes is crucial to maintaining a consistent learning environment, and minimizing student frustration due to technical issues.
Anomaly Detection and Error Tracking: Effective monitoring includes identifying unusual behaviors or patterns in user interactions, and providing detailed insights that help to improve educational content and methodologies.

While tools like Sentry and DataDog offer valuable features, they may not fully meet the unique demands of an e-learning environment. This brings us to the distinct advantages of AppSignal.

You can visit Compare AppSignal to Datadog and Compare AppSignal to New Relic to learn about AppSignal's difference to these tools and why it is a better decision to pick AppSignal for Open edX.

How does AppSignal Benefit Open edX?

AppSignal stands out by catering specifically to the needs of platforms like Open edX, providing a robust set of tools designed to tackle the unique challenges of online learning environments:

All-Inclusive Features: AppSignal offers a comprehensive suite of monitoring tools, including error tracking, performance monitoring, and anomaly detection. This holistic approach eliminates the need to manage multiple services.
Real-Time Analytics and Dashboards: With customizable dashboards, AppSignal provides immediate insights into critical metrics such as error rates, uptime, and response times. You can also create custom metrics like enrollment count and visualize these metrics via a custom dashboard.
Ease of Integration: The AppSignal SDK makes integration straightforward, minimizing setup time and allowing for quick implementation of comprehensive monitoring capabilities.

Incorporating AppSignal not only addresses the immediate operational challenges but also enhances the strategic development of Open edX platforms by providing deep, actionable insights into system performance and user engagement.

AppSignal uses the OpenTelemetry protocol, which provides a robust foundation for observability. This integration brings a wealth of out-of-the-box tools and features, enhancing the monitoring capabilities of your Open edX platform.

Prerequisites

To integrate AppSignal with Open edX, we need the following requirements:

Python 3.8 or higher
Django 1.10 or higher
Docker v24.0.5+ (with BuildKit 0.11+)
Docker Compose v2.0.0+
Open edX Koa version or higher
An AppSignal account — sign up for a free 30-day trial
Basic Django and Open edX knowledge

In this article, we will use Tutor to install and customize Open edX.

How to Install Open edX

In this section, we'll quickly review how to install the Open edX Dev instance on the Ubuntu 22.04 operating system.

To Install Tutor on other operating systems, please visit the official Tutor docs.

Open your terminal and run the following commands (before continuing, make sure you have the necessary libraries and applications mentioned in the Prerequisites section):

sudo apt update -y
sudo apt upgrade -y
sudo apt install python3 python3-pip libyaml-dev
pip install "tutor[full]"
tutor local launch

Provide all the necessary information and a correct URL pointing to your server's IP address.

After finishing the deployment you should be able to visit the URL you provided to see your Open edX instance.

Integrate AppSignal

Let's integrate Open edX with AppSignal to start receiving data in our AppSignal Dashboard.

First, go to the AppSignal login page, create a new Python application, and grab your API KEY. We are going to need that in the next step.

Now let's go to where you installed Tutor. Open the file .local/share/tutor/config.yml and add the following libraries to the OPENEDX_EXTRA_PIP_REQUIREMENTS configuration to install all the necessary libraries for AppSignal:

OPENEDX_EXTRA_PIP_REQUIREMENTS:
  - appsignal==1.3.0
  - opentelemetry-instrumentation-django==0.45b0

Next, run tutor config save and relaunch the platform by running tutor local launch. This should install all the necessary packages for you.

After the deployment completes, we'll initialize AppSignal in our Open edX code base and recreate the Open edX Docker images.

Clone the Quince branch of the edx-platform codebase: git clone https://github.com/openedx/edx-platform/tree/open-release/quince.master. Open edx-platform in your favorite editor and follow these instructions:

In the lms directory, create a new file called __appsignal__.py and add the following code:

import os
from openedx.core.djangoapps.site_configuration.models import SiteConfiguration

# Get APPSIGNAL_PUSH_API_KEY from environment
site = SiteConfiguration.objects.get(site__domain="thelearningalgorithm.ai")
APPSIGNAL_PUSH_API_KEY = os.environ.get("APPSIGNAL_PUSH_API_KEY")
os.environ.setdefault("PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION", "python")

from appsignal import Appsignal
appsignal = Appsignal(
    active=True,
    name="openedx",
    push_api_key=APPSIGNAL_PUSH_API_KEY,
)

In your tutor server, create a new environment variable called APPSIGNAL_PUSH_API_KEY and set the AppSignal API key there.

Next, let's start AppSignal in wsgi. Go to lms/wsgi.py and add the following code:

modulestore() # exiting line

# import appsignal
from lms.__appsignal__ import appsignal

# Start Appsignal
appsignal.start()

You can look at this commit to see how I did it.

Learn more about integrating AppSignal with Django.

Now, create a new repo for your edx-platform on GitHub and push your changes there as a new branch. When you are done, run the following command to build a new image for the Open edX container based on your changes:

tutor images build openedx --build-arg EDX_PLATFORM_REPOSITORY=https://github.com/[YOUR ORGANIZTION]/edx-platform.git --build-arg EDX_PLATFORM_VERSION=[YOUR BRANCH NAME] && tutor local stop && tutor local start -d

If you want to build based on my branch, you can run the following command:

tutor images build openedx --build-arg EDX_PLATFORM_REPOSITORY=https://github.com/amirtds/edx-platform.git --build-arg EDX_PLATFORM_VERSION=feat/appsignal-integration && tutor local stop && tutor local start -d

That's it! Now go to your Open edX site and do some basic interactions like logging in and viewing course content. You should see that your AppSignal Dashboard starts populating with data from your instance.

In this section, we added AppSignal for the LMS part of Open edX. You can follow the same steps in the CMS directory to add monitoring for the studio.

Track Errors with AppSignal

Now head to the AppSignal Dashboard and click on Errors -> Issue List on the left sidebar. Here, you should see all the errors that occur when users visit specific URLs. By clicking on each issue, you should see more detailed information.

Integrate AppSignal with your GitHub repo so that an issue is created in the repo automatically when an error occurs. This way, you won't lose track of errors.

In the Graphs, we can see the error rate, count, and throughput:

Monitor Performance

In the Performance section, click on the Issues list. Here, you should see a breakdown of all the visited URLs and important metrics, including:

Mean: The average response time for all the requests made to a particular endpoint. It provides a general idea of how long it takes for the server to respond. In our context, any mean response time greater than 1 second could be considered a red flag, indicating that our application's performance might not meet user expectations for speed.
Throughput: This is the number of requests that are being handled per second.
Impact: An action's impact on our application, based on its usage compared to other actions.

We can also see graphs showing our application's performance:

This information is highly important for us since it shows the user experience when loading each page.
When numbers are high, we should look for ways to improve the user experience and reduce the mean response time. For example, in our case, we have multiple requests that take longer than 1 second. That means there is room to improve our application's performance.

In the Performance section, we can also monitor background tasks and queries like Celery tasks. We'll look at this in more depth in part two of this series.

Anomaly Detection and Alerts

Here, we can define triggers when there is unusual resource usage on our platform. It is highly important to monitor resource usage and take action so our site doesn't go down when there's a spike in users. Additionally, an important function on the platform might fail, resulting in a high number of errors. By using anomaly detection, we get a notification before it impacts more users.

Let's create 2 triggers: one for high memory usage when it's above 70% and one for % of error rates when they are higher than 20%.

In the Anomaly Detection section on the left sidebar, click on Triggers and follow these steps:

Check the Email notification to receive an email notification when an anomaly happens.

In the Issues list, you can see all anomalies based on defined conditions:

Uptime Monitoring

In my experience with Open edX over the past few years, two scenarios have consistently highlighted the need for robust uptime monitoring:

Post-Deployment Styling Issues: After deployments, CSS links can sometimes change, leading to a 404 error on the CSS resource URL and a broken appearance due to missing styles. This not only impacts the aesthetic appeal of Open edX, but can also confuse and frustrate users.
System Downtimes: Various factors can bring an entire site down, such as database outages or failures in the third-party systems that we've integrated. These downtimes disrupt the learning process and can damage our platform's reputation.

Uptime monitoring is invaluable as it not only confirms our site's operational status, but also alerts us the moment it goes down, allowing us to take quick action to restore functionality.

Regional Response Times: An interesting feature of AppSignal's uptime check is its ability to display response times from different regions. This data provides critical insights into how a site's performance varies globally, which is particularly useful for international educational platforms looking to deliver a consistent user experience across diverse geographical locations.

Public Status Page: You can set up a public status page to communicate real-time site status. This page serves as a reliable source for students to verify whether a site is operational, and can reduce the number of support tickets you get.
For instance, you can view an example here.

Simply add a new URL by clicking on Add Uptime Monitor. In this case, I created one for the CSS resource and one for the main LMS URL.

Host Monitoring

Here, we see how our VM resources are being used by our application through visuals and graphs.

Wrapping Up

In this article, we've explored how AppSignal can be integrated with Open edX to bring robust monitoring capabilities to your learning platform. Without requiring extensive configuration beyond an initial setup, AppSignal provides a rich array of features right out of the box. These features enable real-time insights into application performance, error tracking, and uptime monitoring — all essential tools for any DevOps team dedicated to creating a fail-safe learning environment.

In part two of this two-part series, we will dive deeper into the capabilities of AppSignal by demonstrating how to stream logs from Open edX to AppSignal, monitor background tasks with Celery, track Redis queries, and more.

Until then, happy coding!

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

Monitor the Performance of Your Python Django App with AppSignal

AMIR TADRISI — Wed, 14 Aug 2024 11:00:00 +0000

When we observe a slow system, our first instinct might be to label it as failing. This presumption is widespread and highlights a fundamental truth: performance is synonymous with an application's maturity and readiness for production.

In web applications, where milliseconds can determine the success or failure of a user interaction, the stakes are incredibly high. Performance is not just a technical benchmark, but a cornerstone of user satisfaction and operational efficiency.

Performance encapsulates a system's responsiveness under varying workloads, quantified by metrics such as CPU and memory utilization, response times, scalability, and throughput.

In this article, we will explore how AppSignal can monitor and enhance the performance of Django applications.

Let's get started!

Django Performance Monitoring Essentials

Achieving optimal performance in Django applications involves a multifaceted approach. This means developing applications that run efficiently and maintain their efficiency as they scale. Key metrics are crucial in this process, providing tangible data to guide our optimization efforts. Let's explore some of these metrics.

Key Metrics for Performance Monitoring

Response Time: This is perhaps the most direct indicator of user experience. It measures the time taken for a user's request to be processed and the response sent back. In a Django application, factors like database queries, view processing, and middleware operations can influence response times.
Throughput: Throughput refers to the number of requests your application can handle within a given timeframe.
Error Rate: The frequency of errors (4xx and 5xx HTTP responses) can indicate code, database query, or server configuration issues. By monitoring error rates, you can quickly identify and fix problems that might otherwise degrade user experience.
Database Performance Metrics: These include the number of queries per request, query execution time, and the efficiency of database connections.
Handling Concurrent Users: When multiple users access your Django application simultaneously, it is critical to be able to serve all of them efficiently without delays.

What We Will Build

In this article, we'll construct a Django-based e-commerce store ready for high-traffic events, integrating AppSignal to monitor, optimize, and ensure it scales seamlessly under load. We'll also demonstrate how to enhance an existing application with AppSignal for improved performance (in this case, the Open edX learning management system).

Project Setup

Prerequisites

To follow along, you'll need:

Prepare the Project

Now let's create a directory for our project and clone it from GitHub. We'll install all the requirements and run a migration:

mkdir django-performance && cd django-performance
python3.12 -m venv venv
source venv/bin/activate
git clone -b main https://github.com/amirtds/mystore
cd mystore
python3.12 -m pip install -r requirements.txt
python3.12 manage.py migrate
python3.12 manage.py runserver

Now visit 127.0.0.1:8000. You should see something like this:

This Django application is a simple e-commerce store that provides product lists, details, and a checkout page for users. After successfully cloning and installing the app, use the Django createsuperuser management command to create a superuser.

Now let's create a couple of products in our application. First, we'll shell into our Django application by running:

python3.12 manage.py shell

Create 3 categories and 3 products:

from store.models import Category, Product

# Create categories
electronics = Category(name='Electronics', description='Gadgets and electronic devices.')
books = Category(name='Books', description='Read the world.')
clothing = Category(name='Clothing', description='Latest fashion and trends.')

# Save categories to the database
electronics.save()
books.save()
clothing.save()

# Now let's create new Products with slugs and image URLs
Product.objects.create(
    category=electronics,
    name='Smartphone',
    description='Latest model with high-end specs.',
    price=799.99,
    stock=30,
    available=True,
    slug='smartphone',
    image='products/iphone_14_pro_max.png'
)

Product.objects.create(
    category=books,
    name='Python Programming',
    description='Learn Python programming with this comprehensive guide.',
    price=39.99,
    stock=50,
    available=True,
    slug='python-programming',
    image='products/python_programming_book.png'
)

Product.objects.create(
    category=clothing,
    name='Jeans',
    description='Comfortable and stylish jeans for everyday wear.',
    price=49.99,
    stock=20,
    available=True,
    slug='jeans',
    image='products/jeans.png'
)

Now close the shell and run the server. You should see something like the following:

Install AppSignal

We'll install AppSignal and opentelemetry-instrumentation-django in our project.

Before installing these packages, log in to AppSignal using your credentials (you can sign up for a free 30-day trial). After selecting an organization, click on Add app at the top right of the navigation bar. Choose Python as your language and you'll receive a push-api-key.

Make sure your virtual environment is activated and run the following commands:

python3.12 -m pip install appsignal==1.2.1
python3.12 -m appsignal install --push-api-key [YOU-KEY]
python3.12 -m pip install opentelemetry-instrumentation-django==0.45b0

Provide the app name to the CLI prompt. After installation, you should see a new file called __appsignal__.py in your project.

Now let's create a new file called .env in the project root and add APPSIGNAL_PUSH_API_KEY=YOUR-KEY (remember to change the value to your actual key). Then, let's change the content of the __appsignal__.py file to the following:

# __appsignal__.py
import os
from appsignal import Appsignal

# Load environment variables from the .env file
from dotenv import load_dotenv
load_dotenv()

# Get APPSIGNAL_PUSH_API_KEY from environment
push_api_key = os.getenv('APPSIGNAL_PUSH_API_KEY')

appsignal = Appsignal(
    active=True,
    name="mystore",
    push_api_key=os.getenv("APPSIGNAL_PUSH_API_KEY"),
)

Next, update the manage.py file to read like this:

# manage.py
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys

# import appsignal
from __appsignal__ import appsignal # new line


def main():
    """Run administrative tasks."""
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mystore.settings")

    # Start Appsignal
    appsignal.start() # new line

    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == "__main__":
    main()

We've imported AppSignal and started it using the configuration from __appsignal.py.

Please note that the changes we made to manage.py are for a development environment. In production, we should change wsgi.py or asgi.py. For more information, visit AppSignal's Django documentation.

Project Scenario: Optimizing for a New Year Sale and Monitoring Concurrent Users

As we approach the New Year sales on our Django-based e-commerce platform, we recall last year's challenges: increased traffic led to slow load times and even some downtime. This year, we aim to avoid these issues by thoroughly testing and optimizing our site beforehand. We'll use Locust to simulate user traffic and AppSignal to monitor our application's performance.

Creating a Locust Test for Simulated Traffic

First, we'll create a locustfile.py file that simulates simultaneous users navigating through critical parts of our site: the homepage, a product detail page, and the checkout page. This simulation helps us understand how our site performs under pressure.

Create the locustfile.py in the project root:

# locustfile.py
from locust import HttpUser, between, task

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)  # Users wait 1-3 seconds between tasks

    @task
    def index_page(self):
        self.client.get("/store/")

    @task(3)
    def view_product_detail(self):
        self.client.get("/store/product/smartphone/")

    @task(2)
    def view_checkout_page(self):
        self.client.get("/store/checkout/smartphone/")

In locustfile.py, users primarily visit the product detail page, followed by the checkout page, and occasionally return to the homepage. This pattern aims to mimic realistic user behavior during a sale.

Before running Locust, ensure you have a product with the smartphone slug in the Django app. If you don't, go to /admin/store/product/ and create one.

Defining Acceptable Response Times

Before we start, let's define what we consider an acceptable response time. For a smooth user experience, we aim for:

Homepage and product detail pages: under 1 second.
Checkout page: under 1.5 seconds (due to typically higher complexity).

These targets ensure users experience minimal delay, keeping their engagement high.

Conducting the Test and Monitoring Results

With our Locust test ready, we run it to simulate the 500 users and observe the results in real time. Here's how:

Start the Locust test by running locust -f locustfile.py in your terminal, then open http://localhost:8089 to set up and start the simulation. Set the Number of Users to 500 and set the host to http://127.0.0.1:8000
Monitor performance in both Locust's web interface and AppSignal. Locust shows us request rates and response times, while AppSignal provides deeper insights into our Django app's behavior under load.

After running Locust, you can find information about the load test in its dashboard:

Now, go to your application page in AppSignal. Under the Performance section, click on Actions and you should see something like this:

Mean: This is the average response time for all the requests made to a particular endpoint. It provides a general idea of how long it takes for the server to respond. In our context, any mean response time greater than 1 second could be considered a red flag, indicating that our application's performance might not meet user expectations for speed.
90th Percentile: This is the response time at the 90th percentile. For example, for GET store/, we have 7 ms, which means 90% of requests are completed in 7 ms or less.
Throughput: The number of requests handled per second.

Now let's click on the Graphs under Performance:

We need to prepare our site for the New Year sales, as response times might exceed our targets. Here's a simplified plan:

Database Queries: Slow queries often cause performance issues.
Static Assets: Ensure static assets are properly cached. Use a CDN for better delivery speeds.
Application Resources: Sometimes, the solution is as straightforward as adding more RAM and CPUs.

Database Operations

Understanding Database Performance Impact

When it comes to web applications, one of the most common sources of slow performance is database queries. Every time a user performs an action that requires data retrieval or manipulation, a query is made to the database.

If these queries are not well-optimized, they can take a considerable amount of time to execute, leading to a sluggish user experience. That's why it's crucial to monitor and optimize our queries, ensuring they're efficient and don't become the bottleneck in our application's performance.

Instrumentation and Spans

Before diving into the implementation, let's clarify two key concepts in performance monitoring:

Instrumentation
Spans

Instrumentation is the process of augmenting code to measure its performance and behavior during execution. Think of it like fitting your car with a dashboard that tells you not just the speed, but also the engine's performance, fuel efficiency, and other diagnostics while you drive.

Spans, on the other hand, are the specific segments of time measured by instrumentation. In our car analogy, a span would be the time taken for a specific part of your journey, like from your home to the highway. In the context of web applications, a span could represent the time taken to execute a database query, process a request, or complete any other discrete operation.

Instrumentation helps us create a series of spans that together form a detailed timeline of how a request is handled. This timeline is invaluable for pinpointing where delays occur and understanding the overall flow of a request through our system.

Implementing Instrumentation in Our Code

With our PurchaseProductView, we're particularly interested in the database interactions that create customer records and process purchases. By adding instrumentation to this view, we'll be able to measure these interactions and get actionable data on their performance.

Here's how we integrate AppSignal's custom instrumentation into our Django view:

# store/views.py
# Import OpenTelemetry's trace module for creating custom spans
from opentelemetry import trace
# Import AppSignal's set_root_name for customizing the trace name
from appsignal import set_root_name

# Inside the PurchaseProductView
def post(self, request, *args, **kwargs):
    # Initialize the tracer for this view
    tracer = trace.get_tracer(__name__)

    # Start a new span for the view using 'with' statement
    with tracer.start_as_current_span("PurchaseProductView"):
        # Customize the name of the trace to be more descriptive
        set_root_name("POST /store/purchase/<slug>")

        # ... existing code to handle the purchase ...

        # Start another span to monitor the database query performance
        with tracer.start_as_current_span("Database Query - Retrieve or Create Customer"):
            # ... code to retrieve or create a customer ...

            # Yet another span to monitor the purchase record creation
            with tracer.start_as_current_span("Database Query - Create Purchase Record"):
                # ... code to create a purchase record ...

See the full code of the view after the modification.

In this updated view, custom instrumentation is added to measure the performance of database queries when retrieving or creating a customer and creating a purchase record.

Now, after purchasing a product in the Slow events section of the Performance dashboard, you should see the purchase event, its performance, and how long it takes to run the query.

purchase is the event we added to our view.

Using AppSignal with an Existing Django App

In this section, we are going to see how we can integrate AppSignal with Open edX, an open-source learning management system based on Python and Django.

Monitoring the performance of learning platforms like Open edX is highly important, since a slow experience directly impacts students' engagement with learning materials and can have a negative impact (for example, a high number of users might decide not to continue with a course).

Integrate AppSignal

Here, we can follow similar steps as the Project Setup section. However, for Open edX, we will follow Production Setup and initiate AppSignal in wsgi.py. Check out this commit to install and integrate AppSignal with Open edX.

Monitor Open edX Performance

Now we'll interact with our platform and see the performance result in the dashboard.

Let's register a user, log in, enroll them in multiple courses, and interact with the course content.

Actions

Going to Actions, let's order the actions based on their mean time and find slow events:

As we can see, for 3 events (out of the 34 events we tested) the response time is higher than 1 second.

Host Metrics

Host Metrics in AppSignal show resource usage:

Our system isn't under heavy load — the load average is 0.03 — but memory usage is high.

We can also add a trigger to get a notification when resource usage reaches a specific condition. For example, we can set a trigger for when memory usage gets higher than 80% to get notified and prevent outages.

When we hit the condition, you should get a notification like the following:

Celery Tasks Monitoring

In Open edX, we use Celery for async and long-running tasks like certificate generation, grading, and bulk email functionality.
Depending on the task and the number of users, some of these tasks can run for a long time and cause performance issues in our platform.

For example, if thousands of users are enrolled in a course and we need to rescore them, this task can take a while. We might get complaints from users that their grade is not reflected in the dashboard because the task is still running. Having information on Celery tasks, their runtime, and their resource usage gives us important insights and possible points of improvement for our application.

Let's use AppSignal to trace our Celery tasks in Open edX and see the result in the dashboard. First, make sure the necessary requirements are installed. Next, let's set our tasks to trace Celery performance like in this commit.

Now, let's run a couple of tasks in the Open edX dashboard to reset attempts and rescore learners' submissions:

We'll go to the Performance dashboard in AppSignal -> Slow events and we will see something like:

By clicking on Celery, we'll see all the tasks that have run on Open edX:

This is great information that helps us see if our tasks are running longer than expected, so we can address any possible performance bottlenecks.

And that's it!

Wrapping Up

In this article, we saw how AppSignal can give us insights into our Django app's performance.

We monitored a simple e-commerce Django application, including metrics like concurrent users, database queries, and response time.

As a case study, we integrated AppSignal with Open edX, revealing how performance monitoring can be instrumental in enhancing the user experience, particularly for students using the platform.

Happy coding!

P.S. If you'd like to read Python posts as soon as they get off the press, subscribe to our Python Wizardry newsletter and never miss a single post!

How to Optimize Your React App for SEO with Sanity

AMIR TADRISI — Tue, 03 May 2022 13:07:19 +0000

Introduction to SEO

What is SEO?

Search Engine Optimization (SEO) is the practice of increasing the quantity and quality of traffic to your website through organic search engine results. This is done by improving the ranking of your website on search engines. The higher your website ranks, the more likely it will be on the first page of search results and more people will visit it.

Why SEO is important?

68% of online experiences begin with a search engine and 75% of people never scroll past the first page of the search engine results. This means without SEO, your web application could be missing out on the potential growth, success and will be hidden from sight.

How does SEO work?

SEO process can be divided into 3 steps:

Crawling: Crawlers can be thought of as "residents" of the search engine that go out and visit web pages. They then feed information back to the Search Engine about these web pages which are indexed into a database called the index for later retrieval.
Indexing: When people does a Google search, results are not only based on what words are most frequent in your content, but also on which content is most relevant to the user. This means that search engines like Google care about metadata such as titles and meta descriptions when they're ranking your web page in results for a certain keyword.
Ranking: Once the search engine has created it's an index of your website, then it tries to rank all indexed pages that are relevant to a user's search query. The goal of search engines is to provide the user with the best results that match their query.

SEO best practices

Now that we know how SEO works, let’s see what are the best practices for it.

Be a good citizen and make a useful website

Before we go any further, this is probably the most important advice: write useful content! It doesn't matter how much effort you put into optimizing your React app if no one is there to see it. If you're publishing content that people care about, then other internet users are likely to link to your site and share it with their friends, which increases the likelihood of being visible in search engine results even more.

Get descriptive with your titles & meta descriptions

This means that instead of writing titles like "Home", you should write something more descriptive. This is because search engine bots are not intelligent enough to figure out what your page is about based on different words being close to each other..

Create unique content for every page of your website

Having duplicate content on your site is bad for SEO. For example, if you have multiple pages with the same content - search engines will not be able to properly rank each page.

Optimize your page speed

Search engines like Google care about page speed.

An important thing to keep in mind here is that GoogleBot, which crawls the web looking for content, has a very low bandwidth. That means that it shouldn't have trouble crawling any of your website's pages if they don’t take long time to load.

The time it takes for your page to load can have a major impact on how many visitors stick around. Pages that take less than 2 seconds have an average bounce rate 9%, while pages loaded within 5 show 38%.

Optimize Images

We all know that images are a great way to add more diversity and creativity on our pages, but there is something you should do when using them. Optimizing an image will lead not only increase how fast Google Bot crawls through your content; it'll also improve loading speeds for users!

Include a variety of internal links

Internal links are a great way to make your website more accessible for both humans and search engines. They give Google additional signals that the content on these pages is relevant, which will help rank them higher in results; while also making it easier than ever before with all of their extra information!

Create the blog application

In this section, we create a new react.js application called sanity-blog that connects to Sanity.

The completed project can be found on this Github repo! You can also check the live version of this application

Setup a new React application

⚠️ Before you start, make sure Node.js is installed in your environment. To learn more visit the Node website.

Open your Shell and run the following command.

npx create-react-app sanity-blog

Next, run this command to change the directory to the project you created

cd sanity-blog

To ensure that everything is operating fine, execute the following command in your terminal. Your React application should open in http://localhost:3000 on your browser.

npm start

Now let's install a few of the dependencies we'll need. Run the following command in the project's root:

npm install @portabletext/reactimport @portabletext/react @sanity/image-url react-router-dom
npm install -D tailwindcss postcss autoprefixer @tailwindcss/typography
npx tailwindcss init

Next, let’s set up TailwindCSS for styling our front-end. Go to the src/index.css and replace the file content with

@tailwind base;
@tailwind components;
@tailwind utilities;

Now go to the ./tailwind.config.js file and replace it with the following

module.exports = {
  content: ["./src/**/*.{html,js}"],
  theme: {
    extend: {},
  },
  plugins: [require("@tailwindcss/typography")],
}

Setup Sanity

Our goal here is:

Create a new Sanity project for our blog
Customize the default blog Schema to add SEO fields
Add some content

Let's get started by installing Sanity CLI on our local environment.

⚠️ Before running these commands make sure you are in the sanity-blog directory.

npm i -g @sanity/cli

When the Sanity CLI is installed successfully we can activate initialize Sanity in our React project.

sanity init

Now you see a couple of questions in the command line. Answer them as the following:

? Select project to use **Create new project**
? Your project name: **Sanity Blog**
? Use the default dataset configuration? **Yes**
? Project output path: **[Choose default path]**
? Select project template **Blog (schema)**

Once finished you should see a new folder called sanityblog under the sanity-blog head over there and run the following command to start the Sanity Studio.

cd sanityblog
sanity start

After running these commands you should see this

sanity start
✔ Checking configuration files...
⠧ Compiling...webpack built 2f98f0cdc718c8744e79 in 11987ms
✔ Compiling...
Content Studio successfully compiled! Go to http://localhost:3333

Head over to http://localhost:3333 to open Sanity Studio.

Now, let’s add some new fields to our post model so that it can be optimized for search engines.

In your code editor go to the sanityblog → schemas and open the post.js file.

post.js location in the project

Replace the content of this file with the following:

export default {
  name: 'post',
  title: 'Post',
  type: 'document',
  groups: [
    {
      name: 'seo',
      title: 'SEO',
    },
  ],
  fields: [
    {
      name: 'title',
      title: 'Title',
      type: 'string',
    },
    {
      name: 'seoTitle',
      title: 'SEO Title',
      group: 'seo',
      validation: Rule => [
        Rule.required().min(40).max(50).error('SEO titles between 40 and 50 characters with commonly searched words have the best click-through-rates'),
      ],
      type: 'string',
    },
    {
      name: 'seoDescription',
      title: 'SEO Description',
      group: 'seo',
      validation: Rule => [
        Rule.required().min(50).max(156).error('Good SEO descriptions utilize keywords, summarize the story and are between 140-156 characters long.'),
      ],
      type: 'text',
    },
    {
      name: "ogTitle",
      title: "Open Graph Title",
      group: 'seo',
      validation: Rule => [
        Rule.required().min(40).max(50).error('SEO titles between 40 and 50 characters with commonly searched words have the best click-through-rates'),
      ],
      type: "string",
    },
    {
      name: "ogDescription",
      title: "Open Graph Description",
      group: 'seo',
      validation: Rule => [
        Rule.required().min(50).max(156).error('Good SEO descriptions utilize keywords, summarize the story and are between 140-156 characters long.'),
      ],
      type: "text",
    },
    {
      name: "ogImage",
      title: "Open Graph Image",
      group: 'seo',
      type: "image",
    },
    {
      name: 'slug',
      title: 'Slug',
      type: 'slug',
      options: {
        source: 'title',
        maxLength: 96,
      },
    },
    {
      name: 'author',
      title: 'Author',
      type: 'reference',
      to: {type: 'author'},
    },
    {
      name: 'mainImage',
      title: 'Main image',
      type: 'image',
      options: {
        hotspot: true,
      },
    },
    {
      name: 'categories',
      title: 'Categories',
      type: 'array',
      of: [{type: 'reference', to: {type: 'category'}}],
    },
    {
      name: 'publishedAt',
      title: 'Published at',
      type: 'datetime',
    },
    {
      name: 'body',
      title: 'Body',
      type: 'blockContent',
    },
  ],

  preview: {
    select: {
      title: 'title',
      author: 'author.name',
      media: 'mainImage',
    },
    prepare(selection) {
      const {author} = selection
      return Object.assign({}, selection, {
        subtitle: author && `by ${author}`,
      })
    },
  },
}

We added 2 important changes to the default schema:

We've added a new group called SEO that will be appearing as a tab on the Post page. This group contains all the necessary fields for SEO.

We have a whole new set of fields for meta title, description, open graph title, description, and image. These are all validated so that they contain the proper length to produce the best result in SEO.

Finally, let’s create a sample blog post in Sanity Studio

Connecting Sanity to our React app

To connect Sanity to the React app, let’s install Sanity Client first. Go to the root of the project and run the following command

npm install @sanity/client

Next, we need to create an instance of Sanity Client and setup in our project. To do so under the src folder create a new file called client.js Then, inside that file, add the following code:

import sanityClient from "@sanity/client";

export default sanityClient({
    apiVersion: "2022-04-06",
  projectId: "Your Project ID Here",
  dataset: "production",
  useCdn: true,
});

💡 To find the projectId you can go to the sanityblog/sanity.json and search for the projectId

Finally, let’s add react app URL to the Sanity Project CORS origins. In the command line, go to the sanityblog folder and run the following and answer yes to the question.

sanity cors add http://localhost:3000

Create React components

Under the src folder create a new folder called components let’s add AllPosts.js and OnePost.js there

The AllPosts.js file should contain

import React, { useState, useEffect } from 'react';
import sanityClient from "../client"
import imageUrlBuilder from "@sanity/image-url";

const builder = imageUrlBuilder(sanityClient);
function urlFor(source) {
  return builder.image(source);
}

export default function AllPosts() {
    const [posts, setPosts] = useState([]);
    const [loading, setLoading] = useState(true);
    useEffect(() => {
            // This is GROQ syntax for our query, to learn more about it, check out the docs at https://www.sanity.io/docs/groq
      sanityClient.fetch(
        `*[_type == "post"] | order(publishedAt desc) {
          title,
          publishedAt,
          slug,
          body,
          "authorName": author->name,
          "authorImage": author->image,
          mainImage{
            asset->{
              _id,
              url
             }
           },
          categories {
            title,
            slug,
          },
        }`
        )
        .then(posts => {
          setPosts(posts);
          setLoading(false);
        })
    }, []);

    return loading ? (
        <div>Loading...</div>
      ) : ( 
        <div className="relative bg-gray-50 pt-16 pb-20 px-4 sm:px-6 lg:pt-24 lg:pb-28 lg:px-8">
        <div className="absolute inset-0">
          <div className="bg-white h-1/3 sm:h-2/3" />
        </div>
        <div className="relative max-w-7xl mx-auto">
          <div className="text-center">
            <h2 className="text-3xl tracking-tight font-extrabold text-gray-900 sm:text-4xl">From the blog</h2>
            <p className="mt-3 max-w-2xl mx-auto text-xl text-gray-500 sm:mt-4">
              Lorem ipsum dolor sit amet consectetur, adipisicing elit. Ipsa libero labore natus atque, ducimus sed.
            </p>
          </div>
          <div className="mt-12 max-w-lg mx-auto grid gap-5 lg:grid-cols-3 lg:max-w-none">
            {posts.map((post) => (
              <div key={post.slug.current} className="flex flex-col rounded-lg shadow-lg overflow-hidden">
                <a href={`/${post.slug.current}` } className="block mt-2">
                    <div className="flex-shrink-0">
                    <img className="w-full object-cover" src={urlFor(post.mainImage).width(100).url()} alt={post.title} />
                    </div>
                    <div className="flex-1 bg-white p-6 flex flex-col justify-between">
                    <div className="flex-1">
                        <p className="text-xl font-semibold text-gray-900">{post.title}</p>
                    </div>
                    <div className="mt-6 flex items-center">
                        <div className="flex-shrink-0">
                            <span className="sr-only">{post.authorName}</span>
                            <img className="h-10 w-10 rounded-full" src={urlFor(post.authorImage).width(100).url()} alt={post.authorName} />
                        </div>
                        <div className="ml-3">
                        <p className="text-sm font-medium text-gray-900">
                            {post.authorName}
                        </p>
                        <div className="flex space-x-1 text-sm text-gray-500">
                            <time dateTime={post.publishedAt}>{post.publishedAt}</time>
                            <span aria-hidden="true">&middot;</span>
                        </div>
                        </div>
                    </div>
                    </div>
                </a>
              </div>
            ))}
          </div>
        </div>
      </div>
      );
}

The OnePost.js file should contain

import React, {useState, useEffect} from 'react';
import sanityClient from '../client';
import imageUrlBuilder from '@sanity/image-url';
import { PortableText } from '@portabletext/react'

const builder = imageUrlBuilder(sanityClient);
function urlFor(source) {
  return builder.image(source);
}

export default function OnePost(){
    const [post, setPost] = useState([]);
    const [loading, setLoading] = useState(true);
    useEffect(() => {
      sanityClient.fetch(
        `*[_type == "post" && slug.current == $slug][0]{
          title,
          publishedAt,
          slug,
          body,
          "authorName": author->name,
          "authorImage": author->image,
          mainImage{
            asset->{
              _id,
              url
            }
          },
          categories {
            title,
            slug,
          },
        }`,
        {slug: window.location.pathname.split('/')[1]}
      )
      .then(post => {
        setPost(post);
        setLoading(false);
      })
    }, []);

    return loading ? (
        <div>Loading...</div>
    ) : (
      <div className="relative py-16 bg-white overflow-hidden">
          <div className="hidden lg:block lg:absolute lg:inset-y-0 lg:h-full lg:w-full">
            <div className="relative h-full text-lg max-w-prose mx-auto" aria-hidden="true">
              <svg
                className="absolute top-12 left-full transform translate-x-32"
                width={404}
                height={384}
                fill="none"
                viewBox="0 0 404 384"
              >
                <defs>
                  <pattern
                    id="74b3fd99-0a6f-4271-bef2-e80eeafdf357"
                    x={0}
                    y={0}
                    width={20}
                    height={20}
                    patternUnits="userSpaceOnUse"
                  >
                    <rect x={0} y={0} width={4} height={4} className="text-gray-200" fill="currentColor" />
                  </pattern>
                </defs>
                <rect width={404} height={384} fill="url(#74b3fd99-0a6f-4271-bef2-e80eeafdf357)" />
              </svg>
              <svg
                className="absolute top-1/2 right-full transform -translate-y-1/2 -translate-x-32"
                width={404}
                height={384}
                fill="none"
                viewBox="0 0 404 384"
              >
                <defs>
                  <pattern
                    id="f210dbf6-a58d-4871-961e-36d5016a0f49"
                    x={0}
                    y={0}
                    width={20}
                    height={20}
                    patternUnits="userSpaceOnUse"
                  >
                    <rect x={0} y={0} width={4} height={4} className="text-gray-200" fill="currentColor" />
                  </pattern>
                </defs>
                <rect width={404} height={384} fill="url(#f210dbf6-a58d-4871-961e-36d5016a0f49)" />
              </svg>
              <svg
                className="absolute bottom-12 left-full transform translate-x-32"
                width={404}
                height={384}
                fill="none"
                viewBox="0 0 404 384"
              >
                <defs>
                  <pattern
                    id="d3eb07ae-5182-43e6-857d-35c643af9034"
                    x={0}
                    y={0}
                    width={20}
                    height={20}
                    patternUnits="userSpaceOnUse"
                  >
                    <rect x={0} y={0} width={4} height={4} className="text-gray-200" fill="currentColor" />
                  </pattern>
                </defs>
                <rect width={404} height={384} fill="url(#d3eb07ae-5182-43e6-857d-35c643af9034)" />
              </svg>
            </div>
          </div>
          <div className="relative px-4 sm:px-6 lg:px-8">
            <div className="text-lg max-w-prose mx-auto">
              <h1>
                <span className="mt-2 block text-3xl text-center leading-8 font-extrabold tracking-tight text-gray-900 sm:text-4xl">
                  {post.title}
                </span>
              </h1>
              <hr className="mt-8 border-b-2 w-24 mx-auto border-gray-200" />
              <figure>
                <img
                  className="w-full h-72 rounded-lg mt-12"
                  src={urlFor(post.mainImage).width(100).url()} 
                  alt={post.title}
                />
                <figcaption className='text-gray-700 text-center pt-2'>Sagittis scelerisque nulla cursus in enim consectetur quam.</figcaption>
              </figure>
              <div className="mt-8 text-xl text-gray-500 leading-8 prose prose-indigo">
                <PortableText value={post.body} />
              </div>
            </div>
          </div>
        </div>
      )
    }

Finally, replace the src/App.js content with the following

import React from 'react';
import AllPosts from './components/AllPosts';
import OnePost from './components/OnePost';
import { BrowserRouter, Route, Routes } from "react-router-dom";
import './App.css';

export default function App() {
  return (
    <BrowserRouter>
      <Routes>
        <Route path="/" element={<AllPosts />} />
        <Route path=":slug" element={<OnePost />} />
      </Routes>
    </BrowserRouter>
  )
}

Optimize your React app for SEO with Sanity

In this section, we learn how to build a seo-friendly React application through practical techniques using Sanity’s features.

Optimizing images

As we said earlier our site performance is crucial for SEO. Images are assets that can harm site performance if they are not in the proper format or size. The process of optimizing your site can be tedious and time-consuming. It is crucial to have an automatic way, so you don't end up spending hours optimizing images.

Sanity has a global content delivery network (CDN) for serving assets. When we upload our blog images, they can be accessed from cdn.sanity.io

When a user requests an asset, it is processed by Sanity’s backend systems and then cached on servers located near end-users. Subsequent requests are served from the cache so that they can quickly respond with high-quality content without having to slow down or load extra resources for each individual request.

Let’s see an example. For sample blog content I downloaded https://unsplash.com/photos/qWwpHwip31M
The file size is 985 KB in JPEG format with 5184 × 3456 dimensions. I went to the Sanity Studio and uploaded it as a blog image. In the OnePost.js component, we have the following

<img
  className="w-full h-72 rounded-lg mt-12"
  src={urlFor(post.mainImage).width(100).url()} 
  alt={post.title}
/>

As you see in for src we are defining width 100. That helps us to reduce the file size and dimensions.

You can check the file in the CDN here https://cdn.sanity.io/images/a4zwcx9l/production/ffa097961aeffbde70f97ab81ece18dc80705907-5184x3456.jpg?w=100

Metadata

Title

The title tag is an essential SEO element that users see when they click to enter your website from search results. It's also one of the main factors Google uses in determining what a page is about, so using relevant keywords here will help you rank higher on SERPs (search engine result pages).

To be able to add the title and the rest of the meta tags, let’s first install React Helmet

At the root of the project run the following command in the terminal

npm install --save react-helmet

Next, let’s add Meta title in Sanity Studio. In the Studio, click on Post and choose the blog you want to add a title for. In the SEO tab, you can fill out the SEO Title

Finally, let’s import Helmet at the top of the file and add the Meta title to OnePost.js components

import React, {useState, useEffect} from 'react';
import sanityClient from '../client';
import imageUrlBuilder from '@sanity/image-url';
import { PortableText } from '@portabletext/react'
import { Helmet } from 'react-helmet';

export default function OnePost(){
            .
            .
            .
        return loading ? (
        <div>Loading...</div>
        ) : (
      <div>
        <Helmet>
          <title>{post.seoTitle}</title>
        </Helmet>
                .
                .
                .
      </div>
      )
}

Description

The description meta tag is an often overlooked SEO element, but it can have a significant effect on your click-through rate. According to Google this part of the page isn't taken into account when ranking pages; however if you use keywords here they will show up bolded in any searches that include them - so make sure those words are relevant!

Next, let’s add Meta Description in Sanity Studio. In the Studio, click on Post and choose the blog you want to add a description for. In the SEO tab, you can fill out the SEO Description

Finally let’s add Meta description to OnePost.js components

            .
            .
            .
import { Helmet } from 'react-helmet';

export default function OnePost(){
            .
            .
            .
      <div>
        <Helmet>
                    <title>{post.seoTitle}</title>
          <meta name="description" content={post.seoDescription} />
        </Helmet>
            .
            .
            .
      </div>
      )
}

Open Graph

known as og is a way to provide meta information to social media like Facebook and Twitter. It is used to describe a webpage and tell these platforms what sort of content the page contains, such as images.

These Open Graph tags are not Search Engine Optimization (SEO) related, but still, have benefits for sharing your content on social media or messaging apps such as WhatsApp and Telegram.

Next, let’s add Open graph information in Sanity Studio. In the Studio, click on Post and choose the blog you want to add og information for. In the SEO tab, you can fill the Open Graph Title, Description, and Image.

Finally let’s add Open Graph description to OnePost.js components

            .
            .
            .
import { Helmet } from 'react-helmet';

export default function OnePost(){
            .
            .
            .
      <div>
        <Helmet>
                    <title>{post.seoTitle}</title>
                    <meta name="description" content={post.seoDescription} />
          <meta property='og:title' content={post.ogTitle} />
          <meta property='og:description' content={post.ogDescription} />
          <meta property='og:image' content={urlFor(post.ogImage).width(300).url()} />
        </Helmet>
            .
            .
            .
      </div>
      )
}

What else may we do to improve our SEO?

Headings

Headings are used to organize the page content and help users understand its structure. They also allow search engines like Google, Bing or Yahoo! (the ones who actually read this stuff) know which parts of your website they should rank higher in importance for visitors' convenience.

There are 6 different heading, H1 to H6. The H1 heading should represent what your webpage focuses on - it must be similar to a title tag.

Headings are used to organizing the content on a page and help users understand its structure. They also allow search engines like Google, Bing, or Yahoo! (the ones who read this stuff) know which parts of your website should rank higher in importance for visitors' convenience.
There are 6 different headings, H1 to H6. The H1 heading should represent what your webpage focuses on - it must be similar to a title tag.

In the src/components/OnePost.js component, we have the following

<h1>
     <span className="mt-2 block text-3xl text-center leading-8 font-extrabold tracking-tight text-gray-900 sm:text-4xl">
    {post.title}
     </span>
</h1>

Adding a sitemap

Although adding a sitemap to your website is no longer as essential as it once was, it's still a good idea. Google can effortlessly crawl your website, but adding a sitemap is still beneficial for ensuring that all of your content gets seen.

There are plugins like react-router-sitemap-generator that can do it for you.

Server-side rendering and Static site generation

By using React we can build single-page apps. A single page application is an application that loads a single HTML page and necessary assets like Javascript files and CSS are required for the application to run. Any interaction with the application only loads the necessary content and it doesn’t need to reload the page. SPA has better performance in comparison with the traditional web app but it can be challenging for SEO because when the search engine crawls the application it finds an empty page container that doesn’t have meaningful content for the bot and it should wait until Javascript loads to render the page content. This can cause a delay in indexing the page content or the crawler can miss important content to index.

One of the best options we have to solve this is to use pre-rendering technology to generate HTML files in advance, instead of having it all done by client-side Javascript. In the following we see 2 pre-render options react developers have to improve react SEO.

Server-side rendering (SSR)

As we said React websites are usually generated on the client-side browser. The idea behind SSR unlike client-side rendering is to render the initial HTML page on the server instead of waiting for the Javascript to be loaded on the browser. So the server handles the initial rendering of the application. The server imports the React application root component and renders it into an HTML file and sends that rendered HTML to the client.

Implementing SSR by yourself is a time-consuming and complex process it’s highly recommended to take a look at javascript frameworks like next.js that are tailored for SSR.

Static site generation (SSG)

SSG like SSR generates a static HTML page in advance instead of leaving it to the client browser. But the difference between SSG with SSR is in static site generation we generate HTML files during the build time, unlike server-side rendering that generates the HTML on each request. I also recommend taking a look at next.js to build SSG React application.

Conclusion

SEO is an essential part of ensuring your website/application appears on search result pages.

We have seen Sanity has a lot of features that help us to deliver high performance React application and provides a way for any user to add important SEO elements to content for better visibility and ranking in search engines.

DEV Community: AMIR TADRISI

Advanced Open edX Monitoring with AppSignal for Python

Streaming Logs to AppSignal

Create a Source

Monitor Celery and Redis using AppSignal

Practical Monitoring: Enhancing Open edX with AppSignal

Site Performance Improvement

Server Resource Monitoring

Custom Metrics

Ensuring Consistent Styling

Email Delivery and Error Handling

Background Job Efficiency for Grading

Wrapping Up

Integrating Open edX with AppSignal

What is Open edX?

Why AppSignal for Open edX?

Open edX Challenges

The Need for Dedicated Monitoring

How does AppSignal Benefit Open edX?

Prerequisites

How to Install Open edX

Integrate AppSignal

Track Errors with AppSignal

Monitor Performance

Anomaly Detection and Alerts

Uptime Monitoring

Host Monitoring

Wrapping Up

Monitor the Performance of Your Python Django App with AppSignal

Django Performance Monitoring Essentials

Key Metrics for Performance Monitoring

What We Will Build

Project Setup

Prerequisites

Prepare the Project

Install AppSignal

Project Scenario: Optimizing for a New Year Sale and Monitoring Concurrent Users

Creating a Locust Test for Simulated Traffic

Defining Acceptable Response Times

Conducting the Test and Monitoring Results

Database Operations

Understanding Database Performance Impact

Instrumentation and Spans

Implementing Instrumentation in Our Code

Using AppSignal with an Existing Django App

Integrate AppSignal

Monitor Open edX Performance

Actions

Host Metrics

Celery Tasks Monitoring

Wrapping Up

How to Optimize Your React App for SEO with Sanity

Introduction to SEO

What is SEO?

Why SEO is important?

How does SEO work?

SEO best practices

Create the blog application

Setup a new React application

Setup Sanity

Connecting Sanity to our React app

Create React components

Optimize your React app for SEO with Sanity

Optimizing images

Metadata

What else may we do to improve our SEO?

Headings

*Adding a sitemap*

Server-side rendering and Static site generation

Conclusion

Adding a sitemap