DEV Community: Tech by Choice

Debug Mode: Django Docker Pycharm

Valerie Phoenix ✊🏾 — Sat, 07 Sep 2024 02:06:20 +0000

Getting your local setup to debug the code you're writing properly takes more time than any dev would like to admit. And let's not forget this is mostly a one-and-done setup, so if we don't write it down, we won't remember. This post is here to solve that exact issue! Let this serve as a written reminder of how to get your local dev environment up and running.

Prerequisites:

PyCharm Professional (for Docker support)
Docker and Docker Compose
Django REST Framework (DRF) Application

This post will not cover details about the Django, Docker, or Docker composer setup outside of the updates needed for debug mode. It assumes you already have a working knowledge of how to get that part to work.

Step 1: Dockerfile Setup for Debugging

Set up your Dockerfile to run in dev mode and allow connections from the PyCharm debugger.

Below is an example Dockerfile:

# Builder stage
FROM python:3.9-slim as builder

RUN chmod 1777 /tmp

# Install system dependencies
RUN apt-get update && apt-get install -y \
    libpq-dev \
    build-essential

WORKDIR /app

# Copy the requirements file into the container
COPY requirements.txt /app/
RUN pip install --no-cache-dir -r requirements.txt > pip_install.log

# Copy the current directory contents into the container
COPY . /app

# Collect static files
RUN python manage.py collectstatic --noinput

# Final stage
FROM python:3.9-slim

# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV DJANGO_SETTINGS_MODULE=template.settings.development

# Set work directory
WORKDIR /app

# Copy files from the builder stage
COPY --from=builder /app /app

# Install pydevd-pycharm for remote debugging and gunicorn for serving
RUN pip install gunicorn pydevd-pycharm==241.17890.14 psycopg2-binary

# Expose necessary ports
EXPOSE 8000 5679  # Web app port and debug port

# Entry point for the container
ENTRYPOINT ["sh", "-c", "python manage.py runserver 0.0.0.0:8000"]

Things to keep in mind about this code

241.17890.14 in pydevd-pycharm==241.17890.14 is going to be different depending on the version of Pycharm you have
We're exposing both 8000 (the web server port) and 5679 (the debugger port) for external access.

Step 2: Docker Compose Configuration

Let's get our docker-compose.yml file going to configure the web service (Django app) along with the database and other services.

Here is a sample docker-compose.yml:

version: '3'

services:
  web:
    environment:
      - DJANGO_ENVIRONMENT=development
      - DB_HOST=host.docker.internal
    build:
      context: .
    command: >
      sh -c "python manage.py migrate &&
             python manage.py collectstatic --noinput &&
             python manage.py runserver 0.0.0.0:8000"
    volumes:
      - .:/app
    ports:
      - "8000:8000"   # Expose web port
      - "5679:5679"   # Expose debugger port
    extra_hosts:
      - "host.docker.internal:host-gateway"
  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=${DB_NAME}
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}

Let's jump into the code breakdown

We're mapping port 8000 for the web server and port 5679 for the PyCharm debugger.
extra_hosts ensures your Docker container can communicate with the host machine using host.docker.internal.

Step 3: Configure PyCharm for Debugging

Create a Python Debug Server Configuration:
In PyCharm navigate to Run ➡️ Edit Configurations.
Click the + button and select Python Debug Server.
Set the Host to 0.0.0.0 or your local machine’s IP address.
Set the Port to 5679 (or the one you expose in your Docker setup).
And hit Save!
Start the Debugger Server:
Start the PyCharm debugger by clicking the Debug button (green bug icon). This will start listening on the port we set.

Step 4: Add Remote Debugging Code to Django

In your Django project, you'll need to add the following code in your manage.py or wsgi.py to connect to the PyCharm debugger:

import pydevd_pycharm

# Connect to the PyCharm debug server
pydevd_pycharm.settrace('host.docker.internal', port=5679, stdoutToServer=True, stderrToServer=True, suspend=False)

This snippet tells your Django app to connect back to the PyCharm debugger running on your host machine. host.docker.internal resolves to the host machine in Docker, and port=5679 matches the one we exposed earlier.

Step 5: Run Docker and Debug

Build and Run Docker: Run the following command to start your containers:

docker-compose up --build

This will build the Docker image and start the services, including Django running in development mode.

2. Set Breakpoints:
Set breakpoints in your Django code within PyCharm. The breakpoints should work because your container will connect to the PyCharm debug server running on port 5679.

3. Trigger Your Code:
Now, trigger any HTTP request in your Django REST Framework API. When the code hits the breakpoint, PyCharm will pause execution, allowing you to inspect the current state and step through the code.

Step 6: Troubleshooting

If you encounter the error bind: address already in use while running Docker, another process already uses port 5679. In this case, you can:

Stop the PyCharm debugger and restart Docker.
Change the port in your docker-compose.yml and PyCharm configuration to avoid conflicts.

Conclusion

This is the setup I use to run my Django REST Framework application in development mode inside a Docker container using PyCharm’s powerful debugger. This setup helps me debug my backend code by stepping through the code line by line, all locally.

By setting up your Docker container to communicate with PyCharm, you simplify writing, testing, and debugging your Django application, making it easier to write code!

Have fun breaking your code!

Django, Celery, and Redis on Railway

Valerie Phoenix ✊🏾 — Thu, 03 Aug 2023 17:29:18 +0000

Introduction

Using Django Rest Framework, Celery, and Redis I created a personalized quiz to help people find roles in tech they would like. The app asks you questions and creates a custom image with your results that needs time to process in the background, so I leverage Celery and Redis to get this work done!

Now the app took me a day to build, but a week to deploy to production. Why? The age old story of hard-to-find, or nonexistent documentation. To break the broken cycle in tech I’ve created this tutorial to cover the following steps:

How to deploy and set up Redis on Railway
How to connect your Django server to your Redis server on Railway
How to run your Celery workers during the build

This blog will not cover:

The basics of Python
Getting started with Django Rest Framework
Getting started with Celery
Getting started with Redis

Prerequisites

Skills needed: Basic knowledge of Django, Redis, Celery
Tools required: A Django Project that’s set up to use Redis & Celery

Or you can just read along for fun if you’re not going to build with the tutorial!

Section 2: Project Overview

Below is a simplified overview of the project structure so you can have a better understanding of how the project is structured
1. For this blog, I’ve bolded the files that we’ll focus on to make sure

tbc/
├── manage.py
├── ➕ Procfile
├── tbc/
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── ➕ celery.py
│   └── wsgi.py
└── quiz/
    ├── migrations/
    │   └── __init__.py
    ├── __init__.py
    ├── admin.py
    ├── apps.py
    ├── models.py
    ├── serializers.py
    ├── ➕ tasks.py
    ├── tests.py
    ├── urls.py
    └── views.py

celery.py

import os

from celery import Celery

# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tbc.settings')

app = Celery('tasks',
             backend=os.getenv('REDIS_URL'),
             broker=os.getenv('REDIS_URL')
             )

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django apps.
app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
    print(f'Request: {self.request!r}')

Setting Up Railway

Create a Redis Server in Railways

Now that the files are all ready for different environments, let’s head over to the project in Railway to add a new Redis instance.

To get started, head over to your Railway dashboard and select the project you would like to add your Redis setup to. Once in the project, click '+ New' to get the service dropdown.
From the dropdown, you want to select Redis by either typing in Redis to have it pop up or selecting the database to find the Redis option.

Adding a Railways Reference Variable

In our code, we use the REDIS_URL environment variable in our [celery.py](http://celeary.py) file so our project can use different Redis URLs for each environment we’re working in.

For the production environment, we want our REDIS_URL to connect to the Redis server we created in the previous step. To do this we leverage the Railway Reference variable for Redis_URL.

ℹ️ The way I think of Railway Reference variables is a quick way to add variables for Railway-based services without having to type or copy and paste the value because the Railway defines the value for you! Want to learn more about this topic? Check out their documentation on variables here.

Running and Monitoring Celery Tasks In Railway

To run the Celery worker, you’ll need to update your Procfile to look like this:
```
web: celery -A tbc worker --loglevel=info & python manage.py migrate && gunicorn tbc.wsgi  --bind 0.0.0.0:$PORT
```
What is this doing?
This allows Railway to run the celery worker when the system deploys the site. Without this task running your Django app won’t be able to connect with the celery worker.

ℹ️ You may be thinking “hey, I have to start the Redis server when I’m running this on my local. Don’t I have to do the same for production?” And the answer is no, because Railway does it for you.

Why are we updating the Procfile file?

When working in our local files we start our Django and Redis server before we run our celery worker and make sure we run all of our migrations.

What is a Procfile file?
The Procfile is how we tell the server how to run these commands and more after Railway kicks off the server.

What does the command we add mean?
By adding celery -A tbc worker --loglevel=info & before the other commands you’re telling the Railway to run the Celery so your task can run. One thing to call out here is the order of tasks is important. Add the celery details first to avoid possible build errors you could run into if celery isn’t running during the app building.

ℹ️ If you’re familiar with Procfile files, you might be used to seeing the file set up like it is below with the celery details under the worker. The reason we’re placing the worker details on the web line is because Railway only reads commands under the web.

```python
web: python manage.py migrate && gunicorn tbc.wsgi
worker: celery -A tbc worker --loglevel=info
```

Ready for Testing

At this stage everything is connected and ready to run to be tested in production! If you've added any logging to any of your task, you'll see this information show up in your Django server in railway.

If you hit all of your workflows and everything is working as expected, congratulations because you did it!

Conclusion

In this blog we’ve set up our prod Railway environment to work with Django, Redis, and Celery!

We’ve been able to cover
1. How to set up our project to support the production environment
2. Cover the steps needed to create a Redis server in Railway
3. How to connect your Django server in production to your Redis server

Further Resources

Links to documentation and more advanced guides

If you made it this far, I want to just say thanks for following along! Would love to hear your feedback and thoughts on this blog post.

GM from Tech by Choice

Valerie Phoenix ✊🏾 — Fri, 19 Nov 2021 22:27:06 +0000

The mission of Tech by Choice should be one that everyone supports and benefits from. To see our vision come true, we must be transparent and community-driven. There should never be one person filling every role to keep things going.

Being a membership-based nonprofit with a flat org structure is not enough to support the vision I have for the Tech by Choice community. I spent the last year searching for ways I can genuinely help and sustain the growth of Tech by Choice. To my surprise web 3 has offered a solution through its support of Decentralized Autonomous Nonprofit organizations Read more about DAOs here. We’ve partnered with a Black female NFT artist during the giving season that will create a generative art project to highlight the importance community has with ensuring tech is diverse view the project here. We will use a percentage of the funds raised to work with a DAO consultant to support the structural change of Tech by Choice. We will continue to behold our nonprofit status, but our structure internally will change to better support the community and our mission.

Historically when an organization makes a significant change like this, people fear that they will water down what they do. For us, this is the complete opposite. This change will help us support our members more directly and communally. To prove this point, the very first proposal I will submit for our community is to find a way to help our members deal with the Black Tax. This is a topic I’ve been vocal about and one that I know the Tech by Choice community has brought up as well. Through this new structure, we will find a way to elevate this burden for our members.

This is just one example from me, a single member of the community. I know together we will create programs and funds that will support several burdens our members face. But to get there, we have to do this together.

The Tech by Choice community is one I care for deeply. I will always make time for our mission to ensure marginalized people know tech has a space for them. I hope those who read this message will see this change as me shifting my power and status within this organization as a way to keep it going. I hope people do not make this change as I walk away from giving up on our community.

Valerie Phoenix
Founder of Tech by Choice

Where We Stand

Valerie Phoenix ✊🏾 — Sun, 07 Jun 2020 11:11:01 +0000

If anyone has any question on where this organization stands in regards to the recent protest over the countless lives lost due to systemic racism that plagues our globe, I'm afraid you don't understand Tech By Choice. Our mission. Our values. Our community. Our leadership. Our board.

Our current social unrest did not start with Emmett Till and did not end with George Floyd. The black lives lost during the week of May 25 was the tipping point of the unjust treatment of Black people that has been going on for the last 400+ years. No amount of words can express the hurt, anger, and fear our community is going through.

We are all grieving.

Recent events had me question several things. How can we thrive if they don't think we matter? If they don't think we deserve civil rights? If they don't see us?

To me, the best part of being black in tech has been using tech to enhance my ability to turn nothing into something. This is why Tech By Choice is on a mission to ensure underserved adults know they can enter, stay, and thrive in STEAM (Science Technology Art Engineering and Math) because we have always thrived. And always will.

But we can't lose hope.

Do not let our silence be confused with being complicit. We stand with every person protesting. They are our hope.

Our leadership and board are working harder than ever to keep this hope going. Stay tuned for the action we take.

In the meantime remember statements can easily become empty promises. Hold us, and others, accountable to these statements.

If you feel inclined to donate to organizations, we ask that you do your homework and lookup these organizations on sites like Guidestar. Find organizations that are doing the work but don't have the funding that well-known organizations have.

Valerie
Founder & CEO
Tech By Choice

#BlackLivesMatter #SayHerName

Resources for people grieving

DEV Community: Tech by Choice

Debug Mode: Django Docker Pycharm

Prerequisites:

Step 1: Dockerfile Setup for Debugging

Step 2: Docker Compose Configuration

Step 3: Configure PyCharm for Debugging

Step 4: Add Remote Debugging Code to Django

Step 5: Run Docker and Debug

Step 6: Troubleshooting

Conclusion

Django, Celery, and Redis on Railway

Introduction

Prerequisites

Section 2: Project Overview

Setting Up Railway

Create a Redis Server in Railways

Adding a Railways Reference Variable

Running and Monitoring Celery Tasks In Railway

Ready for Testing

Conclusion

Further Resources

GM from Tech by Choice

Where We Stand

#BlackLivesMatter #SayHerName

Resources for people grieving

Mental Health

How to recharge

Protesting

Additional Learning