DEV Community: foadlind

Recover a deleted local branch

foadlind — Tue, 08 Feb 2022 07:02:04 +0000

If you have accidentally deleted a branch that was never pushed to a remote, you can easily recover it in Git.
You'll need help from a useful Git utility called reflog. Let's show you how to do it:

Suppose I have a local branch called feature_1, in which I have made a commit:

* ef640e4 (feature_1) Add settings flag for feature_1
* 4015b6f (HEAD -> main) Provide default for product size
* d8dc31c Add db info to settings

Now I, accidentally, delete the feature_1 branch:

$ git branch -D feature_1
Deleted branch feature_1 (was ef640e4).

So now all my work in that branch is gone 😱:

* 4015b6f (HEAD -> main) Provide default for product size.
* d8dc31c Add db info to settings.

Since I had not yet pushed the branch, I don't have a copy of it on the remote, so I have to resort to reflog. Git always keeps a log of your activities in the repository.

By activities, I mean anything that has to do with Git. So if
you have committed anything, made a branch, stashed anything, it will be notes in the reflog. Git documentation defines it like this:

Reference logs, or "reflogs", record when the tips of branches and other references were updated in the local repository.Reflogs are useful in various Git commands, to specify the old value of a reference.

So let's use it in our case:

$ git reflog
4015b6f (HEAD -> main) HEAD@{0}: checkout: moving from feature_1 to main
ef640e4 HEAD@{1}: commit: Add settings flag for feature_1
4015b6f (HEAD -> main) HEAD@{2}: checkout: moving from main to feature_1
...

There it is! Our last commit in feature_1 branch: ef640e4 HEAD@{1}: commit: Add settings flag for feature_1. That is
the location in our commit tree that we want to be at. It is worth mentioning that branches are nothing but "labels". They just point to a commit in the tree. So if we can make the branch label point to that place in the commit tree, we have restored our branch as it was before deletion.

To do that, first, we recreate our lost branch:

$ git checkout -b feature_1
Switched to a new branch 'feature_1'

Basically we are recreating the label. But now it is pointing to the wrong commit. At the moment it is pointing to the
same commit as main. In our new branch, we use the reset command with the --hard flag to set the branch tip to ef640e4 HEAD@{1}: commit: Add settings flag for feature_1.

(feature_1) $ git reset --hard ef640e4
HEAD is now at ef640e4 Add settings flag for feature_1

So if we look at the Git log, we can see that our branch is restored:

* ef640e4 (HEAD -> feature_1) Add settings flag for feature_1
* 4015b6f (main) Provide default for product size
* d8dc31c Add db info to settings

In this example we only had one commit in the branch, so we pointed to that. If you had more commits in your lost branch,
the process is exactly the same. Just point to the last commit in the lost branch, and you are good to go.

Accidentally committed to the wrong branch? Here is how to fix it.

foadlind — Wed, 02 Feb 2022 05:35:04 +0000

If you have made a commit to the wrong branch, there is nothing to worry about. You can easily move the commit to the
correct branch. Let me show you how with an example:

Let's say I made a commit to the main branch that was meant to be in the feature_1 branch (commit 55f0c29 shown below).

* 55f0c29 (HEAD -> main) Add settings flag for feature_1
* 4015b6f Provide default for product size
* d8dc31c Add db info to settings

We want to undo that last commit and keep the changes. To do that, we use the reset command with the --soft flag.

(main)$ git reset --soft HEAD^

HEAD^ means the commit before where HEAD is now. In our case 4015b6f. We use --soft so that Git preserves our
changes. If we look at Git status we will see that our change is still there:

(main)$ git status
On branch main
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    modified:   file.txt

Now we can check out the destination branch and make our commit there. In my case feature_1 branch didn't exist,
so I am creating it here with the -b flag:

(main)$ git checkout -b feature_1
Switched to a new branch 'feature_1'

(feature_1)$ git status
On branch feature_1
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    modified:   file.txt

(feature_1)$ git commit -m "Add settings flag for feature_1"
[feature_1 ef640e4] Add settings flag for feature_1
 1 file changed, 1 insertion(+)

If I look at my Git log I see that HEAD is pointing to the new commit in the new branch:

* ef640e4 (HEAD -> feature_1) Add settings flag for feature_1
* 4015b6f (main) Provide default for product size.
* d8dc31c Add db info to settings.

Undo a merge or a rebase

foadlind — Fri, 28 Jan 2022 07:32:46 +0000

Have you merged the wrong branch into your branch? or maybe you rebased your branch on the main branch but something
went wrong during the process? Don't worry! You can undo a merge/rebase process fairly easily.

Before doing a dangerous operation, like merging or rebasing, Git saves your previous HEAD location in a special
variable called ORIG_HEAD. This way you can always go back to where HEAD was before you started the operation.

Let's say I have a branch called feature_1. I have accidentally merged the wrong branch into it. Instead of merging
main, I merged develop to it (commit af735d2). I would like to undo this operation.

*   af735d2 (HEAD -> feature_1) Merge branch 'develop' into feature_1
|\
| * db7d7a6 (develop) add login button
| * 2f0ae72 (main) add test fixtures
* | 2d487f5 create user profile page
|/
* 2f591bd fix bug #1
* 290fc8b add db config file

Before I did the merge, Git saved the value of the previous HEAD location to the ORIG_HEAD variable. In this case,
2d487f5.
To remove the last commit I can instruct Git to reset to the location of ORIG_HEAD like this:

(feature_1)$ git reset --hard ORIG_HEAD

Now my commit tree is clean, and I am back to where I was before the wrong merge.
Notice that the merge commit is removed:

* db7d7a6 (develop) add login button
* 2f0ae72 (main) add test fixtures
| * 2d487f5 (HEAD -> feature_1) create user profile page
|/
* 2f591bd fix bug #1
* 290fc8b add db config file

You can use the same command for undoing a rebase.

Best place to put queries in a Django project? Model Managers!

foadlind — Tue, 13 Jul 2021 09:09:50 +0000

A question often arises when Django projects start to grow: "Where to put all the queries?"

When it comes to this question, a good concept to have in mind is fat models and thin views. This means that, as much as possible, you should try to keep business logic out of views and closer to your models.

According to Djagno documentation, you should:

Define custom methods on a model to add custom “row-level” functionality to your objects. Whereas Manager methods
are intended to do “table-wide” things, model methods should act on a particular model instance.
This is a valuable technique for keeping business logic in one place – the model.

Queries on a model can be considered "table-wide" logic, so a good place for them is in the model Manager methods.

Here is an example to illustrate this point:

Let's say you have a model called Membership in your project. A Membership is connected to a User and an Organization. Now, in one of your views you want to present all the memberships a user has. You could do it like this:

# models.py
class Membership(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE)

# views.py
def my_memberships(request):
    memberships = Membership.objects.filter(user=request.user)
    return render(request, 'myapp/my_memberships.html', {'memberships': memberships})

This is all good and dandy, but what if you want to make another view that needs to retrieve all the user's memberships? Then you would need to copy the memberships query to the new view. You would be duplicating code!

A better way is to move this query to the Membership model manager:

#models.py
class MembershipManager(models.Manager):
    def get_memberships_for_user(self, user):
        return self.filter(user=user)


class Membership(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE)

    objects = MembershipManager()

#views.py
def my_memberships(request):
    memberships = Membership.objects.get_memberships_for_user(request.user)
    return render(request, 'myapp/my_memberships.html', {'memberships': memberships})

This way, we can reuse the query logic in other places and avoid duplication. Views that use this method will be more readable and self-documenting. Another added benefit is that you can test this query separately without needing to test the entire view.

You can read more about model managers here.

All you need to know to survive in Git!

foadlind — Thu, 08 Jul 2021 08:12:51 +0000

* Photo by milan degraeve on Unsplash

Learning Git can be really overwhelming. There are a lot of new concepts to learn and there are a lot of commands that you need to memorize. Every time you think you know what you are doing, Git throws a new challenge at you and makes you humble.

Of course when you are a Git beginner this happens more often. It can be demotivating and frustrating.

I am here to tell you that you don't need to beat yourself up about it. Even seasoned Git users need to look up commands now and then.

In the beginning you just need to survive. You need to know just enough Git to get by. Just enough so that it gets out of your way, and lets you focus on other activities like coding, building, shipping, etc.

You need some basic Git survival skills!

So what are these Git survival skills?

To survive, there are a subset of Git concepts that you need to understand, and a handful of commands to remember:

Skill 1: Initializing and cloning repositories

Git repositories are not magic. They are just a folder of files with a .git folder inside. Go ahead and list all the files (and hidden files) in a Git repository. You will see a .git folder in there. In that folder, Git keeps a record of the commits that have been made, who made them, when, and so on.

When you initialize a repository using git init inside a folder, you create the .git folder there for the first time. From then on, Git will track everything that happens in your repository.

When you clone a repository, you are basically downloading a folder that includes the project files but also a .git folder. Git will set up things so that you can communicate with where you cloned from (for example GitHub) so that later you can send your changes to it (push) or update from it (pull). To clone a repository you need to provide the URL for that repository:

$ git clone [URL]

Skill 2: Checking the status of your local changes

You should be able to get an overview of your local changes in Git at any time. This is useful just before you commit or if you just want to remember what files you have changed. You can easily do that using the status command in your repository:

$ git status

On branch feature_1
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   file.txt

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    file2.txt

no changes added to commit (use "git add" and/or "git commit -a")

Here you can see that we are inside a branch called feature_1. We have modified a file called file.txt and there is a new file (file2.txt) that is not added to Git yet and therefore it is reported as "Untracked".

Now if you want to know what are the changes in file.txt, you can use the diff command like this:

$ git diff file.txt

diff --git a/file.txt b/file.txt
index be05e9c..163b853 100644
--- a/file.txt
+++ b/file.txt
@@ -1,3 +1,3 @@
 Something
 Something more
-
+Something else

Now that's a bit cryptic. I personally don't like it. To get an easy-to-understand diff, I suggest you use the Git integration in your favorite text editor or IDE. Here is an excellent article describing how to set up VSCode so that it opens from terminal and shows you a good-looking diff.

Skill 3: Staging and Committing

The difference between the two, and the fact that there are two steps to committing, is sometimes confusing for Git beginners. I wrote an article about what the differences are and how you can use staging and committing to your advantage. Give it a read.

Skill 4: Checking the history

Another thing you need is the ability to find things in the repository history. Git history is basically a log of all the things that has happened in the repository.

Most Git GUIs (and Git integrations in text editors and IDEs) offer a visual and intuitive presentation of this log in a form of a tree. You can see the branches, commits, tags, changed files, diffs, etc.

Although you can definitely investigate the Git history in a terminal using the commands below, a Git GUI offers a more
intuitive experience when it comes to investigating the history. I usually recommend learning Git in a terminal first before going to GUIs,
but from this particular aspect, GUIs are better.

If you choose to remain in the terminal, the git log command is your friend. Although the default output is not so helpful, you can create a good-looking visualization of the log using some additional command arguments. You can even create an alias, so you don't need to write those command arguments each time. Read this article to learn how to do it.

You can show the contents of any commit using the commit hash. That's the crazy (but unique) number in front of each commit.

$ git show <commit-hash>

Or you can jump to that commit by checking it out:

$ git checkout <commit-hash>

Don't get scared by the somewhat morbid message from git about a detached HEAD. You can switch back to where you were before, using git switch -.

Skill 5: Branching

Branches are one of the most useful concepts in Git. Let's say you want to experiment on your code, for example refactor something or introduce a new feature. You would typically do this in a branch. That way you don't touch the code that already works. When you are satisfied with your results, you can merge your experiment branch to the main branch.

You can create a new branch with:

$ git branch [branch-name]

You can move to another branch using:

$ git checkout [branch-name]

Skill 6: Merging and resolving merge conflicts

You also need to learn how to merge branches. Merging often seems scary to beginners because they are afraid they will write over other people's work or that they will get
a merge conflict.

To prevent writing over other people's work, always make sure that the branch you are merging into is updated. Check out the destination branch and do a pull before you merge:

$ git pull
Already up to date.

$ git merge [your-branch-name]

Sometimes you end up in a merge conflict state. It's just the reality of working with Git. There is no need to panic.
Just follow this guide, and you will become a Git conflict resolution guru.

Skill 7: Syncing with a remote

Sooner or later you will need to put your project on GitHub (or GitLab, or BitBucket, etc). Or you might need to collaborate with others on the same repository.
So you need to send your changes to a remote repository, and download changes from it. This is when push, pull and fetch come into the picture.

Push is pretty obvious. git push simply sends commits you have done locally in your current branch up to the remote repository. However, if this is the first time you are
pushing a new branch to remote, you need to use a slightly different command:

$ git push -u origin my_new_branch

Remember, this is done only if you have just created my_new_branch and this is the first time you are pushing it to the remote repository. Read more about this here.

Before we get to pull, we need to talk about fetch. git fetch asks the remote about the latest developments. It doesn't change your files. It just downloads the history from remote so that you can visualize what has happened on the remote since the last time you synced.

Pull is like fetch except that it will not only update history, it will also merge the changes from remote to your local files. If you have changed some file and
another colleague has changed the same file and pushed it, you will get a merge conflict on that file if you do a pull. This is not bad. It's just Git asking you for guidance on how to merge these changes. You can follow my merge conflict resolution guide to resolve it.

One important rule is to always do a pull before you push your changes. This way if there is a conflict, you get a chance to solve it locally and do not risk overwriting other people's work.

Beyond survival

Now you are equipped with all the survival skills you need to use Git on a day-to-day basis. My guess is that this list helps you in about 90% of your work with Git.
There are of course more things to learn. For example, when you are comfortable with these basic concepts, you might want to take a look at rebasing and cherry-picking.
You don't need them right now, but they might come in handy one day.

So to recap, you need to know how to:

initialize and clone a repository,
check the status of your work,
stage and commit your changes,
check the history of the repository,
make branches and jump between them,
merge branches and resolve merge conflicts,
sync with a remote using push and pull.

To help you remember all these commands, I have created a cheat sheet that you download. Print it out and put it somewhere that is easy to see.

Staging vs. Committing: an explanation for Git beginners

foadlind — Wed, 17 Mar 2021 12:00:24 +0000

I get these questions quite often:

"What is the difference between staging and committing?"

"Why is there a git add and a git commit? What's the difference?"

What is the difference?

Imagine you are doing online shopping. You browse through the pages of your favorite online store and add some products to your shopping cart. The shopping cart is like staging. Things are there ready to be paid for. You can keep adding stuff to it or remove some from it.

When you are done shopping, you go and pay for the things in your shopping cart. That can be compared to making a commit. Now there is a record somewhere of what you have bought and when. There is a history.

Notice that you can easily change your mind when things are in staging, but once you have made a commit things are hard to change (although not impossible!).

Why are they separated?

The fact that you need to stage and commit separately in Git has many benefits. Generally it is a good idea to make your commits small. And to make sure that all the changes that are included in one commit are related to each other.

For example a commit that fixes a bug in Feature A should not also include an improvement you made to Feature B. If you have been productive 😎 and have done those two things at once, you should not commit everything at once. You should separate them by staging the bug fixes and committing those first and then staging the improvements to Feature B and making a second commit.

That way, if someone looks at the Git history of your project, they are not confused by one giant commit that did a lot of things. They can see clearly that you performed two different tasks: one bug fix in Feature A and one improvement to Feature B.

How to stage and commit

Ok so now that you know the difference between staging and committing, let's see how to do these things in Git:

To add the changes you made to a file to staging, use this command

$ git add <path-to-file>

If you want to add all the files at once, put a dot instead of path to each file, like this:

$ git add .

All the changes will be staged, ready to be committed.

To commit use the command:

$ git commit -m "some descriptive message"

I have created a Git cheat sheet for beginners to make it easier to look up these basic commands. You can grab a free copy from here.

Git vs. GitHub: an explanation for absolute beginners

foadlind — Tue, 16 Mar 2021 09:21:58 +0000

This is a question a lot of Git beginners have:

"What is the difference between Git and GitHub?"

It can be confusing to understand their difference when you first encounter these new concepts. In this article I will try to give you some context for how Git and GitHub are related and what that means to you.

Git is the version control software you can use locally on your machine to keep track of changes you make to your projects. Git is an open-source project itself that was made by the team developing Linux operating system to keep track of the changes to their code. There are other version control software beside Git (Subversion, Mercurial, etc), but Git is by far the most common and most powerful.

GitHub is a web service that allows you to store your projects in the cloud using Git. You don't need GitHub to be able to use Git. But if you want to back up your projects in the cloud or share it with others, you can use GitHub.

GitHub is not the only option though. There are at least two other major ones: GitLab and Bitbucket.

Apart from storing your project files and your project's Git history, these services provide other tools that are useful for collaborating with others. Things like code review tools, project management tools, team communication tools, bug tracking, etc.

Sooner or later you will need to collaborate with others on a project. Usually you will do that by uploading your code (pushing it) to GitHub. Then others can download (clone) your code and start working on it. When they have made some changes and want to share it with you, they push it up to GitHub. Then you can pull the changes down to your local Git and see what they have done. This is the ground of all collaboration done on GitHub and is the central workflow for all Git projects.

In order for Git to send your project to GitHub, you need to give your project a remote address. You can go to GitHub and create a home for your project, called a repository. Then you give the address of that repository to Git. After that you are able to just push your files to its new remote home on GitHub.

You can also download other people's work from GitHub. This process is called cloneing. You clone a repository like this:

$ git clone <address-of-the-repository>

You can find the address of the repository from it's GitHub page. It usually looks something like this: git@github.com:username/some-cool-project-name.git

So in summary, Git is a version control system and GitHub is a web service that helps you collaborate with others using Git.
I hope this article helped to clear some of the confusion around Git and GitHub! 🙂
I have also created a Git cheat sheet for beginners. If you are interested, grab a free copy from here.

Authentication with Django REST Framework

foadlind — Fri, 26 Feb 2021 06:04:14 +0000

In this post I will go through some common authentication mechanisms that are used with Django REST Framework (DRF). Some of these methods are included in DRF, and the others are custom authentication packages that implement popular schemes not included in DRF. We will go through how each scheme works and when it should be used. I have also included some resources for learning how to implement each.

But before we dig in, we need to look at how authentication works in DRF.

Authentication in Django REST Framework

Authentication is a way for your service to determine whether the user is who they claim to be. When that is determined, your service can check whether they are allowed to access certain information.

In DRF, authentication is the first check that happens upon receiving a request. All other checks (like permissions, throttling, etc) are done afterwards. You can have several ways of authenticating users for the same API service, each used for a different type of client.
You can set your API's authentication schemes globally in your setting file:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.BasicAuthentication',
        'rest_framework.authentication.SessionAuthentication',
    ]
}

In this case DRF will first try the BasicAuthentication scheme for each request. If that fails, it will try the next authentication scheme in the list and so on.

You can also override these global settings for each view individually. For example this view will only allow requests that use the TokenAuthentication scheme:

class SomeSpecialView(APIView):
    authentication_classes = [TokenAuthentication]

If you do choose to have specific authentication schemes for specific views make sure you always have a global one set in your settings file to fall back on.

A tour of common authentication schemes

BasicAuthentication

This is the most ... uh ... basic authentication scheme you can have. Users identify themselves with a username and password. This is also the least secure scheme. Users will be sending their precious passwords over the wire. Only use it in testing. If you absolutely must use it, make sure your API is only available over HTTPS.

You can implement it by just importing it from DRF and adding it to to a specific view or to your settings.py as a default authentication class.

Users then send requests to the API and include HTTP Basic Authentication header:

GET https://example.com/api/books/123/
Authorization: Basic myusername mypasswd

TokenAuthentication

In this scheme you give each user a unique token and they include it in every request they make. This is appropriate for client-serer applications, for example for desktop or mobile apps that need to access your API. Note that anyone that gets hold of the token can use it and authenticate themselves as a user.

To implement TokenAuthentication you need to add rest_framework.authtoken to the list of INSTALLED_APPS in your settings.py file. This app takes care of creation, validation and management of user tokens in your database. Therefore you need to run migrations after you add it so that it can create token management tables in the database. Then you need to a way to create token for each user and give it to them. For example in your web app you can make a page where logged in users can see their token. You create tokens for your users like this:

# views.py
from rest_framework.authtoken.models import Token

@login_required
def user_token_view(request):
    token = Token.objects.create(user=request.user)
    return render(request, 'myapp/user_token.html', {'token': token.key})

You should provide a way for users to generate a new token (and therefore revoke the old one) in case they feel like the token has been compromised.

Your users then can send requests to your API and authenticate themselves by adding the token to the Authorization HTTP header:

GET https://example.com/api/books/123/
Authorization: Token 9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b

SessionAuthentication

This scheme is particularly good for Single Page Applications where the user logs in to your app, creates a session in Django and then the app sends AJAX requests to the server to fill the page with relevant information.
This scheme works based on Django's session framework so make sure it is included in INSTALLED_APPS in your settings.py files. Read more about Django sessions here.

Django has excellent documentation on how to securely make AJAX calls using Django sessions.

OAuth2

OAuth2 is a popular standard for authentication. It provides guidelines for user authentication as well as authenticating third-party apps on behalf of the user. Here and here you can read more about details of each authentication "flow" and their appropriate use cases.

You can find the technical standard specification here.

Thankfully you don't need to implement this giant standard yourself. You can use the excellent Django OAuth Toolkit package. They also have a specific section in their documentation for using it with DRF.

JSON Web Token (JWT) Authentication

This is a new and popular standard that works similar to TokenAuthentication except that it does not need to save tokens in the database. Your app generates JWTs that are signed using a secret. Then you give this token to the user to include in their request just as in TokenAuthentication.

Every time you receive the token, you decode it and check its signature. If it's valid you know that it has not been tampered with. The token is actually nothing but a json string that includes some claims. These claims can be the username, token expiry date, etc. You can read more about the details of the token here.

For JWT authentication you can use the Simple JWT package that implements it for DRF.

A word on Security

I hope this guide has given you a good overview of the most common authentication schemes to use with Django REST Framework. Be aware that no authentication scheme is perfect from the security perspective. Make sure you do extensive research before choosing an authentication scheme and always follow best practices provided by Django and DRF documentations. You are not only responsible for the security of your app, you are also responsible for the security of your users and their data.

Creating custom manage.py commands for Django

foadlind — Sat, 10 Oct 2020 17:40:27 +0000

For my recent project pyremote, I needed to run a script every night to remove old job postings from the database. I decided to implement it as a custom manage.py command so that I can easily run it from the command line and add it to a nightly cron job on the server.

To create a custom command you need to add a management directory to your Django app. In there create another directory called commands and in that one add a Python file and call it the way you want the command to be called. In my case I called it purge_old_jobs.py:

pyremote/
    __init__.py
    management/
        commands/
            purge_old_jobs.py
    models.py
    views.py
    ...

The Python file you create must define a class called Command that inherits from BaseCommand. Create a method named handle in your Command class. The actual logic of the command goes in this method.

You can also define add_argument method where you specify any command line arguments you would like to pass to the custom command when you invoke it. These arguments are then available to the handle method in a dictionary called options.

# purge_old_jobs.py
import datetime
from django.core.management.base import BaseCommand
from pyremote.models import Job


class Command(BaseCommand):
    help = 'Cleans database from job posts older than the given number of days.'

    def add_arguments(self, parser):
        parser.add_argument('days', type=int, help='Retention period in days.')

    def handle(self, *args, **options):
        retention_period = datetime.timedelta(days=options['days'])
        expiry_date = datetime.date.today() - retention_period
        number_deleted, * _ = Job.objects.filter(creation_date__lt=expiry_date).delete()
        self.stdout.write(f'Number of expired jobs: {number_deleted}')

Now I can run this command to clean the database from jobs that are added to the page more than 30 days ago:

$ python manage.py purge_old_jobs 30

Dockerizing a Django + MySQL project

foadlind — Mon, 05 Oct 2020 13:52:22 +0000

If you are here, I assume you have already embraced the merits of using Docker to set up your development environment such that it matches your staging and production environments.

In this guide I show you how to run your Django app in one Docker container and your MySQL database in another and make them talk to each other.

We are going to manage and run these two containers using Docker Compose.

👉 If you want to learn how to do this for a Django + Postgres app read this post.

I am assuming you already have a Django project locally that is using a local database and that your project has a requirements.txt file. You should also install Docker on your machine if you haven't already.

Let's dive right in:

The first step is to create a Dockerfile in the root of your project (this is where you have your requirements.txt and manage.py files).


 Dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED 1
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt
COPY . /app

Let's go through that line by line:

FROM python:3.8 is the base image your Docker image will be built upon. This will be pulled from DockerHub.
ENV PYTHONUNBUFFERED 1 ensures that the output Django writes to the terminal comes out in real time without being buffered somewhere. This makes your Docker logs useful and complete.
WORKDIR /app creates a new folder in your container called app which will be your project's root inside the container. It then sets that as the work directory in which the subsequent commands will be executed in.
COPY requirements.txt /app/requirements.txt copies your local requirements.txt file to the Docker container.
RUN pip install -r requirements.txt will make sure you have all your project dependencies installed inside the container.
COPY . /app and finally it's time to copy your project's content into the Docker container.

Now create a new file in your local project root. This one is called docker-compose.yml and is used for configuring Docker Compose.


 yml
version: '3'
services:
  db:
    image: mysql:8
    ports:
      - "3306:3306"
    environment:
      - MYSQL_DATABASE='mydatabase'
      - MYSQL_USER='root'
      - MYSQL_PASSWORD='some_password'
      - MYSQL_ROOT_PASSWORD='some_password'
      - MYSQL_HOST=''
    volumes:
      - /tmp/app/mysqld:/var/run/mysqld
      - ./db:/var/lib/mysql
  web:
    build: .
    command: python manage.py runserver 0.0.0.0:8000
    ports:
      - "8000:8000"
    volumes:
      - .:/app
      - /tmp/app/mysqld:/run/mysqld
    depends_on:
      - db

This defines two services: db and web.

db uses an official MySQL image from DockerHub. We attach local port 3306 to port 3306 of this container. We then set a bunch of environment variables that are required for MySQL. Make sure you also set these in your Django database settings. (You can put these in a .env file to avoid committing sensitive information to version control. Read here to learn how.)

volume define two volume mappings:

The first one maps the contents of var/run/mysqld of the container to a local folder on your machine. This file contains socket information that enables the web service (your Django app) to talk to the database service.
./db:/var/lib/mysql creates a local db folder in your project root so that database information can be saved if you destroy the db service.

web build its image using the Dockerfile we created earlier. It then runs the Django server and exposes port 8000 to your machine. We map your local project folder to the app folder and we bring in the socket file. We tell Docker Compose that this service is dependant on the db service.

Now all we have to do is to start everything up. In your project root run:


 bash
$ docker-compose up

This should build your images and start the containers. Open a browser and go to 0.0.0.0:8000. You should see your app. Now if you open a new terminal and run a docker ps you will see the web and db containers are running:


 bash
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                        NAMES
77a320233bf2        my_project_web      "python manage.py ru…"   4 minutes ago       Up 4 minutes        0.0.0.0:8000->8000/tcp       my_project_web_1
66324c20d91e        mysql:8             "docker-entrypoint.s…"   4 minutes ago       Up 4 minutes        0.0.0.0:3306->3306/tcp       my_project_db_1

If you want to stop Docker Compose, press Ctrl+C in the first terminal or run docker-compose stop in the project root folder in another terminal.

To run manage.py commands in the Django container (web):


 bash
$ docker-compose run --rm web python manage.py migrate

Validating Django models that are created without forms

foadlind — Thu, 28 May 2020 17:33:33 +0000

Usually instances of models (objects) are created using a ModelForm. You know the drill: The user inputs some values in the form, hits submit and sends a POST request with the data. In the view function/class, we do a model validation using form.is_valid() and then save the object in the database. This gives a basic level of validation for the model fields.

But what if we want to do a more comprehensive validation on the object? And what about a scenario where we want to create objects in some other way than using forms? Let's say through an API call. How do we validate the objects in that case?

That is what validators are for. Validators are callable functions that will check the object fields and raise ValidationError if they contain values that don't meet our desired criteria. In case of ModelForm the validation is done when we call form.is_valid(). Note that if we create an instance of the model ourselves, validators will not be run automatically. So we need to validate the objects first before saving them.

Let's say we have a model called Person. Each person has a first name, a last name and an age. And for some reason, we don't want to save the person in our database if their age is less than 18. We can create a validator that checks for that.

In models.py:

from django.db import models
from django.core.exceptions import ValidationError

def validate_age(value):
    if value < 18:
        raise ValidationError(
            ('Age must be at least 18.'),
            params={'value': value},
        )

class Person(models.Model):
    first_name = models.CharField()
    last_name = models.CharField()
    age = models.IntegerField(validators=[validate_age])

Now just before we save the object, we ask django to validate it by calling the full_clean() method.

In views.py:

from myapp.models import Person
from django.core.exceptions import ValidationError

# let's create an object
person = Person()
person.first_name = "John"
person.last_name  = "Doe"
person.age = 17

try:
    person.full_clean()
    person.save()

except ValidationError:
    # Do something with the error here.
    # Maybe display to user or log it somewhere.

person.full_clean() will run the relevant validators for all fields of the person object. In this case it will run the validate_age function on the person.age field and throws an exception. This prevents this object from being saved. So young John will not be saved into the database! 🙂

Need to stash untracked files? There is a way!

foadlind — Mon, 23 Mar 2020 14:29:07 +0000

You are in the middle of some work. You have created a bunch of new files and edited some existing ones when your boss comes and tells you to deal with an emergency bug fix. You realize you need to switch branches but your work is incomplete and you don't want to commit it just yet.

Stashing your changes is the best way to go. However, the default behavior of git stash is to stash only the tracked files. The untracked files will remain in your repo. Sometimes that is ok. You can have them there while you switch branches and deal with the emergency. But, in some occasions, their existence might interfere with other things (like changing the behavior of the software, or getting accidentally added to a commit).

I prefer to stash the untracked files every time I need to switch branches. That way I get a clean repo to work with. Less thinking required, and less room for error. It's the safest option.

So how do you stash both tracked AND untracked files?

Using:

$ git stash --include-untracked
# or the shorter alternative:
$ git stash -u

"Will this include the files in my .gitignore?"

No. This will only include the untracked files you see when you do git status. If you want to stash files included in .gitignore, use git stash --all.

"How do I see what I have just stashed?"

To see all tracked files you just stashed use git show stash. To see all the untracked files you just stashed use git show stash@{0}^3. Unfortunately there is no way, that I know of, to see both the tracked and untracked stashed files using one command. This is due to the way Git stores and applies stashed changes.