DEV Community: Endalkachew Biruk

JavaScript Promises and Async Await

Endalkachew Biruk — Sat, 16 Jul 2022 14:42:27 +0000

In the previous article we have seen what JavaScript callbacks are and what potential issues they have. In this article
we are going to explore JavaScript Promises. How they work, potential issues with promises and, how async and await solves these issues. If you haven't read the previous article I recommend you do. JavaScript Callbacks Article

Promises

Have you ever been to a busy restaurant without a reservation? When this happens, the restaurant needs a way to get back in
contact with you when a table opens up. Historically, they'd just take your name and yell it when your table was ready. Then,
as naturally occurs, they decided to start getting fancy. One solution was, instead of taking your name, they'd take your
number and text you once a table opened up. This allowed you to be out of yelling range but more importantly, it allowed them
to target your phone with ads whenever they wanted. Sound familiar? It should! OK, maybe it shouldn't. It's a metaphor for
callbacks! Giving your number to a restaurant is just like giving a callback function to a third party service. You expect the restaurant to text you when a table opens up, just like you expect the third party service to invoke your function when and how they said they would.
Once your number or callback function is in their hands though, you've lost all control.

Thankfully, there is another solution that exists. One that, by design, allows you to keep all the control. You've
probably even experienced it before - it's that little buzzer thing they give you. You know, this one.

If you've never used one before, the idea is simple. Instead of taking your name or number, they give you this
device. When the device starts buzzing and glowing, your table is ready. You can still do whatever you'd like as
you're waiting for your table to open up, but now you don't have to give up anything. In fact, it's the
exact opposite. They have to give you something. There is no inversion of control.

The buzzer will always be in one of three different states - pending, fulfilled, or rejected.

pending is the default, initial state. When they give you the buzzer, it's in this state.

fulfilled is the state the buzzer is in when it's flashing and your table is ready.

rejected is the state the buzzer is in when something goes wrong. Maybe the restaurant is about to
close or they forgot someone rented out the restaurant for the night.

Again, the important thing to remember is that you, the receiver of the buzzer, have all the control. If the
buzzer gets put into fulfilled, you can go to your table. If it gets put into fulfilled and you
want to ignore it, cool, you can do that too. If it gets put into rejected, that sucks but you can go
somewhere else to eat. If nothing ever happens and it stays in pending, you never get to eat but you're
not actually out anything.

Now that you're a master of the restaurant buzzer thingy, let's apply that knowledge to something that matters.

If giving the restaurant your number is like giving them a callback function, receiving the little buzzy thing is like receiving what's called a "Promise".

As always, let's start with why. Why do Promises exist? They exist to make the complexity of making
asynchronous requests more manageable. Exactly like the buzzer, a Promise can be in one of three
states, pending, fulfilled or rejected. Unlike the buzzer, instead of these states representing
the status of a table at a restaurant, they represent the status of an asynchronous request.

If the async request is still ongoing, the Promise will have a status of pending. If the async request
was successfully completed, the Promise will change to a status of fulfilled. If the async request
failed, the Promise will change to a status of rejected. The buzzer metaphor is pretty spot on, right?

Now that you understand why Promises exist and the different states they can be in, there are three more questions
we need to answer.

How do you create a Promise?
How do you change the status of a promise?
How do you listen for when the status of a promise changes?

1) How do you create a Promise?

This one is pretty straight forward. You create a new instance of Promise.

const promise = new Promise()

2) How do you change the status of a promise?

The Promise constructor function takes in a single argument, a (callback) function. This function is going
to be passed two arguments, resolve and reject.

resolve - a function that allows you to change the status of the promise to fulfilled

reject - a function that allows you to change the status of the promise to rejected.

In the code below, we use setTimeout to wait 2 seconds and then invoke resolve. This will change the
status of the promise to fulfilled.

const promise = new Promise((resolve, reject) => {
    setTimeout(() => {
        resolve() // Change status to 'fulfilled'
    }, 2000)
})

We can see this change in action by logging the promise right after we create it and then again roughly
2 seconds later after resolve has been called.

Notice the promise goes from <pending> to <resolved>.

3) How do you listen for when the status of a promise changes?

In my opinion, this is the most important question. It's cool we know how to create a promise and change its
status, but that's worthless if we don't know how to do anything after the status changes.

One thing we haven't talked about yet is what a promise actually is. When you create a new Promise, you're
really just creating a plain old JavaScript object. This object can invoke two methods, then, and catch.
Here's the key. When the status of the promise changes to fulfilled, the function that was passed to .then will
get invoked. When the status of a promise changes to rejected, the function that was passed to .catch will be
invoked. What this means is that once you create a promise, you'll pass the function you want to run if the async request
is successful to .then. You'll pass the function you want to run if the async request fails to .catch.

Let's take a look at an example. We'll use setTimeout again to change the status of the promise to fulfilled after
two seconds (2000 milliseconds).

function onSuccess() {
    console.log("Success!")
}

function onError() {
    console.log("💩")
}

const promise = new Promise((resolve, reject) => {
    setTimeout(() => {
        resolve()
    }, 2000)
})

promise.then(onSuccess)
promise.catch(onError)

If you run the code above you'll notice that roughly 2 seconds later, you'll see "Success!" in the console. Again
the reason this happens is because of two things. First, when we created the promise, we invoked resolve after
~2000 milliseconds - this changed the status of the promise to fulfilled. Second, we passed the onSuccess function
to the promises' .then method. By doing that we told the promise to invoke onSuccess when the status of the
promise changed to fulfilled which it did after ~2000 milliseconds.

Now let's pretend something bad happened and we wanted to change the status of the promise to rejected.
Instead of calling resolve, we would call reject.

function onSuccess() {
    console.log("Success!")
}

function onError() {
    console.log("💩")
}

const promise = new Promise((resolve, reject) => {
    setTimeout(() => {
        reject()
    }, 2000)
})

promise.then(onSuccess)
promise.catch(onError)

Now this time instead of the onSuccess function being invoked, the onError function will be invoked since we called reject.

Now that you know your way around the Promise API, let's start looking at some real code.

Remember the last async callback example we saw earlier?

function getUser(id, onSuccess, onFailure) {
    $.getJSON({
        url: `https://api.github.com/users/${id}`,
        success: onSuccess,
        error: onFailure,
    })
}

function getWeather(user, onSuccess, onFailure) {
    $.getJSON({
        url: getLocationURL(user.location.split(",")),
        success: onSuccess,
        error: onFailure,
    })
}

$("#btn").on("click", () => {
    getUser(
        "endalk200",
        (user) => {
            getWeather(
                user,
                (weather) => {
                    updateUI({ user, weather: weather.query.results })
                },
                showError
            )
        },
        showError
    )
})

Is there any way we could use the Promise API here instead of using callbacks? What if we wrap our AJAX
requests inside of a promise? Then we can simply resolve or reject depending on how the request goes.
Let's start with getUser.

function getUser(id) {
    return new Promise((resolve, reject) => {
        $.getJSON({
            url: `https://api.github.com/users/${id}`,
            success: resolve,
            error: reject,
        })
    })
}

Nice. Notice that the parameters of getUser have changed. Instead of receiving id, onSuccess, a
nd onFailure, it just receives id. There's no more need for those other two callback functions because we're no
longer inverting control. Instead, we use the Promise's resolve and reject functions. resolve will be invoked
if the request was successful, reject will be invoked if there was an error.

Next, let's refactor getWeather. We'll follow the same strategy here. Instead of taking in onSuccess and onFailure callback
functions, we'll use resolve and reject.

function getWeather(user) {
    return new Promise((resolve, reject) => {
        $.getJSON({
            url: getLocationURL(user.location.split(",")),
            success: resolve,
            error: reject,
        })
    })
}

Looking good. Now the last thing we need to update is our click handler. Remember, here's the flow we want to take.

Get the user's information from the Github API.
Use the user's location to get their weather from the Yahoo Weather API.
Update the UI with the user's info and their weather.

Let's start with #1 - getting the user's information from the Github API.

$("#btn").on("click", () => {
    const userPromise = getUser("endalk200")
    userPromise.then((user) => {})
    userPromise.catch(showError)
})

Notice that now instead of getUser taking in two callback functions, it returns us a promise that we can call .then and .catch on.
If .then is called, it'll be called with the user's information. If .catch is called, it'll be called with the error.

Next, let's do #2 - Use the user's location to get their weather.

$("#btn").on("click", () => {
    const userPromise = getUser("endalk200")
    userPromise.then((user) => {
        const weatherPromise = getWeather(user)
        weatherPromise.then((weather) => {})
        weatherPromise.catch(showError)
    })
    userPromise.catch(showError)
})

Notice we follow the exact same pattern we did in #1 but now we invoke getWeather passing it the user object we
got from userPromise.

Finally, #3 - Update the UI with the user's info and their weather.

$("#btn").on("click", () => {
    const userPromise = getUser("endalk200")
    userPromise.then((user) => {
        const weatherPromise = getWeather(user)
        weatherPromise.then((weather) => {
            updateUI({ user, weather: weather.query.results })
        })
        weatherPromise.catch(showError)
    })
    userPromise.catch(showError)
})

Our new code is better, but there are still some improvements we can make. Before we can make those improvements though, there are two
more features of promises you need to be aware of, chaining and passing arguments from resolve to then.

Chaining

Both .then and .catch will return a new promise. That seems like a small detail but it's important because
it means that promises can be chained.

In the example below, we call getPromise which returns us a promise that will resolve in at least 2000 milliseconds.
From there, because .then will return a promise, we can continue to chain our .thens together until we
throw a new Error which is caught by the .catch method.

function getPromise() {
    return new Promise((resolve) => {
        setTimeout(resolve, 2000);
    });
}

function logA() {
    console.log("A");
}

function logB() {
    console.log("B");
}

function logCAndThrow() {
    console.log("C");
    throw new Error();
}

function catchError() {
    console.log("Error!");
}

getPromise()
    .then(logA); // A
    .then(logB) // B
    .then(logCAndThrow) // C
    .catch(catchError); // Error!

Cool, but why is this so important? Remember back in the callback section we talked about one of the downfalls of callbacks
being that they force you out of your natural, sequential way of thinking. When you chain promises together, it doesn't force
you out of that natural way of thinking because chained promises are sequential. getPromise runs then logA runs then logB runs then....

Just so you can see one more example, here's a common use case when you use the fetch API. fetch will return you
a promise that will resolve with the HTTP response. To get the actual JSON, you'll need to call .json. Because of chaining,
we can think about this in a sequential manner.

fetch("/api/user.json")
    .then((response) => response.json())
    .then((user) => {
        // user is now ready to go.
    })

Now that we know about chaining, let's refactor our getUser/getWeather code from earlier to use it.

function getUser(id) {
    return new Promise((resolve, reject) => {
        $.getJSON({
            url: `https://api.github.com/users/${id}`,
            success: resolve,
            error: reject,
        })
    })
}

function getWeather(user) {
    return new Promise((resolve, reject) => {
        $.getJSON({
            url: getLocationURL(user.location.split(",")),
            success: resolve,
            error: reject,
        })
    })
}

$("#btn").on("click", () => {
    getUser("endalk200")
        .then(getWeather)
        .then((weather) => {
            // We need both the user and the weather here.
            // Right now we just have the weather
            updateUI() // ????
        })
        .catch(showError)
})

It looks much better, but now we're running into an issue. Can you spot it? In the second .then we want to call updateUI.
The problem is we need to pass updateUI both the user and the weather. Currently, how we have it set up,
we're only receiving the weather, not the user. Somehow we need to figure out a way to make it so the promise
that getWeather returns is resolved with both the user and the weather.

Here's the key. resolve is just a function. Any arguments you pass to it will be passed along to the function given to .then.
What that means is that inside of getWeather, if we invoke resolve ourself, we can pass to it weather and user.
Then, the second .then method in our chain will receive both user and weather as an argument.

function getWeather(user) {
    return new Promise((resolve, reject) => {
        $.getJSON({
            url: getLocationURL(user.location.split(",")),
            success(weather) {
                resolve({ user, weather: weather.query.results })
            },
            error: reject,
        })
    })
}

$("#btn").on("click", () => {
    getUser("endalk200")
        .then(getWeather)
        .then((data) => {
            // Now, data is an object with a
            // "weather" property and a "user" property.
            updateUI(data)
        })
        .catch(showError)
})

You can play around with the final code here

It's in our click handler where you really see the power of promises shine compared to callbacks.

// Callbacks 🚫
getUser(
    "endalk200",
    (user) => {
        getWeather(
            user,
            (weather) => {
                updateUI({ user, weather: weather.query.results })
            },
            showError
        )
    },
    showError
)

// Promises ✅

getUser("endalk200")
    .then(getWeather)
    .then((data) => updateUI(data))
    .catch(showError)

Following that logic feels natural because it's how we're used to thinking, sequentially. getUser then getWeather then update the UI with the data.

Now it's clear that promises drastically increase the readability of our asynchronous code, but is there a way
we can make it even better? Assume that you were on the TC39 committee and you had all the power to add new features to the
JavaScript language. What steps, if any, would you take to improve this code?

$("#btn").on("click", () => {
    getUser("endalk200")
        .then(getWeather)
        .then((data) => updateUI(data))
        .catch(showError)
})

As we've discussed, the code reads pretty nicely. Just as our brains work, it's in a sequential order. One issue that we did run
into was that we needed to thread the data (users) from the first async request all the way through to the last .then.
This wasn't a big deal, but it made us change up our getWeather function to also pass along users. What if we just
wrote our asynchronous code the same way which we write our synchronous code? If we did, that problem would go away entirely and it
would still read sequentially. Here's an idea.

$("#btn").on("click", () => {
    const user = getUser("endalk200")
    const weather = getWeather(user)
    updateUI({ user, weather })
})

Well, that would be nice. Our asynchronous code looks exactly like our synchronous code. There's no extra steps our brain needs
to take because we're already very familiar with this way of thinking. Sadly, this obviously won't work. As you know, if we
were to run the code above, user and weather would both just be promises since that's what getUser and getWeather
return. But remember, we're on TC39. We have all the power to add any feature to the language we want. As is, this code would be really
tricky to make work. We'd have to somehow teach the JavaScript engine to know the difference between asynchronous function invocations
and regular, synchronous function invocations on the fly. Let's add a few keywords to our code to make it easier on the engine.

First, let's add a keyword to the main function itself. This could clue the engine to the fact that inside of this function, we're
going to have some asynchronous function invocations. Let's use async for this.

$("#btn").on("click", async () => {
    const user = getUser("endalk200")
    const weather = getWeather(user)
    updateUI({ user, weather })
})

Cool. That seems reasonable. Next let's add another keyword to let the engine know exactly when a function being invoked is
asynchronous and is going to return a promise. Let's use await. As in, "Hey engine. This function is asynchronous
and returns a promise. Instead of continuing on like you typically do, go ahead and 'await' the eventual value of the
promise and return it before continuing". With both of our new async and await keywords in play, our new code
will look like this.

$("#btn").on("click", async () => {
    const user = await getUser("endalk200")
    const weather = await getWeather(user.location)
    updateUI({ user, weather })
})

Pretty slick. We've invented a reasonable way to have our asynchronous code look and behave as if it were synchronous.
Now the next step is to actually convince someone on TC39 that this is a good idea. Lucky for us, as you probably guessed
by now, we don't need to do any convincing because this feature is already part of JavaScript and it's called Async/Await.

async functions return a promise

Now that you've seen the benefit of Async/Await, let's discuss some smaller details that are important to know. First, anytime you add async to a function, that function is going to implicitly return a promise.

async function getPromise() {}

const promise = getPromise()

Even though getPromise is literally empty, it'll still return a promise since it was an async function.

If the async function returns a value, that value will also get wrapped in a promise. That means you'll have
to use .then to access it.

async function add(x, y) {
    return x + y
}

add(2, 3).then((result) => {
    console.log(result) // 5
})

await without async is bad

If you try to use the await keyword inside of a function that isn't async, you'll get an error.

$("#btn").on("click", () => {
    const user = await getUser("endalk200"); // SyntaxError: await is a reserved word
    const weather = await getWeather(user.location); // SyntaxError: await is a reserved word
    updateUI({ user, weather });
});

Here's how I think about it. When you add async to a function it does two things. It makes it so the
function itself returns (or wraps what gets returned in) a promise and makes it so you can use await inside of it.

Error Handling

You may have noticed we cheated a little bit. In our original code we had a way to catch any errors using .catch.
When we switched to Async/Await, we removed that code. With Async/Await, the most common approach is to wrap your code
in a try/catch block to be able to catch the error.

$("#btn").on("click", async () => {
    try {
        const user = await getUser("endalk200")
        const weather = await getWeather(user.location)
        updateUI({ user, weather })
    } catch (e) {
        showError(e)
    }
})

That concludes our two part series on JavaScript callbacks and, async and await. If you have questions or feedback I would appreciate it if you leave a comment below.

Mastering JavaScript Callbacks

Endalkachew Biruk — Tue, 29 Mar 2022 15:22:27 +0000

Asynchronous processing is one of the most important aspects of the JavaScript language. JavaScript is a single-threaded language, so it needs a mechanism to deal with the actions like performing time-intensive and potentially unreliable activities like making REST API calls, reading files from the disk, or interacting with user input.

In JavaScript the mechanism used is an event loop, which processes a callback queue in order from oldest item to newest. Each item in the queue is a message that’s processed completely before moving on to the next message. Each message is a function that returns a response to code elsewhere in the application. Functions used in this way are called callback functions.

Because of callbacks, JavaScript can use a number of techniques for getting around its single-threaded nature to implement asynchronous processing. In this and the next article we will try to explain these concepts, So buckle up it's gonna be a bumpy ride 😁.

In this one we are going to see JavaScript callbacks. We are going to explore what they are, why and where we use them. We will also see the potential issues with Callbacks and in the next article we are going to explore what promises are, the issues associated with using them and how async await solves those issues.

Callbacks

I'm going to assume you know exactly 0 about callbacks. If I'm assuming wrong, just scroll down a bit.

When I was first learning JavaScript, it helped me to think about functions as machines. These machines can do anything you want them to. They can even accept input and return a value. Each machine has a button on it that you can press when you want the machine to run.

function add(x, y) {
    return x + y;
}

add(2, 3); // Press the button, run the machine.

Whether I press the button, you press the button, or someone else presses the button it doesn't matter. Whenever the button is pressed, like it or not, the machine is going to run.

function add(x, y) {
    return x + y;
}

const me = add;
const you = add;
const someoneElse = add;

me(2, 3); // Press the button, run the machine.
you(2, 3); // Press the button, run the machine.
someoneElse(2, 3); // Press the button, run the machine.

In the code above we assign the add function to three different variables, me, you, and someoneElse. It's important to note that the original add and each of the variables we created are pointing to the same spot in memory. They're literally the exact same thing under different names. So when we invoke me, you, or someoneElse, it's as if we're invoking add. Now, what if we take our add machine and pass it to another machine? Remember, it doesn't matter who presses the () button, if it's pressed, it's going to run.

function add(x, y) {
    return x + y;
}

function addFive(x, addReference) {
    return addReference(x, 5); // 15 - Press the button, run the machine.
}

addFive(10, add); // 15

Your brain might have got a little weird on this one, nothing new is going on here though. Instead of "pressing the button" on add, we pass add as an argument to addFive, rename it addReference, and then we "press the button" or invoke it.

This highlights some important concepts of the JavaScript language. First, just as you can pass a string or a number as an argument to a function, so too can you pass a reference to a function as an argument. When you do this the function you're passing as an argument is called a callback function and the function you're passing the callback function to is called a higher order function.

Because vocabulary is important, here's the same code with the variables re-named to match the concepts they're demonstrating.

function add(x, y) {
    return x + y;
}

function higherOrderFunction(x, callback) {
    return callback(x, 5);
}

higherOrderFunction(10, add);

This pattern should look familiar, it's everywhere. If you've ever used any of the JavaScript Array methods, you've used a callback. If you've ever used lodash, you've used a callback. If you've ever used jQuery, you've used a callback.

[1, 2, 3].map((i) => i + 5);

_.filter([1, 2, 3, 4], (n) => n % 2 === 0);

$("#btn").on("click", () => console.log("Callbacks are everywhere"));

In general, there are two popular use cases for callbacks. The first, and what we see in the .map and _.filter
examples, is a nice abstraction over transforming one value into another. We say "Hey, here's an array and a function. Go ahead and get me a new value based on the function I gave you". The second, and what we see in the jQuery example, is delaying execution of a function until a particular time. "Hey, here's this function. Go ahead and invoke it whenever the element with an id of btn is clicked." It's this second use case that we're going to focus on, "delaying execution of a function until a particular time".

Right now we've only looked at examples that are synchronous. As we talked about at the beginning of this post, most of the apps we build don't have all the data they need up front. Instead, they need to fetch external data as the user interacts with the app. We've just seen how callbacks can be a great use case
for this because, again, they allow you to "delay execution of a function until a particular time". It doesn't take much imagination to see how we can adapt that sentence to work with data fetching. Instead of delaying execution of a function until a particular time, we can delay execution of a function until we have the data we need. Here's probably
the most popular example of this, jQuery's getJSON method.

// updateUI and showError are irrelevant.
// Pretend they do what they sound like.
const id = "endalk200";

$.getJSON({
    url: `https://api.github.com/users/${id}`,
    success: updateUI,
    error: showError
});

We can't update the UI of our app until we have the user's data. So what do we do? We say, "Hey, here's an object. If
the request succeeds, go ahead and call success passing it the user's data. If it doesn't, go ahead and call error
passing it the error object. You don't need to worry about what each method does, just be sure to call them when you're
supposed to". This is a perfect demonstration of using a callback for async requests.

At this point, we've learned about what callbacks are and how they can be beneficial both in synchronous and asynchronous code. What we haven't talked yet is the dark side of callbacks. Take a look at this code below. Can you tell what's happening?

// updateUI, showError, and getLocationURL are irrelevant.
// Pretend they do what they sound like.
const id = "endalk200";

$("#btn").on("click", () => {
    $.getJSON({
        url: `https://api.github.com/users/${id}`,
        success: (user) => {
            $.getJSON({
                url: getLocationURL(user.location.split(",")),
                success(weather) {
                    updateUI({ user, weather: weather.query.results });
                },
                error: showError
            });
        },
        error: showError
    });
});

Notice we've added a few more layers of callbacks. First, we're saying don't run the initial AJAX request until the element with an id of btn is clicked. Once the button is clicked, we make the first request. If that request succeeds, we make a second request. If that request succeeds, we invoke the updateUI method passing it the data we got from both requests. Regardless of if you understood the code at first glance or not, objectively it's much harder to read than the code before. This brings us to the topic of "Callback Hell".

As humans, we naturally think sequentially. When you have nested callbacks inside of nested callbacks, it forces you out of your natural way of thinking. Bugs happen when there's a disconnect between how your software is read and how you naturally think.

Like most solutions to software problems, a commonly prescribed approach for making "Callback Hell" easier to consume is to modularize your code.

const getUser = (id, onSuccess, onFailure) => {
    $.getJSON({
        url: `https://api.github.com/users/${id}`,
        success: onSuccess,
        error: onFailure
    });
};

const getWeather = (user, onSuccess, onFailure) => {
    $.getJSON({
        url: getLocationURL(user.location.split(",")),
        success: onSuccess,
        error: onFailure
    });
};

$("#btn").on("click", () => {
    getUser(
        "endalk200",
        (user) => {
            getWeather(
                user,
                (weather) => {
                    updateUI({ user, weather: weather.query.results });
                },
                showError
            );
        },
        showError
    );
});

OK, the function names help us understand what's going on, but is it objectively "better"? Not by much. We've put a band-aid over the readability issue of Callback Hell. The problem still exists that we naturally think sequentially and, even with the extra functions, nested callbacks break us out of that sequential way of thinking.

The next issue of callbacks has to do with inversion of control. When you write a callback, you're assuming that the program you're giving the callback to is responsible and will call it when (and only when) it's supposed to. You're essentially inverting the control of your program over to another
program. When you're dealing with libraries like jQuery, lodash, or even vanilla JavaScript, it's safe to assume that the callback function will be invoked at the correct time with the correct arguments. However, for many third-party libraries, callback functions are the interface for how you interact with them. It's entirely plausible that a third party library could,
whether on purpose or accidentally, break how they interact with your callback.

const criticalFunction = () => {
    // It's critical that this function
    // gets called and with the correct arguments.
};

thirdPartyLib(criticalFunction);

Since you're not the one calling criticalFunction, you have 0 control over when and with what argument it's invoked. Most of the time this isn't an issue, but when it is, it's a big one.

In the next article we are going to explore JavaScript promises and how they can provide a potential solution to the
inversion of control problem.

You can find the next article on JavaScript promises and, async and await here

Production-Ready Docker Configuration With DigitalOcean Container Registry Part II

Endalkachew Biruk — Wed, 03 Nov 2021 11:54:24 +0000

Prerequisite- If you didn’t read part I of this article I suggest you read that first here. I am assuming you already have a working DigitalOcean account. If you don’t have an account create an account using this link and get 100 USD free credit.

Objective- In part I of this article series we discussed the two dockerfiles powering the web server and our database. We also discussed some configuration files and scripts needed to run and operate our containers. In this article, we are going to build our docker image, push it to DigitalOcean container registry and deploy it to DigitalOcean App platform. We will also setup CI/CD for the build and deploy process using GitHub actions.

To get full code for this blog clone the repository.

git clone https://github.com/endalk200/prod-django-docker-config.git

or download the zip file here.

We will do this step by step.

Step 1- Install and configure `doctl`

doctl is CLI for interacting with DigitalOcean APIs.

To install this CLI refer to this official documentation. It has instructions for each platform and package managers. After you install the CLI you have to authenticate the CLI to interact with DigitalOcean APIs. To do that you have to generate AuthToken first. Go to your DigitalOcean dashboard, and click the API then Generate New Token Button

Give your token a name and generate the token. You will see the token generated in a table (Notice: This token is only visible to you once so copy the token and store it in a secure location). Now that you have you access token lets authenticated the CLI from your terminal run

doctl auth init

After that, you will be prompted to enter the access token, Paste it there and click enter. After that, you should see a message like this.

Validating token: OK

Now you have installed and configured your CLI to talk to DigitalOcean APIs you are good to go.

Step 2 — Build and push your image to container registry

Before we build our docker image we need to create container registry on DigitalOcean. Go to the **Container Registry **and you will be prompted with the following.

Enter the name (the name should be unique) of the container registry you want to create, chose a plan that suits your needs, and **Create Registry. **After creating your registry you will be redirected to your registry page like this.

Copy the registry URL (we are going to need that URL when we push to this registry).

By default, docker pulls and pushes Docker images from docker hub. We want docker to point to the container registry we just created. We can do that by running the following command.

doctl registry login

This command will tell docker to point to DigitalOcean for container registry.

To build our docker image run the following command in the root directory of the project.

docker build -t my-image-name:latest

In this command, we are tagging our image with the name my-image-name After that, we are telling Docker to tag the version of this docker image latest

The command will build the docker image and store it locally. What we want is to push the image to our container registry. To do that run

docker tag my-image-name:latest registry.digitalocean.com/my-registry1r43d/my-image-name:latest

This command is tagging our docker image we just build and pointing it to the container registry we created above. To push our container use the following command.

docker push registry.digitalocean.com/my-registry1r43d/my-image-name:latest

Step 3 —Setup CI/CD for the build process using GitHub actions

Now that we built our Docker image and push it to DigitalOcean container registry let's configure CI/CD for this process.

Before we setup CI/CD with GitHub we need to create a repository secret in our GitHub repository settings tab. Name the secret CONTAINER_REGISTRY_TOKEN and the value of the token we created above. After that create .github/workflows directory in you root project and create a file with the name of docker-image-build-push.yml and put the following yml config in it.

name: Build and push docker imageon:  

push:    
    branches:      
        - release

jobs:  
    build:    
        runs-on: ubuntu-latest    
        steps:      
            - name: Checkout master        
              uses: actions/checkout@main      
            - name: Install doctl        
              uses: digitalocean/action-doctl@v2        
              with:          
                  token: ${{ secrets.CONTAINER_REGISTRY_TOKEN }}
            - name: Build container image        
              run: docker build -t my-image-name:latest
            - name: Log in to DigitalOcean Container Registry    
              run: doctl registry login --expiry-seconds 1200
            - name: Tag the container being built
              run: docker tag my-image-name:latest registry.digitalocean.com/my-registry1r43d/my-image-name:latest
            - name: Push image to DigitalOcean Container Registry
              run: docker push registry.digitalocean.com/my-registry1r43d/my-image-name:latest

Step 4 — Deploy your container to app platform

App platform is DigitalOceans PAAS offering. We can deploy to this platform from our terminal using the CLI we installed. First, create yml file with the following config and name it spec.yml

name: my-project
services:
- name: my-project-image
  image:
    registry_type: "DOCR"
    registry: "my-registry1r43d"
    repository: "my-image-name"
    tag: "latest"
  run_command: start.sh server --worker-tmp-dir /dev/shm
  envs:
  - key: DEBUG
    scope: RUN_AND_BUILD_TIME
    value: "False"
  http_port: 8080
  instance_count: 1
  instance_size_slug: basic-xs
  routes:
  - path: /
  source_dir: /

The above config file tells DigitalOcean how to create your instance and run it.

To create the instance run the following command.

doctl apps create --spec spec.yml

After that command is executed your web app will be live with HTTPS and you can see the process in your dashboard.

Conclusion

This two-part article attempted to cover production level docker configuration and how you can build and deploy the image to DigitalOcean container registry. The deployment process guide is only meant to get you started there are many considerations when deploying your web application.

To get the full code and try it for yourself, It is on GitHub here.

Please comment on any gaps or improvements in the above setup. Follow me for more articles like this one.

Use the following link and get 100 USD in DigitalOcean free credit.

Follow me to get articles like this and more.

Production-Ready Docker Configuration With DigitalOcean Container Registry Part I

Endalkachew Biruk — Wed, 03 Nov 2021 11:53:13 +0000

Prerequisites-This article assumes a basic understanding of Docker and Django.

Objective

Part I of this article series will be setting up a basic Django web app with development level docker configuration using docker-compose and production level docker configuration. We will also discuss in detail the rationale behind our docker configuration.
Part II we will be building our docker image, push it to DigitalOcean container registry, and setup CI/CD pipeline with GitHub actions. We will also be deploying our built docker image to app platform and setup CI/CD using GitHub actions.

Docker has revolutionized software development and has proven to be the nucleus of new-age development practices like CI-CD, distributed development, and collaboration.

Still, there isn’t any popular consensus on what are good docker development principles and guidelines. Dockerfiles written for Java or any other programming language don’t directly translate to Python.

This article discusses an opinionated, production-ready Docker setup for Django applications which can be used in docker-compose files or with Kubernetes clusters. Our requirement further extends for containers to be scaled up and down without any side effects.

Note: Even though our docker configuration is production-ready the Django application itself is in no way ready for production.

If you need the code without going into the reasoning, a sample Django repo with Docker setup is available for download on Github, here.

So without further ado, let's start. The tech stacks we are using are:

Celery is used for background tasks, with Redis as the celery backend.
Celery beat is used for cron jobs, to schedule periodic tasks.
Flower is used for background tasks monitoring.
We are using PostgreSQL as our Database.

Both Django server and celery will be run from one docker image thus they share the same docker configuration. For database backup and restore purposes we are going to use a custom image for our PostgreSQL with maintenance commands and scripts. Thus we have two dockerfiles, one for our Django server and one for PostgreSQL. We will discuss both docker configs in detail below.

Django Docker Configuration

Let's go over our first dockerfile which runs the web server and celery.

# Section 1- Basic parameters
ARG PYTHON_VERSION=3.9-slim-buster
ARG BUILD_ENVIRONMENT=production
ARG APP_HOME=/app

# Section 2- Set the python base image
FROM python:${PYTHON_VERSION}

# Section 3- Python interpreter flags
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

# Section 4- Compiler and OS libraries
RUN apt-get update && apt-get install --no-install-recommends -y \
    build-essential \
    # psycopg2 dependencies
    libpq-dev \

    # Translations dependencies
    gettext \

    # cleaning up unused files
    && apt-get purge -y --auto-remove -o \
    APT::AutoRemove::RecommendsImportant=false \
    && rm -rf /var/lib/apt/lists/

# Section 5- Project libraries and User Creation
COPY requirements.txt /tmp/requirements.txt

RUN pip install --no-cache-dir -r /tmp/requirements.txt \
    && rm -rf /tmp/requirements.txt \

RUN useradd -U app_user \
    && install -d -m 0755 -o app_user -g app_user /app/static

# Section 6- Code and User Setup
WORKDIR ${APP_HOME}

USER app_user:app_user

COPY --chown=app_user:app_user . ${APP_HOME}

RUN chmod +x ./*.sh && chmod +x ./postgresql/maintenance/*.sh && \
    chmod +x ./postgresql/maintenance/_sourced/*.sh

# Section 7- Docker Run Checks and Configurations
ENTRYPOINT [ "./entrypoint.sh" ]

CMD [ "./start.sh", "server" ]

Let’s explore each section of our Dockerfile:

Section 1- Basic Parameters

# Step 1- Set arguments used throughout the build
ARG PYTHON_VERSION=3.9-slim-buster
ARG BUILD_ENVIRONMENT=production
ARG APP_HOME=/app

In the first section of our Dockerfile, we are declaring arguments or variables we are going to use throughout the build process. This is good practice for maintenance and updates later on. If you want to update or change one config you don’t need to go through all of your docker configurations.

Section 2- Base Image

We have selected python:3.9-slim-buster as the base image. While choosing a base image key consideration is its size, as a bigger base image results in a bigger docker image size. Developers prefer alpine flavor due to its small size and for languages such as Java or Scala, in most cases, it is the right way to go. Alpine is a minimal Docker image based on Alpine Linux.

But for Python applications, many requisite libraries are not supported by alpine flavor out of the box. It means you would end up downloading dependencies on alpine flavor which will result in bigger image size. This also means, greater image build time and application incompatibility. The slim flavor sits between alpine and full version and hits the sweet spot in terms of size and compatibility.

If you want to dig deep into this topic this article can get you started.

Section 3- Python Interpreter Flags

*# Section 3- Python interpreter flags
*ENV PYTHONUNBUFFERED *1
*ENV PYTHONDONTWRITEBYTECODE *1*

We have set two flags PYTHONUNBUFFERED and PYTHONDONTWRITEBYTECODE to non-empty values to modify the behavior of the Python interpreter.

When set to a non-empty value, PYTHONUNBUFFERED will send python output straight to the terminal(standard output) without being buffered. This helps in two ways. Firstly, this allows us to get logs in real-time. Secondly, in case of container crash, it ensures that you receive output and hence, the reason for failure.

We are also setting PYTHONDONTWRITEBYTECODE to a non-empty value. This ensures that the Python interpreter doesn’t generate .pyc files which apart from being useless in our use-case, can also lead to few hard-to-find bugs due to caching.

Section 4- Compiler and OS libraries

# Section 4- Compiler and OS libraries
RUN apt-get update && apt-get install --no-install-recommends -y \
    build-essential \
    # psycopg2 dependencies
    libpq-dev \

    # Translations dependencies
    gettext \

    # cleaning up unused files
    && apt-get purge -y --auto-remove -o \
    APT::AutoRemove::RecommendsImportant=false \
    && rm -rf /var/lib/apt/lists/

Commands in this section install compilers, tools, and OS-level libraries. For e.g. apt-get update , as you may already know, update the list of available packages. It doesn’t update packages themselves, just fetches their latest versions.

apt-get install -y --no-install-recommends build-essential \
libpq-dev gettext

The build-essential contains a collection of meta-packages that are necessary to compile software. This includes, but is not limited to, GNU debugger, g++/GNU compiler collection, and a few other tools and libraries. The complete list of build-essential packages can be found here. As per official documentation libpq-dev contains,

Header files and static library for compiling C programs to link with the libpq library in order to communicate with a PostgreSQL database backend.

Since libpq-dev contains libraries concerning the PostgreSQL database, feel free to drop this if you are using some other database and install the requisite for that database.

The flag --no-install-recommends skips the installation of other recommended packages. This is done to reduce docker image size. Please note that dependent packages mandatory for our packages are still getting installed. gettext is a Linux package that facilitates translations. If you want to know more refer here

apt-get purge -y --auto-remove -o \ APT::AutoRemove::RecommendsImportant=false* \

In this command, we are cleaning up our package repository by removing orphaned packages we don’t need.

rm -rf /var/lib/apt/lists/

Cleaning /var/lib/apt/lists/* can easily reduce your docker image size by ~5%-25%. The apt-get update command updates versions of the list of packages that are not required in our Dockerfile after installing build-essential and libpq-dev . Hence, in this step, we clean out all the files added.

Section 5- Project libraries and User Creation

In this section, we install the project libraries mentioned in requirements.txt and create a user who will be a non-root user for security purposes.

COPY requirements.txt /tmp/requirements.txt

If you notice, instead of copying the whole project, which we do eventually in Section 6, we are only copying requirements.txt. Then we are installing all the libraries mentioned in it. This is done so because Docker works on the principle of layers. If there is any change in a layer, all the subsequence layers will be re-processed. Hence, copying only requirements.txt ensures that installation is reused across docker builds. This layer is dropped if there is a change in the requirements.txt file itself. Had we copied the entire project of Section 6 here, each new commit or change in code would lead to invalidating of these layers and re-installation of libraries.

RUN pip install --no-cache-dir -r /tmp/requirements.txt
&& rm -rf /tmp/requirements.txt \

In this stage, we are installing all the project dependencies mentioned in requirements.txt. The --no-cache-dir flag is used to disable caching during pip installation. By default, pip caches installation files(.whl etc) and source files(.tar.gz etc). In docker installation, we don’t reinstall using the cache hence disabling it will reduce image size. Then we remove the requirements file we copied to /tmp directory

useradd -U app_user

Here, we are creating a non-root user app_user using the useradd command. By default, Docker runs container processes as root inside of a container. This is a bad practice since attackers can gain root access to the Docker host if they manage to break out of the container (source). The -U flag creates a user group with the same name.

install -d -m 0755 -o app_user -g app_user /app/static

At the end of the section, we are creating a folder app/static and giving our user app_user ownership to it. This folder will be used by Django to collect all static resources of our project by running the command python manage.py collectstatic .

Section 6- Code and User Setup

WORKDIR ${APP_HOME}

We start this section by setting the working directory. The WORKDIR instruction sets the working directory for subsequent commands. Since we don’t want to copy our code to the root folder, we are copying it to /app folder.

USER app_user:app_user

Then we are setting the non-root user created at the end of Section 5 as the owner of subsequent commands. As mentioned earlier, this will improve our security.

COPY --chown=app_user:app_user . ${APP_HOME}

With everything set up, we copy the project into the docker image. Any code change will only result in an update in this and subsequent layers of docker, hence resulting in reduced docker image build time. While copying we are providing the content’s ownership to our user app_user created in Section 4.

RUN chmod +x ./*.sh && chmod +x ./postgresql/maintenance/*.sh && \
    chmod +x ./postgresql/maintenance/_sourced/*.sh

At the end of this section, we are giving executable permission to our two scripts files i.e. entrypoint.sh and start.sh and all scripts we use to maintain our database. We will go into detail about these two files after the end of Section 6.

Section 7- Docker Run Checks and Configurations

ENTRYPOINT [ "./entrypoint.sh" ]

The ENTRYPOINT section of a Dockerfile is always executed, hence we would like to hitch it for validations and Django commands such as migrate. The CMD is overridden by the command section in a docker-compose file so the value given here, serves as a default.

CMD [ "./start.sh", "server" ]

For a better understanding of what we are trying to do with ENTRYPOINT and CMD let’s look at the corresponding files entrypoint.sh and start.sh which are invoked by them.

entrypoint.sh

#!/bin/bash

set -o errexit
set -o pipefail
set -o nounset

postgres_ready() {
    python << END
import sys
from psycopg2 import connect
from psycopg2.errors import OperationalError
try:
    connect(
        dbname="${DJANGO_POSTGRES_DATABASE}",
        user="${DJANGO_POSTGRES_USER}",
        password="${DJANGO_POSTGRES_PASSWORD}",
        host="${DJANGO_POSTGRES_HOST}",
        port="${DJANGO_POSTGRES_PORT}",
    )
except OperationalError:
    sys.exit(-1)
END
}

redis_ready() {
    python << END
import sys
from redis import Redis
from redis import RedisError
try:
    redis = Redis.from_url("${CELERY_BROKER_URL}", db=0)
    redis.ping()
except RedisError:
    sys.exit(-1)
END
}

until postgres_ready; do
    >&2 echo "Waiting for PostgreSQL to become available..."
    sleep 5
done
>&2 echo "PostgreSQL is available"

until redis_ready; do
    >&2 echo "Waiting for Redis to become available..."
    sleep 5
done
>&2 echo "Redis is available"

python3 manage.py collectstatic --noinput

exec "$@"

Let’s look at the above entrypoint.sh, though in lesser detail than Dockerfile.

Docker provides a default entrypoint /bin/sh . In most systems, it is a symbolic link, and in the case of Ubuntu it is linked to, /bin/bash but in some scenarios, this assumption could be wrong(source). Hence we will be explicitly linking it to /bin/bash.

Section 1- Bash options

set -o errexit
set -o pipefail
set -o nounset

Here, we are setting few bash options. The errexit fails the script on the first encounter of error and doesn’t proceed further, which is default bash behavior. The pipefail means that if any element of the pipeline fails, then the pipeline as a whole will fail. The nounset forces error whenever an unset variable is extended.

Section 2: Health of dependent services

Earlier, we had assumed that our application is using PostgreSQL database and Redis as celery backend. In this section, we are checking if both services are up and if not, we wait for them to come up.

Similarly, you may add other such critical services which are necessary for the normal functioning of your application.

Section 3- Idempotent Django commands

python manage.py collectstatic --noinput

There are many Django management commands which we need to run before starting our Django server. This includes commands to collect all static resources, collectstatic. Djangos makemigrations and migrate command should not be run at container runtime due to the following reasons:

In dev environments you typically spin up one server, but in production you’re likely spinning up more than one. So now instead of one process doing schema migration, you have multiple processes trying to do multiple identical schema migrations at the same time.

Depending on your database, the migration tool you’re using, and the kind of migration you’re doing, parallel schema upgrades might break your database in a variety of ways.
You don’t want a broken database!

If you always do schema upgrades as part of the application startup you also end up mentally coupling schema migrations and code upgrades.*** In particular, you’ll start assuming that you only ever have new code running with the latest schema.***

Why is that assumption a problem? From most to least common:
1. Sometimes you need to rollback a broken code upgrade. If you assume you always have new code with a new schema, you can end up in a situation where your new code is broken, but you can’t easily rollback to older code because you’ve done an irreversible schema change.
2. To minimize downtime on upgrades, you want to have a brief moment where both old and new versions of your application are running in parallel. If your schema migration breaks old code, you can’t do that.
3. To catch bugs in new code, you might want to do a canary deploy. That is, upgrade only one or two of your many processes and see if they break.

The only thing which should be kept in mind is that all these commands should be idempotent i.e. multiple runs of these commands should not have any side-effect on the state of our application. Idempotency is required here because, suppose if Kubernetes is scaling these containers, multiple instances will be running and they will interfere will each other.

In fact, any idempotent operation can be executed here, not just Django commands.

start.sh

We are using start.sh file, to leverage the same Dockerfile and commands to run containers for Django server, Celery workers, Celery Beat and Flower, by having different arguments for each.

#!/bin/bash

cd /app

if [ $# -eq 0 ]; then
    echo "Usage: start.sh [PROCESS_TYPE](server/beat/worker/flower)"
    exit 1
fi

PROCESS_TYPE=$1

if [ "$PROCESS_TYPE" = "server" ]; then
    if [ "$DJANGO_DEBUG" = "true" ]; then
        gunicorn \
            --reload \
            --bind 0.0.0.0:8000 \
            --workers 2 \
            --worker-class eventlet \
            --log-level DEBUG \
            --access-logfile "-" \
            --error-logfile "-" \
            dockerapp.wsgi
    else
        gunicorn \
            --bind 0.0.0.0:8000 \
            --workers 2 \
            --worker-class eventlet \
            --log-level DEBUG \
            --access-logfile "-" \
            --error-logfile "-" \
            dockerapp.wsgi
    fi
elif [ "$PROCESS_TYPE" = "beat" ]; then
    celery \
        --app dockerapp.celery_app \
        beat \
        --loglevel INFO \
        --scheduler django_celery_beat.schedulers:DatabaseScheduler
elif [ "$PROCESS_TYPE" = "flower" ]; then
    celery \
        --app dockerapp.celery_app \
        flower \
        --basic_auth="${CELERY_FLOWER_USER}:${CELERY_FLOWER_PASSWORD}" \
        --loglevel INFO
elif [ "$PROCESS_TYPE" = "worker" ]; then
    celery \
        --app dockerapp.celery_app \
        worker \
        --loglevel INFO --loglevel INFO -P gevent --concurrency=100

fi

In the above script, we are using gunicorn to run our application server which is recommended approach for production. The python manage.py runserver command should be used only in the development setup.

PostgreSQL Docker Configuration

Now lets take a look at our other dockerfile for our database

# Step 1- Set arguments used throughout the build
ARG POSTGRES_VERSION=13.3-alpine

# Step 2- Set the postgresql base image
FROM postgres:${POSTGRES_VERSION}

# Step 3- Copy Postgresql configuration files and maintenance scripts

COPY ./postgresql/maintenance /usr/local/bin/maintenance
RUN chmod +x /usr/local/bin/maintenance/

RUN mv /usr/local/bin/maintenance/* /usr/local/bin \
&& rmdir /usr/local/bin/maintenance

Step 1 & Step 2 Argument Setup and Image Version

In steps 1 and 2 we are setting up argument and pulling PostgreSQL image from dockerhub

Step 3 Copy Postgresql configuration files and maintenance scripts

In step 3 we are copying all config files and maintenance files for PostgreSQL to /user/local/bin/This directory is where all binary files for linux are stored which makes it easier to invoke those scripts. Let's look at those scripts

backup.sh and backups.sh

These two bash scripts hold our backup-related scripts.

working_dir="$(dirname ${0})"

source "${working_dir}/_sourced/constants.sh"
source "${working_dir}/_sourced/messages.sh"

message_welcome "Backing up the '${POSTGRES_DB}' database..."

if [[ "${POSTGRES_USER}" == "postgres" ]]; then
message_error "Backing up as 'postgres' user is not supported. Assign 'POSTGRES_USER' env with another one and try again."

exit 1

fi

export PGHOST="${POSTGRES_HOST}"
export PGPORT="${POSTGRES_PORT}"
export PGUSER="${POSTGRES_USER}"
export PGPASSWORD="${POSTGRES_PASSWORD}"
export PGDATABASE="${POSTGRES_DB}"

backup_filename="${BACKUP_FILE_PREFIX}_$(date +'%Y_%m_%dT%H_%M_%S').sql.gz"

pg_dump | gzip > "${BACKUP_DIR_PATH}/${backup_filename}"

message_success "'${POSTGRES_DB}' database backup '${backup_filename}' has been created and placed in '${BACKUP_DIR_PATH}'."

We are exporting environment variables to connect to the database and preparing a backup filename from the date.

pg_dump | gzip > “${BACKUP_DIR_PATH}/${backup_filename}”

This command will dump all the content of the database to the backup file in the backup directory.

To run our container we are running

docker-compose up

The other essential commands are

# To backup the database
docker-compose exec postgres backup

# To restore created backup
docker-compose exec postgres restore backup_2021_03_13T09_05_07.sql.gz

Part I of this article is fairly long, We are going to leave it there for this one. In the next article, we are going to build our docker image and push it to DigitalOcean container registry. We will also setup CI/CD for the build process, after that we are going to deploy our docker image to DigitalOcean App platform and setup CI/CD for deployment using GitHub actions.

Production-Ready Docker Configuration With DigitalOcean Container Registry Part II

To get the full code and try it for your self, It is on GitHub here.

Please comment on any gaps or improvements in the above setup. Follow me for more articles like this one.

Use the following link and get 100 USD in DigitalOcean free credit.

Getting started with containerizing ReactJs application — development and production environment setup.

Endalkachew Biruk — Wed, 03 Nov 2021 11:42:29 +0000

Overview

Before the wide adoption of containerization and docker developers develop their web apps and everything works perfectly in development machine but when they push it to the production environment it doesn’t work. This was because the development environment and production environment are not the same and that causes problems. There is also another issue if there is an error or a bug in the production environment it is very difficult to figure out what is wrong and solve that issue since that issue might not be in your local setup.

This is where containerization technologies like docker come in. Docker solves the above issues by packaging your web app with all your dependencies and shipping it to production. This container is self-sufficient and can be run anywhere, The environment running the container doesn’t care what is in the container (It just works). When you use Docker for development and production you will not run into unforeseen production issues or bugs since you are using the same container you have been running locally in production.

Getting Started

In this article, we are going to go over docker development and production docker setup for ReactJs applications. We will build a basic ReactJs application and containerize the app both for development and production. We will use Nginx for serving production build of our ReactJs application.

Nginx is open-source software and a web server for reverse proxying caching and load balancing. Nginx also provides HTTP server capabilities. And it is mainly designed for maximum performance and stability. Nginx provides functions like IMAP, POP3, and SMTP as a proxy server for email.

If you want to clone the app and try it locally use the following git command. and check out the reactjs-docker-setup branch.

git clone [https://github.com/endalk200/blog-repository.git](https://github.com/endalk200/blog-repository.git)
cd ./blog-repository
git checkout reactjs-docker-setup

We have two environments that our web app runs in. The first one is the development environment where we want hot reloading when making code changes. The other environment is the production environment. In production, our ReactJs app is optimized and built to static assets and served through a web server or a CDN. In our case, we are going to use Nginx to serve our static assets.

We are going to make sure both our production and development docker configurations are as close to each other as possible to avoid unforeseen issues once deployed.

When we build our react application the output is optimized static assets (html, css and js file). To serve these files we are going to use Nginx, Popular web server and reverse proxy server.

Our file structure looks like this

File structure

The most important files we need to see are Dockerfile both in local and production directory anddocker-compose.yaml file. Let’s break it down depending on the environment.

Development

Since in development we want hot realoading we are going to mount our local directory to our container. We will achieve this using docker-compose.yaml file. Let’s start with our dockerfile.

FROM node:alpine AS dependencies

We are going to use alpine as a base image. It is a minimal Linux build with necessary packages. We are going to use a multi-stage build and we will name this stage dependencies where we install all OS level and application dependencies.

RUN apk add --no-cache libc6-compat

libc6-compat is needed in nodeJs build systems. If you want to now more about this refer here

WORKDIR /app

We are going to set our working directory to /app

COPY package.json package-lock.json ./

Then we are going to copy our package.json and package-lock.json files to install our app dependencies.

RUN npm ci

This command is similar to [npm install](https://docs.npmjs.com/cli/v7/commands/npm-install), except it's meant to be used in automated environments such as test platforms, continuous integration, and deployment -- or any situation where you want to make sure you're doing a clean install of your dependencies.

In short, the main differences between using npm install and npm ci are:

The project must have an existing package-lock.json or npm-shrinkwrap.json.
If dependencies in the package-lock do not match those in package.json, npm ci will exit with an error, instead of updating the package-lock.
npm ci can only install entire projects at a time: individual dependencies cannot be added with this command.
If a node_modules is already present, it will be automatically removed before npm ci begins its install.
It will never write to package.json or any of the package-locks: installs are essentially frozen.

Unto the second stage which is the runner stage, where we run our application.

FROM node:alpine AS runner

WORKDIR /app

COPY . .

COPY --from=dependencies /app/node_modules ./node_modules

In the above config, we are setting the working directory where we run our application to /app. Then we are copying all files in our local directory to the container. NOTE: All files in.dockerignore file will not be added to the container.

ENTRYPOINT ["npm", "run", "start"]

The last one will be our ENTRYPOINT . This is where the command to run the application goes. npm run start will run our web app locally.

Now that we have seen our Dockerfile we are going to see out docker-compose.yaml file.

version: '3'

services:
    web:
        build:
            context: .
            dockerfile: ./local/Dockerfile
    image: reactapp_image
    container_name: reactapp_container
    ports:
        - 3000:3000
    volumes:
        - ./:/app

Let’s break this down. basically, we are telling Docker to use the current directory as a context where to run everything and use the Dockerfile found in the local directory. Then we are telling docker to name the image built reactapp_image and the container reactapp_container We are also telling docker to map our local machines port 3000 to the containers port3000 . Then since we want all code changes we make locally to be reflected in our dev server we are going to map our current directory to the containers /app directory where our application runs in.

To run our development server for the first time we have to build our image and spin it up. To do that we will execute the following command.

docker-compose up --build

After that first time, you just have to run docker-compose up to spin up the container.

To tear down the running container use the command.

docker-compose down

For some reason, if you have to run some kind of command inside the container, first run docker ps to list all running containers with their container ID. Then use the ID and run the following command.

docker exec -ti f6d6a1d9a076 bash

Assuming f6d6a1d9a076 is your container ID it will open up an interactive bash session where you can run your commands right in the container.

Production

Now that we have seen our development docker config let’s see the production one. The production docker config only needs the Dockerfile to run.

The first stage of our Dockerfile is the same as the above config. Let's start with the build stage.

FROM node:alpine AS builder

WORKDIR /app

COPY . .

COPY --from=dependencies /app/node_modules ./node_modules

RUN npm run build && npm install --production --ignore-scripts --prefer-offline

In this stage, we are choosing alpine as build stage base image. We are also setting the working directory to /app then we are copying all files in the current directory to the /app directory. After that, we are copying all dependencies downloaded and stored in the /app/node_modules directory to node_modules directory from the previous dependencies stage.

After that, we are running npm run build after npm install --production --ignore-scripts --prefer-offlineThis will build an optimized build of our ReactJs application.

The next stage is where we run our web server using Nginx This is where things get interesting. We are going to use the default Nginx config since we don’t need anything other than that to run our app.

Nginx has some default configs and behaviors we need to know. Before getting into our docker config let’s see a few of them.

Nginx serves static assets found in /usr/share/nginx/html directory. So all static assets like HTML, CSS, and Js need to be in this directory in order forNginx to serve them (We can also change this behavior by editing the default config file). By default Nginx runs on port 80

*# Nginx configuration and runing stage.
*FROM nginx:alpine AS runner

# Set working directory to nginx asset directory
WORKDIR /usr/share/nginx/html

# Remove default nginx static assets
RUN rm -rf ./*

# Copy static assets from builder stage
COPY --from=builder /app/build .

# Containers run nginx with global directives and daemon off
ENTRYPOINT ["nginx", "-g", "daemon off;"]

We are choosing nginx:alpine as our runner since it has all packages we need and it is minimal in size.

We are also setting our working directory to /user/share/nginx/html the default location where static assets are served from. Then we are removing all assets found in that directory using this directive RUN rm -rf ./* . Then we will copy our optimized build from the builder stage to the current working directory. Then we are going to run Nginx web server using the command nginx -g daemon off

To build our docker image from this config run

docker build -t reactjs-prod -f ./production/Dockerfile .

We are naming the image reactjs-prod and we are going to use the production Dockerfile in the production directory.

To run the built image in interactive mode we will use the following docker command.

docker run -it -p 80:80 reactjs-prod:latest

The -it flag tells docker to run the container in interactive mode so that we can see the log. The -p flag tells docker to map port 80 of our machine to port 80 of our container. This command will spin up the production docker container running Nginx server.

Where to go from here

If you want more control over how Nginx serves the static assets and how it behaves you can replace the default Nginx config file find in /etc/nginx/nginx.conf with your config file. As an example, you can customize the above runner stage as follows.

*# Nginx configuration and runing stage.
*FROM nginx:alpine AS runner

# Set working directory to nginx asset directory
WORKDIR /usr/share/nginx/html

# Remove default nginx static assets
RUN rm -rf ./*

# Remove the default nginx config file
RUN rm -rf /etc/nginx/nginx.conf

# Copy custom nginx config to the directory
COPY ./nginx/nginx.con /etc/nginx/

# Copy static assets from builder stage
COPY --from=builder /app/build .

# Containers run nginx with global directives and daemon off
ENTRYPOINT ["nginx", "-g", "daemon off;"]

The above config assumes you have a custom Nginx config in nginx directory.

Conclusion

That's it for this one, If you want more explanation on this article leave a comment and I will get back to you.

To get the full code and try it for yourself, It is on GitHub here.

Please comment on any gaps or improvements or just tell me your thoughts down in the comments. Follow me for more in-depth articles like this one.

Use the following link and get 100 USD in DigitalOcean free credit.

DEV Community: Endalkachew Biruk

JavaScript Promises and Async Await

Promises

1) How do you create a Promise?

2) How do you change the status of a promise?

3) How do you listen for when the status of a promise changes?

Chaining

async functions return a promise

await without async is bad

Error Handling

Mastering JavaScript Callbacks

Callbacks

Production-Ready Docker Configuration With DigitalOcean Container Registry Part II

Step 1- Install and configure doctl

Step 2 — Build and push your image to container registry

Step 3 —Setup CI/CD for the build process using GitHub actions

Step 4 — Deploy your container to app platform

Production-Ready Docker Configuration With DigitalOcean Container Registry Part I

Django Docker Configuration

Section 1- Basic Parameters

Section 2- Base Image

Section 3- Python Interpreter Flags

Section 4- Compiler and OS libraries

Section 5- Project libraries and User Creation

Section 6- Code and User Setup

Section 7- Docker Run Checks and Configurations

entrypoint.sh

Section 1- Bash options

Section 2: Health of dependent services

Section 3- Idempotent Django commands

start.sh

PostgreSQL Docker Configuration

Step 1 & Step 2 Argument Setup and Image Version

Step 3 Copy Postgresql configuration files and maintenance scripts

Production-Ready Docker Configuration With DigitalOcean Container Registry Part II

Getting started with containerizing ReactJs application — development and production environment setup.

Overview

Getting Started

Development

Production

Where to go from here

Conclusion

Step 1- Install and configure `doctl`