DEV Community: Sourav Atta

Writing a Dockerfile for Flask App

Sourav Atta — Fri, 21 May 2021 10:51:54 +0000

Flask is a Python web framework which is used to develop web applications. In this post, we will try to deploy the web application on our machine using Docker container.

Containerising an application is packaging the application and its requirements, so that it can be used to deploy in any machine on a go. We will create a Docker image of the web-app using the Dockerfile and then we will run the docker container to access the web application in our browser.

Getting Started

We will use the basic flask application created by Mindy McAdams. Github repo url: https://github.com/macloo/basic-flask-app

Clone the repo in your machine https://github.com/macloo/basic-flask-app using the below command:

git clone https://github.com/macloo/basic-flask-app

I hope that Docker is already installed in your system, but if it is not installed then follow the steps given in Install Docker Engine

Now, everything is setup. We will start writing Dockerfile for the flask application.

Containerising the Flask App

We will now create a Dockerfile for the flask app which will be used to create the image of the app.

Follow the steps below to create the Dockerfile:

1) Clone the Github repo if not done.
2) Change the directory to basic-flask-app/ and create a file called Dockerfile

cd basic-flask-app/
touch Dockerfile

3) Use your favorite editor, to edit the Dockerfile and paste the following:

ARG APP_IMAGE=python:3.6.1-alpine

FROM $APP_IMAGE AS base

FROM base as builder

RUN mkdir /install
WORKDIR /install

COPY requirements.txt /requirements.txt

RUN pip install --install-option="--prefix=/install" -r /requirements.txt

FROM base
ENV FLASK_APP routes.py
WORKDIR /project
COPY --from=builder /install /usr/local
ADD . /project

ENTRYPOINT ["python", "-m", "flask", "run", "--host=0.0.0.0"]

Breaking up the Dockerfile

Let's see each line of the Dockerfile

ARG APP_IMAGE=python:3.6.1-alpine

ARG are also known as build-time variables i.e. they can be set during the image build with --build-arg and you can’t access them anymore once the image is built. Here, we take a variable APP_IMAGE to give the base image name. Default value: python:3.6.1-alpine

FROM $APP_IMAGE AS base

We are using the concept of multi-stage builds to optimize the size of the docker image. More about multi-stage builds in here. Here, FROM initializes the build stage and sets the base image.

FROM base as builder

Here, we are setting the alias name builder for the base image. In this image, we will only install the dependencies packages.

RUN mkdir /install

RUN is used to run a specific command and it creates a writable container layer. Here, we are running a command to create a directory called install. Thus, the name builder :P

WORKDIR /install

WORKDIR is setting up the working directory of a Docker container at any given time. Any RUN , CMD , ADD , COPY , or ENTRYPOINT command will be executed in the specified working directory. Here, we are making the install directory as working directory.

COPY requirements.txt /requirements.txt

COPY as the name suggest is used to copy the file from your Docker client's current directory. Here, we are copying the file requirements.txt

RUN pip install --install-option="--prefix=/install" -r /requirements.txt

Here, we are using pip to install all the packages required to build the flask app. The packages are mentioned in the requirements.txt file.

FROM base

Again, now after the installation of the required packages, we are now taking the same image (used above) as the base image.

ENV FLASK_APP routes.py

To run a flask app, either we need to use flask command or python's -m switch with flask. But, before that we need to export a variable called FLASK_APP to specify how to load the application.

Note: This will start a development web server. But, for production deployment, you need to use production-ready web server like uWSGI

WORKDIR /project

We are setting project directory as the working directory.

COPY --from=builder /install /usr/local

We are copying all the installed binary packages installed in the later base image to the path /usr/local in the current base image.

ADD . /project

ADD command is also used to copy the files/directories and it also, copies and extract the compressed file automatically. Here, we are copying files/directories from current local directory to container's project directory.

ENTRYPOINT ["python", "-m", "flask", "run", "--host=0.0.0.0"]

ENTRYPOINT is to identify which executable should be run when a container is started from your image. Here, we will run the flask app using python's -m and --host=0.0.0.0 will make the server publicly accessible.

Running the Docker Container

We have the Dockerfile created in above section. Now, we will use the Dockerfile to create the image of the flask app and then start the flask app container.

Follow the below steps to run the container:

1) Building the Docker image using the docker build command.

docker build -t basic-flask:latest --build-arg APP_IMAGE=python:3.9.5-alpine -f Dockerfile .

The above command will build the image with tag basic-flask:latest. We have given the --build-arg option to mention the base image name to be used to iniate the image build.

Note: If we use the option --build-arg in docker build command, it will overwrite the default value of the variable used in the Dockerfile. But, we run the docker build command without using the option --build-arg it will take the default value.

2) Once the command runs successfully, run the below command:

docker images

This command will list all the docker images and you can also see the image basic-flask:latest in the list.

3) Now, run the below command to start the container from the image build in step 2.

docker container run -p 5000:5000 -dit --name flaskApp basic-flask:latest

This command will run the application at port 5000. The various options used are:

-p: publish the container's port to the host port.
-d: run the container in the background.
-i: run the container in interactive mode.
-t: to allocate pseudo-TTY.
--name: name of the container

4) Check the status of the docker container using the command:

docker container ps

You can see that your container is in running mode with several other details.

Try to access the flask application from your browser. Visit the URL http://localhost:5000/ and verify the output:

Try the above example out and please let me know in the comments if you have any doubts or have a better form of the Dockerfile to build the flask app.

Writing an effective GROK pattern

Sourav Atta — Wed, 05 May 2021 06:40:44 +0000

Grok is one of the popular Logstash filters which is used to parse the unstructured log data to a meaningful format.

Logstash ships with 120 default built-in patterns. You can find them here: https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

Also, some of the patterns can be referred from https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns
I personally prefer the above link for constructing grok pattern.

Now, there may be cases when these grok patterns won't fit. So, we have a regular expression library Oniguruma, which can be combined with grok to create powerful patterns.

Grok Syntax

%{SYNTAX:SEMANTIC}

SYNTAX is the default grok patterns
SEMANTIC is the key

Oniguruma Syntax

(?<field_name>regex pattern)

field_name is the key
regex pattern is the placeholder to add your regex

How to use?

Let's try to create a pattern to parse unstructured log data.

Sample Log Data

09:33:45,416 (metrics-logger-reporter-1-thread-1) type=GAUGE, name=notifications.received, value=2

Required fields from log data

Field	Field Value
timestamp	09:33:45,416
logthread	metrics-logger-reporter-1-thread-1
type	GAUGE
name	notifications.received
value	2

Grok Pattern

We will use Grok Debugger to test our pattern to match the log data.

Let's disintegrate the log data to create a pattern that matches a particular field:

Field	Pattern
timestamp	%{TIME}
type	%{DATA}
name	%{DATA}
value	%{POSINT}

The field thread, can be a combination of the alphanumeric characters.

So, we need to use oniguruma to match the field logthread. Considering the syntax of oniguruma, we need to create a regex pattern that will match the value of the field logthread

Constructing Regex Pattern

We now use Regex Checker that will help us to construct and test the regex pattern for the value of field logthread

The (?:[()a-zA-Z\d-]+) non-capturing group matches single character present in the list below:

+ greedy match i.e. matches the previous token between one and unlimited times, as many times as possible
() matches a single character in the list ()
a-z matches a single character in the range between a and z
A-Z matches a single character in the range between A and Z
\d matches a digit
- matches the character -

Oniguruma

The final Oniguruma pattern for the field logthread:

(?<logthread>(?:[()a-zA-Z\d-]+))

Grok Pattern + Oniguruma (Final Pattern)

The final pattern that will match the log data:

%{TIME:timestamp} \((?<logthread>(?:[()a-zA-Z\d-]+))\) type=%{DATA:type}, name=%{DATA:name}, value=%{POSINT:value}

Output of the pattern

{
  "timestamp": [
    [
      "09:33:45,416"
    ]
  ],
  "HOUR": [
    [
      "09"
    ]
  ],
  "MINUTE": [
    [
      "33"
    ]
  ],
  "SECOND": [
    [
      "45,416"
    ]
  ],
  "logthread": [
    [
      "metrics-logger-reporter-1-thread-1"
    ]
  ],
  "type": [
    [
      "GAUGE"
    ]
  ],
  "name": [
    [
      "notifications.received"
    ]
  ],
  "value": [
    [
      "2"
    ]
  ]
}

Conclusion

The combination of Grok Pattern and Oniguruma is a perfect pair. Tha pairing can help to transform any complex logs into structured data. Give it a try using Grok Pattern + Oniguruma in Logstash !!

Let me know in the comments if you have any better way of doing or facing any problem with the above example.