DEV Community

loading...
Cover image for Jun, 21 #CloudGuruChallenge

Jun, 21 #CloudGuruChallenge

Justin Wheeler
・5 min read

TL;DR

I completed another #CloudGuruChallenge. This month we were challenged to improve a python app's performance by implementing ElastiCache for Redis.

I thought this challenge was really interesting because application performance is such a big topic in the industry. I've seen Python and Redis used multiple times already.

Unfortunately due to the nature of the services required for this challenge I was unable to take a serverless approach. That also means that I would have already tore down the resources by the time you're reading this. Don't worry though I took some pictures for us to enjoy!

Database Response

Database Response

Cache Response

Cache Response

If you'd like to checkout the code or connect on LinkedIn then see below.

GitHub
https://github.com/wheelers-websites/CloudGuruChallenge_21.06

LinkedIn
https://www.linkedin.com/in/justin-wheeler-the-cloud-architect/

In addition to the requirements I decided to go above and beyond. If that interests you then keep reading!


Architecture

I decided to manage the architecture for the application and the network separately since the costs vary drastically between both among other reasons.

Network Architecture

Network Architecture

  • VPC
  • 4 Public Subnets
  • 4 Private Subnets
  • Public Route Table
  • Private Route Table
  • Internet Gateway
  • NAT Gateway
  • Public Security Group (HTTP and SSH)
  • Private Security Group (PostgreSQL and Redis)

Application Architecture

Application Architecture

  • EC2 Instance
  • EC2 KeyPair
  • ElasticCache Redis Instance
  • ElasticCache Subnet Group
  • RDS PostgreSQL Instance
  • RDS Subnet Group

It's true that this configuration is probably overkill for this simple application. However, I wanted to push myself to design for a production scenario.

Here are some next steps that would improve this architecture that I deemed out of scope.

HTTPS Ingress

  • update public security group to permit HTTPS instead of HTTP
  • setup NGINX to use HTTPS

Failover NAT Gateway

  • setup failover NAT Gateway in another AZ
  • update private route table to use one or both NAT Gateways

Multiple Application Servers w/ Load Balancer

  • setup the app to run as part of an ASG instead
  • setup an ALB/NLB in front of the ASG to load balance traffic

Terraform

Much like in previous projects I decided to use Terraform here for Infrastructure as Code (IaC). It seems that every time I use Terraform I find something new that I like about it.

Below are a list of new found features that I recently learned about and used on this project.

locals block

This feature allows users to define local variables inside their templates.

https://www.terraform.io/docs/language/values/locals.html

locals {
  availability_zones = [
    "us-east-1a",
    "us-east-1b",
    "us-east-1c",
    "us-east-1d"
  ]
  cidr_anywhere_block = "0.0.0.0/0"
  ...
}

local.cidr_anywhere_block
Enter fullscreen mode Exit fullscreen mode

file function

This feature allows you to easily pass a file into the resource declaration. That allowed me to bootstrap my EC2 instance while managing that script outside of the template itself. No crazy formatting required here!

https://www.terraform.io/docs/language/functions/file.html

resource "aws_instance" "server" {
  ...
  user_data = file("user-data.sh")
}
Enter fullscreen mode Exit fullscreen mode

random_shuffle provider

This feature allows you to shuffle a list of values and return x number of values back. I thought this was far more clever than simply providing a single value and more flexible too!

https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/shuffle

variable "subnet" {
  description = "subnet ids"
  type = object({
    private = list(string)
    public  = list(string)
  })
}

subnet = {
  ...
  public  = [
    "subnet-01b36f97fa8490e43",
    "subnet-0a4272a81af160127",
    "subnet-03939599dd92f0f07",
    "subnet-0b6f2c6a35b31e4d5"
  ]
}

resource "random_shuffle" "public_subnet" {
  input        = var.subnet.public
  result_count = 1
}

subnet_id = random_shuffle.public_subnet.result[0]
Enter fullscreen mode Exit fullscreen mode

count meta-argument

This feature allows you to provide looping behavior to your templates so you don't need to repeat yourself.

https://www.terraform.io/docs/language/meta-arguments/count.html

resource "aws_subnet" "public" {
  count = length(local.cidr_public_blocks)
  cidr_block        = local.cidr_public_blocks[count.index]
  ...
}
Enter fullscreen mode Exit fullscreen mode

GitHub (Terraform)


Flask, Gunicorn, and Styles

I changed the default application that was provided by David Thomas to use some new found knowledge on related topics.

https://github.com/ACloudGuru/elastic-cache-challenge

Jinja Expressions

This allowed me to put logic into my HTML templates similar to ngIf for Angular. I mainly did this because when testing I didn't want to see empty sections drawn on the screen. Now they will be hidden if the values are not passed into the render_template function.

{% if db_host %}
<div class="section">
    <span class="section-title">
        Database Host
    </span>
    <span>{{ db_host }}</span>
</div>
{% endif %}
Enter fullscreen mode Exit fullscreen mode

Furthermore, this allowed me to add an error display on the screen that is hidden if there is no error. Since I know there is a risk with displaying some error logs to the user I modified the code to only pass the actual error if the DEBUG variable is set appropriately on the server.

debug = os.environ.get('DEBUG')
debug = debug.lower() in ['1', 't', 'true', 'y', 'yes'] if debug is not None else False

...
try:
    config = ini('postgres')
    db_host = config['host']
    db_version = select_version()

except Exception as e:
    print('index() error:', e)
    error = e if debug else "we're currently experiencing technical difficulties"

return render_template(
    html,
    error=error,
    db_host=db_host,
    db_version=db_version
)
Enter fullscreen mode Exit fullscreen mode

Error Message

Jinja Inheritance

This allowed me to create a base template and reuse that in other templates, if I had other templates, so I could keep a consistent style app-wide with low maintenance.

<head>
  <meta charset="UTF-8" />
  <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}" />
  <title>CloudGuruChallenge_21.06</title>
</head>
<body>
  {% block content %}
  {% endblock %}
</body>
Enter fullscreen mode Exit fullscreen mode
{% extends 'base.html' %}
{% block content %}
...
<div class="section">
    <span class="section-title">
        Elapsed Time
    </span>
    <span>{{ g.request_time() }}</span>
</div>
{% endblock %}
Enter fullscreen mode Exit fullscreen mode

Gunicorn Server

I had quickly learned that the built-in flask server is not intended to work in production. Although, it may have been fine for this simple project I wanted to go the extra mile to setup gunicorn instead.

Once installed I just ran the app as a daemon in my bootstrap script. e.g. gunicorn -D app:app

Styling

I couldn't stand the bare HTML page and decided to use some CSS to spruce it up a tad. In addition to styling I wanted to leverage flask's static assets and url_for as well to solidify that knowledge.

<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}" />

GitHub (Python)


Conclusion

Overall I really enjoyed this challenge. I can't wait to see what could possibly be next. Regardless I will be up the task!

I'd love to connect on LinkedIn and hear your feedback. If you completed the challenge as well I would like to see your work.

Cheers!

GitHub
https://github.com/wheelers-websites/CloudGuruChallenge_21.06

LinkedIn
https://www.linkedin.com/in/justin-wheeler-the-cloud-architect/

Discussion (0)