DEV Community: Paritosh

DockerHub RateLimiting; Solving via Nexus OSS

Paritosh — Sun, 16 Jan 2022 17:41:35 +0000

What is this about?

Docker is continuously moving towards commercialising their services, which is in a way correct thing to do to keep the ship sailing. One such change they brought in November 2020 is to rate limit the API calls being made to Docker Hub for Docker image pulls.
How we got impacted?
At our organisation, we run our CI system at fairly large scale, where there are thousands of developers actively consume the central CI system for their code commits. All the repos are Docker compatible hence most of them have rely on DockerHub for official images like centos, java, node, python, etc.

So, when this rate limiting was introduced it intermittently impacted our CI system where builds started failing with exceptions from Docker Hub like below :-

ERROR: toomanyrequests: Too Many Requests.
You have reached your pull rate limit. 
You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits. 
You must authenticate your pull requests.

Solution: Sonatype Nexus — Docker Registry

In my opinion Sonatype Nexus is an amazing, must have tool in a large scale CI system. We started using Nexus for proxying Java, NodeJS and also hosting private packages too.
So it was an obvious choice to use Nexus for solving DockerHub rate limiting issue. By proxying DockerHub and reducing dependency on DockerHub.

Docker Hub is the common registry used by all image creators and consumers. To reduce duplicate downloads and improve download speeds for your developers and CI servers, you should proxy Docker Hub and any other registry you use for Docker images.

We created a proxy repo in Nexus for DockerHub and added a small configuration to mirror the registry in our CI servers (which are autoscaled, so it was a change in the AMI).

How it works?

The first time you request an image from your local registry mirror, it pulls the image from the public Docker registry and stores it in Nexus before handing it back to the CI servers. On subsequent requests, the Nexus registry mirror is able to serve the image from its own storage.
/etc/docker/daemon.json

{
  "registry-mirrors": ["http://nexus.domain.org:18000"]
}

Simple and effective !

For detailed information on configuration do check the official documentation of Nexus here https://help.sonatype.com/repomanager3/formats/docker-registry/proxy-repository-for-docker

Looking for More…!!

Pls do follow. I am planning to post more such blogs which are practical and scalable solutions around Docker, AWS, Python.
In case any help is required for setup or approach pls don’t think twice, before asking me.

Cheers!

Deploying Django Apps; Containers

Paritosh — Sun, 22 Oct 2017 14:58:30 +0000

Intro

Lately I observed some weird practices that my fellow Engineers follow for developing and especially deploying their Django application in production.

I did bit of research about best practices for a Django project development life cycle & below is what I feel the most reliable way to develop & deploy a Django application end-to-end.

I don't intend to define the coding practices...

Though trying to set the perspective of creating new project & shed some light to small details that we tend to miss that eventually lead to bad Engineering.

Unit test cases in Django

Writing test cases is a tool that developer do ignore or don't plan ahead and hence there are repeated iterations to make sure that application functionality is working fine after every major code change.

Unit tests are important if the application's scope is wide. Though this practice is being followed for larger application but overlooked for small or medium scale projects.

Code style enforcement using Flake8

Flake8 points out the errors & violations like a variable declared but never used in a class. It is very handy and helps in setting fundamental guidelines for every developer in the team.

Creating Docker images for Django application

Containerisation has huge benefits & being the torch bearer of Docker in my organisation I have figured that the best possible solution to deploy Django
applications is to create Docker images & ship them to production.

Creating a Dockerfile that define setting up python environment required by the Django application. Using this for creating Docker images is the most reliable way to ship & deploy Django application in production.

Setting Docker Registry using Harbor

In the process to get Docker adopted in my team, it was necessary to get an in-house docker registry which people can use to push the Docker images of the projects they were working on.

VMware's Harbor is an enterprise grade registry server which supports LDAP based user login to Docker registry. That is Engineers can use their existing user credentials to use in-house Docker registry.

Code review & Jenkins CI for a Django project

Code review process using conventional tools like Gerrit & Jenkins offer good features to automate code verification and Docker image creation part using above mentioned strategy.

This is the most reliable way to develop, ship & deploy applications as per my best knowledge of Django. Do share if there are any other better strategies or tools to achieve this common goal.

LB Console for BigIp F5

Paritosh — Sat, 14 Oct 2017 17:15:24 +0000

Introduction

LB Console is an application that wraps functionalities provided by BigIp F5 UI to configure load balancer. BigIP F5 load balancer is one of the extensively used load balancer (at least in e-commerce domain).

Problem statement

Though BigIp F5 has very effective UI but in terms of usability users are required to have some understanding of how F5 works.

Things get much complex for teams who want to integrate monitoring application, deployment application & other planned activities. As there is a learning curve required to understand basic working of F5 plus the API integration at multiple levels feels like an overhead for the consumer.

Solution

An application that performs all required transactions with BigIp F5. That exposes REST endpoints that can be used for integration with other applications.

Have a simpler UI that is user friendly & fast. A tracking mechanism that keeps log of all critical transaction.

Tech stack

Python Django framework
PyControl
ElasticSearch
Python Boto3
Docker

RESTful approach

Implementing a simple RESTful approach proved to be extremely useful as it enabled teams to easily integrate the same with deployment strategies and moving application from data centres to cloud (AWS & OpenStack).

Django DB Caching

As the volume of pools and servers configured at F5 were moderatley large ~3k servers under ~1K pools, read calls from LB Console to BigIp F5 were considerable. Hence to minimise # of calls to F5 Django DB based caching came in handy.

This enabled the application to query directly to F5 only when the required data was not present in the cache. Result from every F5 query being saved in cache with a short TTL (300s) ensured that data being served almost instantly & F5 query calls only in the event of cache miss occurrence.

ElasticSearch based logging

ElasticSearch based logging each of the create/update/delete transaction helped in keeping critical data readily available for consumers. User interface being the primary consumer to display logs per server or pool, later the same was integrated with our IRC channel for users to query the logs from IRC itself.

Docker

Conventionally Django projects were shipped as tar files in production or in some cases deployment via Git was practiced. So to bring some reliablility and standardisation in deployment approach Docker came very handy plus usual benefits of using containerisation.

LB Console application also solves complex problems like:

Traffic switching within data centres.
Managing DNS entries in AWS Route53.