Intro
If you care just a little about technology you are aware of the plethora of different available (many of which quite similar) platforms, raising the need for portability of software products among them. Sometime in the quest for easily provisionable apps accross environments, from the first Makefile to the current orchestrated container, the 12-Factor Methodology was established (though a lot closer in time to the latter).
Its notable principles are the use of declarative setup to facilitate provision automation (like a Dockerfile or Ansible playbook-based pipeline instead of a bash script defining each installation step in detail or a manual setup), a clean contract with the local environment to maximize portability, suitability for deployment in modern cloud platforms, minimum divergence between production & development instances, support for Continuous Deployment and upscaling without major changes to tooling, architecture or development practices.
According to the 12 Factor website, this methodology was developed by Heroku employees, the current standard for Platform-as-a-Service cloud deployment solution, after a lot of competent observation on the problems of scaling, changing and porting web applications.
Most of these recommendations are not confined to cloud-based web development, and many of them are quite intuitive; you probably have seen some of these advices if you followed coding tutorials on the web. However, these recommendations are influent, and now is a chance to think about the why of it.
The 12 Factors
1. Codebase
One codebase in version control for many deploys.
Example: Little Johnny has just developed a feature for his app and committed the changes to local development branch. After passing unit tests, he will then push his local development branch to the staging environment in the cloud to manually test the feature in a scenario closely resembling production, and if all is ok then he will merge the changes to the user-facing production environment.
2. Dependencies
Explicitly declare & isolate dependencies.
Example: using a version manager like rbenv to install Ruby libraries declared in a Gemfile and associating these libraries with rbenv's shimmed version instead of manually installing each gem for the system's version of ruby; using pip to install packages declared in requirements.txt from within a Python virtual environment.
3. Config
Store config in environment, isolating it from the code.
Example: whether you decide to manually export your database address as an environment variable or declare it in a database_addresses.env.local file (though you should do the latter), the codebase (which does not contain hard-coded data and never will) will be able to successfully read the required data in both cases.
4. Backing Services
Treat backing services as attachable resources, easily changing one from another.
Example: when Little Johnny wanted to switch his local MongoDB instance for one hosted in MongoDB's servers, all he had to do was update the database address (which was stored in the environment, isolated from the code) to make the changes, as the app was setup to make no distinction from local or third-party services.
5. Build, Release, Run
Separate build, release & run stages.
Example: Little Johnny's friend directly deployed his codebase to the production environment with an obscure bug which returned an error every time a customer tried to make a purchase, causing him to lose sales and money, very bad! Little Johnny is glad that his deployment pipeline first checks whether the current codebase properly builds without errors, then proceeds to the release phase in which the app loads staging configuration and goes through a set of end-to-end tests and then finally after all tests pass the new code gets deployed to the production environment flawlessly loading the production configuration.
6. Processes
App should be able to run as one (or more) stateless process(es).
Example: Little Johnny wants to apply for a developer position in New Innovation Entrepreneur Company. After filling yet another set of forms asking for previous experience, projects and education, he realizes he has to complete a skill evaluation test, and decides to come back later, shutting down his computer. After logging in again to complete the test, he realizes all of his data has vanished and he has to fill everything back again if he wants to complete all the job application's steps! New Innovation Entrepreneur Company should have been more mindful and written Little Johnny's CV data into perennial storage (like a database) instead of assuming that the cached data in Little Johnny's browser would have been available for a future request (in this case, the skill evaluation test).
7. Port Binding
Export visible and self-containing services through port-binding.
Example: Little Johnny was developing a Java web app and thinking about how he would configure Tomcat to listen to requests and redirect the data into his app... until he remembered this would violate Factor 7!
Instead, he decided to declare Jetty as a dependency, keeping the HTTP service inside the app instead of configuring an external web server and then injecting its functionalities. Now, whenever he wants to instantiate another server for this app, all he has to do is installing dependencies and running the app, isn't it convenient?
8. Concurrency
Instead of vertically scaling a single app instance by transferring it to a very powerful machine, the app should be broken in smaller processes, allowing for horizontal scaling through the process model.
Example: Little Johnny goes to school. In there, the teacher is giving a class on horizontal process scale, giving a PHP app running on an Apache server on top a Linux machine as an example. In a web-app-applied strategy similar to the Unix process model, processes are classified by type and new processes are created according to the required workload: there can be an over-representation of web processes if there is a spike of web server connections, or more clock processes if there is a lot of server task scheduling. The objective is to optimize attention to the task being required at the moment, keeping the workflow dynamic and flexible. While this will usually be done at the OS level away from a web developer's concerns, there are tools such as Foreman which help developers to have more access and control regarding processes and their types. This is a lot more effective than simply having a bunch of generic processes receiving equal system resources instead of optimizing computational power distribution by process type.
9. Disposability
Prioritize fast startup and graceful shutdown; processes should be able to quickly start and stop.
Example: Little Johnny has made a deal with a famous YouTuber to sell water from the Fountain of Youth. They are expecting a very big spike in sales once the YouTuber announces the product in his upcoming live broadcast, and the sales are expected to be about 10% of the spike each day for the upcoming weeks. No problem at all, as Little Johnny orchestrates the existing web containers for the store to multiply at the time of the live then effortlessly downs most of them the next day, keeping the main store functionality essentially what it was from the start, just using more or less server instances to keep up with the required computing power for the moment.
10. Development & Production Parity
Development, staging and production environments should be as similar as possible.
Example: "It works in my machine!", says Little Johnny to his manager at New Innovation Entrepreneur Company. It indeed does: all unit tests are passing and so are the manual tests. However, once deployed to the cloud, there are some unit tests which insist on failing. They then discover that Little Johnny's query is perfectly fine for his local instance of PostgreSQL, but it fails when executed at the MySQL instance that is used for production. They then mirror both environments, try again and... it fails! Turns out some other developer committed code that would make Little Johnny's code cause an error. The manager then assembles the team for a new workflow strategy: they are all using Docker in their personal machines, completely mirroring production stack, and are also committing local changes to a centralized Dev branch every Thursday, to keep the time gap between versions to a minimum.
11. Logs
Logs should be captured as a continuous event stream performed by a service decoupled from the main app.
Example: Little Johnny's server went down... but why? He tries to check the system's logfile but it's so big and disorganized... maybe he should have treated logs as a continuous event stream and timestamp each event to at least know what happened when. After a while, he discovers that the server went down due to a lack of memory, as it was holding a 20 Gb log file... guess it would be better to keep logs as an external service. Little Johnny then configures his environment to use the open-source Fluentd to centralize different log sources and then route the logfile to an external data warehouse. Next time, debugging production issues will be a way less traumatic experience.
12. Admin Processes
Administrative (which in this context means 'general maintenance' and not 'root user') tasks should be one-off and run via the codebase.
Example: Little Johnny developed a store with a quite smart system: he displays at the main page a list of the 10 most sold products during the last hour, which is a quick and easy process since this data gets stored in Redis. Every 60 minutes a locally configured cron job cleans local Redis cache, in the app's codebase there is a feature which rebuilds the list with new data if there is no data in a specific Redis key. The system works wonders until Little Johnny kills his cloud instance for a brief maintenance and quickly gets another one online, however, forgetting to register the Redis-cleaning cron job. The product list is built once and never updated, making him lose sales by missing the advertisement of trending products, until he finally realizes that triggering Redis cache-cleaning should be done from the app codebase instead of relying on OS-level tasks.
Conclusion
You know (or will find out) that these guidelines exist for a reason. Even though following all of them can be bothersome at times, it is very rewarding and time-saving in the longer term. I hope this article has helped you; if you need further clarification from any of these points click on the factor name for a more detailed explanation.
Top comments (1)
Even if I already knew those rules, it's good to have a reminder about them.
Nowadays, they are truly mandatory to understand if you want to design a scalable and robust application!
Thank you!