loading...
Cover image for wal-e Postgres Backups - Building SaaS #32

wal-e Postgres Backups - Building SaaS #32

mblayman profile image Matt Layman Originally published at mattlayman.com ・2 min read

Building SaaS (48 Part Series)

1) Building SaaS with Python and Django #1 2) Reporting scraped data - Building SaaS #2 3 ... 46 3) Multiple Stripe plans - Building SaaS #3 4) Using a background scheduler - Building SaaS #4 5) Updating data models - Building SaaS #5 6) Third party integration modeling - Building SaaS #6 7) Displaying third party data - Building SaaS #7 8) Connecting third party services - Building SaaS #8 9) Finishing third party integration - Building SaaS #9 10) Admin dashboards - Building SaaS #10 11) Semi-automated tasks - Building SaaS #11 12) Automation aides - Building SaaS #12 13) Deploying with Ansible - Building SaaS #13 14) Ansible Cranked to 11 - Building SaaS #14 15) Feature Flags with Django Waffle - Building SaaS #15 16) Feature Flags in Action - Building SaaS #16 17) Canceling Stripe Subscriptions - Building SaaS #17 18) Completing Account Deactivation - Building SaaS #18 19) Pip-tools and App Packaging - Building SaaS #19 20) Making a Shiv App - Building SaaS #20 21) Shiv zipapps and CI - Building SaaS #21 22) Upload to S3 with CircleCI orbs - Building SaaS #22 23) It's Alive! A Django Shiv app - Building SaaS #23 24) In the Guts of a Shiv App - Building SaaS #24 25) It's Permissions, Dummy! - Building SaaS #25 26) Connecting Shiv Apps with Ansible - Building SaaS #26 27) Plug the Shiv App Into Nginx - Building SaaS #27 28) Webpack and collectstatic in CI - Building SaaS #28 29) Add Static Assets to Deployment - Building SaaS #29 30) Ripping Out Node.js - Building SaaS #30 31) Celery In A Shiv App - Building SaaS #31 32) wal-e Postgres Backups - Building SaaS #32 33) Get Out, Git! - Building SaaS #33 34) Bring in the WhiteNoise, Bring in Da Funk - Building SaaS #34 35) Deploying WhiteNoise - Building SaaS #35 36) Configurama - Building SaaS #36 37) Lessons From A Failed SaaS - Building SaaS #37 38) New Project, Who Dis? - Building SaaS #38 39) django-environ and django-debug-toolbar - Building SaaS #39 40) Make A Custom User Model - Building SaaS #40 41) User Accounts With django-allauth - Building SaaS #41 42) Add Styles To Templates - Building SaaS #42 43) Use Tailwind On A Template - Building SaaS #43 44) Fast Forms With UpdateView - Building SaaS #44 45) Templates and Logic - Building SaaS #45 46) A Week At A Time - Building SaaS #46 47) How To Style Sign Up - Building SaaS #47 48) Onboarding - Building SaaS #48

In this episode, we worked on Postgres database backups and modified the backup tool, wal-e, to use the Shiv app format.

wal-e is Postgres database backup tool that works by managing the Write-Ahead Log (WAL) that a Postgres database produces. The WAL is the log of recent changes that happened in the database. If you have access to a full WAL, then you can conceivably reproduce a Postgres database's data.

When I switched the deployment to use a Shiv app, we now have to remove every use of the existing virtual environment in the system. The wal-e executable is installed in the bin directory of the virtual environment so we need another way to access it.

The way that the Shiv app can execute other binaries is with the [SHIV_ENTRY_POINT](https://shiv.readthedocs.io/en/latest/#shiv-entry-point) environment variable. Using the setuptools-style link of wal_e.cmd:main, we can run the wal-e executable that is available to the Shiv app. Because I already use the envdir command, we only had to make two changes:

  1. Replace {{ venv_path }}/bin/wal-e with /srv/apps/conductor.pyz.
  2. Add a SHIV_ENTRY_POINT file to /etc/wal-e/env with the appropriate setuptools-style value.

Once we made the changes, I deployed things to the staging site with Ansible. From there, we verified that the cron entry was changes with:

$ sudo su -  # change to root
$ su - postgres  # change to postgres account
$ crontab -l  # list cron entries

With the wal-e updates in place, nothing used the Python virtual environment so we could finally delete it!

In the next stream, we're going to begin the process of removing the Git clone to complete the process of simplifying deployment.

This article first appeared on mattlayman.com.

Building SaaS (48 Part Series)

1) Building SaaS with Python and Django #1 2) Reporting scraped data - Building SaaS #2 3 ... 46 3) Multiple Stripe plans - Building SaaS #3 4) Using a background scheduler - Building SaaS #4 5) Updating data models - Building SaaS #5 6) Third party integration modeling - Building SaaS #6 7) Displaying third party data - Building SaaS #7 8) Connecting third party services - Building SaaS #8 9) Finishing third party integration - Building SaaS #9 10) Admin dashboards - Building SaaS #10 11) Semi-automated tasks - Building SaaS #11 12) Automation aides - Building SaaS #12 13) Deploying with Ansible - Building SaaS #13 14) Ansible Cranked to 11 - Building SaaS #14 15) Feature Flags with Django Waffle - Building SaaS #15 16) Feature Flags in Action - Building SaaS #16 17) Canceling Stripe Subscriptions - Building SaaS #17 18) Completing Account Deactivation - Building SaaS #18 19) Pip-tools and App Packaging - Building SaaS #19 20) Making a Shiv App - Building SaaS #20 21) Shiv zipapps and CI - Building SaaS #21 22) Upload to S3 with CircleCI orbs - Building SaaS #22 23) It's Alive! A Django Shiv app - Building SaaS #23 24) In the Guts of a Shiv App - Building SaaS #24 25) It's Permissions, Dummy! - Building SaaS #25 26) Connecting Shiv Apps with Ansible - Building SaaS #26 27) Plug the Shiv App Into Nginx - Building SaaS #27 28) Webpack and collectstatic in CI - Building SaaS #28 29) Add Static Assets to Deployment - Building SaaS #29 30) Ripping Out Node.js - Building SaaS #30 31) Celery In A Shiv App - Building SaaS #31 32) wal-e Postgres Backups - Building SaaS #32 33) Get Out, Git! - Building SaaS #33 34) Bring in the WhiteNoise, Bring in Da Funk - Building SaaS #34 35) Deploying WhiteNoise - Building SaaS #35 36) Configurama - Building SaaS #36 37) Lessons From A Failed SaaS - Building SaaS #37 38) New Project, Who Dis? - Building SaaS #38 39) django-environ and django-debug-toolbar - Building SaaS #39 40) Make A Custom User Model - Building SaaS #40 41) User Accounts With django-allauth - Building SaaS #41 42) Add Styles To Templates - Building SaaS #42 43) Use Tailwind On A Template - Building SaaS #43 44) Fast Forms With UpdateView - Building SaaS #44 45) Templates and Logic - Building SaaS #45 46) A Week At A Time - Building SaaS #46 47) How To Style Sign Up - Building SaaS #47 48) Onboarding - Building SaaS #48

Posted on Mar 19 by:

mblayman profile

Matt Layman

@mblayman

Matt Layman is a software engineer from Frederick, MD. He is an open source software maintainer and advocate for Python.

Discussion

markdown guide