Hello, folks!
One thing that bothers me when writing about self-hosting, to have greater control of my data, is that I don't apply those principles to the articles themselves. I mean, there is no immediate risk of Minds or Dev.to taking down my publications, but the mere fact that they could do that leaves me concerned
The Right Tool for the Job
First I've looked at the tools I was already familiar with. I have some old blog where I've posted updates during my Google Summer of Code projects. It uses Jekyll to generate static files, automatically published by GitHub Pages. It works very well when you have the website tied to a version-controlled repository, but it's cumbersome when you need to rebuild container images or replace files in a remote volume even for small changes
When looking for something more dynamic, I initially though about using Plume, since it's easy to integrate with some applications I plan to deploy later, but unfortunately it's not well maintained anymore. As Ghost or Wordpress seem overkill, I ended up opting for the conveniences of WriteFreely: it lets me create and edit posts in-place, with Markdown support and no need to upload new files. However, that comes with a cost: it requires a MySQL[-compatible] database
Contrarian Vibes
It seems easy enough to just deploy a MySQL container and use it, right? Well... It seems that there are some concerns about its licensing and development direction ever since the brand has been bought by Oracle (remember OpenOffice?). That was the motivation for the MariaDB fork, distributed under the GPLv2 license, which nowadays is not even a 100% drop-in replacement for MySQL, but still works for our case
Reputation-related shenanigans aside, one great advantage of picking MariaDB is the ability to use a Galera Cluster. Similarly to what we did for PostgreSQL, I wish to be able to scale it properly, and Galera's replication works in an even more interesting manner, with multiple primary (read-write) instances and no need for a separate proxy!:
(Man, I wish PostgreSQL had something similar...)
Of course that requires a more complex setup for the database server itself, but thanks to Bitnami's mariadb-galera Docker image and Helm chart, I've managed to get to something rather manageable for our purposes:
apiVersion: v1
kind: ConfigMap
metadata:
name: mariadb-config
labels:
app: mariadb
data:
BITNAMI_DEBUG: "false" # Set to "true" for more debug information
MARIADB_GALERA_CLUSTER_NAME: galera
# All pods being synchronized (has to reflect the number of replicas)
MARIADB_GALERA_CLUSTER_ADDRESS: gcomm://mariadb-state-0.mariadb-replication-service.default.svc.cluster.local,mariadb-state-1.mariadb-replication-service.default.svc.cluster.local
MARIADB_DATABASE: main # Default database
MARIADB_GALERA_MARIABACKUP_USER: backup # Replication user
---
# Source: mariadb-galera/templates/secrets.yaml
apiVersion: v1
kind: Secret
metadata:
name: mariadb-secret
labels:
app: mariadb
data:
MARIADB_ROOT_PASSWORD: bWFyaWFkYg== # Administrator password
MARIADB_GALERA_MARIABACKUP_PASSWORD: YmFja3Vw # Replication user password
---
apiVersion: v1
kind: ConfigMap
metadata:
name: mariadb-cnf-config
labels:
app: mariadb
data: # Database server configuration
my.cnf: |
[client]
port=3306
socket=/opt/bitnami/mariadb/tmp/mysql.sock
plugin_dir=/opt/bitnami/mariadb/plugin
[mysqld]
explicit_defaults_for_timestamp
default_storage_engine=InnoDB
basedir=/opt/bitnami/mariadb
datadir=/bitnami/mariadb/data
plugin_dir=/opt/bitnami/mariadb/plugin
tmpdir=/opt/bitnami/mariadb/tmp
socket=/opt/bitnami/mariadb/tmp/mysql.sock
pid_file=/opt/bitnami/mariadb/tmp/mysqld.pid
bind_address=0.0.0.0
## Character set
##
collation_server=utf8_unicode_ci
init_connect='SET NAMES utf8'
character_set_server=utf8
## MyISAM
##
key_buffer_size=32M
myisam_recover_options=FORCE,BACKUP
## Safety
##
skip_host_cache
skip_name_resolve
max_allowed_packet=16M
max_connect_errors=1000000
sql_mode=STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE,ONLY_FULL_GROUP_BY
sysdate_is_now=1
## Binary Logging
##
log_bin=mysql-bin
expire_logs_days=14
# Disabling for performance per http://severalnines.com/blog/9-tips-going-production-galera-cluster-mysql
sync_binlog=0
# Required for Galera
binlog_format=row
## Caches and Limits
##
tmp_table_size=32M
max_heap_table_size=32M
# Re-enabling as now works with Maria 10.1.2
query_cache_type=1
query_cache_limit=4M
query_cache_size=256M
max_connections=500
thread_cache_size=50
open_files_limit=65535
table_definition_cache=4096
table_open_cache=4096
## InnoDB
##
innodb=FORCE
innodb_strict_mode=1
# Mandatory per https://github.com/codership/documentation/issues/25
innodb_autoinc_lock_mode=2
# Per https://www.percona.com/blog/2006/08/04/innodb-double-write/
innodb_doublewrite=1
innodb_flush_method=O_DIRECT
innodb_log_files_in_group=2
innodb_log_file_size=128M
innodb_flush_log_at_trx_commit=1
innodb_file_per_table=1
# 80% Memory is default reco.
# Need to re-evaluate when DB size grows
innodb_buffer_pool_size=2G
innodb_file_format=Barracuda
[galera]
wsrep_on=ON
wsrep_provider=/opt/bitnami/mariadb/lib/libgalera_smm.so
wsrep_sst_method=mariabackup
wsrep_slave_threads=4
wsrep_cluster_address=gcomm://
wsrep_cluster_name=galera
wsrep_sst_auth="root:"
# Enabled for performance per https://mariadb.com/kb/en/innodb-system-variables/#innodb_flush_log_at_trx_commit
innodb_flush_log_at_trx_commit=2
# MYISAM REPLICATION SUPPORT #
wsrep_mode=REPLICATE_MYISAM
[mariadb]
plugin_load_add=auth_pam
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mariadb-state
spec:
serviceName: mariadb-replication-service # Use the internal/headless service name
replicas: 2
selector:
matchLabels:
app: mariadb
template:
metadata:
labels:
app: mariadb
spec:
securityContext: # Container is not run as root
fsGroup: 1001
runAsUser: 1001
runAsGroup: 1001
containers:
- name: mariadb
image: docker.io/bitnami/mariadb-galera:11.5.2
imagePullPolicy: "IfNotPresent"
command:
- bash
- -ec
- |
exec /opt/bitnami/scripts/mariadb-galera/entrypoint.sh /opt/bitnami/scripts/mariadb-galera/run.sh
ports:
- name: mdb-mysql-port
containerPort: 3306 # External access port (MySQL's default)
- name: mdb-galera-port
containerPort: 4567 # Internal process port
- name: mdb-ist-port
containerPort: 4568 # Internal process port
- name: mdb-sst-port
containerPort: 4444 # Internal process port
envFrom:
- configMapRef:
name: mariadb-config
env:
- name: MARIADB_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mariadb-secret
key: MARIADB_ROOT_PASSWORD
- name: MARIADB_GALERA_MARIABACKUP_PASSWORD
valueFrom:
secretKeyRef:
name: mariadb-secret
key: MARIADB_GALERA_MARIABACKUP_PASSWORD
volumeMounts:
- name: previous-boot
mountPath: /opt/bitnami/mariadb/.bootstrap
- name: mariadb-data
mountPath: /bitnami/mariadb
- name: mariadb-cnf
mountPath: /bitnami/conf/my.cnf # Overwrite any present configuration
subPath: my.cnf
- name: empty-dir
mountPath: /tmp
subPath: tmp-dir
- name: empty-dir
mountPath: /opt/bitnami/mariadb/conf
subPath: app-conf-dir
- name: empty-dir
mountPath: /opt/bitnami/mariadb/tmp
subPath: app-tmp-dir
- name: empty-dir
mountPath: /opt/bitnami/mariadb/logs
subPath: app-logs-dir
volumes:
- name: previous-boot
emptyDir: {} # Use a fake directory for mounting unused but required paths
- name: mariadb-cnf
configMap:
name: mariadb-cnf-config
- name: empty-dir
emptyDir: {} # Use a fake directory for mounting unused but required paths
volumeClaimTemplates: # Description of volume claim created for each replica
- metadata:
name: mariadb-data
spec:
storageClassName: nfs-small
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 8Gi
---
# Headless service for internal replication/backup processes
apiVersion: v1
kind: Service
metadata:
name: mariadb-replication-service
labels:
app: mariadb
spec:
type: ClusterIP
clusterIP: None
ports:
- name: mariadb-galera-service
port: 4567
targetPort: mdb-galera-port
appProtocol: mysql
- name: mariadb-ist-service
port: 4568
targetPort: mdb-ist-port
appProtocol: mysql
- name: mariadb-sst-service
port: 4444
targetPort: mdb-sst-port
appProtocol: mysql
publishNotReadyAddresses: true
---
# Exposed service for external access
apiVersion: v1
kind: Service
metadata:
name: mariadb-service
spec:
type: LoadBalancer # Let it be accessible inside the local network
selector:
app: mariadb
ports:
- port: 3306
targetPort: mdb-mysql-port
appProtocol: mysql
Incredibly, it works. My deployment has been running without issue for some time now:
$ kubectl get all -n choppa -l app=mariadb
NAME READY STATUS RESTARTS AGE
pod/mariadb-state-0 1/1 Running 2 (3d1h ago) 5d3h
pod/mariadb-state-1 1/1 Running 2 (3d1h ago) 5d3h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mariadb-replication-service ClusterIP None <none> 4567/TCP,4568/TCP,4444/TCP 5d3h
service/mariadb-service LoadBalancer 10.43.40.243 192.168.3.10,192.168.3.12 3306:31594/TCP 5d3h
NAME READY AGE
statefulset.apps/mariadb-state 2/2 5d3h
(The 2 restarts were due to a power outage that exceeded the autonomy of my no-break's battery)
Solving one Problem to Reveal Another
I just started typing my first self-hosted blog post to realize something was missing: images. On Jekyll I had a folder for that, but on Minds and Dev.to they are hosted somewhere else, e.g. https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v2221vgcikcr05hmnj4v.png
If complete self-hosting is a must, I now need some file server capable of generating shareable links, to be used in my Markdown image components. In summary, Syncthing is great for Dropbox-style backups, but can't share links, NextCloud is too resource-heavy and Seafile is interesting but apparently has proprietary encryption, which left me with the lightweight Filebrowser
I don't expect or intend my file server to ever deal with a huge number of requests, so I've ran it as a simple deployment with a single pod:
kind: PersistentVolumeClaim # Storage requirements component
apiVersion: v1
metadata:
name: filebrowser-pv-claim
labels:
app: filebrowser
spec:
storageClassName: nfs-big # The used storage class (1TB drive)
accessModes:
- ReadWriteOnce
#- ReadWriteMany # For concurrent access (In case I try to use more replicas)
resources:
requests:
storage: 200Gi # Asking for a ~50 Gigabytes volume
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebrowser-config
labels:
app: filebrowser
data: # Application settings file
.filebrowser.json: |
{
"port": 80,
"baseURL": "",
"address": "",
"log": "stdout",
"database": "/srv/filebrowser.db",
"root": "/srv"
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: filebrowser-deploy
spec:
replicas: 1
strategy:
type: Recreate # Wait for the old container to be terminated before creating a new one
selector:
matchLabels:
app: filebrowser
template:
metadata:
labels:
app: filebrowser
spec:
# Run this initial container to make sure at least an empty
# database file exists prior to the main container starting,
# as a workaround for a know bug (https://filebrowser.org/installation#docker)
initContainers:
- name: create-database
image: busybox
command: ["/bin/touch","/srv/filebrowser.db"]
volumeMounts:
- name: filebrowser-data
mountPath: /srv
containers:
- name: filebrowser
image: filebrowser/filebrowser:latest
imagePullPolicy: IfNotPresent
ports:
- name: file-port
containerPort: 80
protocol: TCP
volumeMounts:
- name: filebrowser-readonly
mountPath: /.filebrowser.json
subPath: .filebrowser.json
- name: filebrowser-data
mountPath: /srv
volumes:
- name: filebrowser-readonly
configMap:
name: filebrowser-config
- name: filebrowser-data # Label the volume for this deployment
persistentVolumeClaim:
claimName: filebrowser-pv-claim # Reference volumen create by the claim
---
apiVersion: v1
kind: Service
metadata:
name: filebrowser-service
spec:
type: NodePort # Expose the service outside the cluster with an specific port
selector:
app: filebrowser
ports:
- protocol: TCP
port: 8080
targetPort: file-port
nodePort: 30080
(That's what I did here, make it makes the filebrowser.db file end up visible inside the root folder. It's probably a good idea to use subpaths and mount them separately e.g. srv/filebrowser.db and srv/data for root)
We can't upload or access the files from the Internet yet, but using NodePort an external port in the range 30000-32767 can be used to reach it locally. Use the default username admin and password admin to login and then change it in the settings:
Click on each file you wish to share and the option to generate links will appear on the top. In Markdown syntax, shared images may be annexed with the statement 
One Step Forward. Two Steps Back
All set to deploy WriteFreely, right? As you might guess, no
The application doesn't have an official Docker image, and the custom ones available are either too old or not available for the ARM64 architecture. The repository provided by karlprieb is a good base to build your own, but it lead to crashes here when compiling the application itself. In the end, I found it easier to create one taking advantage of Alpine Linux's packages:
- Dockerfile
FROM alpine:3.20
LABEL org.opencontainers.image.description="Simple WriteFreely image based on https://git.madhouse-project.org/algernon/writefreely-docker"
# Install the writefreely package
RUN apk add --no-cache writefreely
# Installation creates the writefreely user, so let's use it
# to run the application
RUN mkdir /opt/writefreely && chown writefreely -R /opt/writefreely
COPY --chown=writefreely:writefreely ./run.sh /opt/writefreely/
RUN chmod +x /opt/writefreely/run.sh
# Base directory and exposed container port
WORKDIR /opt/writefreely/
EXPOSE 8080
# Set the default container user and group
USER writefreely:writefreely
# Start script
ENTRYPOINT ["/opt/writefreely/run.sh"]
- Entrypoint script (run.sh)
#! /bin/sh
writefreely -c /data/config.ini --init-db
writefreely -c /data/config.ini --gen-keys
if [ -n "${WRITEFREELY_ADMIN_USER}" ] && [ -n "${WRITEFREELY_ADMIN_PASSWORD}" ]; then
writefreely -c /data/config.ini --create-admin "${WRITEFREELY_ADMIN_USER}:${WRITEFREELY_ADMIN_PASSWORD}"
fi
writefreely -c /data/config.ini
Here I've published the image to ancapepe/writefreely:latest on DockerHub, so use it if you wish and have no desire for alternative themes or other custom stuff. One more thing to do before running our blog is to prepare the database to receive its content, so log into the MariaDB server on port 3306 using you root user and execute those commands, replacing username and password to your liking:
CREATE DATABASE writefreely CHARACTER SET latin1 COLLATE latin1_swedish_ci;
CREATE USER 'blog' IDENTIFIED BY 'my_password';
GRANT ALL ON writefreely.* TO 'blog';
Now apply a K8s manifest matching previous configurations and adjusting new ones to your liking:
apiVersion: v1
kind: ConfigMap
metadata:
name: writefreely-config
labels:
app: writefreely
data:
WRITEFREELY_ADMIN_USER: my_user
config.ini: |
[server]
hidden_host =
port = 8080
bind = 0.0.0.0
tls_cert_path =
tls_key_path =
templates_parent_dir = /usr/share/writefreely
static_parent_dir = /usr/share/writefreely
pages_parent_dir = /usr/share/writefreely
keys_parent_dir =
[database]
type = mysql
username = blog
password = my_password
database = writefreely
host = mariadb-service
port = 3306
[app]
site_name = Get To The Choppa
site_description = Notes on Conscious Self-Ownership
host = https://blog.choppa.xyz
editor =
theme = write
disable_js = false
webfonts = true
landing = /login
single_user = true
open_registration = false
min_username_len = 3
max_blogs = 1
federation = true
public_stats = true
private = false
local_timeline = true
user_invites = admin
# If you wish to change the shortcut icon for your blog without modifying the image itself, add here the configmap entry generated by running `kubectl create configmap favicon-config --from-file=<your .ico image path>`
binaryData:
favicon.ico: <binary dump here>
---
apiVersion: v1
kind: Secret
metadata:
name: writefreely-secret
data:
WRITEFREELY_ADMIN_PASSWORD: bXlfcGFzc3dvcmQ=
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: writefreely-deploy
spec:
replicas: 1
selector:
matchLabels:
app: writefreely
template:
metadata:
labels:
app: writefreely
spec:
containers:
- name: writefreely
image: ancapepe/writefreely:latest
imagePullPolicy: "IfNotPresent"
ports:
- containerPort: 8080
name: blog-port
env:
- name: WRITEFREELY_ADMIN_USER
valueFrom:
configMapKeyRef:
name: writefreely-config
key: WRITEFREELY_ADMIN_USER
- name: WRITEFREELY_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: writefreely-secret
key: WRITEFREELY_ADMIN_PASSWORD
volumeMounts:
- name: writefreely-volume
mountPath: /data/config.ini
subPath: config.ini
# Use this if you set the custom favicon.ico image above
- name: writefreely-volume
mountPath: /usr/share/writefreely/static/favicon.ico
subPath: favicon.ico
volumes:
- name: writefreely-volume
configMap:
name: writefreely-config
---
apiVersion: v1
kind: Service
metadata:
name: writefreely-service
spec:
publishNotReadyAddresses: true
selector:
app: writefreely
ports:
- protocol: TCP
port: 8080
targetPort: blog-port
(You may add your own favicon.ico to the image itself if you're building it)
Almost there. Now we just have to expose both our blog pages and file server to the Internet by adding the corresponding entries to our ingress component:
apiVersion: networking.k8s.io/v1
kind: Ingress # Component type
metadata:
name: proxy # Component name
namespace: choppa # You may add the default namespace for components as a paramenter
annotations:
cert-manager.io/cluster-issuer: letsencrypt
kubernetes.io/ingress.class: traefik
status:
loadBalancer: {}
spec:
ingressClassName: traefik # Type of controller being used
tls:
- hosts:
- choppa.xyz
- talk.choppa.xyz
- blog.choppa.xyz
- files.choppa.xyz
secretName: certificate
rules: # Routing rules
- host: choppa.xyz # Expected domain name of request, including subdomain
http: # For HTTP or HTTPS requests
paths: # Behavior for different base paths
- path: / # For all request paths
pathType: Prefix
backend:
service:
name: welcome-service # Redirect to this service
port:
number: 8080 # Redirect to this internal service port
- path: /.well-known/matrix/
pathType: ImplementationSpecific
backend:
service:
name: conduit-service
port:
number: 8448
- host: talk.choppa.xyz
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: conduit-service
port:
number: 8448
- host: test.choppa.xyz # Expected domain name of request, including subdomain
http: # For HTTP or HTTPS requests
paths: # Behavior for different base paths
- path: / # For all request paths
pathType: Prefix
backend:
service:
name: test-service # Redirect to this service
port:
number: 80 # Redirect to this internal service port
- host: blog.choppa.xyz
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: writefreely-service
port:
number: 8080
- host: files.choppa.xyz
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: filebrowser-service
port:
number: 8080
If everything went accordingly, you now have everything in place to log into your blog and start publishing. To get an idea of how your self-hosted articles will look like, pay a visit to the first chapter of this series that I'm starting to publish on my server as well:
Thanks for following along. See you next time
Top comments (0)