5 Self-Hosting Projects for Infrastructure Specialists: Real-World

The infrastructure business isn't something you can learn solely from books or online courses. The real world has its own dynamics, its own strange problems. In my 20-year career, I learned the most from projects where I said "it's fine," worked on them all night, and tinkered with them myself. Especially if you're an aspiring infrastructure specialist or want to solidify your existing knowledge, self-hosting projects on your own server, with your own resources, are invaluable.

In this post, I'll talk about 5 self-hosting projects that will truly make a difference in your career and get your hands dirty. These aren't just about installing an application; each one will give you in-depth experience in areas like networking, security, system administration, and software architecture. I learned a lot while implementing these projects on my own VPS, and sometimes even on an old Raspberry Pi.

1. Running Your Own DNS and Ad-Blocking Server

DNS is the cornerstone of the internet. 80% of network problems often stem from a misconfigured DNS server or an incorrectly cached record. Many times, I've found the root cause of payment gateway access issues on a large e-commerce site to be faulty DNS resolution. Setting up your own DNS server isn't just about blocking ads; it helps you understand how DNS works, how queries are resolved, caching mechanisms, and even modern security layers like DoH (DNS over HTTPS) / DoT (DNS over TLS).

My preference is usually Pi-hole or AdGuard Home. While setting them up, I learned about integrating with a DHCP server, resolving local network device names by using conditional forwarding, and even defining separate DNS policies for some sensitive devices. I remember once detecting and blocking an IoT device on my home network that was sending excessive telemetry, which I identified from my own DNS logs. These types of projects provide not only technical knowledge but also proactive monitoring and troubleshooting skills.

ℹ️ Practical Example: Troubleshooting with DNS Logs

I once saw from my Pi-hole logs that an IoT device was communicating with external IP addresses unexpectedly. This allowed me to detect not only network traffic but also potential security vulnerabilities early on. A log entry like the following immediately reveals the anomaly:

Jun 13 14:30:05 dnsmasq[12345]: query[A] tracking.iotprovider.com from 192.168.1.100
Jun 13 14:30:05 dnsmasq[12345]: gravity blocked tracking.iotprovider.com is 0.0.0.0
Jun 13 14:30:06 dnsmasq[12345]: query[A] data-upload.iotprovider.com from 192.168.1.100
Jun 13 14:30:06 dnsmasq[12345]: /etc/pihole/gravity.list data-upload.iotprovider.com is 0.0.0.0

Regularly following these logs is critical to understanding the "chatter" of devices on your network.

In this project, I also better understood the importance of network segmentation. Issues like which device uses which DNS server and which traffic is allowed to go out form the basis of firewall policies and VLAN segmentation that we implement in corporate networks. For my own side project, I frequently used DNS logs to monitor incoming traffic.

2. Setting Up a Simple CI/CD Pipeline

CI/CD (Continuous Integration/Continuous Deployment) is now indispensable in the world of software development and operations. But often, tools like Jenkins, GitLab CI, or GitHub Actions that we use in corporate projects hide the infrastructure details from us. Setting up your own simple CI/CD system helps you understand the mechanics behind this magic. While working on a production ERP, I personally experienced how much we needed a reliable CI/CD pipeline for rapid and error-free deployment of constantly changing business rules.

I usually prefer self-hosted alternatives like Gitea Actions or Drone CI. I created such a pipeline on my own VPS to automatically deploy a side project of mine, which has a small FastAPI backend and a Vue.js frontend. During this process, I learned to write Dockerfiles, orchestrate services with docker compose, manage services with systemd units, and even deal with OOM (Out Of Memory) errors during builds. Adjusting cgroup limits like MemoryHigh and MemoryMax for a systemd service and analyzing journald logs were some of these experiences.

💡 Simple Dockerfile Example

When containerizing your own application, it's crucial to understand how layers in a Dockerfile are cached and how build times are optimized as a result. A simple Dockerfile like the one below shows how to package a basic Python application:

# Select the base image
FROM python:3.9-slim-buster

# Set the working directory
WORKDIR /app

# Copy and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code
COPY . .

# Define the command to run the application
CMD ["python", "app.py"]

When you run docker build -t myapp . with this Dockerfile, your application turns into a container image.

In this project, you also start thinking about the fundamentals of blue-green or canary deployment strategies. Initially, even a simple rolling update is a success in itself. I personally saw how much time a CI/CD pipeline saved and how it reduced the error rate in small updates I made to my own site.

3. Creating a Monitoring and Alerting System

For an infrastructure specialist, there's nothing worse than being "blind." Are your systems running, is the disk full, is the CPU spiking, is there a slowdown in the network? Without knowing these, you can't be anything more than reactive. Early in my career, I remember times when I didn't notice a disk filling up to 100% until it caused a critical service to crash. Since that day, monitoring has become an obsession for me.

Setting up the Prometheus, Grafana, and Alertmanager trio on my own VPS or home server gave me incredible experience. In this project, I learned to collect server metrics with node_exporter, monitor PostgreSQL metrics with pg_exporter, and visualize this data on Grafana dashboards. With Alertmanager, I configured it to receive notifications via Slack or email when my defined thresholds were exceeded. For example, I receive an alert immediately if a PostgreSQL replication lag exceeds a certain number of seconds.

⚠️ WAL Bloat and Disk Fullness Alert

In PostgreSQL databases, WAL (Write-Ahead Log) files are vital for replication and recovery. However, misconfiguration or long-running transactions can lead to WAL bloat and quickly fill up the disk. Monitoring this situation with Prometheus and receiving alerts with Alertmanager is critical. In a client's project, a WAL rotation alarm went off at 03:14 AM on April 28th, and thanks to this, we were able to intervene before the disk filled up.

# prometheus.yml snippet for PostgreSQL WAL monitoring
scrape_configs:
  - job_name: 'postgresql'
    static_configs:
      - targets: ['localhost:9187'] # pg_exporter port

# alertmanager.yml snippet for WAL bloat alert
groups:
  - name: 'postgresql_alerts'
    rules:
      - alert: HighWALDiskUsage
        expr: pg_wal_size_bytes / pg_database_size_bytes{datname="your_db"} > 0.10
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "PostgreSQL WAL disk usage is high ({{ $labels.instance }})"
          description: "WAL files occupy more than 10% of the database size on {{ $labels.instance }}."

This type of alert provides an opportunity for proactive intervention and prevents major problems.

Thanks to these projects, I developed metric collection and analysis capabilities that form the basis of SLO (Service Level Objective) and Error Budget management in corporate environments. I also learned to prevent log services from consuming excessive resources by adjusting journald rate limit settings.

4. Setting Up a Database Cluster (PostgreSQL Replication)

Databases are the heart of every application. High availability and data integrity are undisputed priorities, especially in enterprise software. While working on a production ERP, I saw how database outages directly led to production downtime and how high the cost of that was. Therefore, setting up and managing database replication is a topic every infrastructure specialist must experience.

For the backend of my own side project, I set up physical replication (streaming replication) on PostgreSQL. During this process, I learned to take a copy from the primary server with pg_basebackup, configure postgresql.conf and pg_hba.conf settings, create the recovery.conf (or standby.signal in modern PostgreSQL versions) file, and monitor the replication status. I remember once how WAL files accumulated on the primary server and filled up the disk due to a wrong setting, and how I temporarily resolved this situation with the pg_wal_replay_pause() function.

💡 PostgreSQL Replication Setup Steps

Physical replication is a critical component for database high availability. Here are the basic steps and important commands:

1. postgresql.conf settings on the Primary Server:

ini
wal_level = replica
archive_mode = on
archive_command = 'cp %p /mnt/server/archivedir/%f' # Archive WAL files
max_wal_senders = 10 # Number of replication connections allowed
wal_keep_size = 5GB # wal_keep_segments in older versions
listen_addresses = '*' # Allow external connections


1. pg_hba.conf setting: Grant permissions for the replication user.


host replication replicator 192.168.1.0/24 md5


1. Create replication user on the primary server:

sql
CREATE USER replicator REPLICATION LOGIN CONNECTION LIMIT 10 ENCRYPTED PASSWORD 'your_password';


1. Create Standby Server:

bash
pg_basebackup -h primary_ip -U replicator -D /var/lib/postgresql/14/main -P -Xs -R


This command takes an initial backup from the primary server and automatically creates the `standby.signal` file to set up the standby server.

1. Monitor replication:

sql
SELECT * FROM pg_stat_replication;


This project also opens the door to advanced database topics such as connection pool tuning, read replica routing strategies, and partitioning strategies. In a client's project, I saw how the primary database slowed down due to heavy reporting load and how I solved this problem by redirecting to read replicas. Database performance regressions and vacuum monitoring were also part of these experiences.

5. Configuring Your Own VPN or ZTNA Gateway

Remote access and network security are indispensable in today's business world. This topic has become even more critical, especially after the pandemic. I set up my own VPN gateway to both secure my own network and understand the logic behind corporate VPN topologies. This allowed me to personally experience the fundamental principles of ZTNA (Zero Trust Network Access) architectures used for accessing sensitive systems like "an internal banking platform."

I set up a VPN server on my own VPS using solutions like WireGuard or OpenVPN. During this process, I learned to write iptables rules, understand NAT (Network Address Translation) and routing tables, and even monitor authentication attempts with tools like fail2ban. Additionally, segmenting the network and controlling egress to limit access to certain resources allowed me to see the practical application of zero-trust principles. I once tunneled all traffic leaving my home network through my own VPN, preventing my ISP from monitoring my DNS queries.

🔥 Misconfiguration and Security Risk

When setting up a VPN/ZTNA gateway, an incorrect iptables rule or incomplete segmentation can expose your network to unexpected risks. Especially leaving ACCEPT instead of DROP in the forward chain can allow unwanted traffic to enter.

# Only forward traffic from the VPN interface to the LAN (example)
sudo iptables -A FORWARD -i wg0 -o eth0 -j ACCEPT
sudo iptables -A FORWARD -i eth0 -o wg0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
# By default, block all other forward traffic
sudo iptables -P FORWARD DROP

Writing and testing such rules carefully is vital for your network security. Using fail2ban to monitor failed VPN login attempts and temporarily block IP addresses also increases the security layer.

These experiences helped me better understand the firewall policies, VLAN segmentation, and switch hardening techniques like IP source guard that we use in corporate networks. In my own Android spam application, when protecting backend API requests with rate limiting, I used the security knowledge gained from these VPN experiences.

Conclusion: Experience is Priceless

These 5 projects will not only give you technical skills but also develop your problem-solving ability, debugging processes, and most importantly, your habit of pursuing the "why." You will personally see how everything you read in books can manifest in different ways in the real world. Last month, I was OOM-killed in the backend of my own side project due to a sleep 360 command and how I fixed it with a polling-wait mechanism. Such mistakes are the best learning opportunities.

Remember, at the core of infrastructure expertise lies curiosity, patience, and continuous learning. These projects will take you a step further on this journey and make you much more prepared for the real-world problems you will encounter in your career. Failures will be your best teachers. Go ahead, rent a VPS or set up an old computer, and embark on this adventure.

Comments

[The Lazy Girl (⁠◕⁠ᴗ⁠◕⁠✿⁠)]

❤️