DEV Community: Mihai Farcas

The $3/Month Enterprise-Ready Automation Stack: Zero-Trust n8n With One Command (n8n, Cloudflare, UpCloud, Pulumi)

Mihai Farcas — Tue, 07 Apr 2026 17:44:04 +0000

I run one command and get a fully working n8n instance — HTTPS, zero-trust tunnel, PostgreSQL, the works. One more command and it's all gone. Total cost: $3/month.

No SSH. No open ports. No clicking around in dashboards.

Here's exactly how it works.

The Problem With Self-Hosted Automation

Every self-hosting tutorial I find tells me to do the same thing: open ports 80 and 443, set up Nginx, wrestle with Let's Encrypt, and pray nobody scans my server before I've hardened it.

Then there's the database. Most n8n guides default to SQLite. It works — until two workflows fire at the same time and the whole thing locks up.

I don't want to babysit a server. I want to deploy, use it, and tear it down when I'm done. No manual steps. No leftover infrastructure.

So I built exactly that.

What You Get

One pulumi up command creates:

A $3/month UpCloud server (1 CPU, 1 GB RAM, 10 GB storage, Ubuntu 24.04)
A Cloudflare Tunnel — zero-trust, outbound-only connection. No open ports.
Automatic HTTPS and DDoS protection via Cloudflare's edge
PostgreSQL 18 for concurrent workflow execution
n8n running behind the tunnel, accessible at your custom domain
DNS records pointed at the tunnel automatically

One pulumi destroy removes everything. Server, tunnel, DNS — gone.

Why Zero-Trust Matters Here

Traditional setups expose your server to the internet. You open ports, configure firewalls, manage certificates. Every open port is an attack surface.

Cloudflare Tunnels flip this model. The cloudflared daemon on your server creates an outbound-only connection to Cloudflare's network. Traffic flows in reverse:

User → Cloudflare (HTTPS + DDoS) → Tunnel → n8n container

Your server has zero open inbound ports. No firewall rules to manage. No certificates to renew. Cloudflare handles all of it at the edge.

The 6-Pillar Stack

Each tool has one job:

Pulumi — Infrastructure as Code. Provisions everything.
UpCloud — $3/month compute (Frankfurt, DE).
Cloudflare — Zero-trust tunnel + HTTPS + DDoS protection.
Docker — Container runtime via Docker Compose.
n8n — Workflow automation engine.
PostgreSQL 18 — Production database for concurrent execution.

How It Wires Together

The entire deployment is 180 lines of TypeScript. Here's what happens when you run pulumi up:

1. Pulumi Creates the Cloudflare Tunnel

const tunnel = new cloudflare.ZeroTrustTunnelCloudflared("n8n-tunnel", {
  accountId: cfAccountId,
  name: "n8n-tunnel",
  configSrc: "cloudflare",
  tunnelSecret: crypto.randomBytes(32).toString("base64")
});

A 32-byte random secret gets generated. The tunnel routes traffic from your domain to http://n8n:5678 inside the Docker network.

2. The Tunnel Token Gets Fetched via API

const tunnelToken = cloudflare.getZeroTrustTunnelCloudflaredTokenOutput({
  accountId: cfAccountId,
  tunnelId: tunnel.id
});

This is the key trick. Instead of SSHing into the server after boot to inject the token, Pulumi fetches it directly from Cloudflare's API. The token gets baked into the cloud-init script — zero post-deploy steps.

3. DNS Points to the Tunnel

const dnsRecord = new cloudflare.DnsRecord("n8n-dns", {
  zoneId: cfZoneId,
  name: domain,
  type: "CNAME",
  content: pulumi.interpolate`${tunnel.id}.cfargotunnel.com`,
  proxied: true
});

A proxied CNAME record. Cloudflare's edge handles TLS termination and DDoS filtering before traffic ever reaches your server.

4. Cloud-Init Bootstraps the Server

Pulumi builds a cloud-init script that:

Installs Docker Engine
Writes docker-compose.yml with all secrets baked in
Writes the tunnel token to .env
Runs docker compose up -d

The server boots, runs this script, and 60 seconds later n8n is live. No SSH required.

5. Three Containers, One Compose File

services:
  postgres:
    image: postgres:18-alpine
    # Health check ensures n8n waits for DB

  n8n:
    image: n8nio/n8n:latest
    depends_on:
      postgres:
        condition: service_healthy
    # Full PostgreSQL config, HTTPS via tunnel

  cloudflared:
    image: cloudflare/cloudflared:latest
    command: tunnel run
    # Token from .env file

PostgreSQL starts first. n8n waits for the health check. cloudflared connects the tunnel. The Docker network handles service discovery — n8n:5678 just works.

The Setup (5 Minutes)

Prerequisites

An UpCloud account (API credentials)
A Cloudflare account with a domain (API token with Zone:DNS:Edit + Account:Tunnels:Edit)
Pulumi CLI installed
Node.js 18+

Deploy

# Clone and install
cd infra && npm install

# Create .env from the template (in the project root)
cp ../.env.example ../.env
# Edit .env — fill in your API tokens, domain, and Cloudflare IDs
# SSH key is auto-detected from ~/.ssh/ (no config needed)

# Load env vars and set secrets
set -a && source ../.env && set +a
pulumi config set --secret postgresPassword "$(openssl rand -hex 16)"
pulumi config set --secret n8nBasicAuthUser "admin"
pulumi config set --secret n8nBasicAuthPassword "$(openssl rand -hex 16)"
pulumi config set --secret n8nEncryptionKey "$(openssl rand -hex 32)"

# Deploy everything
pulumi up

That's it. Visit https://n8n.yourdomain.com and log in.

Tear Down

pulumi destroy

Server, tunnel, DNS — all removed. Nothing left running. Nothing left billing.

What About Persistence?

PostgreSQL data is saved as a Docker volume on the server itself. Your workflows, credentials, and execution history survive container restarts and docker compose down / up cycles.

What they don't survive is pulumi destroy — that deletes the server and everything on it.

For production, consider switching to a managed PostgreSQL service (UpCloud, Aiven, Neon) or mounting resilient block storage for the database volume. Either option decouples your data from the server lifecycle.

What About n8n Cloud?

n8n Cloud is the easiest way to get started. Managed hosting, automatic updates, built-in auth, and zero infrastructure to think about. If you don't want to manage servers, it's the right choice.

This self-hosted stack is a different tradeoff. You get full control over the environment — custom domains, your own database, and the ability to spin up or tear down the whole thing in seconds. You also manage the infrastructure.

Both are valid paths. Pick the one that fits how you work.

The Full Lifecycle

Here's the entire flow, start to finish:

You (one-time): Fill in a .env file (API tokens, domain, Cloudflare IDs). Run 4 secret config commands. SSH key is auto-detected.

Pulumi (automated): Creates tunnel → fetches token → configures routes → creates DNS → builds cloud-init → provisions server.

Server (automated): Boots → installs Docker → writes configs → starts containers.

Result: n8n live at your domain with HTTPS, zero-trust networking, and PostgreSQL. No SSH. No open ports. $3/month.

Done? pulumi destroy. Everything gone.

Want to try it? The full source code is on GitHub. Clone the repo, set your API keys, and deploy in 5 minutes.

Need help with your setup? I build and consult on automation infrastructure professionally. Reach out and let's talk about your use case.

Automate Your Homelab Like a Pro with Ansible!

Mihai Farcas — Wed, 02 Apr 2025 07:02:04 +0000

Spending too much time manually configuring servers in your homelab? Wish you could automate setup, updates, and deployments reliably? Ansible is a powerful yet simple, agentless automation tool that can tame your infrastructure, whether it's a personal lab or a production environment.

This article walks through practical examples directly from my latest YouTube video, showing you how to automate Proxmox updates, VM setups, PostgreSQL configuration, and even handle secrets securely with Ansible Vault.

If you'd rather watch a youtube video than read:

If not, let's dive in!

Why Ansible for Your Homelab (and Beyond)?

So, why choose Ansible over manual configurations or complex scripts? Here are a few compelling reasons:

Simplicity & Readability

Ansible uses YAML for its playbooks, which is incredibly easy for humans to read and write. You describe the state you want your systems to be in, rather than scripting every single step. This makes your automation understandable and maintainable.

Agentless Architecture

This is a huge advantage! You don't need to install or manage special agent software on your servers (nodes). Ansible communicates primarily over standard SSH for Linux/Unix systems (or WinRM for Windows), using Python on the remote end (which is usually already there). This means less setup, less resource overhead, and a smaller attack surface.

Idempotency: The Automation Superpower

As highlighted in the video, Ansible operations are typically idempotent. This means you can run a playbook multiple times, and it will only make changes if needed to reach the desired state. Running it again won't break things or cause unintended side effects. This is absolutely crucial for reliable and predictable automation – run your updates, and only the nodes that need updating get touched!

Power & Flexibility ("Batteries Included")

Don't let the simplicity fool you. Ansible comes with a vast library of built-in modules that handle thousands of common tasks – installing packages (apt, yum), managing services (service, systemd), copying files (copy), generating configs from templates (template), managing users (user), interacting with cloud providers, configuring databases (like PostgreSQL!), and much more. You describe what you want done, and the module handles the how.

Getting Started: Prerequisites

Before you start automating, you need a couple of things set up.

1. On Your Control Machine (Where you run Ansible):

Python 3: Ansible is built with Python. Make sure you have Python 3 installed. You can check with:
```
python3 --version
```
Ansible: Install Ansible itself. Using pip (Python's package manager) is often recommended for getting the latest version: pip3 install ansible. Alternatively, use your OS package manager (like apt or brew). Check your installation with:
```
ansible --version
```
(Optional) Ansible VS Code Extension: If you use Visual Studio Code, the official Ansible extension by Red Hat provides excellent syntax highlighting and autocompletion.
Ansible Packages: Some Ansible community collections might need to be installed separately using commands like ansible-galaxy collection install community.postgresql for example.

2. Access to Your Target Nodes (Your Homelab Servers):

SSH Access: Since Ansible is agentless for Linux/Unix, it relies on SSH to connect and execute commands. You need SSH access from your control machine to your homelab servers.
SSH Key Authentication (Recommended): For seamless automation, set up SSH key-based authentication so Ansible doesn't need to ask for passwords. The easiest way is to copy your public SSH key (e.g., ~/.ssh/id_ed25519.pub) to your target nodes using ssh-copy-id. Replace root@<server_ip> with the appropriate user and IP for each server:
```
ssh-copy-id -i ~/.ssh/id_ed25519.pub root@<your_homelab_server_ip>
```
(Note: For Windows targets, Ansible uses WinRM instead of SSH, which requires different setup steps not covered in detail here.)

Understanding Your Ansible Project: Structure & Core Concepts

A well-structured Ansible project makes automation easier to manage and scale. Here's a typical layout, incorporating the core concepts:

your-ansible-project/
├── ansible.cfg         # Project-specific Ansible configuration
├── inventory/          # Defines WHAT servers Ansible manages
│   └── hosts.ini       # Your inventory file(s)
├── playbooks/          # Contains your automation instructions (playbooks)
│   ├── proxmox.yml
│   ├── postgres.yml
│   └── ...
├── vars/               # Stores variables, including secrets
│   ├── common_vars.yml
│   └── secrets.yml     # Encrypted secrets using Ansible Vault
└── roles/              # (Optional) Reusable units of automation (Not covered in detail here)

[Image: Simple diagram showing this directory structure]

Let's break down each part:

inventory/ (The "What"): This directory holds your Inventory file(s) (often hosts.ini or inventory.yml). This tells Ansible which servers (nodes) it should manage. You can list servers individually or group them logically (like [proxmox_nodes] or [webservers]). Your script showed an example inventory.ini:
```
[proxmox_nodes]
pve ansible_host=192.168.31.100
srv-1 ansible_host=192.168.31.101
srv-2 ansible_host=192.168.31.110 # The new node

[k3s_nodes]
k3s-master-1 ansible_host=192.168.31.50 ansible_user=k3s ansible_become_password="{{ k3s_password }}"
# ... other nodes ...

[all:vars]
# Global variables can go here
ansible_user=root # Default user if not specified per host/group
```
[Note: Variables like ansible_host, ansible_user, and group variables ([all:vars]) can also be defined here.]

playbooks/ (The "How"): This is where your Playbooks live. Playbooks are written in YAML and define a series of Tasks to be executed on hosts defined in your inventory. Each task typically uses an Ansible Module (the building blocks like apt, service, template, command, postgresql_user) to perform a specific action. Remember, good modules are idempotent. A basic task structure looks like this:

---
- name: Example Play # Descriptive name for the whole play
  hosts: proxmox_nodes # Which group(s) from inventory to target
  become: true # Execute tasks with root privileges (like sudo)

  tasks:
    - name: Update Proxmox # Descriptive name for this task
      apt: # The module being used
        update_cache: yes # Module parameters
        upgrade: dist

vars/ (The Data & Secrets): Store your variables here, separating them from your playbooks for better organization. You can have files for common variables (common_vars.yml) and crucially, for secrets (secrets.yml). Never commit plain-text passwords! Use Ansible Vault to encrypt sensitive files:

# Create a new encrypted file
ansible-vault create vars/secrets.yml

# Edit an existing encrypted file
ansible-vault edit vars/secrets.yml

# Run playbook asking for password
# ansible-playbook my_playbook.yml --ask-vault-pass

# Or better, use a password file (add vault.pass to .gitignore!)
ansible-playbook my_playbook.yml --vault-password-file=./vault.pass

Your secrets.yml might contain things like db_password: your_secret_password.

When you run ansible-vault create or edit, Ansible opens your default text editor ($EDITOR). Inside the editor, you just write standard YAML key-value pairs with your secrets, like this:

```yaml
# vars/secrets.yml
# --- This is just plain YAML while you edit! ---
k3s_password: "your_k3s_sudo_password"
app1_db_password: "supersafepassword123"
some_api_key: "xyz789abc123"
some_secret_value: 1234
```

Once you save and close the editor, Ansible encrypts the content. The actual file on disk becomes unreadable text, which is safe to commit to version control. This file is encrypted and can be decrypted with the password you picked when creating the vault!

ansible.cfg (The Settings): This optional file defines project-specific Ansible settings, overriding system defaults. Your script mentioned useful settings:

[defaults]
inventory = ./inventory/hosts.ini  # Default inventory file location
remote_user = root              # Default user to connect as
private_key_file = ~/.ssh/id_ed25519 # Default SSH key
host_key_checking = False       # Disable host key checking (use with caution!). Acceptable in trusted homelabs, but avoid in production - Disabling key checking skips verification of the remote host's identity, making you vulnerable to man-in-the-middle attacks.

[privilege_escalation]
become = True                   # Run tasks with privileges by default
become_method = sudo            # How to gain privileges
become_user = root              # Which user to become
become_ask_pass = False         # Don't ask for sudo password (assumes NOPASSWD)

Organizing your project this way keeps things tidy and makes your automation easier to understand, maintain, and share.

Homelab Automation in Action: Examples

Theory is great, but let's see Ansible solve real homelab problems, based on the examples from the YouTube video.

Before we look at example, do me a solid favor and check out my YouTube Channel, Let's Talk Dev

Let's Talk Dev - YouTube

I share my insights and tips on software engineering topics such as web development, databases, cloud computing, and more. Hi, I'm Mihai and I'm a full-stack software engineer with a passion for creating innovative and user-friendly solutions using web and cloud technologies. I have a home lab where I like to experiment with Kubernetes and other cutting-edge technologies. I'm also a big fan of open-source software. When I'm not coding, I enjoy photography, reading books, learning about finance and entrepreneurship, and watching movies.

youtube.com

Who knows, you might find those videos helpful and even subscribe!

1. Fixing Proxmox Subscription Error by Adding the Non-Subscription Repositories

Problem: Dealing with the Proxmox "no subscription" warnings by switching repositories, and keeping all Proxmox nodes consistently updated.
Solution: Use Ansible's dedicated apt_repository module to manage APT sources correctly, and the apt module for updates.

Fixing Repositories (using apt_repository):

# playbooks/proxmox_config.yml (Example Snippet)
---
- name: Fix Proxmox repository errors by adding the no-subscription repository
  hosts: proxmox_nodes
  become: true
  gather_facts: false # Facts not strictly needed for this part
  # vars_files: # Only needed if secrets were used here
  #  - ../vars/secrets.yml

  tasks:
    - name: Disable Proxmox enterprise repository
      ansible.builtin.apt_repository:
        repo: "deb [https://enterprise.proxmox.com/debian/pve](https://enterprise.proxmox.com/debian/pve) bookworm pve-enterprise" # Adjust 'bookworm' for your PVE version
        state: absent
        filename: pve-enterprise # Explicitly target the correct file

    - name: Add Proxmox no-subscription repository
      ansible.builtin.apt_repository:
        repo: "deb [http://download.proxmox.com/debian/pve](http://download.proxmox.com/debian/pve) bookworm pve-no-subscription" # Adjust 'bookworm'
        state: present
        filename: pve-no-subscription # Use a distinct filename

    - name: Disable Proxmox enterprise ceph repository (if applicable)
      ansible.builtin.apt_repository:
        repo: "deb [https://enterprise.proxmox.com/debian/ceph-quincy](https://enterprise.proxmox.com/debian/ceph-quincy) bookworm enterprise" # Adjust 'quincy'/'bookworm'
        state: absent
        filename: ceph-enterprise # Explicitly target the correct file

    - name: Add Proxmox no-subscription ceph repository (if applicable)
      ansible.builtin.apt_repository:
        repo: "deb [http://download.proxmox.com/debian/ceph-quincy](http://download.proxmox.com/debian/ceph-quincy) bookworm no-subscription" # Adjust 'quincy'/'bookworm'
        state: present
        filename: ceph-no-subscription # Use a distinct filename

    - name: Update apt cache after repo changes
      ansible.builtin.apt:
        update_cache: true

Explanation: This automation uses the ansible.builtin.apt_repository module, which is the idiomatic way to manage APT repositories. state: absent ensures the enterprise repositories are removed or commented out in their respective files (specified by filename), while state: present ensures the no-subscription repositories are added correctly. Finally, apt update_cache: true refreshes the package list.

2. Keeping Proxmox up to date!

Upgrading Proxmox Nodes (upgrade-proxmox.yml logic):

# playbooks/proxmox_upgrade.yml (Example Snippet)
---
- name: Upgrade Proxmox Nodes
  hosts: proxmox_nodes
  become: true

  tasks:
    - name: Update apt cache and perform dist-upgrade
      ansible.builtin.apt:
        update_cache: yes # Good practice before upgrade
        upgrade: dist # Handles kernel/PVE upgrades correctly
        autoclean: yes # Clean up downloaded package files
        autoremove: yes # Remove unused dependencies

    - name: Check if reboot is required
      ansible.builtin.stat:
        path: /var/run/reboot-required
      register: reboot_required_file

    - name: Reboot the server if required
      ansible.builtin.reboot:
        msg: "Reboot initiated by Ansible due to OS updates"
        connect_timeout: 5
        reboot_timeout: 600 # Wait up to 10 mins for reboot
        pre_reboot_delay: 0
        post_reboot_delay: 30 # Wait 30s after reboot before continuing
        test_command: uptime # Command to test if server is back up
      when: reboot_required_file.stat.exists # Only reboot if flag file exists

Explanation: This automation uses the apt module for the upgrade process (upgrade: dist is important for Proxmox). Crucially, it checks if the system flagged a required reboot (stat module) and then uses the reboot module only when that flag exists (when condition). Idempotency ensures nodes already up-to-date are simply skipped by the apt module.

3. Managing VMs Consistently

Problem: Ensuring all your VMs (e.g., Ubuntu servers running Docker or k3s) have essential tools like Python and specific libraries installed.

Solution: Target a group of VMs in your inventory and use the apt module.

# playbooks/vm_setup.yml (Example Snippet)
---
- name: Setup Base Packages on VMs
  hosts: k3s_nodes # Or a group like [all_vms]
  become: true # Needs sudo/root privileges
  # Assumes ansible_become_password might be needed (defined in inventory/vault)

  tasks:
    - name: Install Python, Pip, and other required packages
      ansible.builtin.apt:
        name:
          - python3
          - python3-pip
          - python3-psycopg2 # For PostgreSQL interaction later
          - python3-docker   # For Docker interaction later
        state: present
        update_cache: yes

Explanation: This playbook targets a specific group (k3s_nodes). The apt module ensures the listed packages are installed (state: present). If a package is already installed, Ansible makes no changes.

4. Automating PostgreSQL Initialization and Configuration

Problem: Manually creating database users and databases for different applications is repetitive and error-prone.

Solution: Use service-specific Ansible modules for more complex tasks.

# playbooks/postgres_setup.yml (Example Snippet)
---
- name: Configure PostgreSQL Users and Databases
  hosts: database_servers # Assuming a group for your DB server(s)
  become: true
  become_user: postgres # Run postgres tasks as the 'postgres' OS user, alternatively if postgres runs as root, keep using root.

  vars_files:
    - ../vars/secrets.yml # Load sensitive variables like passwords

  tasks:
    - name: Ensure database user exists for App1
      community.postgresql.postgresql_user:
        name: app1_user
        password: "{{ app1_db_password }}" # Password from vault
        state: present
      # This module handles creating the user idempotently

    - name: Ensure database exists for App1
      community.postgresql.postgresql_db:
        name: app1_db
        owner: app1_user
        state: present
      # This module handles creating the database idempotently

Explanation: This example uses modules from the community.postgresql collection (postgresql_user, postgresql_db). It runs tasks as the postgres system user (become_user) and pulls the password from an Ansible Vault encrypted file (vars_files and {{ app1_db_password }}). These modules are designed to be idempotent.

These examples show how Ansible moves beyond simple commands to reliably manage configurations, updates, and service setups across your homelab infrastructure.

Conclusion

As we've seen, Ansible is a fantastic tool for bringing order and efficiency to your homelab (and potentially your day job!). By defining your infrastructure as code, you reduce tedious manual work, ensure configurations are consistent across machines, and gain confidence through idempotent operations. The examples here – fixing Proxmox repos, updating nodes, setting up VMs, and configuring PostgreSQL – are just the beginning. Think about initial server setups, Docker container deployments, or keeping k3s clusters up-to-date; Ansible can handle it all.

Remember, the key concept is idempotency: run your playbooks as often as you need, knowing they'll only make changes necessary to reach your desired state.

I hope this gives you a solid starting point! For the full walkthrough and live demos, make sure to check out the companion Ansible Automation YouTube video.

What's the first thing you plan to automate in your homelab using Ansible? Let me know in the comments below!

If you found this article and the video helpful, consider subscribing to Let's Talk Dev on YouTube for more software engineering and DevOps content, and follow me here on Dev.to! Happy automating!

About me

I'm Mihai Farcas, a software architect with over a decade of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Functional Programming Face-Off: Python vs JavaScript vs Go!

Mihai Farcas — Wed, 05 Mar 2025 20:50:20 +0000

Functional programming has been gaining popularity in recent years, and for good reason. It offers a number of benefits over traditional imperative programming, such as improved code readability, testability, and maintainability.

In this article, we'll compare and contrast how three popular programming languages - Python, JavaScript, and Go - handle functional programming concepts. We'll explore how each language supports features like higher-order functions, immutability, and function composition, and discuss their strengths and weaknesses.

If you'd rather watch than read:

Check out the accompanying YouTube video!

1. Defining and Calling Functions

At the heart of functional programming lies, unsurprisingly, the function. The way functions are defined and invoked can significantly impact a language's suitability for functional programming paradigms. We'll examine how Python, JavaScript, and Go handle function definitions, looking at syntax, typing, and overall structure, setting the stage for more advanced functional concepts. Specifically how similar (or different) the syntax is, and the implications of different approaches to type systems (static, dynamic, type hints).

Python

def square(i: int) -> int:
    return i * i

result = square(5)
print(result)

In Python, functions are defined using the def keyword followed by the function name, parentheses for parameters, and a colon. The code block within the function is indented. Type hints can be optionally added to specify the expected types of parameters and return values.

JavaScript

const square = i => i * i;

const result = square(5);
console.log(result);

JavaScript offers a more compact syntax for defining functions, particularly with arrow functions. Arrow functions use the => operator, providing a concise way to express a function's logic.

Go

package main

import (
    "fmt"
)

func square(i int) int {
    return i * i
}

func main() {
    result := square(5)
    fmt.Println(result)
}

Go, a statically-typed language, requires explicit type declarations for both function parameters and return values. Functions are defined using the func keyword, followed by the function name, a parenthesized list of parameters (with their types), and the return type. The function body is enclosed in curly braces {}. Go's syntax is very similar to python in that it uses a keyword, followed by the name, followed by parameters. However, Go's strict typing and the need for explicit imports, even for basic operations like printing to the console (using the fmt package), introduce a slightly higher level of formality compared to Python and JavaScript. The Go example defines a function that, like the Python and JavaScript counterparts, takes an integer and returns its square.

However, Go differs significantly in its overall structure. Every Go file belongs to a package, and the package main declaration is special. It signifies that the file is part of the main package, which is the entry point for executable programs. Without package main and the func main() within it, Go wouldn't know where to begin executing your code. It's a fundamental difference from Python and JavaScript, where you don't need such explicit entry point declarations for simple scripts. Another key distinction is Go's requirement for explicit imports, even for basic functionalities like printing to the console. In the provided example, fmt.Println is used, requiring an import of the fmt (format) package.

2. Higher-Order Functions

This section delves into a cornerstone of functional programming: treating functions as first-class citizens. This means functions can be passed as arguments to other functions, returned as values from functions, and generally treated like any other variable. A direct consequence of this is the ability to create higher-order functions – functions that either take other functions as arguments or return them (or both). This allows for highly flexible and reusable code. We will also be touching on Closures, though the dedicated section for them comes later.

Python

def applyOperation(operation, a: int, b: int) -> int:
    return operation(a, b)


def add(a: int, b: int) -> int:
    return a + b


def subtract(a: int, b: int) -> int:
    return a - b


result = applyOperation(add, 10, 5)
print(result)  # Output: 15

result = applyOperation(subtract, 10, 5)
print(result)  # Output: 5

Python supports higher-order functions elegantly. In the Python snippet, applyOperation is a higher-order function because it takes another function (operation) as an argument. add and subtract are regular functions that perform addition and subtraction, respectively. applyOperation then calls the passed-in function (operation) with the provided arguments (a and b). Notice that the type hints are optional in Python. While the example does use type hints for clarity (specifying that operation should be a function taking two integers and returning an integer, and that a and b are integers), they are not enforced at runtime by default. This makes the Python code concise and similar in that aspect to JavaScript.

JavaScript

function applyOperation(operation, a, b) {
  return operation(a, b);
}

function add(a, b) {
  return a + b;
}

function subtract(a, b) {
  return a - b;
}

let result = applyOperation(add, 10, 5);
console.log(result); // Output: 15

result = applyOperation(subtract, 10, 5);
console.log(result); // Output: 5

JavaScript, like Python, treats functions as first-class citizens and readily supports higher-order functions. The JavaScript snippet demonstrates the same applyOperation, add, and subtract structure as the Python example. applyOperation accepts a function (operation) and calls it with arguments a and b. Because JavaScript is dynamically typed, there are no type annotations. The code is very concise and mirrors the Python example in its functional approach.

Go

package main

import "fmt"

func applyOperation(operation func(int, int) int, a int, b int) int {
    return operation(a, b)
}

func add(a, b int) int {
    return a + b
}

func subtract(a, b int) int {
    return a - b
}

func main() {
    result := applyOperation(add, 10, 5)
    fmt.Println(result) // Output: 15

    result = applyOperation(subtract, 10, 5)
    fmt.Println(result) // Output: 5
}

Go also supports higher-order functions, although with its characteristic explicitness due to static typing. In the Go snippet, applyOperation is again the higher-order function. However, the type of the operation parameter is explicitly declared: func(int, int) int. This signifies that operation must be a function that takes two int arguments and returns an int. This explicit type declaration is a key difference from Python (with optional type hints) and JavaScript (dynamically typed). The add and subtract functions are defined with their types, and applyOperation calls the provided function just like in the other languages. While all three languages achieve the same result, Go's static typing makes the function signatures more verbose. This reinforces Go's emphasis on explicitness and compile-time type safety.

3. Closures

Closures are a powerful feature often used in conjunction with higher-order functions. A closure is a function that "remembers" its surrounding environment (its lexical scope) even after the outer function that created it has finished executing. This means the inner function can access and modify variables defined in the outer function's scope, even if the outer function has returned. This ability to "capture" state is crucial for many functional programming patterns.

Python

def createCounter():
    count = 0

    def counter():
        nonlocal count  # count is not a local variable,
        # it refers to the count variable in the outer scope
        count += 1
        return count

    return counter


counter = createCounter()
print(counter())
print(counter())

The Python snippet demonstrates a classic closure example with createCounter. createCounter is a function that returns another function, counter. The key here is the nonlocal count declaration within counter. Because Python has stricter scoping rules, you need to explicitly tell the inner function (counter) that count is not a local variable within counter itself, but rather refers to the count variable in the enclosing *scope of createCounter. Without nonlocal, Python would create a new, local count variable inside counter, and the outer count would remain unchanged. The returned counter function *"closes over" the count variable, maintaining its state between calls.

JavaScript

const createCounter = () => {
  let count = 0;
  return () => {
    count++;
    return count;
  };
};

const counter = createCounter();
console.log(counter());
console.log(counter());

JavaScript handles closures very naturally. In the JavaScript snippet, createCounter also returns an anonymous function (the closure). This inner function increments and returns the count variable. Crucially, JavaScript's scoping rules automatically allow the inner function to access and modify the count variable from the createCounter scope, even after createCounter has completed. There's no need for a special keyword like Python's nonlocal; the closure behavior is implicit. This makes the JavaScript code very concise.

Go

package main

import "fmt"

func createCounter() func() int {
    count := 0
    return func() int {
            count++
            return count
    }
}

func main() {
    counter := createCounter()
    fmt.Println(counter())
    fmt.Println(counter())
}

Go's closure mechanism is similar to JavaScript's. The Go snippet defines createCounter, which returns an anonymous function that increments and returns the count variable. Like JavaScript, and unlike Python, there's no need for any special declaration to indicate that count is from the outer scope. The inner function automatically has access to and can modify count. The returned function "closes over" the count variable, preserving its state between invocations. Go's approach, like JavaScript's, is straightforward and doesn't require extra keywords for closure behavior. The syntax, returning a func() int, clearly expresses that createCounter returns a function that takes no arguments and returns an integer.

4. Partial Application

Partial application is a technique for creating new functions from existing ones by pre-filling some of their arguments. It's a way to specialize a general function into a more specific one. Instead of calling a function with all its arguments at once, you "fix" some of the arguments, creating a new function that expects only the remaining arguments. This can lead to more reusable and readable code.

Python

def createGreeting(greeting: str):
    def greetingFn(name: str) -> str:
        return f"{greeting} {name}"

    return greetingFn


firstGreeting = createGreeting("Well, hello there ")
secondGreeting = createGreeting("Hola ")
print(firstGreeting("Remi"))
print(secondGreeting("Ana"))

In the Python snippet, createGreeting is a function that demonstrates partial application. It takes a greeting string as input and returns a new function (greetingFn). This returned function takes a name string and combines it with the pre-filled greeting. When you call createGreeting("Well, hello there "), you're not getting the final greeting string; you're getting a new function (firstGreeting) that has the "Well, hello there " greeting already "baked in." You can then call firstGreeting with different names to get the complete greeting. The same principle applies to secondGreeting, prefilling a different initial greeting. The type hints help show the flow: createGreeting takes a string and returns a function that takes a string and returns a string.

JavaScript

const createGreeting = (greeting) => {
  return (name) => {
    return `${greeting} ${name}`;
  };
};

const firstGreeting = createGreeting("Well, hello there ");
const secondGreeting = createGreeting("Hola ");
console.log(firstGreeting("Remi"));
console.log(secondGreeting("Ana"));

The JavaScript snippet achieves the same result using a concise arrow function syntax. createGreeting takes a greeting and returns another arrow function that takes a name. This inner function uses template literals (backticks) to create the final greeting string. Like the Python example, calling createGreeting with a greeting creates a new, specialized function (firstGreeting, secondGreeting) with that greeting pre-applied. The nested arrow functions clearly illustrate the process of creating a function that returns another function.

Go

package main

import "fmt"


func createGreeting(greeting string) func(string) string {
    return func(name string) string {
            return fmt.Sprintf("%s %s", greeting, name)
    }
}

func main() {
    firstGreeting := createGreeting("Well, hello there ")
    secondGreeting := createGreeting("Hola ")
    fmt.Println(firstGreeting("Remi"))
    fmt.Println(secondGreeting("Ana"))
}

The Go snippet implements partial application in a similar manner. createGreeting takes a greeting string and returns a function that takes a name string and returns the combined string. Go's explicit type signature, func(string) string, clearly indicates that createGreeting returns a function. The use of fmt.Sprintf is Go's way of formatting strings, analogous to Python's f-strings and JavaScript's template literals. The result is the same: firstGreeting and secondGreeting become specialized functions with their respective greetings pre-filled. Go's syntax, while slightly more verbose due to type declarations, achieves the same functional goal.

5. Currying

Currying is a technique closely related to partial application, but with a subtle, yet important, difference. While partial application allows you to pre-fill any number of arguments, currying transforms a function that takes multiple arguments into a sequence of functions, each taking a single argument. Each function in the sequence returns the next function in the sequence, until all arguments have been supplied, at which point the final result is returned. Currying isn't built-in to any of these languages; we need to implement a curry function ourselves.

Python

def curry(fn):
    def curried(*args):
        if len(args) >= fn.__code__.co_argcount:
            return fn(*args)
        else:
            return lambda *args2: curried(*(args + args2))

    return curried


def add(a: int, b: int, c: int) -> int:
    return a + b + c


result = curry(add)(1)(2)(3)
print(result)  # Output: 6

The Python snippet implements a general-purpose curry function. This function is more complex than the previous examples because it needs to handle functions with an arbitrary number of arguments. It uses fn.__code__.co_argcount to determine the number of arguments the original function (fn) expects. The inner curried function uses *args to collect arguments. If enough arguments have been collected (the length of args is greater or equal to the number of arguments that fn expects), it calls the original function fn with those arguments. Otherwise, it returns a new lambda function that takes more arguments (*args2), combines them with the existing arguments (args + args2), and recursively calls curried again. This recursive process continues until all arguments are provided. The add function, which takes three arguments, is then curried and called with one argument at a time: curry(add)(1)(2)(3).

JavaScript

function curry(fn) {
  return function curried(...args) {
    if (args.length >= fn.length) {
      return fn.apply(this, args);
    } else {
      return (...args2) => curried.apply(this, args.concat(args2));
    }
  };
}

function add(a, b, c) {
  return a + b + c;
}

const result = curry(add)(1)(2)(3);
console.log(result); // Output: 6

The JavaScript snippet provides a similar curry function implementation. It uses fn.length to get the number of arguments the original function expects. The curried function uses the rest parameter syntax (...args) to gather arguments. If enough arguments are present, it calls the original function fn using apply (to handle the this context correctly). Otherwise, it returns a new function that takes more arguments (...args2), concatenates them with the existing arguments using concat, and recursively calls curried. Like the Python version, this allows us to call a curried function with one argument at a time, as shown with curry(add)(1)(2)(3). The Javascript and Python versions are strikingly similar.

Go

The Go snippet presents a significantly more complex implementation of curry. This is where the differences between Go's static typing and the dynamic typing of Python and JavaScript become most apparent. Because Go is statically typed, creating a generic curry function that works with functions of any arity (number of arguments) and any type requires using the reflect package. Reflection allows us to inspect and manipulate types at runtime, which is necessary because the curry function doesn't know the types of the function it's currying beforehand.

package main

import (
    "fmt"
    "reflect"
)


func curry(f interface{}) interface{} {
    ft := reflect.TypeOf(f)
    if ft.Kind() != reflect.Func {
        panic("curry: argument must be a function")
    }

    numArgs := ft.NumIn()
    args := make([]reflect.Value, 0, numArgs)

    var curried func(x reflect.Value) interface{}
    curried = func(x reflect.Value) interface{} {
        args = append(args, x)
        if len(args) == numArgs {
            return reflect.ValueOf(f).Call(args)[0].Interface()
        } else {
            return curried
        }
    }

    return curried
}

func add(a, b, c int) int {
    return a + b + c
}

func main() {
    curriedAdd := curry(add)

    add1 := curriedAdd.(func(reflect.Value) interface{})(reflect.ValueOf(1))
    add12 := add1.(func(reflect.Value) interface{})(reflect.ValueOf(2))
    result := add12.(func(reflect.Value) interface{})(reflect.ValueOf(3)).(int)

    fmt.Println(result) // Output: 6
}

The curry function first checks if the input f is actually a function using reflect.TypeOf(f).Kind() != reflect.Func. It then determines the number of expected arguments using ft.NumIn(). It uses a slice args (of type reflect.Value) to store the accumulated arguments. The curried function (a closure) is defined recursively. When called with an argument x, it appends x to args. If enough arguments have been collected, it calls the original function using reflect.ValueOf(f).Call(args)[0].Interface(). This line is crucial: it converts the function f to a reflect.Value, calls it with the collected arguments, extracts the result (which is also a reflect.Value), and then converts it back to a regular Go interface{} using .Interface(). If not enough arguments have been collected, it returns itself (curried), continuing the currying process.

The main function shows how to use the curried function. Because the curry function returns the most generic type interface{}, we need multiple type assertions (e.g., .(func(reflect.Value) interface{})) to convert the intermediate curried functions to their correct types before we can call them. This is much more verbose than the Python and JavaScript examples, highlighting the added complexity of implementing currying in a statically-typed language without generics (prior to Go 1.18). The final result is retrieved and printed, demonstrating that the currying works, but at the cost of significantly more complex code. This example clearly shows that functional programming patterns like currying, while possible in Go, are often more natural and concise in dynamically typed languages.

6. Immutability

Immutability is a core principle of functional programming. An immutable object is one whose state cannot be changed after it's created. This contrasts with mutable objects, which can be modified in place. Immutability simplifies reasoning about code, prevents unintended side effects, and is crucial for concurrent programming (as immutable data is inherently thread-safe). Different languages have varying levels of built-in support for immutability.

Python

import copy

points1 = [{"x": 1, "y": 2}, {"x": 3, "y": 4}]

# Shallow Copy
points2 = copy.copy(points1)
points2.append({"x": 5, "y": 6})  # Modifies only points2
points2[0]["x"] = 10  # Modifies the object in both lists

print(points1)  # Output: [{'x': 10, 'y': 2}, {'x': 3, 'y': 4}] (affected!)
print(points2)  # Output: [{'x': 10, 'y': 2}, {'x': 3, 'y': 4}, {'x': 5, 'y': 6}]

# Deep Copy
points3 = copy.deepcopy(points1)
points3[0]["x"] = 20  # Modifies only points3

print(points1)  # Output: [{'x': 10, 'y': 2}, {'x': 3, 'y': 4}] (unaffected)
print(points3)  # Output: [{'x': 20, 'y': 2}, {'x': 3, 'y': 4}]

Python's built-in data structures, like lists and dictionaries, are mutable. This means you can modify them directly after creation. The Python snippet highlights the importance of understanding the difference between shallow and deep copies when working with nested mutable objects.

Shallow Copy (copy.copy()): A shallow copy creates a new top-level object, but the elements within that object are still references to the original objects. So, if you have a list of dictionaries, a shallow copy will create a new list, but the dictionaries inside the new list will be the same dictionaries as in the original list. Modifying a nested object in the shallow copy will affect the original object as well. As shown, appending to points2 affects only the copy because append creates a new array, but modifying points2[0]["x"] modifies the underlying object referenced by both points1 and points2.
Deep Copy (copy.deepcopy()): A deep copy recursively creates new copies of all objects, including nested ones. This means modifying a nested object in a deep copy will not affect the original object. As shown, after a deep copy, modifying points3[0]["x"] only affects points3, leaving points1 unchanged. You need to explicitly use copy.deepcopy() to achieve true immutability with nested mutable objects in Python.

JavaScript

const points1 = [{ x: 1, y: 2 }, { x: 3, y: 4 }];
const points2 = [...points1]; // Creates a shallow copy

points2.push({ x: 5, y: 6 }); // Modifies only points2
points2[0].x = 10; // Modifies the object in both arrays

console.log(points1); // Output: [{ x: 10, y: 2 }, { x: 3, y: 4 }] (affected!)
console.log(points2); // Output: [{ x: 10, y: 2 }, { x: 3, y: 4 }, { x: 5, y: 6 }]

JavaScript, like Python, has mutable objects (arrays and objects) by default. The JavaScript snippet uses the spread operator (...) to create a shallow copy of an array of objects.

Spread Operator for Arrays: The spread operator, when used with an array, creates a shallow copy. Similar to Python's copy.copy(), it creates a new array, but the objects within the array are still references to the same objects in the original array. Appending to points2 doesn't modify points1, as a new array instance is created, but changing points2[0].x does modify the corresponding object in points1 because it's a shared reference.
Deep Copying in JavaScript: JavaScript doesn't have a built-in deep copy function like Python's copy.deepcopy(). You typically need to use a custom function or a library (like Lodash's _.cloneDeep) to perform a deep copy. One common (but potentially problematic) approach is to use JSON.parse(JSON.stringify(object)), but this only works for objects that can be safely serialized to JSON (no functions, circular references, etc.). The lack of a standard, built-in deep copy mechanism is a notable weakness of JavaScript in terms of supporting immutability.

Go

package main

import "fmt"

type Point struct {
    X int
    Y int
}

func main() {
    points1 := []Point{{1, 2}, {3, 4}}
    points2 := points1 // Creates a copy of the slice header and underlying array

    // Appending creates a new slice, doesn't modify points1
    points2 = append(points2, Point{5, 6})

    // Modifying an element in points2 creates a copy of that element.
    points2[0].X = 10

    fmt.Println(points1) // Output: [{1 2} {3 4}] (original unaffected)
    fmt.Println(points2) // Output: [{10 2} {3 4} {5 6}]
}

Go takes a different approach to immutability. Go's structs are value types. When you assign a struct to a new variable or pass it to a function, a copy of the struct is created. This promotes immutability by default.

Value Types: In the Go snippet, Point is a struct. When points2 := points1 is executed, points2 receives a copy of the slice header. Importantly, in this case, the underlying array elements (the Point structs) are also copied because they are value types.
append: The append function in Go is designed to work well with immutability. When you append to a slice, Go might create a new underlying array if the existing array doesn't have enough capacity. This means that append often returns a new slice, leaving the original slice untouched. As demonstrated, points2 = append(points2, Point{5, 6}) creates a new slice for points2, leaving points1 unmodified.
Modifying Struct Fields: Because of the value semantics of structs, changing a field within the struct inside a slice in Go doesn't change the value of the struct in another.
In the example provided, modifying points2, does not affect points1.

Go's design, with its emphasis on value types for structs and the behavior of append, encourages immutability. While you can use pointers to create mutable behavior in Go, the language's default behavior for structs promotes immutability, making it easier to write code with fewer side effects. This is a significant advantage of Go for functional programming.

7. Map, Filter, Reduce

Map, filter, and reduce are fundamental higher-order functions in functional programming that operate on collections of data (like lists or arrays). They provide a concise and declarative way to transform and process data without using explicit loops.

Map: Applies a given function to each element of a collection, producing a new collection with the transformed elements. The original collection remains unchanged.
Filter: Creates a new collection containing only the elements from the original collection that satisfy a given condition (a predicate function).
Reduce: Combines all elements of a collection into a single value using a specified function. This function takes an accumulator and the current element, updating the accumulator with each iteration.

Python

from functools import reduce

# Map: Square each number
numbers = [1, 2, 3, 4, 5]
squares = list(map(lambda n: n * n, numbers))
print("Squares:", squares)

# Filter: Get even numbers
evens = list(filter(lambda n: n % 2 == 0, numbers))
print("Evens:", evens)

# Reduce: Sum all numbers
sum_of_numbers = reduce(lambda acc, n: acc + n, numbers, 0)  # 0 is the initial value
print("Sum:", sum_of_numbers)

Python has built-in support for map, filter, and reduce.

map: The map function takes a function and an iterable (like a list) as arguments. In the example, lambda n: n * n is an anonymous function (a lambda) that squares a number. map applies this function to each element of the numbers list. The result of map is an iterator, so we use list() to convert it to a list for printing.
filter: The filter function also takes a function and an iterable. The function (lambda n: n % 2 == 0) acts as a predicate, returning True for even numbers and False otherwise. filter returns an iterator containing only the elements for which the predicate returns True. Again, we use list() to convert it to a list.
reduce: reduce is not directly built-in; it's part of the functools module. It takes a function, an iterable, and an optional initial value. The function (lambda acc, n: acc + n) takes the accumulator (acc) and the current element (n), returning the updated accumulator. reduce applies this function cumulatively to the items of the sequence, from left to right, so as to reduce the sequence to a single value. In the snippet, it calculates the sum of all numbers in the list.

JavaScript

// Map: Square each number
const numbers = [1, 2, 3, 4, 5];
const squares = numbers.map(n => n * n);
console.log("Squares:", squares);

// Filter: Get even numbers
const evens = numbers.filter(n => n % 2 === 0);
console.log("Evens:", evens);

// Reduce: Sum all numbers
const sum = numbers.reduce((acc, n) => acc + n, 0); // 0 is the initial value
console.log("Sum:", sum);

JavaScript also provides map, filter, and reduce as built-in methods on arrays. This makes the syntax very concise and readable.

map: The map method is called directly on the numbers array. It takes a callback function (n => n * n) that's applied to each element, creating a new array (squares) with the results.
filter: The filter method, similarly, is called on the array. It takes a predicate function (n => n % 2 === 0) and returns a new array (evens) containing only the elements that satisfy the predicate.
reduce: The reduce method is also called on the array. It takes a callback function ((acc, n) => acc + n) and an initial value (0). The callback function takes the accumulator (acc) and the current element (n), returning the updated accumulator. reduce iterates through the array, accumulating the result into the sum variable. The JavaScript syntax for these operations is very clean and directly integrated into the array object.

Go

package main

import (
    "fmt"
)

func Map[T, U any](slice []T, f func(T) U) []U {
    result := make([]U, len(slice))
    for i, v := range slice {
        result[i] = f(v)
    }
    return result
}

func Filter[T any](slice []T, f func(T) bool) []T {
    var result []T
    for _, v := range slice {
        if f(v) {
            result = append(result, v)
        }
    }
    return result
}

func Reduce[T, U any](slice []T, initial U, f func(U, T) U) U {
    result := initial
    for _, v := range slice {
        result = f(result, v)
    }
    return result
}

func main() {
    numbers := []int{1, 2, 3, 4, 5}

    // Map: Square each number
    squares := Map(numbers, func(n int) int { return n * n })
    fmt.Println("Squares:", squares)

    // Filter: Get even numbers
    evens := Filter(numbers, func(n int) bool { return n%2 == 0 })
    fmt.Println("Evens:", evens)

    // Reduce: Sum all numbers
    sum := Reduce(numbers, 0, func(acc int, n int) int { return acc + n })
    fmt.Println("Sum:", sum)
}

Go, unlike Python and JavaScript, does not have built-in map, filter, and reduce functions. However, you can easily implement them using Go's for...range loops and functions. The Go snippet shows generic implementations of Map, Filter, and Reduce using Go 1.18+ generics.

Map: The provided Map function takes a slice of type T, a function that transforms a T to a U, and it returns a slice of type U. The types T and U can be any types. It iterates through the input slice, applies the provided function f to each element, and stores the result in a new slice.
Filter: The Filter function takes a slice of type T and a predicate function (a function that returns a boolean). It iterates through the slice, and for each element where the predicate function returns true, it appends the element to a new slice.
Reduce: The Reduce function takes a slice of type T, an initial value of type U, and a function which takes a value of type U and a value of type T, returning another U.

While Go requires more code to achieve the same functionality as Python and JavaScript's built-ins, the generic implementations provide flexibility and type safety. The use of anonymous functions (like func(n int) int { return n * n }) makes the code resemble the lambda expressions used in Python and JavaScript.

8. Function Composition

Function composition is the process of combining two or more functions to create a new function. The output of one function becomes the input of the next, forming a pipeline of operations. This is a powerful way to build complex logic from simpler, reusable building blocks, enhancing code modularity and readability. It avoids creating intermediate variables and nested function calls.

Python

from functools import reduce


def compose(*funcs):
    return lambda x: reduce(lambda acc, f: f(acc), reversed(funcs), x)


def add_one(x):
    return x + 1


def square(x):
    return x * x


def multiply_by_two(x):
    return x * 2


composed = compose(square, multiply_by_two, add_one)  # Order is correct
result = composed(5)  # ((5 + 1) * 2)^2 = 144
print("Composed result:", result)

The Python snippet defines a compose function that takes a variable number of functions (*funcs) as arguments. It returns a new function (a lambda) that takes an initial value x. Inside the lambda, it uses reduce to apply the functions in sequence. The trick here is reversed(funcs). reduce normally applies functions from left to right, but in function composition, we usually want the rightmost function to be applied first. reversed(funcs) reverses the order of functions, so reduce effectively applies them from right to left. The functions add_one, square, multiply_by_two demonstrates what functions can be composed. The order matters in composition. The comment ((5 + 1) * 2)^2 = 144 shows how the functions are applied step by step, starting with add_one.

JavaScript

const compose = (...funcs) => x => funcs.reduceRight((acc, f) => f(acc), x);

const addOne = x => x + 1;
const square = x => x * x;
const multiplyByTwo = x => x * 2;

const composed = compose(square, multiplyByTwo, addOne); // Order is correct
const result = composed(5); // ((5 + 1) * 2)^2 = 144
console.log("Composed result:", result);

The JavaScript snippet defines a compose function using arrow function syntax. It's very concise. ...funcs collects all function arguments into an array. The returned function takes an initial value x and uses funcs.reduceRight((acc, f) => f(acc), x). The reduceRight method is the key here; it's a built-in array method that applies a function against an accumulator and each element of the array (from right to left), reducing it to a single value. This achieves the correct order of function application directly, without needing to reverse the function list like in Python.

Go

package main

import "fmt"

func compose[T any](funcs ...func(T) T) func(T) T {
    return func(x T) T {
        result := x
        for _, f := range funcs {
            result = f(result)
        }
        return result
    }
}

func addOne(x int) int {
    return x + 1
}

func square(x int) int {
    return x * x
}

func multiplyByTwo(x int) int {
    return x * 2
}

func main() {
    composed := compose(addOne, multiplyByTwo, square)
    result := composed(5) // ((5 + 1) * 2)^2 = 144
    fmt.Println("Composed result:", result)
}

The go snippet defines a compose function takes advantage of Go 1.18+ generics.
It is defined to take a variadic number of functions (funcs ...func(T) T). The T represents any data type. The key difference is in how the functions are applied. It uses a simple for...range loop to iterate through the funcs slice in the order they were provided. Unlike Python and JavaScript, this Go compose function applies functions from left to right. This is a deliberate choice to make the code simpler, and in the provided main() function, this is accounted for. The composed variable receives the composition, but the functions are listed in the reverse order (addOne, multiplyByTwo, square) compared to what would be mathematically expected. The final call to composed(5) produces correct results.

Conclusion

Each of these three languages brings its own unique approach to functional programming. Python and JavaScript, with their dynamic typing and built-in support for many functional features, offer a relatively lower barrier to entry. Go, on the other hand, while requiring a bit more effort to implement some functional concepts, provides strong immutability support and explicitness, which can be advantageous in certain scenarios.

Subscribe for More!

If you enjoyed this comparison, be sure to subscribe to my YouTube channel for more content on programming languages, software development, and other tech topics.

Let me know in the comments which language you prefer for functional programming and why!

Let's Talk Dev - YouTube

youtube.com

About me

I'm Mihai Farcas, a software engineer with a few years of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Secure Self-Hosting with Cloudflare Tunnels and Docker: Zero Trust Security

Mihai Farcas — Wed, 08 Jan 2025 11:07:20 +0000

For many software engineers, the allure of self-hosting is undeniable. It represents autonomy, a playground for experimentation and helps you learn about the infrastructure that makes the internet work.

However, the freedom of self-hosting comes with a critical responsibility: security. Exposing your home network to the internet is like opening your front door to everyone. Without proper protection, your home setup can quickly become a target.

This article explains the key methods and tools for securing your self-hosted applications, changing it from a weak spot into a secure setup. We'll go through DNS, look at the threat of DDoS attacks, and show how Cloudflare Tunnels, especially with Docker, provide a strong solution for achieving zero-trust security for your self-hosted environment.

To complement this written guide, I've created two accompanying YouTube videos:

This video delves into the underlying concepts of DNS, DDoS attacks, reverse proxies, and how Cloudflare's infrastructure works to mitigate these threats.

This video provides a hands-on walkthrough, demonstrating how to set up Cloudflare Tunnels with Docker, securing a sample web application step-by-step.

Why Self-Hosting Needs Special Attention

When you self-host a website, you're essentially making your home network a part of the public internet. This drastically increases your attack surface. Let's look at some key implications:

Your Public IP Address: This is your home network's "address" on the internet. Traditionally, to make your website accessible, you'd point your domain's DNS records to this IP. This direct association creates a clear target for malicious actors.
Open Ports: To serve web traffic, you typically need to open ports 80 (HTTP) and 443 (HTTPS) on your router, forwarding them to your web server. Open ports are like open doors, inviting potential intruders to probe your network for weaknesses.
Dynamic IP Addresses: Most residential ISPs assign dynamic IP addresses, which change periodically. This makes self-hosting less reliable because your website's DNS records need to be updated constantly to point to the correct IP. If they're not, your site can become unreachable. We'll see how to solve this problem later on.

What is DNS and how does it work?

Domain Name System (DNS) is like the internet's phone book. It's the technology that translates human-friendly website names (like www.example.com) into the numerical IP addresses that computers use to communicate. Without DNS, browsing the web would be incredibly difficult.

However, for self-hosting, the traditional DNS system has a weakness: it can expose your home IP address. When someone looks up your domain, the DNS lookup process eventually leads back to your home's IP address, if you are using a basic self hosting setup.

DDoS Attacks: The Digital Siege

Once a malicious actor gets hold of your home IP address, which as we saw, can be exposed through the DNS resolution process, they can launch a Distributed Denial of Service (DDoS) attack.

DDoS attacks are a major threat to self-hosted websites. In a DDoS attack, attackers use a network of infected computers (called a "botnet") to flood your server with traffic. This overwhelms your server's resources, making your website slow or completely inaccessible to legitimate users.

For self-hosted sites, a DDoS attack can be devastating. Not only can it take your website offline, but the flood of traffic can also disrupt your entire home network, affecting other services and even your household's internet access.

If you're interested in diving deeper into topics like this, including software engineering, self-hosting, home labs, Docker, Kubernetes, and other DevOps practices, head over to the Let's Talk Dev YouTube channel.

Let's Talk Dev - YouTube

youtube.com

Cloudflare: Your First Line of Defense

Cloudflare, at its core, is a Content Delivery Network (CDN) with a strong focus on security. It acts as an intermediary between your users and your origin server, providing numerous benefits:

Reverse Proxy: Cloudflare operates as a reverse proxy, meaning all traffic to your website flows through its network first. This masks your origin server's IP address, shielding it from direct exposure.
DNS Resolution Through Cloudflare: When you use Cloudflare's DNS services, the resolution process is handled by their vast network of globally distributed DNS servers. This not only improves performance but also adds another layer of obfuscation, preventing attackers from easily discovering your home IP through DNS queries.
Basic DDoS Mitigation: Cloudflare's network is designed to absorb and filter malicious traffic. Their anycast network distributes traffic across numerous data centers, making it difficult for attackers to overwhelm a single point. It also identifies and blocks known malicious traffic patterns, preventing them from reaching your server.

Limitations of Traditional Cloudflare Protection for Self-Hosting

While Cloudflare's basic DNS and DDoS protection offer a substantial improvement, they are not a complete solution for self-hosting scenarios. Here's why:

To allow Cloudflare to communicate with your web server, you still need to open ports 80 and 443 on your router and forward them to your server. This keeps a potential vulnerability open. Attackers can directly target those open ports, attempting to exploit any weaknesses in your web server (i.e. Nginx) or other services running on your network.

This also doesn't get rid of the dynamic IP address issue. If your home IP address changes, your website becomes inaccessible until you manually update your DNS records on Cloudflare. This manual intervention is prone to errors and can lead to downtime. There are more automated solutions out there, however they all rely on making the change on a schedule dinterval, which still leads to downtime.

Also, while Cloudflare hides your IP from direct DNS queries, determined attackers might still uncover it through other means, such as examining email headers or exploiting vulnerabilities in third-party services you use.

Cloudflare Tunnels: The Zero Trust Solution

Cloudflare Tunnels provide a paradigm shift in how you expose self-hosted services to the internet. Instead of opening inbound ports and relying on traditional DNS pointing to your home IP, Tunnels create a secure, outbound-only connection from your server to the Cloudflare network. This fundamentally changes the security model to one of "zero trust."

How Cloudflare Tunnels Work: A Deep Dive

cloudflared Daemon: You install a lightweight agent called cloudflared on your server (or within a Docker container, as we'll see later). This daemon is responsible for establishing and maintaining the secure tunnel.
Outbound-Only Connection: cloudflared initiates an outbound connection to Cloudflare's edge network. This is crucial because it eliminates the need to open any inbound ports on your router, drastically reducing your attack surface.
Tunnel Creation: You create a tunnel in the Cloudflare dashboard, giving it a unique name.
Service Configuration: You configure the tunnel to route traffic to specific services running on your local network. For instance, you can specify that traffic for www.example.com should be routed to your web server running on port 80. So you can choose which services are accessible through the tunnel and which aren't.
DNS Integration: Finally, when creating the tunnel, DNS records in Cloudflare are automatically configured to point to the Cloudflare network, specifying the tunnel to be used for each service. So even if your home IP changes, the tunnel will still be opened and connected to Cloudflare's network.

To recap, the benefits of using Cloudflare Tunnels are:

Not reliant on opened ports
Dynamic IP resilience
Granular access control
Better security
Simplified management

Docker and Cloudflare Tunnels

You can run the cloudflared daemon within a separate Docker container alongside your application containers.

You can also use cloudflared in a docker-compose.yml. Docker Compose simplifies the process of defining and managing multi-container applications. You can define your web application, database, and cloudflared container in a single docker-compose.yml file, ensuring they call all communicate between each other easily through Docker's internal DNS.

Refer to my accompanying YouTube video for a detailed step-by-step guide on setting up Cloudflare Tunnels with Docker. The video covers:

Creating and managing tunnels through the Cloudflare dashboard
Installing and configuring cloudflared within a Docker container
Using docker compose (and Portainer) to expose a self hosted service to the internet!

The tldr; is:

cloudflared:
    image: cloudflare/cloudflared:latest
    command: tunnel --no-autoupdate run
    environment:
      - "TUNNEL_TOKEN=${TUNNEL_TOKEN}"

Embracing Secure Self-Hosting

Self-hosting offers a unique blend of control, flexibility, and learning opportunities. However, security should always be paramount. Cloudflare Tunnels, provide a robust and elegant solution for securing your self-hosted environment.

About me

I'm Mihai Farcas, a software engineer with a few years of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Kubernetes Debugging Made Easy with mirrord

Mihai Farcas — Fri, 25 Oct 2024 18:04:46 +0000

Developing and debugging applications in a Kubernetes cluster can be complex. However, mirrord, an open-source project by MetalBear, offers a solution to simplify this process.

The Use Case

Imagine you're a software engineer working on a complex project involving multiple interconnected microservices. You encounter a bug that only occurs in the staging environment. Reproducing this bug locally is a nightmare for two main reasons:

Running the application locally requires a complex setup with multiple microservices, making it difficult to replicate the staging environment accurately.
The bug may be data-dependent, and you lack the specific data present in the staging environment. Downloading this data from the staging database is a convoluted process.

This is where mirrord comes in!

What is mirrord?

mirrord is an open-source tool designed to simplify the development and debugging of applications running in Kubernetes clusters. It achieves this by enabling developers to work on their code directly within the cluster, eliminating the need for extensive local setups or continuous deployments to staging environments.

How does it work?

mirrord operates as a "mirror" (hence the name) between your local development environment and the Kubernetes cluster. It intercepts requests destined for your application pods in the cluster and duplicates them, sending a copy to your local machine.

This allows you to run and debug your code locally while interacting with real production data and services within the cluster. In essence, it's like having a magic portal that connects your local machine to the live Kubernetes environment without affecting actual users.

Example: Debugging a Production-Only Bug

The video demonstrates how mirrord can help debug an upper-environment-only bug. The application, written in Go, runs fine locally but displays an error in the production Kubernetes cluster. In the following video I show how to use the mirrord along with it's VS Code extension to debug the application. mirrord allows you to select a target to mirror (pod or deployment) and provides access to the same resources and environment variables as the remote application.

Keep in mind you have to have kubectl access to the Kubernetes cluster you want to target pods in.

Who is Mirrord for?

Developers Tired of the "Push-and-Pray" Development Cycle

If you're tired of the constant cycle of pushing code to a staging environment and praying it works, Mirrord is for you. With mirrord, you can see your code running in a near-production environment instantly, without waiting for deployments.

Teams Struggling with Complex Local Setups

Setting up a local environment that accurately mirrors a production Kubernetes cluster can be a real headache, especially in microservice architectures. mirrord eliminates this pain by allowing you to develop and debug directly in the cluster.

Developers Who Need to Debug with Production-Like Data

Debugging with real-world data is crucial for finding and fixing those tricky bugs that only show up in upper environments. mirrord lets you do this safely, without impacting your live application.

Anyone Facing Challenges in Reproducing and Fixing Environment-Specific Bugs

Some bugs just won't show up in your local environment, no matter how hard you try to replicate it. mirrord makes it easy to reproduce and fix these environment-specific bugs by giving you direct access to the target environment.

TLDR; mirrord is for any developer who wants to simplify Kubernetes development and debugging, improve their workflow, and ship higher-quality code faster.

For more details check out the documentation on the mirrord website along with the code repository I created to demonstrate mirrod's features.

About me

I'm Mihai Farcas, a software engineer with a few years of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Why Go is Popular Right Now and Why I Started Learning Go as a Node.js Developer

Mihai Farcas — Tue, 22 Oct 2024 07:28:06 +0000

As a Node.js developer for over a decade, I've built countless web applications, APIs, and microservices. Node.js has been my go-to language for its versatility and vast ecosystem. However, I've recently become increasingly interested in Go, particularly its performance, concurrency model, and growing popularity in the cloud-native space. This interest led me to start learning Go, and I'm thrilled to share my experiences and insights with you all.

Reasons for Go's Popularity

Go has been steadily gaining popularity among developers, and for good reason. It offers a unique blend of features that make it a compelling choice for building modern, high-performance applications.

Performance: Go's compiled nature and efficient garbage collection result in impressive execution speed, often outperforming interpreted languages like Node.js and Python.
Concurrency: Go's built-in concurrency features, through goroutines and channels, make it easier to write concurrent code, which is essential for building scalable applications.
Simplicity: Go's syntax is clean, minimal, and easy to learn, reducing cognitive overhead and promoting code readability.
Type Safety: Go's static typing helps catch errors during compilation, reducing runtime surprises and improving code reliability.
Tooling: Go's comprehensive standard library and rich ecosystem of third-party tools provide developers with the necessary resources to build and maintain robust applications.

Go's sweet spot lies in its ability to balance performance, concurrency, and simplicity. It's a language that allows you to write efficient, scalable code without sacrificing readability or ease of development.

Go's popularity has also been fueled by the rise of Kubernetes. As Kubernetes has become the de facto standard for container orchestration, Go has benefited from its close association. Kubernetes is written in Go, and many of its components and extensions are also built using Go.

Why I Started Learning Go

Beyond the technical benefits, I was drawn to Go's simplicity and elegance. Coming from the world of JavaScript with its occasional quirks and complexities, Go's minimalist syntax and clear design principles were a breath of fresh air. The language's focus on readability and maintainability aligned with my belief in writing clean, understandable code.

I've been writing code in a functional-lite style in Node.js, and Go allows me to continue with that as it has many of the traits that make this possible in JS. Go's support for first-class functions, closures, and immutability makes it well-suited for functional programming paradigms. This was a significant advantage, as I could leverage my existing functional programming skills while exploring Go's unique features. Check out my video about functional programming in JavaScript - many of the concepts apply to Go as well.

I was particularly interested in Kubernetes and the possibility of writing Kubernetes extensions and operators in Go. Kubernetes has become the de facto standard for container orchestration, and Go's close association with Kubernetes made it a natural choice for extending its functionality. The prospect of building custom Kubernetes tools and controllers in Go further fueled my interest in learning the language.

If you're interested in similar topics, check out my YouTube channel, where I share content about Kubernetes, functional programming, and many other software development topics:

Let's Talk Dev - YouTube

youtube.com

Transitioning from Node.js to Go:

While I found several similarities between Go and Node.js, particularly in their approach to concurrency and support for functional programming paradigms, there are also some notable differences that shaped my transition experience. Initially, I felt a bit disoriented switching between the two languages, but as I gained more experience with Go, the transition became smoother.

Go's strict type system and compilation process were significant changes from Node.js with TypeScript. While TypeScript introduces static typing to JavaScript, Go's type system is more strict and enforced during compilation. This stricter approach reduces runtime errors and improves code reliability. Go's compilation process also differs from TypeScript's transpilation process. Go compiles directly to machine code, resulting in faster execution speeds compared to TypeScript, which is transpiled to JavaScript that then runs on a JavaScript engine.

A big difference was Go's minimalist approach to error handling. Unlike Node.js's reliance on exceptions, Go treats errors as values, requiring explicit handling. This approach, while initially surprising, encouraged me to write more robust and error-aware code. It also made me realize the importance of carefully considering error handling strategies in my Node.js projects. I must say that I much prefer Go’s approach to error handling!

Go's built-in support for concurrency through goroutines and channels was a significant advantage over Node.js's single-threaded model with its event loop and asynchronous callbacks. Goroutines, Go's lightweight concurrency units, allowed me to write concurrent code without the overhead of managing threads manually. They are managed by the Go runtime, and multiple goroutines can be multiplexed onto a smaller number of OS threads. This is unlike Node.js, where the developer has to make use of the event loop and manage asynchronous code and callbacks. Channels, the communication mechanism between goroutines, provided a safe and efficient way to synchronize data between concurrent processes. They can be used to pass data and also to signal that a certain task is done.

For other Node.js developers looking to learn Go, I recommend embracing the type system, understanding the concurrency model, and adopting Go’s error handling approach. There are plenty of resources available online, including tutorials, documentation, and community forums, to help you get started with Go. You can also check out my video about how I started learning Go:

Benefits of Learning Go

Learning Go has been a rewarding experience, and I encourage other Node.js developers to explore this powerful language. By taking this step, you'll be investing in your own growth and development as a software engineer. Here are some of the benefits I've gained from learning Go:

Improved performance and scalability of applications: Go's compiled nature and efficient memory management translate to faster execution speeds and reduced resource consumption. This can lead to significant improvements in application performance, especially for high-traffic web services and data-intensive applications.
Enhanced understanding of concurrency and parallelism: Go's built-in concurrency features, using goroutines and channels, provide an elegant and efficient way to handle concurrent operations. Learning Go can deepen your understanding of concurrency concepts.
Increased productivity and code maintainability: Go's simplicity, strong typing, and tooling contribute to increased developer productivity. The language's focus on readability and clarity makes it easier to write, understand, and maintain code, even in large and complex projects.
Exposure to a new programming language and its community: Learning a new language like Go exposes you to new ideas, paradigms, and approaches to problem-solving. It also opens doors to a vibrant and supportive community of Go developers, where you can learn, collaborate, and contribute.

Conclusion:

Go's popularity is a testament to its ability to address the challenges of building modern, high-performance applications. Its performance, concurrency model, simplicity, and tooling have made it a compelling choice for developers across various domains.

My journey into Go as a Node.js developer has been a rewarding experience. The transition, while initially challenging, has broadened my perspective and enhanced my skill set. I encourage other Node.js developers to consider learning Go and explore its potential for building robust and scalable applications.

About me

I'm Mihai Farcas, a software engineer with a few years of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Prompt Engineering and Other Bullshit Jobs in Tech

Mihai Farcas — Wed, 28 Aug 2024 20:31:22 +0000

In the tech industry, we love our buzzwords. "Prompt engineering" is the latest buzzword, a shiny new term for a role that shouldn’t and probably doesn’t exist. As usual, the internet and tech companies have latched onto this buzzword and are now creating entire "prompt engineering" teams.

I'm calling BS. Sure, there's a skill to getting chatbots to spit out what you want, but it's not that hard to do, nor does it deserve its own “role” in a company. It's more like knowing how to sweet-talk or clearly define what you want.

The Bullshit Job Phenomenon in Tech

Bullshit jobs are not a new phenomenon in tech. Long before "prompt engineer" became a thing, the tech world was already a breeding ground for pointless positions.

The Compliance Officer Who Clicks "Approve": In the wake of scandals and regulations, companies hired hordes of compliance officers. Many of them spent their days rubber-stamping documents, ensuring the illusion of oversight rather than actual scrutiny.

The "Innovation Strategist" Who Never Innovates: With titles like "Chief Innovation Officer" or "VP of Disruption," these folks were paid handsomely to brainstorm ideas that rarely see the light of day. Their real job? To make the company look forward-thinking.

The "Customer Experience Manager" Who Doesn't Talk to Customers: Tasked with improving the customer experience, these employees often have little contact with actual customers. Instead, they focused on internal metrics and reports, creating the illusion of progress without actually making a difference.

In David Graeber's book Bullshit Jobs, he describes the phenomenon of pointless jobs that exist only to make companies or individuals appear busy and important. Think of all those endless meetings that could've been an email. He argues that these jobs are not only useless but also soul-crushing for the people who hold them.

The rise of "prompt engineering" seems to fit Graeber's definition of a bullshit job perfectly. It's a job that exists solely because of the hype surrounding AI, not because there's any real need for it. It exists because it sounds cutting-edge and cool, not because it solves real problems.

What "Prompt Engineers" Actually Do (or Don't Do)

So, you're probably wondering, what do these so-called "prompt engineers" actually do all day? Well, the truth is, it's not as glamorous as it sounds. The reality is that much of what's touted as "prompt engineering" is just the same old stuff dressed up in fancy new tech jargon.

The fundamental skills remain the same. If you were a marketer before, you still need marketing expertise to craft effective prompts for AI. If you were a coder, you still need coding knowledge to guide the AI in generating functional code. AI is a tool, not a replacement for domain knowledge.

It boils down to crafting clear and specific instructions for AI models. Think of it like giving directions to a particularly literal-minded friend. You need to be explicit, avoid ambiguity, and anticipate potential misunderstandings. It's less about engineering and more about effective communication.

Of course, there are some nuances involved. Prompt engineers might experiment with different phrasing, iterate on instructions, and fine-tune parameters to optimize the AI's output. But let's be honest, it's not exactly rocket science. If you can write a clear email or follow a recipe, you've got the basic skills down.

You can confidently dismiss the "prompt engineer" fad. Prompting LLMs or AI is not a job; it's a skill, and a basic one at that. It certainly won't be a key differentiator in the job market.

Domain Expertise vs. Prompt Engineering: What Employers Really Want?

Let's cut through the BS. The real value in today's AI-driven world isn't in "prompt engineering" – it's in good old-fashioned domain expertise. AI is a tool, and like any tool, it's only as good as the person wielding it. You can give a monkey a paintbrush, but it won't create a masterpiece - actually never mind, that would probably sell for millions. Similarly, you can give someone with zero marketing experience the ability to prompt an AI, but they won't generate a winning campaign.

The hype around "prompt engineering" might be driven by the same factors that create bullshit jobs. It's a shiny new title that creates an illusion of specialized knowledge and importance, masking the reality that the core tasks remain unchanged. Companies might be tempted to hire "prompt engineers" to project an image of being at the forefront of AI innovation, even if the role itself adds little genuine value.

Employers ultimately care about results. They might think they need people with AI prompting skills, but what they really need is someone who can deliver. And delivering results requires experience, expertise, and intelligence. AI can assist in the process, but it can't replace the human element that drives success.

The future of work isn't about becoming a "prompt engineer." It's about becoming an expert in your field and learning how to leverage AI to enhance your skills and productivity. AI can automate mundane tasks, generate ideas, and provide insights, but it can't replace the critical thinking, creativity, and problem-solving skills that come with true expertise.

The Future of Work in the Age of AI

The future belongs to those who can adapt and evolve, not those who chase fleeting trends. Instead, invest in yourself and your skills. Focus on building your knowledge in your chosen field. The real winners in the age of AI will be those who combine their domain expertise, with their unique set of skills and experience and use AI as a tool to enhance these.

"Prompt engineering" is largely a myth. Don't be fooled by the hype. Focus on what you're good at, and let the AI tools help you where they actually can. The essence of meaningful work lies in its ability to create value, solve problems, and contribute to the world. AI can be a powerful ally in this endeavor, but it's the human element - our knowledge, creativity, and passion - that will ultimately define the future of work.

About me

I'm Mihai Farcas, a software engineer with a few years of experience under my belt. I'm passionate about writing code and love sharing knowledge with fellow developers.

My YouTube channel, "Let's Talk Dev," is where I break down complex concepts, share my experiences (both the good and the face-palm moments).

Let's Talk Dev - YouTube

youtube.com

Connect with me:

Website: https://mihai.ltd
YouTube: https://www.youtube.com/@letstalkdev
GitHub: https://github.com/mihailtd
LinkedIn: https://www.linkedin.com/in/mihai-farcas-ltd/

Simplify PostgreSQL Deployments with Kubernetes & GitOps (with Crunchy Data Operator)

Mihai Farcas — Mon, 22 Jul 2024 21:17:15 +0000

Ever felt frustrated with the complexity of running PostgreSQL on Kubernetes? Manually setting up data volumes, services, deployments, managing backups, managing secrets holding credentials, not to mention high availability... There is a lot of complexity in running stateful applications in Kubernetes. This is especially true if you're aiming to achieve this in a declarative GitOps way.

There is a way to simplify this process, to deploy PostgreSQL declaratively, and have a production-ready setup with just a few lines of declarative yaml.

That's where the Crunchy Data Postgres Operator comes in.
In this article, let's deploy PostgreSQL declaratively, creating a production-ready setup on Kubernetes.

If you'd rather watch than read:

Installing Crunchy Data Postgres Operator

To follow along, you need to have a kubernetes cluster running. On my local machine I will be using Rancher Desktop - see my video on that topic. Or you can do so in the cloud or even in your own homelab, and wouldn’t you know, I have a video on how to easily set up a kubernetes homelab as well.

Let’s get back to the task at hand. Installing PGO is not that complicated, however it presented a few challenges when I initially started out, especially since I was a beginner in Kubernetes back then. So I am aiming to simplify this process for you.

To help us out Crunchy has prepared a github repo with code examples. You can find that in the quick start guide from the documentation.

Let’s clone that repo locally and see what’s inside.

There are a couple of ways to install PGO, or actually most things in kubernetes. One is through Helm and one is through Kustomize.

I will use Kustomize, it is the closest thing to native kubernetes manifests and I believe the easiest to figure out what exactly is being installed.

You'll want to check out the folder conveniently named install. And yet again, there are a couple of ways of installing it. One is the single namespace install, where PGO will manage Postgres instances in, well, a single namespace, and the other one is a cluster-wide installation where PGO can manage postgres instances in any namespace in the cluster. The main difference is in the permissions you give PGO within your cluster.

This type of cluster-wide operator is important and we will see that later on we'll talk about the interesting topic of Platform Development.

The cluster-wide installation manifests can be found in the default folder.

Here we can check the manifest file, it references other manifests, but we can understand what it’s trying to do: install a CRD or a custom resource definition, which is how you extend the kubernetes API, it applies some role based access controls (RBAC), and some thing called “manager” which is just a deployment, a pod that will run an image with the code that makes all of this possible.

OK let’s try to apply these manifests, kubectl apply -k ./kustomize/install/default and here we get the first error. The annotation is too long. To fix it you just need server-side apply. I added the --force-conflicts flag as well but that’s optional. Here is the final command:

kubectl apply -k ./kustomize/install/default --server-side --force-conflicts

And that’s it, we now have the Crunchy Data Postgres Operator installed!

Deploying PostgreSQL using PGO

In the examples repository you can find manifests that showcase the options you have when deploying PostgreSQL. Let's take a look at /kustomize/postgres/postgres.yaml.

With PGO installed we can now create a custom resource of kind PostgresCluster.
In the manifest we can define the Postgres image and version to use, in this case version 16.
We can have multiple instances in the same manifest and each instance can be a replica set with multiple replicas.
For example here there is a single instance named instance1.
We need to allocate some storage - 1 gb should be enough.

Another required field to have a minimal postgres cluster is the backup configuration. It’s nice that they "force" you to have at least a minimal backup configuration.

In a production scenario the backups should be stored at least on a separate drive or even better, offsite or in the cloud.

There are a bunch of options you can configure here, and after we apply the manifest, PGO will take care of backups automatically.

As you can see there are a few pods that started, you can see one pod for that postgres instance. There is also a backup job that ran immediately and now it’s stopped. It also created some kubernetes services so you can connect to this postgres instance.

Speaking of connecting to postgres, one of the best things about this operator is that it manages credentials in kubernetes secrets automatically!

You should see a secret for each user configured for the database. Let’s check one of these secrets out to see what’s in there: a few properties, basically everything you need to successfully connect to the database!

No need to manually configure secrets, this is all done for us!

Actually let me show you how easy it is to just create a user and a new database.
There is a users key in the manifest, and we can give it a name, we can specify the databases this new user has access to. The nice thing is that if the database doesn’t exist PGO will create it. Let’s call it testdb. And we can give this new user SUPERUSER access:

  users:
    - name: test
      databases:
        - testdb
      options: SUPERUSER

Now it’s as simple as applying this new manifest and there we go, a new secret has been created for this new user with all the credentials.
More importantly behind the scene, all the required operations were done for us, creating the database, creating the user and so on. And the beauty of it all is that it’s all declarative!

Deploying PGAdmin

I’d really love to test all of this in a visual way, if only there was a UI interface to see what’s going on in a Postgres database. Oh wait, PGO can also manage a PGAdmin installation for us! Let’s check out the manifest for that: ./kustomize/pgadmin/pgadmin.yaml

As you can see the other custom resource that this operator installed is of kind PGAdmin.
Let’s just apply this example manifest. We can see a pod PGAdmin was created, let’s port forward and open it in a browser.

Credentials for the PGAdmin web interface are also stored in a kubernetes secret automatically.

High Availability

Scaling a PostgreSQL instance with PGO is as easy as flipping an integer:

apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata:
  name: hippo-ha
spec:
  image: registry.developers.crunchydata.com/crunchydata/crunchy-postgres:ubi8-16.3-1
  postgresVersion: 16
  instances:
    - name: pgha1
      replicas: 2

Just change the number of replicas to whatever you need, and PGO will take care of it. See ./kustomize/high-availability/ha-postgres.yaml for a more complete example that also includes pgbouncer.

What is a Kubernetes Operator?

Kubernetes is great at managing stateless applications, it has everything you need out of the box. But stateful applications like PostgreSQL, which store persistent data and require special care and procedures to run, that’s a bit more tricky. This is where Kubernetes Operators come in. They are software extensions that enhance Kubernetes' capabilities, specifically designed to automate the management of complex, stateful applications like PostgreSQL databases.

Like we saw earlier, operators extend the kubernetes API with new types of resources, in this case we had the PostgresCluster and the PGAdmin resource types.

The operators usually also run a container in the background and can handle certain administrative tasks for you, for example they can scale Postgres instances, automate backup and restores, disaster recovery etc.

And most importantly they can do so in a declarative way!

Platform Development

This leads me to the other topic I wanted to cover in the video and that is Platform Development.

This is a concept where you have a platform team that manages a Kubernetes cluster that has all of these operators installed.

This platform then enables self-provisioning and managing of resources without complex scripting, manual actions - it’s all done through declarative configuration!

This is especially important for a few use cases, such as:

Spinning up new environments like staging or test environments
Accelerating development cycles and productivity
Faster iteration on new ideas
Implementing GitOps principles ensuring consistency, repeatability and auditability.

So there you have it! The Crunchy Data Postgres Operator takes the complexity out of running PostgreSQL on Kubernetes. Head over to their GitHub page and give it a try. I'd love to hear about your experiences in the comments below!

Don't forget to subscribe to my youtube channel for more content like this!

Let's Talk Dev - YouTube

youtube.com

Hosting for Web Apps: Cloud vs. Homelab vs. Hybrid – Which Saves You Money?

Mihai Farcas — Sun, 14 Apr 2024 13:42:07 +0000

Cloud providers love to promise cheap, scalable web app hosting. The reality is often far more complex. Let's cut through the marketing jargon and uncover the true cost of cloud solutions, the potential savings of a homelab, and when a hybrid approach might be your smartest move.

When "Simple" Gets Costly

Building a solid online presence is crucial in today's world, and a strong portfolio website can open doors to new opportunities. That's why I decided to create my own "Mihai @ Let's Talk Dev" portfolio website, opting for a stack using Nuxt.js, Directus CMS, and PostgreSQL. While the development went smoothly, finding the right hosting solution proved far more complicated than I anticipated.

If you'd rather watch than read, check out my video on this topic here:

Vercel: The Maze of Pricing

Initially, Vercel seemed like a great choice for the frontend. Their promises of "unlimited this" and "free that" are enticing... until you try to decipher their pricing structure. Even with experience, I found it overwhelming, and it's easy to imagine how intimidating this would be for newcomers.

One particularly obscure detail was the PostgreSQL compute time limit on their "Hobby" tier. While 60 hours per month sound generous, there's no clear definition of what constitutes a "compute hour." This lack of transparency makes it incredibly difficult to estimate your real-world costs for a web app.

Let's consider two scenarios:

A Low-Traffic Backend: If your web app powers a small portfolio site or a simple API with infrequent queries, you might stay within the free tier.
A Real-Time Dashboard: Now, picture a dashboard constantly polling for updates. These frequent requests, even if simple, could blow past those 60 hours in no time, landing you with a bill for each additional hour.

...And here's where I want you to think honestly: Looking at this price table, would you be able, right now, to predict with certainty how quickly your project might outgrow Vercel's free tier? Can you really say for sure which of those limits you'll hit first – the storage, the requests, or that mysterious "compute time"? Or, even, how much will your first month’s invoice be?

The Cost of Convenience: Supabase

Seeking a simpler solution, I considered Supabase. Their free tier looked promising at first. But then I hit a major snag: they sleep inactive databases, and you have to manually wake them up from the dashboard.

Now, my web app wouldn't get a ton of traffic. I could realistically see a week with no visitors. Could I really guarantee at least some activity to keep my database awake? Should I resort to automated scripts as a workaround? That seems like a hack, and suddenly, even basic availability jumps you into paid tiers.

And then there's the storage issue. Their free storage is tiny – maybe enough for a simple portfolio, but a slightly more complex project would easily exceed that limit. It seems like this would be the first hurdle forcing you into a paid tier, even if everything else about your project is small-scale. And then that next tier likely over-provisions the database to something you don't really need – the jump is just too drastic.

This made me question whether these supposedly simpler services are really as straightforward as they seem.

Digital Ocean: Better

Unlike Vercel and Supabase, which are Platform-as-a-Service (PaaS) providers, Digital Ocean takes an Infrastructure-as-a-Service (IaaS) approach. This means they focus on renting you raw resources like virtual machines ("droplets"), instead of pre-packaged web app solutions. This offers more granular control over costs compared to the often opaque tiered pricing of PaaS solutions.

So, at least with platforms like Digital Ocean, the pricing is transparent. However, their managed database options start off affordable but quickly become expensive as you need more resources. Even a modest performance boost can easily double your bill.

It's important to note that 1GB of RAM and 1 CPU core isn't really a performant database setup, especially for anything beyond a small project. And this price is just for a single node, not a cluster. On the plus side, they do include free daily backups for 7 days, which is a nice touch.

I would also need one or two droplets where I could host the Nuxt page itself and the Directus CMS. So you’re looking at upwards of $20 for a simple website like this.

Why Cloud Costs Cause Headaches

Before diving into specific pitfalls, let's understand the root of the problem with cloud hosting for many web apps:

Complexity Overload

Cloud providers offer a dizzying array of services designed for massive enterprises. For most of us, this is overkill. Think of AWS: you start needing a host (EC2), then storage (S3), a database (RDS), maybe a CDN... It's like an onion, layer upon layer of services, each with its own pricing model. This makes accurately budgeting for your web app almost impossible.

Hidden Fees

Data transfer, egress charges... ever tried to calculate the cost of moving your data off a cloud platform? It'll make you question your tech choices (and maybe consider investing in a fast SSD instead!). The nickel-and-diming is rampant, and thankfully, the European Union is taking action. Their new Data Act aims to force transparency around these fees, but it highlights how insidious the problem has become.

Vendor Lock-In

Those shiny, cloud-exclusive tools are tempting. But good luck migrating if your costs suddenly skyrocket! You might end up feeling trapped, with the cloud feeling more like a mortgage than a flexible solution.

Security and Surprise Invoices

As Web Dev Cody showed in his video, cloud security isn't a "set it and forget it" deal. Protection during attacks can get incredibly expensive. Planning is key! If your self-hosted server gets DDoS'ed, downtime is likely. But, if your cloud service gets hit, you might face a shocking bill. The right choice depends on your web app's needs: can you tolerate downtime, or is an unexpected invoice worse?

Pitfalls

Understanding pitfalls like these is crucial for avoiding unexpected costs. Here are five of the most common ones I've encountered:

Pitfall 1: The Illusion of "Per Request" Pricing

Charges by Rows Fetched: Let's say your shiny cloud database charges per row fetched. Sounds reasonable, right? But one unoptimized query without an index? Suddenly, you're scanning thousands of rows to find that single piece of data you need ...and now your 'cheap' database query just bankrupted you.
Charges per Requests: Requests are not users. My simple portfolio sees about 1500 requests for 50 unique users. That's because every image, CSS file, and JavaScript snippet that is sent in chunks is a request. And that’s for my simple portfolio, for a more dynamic website, and the bill stacks up fast.
Unpredictability: Unless you can predict exactly how every user will interact with your site, estimating your true cost at the end of the month is next to impossible.

Pitfall 2: The "Generosity" Trap

The Bait: Ample free storage, bandwidth, etc., lure you in. But that one crucial feature you really need? Strangely limited...forcing a paid upgrade.
Overprovisioning: And of course, that upgrade tier forces you into a package with way more resources than you need.
Read the Fine Print: Try to understand what each item on the pricing page means, before you get hooked!

Pitfall 3: The Cost of Escape

Data Egress: Extracting your data out of the cloud? You'll often pay steep fees for the privilege, especially at scale.
Tool Lock-in: Proprietary cloud tools may seem amazing at first, but are designed to make switching a nightmare, potentially costing you a small fortune in development time.

Pitfall 4: Dangerous Ignorance

Abstraction Blindness: Many devs treat cloud services as magic black boxes. They don't realize the underlying tech (PostgreSQL, Nginx, etc.) and how that impacts cost. Imagine unknowingly deploying a database configuration that costs ten times more than a perfectly suitable alternative!
The "Just Pay" Mentality: Without understanding the fundamentals, devs easily accept inflated prices, assuming "this is just how the cloud works." But cloud pricing is often arbitrary. Understanding and questioning those costs is important! And this perfectly sets us up for the next pitfall:

Pitfall 5: The "Managed" Illusion

Managed Doesn't Mean Problem-Free: Managed databases and automatic backups are fantastic tools. However some developers confuse them with a set-it-and-forget-it solution, but they often lure developers into a false sense of security.
What You Still Need to Do:
- Performance Tuning: Managed services don't optimize your queries for you. Indexing, database design are still your responsibility. Not understanding this leads to unnecessary costs through inefficient setup or over-reliance on auto-scaling – both of which cloud providers happily profit from.
- Backup Strategy: Automatic backups are great, but what's your disaster recovery plan? How often do you test restores?
- Cost Control and Monitoring: Managed services can scale automatically, often without you fully realizing it. You still need to monitor usage. Cloud providers DO offer monitoring tools, but they often come with their own costs or limitations. Don't assume you're getting accurate oversight for free!

Solutions for my portfolio website

So, does this mean cloud computing is evil? Absolutely not! I don’t even think the services I outlined here are bad, they just helped me illustrate some points and misconceptions. But, you need to be armed with knowledge. Here’s what ended up working for my portfolio website:

Finding the Right Fit: I used Cloudflare Workers for the Nuxt frontend application. It is simple to use and has a generous free tier. Cloudflare also acts as your DNS manager, and offers performance optimizations and DDOS protection for free! This seems like a smart strategy on their part – support smaller projects with potential to grow, and they get word of mouth recommendations, case and point, I’m mentioning their services in this video!
Homelab Love: For the CMS, my trusty homelab server stepped up. This is where I host Directus and PostgreSQL. To see some more information on how I built my homelab server, check out this video! Yes, electricity costs a bit, but at least my server isn't sending me surprise invoices. It’s a predictable cost and I'm in total control.
Be Strategic about Self-Hosting: Choose what goes on your servers for example, non mission critical services, staging environments and so on. and what goes in the cloud or on managed services. For example, I self-host multiple services that use the same database instance, sharing the costs. Plus, I can use the same Directus instance to manage content for multiple websites. And for securely exposing the services from my home network, I use Cloudflare Tunnels – a great option for controlled exposure of self-hosted services. This was again, free, if you can believe it! So all in all I am only paying for electricity to host my website.
Price Conscious Architecting: When I do use the cloud, I scrutinize every component's cost. Open-source tools help break free of vendor lock-in and give me more flexibility in the long run.
But beyond that, I'm investing time into learning the underlying technologies – things like Kubernetes and cloud-native applications. This means that if I need to, I can move these workloads between cloud providers, or even bring them to my homelab without major headaches.

Take time to understand what you actually NEED and shop smartly. Don’t be afraid to do it yourself, and learn the hard things, trust me it pays off!

Let's keep this conversation going. Drop your cloud pricing horror stories in the comments, and let's learn from each other. Maybe your story will help someone else avoid a nasty cloud invoice.

Or maybe you think my criticism is unfair - write me a comment about that, I read all of them!

And before you go, we’re getting very close to 1000 subscribers!
Please consider subscribing to the Let's Talk Dev YouTube channel:

Let's Talk Dev - YouTube

youtube.com

Here is a video of how I built the UI for the landing page of my portfolio website I talked about in this article:

Better Postgres Database Migrations and Testing by Ditching Bloated ORMs

Mihai Farcas — Sat, 02 Mar 2024 10:21:35 +0000

Managing PostgresSQL schema can be a challenging task, especially when dealing with bloated ORMs. Learning new syntaxes that eventually translate to SQL but are almost always worse than writing pure SQL can be frustrating. In this blog post, we will explore a better approach to managing Postgres database migrations and testing, using a tool called Graphile Migrate. We will discuss the important principles to follow and how Graphile Migrate simplifies the workflow. So, let's jump right in!

Before I continue, a couple of useful links:

The GitHub code repository containing an example of a database migration and integration test.
The video version of this article which contains additional practical examples.

Principle #1: Writing Schema Migrations in the Language of Your Database:

The best way to write schema migrations is in the native language of your database, which is SQL. Graphile Migrate allows you to write migrations in pure SQL, freeing you from being tied to a particular tech stack. By using SQL, you can easily switch between different migration tools without rewriting your migration scripts.

Embracing the native language of your database — SQL — and harnessing its robust features will empower you as a software engineer. SQL is the key to querying, manipulating, and optimizing data efficiently. Whether you’re writing complex queries or designing efficient indexes, SQL proficiency is essential for successful database management.
So dive into SQL tutorials, practice writing queries, and explore the intricacies of database design.

Principle #2: Idempotency

Graphile Migrate emphasizes the important concept of idempotency in database migrations. An idempotent operation can be run multiple times without altering the results. This means that if a migration script is run multiple times, it will always produce the same outcome. In other words an operation is considered idempotent if it can be executed multiple times without altering the final state.
This property ensures predictability and stability during the migration process.

Getting Started with Graphile Migrate:

To get started with Graphile Migrate, you'll need to install it and run the initialization script (I'm using Bun, feel free to use NodeJS & npm / pnpm / yarn):

bun add graphile-migrate
bun graphile-migrate init

This will set up the basic folder structure and configuration file. The configuration file contains connection strings for your Postgres database, and you can customize it to suit your needs. Graphile Migrate also provides features like setting placeholders which are special strings (i.e. :DATABASE_NAME or :DATABASE_USER) that will evaluate to a value in the final migration. This value can be evaluated from env variables for example. You can also define hooks for executing custom commands during or after the migration process.

Fast and Seamless Migration Workflow:

One of the key advantages of Graphile Migrate is its speed. It runs migrations extremely fast, making the development workflow seamless. The watch mode allows you to continuously run the migration command, automatically executing the migration whenever there are changes in the SQL file. This immediate feedback enables quick iteration and boosts developer confidence that the migration scripts are correct.

Principle #3: Roll Forward Migrations

Graphile Migrate exclusively supports roll forward migrations. Unlike rollbacks, which revert changes, roll forward migrations only move forward in the migration history. Here’s why this approach is beneficial:

No Need for Rollbacks: By focusing solely on forward progress, developers avoid the complexity of managing backward migrations. This simplifies the schema evolution process.
Consistently Reproducible Schema: Roll forward migrations maintain a clean and reproducible schema, making it easier to collaborate and deploy changes across development, staging, and production environments.

Principle #4: Testing Database Migrations:

By writing integration tests for your migrations, you can ensure their correctness and catch any issues early. Testing also plays a crucial role in maintaining a consistently reproducible chain of events, which leads to the same final state every time.
An example of a database integration test would be:

spawn a new test database
run migrations
seed test data for different scenarios
execute functions, triggers etc.
make assertions
clean up the data

Conclusion

Ditching bloated ORMs and adopting a better approach to Postgres database migrations can greatly enhance your development workflow. By following the principles of writing migrations in SQL, ensuring idempotency, and leveraging the power of tools like Graphile Migrate, you can simplify the process and achieve a cleaner, consistently reproducible schema. So why not give it a try and level up your database migration game?

Remember, Your mastery of SQL will elevate your development skills and enable you to build robust, scalable applications. Happy migrating!

Note: To learn more about the advantages of PostgreSQL and why it's a strong choice even for non-relational data, check out this video:

Don't forget to subscribe to the Let's Talk Dev YouTube channel!

Set up a Kubernetes cluster in under 5 minutes with Proxmox and k3s

Mihai Farcas — Tue, 30 Jan 2024 16:02:18 +0000

Introduction

In this blog post, we will explore how to quickly and easily add a new node to your Kubernetes cluster in under 5 minutes. I'll walk you through the process of setting up a compact mini PC and demonstrate how to integrate it seamlessly into your existing Kubernetes cluster. By following this guide, you'll be able to expand the capabilities of your cluster and gain valuable insights into the inner workings of cloud computing.

Video Version

Compact Mini PCs: A Cost-Effective Solution

First, let's take a look at the compact mini PC that I'll be using for this setup. These machines are not only affordable but also powerful enough to perform complex computing tasks. With their 8th Gen Core i7 processors, 16GB of RAM, and fast NVMe storage, these mini PCs offer exceptional performance while consuming minimal power. In fact, they draw only about 10 to 30 watts of power (depending on load), making them a budget-friendly alternative to cloud computing.

PRO Tip: There are a lot of great deals to be found on second hand Mini PCs

Why Choose Kubernetes?

Before I dive into the setup process, let's briefly discuss why Kubernetes is the platform of choice for building scalable and efficient applications. Kubernetes has gained immense popularity, especially in the enterprise world, due to its ability to streamline containerized application deployment and management. By learning Kubernetes, you can open up new career opportunities and gain a better understanding of the services cloud vendors provide.

Setting Up Your New Node

To add a new node to your Kubernetes cluster, I'll utilize Proxmox, a powerful tool for managing clusters of servers. Installing Proxmox is a straightforward process that can be completed by booting from a USB with the Proxmox ISO file. Once the installation is complete, you can go to the IP address that was assigned to the machine during installation on the port 8006 and you will be greeted by the Proxmox web UI.
If this is the first node, then you instead need to click on "Create Cluster".
Since I already have a cluster, I'll join this new machine to it. This is as simple as going to to the existing cluster's Proxmox web UI, click on Datacenter -> Cluster -> Join Information. Copy the join information, go back to the new node's Proxmox web UI, click "Join Cluster" paste the information, input the cluster's password, and click join. Wait for a few seconds until the join is successful. From now on, you only use the cluster's web UI.

Check out the official quick start guide.

Creating a Virtual Machine in Proxmox

Now that our new machine is part of the cluster, it's time to create a virtual machine (VM) that will serve as our new Kubernetes worker node. I'll use Ubuntu Server as the base operating system for this VM, as it provides a solid foundation for running Kubernetes. In Proxmox, creating the VM is as simple as giving it a name, allocating resources such as storage, CPU cores, and RAM, and then booting it up.

Installing Kubernetes with K3S

Once the VM is up and running with Ubuntu Server, we're ready to install Kubernetes. To keep things lightweight and beginner-friendly, we'll use K3S, a minimalistic version of Kubernetes developed by the Rancher team. By following a quick start guide, setting up K3S on our Ubuntu server becomes a breeze. We'll ensure that the environment variables "K3S_URL" and "K3S_TOKEN" are set correctly to enable the worker node to join the existing cluster.
However, if this is the first node (the control plane) of this Kubernetes cluster, you don't need to set those variables, instead just run the installation command.
This takes care of everything, including ensuring that k3s starts upon rebooting the VM.

Check out the official K3s quick start guide.

Expanding Your Possibilities with Kubernetes

With our new node successfully integrated into the Kubernetes cluster, the possibilities are endless. From hosting websites to running various applications and databases, you can easily leverage the power of Kubernetes for your projects. In the near future, stay tuned as we tackle exciting projects such as building a portfolio website from scratch, covering everything from design and front-end development to back-end development, deployment to Kubernetes, and hosting online.

Monitoring Power Consumption with Smart Plug

To keep track of the power consumption of our server, I'll include a smart plug in my setup. This handy device allows us to accurately measure the power consumed by the mini PC. Monitoring power consumption not only helps us optimize efficiency but also sheds light on the cost-effectiveness of having our own Kubernetes cluster compared to utilizing cloud services.
In this case I am running 3 Mini PCs totaling 24 threads, 48 GB of RAM and NVME storage, all under 50W.

Conclusion

In this blog post, I demonstrated how to add a new node to your Kubernetes cluster in under 5 minutes using a compact mini PC. By following the step-by-step guide, you can expand the capabilities of your cluster and gain valuable insights into cloud computing. From cost-effectiveness to versatility, building and managing your own Kubernetes cluster opens up a world of possibilities for hosting various applications and accelerating your learning journey. Stay tuned for more exciting projects and developments on our channel!

Thank you for reading and following. I am looking forward to sharing more insightful content with you in the future.

Consider Subscribing to my YouTube channel:

Let's Talk Dev - YouTube

youtube.com

A Year of Self-Hosting: 6 Open-Source Projects That Surprised Me in 2023

Mihai Farcas — Tue, 19 Dec 2023 10:19:17 +0000

Are you ready to supercharge your development workflow and take your projects to the next level? Look no further! In this post, I'm going to introduce you to the most innovative and useful open source tools that I discovered in 2023. These tools have revolutionized the way I create amazing apps, automate tasks, run machine learning models, and more. The best part? They're all self-hosted, giving you full control over your data and privacy. Don't miss out on these game-changers!

Watch in video form

Hanko: Passwordless Authentication Made Easy Have you ever wished for a more secure and convenient way of authentication? With Hanko, you can say goodbye to passwords and welcome a passwordless future. Hanko is a self-hosted authentication platform that seamlessly integrates both social and passwordless login into any web application with just a few lines of code. No more password headaches or security concerns. Get started with Hanko and experience the future of authentication.

hanko.io

Directus: Low Code Backend Generation Creating and managing your data model can be a breeze with Directus. This low code, GraphQL or REST backend generator allows you to effortlessly build and customize your data model using an intuitive no-code interface. Once you have your data model set up, Directus provides you with a fully-featured API that you can connect to any front-end framework of your choice. Whether it's for a headless CMS or a data-rich app, Directus has got you covered.

The Headless CMS + Backend for Every Custom Build

Built for developers who need more than a CMS. Manage complex content, handle digital assets, and control permissions through an intuitive Studio.

directus.io

n8n: Your Self-Hosted Automation Platform Imagine having your own automation platform that connects various apps and services, just the way you want. n8nmakes it possible! This self-hosted automation platform allows you to create visually stunning workflows and automate tasks using an intuitive user interface. With a vast array of integrations, it supports almost any service you can imagine. Build complex automations effortlessly and take full control of your workflows with n8n.

AI Workflow Automation Platform - n8n

n8n is a workflow automation platform that uniquely combines AI capabilities with business process automation, giving technical teams the flexibility of code with the speed of no-code.

n8n.io

PostgresML: Machine Learning Inside PostgreSQL Combine the power of machine learning with the reliability of PostgreSQL with PostgresML. This complete MLOps platform, integrated as a PostgreSQL extension, opens up new possibilities for running machine learning models directly within your database. With support for text classification, generation, and summarization, you can fine-tune large language models with your own data without it ever leaving your PostgreSQL database. Experience the seamless integration of machine learning and PostgreSQL with PostgresML.

postgresml / postgresml

Postgres with GPUs for ML/AI apps.

Postgres + GPUs for ML/AI applications.

| Documentation | Blog | Discord |

Why do ML/AI in Postgres?

Data for ML & AI systems is inherently larger and more dynamic than the models. It's more efficient, manageable and reliable to move models to the database, rather than constantly moving data to the models.

Architecture

PostgresML is a powerful Postgres extension that seamlessly combines data storage and machine learning inference within your database. By integrating these functionalities, PostgresML eliminates the need for separate systems and data transfers, enabling you to perform ML operations directly on your data where it resides.

Features at a glance

In-Database ML/AI: Run machine learning and AI operations directly within PostgreSQL
GPU Acceleration: Leverage GPU power for faster computations and model inference
…

View on GitHub

SuperDuperDB: Machine Learning Made Easy for Database Users If you're a database user who wants to dive into the world of machine learning, SuperDuperDB is here to assist you. This innovative project helps you create, train, and deploy machine learning models directly within your existing database. Similar to PostgreML, SuperDuperDB aims to make machine learning accessible for anyone who uses a database. Stay tuned for an upcoming video where I'll compare SuperDuperDB with PostgresML and delve deeper into this exciting tool.

superduperdb.com

Please take a moment to subscribe to my YouTube channel where I share more insights and practical tips / tutorials like this

Let's Talk Dev - YouTube

youtube.com

Hugging Face: Collaboration and Open Source ML Models Discover the collaborative power of Hugging Face, where the machine learning community comes together to share models, datasets, and applications. With hundreds of models at your fingertips, you can find pre-trained ML models for various applications such as text generation, classification, translation, and summarization. Hugging Face is free, transparent, and fosters a thriving community of learners and contributors. Explore, learn, and leverage the power of Hugging Face for your projects.

Hugging Face

In conclusion, the year 2023 has brought a wave of groundbreaking open source projects that have transformed the way I create, automate, and innovate. From passwordless authentication to low code backends, self-hosted automation to machine learning inside databases, the possibilities are limitless. Don't let these game-changing tools slip under your radar. Embrace the power of open source and revolutionize your development journey today!