DEV Community: Tristan Kilhwan Chai

[Homelab AI Project] Ep.7 - Designing a Lightweight Monorepo Architecture with Node.js and Python

Tristan Kilhwan Chai — Tue, 28 Jul 2026 04:24:31 +0000

1. Background: "Spring Boot is Great, but My 16GB RAM is Priceless"

After wrapping up the frontend optimization with Astro and React (Episode 6), it was time to design the backend services.

When designing multi-module architectures, the default choice is often Java and Spring Boot (Gradle Multi-Module). With strong typing, mature DI containers, and structured layering, it is undeniably a solid framework.

However, my mini-PC homelab (Ryzen 5500U, 16GB RAM) runs under strict resource constraints. The core database and utility containers already consume a substantial portion of memory:

PostgreSQL 16 + pgvector (Capped at 4GB for caching vector indexing structures)
Redis 7.2 Alpine (Capped at 1GB for queuing and caching layers)
Prometheus + Grafana + Loki (~1GB for the APM and logging stack)
Gitea + Actions Runner (~200MB for CI/CD pipelines)

In this environment, booting separate Spring Boot instances for the news collector, AI processor, and mail sender would instantly swallow 300MB to 500MB of RAM per instance. Running three JVM processes would eat up 1.5GB of memory before processing a single news article, inviting OS OOM (Out Of Memory) crashes during compute-intensive vector operations.

Furthermore, implementing RAG processing requires Python library bindings (like numpy or sentence-transformers). Choosing Spring Boot meant I would eventually have to launch a separate Python runner anyway, doubling the memory footprint.

Therefore, I chose a resource-optimized stack (ADR-002):

Node.js (TypeScript) Workspaces: Running high-concurrency async I/O jobs under 50MB of RAM per instance.
Python RAG Core: Constructing the embedding calculations and pgvector pipelines directly in Python.
Monorepo Structure: Housing both languages inside a unified repository to keep pipeline configurations clean.

2. The Solution: Node.js & Python Monorepo Layout

My multi-module layout (ADR-002) is organized as follows:

# Monorepo directory structure
AI_News/
├── packages/
│   ├── core/           # [Node.js/TS] Shared database configurations, entities, and schemas
│   ├── collector/      # [Node.js] Asynchronous RSS news collector (batch job)
│   ├── ai-engine/      # [Python] pgvector + OpenRouter RAG summarizer
│   ├── sender/         # [Node.js] Redis Queue based mass email sender
│   └── frontend/       # [Astro/React] Frontend analytics dashboard
├── infra/              # Docker Compose configurations and environment files (.env)
├── docs/               # Architecture Decision Records (ADRs) and blog drafts
├── package.json        # Root npm workspaces configuration
└── tsconfig.json       # Shared compiler options for TypeScript modules

The core component is packages/core. Since the collector, sender, and frontend dashboard all belong to the Node.js/TypeScript stack, sharing common database entities, DTO schemas, and environment validators inside core keeps code duplication to absolute zero.

3. Implementation: Root Setup & Module Scaffolding

(1) Root Configuration (`package.json`)

I configured npm workspaces in the root JSON file to manage local dependency paths:

// package.json (root)
{
  "name": "@ai-news/root",
  "private": true,
  "workspaces": [
    "packages/core",
    "packages/collector",
    "packages/sender",
    "packages/frontend"
  ],
  "scripts": {
    "dev:frontend": "npm run dev -w @ai-news/frontend",
    "build:frontend": "npm run build -w @ai-news/frontend",
    "start:collector": "npm start -w @ai-news/collector",
    "start:sender": "npm start -w @ai-news/sender"
  }
}

(2) Shared TypeScript Rules (`tsconfig.json`)

I established standard compiler options at the root, allowing submodules to extend the configuration:

// tsconfig.json (root)
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "esModuleInterop": true,
    "strict": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "declaration": true,
    "sourceMap": true
  }
}

(3) The Core Package (`packages/core`)

Below is the scaffolding setup for the shared TypeScript package:

// packages/core/package.json
{
  "name": "@ai-news/core",
  "version": "0.1.0",
  "private": true,
  "main": "./dist/index.js",
  "types": "./dist/index.d.ts",
  "scripts": {
    "build": "tsc"
  }
}

// packages/core/src/types/news.ts
export interface NewsItemDTO {
  id?: string;
  title: string;
  link: string;
  source: string;
  rawContent: string;
  publishedAt: Date;
}

export interface SummaryDTO {
  newsId: string;
  summaryText: string;
  modelUsed: string;
  costIncurred: number;
  processedAt: Date;
}

4. Troubleshooting & Debugging Logs

🚨 Issue 1: npm Workspaces Hoisting and Missing Local Build Files

After running npm install, trying to boot sub-packages threw compile-time errors: Module not found when referencing @ai-news/core.

Cause: npm workspaces hoist third-party packages to the root node directory. Because the local core package @ai-news/core was not built into compiled .js assets (dist/), referencing submodules crashed during runtime resolve steps.
Resolution: I configured package pipeline triggers inside the root script runner, guaranteeing @ai-news/core is built prior to booting any dependent workspaces:

  // package.json (root)
  "scripts": {
    "build:core": "npm run build -w @ai-news/core",
    "predev:frontend": "npm run build:core",
    "prestart:collector": "npm run build:core"
  }

🚨 Issue 2: Docker Bridge DNS Resolving Failures (Connection Refused)

Routing API calls from the Node.js collector to the Python AI engine via http://localhost:8000/summarize resulted in crash errors: ECONNREFUSED inside container task runners.

Cause: In containerized spaces, localhost points to the isolated loopback interface of that specific container. To communicate with external containers, processes must resolve network domains using Docker's internal bridge DNS resolver.
Resolution: I targeted the Docker Compose service name (trisoft-ai-engine) as the domain destination, mapping host targets to http://trisoft-ai-engine:8000 to establish container-to-container connections.

5. Results & Metrics: Spring Boot vs. Node.js/Python

I measured the server memory footprint (RSS) against a simulated Java/Spring Boot multi-module deployment under idle conditions:

Metric	Spring Boot (JVM, 3 instances)	Node.js (2 instances) + Python (1 instance)	Improvement
Constant RAM Footprint	~720 MiB	~124 MiB (Node: 44MB, Python: 80MB)	-82.7% (RAM Saved)
Cold Start Duration	~8.2s	~0.9s	-89.0%
16GB Host RAM Saved	~95.5% conserved	~99.2% conserved (600MB preserved)	+3.7% headroom

By deploying lightweight Node.js script runtimes for collectors and senders, I freed up 600MB of RAM. This memory buffer was funneled into the pgvector similarity index caches, enabling faster semantic processing pipelines.

6. Next Up

With the Monorepo skeleton verified, Episode 8 will dive into our data harvesting layers: Building a Node.js RSS News Collector Scheduler — Pipeline Steps and DB storage.

[Homelab AI Project] Ep.6 - Aiming for a 0MB Memory Footprint on Frontend Servers with Astro and React Islands

Tristan Kilhwan Chai — Wed, 22 Jul 2026 08:01:09 +0000

1. Background: "Zero RAM Left to Waste on Frontend Servers"

After establishing my Gitea-based CI/CD pipeline in Episode 5, it was time to build the frontend dashboard to display my AI news pipeline stats.

For modern web apps, the default tech stack is almost always Next.js (App Router). While Next.js provides excellent developer velocity, it suffers from a massive architectural drawback when deployed on hardware-constrained homelabs:

Persistent Node.js Runtime Footprint: Running a Next.js app in SSR/ISR mode requires a continuous Node.js server instance running in the background. This instantly swallows 150MB to 300MB of idle RAM.
The Threat of the OOM Killer: In my homelab (Ryzen 5500U, 16GB RAM), memory must be preserved for PostgreSQL page caches, Redis queues, and intensive Python/RAG embedding operations. Wasting 200MB+ on a frontend server would invite out-of-memory crashes.

What about standard React SPAs (e.g., Vite builds)? They compile to static assets, allowing Nginx static serving (0MB constant runtime RAM). However, they enforce Client-Side Rendering (CSR), meaning users must download hundreds of kilobytes of JavaScript before anything is painted, leading to sluggish LCP and poor SEO crawlers indexation.

I needed a system that shipped close to 0 bytes of initial JavaScript, loaded instantly, scored 100/100 on Google Lighthouse SEO, and still let me build interactive charts using the rich React ecosystem.

This is why I chose Astro and React's Island Architecture.

2. The Solution: Island Architecture (Astro + React)

Astro is a static-first framework. At build time, it processes components and outputs pure static HTML/CSS. Since I serve these files using Nginx, I allocate 0MB constant runtime memory for a Node.js frontend process.

To support dynamic interfaces (like my analytics charts), I use Island Architecture.

graph TD
    subgraph HTML_Page["Astro Static Page (HTML / No JS)"]
        Hero["Hero Section (Static HTML)"]
        Arch["Architecture Diagram (Static SVG/HTML)"]

        subgraph React_Island_1["React Island (Hydrated)"]
            ChartComponent["Live Demo Chart (React + Chart.js)"]
        end

        subgraph React_Island_2["React Island (Hydrated)"]
            LogViewer["Live Log Streamer (React)"]
        end

        Journal["Journal Cards (Static HTML)"]
    end

    HTML_Page --> Hero
    HTML_Page --> Arch
    HTML_Page --> React_Island_1
    HTML_Page --> React_Island_2
    HTML_Page --> Journal

    style React_Island_1 fill:#61dafb,stroke:#333,stroke-width:2px
    style React_Island_2 fill:#61dafb,stroke:#333,stroke-width:2px

The layout and static descriptions render instantly as plain HTML. Interactive widgets are marked as independent Islands (e.g. <LiveDemoChart client:visible />). The browser only requests the React bundle and hydrating JavaScript when the scroll position hits the visible chart viewport, bypassing massive initial payloads.

3. Implementation: Packages & Configuration

(1) Package Directory Layout

To scale into a monorepo later, I organized the frontend code under packages/frontend/:

# packages/frontend/
├── public/
├── src/
│   ├── components/       # Astro & React Components
│   │   ├── Header.astro
│   │   ├── Hero.astro
│   │   └── LiveDemoChart.jsx (React)
│   ├── layouts/          # Layout Templates
│   │   └── Layout.astro
│   ├── pages/            # File-based Routing
│   │   └── index.astro
│   └── styles/           # Vanilla CSS Styling
│       └── global.css
├── astro.config.mjs
├── package.json
└── tsconfig.json

(2) Astro Configuration (`astro.config.mjs`)

I enabled the React integration and enforced strict static file compilation:

import { defineConfig } from 'astro/config';
import react from '@astrojs/react';

export default defineConfig({
  integrations: [react()],
  output: 'static', // Forces build-time static HTML file generation
  build: {
    format: 'file', // Generates concrete physical files (index.html)
  }
});

(2.5) Essential Node.js Package Configuration (`package.json` & `tsconfig.json`)

We initialized modular dependencies for Astro and React, and configured TypeScript parameters:

// packages/frontend/package.json
{
  "name": "@ai-news/frontend",
  "type": "module",
  "version": "0.1.0",
  "scripts": {
    "dev": "astro dev",
    "start": "astro dev",
    "build": "astro build",
    "preview": "astro preview",
    "astro": "astro"
  },
  "dependencies": {
    "@astrojs/react": "^3.0.0",
    "astro": "^4.0.0",
    "chart.js": "^4.4.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-chartjs-2": "^5.2.0"
  },
  "devDependencies": {
    "@types/react": "^18.2.0",
    "@types/react-dom": "^18.2.0",
    "typescript": "^5.0.0"
  }
}

// packages/frontend/tsconfig.json
{
  "extends": "astro/tsconfigs/strict",
  "compilerOptions": {
    "jsx": "react-jsx",
    "jsxImportSource": "react",
    "baseUrl": ".",
    "paths": {
      "@/*": ["src/*"]
    }
  }
}

(3) Lightweight CSS Design Tokens (`global.css`)

To keep builds clean and eliminate dependencies, I shunned CSS frameworks (like Tailwind) and built a custom dark theme using CSS variables:

:root {
  --font-sans: 'Outfit', 'Inter', -apple-system, sans-serif;

  /* Harmonious Dark Palette */
  --bg-primary: hsl(224, 25%, 12%);
  --bg-secondary: hsl(224, 25%, 16%);
  --bg-tertiary: hsl(224, 25%, 20%);
  --border-color: hsl(224, 15%, 25%);

  --text-primary: hsl(210, 40%, 98%);
  --text-secondary: hsl(215, 20%, 75%);

  /* Accent Colors */
  --accent-cyan: hsl(180, 100%, 50%);
  --accent-purple: hsl(270, 100%, 65%);
  --gradient-accent: linear-gradient(135deg, var(--accent-cyan), var(--accent-purple));

  --shadow-lg: 0 10px 30px rgba(0, 0, 0, 0.5);
  --transition-smooth: all 0.3s cubic-bezier(0.4, 0, 0.2, 1);
}

body {
  background-color: var(--bg-primary);
  color: var(--text-primary);
  font-family: var(--font-sans);
  margin: 0;
  padding: 0;
  line-height: 1.6;
}

.card-glass {
  background: rgba(36, 44, 60, 0.6);
  backdrop-filter: blur(12px);
  border: 1px solid var(--border-color);
  border-radius: 12px;
  box-shadow: var(--shadow-lg);
  transition: var(--transition-smooth);
}

.card-glass:hover {
  border-color: var(--accent-cyan);
  transform: translateY(-2px);
}

(4) React Island Live Chart Component (`LiveDemoChart.jsx`)

This component is designed to render asynchronously on the client-side once hydration occurs, bypassing SSR window is not defined errors. It utilizes Chart.js to map news parsing and AI summarization statistics.

// packages/frontend/src/components/LiveDemoChart.jsx
import React, { useEffect, useState } from 'react';
import {
  Chart as ChartJS,
  CategoryScale,
  LinearScale,
  PointElement,
  LineElement,
  Title,
  Tooltip,
  Legend,
  Filler
} from 'chart.js';
import { Line } from 'react-chartjs-2';

ChartJS.register(
  CategoryScale,
  LinearScale,
  PointElement,
  LineElement,
  Title,
  Tooltip,
  Legend,
  Filler
);

export default function LiveDemoChart() {
  const [chartData, setChartData] = useState(null);

  useEffect(() => {
    // Safely run inside the client environment
    const data = {
      labels: ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'],
      datasets: [
        {
          label: 'Collected News',
          data: [65, 78, 72, 89, 95, 110, 120],
          borderColor: 'rgb(0, 242, 254)',
          backgroundColor: 'rgba(0, 242, 254, 0.1)',
          fill: true,
          tension: 0.4,
        },
        {
          label: 'AI Summaries',
          data: [12, 15, 14, 18, 20, 24, 25],
          borderColor: 'rgb(186, 85, 211)',
          backgroundColor: 'rgba(186, 85, 211, 0.1)',
          fill: true,
          tension: 0.4,
        }
      ]
    };
    setChartData(data);
  }, []);

  if (!chartData) {
    return (
      <div style={{ height: '300px', display: 'flex', alignItems: 'center', justifyContent: 'center', color: 'var(--text-secondary)' }}>
        Loading chart...
      </div>
    );
  }

  const options = {
    responsive: true,
    maintainAspectRatio: false,
    plugins: {
      legend: {
        labels: {
          color: 'rgb(210, 220, 240)',
          font: {
            family: 'Outfit'
          }
        }
      }
    },
    scales: {
      x: {
        grid: {
          color: 'rgba(255, 255, 255, 0.05)'
        },
        ticks: {
          color: 'rgb(215, 220, 235)'
        }
      },
      y: {
        grid: {
          color: 'rgba(255, 255, 255, 0.05)'
        },
        ticks: {
          color: 'rgb(215, 220, 235)'
        }
      }
    }
  };

  return (
    <div style={{ height: '300px', width: '100%' }}>
      <Line data={chartData} options={options} />
    </div>
  );
}

4. Troubleshooting & Debugging logs

🚨 Issue 1: `window is not defined` Build Failure

Compiling the project with npm run build failed instantly with a compilation error:
ReferenceError: window is not defined inside the React chart component.

Cause: Astro attempts to prerender all components on the server side (SSR) to spit out clean HTML during compilation. Because Node.js lacks browser globals like window or document, libraries parsing canvas dimensions crash during static rendering.
Resolution: Force Astro to skip SSR compilation for client-dependent modules by applying the client:only="react" directive:

  <!-- Bypasses server-side build render; mounts directly in the browser -->
  <LiveDemoChart client:only="react" />

Alternatively, wrapping browser API declarations inside React's useEffect hook ensures code only runs after hydration.

🚨 Issue 2: Style Flash (FOUC) during Page Reloads

Refreshing the dashboard caused a flash of unstyled content (FOUC)—showing raw black-on-white text for ~100ms before snapping to our dark theme.

Cause: Styles scoped within React Islands were asynchronously loaded and parsed only after the React bundles hydrated.
Resolution: Decoupled the styling from React component scopes. Imported global styles (global.css) directly in the static Astro layout header (Layout.astro), allowing the browser to parse all CSS classes instantly alongside the initial HTML stream.

🚨 Issue 3: Missing Node.js/npm Runtime on the Homelab Server

After SSHing into my Rocky Linux server, I attempted to install dependencies using npm install only to be greeted by bash: npm: command not found.

Cause: In the excitement of building my homelab infrastructure, I forgot to provision the Node.js/npm runtime on the host OS.
Resolution: Installed NVM (Node Version Manager) and ran nvm install --lts to neatly implement the latest LTS release. NVM is preferred on homelabs to avoid global permission friction.

🚨 Issue 4: Empty Boilerplate `package.json` and `EJSONPARSE` Failures

With Node.js running, I triggered npm install again, which immediately crashed with npm error code EJSONPARSE: Unexpected end of JSON input.

Cause: While the folder structure was created, the core package.json was left completely blank. The npm client crashes instantly when parsing empty or malformed JSON payloads.
Resolution: Populated the template with essential Astro and React hooks (@astrojs/react, astro, react, react-dom) alongside a sound tsconfig.json template, which restored the installation process.

🚨 Issue 5: Astro's Host Header Restrictions over Cloudflare Tunnel

Binding my server port (4321) to a custom domain via Cloudflare Tunnel resulted in 400 Bad Request or host header rejection errors on external browsers.

Cause: Astro 4.x natively blocks unexpected host headers to prevent host header injection attacks. Additionally, bound loopback addresses (127.0.0.1) do not receive tunneled proxy streams gracefully.
Resolution: We must run the dev server with npm run dev -- --host 0.0.0.0 and configure server: { allowedHosts: ['ainews.tristan.team'] } in astro.config.mjs. However, during the static build verification stage, the astro preview command ignored configuration file host configurations due to config-merging bugs between Astro and Vite, consistently displaying the Blocked request. This host is not allowed warning page. Since the core philosophy of Episode 6 is "rejecting persistent Node.js web servers and serving static compiled HTML via Nginx to keep RAM usage at 0MB," fighting with Vite's dev-only security checks on astro preview was pointless. Instead, we can verify the output by launching a lightweight static web server (npx serve dist -l 4321), which behaves exactly like the production Nginx static serving configuration.

5. Results & Metrics: Next.js vs. Astro

Here are the performance metrics compared to a default Next.js (App Router) project:

Metric	Next.js (App Router, blank page)	Astro (Static + 1 React Island)	Improvement
Server RAM Footprint	240 MiB	0 MiB (Static Nginx serving)	-100% (No runtime node)
Initial JS Size	~82.4 kB	7.2 kB (Excluding React core)	-91.2%
Lighthouse Performance	88 / 100	100 / 100	+12 points
TTI (Time to Interactive)	0.8s	0.15s	-81.2%

By using Astro, I saved 240MB of memory on my server and reduced the initial JS footprint down to a mere 7.2KB, securing a perfect 100/100 Lighthouse Performance score even on lower-end hardware configurations.

6. Next Up

With my frontend skeleton running efficiently, Episode 7 shifts focus to my core codebase. I will walk through configuring my monorepo architecture: Monorepo Structure Design — Decoupling Node.js & Python modules and sharing core modules.

[Homelab AI Project] Ep.5 - Building a Lightweight Self-Hosted CI/CD Pipeline with Gitea and Act Runner

Tristan Kilhwan Chai — Wed, 22 Jul 2026 06:24:23 +0000

1. The Challenge: Keeping Git and CI/CD under 150MB of RAM

In Episode 4, we decoupled our PostgreSQL database, Redis task queue, and RSS crawler module into isolated Docker containers with strict memory ceilings. However, a manual deployment loop—SSHing to the server, pulling source code, and running rebuild commands—violates basic principles of engineering efficiency.

We needed a local, self-hosted Git server and an automated CI/CD pipeline. The industry standard, GitLab, comes with a steep recommendation:

"GitLab requires at least 4GB of RAM (8GB recommended) to function smoothly."

Allocating 25% of our entire 16GB homelab memory to a tool that simply hosts repos and triggers pipelines is a critical FinOps failure. That 4GB of RAM is better utilized feeding our database page cache or Python RAG pipeline.

Our target was to achieve a git-triggered deployment pipeline with:

Under 150MB active RAM consumption.
Support for GitHub Actions YAML syntax (to preserve portable configuration).
Direct container control to rolling-update the crawler service.

The solution: Gitea paired with Gitea Actions Runner (Act Runner). Written in Go, Gitea runs natively in lightweight environments, and its Act Runner executes standard GitHub Actions workflow files seamlessly.

2. Topology: Internal Network Bridging and Docker Daemon Delegation

To deploy this without exposing raw ports to the internet or introducing unnecessary routing overheads, we set up two integration boundaries:

① Host Docker Daemon Binding (`/var/run/docker.sock`)

The Act Runner container maps the host's /var/run/docker.sock file. When a build job triggers a container update, it passes commands directly to the host's Docker daemon, allowing the runner container to remain lightweight.

② Loopback and Internal DNS Direct Routing

We bind both Gitea and Act Runner to the internal docker bridge network (ainews-internal). The runner polls Gitea using the internal DNS address (http://ainews-git:3000), ensuring build traffic never loops through external routers or Cloudflare edge servers.

graph TD
    subgraph Host_OS ["Rocky Linux 10.1 (Host)"]
        subgraph Int_Net ["ainews-internal Network (Isolated)"]
            Gitea["Gitea Server (ainews-git) <br> [Port: 3000, 2222]"]
            Runner["Act Runner (ainews-runner)"]
        end

        Docker_Sock["/var/run/docker.sock <br> (Host Docker Socket)"]
        Target_Cont["ainews-collector <br> (Collector Container)"]
    end

    Local_Dev["Local Developer PC <br> (git push)"] -- "Cloudflare Tunnel <br> (https://git.your-domain.com)" --> Gitea
    Runner -- "Polling <br> (http://ainews-git:3000)" --> Gitea
    Runner -- "docker compose up -d --build <br> (Control Daemon)" --> Docker_Sock
    Docker_Sock --> Target_Cont

3. Setup: infra/docker-compose.gitea.yml & Workflow

📄 `infra/docker-compose.gitea.yml`

version: '3.8'

networks:
  ainews-internal:
    external: true

services:
  ainews-git:
    image: gitea/gitea:1.22
    container_name: ainews-git
    restart: always
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - GITEA__database__DB_TYPE=postgres
      - GITEA__database__HOST=ainews-db:5432
      - GITEA__database__NAME=giteadb
      - GITEA__database__USER=tristan
      - GITEA__database__PASSWD=super-secret-password-change-me
    volumes:
      - ./volumes/gitea:/data:z
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "2222:22"
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M

  ainews-runner:
    image: gitea/act_runner:latest
    container_name: ainews-runner
    restart: always
    depends_on:
      - ainews-git
    environment:
      - CONFIG_FILE=/config.yaml
      - GITEA_INSTANCE_URL=http://ainews-git:3000
      - GITEA_RUNNER_REGISTRATION_TOKEN=your_runner_registration_token_here
      # Map custom label for our Rocky Linux environment instead of the default ubuntu
      - GITEA_RUNNER_LABELS=rocky-linux:docker://node:22-slim
    volumes:
      - ./volumes/act_runner:/data:z
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M

📄 `.gitea/workflows/deploy.yml`

name: AI News Auto Deploy

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: rocky-linux # Custom runner label configured in config.yaml / runner labels
    steps:
      - name: Checkout Source Code
        uses: actions/checkout@v4

      - name: Rebuild & Restart Target Service
        run: |
          echo "Start deploying AI News application..."
          docker compose -f infra/docker-compose.yml up -d --build ainews-collector
          echo "Deployment finished successfully!"

4. Real-World Troubleshooting

🚨 Issue 1: Runner Loopback Failure via External Domain Addresses

Symptom: Pointing GITEA_INSTANCE_URL to https://git.your-domain.com threw socket timeout and connection refused errors.
Cause: Requests from the runner routed out through the WAN and back into Cloudflare Tunnel. Lacking NAT loopback translation rules inside the local router, internal packets were dropped.
Resolution: Keep traffic inside the bridge network by utilizing Gitea's internal container alias (http://ainews-git:3000).

🚨 Issue 2: Docker Socket Bind Permission Denied (GID Mismatch)

Symptom: Runner container threw Permission denied: failed to connect to /var/run/docker.sock during deployment steps.
Cause: The host's socket requires root or docker group membership. The runner's inner container user (UID 1000) was not mapped to the host's docker GID (often 999 or 992).
Resolution: Configured GID group delegation by granting the runner process elevated socket access inside the Docker config, or configuring group mapping settings to line up internal UIDs with the host's Docker socket groups.

🚨 Issue 3: Missing 'Actions' Repository Settings UI

Symptom: Repository landing page did not display the Actions configuration tab.
Cause: Out of resource safety concerns, Gitea disables actions processing on initial startup.
Resolution: Appended configuration flags to Gitea's custom configurations file:

  # edit volumes/gitea/gitea/conf/app.ini
  [actions]
  ENABLED = true

A quick container restart loaded the configurations, activating the menu.

🚨 Issue 4: Disappearing Host Port Bindings (3000) leading to 502 Bad Gateway

Symptom: Gitea container failed to resolve external connections, routing to a 502 Bad Gateway error. Running docker ps revealed host port bindings (0.0.0.0:3000->3000/tcp) were missing, showing only raw container ports (22/tcp, 3000/tcp).
Cause: The Gitea service was connected solely to the infra_ainews-internal network. By default, the Docker daemon drops and blocks all host port forwarding rules for containers isolated purely inside internal: true bridge networks for strict access segregation.
Resolution: Map both ainews-internal and the public bridge network ainews-external to Gitea in docker-compose.gitea.yml:

      networks:
        - ainews-internal
        - ainews-external  # <- Allow host port forwarding by joining public network

Restarting the stack created the correct iptables forwarding rules, resolving the bad gateway immediately.

🚨 Issue 5: 'token is empty' Registration Failure for Gitea Act Runner

Symptom: The runner container looped on startup, throwing Error: token is empty even though we mapped the token in the configuration.
Cause: Depending on the specific image tag, Gitea Act Runner expects either GITEA_RUNNER_TOKEN or GITEA_RUNNER_REGISTRATION_TOKEN. Environment parser mismatch passed an empty string inside the final configuration layout.
Resolution: Map both environment variables to the registration token inside docker-compose.gitea.yml:

        - GITEA_RUNNER_TOKEN=your_token_here
        - GITEA_RUNNER_REGISTRATION_TOKEN=your_token_here

Enforcing both variables bypasses image version compatibility issues, registering the runner instantly.

5. Results & Metrics

Here are the resource footprint improvements compared to GitLab:

Idle Hardware Resource Usage Comparison

Metric	GitLab CE	Gitea (PostgreSQL)	Gitea Act Runner
Idle Memory Footprint	4.2 GiB	118 MiB	28 MiB
Idle CPU Utilization	~2.5%	~0.05%	~0.01%
Active Build Memory	5.0+ GiB	~180 MiB	~210 MiB (Host Socket)

96.5% Memory Savings: Instead of dedicating 4.2GB to GitLab, Gitea and its runner require only 146MB, freeing up over 4GB of raw hardware capacity for database index caching and memory-intensive Python vector searches.
5-Second Deployment Lead Time: Pushing code updates triggers the local agent immediately, completing target rebuilds in 4.8 seconds with zero human interaction.

6. Next Up

With container structures and CI/CD pipelines isolated, Episode 6 explores client interfaces. We will configure our frontend using Astro and React's Island Architecture to maintain high rendering speeds and minimal Javascript runtimes.

[Homelab AI Project] Ep.4 - Isolating the Service Layer with Docker Compose

Tristan Kilhwan Chai — Thu, 16 Jul 2026 06:46:48 +0000

Series: Building a Personalized AI News Pipeline (4/23)

Executive Summary

This post details how to design a multi-service container infrastructure on a single homelab server (Ryzen 5500U, 16GB RAM) using Docker Compose. We implement hardware resource caps via cgroups (allocating 4GB to PostgreSQL and 1GB to Redis) and establish a dual-network topology (Internal vs. External) to prevent resource contention and secure data stores. We also share troubleshooting lessons on enabling Linux memory controllers, Docker DNS search ordering, and environment parser edge cases.

References: docker-compose resource limits docs / Linux cgroups documentation

1. The Challenge: Monolithic Mess vs. Virtualization Overhead

Our AI News Pipeline consists of four distinct architectural components:

RSS Collector (Node.js): Constantly crawls tech feeds.
AI Processing Engine (Python): Handles embedding generation, Vector DB querying (RAG), and LLM orchestration.
Database (PostgreSQL + pgvector): Stores articles, metadata, and high-dimensional vectors.
Queue & Cache (Redis 7.2 Alpine): Orchestrates tasks and caches API requests.

In a naive setup, one might install these services directly on the host OS (dnf install postgresql-server redis). However, in a production-grade setting, this leads to:

Resource Contention: Heavy embedding computation in Python can spike CPU and memory, triggering the host OS's Out-Of-Memory (OOM) Killer to terminate PostgreSQL.
Dependency Conflicts: Upgrades to system libraries for one service can break dependencies for another.

To avoid this, we could use hypervisor-level virtualization (such as Proxmox or ESXi) to allocate resources statically. But on a mini PC with only 16GB of RAM, running multiple VM operating systems introduces unacceptable virtualization memory overhead.

The optimal solution is containerization with Docker Compose -- providing lightweight process isolation, strict resource control, and seamless portability from local environment to our production Rocky Linux server.

2. The Architecture: Cgroups and Network Segmentation

To adapt enterprise-grade network security and resource scheduling principles to our homelab, we implemented three key design patterns:

1. Resource Constraints via cgroups

Docker utilizes Linux kernel cgroups to restrict container workloads. We enforced hard limits to protect host OS stability:

PostgreSQL (ainews-db): Limit to 4GB RAM, 2.0 CPUs. Enough for pgvector indexing and cache buffers, but capped to prevent host OS starvation.
Redis (ainews-queue): Limit to 1GB RAM, 1.0 CPU. Activates eviction policies if memory usage exceeds this cap.

2. Dual-Network Partitioning

Containers should not share a single flat network. We separated traffic into two custom bridge networks:

ainews-internal: Only contains PostgreSQL (ainews-db) and Redis (ainews-queue). It has no outbound internet route (internal: true) and is inaccessible from the outside world.
ainews-external: Contains the public-facing Frontend (Astro), the Collector (which needs outbound internet access to fetch RSS feeds), and the Cloudflare Tunnel agent (cloudflared).

3. Complete Configuration Decoupling

All secrets, ports, and volume paths are abstracted into a local .env file, allowing us to keep configuration separate from the core stack declaration.

graph TD
    subgraph Host_OS ["Rocky Linux 10.1 (Host)"]
        subgraph Ext_Net ["ainews-external Network"]
            CF_Tunnel["Cloudflare Tunnel (cloudflared)"]
            Astro_FE["Astro + React Frontend"]
            Collector["Node.js RSS Collector"]
            AI_Engine["Python AI Engine"]
        end

        subgraph Int_Net ["ainews-internal Network"]
            Postgres["PostgreSQL + pgvector (ainews-db) <br> [cgroups limit: 4GB]"]
            Redis["Redis 7.2 Alpine (ainews-queue) <br> [cgroups limit: 1GB]"]
        end
    end

    CF_Tunnel --- Astro_FE
    Astro_FE --- Postgres
    Collector --- Postgres
    Collector --- Redis
    AI_Engine --- Postgres
    AI_Engine --- Redis

    style Postgres fill:#336791,stroke:#fff,stroke-width:2px,color:#fff
    style Redis fill:#D82C20,stroke:#fff,stroke-width:2px,color:#fff
    style CF_Tunnel fill:#F38020,stroke:#fff,stroke-width:2px,color:#fff
    style Ext_Net fill:#e1f5fe,stroke:#0288d1,stroke-width:1px,stroke-dasharray: 5 5
    style Int_Net fill:#efebe9,stroke:#5d4037,stroke-width:1px,stroke-dasharray: 5 5

3. Setup Files: infra/docker-compose.yml & .env.example

infra/.env.example

PROJECT_NAME=ai-news

POSTGRES_USER=tristan
POSTGRES_PASSWORD=super-secret-password-change-me
POSTGRES_DB=ainews
POSTGRES_PORT=5432
DB_VOLUME_PATH=./volumes/postgres

REDIS_PORT=6379
REDIS_VOLUME_PATH=./volumes/redis

NODE_ENV=production

infra/docker-compose.yml

version: '3.8'

networks:
  ainews-internal:
    driver: bridge
    internal: true
  ainews-external:
    driver: bridge

services:
  # 1. PostgreSQL DB with pgvector
  ainews-db:
    image: postgres:16
    container_name: ainews-db
    restart: always
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - ${DB_VOLUME_PATH}:/var/lib/postgresql/data:z # :z is required for SELinux context
    ports:
      - "${POSTGRES_PORT}:5432"
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4096M

  # 2. Redis Cache & Queue
  ainews-queue:
    image: redis:7.2-alpine
    container_name: ainews-queue
    restart: always
    command: redis-server --appendonly yes --requirepass ${POSTGRES_PASSWORD}
    volumes:
      - ${REDIS_VOLUME_PATH}:/data:z
    ports:
      - "${REDIS_PORT}:6379"
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1024M

  # 3. Node.js RSS Collector
  ainews-collector:
    image: node:22-slim
    container_name: ainews-collector
    restart: always
    working_dir: /usr/src/app
    environment:
      NODE_ENV: ${NODE_ENV}
      DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@ainews-db:5432/${POSTGRES_DB}      REDIS_URL: redis://:${POSTGRES_PASSWORD}@ainews-queue:6379/0    networks:
      - ainews-internal # Primary network first to prioritize internal DNS resolution
      - ainews-external
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M

4. Real-World Troubleshooting

Issue 1: Cgroup Memory Limits Ignored under Standalone Compose

After starting the stack, running docker stats revealed that resource caps were not being enforced; the memory limits showed the host PC's full memory allocation (16GB).

Cause: On certain installations of modern RHEL/Rocky Linux environments running cgroups v2, memory controllers may not automatically map limits down to containers unless the system boot parameters explicitly delegate memory swapping and namespace control.
Resolution: Update the GRUB kernel parameters using grubby to ensure the memory cgroup controller is active at boot:

  # Check if memory controller is enabled in cgroups
  cat /proc/cgroups | grep memory

  # If disabled, enable memory and swap accounting via grubby
  sudo grubby --update-kernel=ALL --args="cgroup_enable=memory swapaccount=1"
  sudo reboot

After restarting, docker stats correctly recognized the limits (4GiB for DB, 1GiB for queue).

Issue 2: DNS Resolution Failures inside Dual-Network Containers

The collector container crashed immediately on startup, failing to connect to the DB. The logs outputted:
Error: getaddrinfo ENOTFOUND ainews-db

Cause: When a container is attached to multiple bridge networks (in this case, both ainews-internal and ainews-external), Docker's internal DNS resolver configures the search path based on the sequence in which networks are declared. If the external network is registered first, DNS requests for internal hosts (like ainews-db) may get routing priority on the external network gateway, failing to resolve the internal hostname.
Resolution: Reorder the networks block under the service definition in docker-compose.yml, placing the private network (ainews-internal) at the top of the array:

  networks:
    - ainews-internal  # Prioritize private network DNS
    - ainews-external

This forces the container's resolver to query the internal bridge first, resolving internal hostnames instantaneously.

Issue 3: Docker Compose .env Parsing Failure with Secret Special Characters

The database container failed to launch, reporting authentication syntax errors even though the password matched our .env configuration.

Cause: We generated a strong password containing a # character. The standard Docker Compose environment parser reads .env lines literally and interprets # as the beginning of a comment. Consequently, everything after the # character was stripped out, leaving PostgreSQL with a truncated password.
Resolution: Wrap any variable values containing special characters in double quotes (") inside the .env file to escape the parser:

  POSTGRES_PASSWORD="my#secret$password"

5. Results & Metrics

Here are the resource usage statistics captured under active crawling load:

Container	Memory Limit	Memory (Idle)	Memory (Peak)	CPU Limit	CPU (Peak Load)
ainews-db	4096 MiB	~120 MiB	1.8 GiB (RAG Queries)	2.0 Cores	~45%
ainews-queue	1024 MiB	~12 MiB	~85 MiB (High Queue Load)	1.0 Core	~5%
ainews-collector	512 MiB	~65 MiB	~180 MiB (Parallel Parsing)	1.0 Core	~35%

Zero Resource Contention: No matter how computationally intensive our RAG indexing operations get, ainews-db is strictly locked out of hogging the entire host memory, preserving a safe overhead of 1.5GB for host OS systemd tasks and the Cloudflare Tunnel daemon.
Port Scans Mitigated: Because our public-facing ports are bound internally and exposed strictly via cloudflared edge routing, external port scanning attempts against our home IP register a complete drop (closed status), keeping server audit logs clean of noise.

6. Next Up

With our backend services isolated and secured in Docker Compose, the next challenge is managing deployments. In Episode 5, we will set up Gitea (a lightweight 100MB self-hosted Git server) and Act Runner to configure a seamless, local CI/CD pipeline on push events.

[Homelab AI Project] Ep.3 - Zero Inbound Ports. Full Remote Dev Access. The Cloudflare Tunnel Architecture.

Tristan Kilhwan Chai — Tue, 14 Jul 2026 00:44:29 +0000

This post is Episode 3 of my Homelab AI Project series, where I document building an AI news automation pipeline on a mini PC homelab server instead of the cloud.

Korean version available on Velog---## Executive Summary

This post details how to build a Zero-Trust, zero-port-forwarding remote development infrastructure for a homelab server using Cloudflare Tunnel (cloudflared) and Cloudflare Access OTP. We'll cover the complete architecture - from OS-native systemd installation on the server side, to ProxyCommand-based VS Code Remote-SSH configuration on the client side - plus the real-world troubleshooting you won't find in the official docs.

1. The Security Anti-Pattern: Why Port Forwarding Is a Liability

Opening inbound ports via router port forwarding is the default "quick fix" for homelab remote access. The consequences are immediate and measurable.

Here's real Fail2ban log data captured 12 hours after enabling port 22 forwarding as a controlled test on our Rocky Linux 10.1 node:

# /var/log/fail2ban.log - 12 hours post-exposure
2026-07-10 02:14:38 fail2ban.actions: NOTICE [sshd] Ban 185.224.128.x  (CN)
2026-07-10 02:17:12 fail2ban.actions: NOTICE [sshd] Ban 45.83.65.x     (RU)
2026-07-10 03:01:55 fail2ban.actions: NOTICE [sshd] Ban 91.240.118.x   (RO)
...
# Total unique IPs banned in 12 hours:      247
# Total failed login attempts logged: 12,481

Over 12,000 automated brute-force attempts in half a day. Each blocked attempt still consumes SSH daemon processing cycles and triggers Fail2ban's iptables rule updates - a persistent, invisible tax on server resources.

Our architecture mandate: Zero public IP exposure. Zero inbound ports open.

2. The Architecture: Outbound-Only Tunnel Model

Cloudflare Tunnel (cloudflared) inverts the traditional connection model entirely.

Instead of opening inbound ports to accept connections, we run a lightweight cloudflared daemon inside our private network that establishes a persistent outbound HTTPS/TCP connection to Cloudflare's global edge network (321+ PoPs as of 2026).

The key security property: Malicious packets - port scans, brute-force attempts, vulnerability probes - never reach the homelab server. They terminate at Cloudflare's edge and are silently dropped before any traffic is forwarded into the private network.

The connection flow:

cloudflared agent (homelab) -> establishes persistent outbound tunnel to Cloudflare Edge
Developer (laptop) -> connects to ssh.your-domain.com (Cloudflare Edge)
Cloudflare Edge -> issues OTP challenge, verifies identity
Cloudflare Edge -> forwards verified traffic through existing tunnel
cloudflared -> proxies to local SSH daemon (port 22)

3. Server-Side Implementation: OS-Native systemd Installation

A critical architectural decision must be made upfront: do not run cloudflared as a Docker container.

If the Docker daemon crashes or fails to restart after an update, the cloudflared container dies with it - taking your entire SSH tunnel access down. You cannot diagnose what went wrong because the tunnel that would provide SSH access is itself inside the broken Docker stack. This is the equivalent of storing a lifeboat inside the hull of the ship.

cloudflared must live at the OS systemd layer, independent of Docker entirely. If Docker goes down, the systemd-managed tunnel remains alive, allowing SSH access to diagnose and recover the situation.

# 1. Install cloudflared binary via RPM (Rocky Linux / RHEL-based)
curl -L --output cloudflared.rpm \
  https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-x86_64.rpm
sudo rpm -i cloudflared.rpm

# 2. Verify installation
$ cloudflared --version
cloudflared version 2026.5.1 (built 2026-05-01-1234 UTC)

# 3. Register as systemd service using tunnel token from Cloudflare Zero Trust dashboard
sudo cloudflared service install $TUNNEL_TOKEN

# 4. Enable service and configure for automatic start at boot
sudo systemctl enable --now cloudflared

# 5. Verify status and measure resource consumption
$ systemctl status cloudflared
* cloudflared.service - Cloudflare Tunnel
     Loaded: loaded (/etc/systemd/system/cloudflared.service; enabled)
     Active: active (running) since 2026-07-10 09:12:34 KST; 3 days ago
   Main PID: 1842 (cloudflared)
      Tasks: 8
     Memory: 21.4M
        CPU: 1.234s

*Memory: 21.4 MB. CPU accumulated: 1.234 seconds over 3 days of uptime*This post is Episode 3 of my Homelab AI Project series, where I document building an AI news automation pipeline on a mini PC homelab server instead of the cloud.

Korean version available on Velog---## Executive Summary

1. The Security Anti-Pattern: Why Port Forwarding Is a Liability

Opening inbound ports via router port forwarding is the default "quick fix" for homelab remote access. The consequences are immediate and measurable.

Here's real Fail2ban log data captured 12 hours after enabling port 22 forwarding as a controlled test on our Rocky Linux 10.1 node:

# /var/log/fail2ban.log - 12 hours post-exposure
2026-07-10 02:14:38 fail2ban.actions: NOTICE [sshd] Ban 185.224.128.x  (CN)
2026-07-10 02:17:12 fail2ban.actions: NOTICE [sshd] Ban 45.83.65.x     (RU)
2026-07-10 03:01:55 fail2ban.actions: NOTICE [sshd] Ban 91.240.118.x   (RO)
...
# Total unique IPs banned in 12 hours:      247
# Total failed login attempts logged: 12,481

Our architecture mandate: Zero public IP exposure. Zero inbound ports open.

2. The Architecture: Outbound-Only Tunnel Model

Cloudflare Tunnel (cloudflared) inverts the traditional connection model entirely.

The connection flow:

cloudflared agent (homelab) -> establishes persistent outbound tunnel to Cloudflare Edge
Developer (laptop) -> connects to ssh.your-domain.com (Cloudflare Edge)
Cloudflare Edge -> issues OTP challenge, verifies identity
Cloudflare Edge -> forwards verified traffic through existing tunnel
cloudflared -> proxies to local SSH daemon (port 22)

3. Server-Side Implementation: OS-Native systemd Installation

A critical architectural decision must be made upfront: do not run cloudflared as a Docker container.

# 1. Install cloudflared binary via RPM (Rocky Linux / RHEL-based)
curl -L --output cloudflared.rpm \
  https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-x86_64.rpm
sudo rpm -i cloudflared.rpm

# 2. Verify installation
$ cloudflared --version
cloudflared version 2026.5.1 (built 2026-05-01-1234 UTC)

# 3. Register as systemd service using tunnel token from Cloudflare Zero Trust dashboard
sudo cloudflared service install $TUNNEL_TOKEN

# 4. Enable service and configure for automatic start at boot
sudo systemctl enable --now cloudflared

# 5. Verify status and measure resource consumption
$ systemctl status cloudflared
* cloudflared.service - Cloudflare Tunnel
     Loaded: loaded (/etc/systemd/system/cloudflared.service; enabled)
     Active: active (running) since 2026-07-10 09:12:34 KST; 3 days ago
   Main PID: 1842 (cloudflared)
      Tasks: 8
     Memory: 21.4M
        CPU: 1.234s

Memory: 21.4 MB. CPU accumulated: 1.234 seconds over 3 days of uptime - effectively idle. The OS boots cloudflared before Docker even starts, and it continues running regardless of Docker's health.

4. Zero-Trust Access Policy Configuration

Within the Cloudflare Zero Trust Dashboard (Access -> Applications), we define two types of access policies:

Web Service Routes:

git.your-domain.com -> internal Gitea container (gitea:3000)
dashboard.your-domain.com -> internal Astro dashboard (astro:4321)

SSH Service Route with MFA Guardrail:

ssh.your-domain.com -> local SSH daemon (localhost:22)
Policy: Include -> Email -> your-email@your-domain.com- Auth Method: One-time PIN (OTP) - enforced at edge before any tunnel traffic is forwarded

The policy enforcement model is critical: unauthenticated connection attempts to ssh.your-domain.com are terminated at the Cloudflare edge. The OTP challenge is issued before a single packet reaches the homelab server.

5. Client-Side Configuration: Bridging the Tunnel Locally

The developer's external laptop requires a local cloudflared binary to function as the SSH proxy client.

Step 1: Install cloudflared on the client machine

# Windows
winget install Cloudflare.cloudflared

# macOS
brew install cloudflare/cloudflare/cloudflared

Step 2: SSH Config with ProxyCommand

# ~/.ssh/config
Host home-server
    HostName ssh.your-domain.com
    User your-username
    Port 22
    ProxyCommand cloudflared access ssh --hostname %h
    # Windows: use absolute path if ProxyCommand cannot resolve PATH
    # ProxyCommand "C:\Tools\cloudflared.exe" access ssh --hostname %h
    IdentityFile ~/.ssh/id_rsa_homelab
    TCPKeepAlive yes
    ServerAliveInterval 60

Step 3: VS Code Remote-SSH connection

F1 -> SSH: Connect to Host... -> home-server

On first connection, a browser window opens to the Cloudflare Access OTP verification page. After email OTP verification, the tunnel session is established and VS Code's status bar shows [SSH: home-server].

6. Real-World Troubleshooting

Issue 1: `ProxyCommand` fails silently on Windows when the binary path contains spaces

winget installs cloudflared.exe to a path containing the Windows username (e.g., C:\Users\YourUsername\...). SSH config's ProxyCommand parser splits on spaces, causing the command to fail with a misleading error:

ssh: connect to host ssh.your-domain.com port 22: No such file or directory

Fix: Copy cloudflared.exe to a space-free path (e.g., C:\Tools\cloudflared.exe) and reference it directly:

ProxyCommand "C:\Tools\cloudflared.exe" access ssh --hostname %h

Issue 2: First connection hangs without triggering the OTP browser flow

The ProxyCommand silently times out on the first connection attempt if no local Cloudflare Access session token exists. You must initialize the local token cache explicitly:

cloudflared access login https://ssh.your-domain.com
# Browser opens -> OTP verification -> token cached to ~/.cloudflared/
# Subsequent SSH connections via ProxyCommand reuse this cached token

Issue 3: `cloudflared service install` embeds TUNNEL_TOKEN in plaintext

Running sudo cloudflared service install $TUNNEL_TOKEN writes the token directly into the ExecStart line of /etc/systemd/system/cloudflared.service. The token is also visible via ps aux and systemctl show. A leaked tunnel token enables full tunnel hijacking.

Fix: Use an EnvironmentFile for secret injection:

# 1. Isolate the token in a root-only env file (chmod 600)
sudo mkdir -p /etc/cloudflared
sudo tee /etc/cloudflared/env << 'EOF'
TUNNEL_TOKEN=your-actual-token-here
EOF
sudo chmod 600 /etc/cloudflared/env
sudo chown root:root /etc/cloudflared/env

# 2. Override the service file
sudo systemctl edit cloudflared

# /etc/systemd/system/cloudflared.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/cloudflared tunnel run
EnvironmentFile=/etc/cloudflared/env

sudo systemctl daemon-reload && sudo systemctl restart cloudflared
$ ps aux | grep cloudflared
root 1842  0.0  0.1 cloudflared tunnel run  # Token absent from args

7. Architecture Comparison

Metric	Port Forwarding	Cloudflare Tunnel (OS Native)
Inbound ports exposed	SSH 22, HTTP 80/443	0
Public IP visibility	Exposed in DNS A records	Hidden (Cloudflare IPs only)
Brute-force attack surface	Reaches server directly	Terminated at edge
Authentication layers	SSH key only	Edge OTP + SSH key (2-factor)
Dynamic IP handling	Requires DDNS	Handled automatically
SSH access if Docker fails	N/A	[OK] Survives - OS layer is independent
Monthly cost	$0 (but attack surface)	$0 (Free Tier)
Server resource overhead	CPU ~3% (Fail2ban)	Memory 21 MB, CPU ~0 (idle)

8. Next Episode

In Episode 4, we design the multi-service Docker Compose architecture - separating PostgreSQL, Redis, the Node.js collector, and the AI engine into isolated Docker networks with controlled inter-service communication.

Follow the series: Velog (Korean) | Dev.to

[Homelab AI Project] Ep.2 - Rocky Linux 10.x OS Hardening & Kernel Tuning

Tristan Kilhwan Chai — Mon, 13 Jul 2026 07:48:35 +0000

[Core] Executive Summary

An in-depth systems engineering guide to turning a budget Mini PC running bare-metal Rocky Linux 10.x into a resilient, high-concurrency database/message-queue node. This post covers optimizing base OS RAM usage under 150MB, securing ports with Fail2ban, configuring physical disk partition isolation, installing base host runtimes (Docker Engine & Docker Compose), and tuning core kernel parameters for Redis and PostgreSQL.

1. Introduction: Between Cloud Mazes and Terminal Dark Worlds

Deploying virtualized servers on public clouds like AWS is the industry standard for production environments. However, that convenience comes with a cost: staring at five browser tabs, hunting for hidden options, and clicking dropdown selectors like you're playing a hidden-object game in the AWS console.

To escape that, we retreat to the terminal--an unforgiving, dark space where a single typo can reduce your entire system to ashes. In the end, it's just about what risks you are willing to own and manage. The moment that lone cursor starts blinking, the reality of bare-metal homelabs hits you with a barrage of systems-level constraints:

IP Instability (DHCP): The home router changing the server IP on a whim, breaking all internal service links.
Immediate Port Scans: Automated bots knocking on Port 22 within minutes of bringing the server online.
Storage Disasters: Docker logging and DB data accumulating until the root filesystem hits 100% capacity, triggering a fatal OS kernel panic and locking you out of SSH.

If we don't isolate and secure these physical hardware risks at the OS layer, whatever shiny Node.js pipeline or RAG orchestration we build on top will just be a house of cards. Here is how I constructed the resilient foundation for my homelab server--along with a single script to automate the entire process.

2. Operating System Selection: "Why I Chose Rocky Linux 10.x Over Ubuntu"

Ubuntu Server is the absolute standard. It has the largest ecosystem, the most resources, and is a rock-solid choice for both home and enterprise production. I love Ubuntu and use it frequently.

However, for this project, I set aside Ubuntu's cozy ecosystem. Driven by personal familiarity, pure control, and a bit of extreme "hunger spirit" for efficiency, I opted for Rocky Linux 10.x (Minimal ISO).

Speed of Familiarity: You troubleshoot faster when you use tools that match your muscle memory. The robust directory layout of RHEL-family distributions and the strictness of its package manager make me feel more in control.
Extreme Memory Diet: By installing the Minimal ISO, I stripped out every unnecessary daemon and GUI component (like Gnome). The base RAM consumption at boot is a mere 140MiB.

$ free -h
               total        used        free      shared  buff/cache   available
Mem:            15Gi       140Mi        14Gi       8.0Mi       480Mi        15Gi
Swap:          2.0Gi          0B       2.0Gi

This minimal footprint ensures that almost 99% of my Mini PC's 16GB RAM is reserved entirely for memory-intensive PostgreSQL (pgvector) calculations and the Redis 7.2 queue.

3. Storage Design: Split Partition Isolation (Disk A vs. Disk B)

My Mini PC has two 512GB SSDs. Simply merging them into a single logical volume (LVM) was out of the question. To prevent a storage bottleneck from taking down the entire system, I isolated the partitions physically.

graph TD
    subgraph Mini_PC["Mini PC (Rocky Linux 10.x)"]
        subgraph Hardware["Physical Hardware"]
            SSD_A["SSD A (512GB)"]
            SSD_B["SSD B (512GB)"]
        end
        subgraph OS_Layer["OS Layer (RAM < 200MB)"]
            Kernel["Kernel (vm.overcommit_memory=1, vm.swappiness=10, net.core.somaxconn=1024)"]
            Firewall["Firewalld (White-listed IPs)"]
            Fail2ban["Fail2ban (Bans brute-force IPs)"]
            SSH["SSH Daemon (Port xxxx, No Root Login)"]
        end
        subgraph Storage_Mounts["Storage Mounts"]
            RootMount["/ (Root Filesystem)"]
            DataMount["/data (Data Filesystem)"]
        end
        subgraph Docker_Layer["Docker Container Layer (SELinux :z context)"]
            PostgreSQL["PostgreSQL 16 + pgvector"]
            Redis["Redis 7.2 (Queue/Cache)"]
            Apps["Collector, AI Engine, Sender"]
        end
    end

    SSD_A --> RootMount
    SSD_B --> DataMount
    RootMount --> OS_Layer
    DataMount -.->|Volume Bind Mount :z| Docker_Layer

Disk A (SSD 1 - 512GB): Mounted at / (Root). Contains only system configuration files and packages. Its size remains static.
Disk B (SSD 2 - 512GB): Mounted at /data.
- High-frequency read/write operations (Gitea repositories, workspace, PostgreSQL pgvector data directory, Redis AOF/RDB snapshots) are exiled to this /data mount.

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        492G  2.1G  465G   1% /
/dev/sdb1        492G   24K  467G   1% /data

The benefit is clear:
If a database log overflows or backup files consume 100% of the /data partition, the OS on Disk A stays alive. Essential system logs (syslog, journald) and system processes continue to execute, ensuring SSH access remains available for root administrator recovery. If the OS completely crashes, I can wipe Disk A, reinstall the OS, remount Disk B, and restore the service with zero data loss.

4. Network Configuration & Security Hardening

First, I set up a static IP within the local network using NetworkManager CLI (nmcli):

# nmcli manual static IP assignment
sudo nmcli connection modify eth0 ipv4.addresses "192.168.0.100/24"
sudo nmcli connection modify eth0 ipv4.gateway "192.168.0.1"
sudo nmcli connection modify eth0 ipv4.dns "1.1.1.1,8.8.8.8"
sudo nmcli connection modify eth0 ipv4.method manual
sudo nmcli connection up eth0

To block brute-force scanners, I disabled Root login and changed the default SSH port in /etc/ssh/sshd_config:

Port xxxx
PermitRootLogin no

Next, I deployed Fail2ban to act as a sentinel. Any IP that fails login 3 times is banned for 24 hours:

# /etc/fail2ban/jail.local
[sshd]
enabled = true
port = xxxx
filter = sshd
logpath = /var/log/secure
maxretry = 3
findtime = 600
bantime = 86400  # 24-hour ban

Security Note: This firewall hardening is just a baseline protection for the local network. Opening port forwarding on your router to expose this SSH port directly to the public internet will still fill your Fail2ban logs with thousands of blocks. The proper way to connect remotely without exposing any ports is through Cloudflare Tunnel, which we will set up in Episode 3.

5. Kernel Optimizations

Configure /etc/sysctl.d/99-sysctl.conf to handle high-concurrency requests and Redis database snapshots.

# 1. Prevent background persistence failures in Redis (BGSAVE)
vm.overcommit_memory = 1

# 2. Prevent latency spikes by limiting aggressive swapping
vm.swappiness = 10

# 3. Prevent packet drops under high-concurrency requests
net.core.somaxconn = 1024

Apply immediately:

sudo sysctl -p /etc/sysctl.d/99-sysctl.conf

[Warning] Today's Major Gotcha: SELinux vs Docker

Rocky Linux enables SELinux in Enforcing mode by default. If you attempt a standard Docker volume mount (- /data/postgres:/var/lib/postgresql/data), the container process will throw a Permission Denied error when writing to the host directory.

The Fix: Do not disable SELinux. Instead, append the :z flag in your Docker Compose volume bindings to trigger automatic SELinux security context labeling (e.g., - /data/postgres:/var/lib/postgresql/data:z).

6. Initialization Script (`setup.sh`)

To ensure this entire environment setup--including installing base host runtimes (Docker Engine, Docker Compose Plugin) and applying kernel/security hardening--is reproducible and safe from typos, use the following bash initialization script:

#!/bin/bash
# Rocky Linux 10.x Initialization & Optimization Script
# Execution: sudo bash setup.sh

set -e

echo "[+] Rocky Linux 10.x Optimization Script Started"

# 1. DNF Optimization
echo "[+] Optimizing DNF package manager..."
sudo tee -a /etc/dnf/dnf.conf <<EOF
max_parallel_downloads=10
fastestmirror=True
EOF

# 2. Update and Utilities Installation
echo "[+] Updating system packages & installing utilities..."
sudo dnf update -y
sudo dnf install -y epel-release dnf-plugins-core
sudo dnf install -y fail2ban git curl policycoreutils-python-utils

# 3. Docker Engine & Docker Compose Plugin Installation
echo "[+] Installing Docker Engine & Docker Compose Plugin..."
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker

# 4. Physical Partition Isolation Check
echo "[+] Checking physical partition isolation..."
df -h | grep "/data" || echo "[!] WARNING: /data mount point not detected. Ensure SSD B is mounted before running containers."

# 5. Firewall Configuration
echo "[+] Configuring firewalld for secure access..."
sudo systemctl enable --now firewalld
sudo firewall-cmd --new-zone=homelab-zone --permanent
sudo firewall-cmd --zone=homelab-zone --add-source=192.168.0.0/24 --permanent
sudo firewall-cmd --zone=homelab-zone --add-port=xxxx/tcp --permanent
sudo firewall-cmd --zone=homelab-zone --add-masquerade --permanent
sudo firewall-cmd --reload

# 6. SSH Daemon Hardening
echo "[+] Hardening SSH daemon settings (Port xxxx, No Root)..."
sudo sed -i 's/#Port 22/Port xxxx/' /etc/ssh/sshd_config
sudo sed -i 's/#PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
sudo sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config

# Register custom port to SELinux
echo "[+] Registering custom SSH port to SELinux..."
sudo semanage port -a -t ssh_port_t -p tcp xxxx || sudo semanage port -m -t ssh_port_t -p tcp xxxx
sudo systemctl restart sshd

# 7. Fail2ban Setup
echo "[+] Setting up Fail2ban brute-force protection..."
sudo tee /etc/fail2ban/jail.local <<EOF
[sshd]
enabled = true
port = xxxx
filter = sshd
logpath = /var/log/secure
maxretry = 3
findtime = 600
bantime = 86400
EOF
sudo systemctl enable --now fail2ban

# 8. Kernel Optimizations
echo "[+] Injecting Kernel parameter optimizations..."
sudo tee /etc/sysctl.d/99-sysctl.conf <<EOF
vm.overcommit_memory = 1
vm.swappiness = 10
net.core.somaxconn = 1024
EOF
sudo sysctl -p /etc/sysctl.d/99-sysctl.conf

echo "[+] System configuration successfully completed!"

7. Next Episode

Now we have a solid bare-metal platform running on under 150MB of idle RAM, hardened with a firewall, and isolated at the disk level. But coding strictly within the local home network is a bottleneck. We need a way to access and develop on this server securely from cafes or external workspaces.

In Episode 3, we will establish external access without port forwarding using Cloudflare Tunnel (Zero-Trust). We will cover both HTTPS routing for web applications and a secure SSH tunneling architecture using Cloudflare Access OTP and VS Code Remote-SSH.

[Homelab AI Project] Ep.01 - Dragging an Idle Mini PC into the Production Game (Prologue)

Tristan Kilhwan Chai — Thu, 09 Jul 2026 00:46:19 +0000

0. Dragging an Idle Mini PC into the Game

Today, everyone talks about AI by throwing massive cloud budgets at heavy GPU instances. But my architect instinct always asks a different question:
"Do we really need a massive cloud budget to build a resilient, production-grade enterprise system?"

That's when an idle mini PC collecting dust in my room caught my eye. A modest AMD Ryzen 5500U with DDR4 16GB RAM—the kind of low-spec hardware you easily find on second-hand markets—is now my official server.

On this limited hardware, I'm spinning up Rocky Linux simply because it feels way more comfortable to me than Ubuntu. Without spending a single penny on cloud hosting—well, except for my home electricity bill... wait, anyway—I am kicking off a comprehensive series documenting how to build a production-grade AI Messaging and RAG Platform from scratch.

1. The Architectural Blueprint

This is not a simple tutorial or a casual toy project. To extract maximum efficiency out of this limited hardware, we will implement high-level engineering patterns used in real enterprise environments:

Resource Isolation: Setting up Docker containers on Rocky Linux 10.1 with strict CPU/Memory limits to completely prevent resource contention.
Zero-Trust Private Networking: Utilizing Cloudflare Tunnel to expose services securely to the edge without opening any router ports or risking public IP exposure.
High-Throughput Async Pipelines: Designing a multi-module Node.js and Python backend controlled by Redis Queue to manage event-driven throttling and backpressure during heavy data ingestion.
FinOps-Focused AI & RAG: Integrating a local Vector DB (pgvector) to de-duplicate data and implementing hybrid AI routing algorithms to keep LLM token costs under absolute control.

2. Why Am I Doing This?

Because I firmly believe that true engineering excellence is defined by architectural density, not the size of your capital. This series is dedicated to indie developers, startup architects, and tech enthusiasts who want to build high-performance systems under tight budget realities.

I wanted to use this mini PC as a personal playground to realize, organize, and test years of accumulated knowledge in backend optimization, network troubleshooting, and infrastructure security.

The stage is set. In the next episode, we open the terminal to tackle the core foundation: Rocky Linux 10.1 environment configuration and strict Docker isolation design.

If you are ready to explore the boundaries of software efficiency, follow along. Let's build.

DEV Community: Tristan Kilhwan Chai

[Homelab AI Project] Ep.7 - Designing a Lightweight Monorepo Architecture with Node.js and Python

1. Background: "Spring Boot is Great, but My 16GB RAM is Priceless"

2. The Solution: Node.js & Python Monorepo Layout

3. Implementation: Root Setup & Module Scaffolding

(1) Root Configuration (package.json)

(2) Shared TypeScript Rules (tsconfig.json)

(3) The Core Package (packages/core)

4. Troubleshooting & Debugging Logs

🚨 Issue 1: npm Workspaces Hoisting and Missing Local Build Files

🚨 Issue 2: Docker Bridge DNS Resolving Failures (Connection Refused)

5. Results & Metrics: Spring Boot vs. Node.js/Python

6. Next Up

[Homelab AI Project] Ep.6 - Aiming for a 0MB Memory Footprint on Frontend Servers with Astro and React Islands

1. Background: "Zero RAM Left to Waste on Frontend Servers"

2. The Solution: Island Architecture (Astro + React)

3. Implementation: Packages & Configuration

(1) Package Directory Layout

(2) Astro Configuration (astro.config.mjs)

(2.5) Essential Node.js Package Configuration (package.json & tsconfig.json)

(3) Lightweight CSS Design Tokens (global.css)

(4) React Island Live Chart Component (LiveDemoChart.jsx)

4. Troubleshooting & Debugging logs

🚨 Issue 1: window is not defined Build Failure

🚨 Issue 2: Style Flash (FOUC) during Page Reloads

🚨 Issue 3: Missing Node.js/npm Runtime on the Homelab Server

🚨 Issue 4: Empty Boilerplate package.json and EJSONPARSE Failures

🚨 Issue 5: Astro's Host Header Restrictions over Cloudflare Tunnel

5. Results & Metrics: Next.js vs. Astro

6. Next Up

[Homelab AI Project] Ep.5 - Building a Lightweight Self-Hosted CI/CD Pipeline with Gitea and Act Runner

1. The Challenge: Keeping Git and CI/CD under 150MB of RAM

2. Topology: Internal Network Bridging and Docker Daemon Delegation

① Host Docker Daemon Binding (/var/run/docker.sock)

② Loopback and Internal DNS Direct Routing

3. Setup: infra/docker-compose.gitea.yml & Workflow

📄 infra/docker-compose.gitea.yml

📄 .gitea/workflows/deploy.yml

4. Real-World Troubleshooting

🚨 Issue 1: Runner Loopback Failure via External Domain Addresses

🚨 Issue 2: Docker Socket Bind Permission Denied (GID Mismatch)

🚨 Issue 3: Missing 'Actions' Repository Settings UI

🚨 Issue 4: Disappearing Host Port Bindings (3000) leading to 502 Bad Gateway

🚨 Issue 5: 'token is empty' Registration Failure for Gitea Act Runner

5. Results & Metrics

Idle Hardware Resource Usage Comparison

6. Next Up

[Homelab AI Project] Ep.4 - Isolating the Service Layer with Docker Compose

Series: Building a Personalized AI News Pipeline (4/23)

Executive Summary

1. The Challenge: Monolithic Mess vs. Virtualization Overhead

2. The Architecture: Cgroups and Network Segmentation

1. Resource Constraints via cgroups

2. Dual-Network Partitioning

3. Complete Configuration Decoupling

3. Setup Files: infra/docker-compose.yml & .env.example

infra/.env.example

infra/docker-compose.yml

4. Real-World Troubleshooting

Issue 1: Cgroup Memory Limits Ignored under Standalone Compose

Issue 2: DNS Resolution Failures inside Dual-Network Containers

Issue 3: Docker Compose .env Parsing Failure with Secret Special Characters

5. Results & Metrics

6. Next Up

[Homelab AI Project] Ep.3 - Zero Inbound Ports. Full Remote Dev Access. The Cloudflare Tunnel Architecture.

1. The Security Anti-Pattern: Why Port Forwarding Is a Liability

2. The Architecture: Outbound-Only Tunnel Model

3. Server-Side Implementation: OS-Native systemd Installation

1. The Security Anti-Pattern: Why Port Forwarding Is a Liability

2. The Architecture: Outbound-Only Tunnel Model

3. Server-Side Implementation: OS-Native systemd Installation

4. Zero-Trust Access Policy Configuration

5. Client-Side Configuration: Bridging the Tunnel Locally

6. Real-World Troubleshooting

Issue 1: ProxyCommand fails silently on Windows when the binary path contains spaces

Issue 2: First connection hangs without triggering the OTP browser flow

Issue 3: cloudflared service install embeds TUNNEL_TOKEN in plaintext

7. Architecture Comparison

8. Next Episode

[Homelab AI Project] Ep.2 - Rocky Linux 10.x OS Hardening & Kernel Tuning

(1) Root Configuration (`package.json`)

(2) Shared TypeScript Rules (`tsconfig.json`)

(3) The Core Package (`packages/core`)

(2) Astro Configuration (`astro.config.mjs`)

(2.5) Essential Node.js Package Configuration (`package.json` & `tsconfig.json`)

(3) Lightweight CSS Design Tokens (`global.css`)

(4) React Island Live Chart Component (`LiveDemoChart.jsx`)

🚨 Issue 1: `window is not defined` Build Failure

🚨 Issue 4: Empty Boilerplate `package.json` and `EJSONPARSE` Failures

① Host Docker Daemon Binding (`/var/run/docker.sock`)

📄 `infra/docker-compose.gitea.yml`

📄 `.gitea/workflows/deploy.yml`

Issue 1: `ProxyCommand` fails silently on Windows when the binary path contains spaces

Issue 3: `cloudflared service install` embeds TUNNEL_TOKEN in plaintext

6. Initialization Script (`setup.sh`)