DEV Community: 프링글리스

Building a Mini DBaaS with Kubernetes: A DBA's Cloud-Native Engineering Journey

프링글리스 — Sun, 20 Jul 2025 08:27:46 +0000

Why Did I Build This?

As a Database Administrator (DBA), I've always been curious about how cloud database services like AWS RDS work internally. Rather than just being a consumer of such services, I wanted to understand the engineering challenges of building a Database-as-a-Service (DBaaS) platform.

I was particularly fascinated by AWS Aurora MySQL's fast snapshot creation and cluster restoration capabilities, and wanted to implement these advanced features myself. I also wanted to build a complete DBaaS platform that supports various databases (PostgreSQL, MySQL, MariaDB) with high availability and automatic failover capabilities.

🎯 Development Motivation and Goals

Node.js Learning: Hands-on project to strengthen backend development capabilities
Kubernetes Understanding: Acquiring cloud-native technologies as a DBA
AWS Aurora-style Feature Implementation: Fast snapshots and cluster restoration
High Availability System Construction: HA clusters with automatic failover
Scaling Feature Implementation: Dynamic resource allocation and horizontal/vertical scaling
Restoration Feature Implementation: AWS Aurora-style fast restoration and cross-instance restoration
Multi-Database Support: Integrated management of PostgreSQL, MySQL, MariaDB

💡 Development Tool Investment

Initially, my development skills were limited, so I invested about $40 to purchase Cursor IDE. This tool provides AI-based code generation and autocomplete features, which greatly helped in writing complex Kubernetes manifests and Node.js backend code. I was able to efficiently write complex YAML files like Helm chart templates and Kubernetes Operator configurations.

The goal was simple: Build a fully functional DBaaS using Node.js and Kubernetes in just one week to gain practical experience with cloud-native technologies and deepen my understanding of distributed systems.

The Challenges

Building a DBaaS involves mastering several complex components:

Database Deployment Automation (supporting multiple database types)
Multi-tenant Isolation (proper resource management)
Backup and Recovery Systems (point-in-time recovery)
High Availability Clustering (automatic failure recovery)
Real-time Monitoring and health checks
AWS Aurora-style Snapshot System (fast backup/restoration)

Major Problems I Encountered

1. Complexity of Kubernetes StatefulSets

Managing stateful databases in Kubernetes was trickier than expected. I had to learn:

Persistent Volume Claims (PVC) for data persistence
CSI VolumeSnapshots for backup/recovery
Proper resource allocation and limits
Namespace isolation for multi-tenancy

Solution: Created custom Helm charts with appropriate StatefulSet configurations for each database type.

2. Multi-Database Support and High Availability Implementation

Each database (PostgreSQL, MySQL, MariaDB) has different deployment patterns:

PostgreSQL: Zalando PostgreSQL Operator for HA clusters (✅ Success)
MySQL/MariaDB: Custom StatefulSet with monitoring exporters (❌ HA implementation failed)
Different configuration requirements and connection patterns

Solution: Built an integrated API that abstracts database-specific differences while leveraging the advantages of each database type. For PostgreSQL, I successfully integrated the Zalando Operator, but MySQL HA cluster implementation was limited to single instances due to complexity.

3. AWS Aurora-style Backup and Recovery System

Implementing Aurora's fast snapshot creation and cluster restoration capabilities was a core goal:

CSI VolumeSnapshots for storage-level backup
Cross-instance backup restoration
Backup verification and testing
5-10 second snapshot creation (Aurora-level performance target)

Solution: Used CSI VolumeSnapshots with hostpath-driver for fast storage-level backup that works across all database types. I actually achieved Aurora-like fast backup performance (5-10 seconds) on empty databases.

4. High Availability Clustering (PostgreSQL vs MySQL)

I discovered interesting differences when setting up HA clusters with automatic failure recovery:

PostgreSQL HA (✅ Success):

Zalando PostgreSQL Operator integration
Master/Replica service separation
Automatic failure detection and recovery

MySQL HA (❌ Failed):

Percona XtraDB Cluster complexity
Difficulty in Group Replication setup
Limitations of Operator patterns

Solution: For PostgreSQL, I successfully integrated the Zalando PostgreSQL Operator for production-grade HA clusters with automatic failure recovery. MySQL is currently limited to single instances, with plans for HA implementation through MySQL Operator or Percona Operator in the future.

What I Built

After one week of development, I had a working DBaaS platform with the following capabilities:

✅ Completed Features

Multi-Database Support: PostgreSQL, MySQL, MariaDB instances
High Availability: PostgreSQL HA clusters with automatic failure recovery
AWS Aurora-style Backup/Recovery: CSI VolumeSnapshot-based fast snapshots (5-10 seconds)
RESTful API: Complete CRUD operations for instance management
Real-time Monitoring: Pod status, resource usage, health checks
Multi-tenant Isolation: Namespace-based resource isolation
Resource Scaling: Dynamic CPU/memory allocation

🚧 Current Limitations

No Web UI: Currently CLI/API only (planned for Phase 1)
MySQL HA: Only PostgreSQL HA clusters supported (limited due to MySQL HA implementation failure)
Monitoring: Basic monitoring only (Prometheus/Grafana planned)
Security: Basic authentication only (JWT/RBAC planned)
Multi-tenancy: Basic namespace isolation only (advanced features planned)

📊 Performance Metrics

Backup Creation: 5-10 seconds (Aurora level, empty database basis)
Database Restoration: Within 30 seconds (empty database basis)
Instance Deployment: Within seconds
HA Failover: Automatic detection and recovery

💡 Note: Backup/restoration times are based on empty databases. In actual production environments, times may vary depending on data size.

Technical Architecture

User Request → Node.js API → Kubernetes → Database Instances
                ↓
        CSI VolumeSnapshots (Aurora-style backup/recovery)
                ↓
    PostgreSQL HA Cluster (Zalando Operator)
                ↓
        Real-time Monitoring and Health Checks

Technology Stack

Backend: Node.js + Express
Orchestration: Kubernetes + Helm
Databases: PostgreSQL, MySQL, MariaDB
High Availability: Zalando PostgreSQL Operator
Backup/Recovery: CSI VolumeSnapshots (Aurora-style)
Monitoring: Real-time pod/helm status tracking
Development Tools: Cursor IDE (AI-based code generation)

Key Learnings

1. Advanced Kubernetes Learning

StatefulSets are powerful for database workloads but complex
CSI VolumeSnapshots provide Aurora-level backup functionality
Namespace isolation is crucial for multi-tenant environments
Resource quotas prevent resource exhaustion

2. Database Operations in Kubernetes

Helm charts make database deployment much easier
Operators provide production-grade database management
Health checks are essential for stable database operations
Configuration management through ConfigMaps is elegant

3. Cloud-Native Patterns

API-first design enables automation and integration
Event-driven architecture improves scalability
Infrastructure as Code through Helm charts
Observability through structured logging and metrics

4. Importance of Development Tools

Cursor IDE's AI-based code generation greatly helped with complex Kubernetes manifest writing
AI tool utilization significantly improved development productivity and learning speed
Appropriate tool investment plays a crucial role in project success

Results

My mini DBaaS can now:

Deploy PostgreSQL, MySQL, MariaDB instances in seconds
Provide high-availability PostgreSQL clusters with automatic failure recovery
Create Aurora-style backups in 5-10 seconds (goal achieved!, empty database basis)
Restore databases within 30 seconds (empty database basis)
Scale resources dynamically
Monitor health and performance in real-time

Next Steps

Based on this experience, I created a comprehensive roadmap for future improvements:

Phase 1 (1-2 weeks)

React web UI for visual management
MySQL HA Cluster Retry (Percona XtraDB Operator or MySQL Operator)
Prometheus + Grafana monitoring stack

Phase 2 (3-4 weeks)

Automated backup scheduling
JWT-based authentication and RBAC
Performance monitoring dashboard

Phase 3 (5-8 weeks)

Advanced multi-tenant features
Security enhancements (encryption, audit logs)
Cloud provider integration

Why This Matters

This project taught me that building cloud services isn't just about technology, but understanding the operational challenges that arise when managing databases at scale. As a DBA, this experience provided:

Deep understanding of cloud-native architectures
Practical experience with Kubernetes and containerization
Insights into how cloud providers solve database challenges
Confidence in handling complex distributed systems
Experience with modern development methodologies using AI tools

Check It Out

The complete source code is available on GitHub:
https://github.com/JungyeolHwang/DBaaS

Quick Start

# Clone repository
git clone https://github.com/JungyeolHwang/DBaaS.git
cd DBaaS

# Run setup script
./scripts/setup.sh

# Start API server
cd backend && npm start

# Create first database
curl -X POST http://localhost:3000/instances \
  -H "Content-Type: application/json" \
  -d '{
    "type": "postgresql",
    "name": "my-first-db",
    "config": {
      "password": "securepass123",
      "storage": "2Gi"
    }
  }'

Conclusion

Building this mini DBaaS was an amazing learning experience. It showed that with the right tools and understanding, you can build production-ready database services even as side projects.

Investing in Cursor IDE, an AI tool, played a significant role in the project's success. I was able to efficiently write complex Kubernetes manifests and Node.js backend code, which was a great help in the early development stages.

Implementing AWS Aurora-style fast snapshot functionality was a core goal, and I achieved the target of 5-10 second backup creation using CSI VolumeSnapshots on empty databases. While actual production environments may have different backup times depending on data size, I was able to implement Aurora-like fast backup performance using storage-level snapshots. However, MySQL HA cluster implementation was more complex than expected, so currently only PostgreSQL is supported, but this is included in future improvement plans.

High Availability System Construction was successful for PostgreSQL through successful integration of the Zalando Operator for HA clusters with automatic failover. However, MySQL HA cluster implementation was more complex than expected, so currently only PostgreSQL is supported, but this is included in future improvement plans.

Scaling features and fast restoration features were also important goals. I implemented scaling through Kubernetes' dynamic resource allocation and achieved Aurora-like fast restoration performance using CSI VolumeSnapshots.

The journey from simple database management to cloud-native engineering was eye-opening. Kubernetes, Helm, and modern DevOps practices completely changed how I think about database operations.

For DBAs who want to expand their skills into cloud-native engineering, I strongly recommend building something similar. Start small, focus on core features, and gradually add complexity. And consider investing in appropriate development tools if needed!

What will your next cloud-native project be?

kubernetes #database #dba #cloud-native #nodejs #postgresql #mysql #mariadb #side-project #engineering #devops #cursor-ide #aws-aurora #ha-clustering

This project was built as a learning exercise to understand cloud-native database services. Feel free to contribute, fork, or use as inspiration for your own projects!

Kubernetes로 Mini DBaaS 구축하기: DBA의 클라우드 네이티브 엔지니어링 도전기

프링글리스 — Sun, 20 Jul 2025 08:25:42 +0000

왜 만들게 되었나요?

데이터베이스 관리자(DBA)로서 항상 AWS RDS 같은 클라우드 데이터베이스 서비스가 내부적으로 어떻게 동작하는지 궁금했습니다. 단순히 이런 서비스의 소비자가 되는 것이 아니라, Database-as-a-Service(DBaaS) 플랫폼을 구축하는 엔지니어링 도전과제를 이해하고 싶었습니다.

특히 AWS Aurora MySQL의 빠른 스냅샷 생성 및 클러스터 복원 기능에 매료되어, 이런 고급 기능들을 직접 구현해보고 싶었습니다. 또한 다양한 데이터베이스(PostgreSQL, MySQL, MariaDB)를 지원하면서 고가용성과 자동 페일오버 기능까지 갖춘 완전한 DBaaS 플랫폼을 만들어보고 싶었습니다.

💡 개발 동기와 목표

Node.js 학습: 백엔드 개발 역량 강화를 위한 실전 프로젝트
Kubernetes 이해: DBA로서 클라우드 네이티브 기술 습득
AWS Aurora 스타일 기능 구현: 빠른 스냅샷과 클러스터 복원
고가용성 시스템 구축: 자동 페일오버가 포함된 HA 클러스터
스케일링 기능 구현: 동적 리소스 할당 및 수평/수직 확장
복원 기능 구현: AWS Aurora 스타일의 빠른 복원 및 크로스 인스턴스 복원
다중 데이터베이스 지원: PostgreSQL, MySQL, MariaDB 통합 관리

💡 개발 도구 투자

처음에는 개발 실력이 부족했기 때문에 Cursor IDE를 약 40달러 투자하여 구매했습니다. 이 도구는 AI 기반 코드 생성과 자동완성 기능을 제공하여, 복잡한 Kubernetes 매니페스트와 Node.js 백엔드 코드 작성에 큰 도움이 되었습니다. 특히 Helm 차트 템플릿과 Kubernetes Operator 설정 같은 복잡한 YAML 파일들을 효율적으로 작성할 수 있었습니다.

목표는 간단했습니다: Node.js와 Kubernetes를 사용해서 1주일 만에 완전히 동작하는 DBaaS를 구축하여 클라우드 네이티브 기술에 대한 실무 경험을 쌓고 분산 시스템에 대한 이해를 깊이하는 것이었습니다.

도전 과제

DBaaS를 구축하는 것은 제가 마스터해야 할 여러 복잡한 컴포넌트가 포함되어 있습니다:

데이터베이스 배포 자동화 (여러 데이터베이스 타입 지원)
멀티 테넌트 격리 (적절한 리소스 관리)
백업 및 복구 시스템 (포인트 인 타임 복구)
고가용성 클러스터링 (자동 장애 복구)
실시간 모니터링 및 헬스 체크
AWS Aurora 스타일 스냅샷 시스템 (빠른 백업/복원)

마주친 주요 문제들

1. Kubernetes StatefulSet의 복잡성

Kubernetes에서 상태 유지 데이터베이스를 관리하는 것은 예상보다 까다로웠습니다. 다음을 배워야 했습니다:

데이터 지속성을 위한 Persistent Volume Claims (PVC)
백업/복구를 위한 CSI VolumeSnapshots
적절한 리소스 할당 및 제한
멀티 테넌시를 위한 네임스페이스 격리

해결책: 각 데이터베이스 타입별로 적절한 StatefulSet 구성이 포함된 커스텀 Helm 차트를 생성했습니다.

2. 다중 데이터베이스 지원과 고가용성 구현

각 데이터베이스(PostgreSQL, MySQL, MariaDB)는 서로 다른 배포 패턴을 가집니다:

PostgreSQL: HA 클러스터를 위한 Zalando PostgreSQL Operator (✅ 성공)
MySQL/MariaDB: 모니터링 익스포터가 포함된 커스텀 StatefulSet (❌ HA 구현 실패)
서로 다른 설정 요구사항과 연결 패턴

해결책: 데이터베이스별 차이점을 추상화하면서 각 데이터베이스 타입의 장점을 활용하는 통합 API를 구축했습니다. PostgreSQL의 경우 Zalando Operator를 성공적으로 통합했지만, MySQL HA 클러스터 구현은 복잡성으로 인해 단일 인스턴스로 제한했습니다.

3. AWS Aurora 스타일 백업 및 복구 시스템

Aurora의 빠른 스냅샷 생성과 클러스터 복원 기능을 구현하는 것이 핵심 목표였습니다:

스토리지 레벨 백업을 위한 CSI VolumeSnapshots
크로스 인스턴스 백업 복원
백업 검증 및 테스트
5-10초 내 스냅샷 생성 (Aurora 수준의 성능 목표)

해결책: 모든 데이터베이스 타입에서 작동하는 빠른 스토리지 레벨 백업을 위해 hostpath-driver가 포함된 CSI VolumeSnapshots를 사용했습니다. 빈 데이터베이스 기준으로 Aurora와 유사한 수준의 빠른 백업 성능(5-10초)을 달성할 수 있었습니다.

4. 고가용성 클러스터링 (PostgreSQL vs MySQL)

자동 장애 복구가 포함된 HA 클러스터 설정에서 흥미로운 차이점을 발견했습니다:

PostgreSQL HA (✅ 성공):

Zalando PostgreSQL Operator 통합
Master/Replica 서비스 분리
자동 장애 감지 및 복구

MySQL HA (❌ 실패):

Percona XtraDB Cluster 복잡성
Group Replication 설정의 어려움
Operator 패턴의 한계

해결책: PostgreSQL의 경우 자동 장애 복구가 포함된 프로덕션급 HA 클러스터를 위해 Zalando PostgreSQL Operator를 성공적으로 통합했습니다. MySQL은 현재 단일 인스턴스로 제한하고, 향후 MySQL Operator나 Percona Operator를 통한 HA 구현을 계획하고 있습니다.

무엇을 만들었나요

1주일의 개발 후, 다음과 같은 동작하는 DBaaS 플랫폼을 갖게 되었습니다:

✅ 완성된 기능들

다중 데이터베이스 지원: PostgreSQL, MySQL, MariaDB 인스턴스
고가용성: 자동 장애 복구가 포함된 PostgreSQL HA 클러스터
AWS Aurora 스타일 백업/복구: CSI VolumeSnapshot 기반 빠른 스냅샷 (5-10초)
RESTful API: 인스턴스 관리를 위한 완전한 CRUD 작업
실시간 모니터링: Pod 상태, 리소스 사용량, 헬스 체크
멀티 테넌트 격리: 네임스페이스 기반 리소스 격리
리소스 스케일링: 동적 CPU/메모리 할당

🚧 현재 한계점들

웹 UI 없음: 현재 CLI/API만 지원 (Phase 1에서 계획)
MySQL HA: PostgreSQL HA 클러스터만 지원 (MySQL HA 구현 실패로 인한 제한)
모니터링: 기본 모니터링만 (Prometheus/Grafana 계획)
보안: 기본 인증만 (JWT/RBAC 계획)
멀티 테넌시: 기본 네임스페이스 격리만 (고급 기능 계획)

📊 성능 지표

백업 생성: 5-10초 (Aurora 수준, 빈 데이터베이스 기준)
데이터베이스 복원: 30초 내 (빈 데이터베이스 기준)
인스턴스 배포: 몇 초 내
HA 페일오버: 자동 감지 및 복구

💡 참고: 백업/복원 시간은 빈 데이터베이스 기준입니다. 실제 운영 환경에서는 데이터 크기에 따라 시간이 달라질 수 있습니다.

기술적 아키텍처

사용자 요청 → Node.js API → Kubernetes → 데이터베이스 인스턴스
                ↓
        CSI VolumeSnapshots (Aurora 스타일 백업/복구)
                ↓
    PostgreSQL HA 클러스터 (Zalando Operator)
                ↓
        실시간 모니터링 및 헬스 체크

기술 스택

백엔드: Node.js + Express
오케스트레이션: Kubernetes + Helm
데이터베이스: PostgreSQL, MySQL, MariaDB
고가용성: Zalando PostgreSQL Operator
백업/복구: CSI VolumeSnapshots (Aurora 스타일)
모니터링: 실시간 pod/helm 상태 추적
개발 도구: Cursor IDE (AI 기반 코드 생성)

주요 학습 내용

1. Kubernetes 심화 학습

StatefulSet은 데이터베이스 워크로드에 강력하지만 복잡함
CSI VolumeSnapshots은 Aurora 수준의 백업 기능 제공
네임스페이스 격리는 멀티 테넌트 환경에 중요
리소스 할당량은 리소스 고갈을 방지

2. Kubernetes에서의 데이터베이스 운영

Helm 차트는 데이터베이스 배포를 훨씬 쉽게 만듦
Operator는 프로덕션급 데이터베이스 관리 제공
헬스 체크는 안정적인 데이터베이스 운영에 필수
ConfigMap을 통한 설정 관리는 우아함

3. 클라우드 네이티브 패턴

API 우선 설계는 자동화와 통합을 가능하게 함
이벤트 기반 아키텍처는 확장성 향상
Helm 차트를 통한 Infrastructure as Code
구조화된 로깅과 메트릭을 통한 관찰성

4. 개발 도구의 중요성

Cursor IDE의 AI 기반 코드 생성이 복잡한 Kubernetes 매니페스트 작성에 큰 도움
AI 도구 활용이 개발 생산성과 학습 속도를 크게 향상시킴
적절한 도구 투자가 프로젝트 성공에 중요한 역할

결과

제 미니 DBaaS는 이제 다음을 할 수 있습니다:

PostgreSQL, MySQL, MariaDB 인스턴스를 몇 초 만에 배포
자동 장애 복구가 포함된 고가용성 PostgreSQL 클러스터 제공
5-10초 내에 Aurora 스타일 백업 생성 (목표 달성!, 빈 데이터베이스 기준)
30초 내에 데이터베이스 복원 (빈 데이터베이스 기준)
리소스를 동적으로 스케일링
실시간으로 헬스와 성능 모니터링

다음 단계

이 경험을 바탕으로 향후 개선을 위한 포괄적인 로드맵을 만들었습니다:

Phase 1 (1-2주)

시각적 관리를 위한 React 웹 UI
MySQL HA 클러스터 재도전 (Percona XtraDB Operator 또는 MySQL Operator)
Prometheus + Grafana 모니터링 스택

Phase 2 (3-4주)

자동화된 백업 스케줄링
JWT 기반 인증 및 RBAC
성능 모니터링 대시보드

Phase 3 (5-8주)

고급 멀티 테넌트 기능
보안 강화 (암호화, 감사 로그)
클라우드 프로바이더 통합

왜 이것이 중요한가

이 프로젝트는 클라우드 서비스를 구축하는 것이 단순히 기술에 관한 것이 아니라, 대규모로 데이터베이스를 관리할 때 발생하는 운영상의 도전과제를 이해하는 것에 관한 것임을 가르쳐주었습니다. DBA로서 이 경험은 다음과 같은 것을 제공했습니다:

클라우드 네이티브 아키텍처에 대한 깊은 이해
Kubernetes와 컨테이너화에 대한 실무 경험
클라우드 프로바이더가 데이터베이스 도전과제를 해결하는 방법에 대한 통찰
복잡한 분산 시스템을 다루는 자신감
AI 도구를 활용한 현대적 개발 방법론 경험

확인해보세요

전체 소스코드는 GitHub에서 확인할 수 있습니다:
https://github.com/JungyeolHwang/DBaaS

빠른 시작

# 저장소 클론
git clone https://github.com/JungyeolHwang/DBaaS.git
cd DBaaS

# 설정 스크립트 실행
./scripts/setup.sh

# API 서버 시작
cd backend && npm start

# 첫 번째 데이터베이스 생성
curl -X POST http://localhost:3000/instances \
  -H "Content-Type: application/json" \
  -d '{
    "type": "postgresql",
    "name": "my-first-db",
    "config": {
      "password": "securepass123",
      "storage": "2Gi"
    }
  }'

결론

이 미니 DBaaS를 구축하는 것은 놀라운 학습 경험이었습니다. 올바른 도구와 이해를 가지고 있다면 사이드 프로젝트로도 프로덕션 준비가 된 데이터베이스 서비스를 구축할 수 있다는 것을 보여주었습니다.

특히 Cursor IDE라는 AI 도구에 투자한 것이 프로젝트 성공에 큰 역할을 했습니다. 복잡한 Kubernetes 매니페스트와 Node.js 백엔드 코드를 효율적으로 작성할 수 있었고, 이는 개발 초기 단계에서 큰 도움이 되었습니다.

AWS Aurora 스타일의 빠른 스냅샷 기능을 구현하는 것이 핵심 목표였는데, CSI VolumeSnapshots를 통해 빈 데이터베이스 기준으로 5-10초 내 백업 생성이라는 목표를 달성할 수 있었습니다. 실제 운영 환경에서는 데이터 크기에 따라 백업 시간이 달라질 수 있지만, 스토리지 레벨 스냅샷을 사용하여 Aurora와 유사한 빠른 백업 성능을 구현할 수 있었습니다.

고가용성 시스템 구축에서는 PostgreSQL의 경우 Zalando Operator를 성공적으로 통합하여 자동 페일오버가 포함된 HA 클러스터를 구현할 수 있었습니다. 다만 MySQL HA 클러스터 구현은 예상보다 복잡하여 현재는 PostgreSQL만 지원하고 있지만, 이는 향후 개선 계획에 포함되어 있습니다.

스케일링 기능과 빠른 복원 기능도 중요한 목표였는데, Kubernetes의 동적 리소스 할당을 통해 스케일링을 구현했고, CSI VolumeSnapshots를 활용하여 AWS Aurora와 유사한 빠른 복원 성능을 달성할 수 있었습니다.

단순한 데이터베이스 관리에서 클라우드 네이티브 엔지니어링으로의 여정은 눈을 뜨게 하는 경험이었습니다. Kubernetes, Helm, 현대적인 DevOps 관행들이 제가 데이터베이스 운영에 대해 생각하는 방식을 완전히 바꿨습니다.

클라우드 네이티브 엔지니어링으로 스킬을 확장하고 싶은 DBA라면, 비슷한 것을 구축해보는 것을 강력히 추천합니다. 작게 시작하고, 핵심 기능에 집중하고, 점진적으로 복잡성을 추가하세요. 그리고 필요하다면 적절한 개발 도구에 투자하는 것도 고려해보세요!

당신의 다음 클라우드 네이티브 프로젝트는 무엇이 될까요?

kubernetes #database #dba #cloud-native #nodejs #postgresql #mysql #mariadb #side-project #engineering #devops #cursor-ide #aws-aurora #ha-clustering

이 프로젝트는 클라우드 네이티브 데이터베이스 서비스를 이해하기 위한 학습 연습으로 구축되었습니다. 기여, 포크, 또는 자신의 프로젝트에 영감으로 사용하세요!

Building a Mini DBaaS with Kubernetes in One Week - Part 3: Kubernetes Integration & Helm Charts

프링글리스 — Sun, 20 Jul 2025 08:00:48 +0000

Introduction

Welcome to Part 3! In the previous parts, we set up our environment and created a basic API server. Today, we'll integrate Kubernetes functionality into our Node.js application and create our first Helm charts for database deployment.

By the end of this post, you'll have:

Kubernetes client integration in Node.js
Custom Helm charts for PostgreSQL, MySQL, and MariaDB
Basic database instance deployment functionality
Error handling for Kubernetes operations

What We'll Build Today

We'll create a system that can:

Deploy database instances using Helm charts
Monitor deployment status in real-time
Handle Kubernetes resource management
Provide proper error handling and logging

Step 1: Adding Kubernetes Dependencies

First, let's add the necessary dependencies to our package.json:

{
  "name": "mini-dbaas-backend",
  "version": "1.0.0",
  "description": "Mini DBaaS API Server",
  "main": "index.js",
  "scripts": {
    "start": "node index.js",
    "dev": "nodemon index.js",
    "test": "jest"
  },
  "dependencies": {
    "express": "^4.18.2",
    "cors": "^2.8.5",
    "helmet": "^7.1.0",
    "dotenv": "^16.3.1",
    "winston": "^3.11.0",
    "joi": "^17.11.0",
    "@kubernetes/client-node": "^0.20.0",
    "yaml": "^2.3.4",
    "uuid": "^9.0.1"
  },
  "devDependencies": {
    "nodemon": "^3.0.2",
    "jest": "^29.7.0"
  },
  "keywords": ["kubernetes", "database", "dbaas", "nodejs"],
  "author": "Your Name",
  "license": "MIT"
}

Install the new dependencies:

cd backend
npm install

Step 2: Creating Kubernetes Service

Let's create a service to handle all Kubernetes operations:

const k8s = require('@kubernetes/client-node');
const { exec } = require('child_process');
const { promisify } = require('util');
const logger = require('../utils/logger');
const ResponseUtil = require('../utils/response');

const execAsync = promisify(exec);

class KubernetesService {
  constructor() {
    this.kc = new k8s.KubeConfig();
    this.kc.loadFromDefault();

    this.k8sApi = this.kc.makeApiClient(k8s.CoreV1Api);
    this.appsV1Api = this.kc.makeApiClient(k8s.AppsV1Api);
    this.storageV1Api = this.kc.makeApiClient(k8s.StorageV1Api);
  }

  // Check cluster connectivity
  async checkCluster() {
    try {
      const response = await this.k8sApi.listNamespace();
      logger.info(`Connected to Kubernetes cluster. Found ${response.body.items.length} namespaces`);
      return ResponseUtil.success({ 
        connected: true, 
        namespaces: response.body.items.length 
      });
    } catch (error) {
      logger.error('Failed to connect to Kubernetes cluster', error);
      return ResponseUtil.error('Failed to connect to Kubernetes cluster', 500, error.message);
    }
  }

  // Create namespace
  async createNamespace(name) {
    try {
      const namespace = {
        apiVersion: 'v1',
        kind: 'Namespace',
        metadata: {
          name: name,
          labels: {
            'app': 'mini-dbaas',
            'managed-by': 'mini-dbaas-api'
          }
        }
      };

      await this.k8sApi.createNamespace(namespace);
      logger.info(`Created namespace: ${name}`);
      return ResponseUtil.success({ namespace: name });
    } catch (error) {
      if (error.statusCode === 409) {
        logger.info(`Namespace ${name} already exists`);
        return ResponseUtil.success({ namespace: name, exists: true });
      }
      logger.error(`Failed to create namespace ${name}`, error);
      return ResponseUtil.error(`Failed to create namespace ${name}`, 500, error.message);
    }
  }

  // Delete namespace
  async deleteNamespace(name) {
    try {
      await this.k8sApi.deleteNamespace(name);
      logger.info(`Deleted namespace: ${name}`);
      return ResponseUtil.success({ namespace: name, deleted: true });
    } catch (error) {
      logger.error(`Failed to delete namespace ${name}`, error);
      return ResponseUtil.error(`Failed to delete namespace ${name}`, 500, error.message);
    }
  }

  // Get pod status
  async getPodStatus(namespace, podName) {
    try {
      const response = await this.k8sApi.readNamespacedPod(podName, namespace);
      const pod = response.body;

      return ResponseUtil.success({
        name: pod.metadata.name,
        namespace: pod.metadata.namespace,
        status: pod.status.phase,
        ready: pod.status.containerStatuses?.[0]?.ready || false,
        restartCount: pod.status.containerStatuses?.[0]?.restartCount || 0,
        image: pod.status.containerStatuses?.[0]?.image,
        createdAt: pod.metadata.creationTimestamp
      });
    } catch (error) {
      logger.error(`Failed to get pod status for ${podName} in ${namespace}`, error);
      return ResponseUtil.error(`Failed to get pod status`, 500, error.message);
    }
  }

  // Get all pods in namespace
  async getPodsInNamespace(namespace) {
    try {
      const response = await this.k8sApi.listNamespacedPod(namespace);
      const pods = response.body.items.map(pod => ({
        name: pod.metadata.name,
        status: pod.status.phase,
        ready: pod.status.containerStatuses?.[0]?.ready || false,
        restartCount: pod.status.containerStatuses?.[0]?.restartCount || 0,
        image: pod.status.containerStatuses?.[0]?.image,
        createdAt: pod.metadata.creationTimestamp
      }));

      return ResponseUtil.success({ pods, count: pods.length });
    } catch (error) {
      logger.error(`Failed to get pods in namespace ${namespace}`, error);
      return ResponseUtil.error(`Failed to get pods`, 500, error.message);
    }
  }

  // Get PVC status
  async getPVCStatus(namespace, pvcName) {
    try {
      const response = await this.k8sApi.readNamespacedPersistentVolumeClaim(pvcName, namespace);
      const pvc = response.body;

      return ResponseUtil.success({
        name: pvc.metadata.name,
        namespace: pvc.metadata.namespace,
        status: pvc.status.phase,
        capacity: pvc.status.capacity?.storage,
        accessModes: pvc.status.accessModes,
        createdAt: pvc.metadata.creationTimestamp
      });
    } catch (error) {
      logger.error(`Failed to get PVC status for ${pvcName} in ${namespace}`, error);
      return ResponseUtil.error(`Failed to get PVC status`, 500, error.message);
    }
  }

  // Execute kubectl command
  async executeKubectl(command) {
    try {
      const { stdout, stderr } = await execAsync(`kubectl ${command}`);
      if (stderr) {
        logger.warn(`kubectl stderr: ${stderr}`);
      }
      return ResponseUtil.success({ output: stdout, command });
    } catch (error) {
      logger.error(`kubectl command failed: ${command}`, error);
      return ResponseUtil.error(`kubectl command failed`, 500, error.message);
    }
  }

  // Execute helm command
  async executeHelm(command) {
    try {
      const { stdout, stderr } = await execAsync(`helm ${command}`);
      if (stderr) {
        logger.warn(`helm stderr: ${stderr}`);
      }
      return ResponseUtil.success({ output: stdout, command });
    } catch (error) {
      logger.error(`helm command failed: ${command}`, error);
      return ResponseUtil.error(`helm command failed`, 500, error.message);
    }
  }
}

module.exports = new KubernetesService();

Step 3: Creating Helm Chart Service

Now let's create a service to manage Helm chart operations:

const { exec } = require('child_process');
const { promisify } = require('util');
const fs = require('fs').promises;
const path = require('path');
const logger = require('../utils/logger');
const ResponseUtil = require('../utils/response');
const k8sService = require('./k8s');

const execAsync = promisify(exec);

class HelmService {
  constructor() {
    this.chartsPath = path.join(__dirname, '../../helm-charts');
  }

  // Deploy database using Helm
  async deployDatabase(namespace, instanceName, dbType, config) {
    try {
      // Create namespace if it doesn't exist
      await k8sService.createNamespace(namespace);

      // Prepare Helm values
      const values = this.prepareHelmValues(dbType, config);
      const valuesFile = path.join(this.chartsPath, `${dbType}-local`, 'values.yaml`);

      // Write values to file
      await fs.writeFile(valuesFile, JSON.stringify(values, null, 2));

      // Deploy using Helm
      const helmCommand = `install ${instanceName} ${this.chartsPath}/${dbType}-local --namespace ${namespace} --values ${valuesFile}`;
      const result = await k8sService.executeHelm(helmCommand);

      if (result.success) {
        logger.info(`Successfully deployed ${dbType} instance: ${instanceName} in namespace: ${namespace}`);
        return ResponseUtil.success({
          instanceName,
          namespace,
          dbType,
          status: 'deploying',
          helmOutput: result.data.output
        });
      } else {
        throw new Error(result.message);
      }
    } catch (error) {
      logger.error(`Failed to deploy ${dbType} instance: ${instanceName}`, error);
      return ResponseUtil.error(`Failed to deploy database instance`, 500, error.message);
    }
  }

  // Delete database using Helm
  async deleteDatabase(namespace, instanceName) {
    try {
      const helmCommand = `uninstall ${instanceName} --namespace ${namespace}`;
      const result = await k8sService.executeHelm(helmCommand);

      if (result.success) {
        logger.info(`Successfully deleted instance: ${instanceName} from namespace: ${namespace}`);
        return ResponseUtil.success({
          instanceName,
          namespace,
          status: 'deleted',
          helmOutput: result.data.output
        });
      } else {
        throw new Error(result.message);
      }
    } catch (error) {
      logger.error(`Failed to delete instance: ${instanceName}`, error);
      return ResponseUtil.error(`Failed to delete database instance`, 500, error.message);
    }
  }

  // Get Helm release status
  async getReleaseStatus(namespace, releaseName) {
    try {
      const helmCommand = `status ${releaseName} --namespace ${namespace} --output json`;
      const result = await k8sService.executeHelm(helmCommand);

      if (result.success) {
        const status = JSON.parse(result.data.output);
        return ResponseUtil.success({
          name: releaseName,
          namespace,
          status: status.info?.status,
          revision: status.version,
          lastDeployed: status.info?.last_deployed,
          description: status.info?.description
        });
      } else {
        throw new Error(result.message);
      }
    } catch (error) {
      logger.error(`Failed to get release status for: ${releaseName}`, error);
      return ResponseUtil.error(`Failed to get release status`, 500, error.message);
    }
  }

  // List all Helm releases
  async listReleases(namespace = null) {
    try {
      const helmCommand = namespace 
        ? `list --namespace ${namespace} --output json`
        : `list --all-namespaces --output json`;

      const result = await k8sService.executeHelm(helmCommand);

      if (result.success) {
        const releases = JSON.parse(result.data.output);
        return ResponseUtil.success({
          releases: releases.map(release => ({
            name: release.name,
            namespace: release.namespace,
            status: release.status,
            revision: release.revision,
            lastDeployed: release.updated
          })),
          count: releases.length
        });
      } else {
        throw new Error(result.message);
      }
    } catch (error) {
      logger.error('Failed to list Helm releases', error);
      return ResponseUtil.error('Failed to list Helm releases', 500, error.message);
    }
  }

  // Prepare Helm values based on database type and config
  prepareHelmValues(dbType, config) {
    const baseValues = {
      global: {
        storageClass: "standard"
      },
      persistence: {
        enabled: true,
        size: config.storage || "1Gi"
      },
      resources: {
        requests: {
          memory: config.memory || "256Mi",
          cpu: config.cpu || "250m"
        },
        limits: {
          memory: config.memoryLimit || "512Mi",
          cpu: config.cpuLimit || "500m"
        }
      }
    };

    switch (dbType) {
      case 'postgresql':
        return {
          ...baseValues,
          auth: {
            postgresPassword: config.password,
            database: config.database || "postgres"
          },
          primary: {
            persistence: {
              enabled: true,
              size: config.storage || "1Gi"
            }
          }
        };

      case 'mysql':
        return {
          ...baseValues,
          auth: {
            rootPassword: config.password,
            database: config.database || "mysql"
          },
          primary: {
            persistence: {
              enabled: true,
              size: config.storage || "1Gi"
            }
          }
        };

      case 'mariadb':
        return {
          ...baseValues,
          auth: {
            rootPassword: config.password,
            database: config.database || "mariadb"
          },
          primary: {
            persistence: {
              enabled: true,
              size: config.storage || "1Gi"
            }
          }
        };

      default:
        throw new Error(`Unsupported database type: ${dbType}`);
    }
  }
}

module.exports = new HelmService();

Step 4: Creating Database Instance Controller

Let's create a controller to handle database instance operations:

const helmService = require('../services/helm');
const k8sService = require('../services/k8s');
const logger = require('../utils/logger');
const ResponseUtil = require('../utils/response');

class InstanceController {
  // Create a new database instance
  async createInstance(req, res) {
    try {
      const { type, name, config } = req.body;
      const namespace = `dbaas-${name}`;

      logger.info(`Creating ${type} instance: ${name} in namespace: ${namespace}`);

      // Validate required fields
      if (!type || !name || !config) {
        return res.status(400).json(
          ResponseUtil.error('Missing required fields: type, name, config', 400)
        );
      }

      // Deploy database using Helm
      const result = await helmService.deployDatabase(namespace, name, type, config);

      if (result.success) {
        res.status(201).json(result);
      } else {
        res.status(500).json(result);
      }
    } catch (error) {
      logger.error('Error creating instance', error);
      res.status(500).json(
        ResponseUtil.error('Failed to create instance', 500, error.message)
      );
    }
  }

  // Get all instances
  async getInstances(req, res) {
    try {
      logger.info('Fetching all instances');

      const result = await helmService.listReleases();

      if (result.success) {
        // Filter only our DBaaS instances
        const dbaasInstances = result.data.releases.filter(release => 
          release.namespace.startsWith('dbaas-')
        );

        res.json(ResponseUtil.success({
          instances: dbaasInstances,
          count: dbaasInstances.length
        }));
      } else {
        res.status(500).json(result);
      }
    } catch (error) {
      logger.error('Error fetching instances', error);
      res.status(500).json(
        ResponseUtil.error('Failed to fetch instances', 500, error.message)
      );
    }
  }

  // Get specific instance
  async getInstance(req, res) {
    try {
      const { name } = req.params;
      const namespace = `dbaas-${name}`;

      logger.info(`Fetching instance: ${name}`);

      // Get Helm release status
      const releaseResult = await helmService.getReleaseStatus(namespace, name);

      if (!releaseResult.success) {
        return res.status(404).json(
          ResponseUtil.error(`Instance ${name} not found`, 404)
        );
      }

      // Get pod status
      const podsResult = await k8sService.getPodsInNamespace(namespace);

      res.json(ResponseUtil.success({
        ...releaseResult.data,
        pods: podsResult.success ? podsResult.data.pods : []
      }));
    } catch (error) {
      logger.error(`Error fetching instance: ${req.params.name}`, error);
      res.status(500).json(
        ResponseUtil.error('Failed to fetch instance', 500, error.message)
      );
    }
  }

  // Delete instance
  async deleteInstance(req, res) {
    try {
      const { name } = req.params;
      const namespace = `dbaas-${name}`;

      logger.info(`Deleting instance: ${name}`);

      // Delete Helm release
      const result = await helmService.deleteDatabase(namespace, name);

      if (result.success) {
        // Delete namespace after a delay to ensure cleanup
        setTimeout(async () => {
          await k8sService.deleteNamespace(namespace);
        }, 5000);

        res.json(result);
      } else {
        res.status(500).json(result);
      }
    } catch (error) {
      logger.error(`Error deleting instance: ${req.params.name}`, error);
      res.status(500).json(
        ResponseUtil.error('Failed to delete instance', 500, error.message)
      );
    }
  }

  // Get instance connection info
  async getConnectionInfo(req, res) {
    try {
      const { name } = req.params;
      const namespace = `dbaas-${name}`;

      logger.info(`Fetching connection info for instance: ${name}`);

      // Get service info
      const serviceResult = await k8sService.executeKubectl(
        `get svc -n ${namespace} -o json`
      );

      if (!serviceResult.success) {
        return res.status(404).json(
          ResponseUtil.error(`Instance ${name} not found`, 404)
        );
      }

      const services = JSON.parse(serviceResult.data.output);
      const dbService = services.items.find(svc => 
        svc.metadata.name.includes(name) && svc.spec.ports
      );

      if (!dbService) {
        return res.status(404).json(
          ResponseUtil.error(`Service not found for instance ${name}`, 404)
        );
      }

      const connectionInfo = {
        host: `${dbService.metadata.name}.${namespace}.svc.cluster.local`,
        port: dbService.spec.ports[0].port,
        service: dbService.metadata.name,
        namespace: namespace
      };

      res.json(ResponseUtil.success(connectionInfo));
    } catch (error) {
      logger.error(`Error fetching connection info for: ${req.params.name}`, error);
      res.status(500).json(
        ResponseUtil.error('Failed to fetch connection info', 500, error.message)
      );
    }
  }
}

module.exports = new InstanceController();

Step 5: Creating Routes

Now let's create the routes for our API:

const express = require('express');
const router = express.Router();
const InstanceController = require('../controllers/InstanceController');
const { validate, schemas } = require('../middleware/validation');

// Create new instance
router.post('/', validate(schemas.instance), InstanceController.createInstance);

// Get all instances
router.get('/', InstanceController.getInstances);

// Get specific instance
router.get('/:name', InstanceController.getInstance);

// Get instance connection info
router.get('/:name/connection', InstanceController.getConnectionInfo);

// Delete instance
router.delete('/:name', InstanceController.deleteInstance);

module.exports = router;

Step 6: Updating Main Server File

Let's update our main server file to include the new routes:

const express = require('express');
const cors = require('cors');
const helmet = require('helmet');
require('dotenv').config();

const app = express();
const PORT = process.env.PORT || 3000;

// Import routes
const instancesRouter = require('./routes/instances');

// Middleware
app.use(helmet());
app.use(cors());
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// Basic logging middleware
app.use((req, res, next) => {
  console.log(`${new Date().toISOString()} - ${req.method} ${req.path}`);
  next();
});

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({
    status: 'healthy',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    environment: process.env.NODE_ENV
  });
});

// API information endpoint
app.get('/', (req, res) => {
  res.json({
    name: 'Mini DBaaS API',
    version: '1.0.0',
    description: 'Database as a Service API built with Node.js and Kubernetes',
    endpoints: {
      health: '/health',
      instances: '/instances',
      'ha-clusters': '/ha-clusters'
    }
  });
});

// Routes
app.use('/instances', instancesRouter);

// Error handling middleware
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(500).json({
    error: 'Something went wrong!',
    message: process.env.NODE_ENV === 'development' ? err.message : 'Internal server error'
  });
});

// 404 handler
app.use('*', (req, res) => {
  res.status(404).json({
    error: 'Endpoint not found',
    path: req.originalUrl
  });
});

// Start server
app.listen(PORT, () => {
  console.log(`🚀 Mini DBaaS API server running on port ${PORT}`);
  console.log(`📊 Health check: http://localhost:${PORT}/health`);
  console.log(`📚 API docs: http://localhost:${PORT}/`);
  console.log(`🗄️  Instances API: http://localhost:${PORT}/instances`);
});

module.exports = app;

Step 7: Creating Basic Helm Charts

Let's create basic Helm charts for our databases. First, let's create the PostgreSQL chart:

apiVersion: v2
name: postgresql-local
description: A Helm chart for PostgreSQL database instances
type: application
version: 0.1.0
appVersion: "15.0"

# Default values for postgresql-local
global:
  storageClass: "standard"

auth:
  postgresPassword: "postgres"
  database: "postgres"

primary:
  persistence:
    enabled: true
    size: "1Gi"
  resources:
    requests:
      memory: "256Mi"
      cpu: "250m"
    limits:
      memory: "512Mi"
      cpu: "500m"

service:
  type: ClusterIP
  port: 5432

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: {{ include "postgresql-local.fullname" . }}
  labels:
    {{- include "postgresql-local.labels" . | nindent 4 }}
spec:
  serviceName: {{ include "postgresql-local.fullname" . }}
  replicas: 1
  selector:
    matchLabels:
      {{- include "postgresql-local.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "postgresql-local.selectorLabels" . | nindent 8 }}
    spec:
      containers:
      - name: postgresql
        image: postgres:15
        ports:
        - containerPort: 5432
          name: postgresql
        env:
        - name: POSTGRES_PASSWORD
          value: {{ .Values.auth.postgresPassword }}
        - name: POSTGRES_DB
          value: {{ .Values.auth.database }}
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
        resources:
          {{- toYaml .Values.primary.resources | nindent 10 }}
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: {{ .Values.global.storageClass }}
      resources:
        requests:
          storage: {{ .Values.primary.persistence.size }}

apiVersion: v1
kind: Service
metadata:
  name: {{ include "postgresql-local.fullname" . }}
  labels:
    {{- include "postgresql-local.labels" . | nindent 4 }}
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: postgresql
      protocol: TCP
      name: postgresql
  selector:
    {{- include "postgresql-local.selectorLabels" . | nindent 4 }}

{{/*
Expand the name of the chart.
*/}}
{{- define "postgresql-local.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create a default fully qualified app name.
*/}}
{{- define "postgresql-local.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "postgresql-local.labels" -}}
helm.sh/chart: {{ include "postgresql-local.chart" . }}
{{ include "postgresql-local.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "postgresql-local.selectorLabels" -}}
app.kubernetes.io/name: {{ include "postgresql-local.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "postgresql-local.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

Step 8: Testing Our Kubernetes Integration

Let's create a test script to verify our setup:

#!/bin/bash

echo "🧪 Testing Kubernetes Integration"

# Test 1: Check cluster connectivity
echo "🔗 Testing cluster connectivity..."
curl -s http://localhost:3000/health | grep -q "healthy" && echo "✅ Health check passed" || echo "❌ Health check failed"

# Test 2: Create PostgreSQL instance
echo "🗄️  Creating PostgreSQL instance..."
CREATE_RESPONSE=$(curl -s -X POST http://localhost:3000/instances \
  -H "Content-Type: application/json" \
  -d '{
    "type": "postgresql",
    "name": "test-postgres",
    "config": {
      "password": "testpass123",
      "storage": "1Gi",
      "database": "testdb"
    }
  }')

echo "Create response: $CREATE_RESPONSE"

# Test 3: List instances
echo "📋 Listing instances..."
LIST_RESPONSE=$(curl -s http://localhost:3000/instances)
echo "List response: $LIST_RESPONSE"

# Test 4: Get instance status
echo "📊 Getting instance status..."
sleep 10
STATUS_RESPONSE=$(curl -s http://localhost:3000/instances/test-postgres)
echo "Status response: $STATUS_RESPONSE"

# Test 5: Get connection info
echo "🔌 Getting connection info..."
CONNECTION_RESPONSE=$(curl -s http://localhost:3000/instances/test-postgres/connection)
echo "Connection response: $CONNECTION_RESPONSE"

echo "🎉 Kubernetes integration test completed!"

Make it executable and run:

chmod +x scripts/test-k8s-integration.sh
./scripts/test-k8s-integration.sh

What We've Accomplished

Today we've successfully:

✅ Integrated Kubernetes client into our Node.js application
✅ Created Helm service for database deployment management
✅ Built instance controller with full CRUD operations
✅ Implemented proper error handling for Kubernetes operations
✅ Created basic Helm charts for PostgreSQL deployment
✅ Added comprehensive logging for debugging
✅ Built RESTful API endpoints for instance management
✅ Implemented validation middleware for API requests

Testing the API

Now you can test your API with these commands:

# Create a PostgreSQL instance
curl -X POST http://localhost:3000/instances \
  -H "Content-Type: application/json" \
  -d '{
    "type": "postgresql",
    "name": "my-postgres",
    "config": {
      "password": "securepass123",
      "storage": "2Gi",
      "database": "myapp"
    }
  }'

# List all instances
curl http://localhost:3000/instances

# Get specific instance status
curl http://localhost:3000/instances/my-postgres

# Get connection information
curl http://localhost:3000/instances/my-postgres/connection

# Delete instance
curl -X DELETE http://localhost:3000/instances/my-postgres

Next Steps

In Part 4, we'll enhance our database instance management with:

MySQL and MariaDB Helm charts
Advanced monitoring and status tracking
Connection pooling and optimization
Backup and recovery preparation

Troubleshooting

Common Issues

1. Kubernetes client connection issues

# Check if kubectl is working
kubectl get nodes

# Verify kubeconfig
kubectl config view

2. Helm chart deployment failures

# Check Helm chart syntax
helm lint helm-charts/postgresql-local

# Test chart installation
helm install test-postgres helm-charts/postgresql-local --dry-run

3. Pod startup issues

# Check pod logs
kubectl logs -f <pod-name> -n <namespace>

# Check pod events
kubectl describe pod <pod-name> -n <namespace>

Summary

We now have a fully functional Kubernetes-integrated API server that can deploy and manage database instances! Our system can:

Deploy PostgreSQL instances using custom Helm charts
Monitor deployment status in real-time
Provide connection information for applications
Handle proper cleanup and resource management

In the next part, we'll add MySQL and MariaDB support, and implement more advanced features. Get ready to see your first database running in Kubernetes! 🚀

Series Navigation:

Part 1: Architecture Overview
Part 2: Environment Setup & Basic API Server
Part 3: Kubernetes Integration & Helm Charts (this post)
Part 4: Database Instance Management
Part 5: Backup & Recovery with CSI VolumeSnapshots
Part 6: High Availability with PostgreSQL Operator
Part 7: Multi-Tenant Features & Final Testing

kubernetes #helm #nodejs #postgresql #api #deployment #tutorial #series #docker #minikube

Building a Mini DBaaS with Kubernetes in One Week - Part 2: Environment Setup & Basic API Server

프링글리스 — Sun, 20 Jul 2025 07:54:28 +0000

Introduction

Welcome back! In Part 1, we discussed the architecture and planning for our mini DBaaS platform. Today, we'll get our hands dirty and set up the development environment, then create the foundation of our Node.js API server.

By the end of this post, you'll have:

A working Kubernetes cluster with minikube
A basic Node.js API server with Express
Initial project structure following best practices
Basic health check and routing setup

Prerequisites Check

Before we start, let's make sure you have all the required tools installed:

# Check Docker
docker --version

# Check Node.js (v18+)
node --version

# Check kubectl
kubectl version --client

# Check Helm
helm version

# Check minikube
minikube version

If any of these are missing, install them first:

Docker Desktop: Download here
Node.js: Download here
kubectl: Installation guide
Helm: Installation guide
minikube: Installation guide

Step 1: Setting Up Kubernetes Environment

Let's start by setting up our local Kubernetes cluster:

# Start minikube with adequate resources
minikube start --cpus=4 --memory=8192 --disk-size=20g

# Enable necessary addons
minikube addons enable csi-hostpath-driver
minikube addons enable volumesnapshots

# Verify cluster is running
kubectl cluster-info
kubectl get nodes

Verify CSI and VolumeSnapshot Support

# Check CSI drivers
kubectl get csidriver

# Check VolumeSnapshot classes
kubectl get volumesnapshotclass

# You should see:
# NAME                    DRIVER                DELETIONPOLICY   AGE
# csi-hostpath-snapclass  hostpath.csi.k8s.io   Delete           1m

Step 2: Setting Up Helm Repositories

We'll use Bitnami charts for our databases:

# Add Bitnami repository
helm repo add bitnami https://charts.bitnami.com/bitnami

# Add Zalando PostgreSQL Operator repository
helm repo add zalando https://opensource.zalando.com/postgres-operator/charts/postgres-operator

# Update repositories
helm repo update

# Verify repositories
helm repo list

Step 3: Creating Project Structure

Let's create a well-organized project structure:

# Create project directory
mkdir mini-dbaas && cd mini-dbaas

# Create backend structure
mkdir -p backend/{controllers,routes,services,middleware,utils}
mkdir -p helm-charts/{postgresql-local,mysql-local,mariadb-local}
mkdir -p k8s/operators
mkdir -p scripts

# Create initial files
touch backend/package.json
touch backend/index.js
touch backend/.env.example
touch backend/.env
touch README.md

Your project structure should look like this:

mini-dbaas/
├── backend/
│   ├── controllers/
│   ├── routes/
│   ├── services/
│   ├── middleware/
│   ├── utils/
│   ├── package.json
│   ├── index.js
│   ├── .env.example
│   └── .env
├── helm-charts/
│   ├── postgresql-local/
│   ├── mysql-local/
│   └── mariadb-local/
├── k8s/
│   └── operators/
├── scripts/
└── README.md

Step 4: Setting Up Node.js Backend

Let's create our Node.js API server:

Package.json Setup

{
  "name": "mini-dbaas-backend",
  "version": "1.0.0",
  "description": "Mini DBaaS API Server",
  "main": "index.js",
  "scripts": {
    "start": "node index.js",
    "dev": "nodemon index.js",
    "test": "jest"
  },
  "dependencies": {
    "express": "^4.18.2",
    "cors": "^2.8.5",
    "helmet": "^7.1.0",
    "dotenv": "^16.3.1",
    "winston": "^3.11.0",
    "joi": "^17.11.0"
  },
  "devDependencies": {
    "nodemon": "^3.0.2",
    "jest": "^29.7.0"
  },
  "keywords": ["kubernetes", "database", "dbaas", "nodejs"],
  "author": "Your Name",
  "license": "MIT"
}

Environment Configuration

# Server Configuration
PORT=3000
NODE_ENV=development

# Kubernetes Configuration
KUBECONFIG_PATH=~/.kube/config

# Database Configuration
METADATA_DB_PATH=./data/metadata.db

# Logging
LOG_LEVEL=info

# Security
JWT_SECRET=your-secret-key-here

Main Server File

const express = require('express');
const cors = require('cors');
const helmet = require('helmet');
require('dotenv').config();

const app = express();
const PORT = process.env.PORT || 3000;

// Middleware
app.use(helmet());
app.use(cors());
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// Basic logging middleware
app.use((req, res, next) => {
  console.log(`${new Date().toISOString()} - ${req.method} ${req.path}`);
  next();
});

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({
    status: 'healthy',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    environment: process.env.NODE_ENV
  });
});

// API information endpoint
app.get('/', (req, res) => {
  res.json({
    name: 'Mini DBaaS API',
    version: '1.0.0',
    description: 'Database as a Service API built with Node.js and Kubernetes',
    endpoints: {
      health: '/health',
      instances: '/instances',
      'ha-clusters': '/ha-clusters'
    }
  });
});

// Error handling middleware
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(500).json({
    error: 'Something went wrong!',
    message: process.env.NODE_ENV === 'development' ? err.message : 'Internal server error'
  });
});

// 404 handler
app.use('*', (req, res) => {
  res.status(404).json({
    error: 'Endpoint not found',
    path: req.originalUrl
  });
});

// Start server
app.listen(PORT, () => {
  console.log(`🚀 Mini DBaaS API server running on port ${PORT}`);
  console.log(`📊 Health check: http://localhost:${PORT}/health`);
  console.log(`📚 API docs: http://localhost:${PORT}/`);
});

module.exports = app;

Step 5: Installing Dependencies

cd backend
npm install

Step 6: Testing Our Basic Setup

Let's test our basic API server:

# Start the server
npm start

# In another terminal, test the endpoints
curl http://localhost:3000/health
curl http://localhost:3000/

You should see responses like:

// GET /health
{
  "status": "healthy",
  "timestamp": "2025-01-27T10:30:00.000Z",
  "uptime": 5.123,
  "environment": "development"
}

// GET /
{
  "name": "Mini DBaaS API",
  "version": "1.0.0",
  "description": "Database as a Service API built with Node.js and Kubernetes",
  "endpoints": {
    "health": "/health",
    "instances": "/instances",
    "ha-clusters": "/ha-clusters"
  }
}

Step 7: Creating Utility Functions

Let's create some utility functions that we'll use throughout the project:

class ResponseUtil {
  static success(data = null, message = 'Success') {
    return {
      success: true,
      message,
      data,
      timestamp: new Date().toISOString()
    };
  }

  static error(message = 'Error occurred', statusCode = 500, details = null) {
    return {
      success: false,
      message,
      statusCode,
      details,
      timestamp: new Date().toISOString()
    };
  }

  static paginated(data, page, limit, total) {
    return {
      success: true,
      data,
      pagination: {
        page: parseInt(page),
        limit: parseInt(limit),
        total,
        pages: Math.ceil(total / limit)
      },
      timestamp: new Date().toISOString()
    };
  }
}

module.exports = ResponseUtil;

const winston = require('winston');

const logger = winston.createLogger({
  level: process.env.LOG_LEVEL || 'info',
  format: winston.format.combine(
    winston.format.timestamp(),
    winston.format.errors({ stack: true }),
    winston.format.json()
  ),
  defaultMeta: { service: 'mini-dbaas-api' },
  transports: [
    new winston.transports.File({ filename: 'logs/error.log', level: 'error' }),
    new winston.transports.File({ filename: 'logs/combined.log' })
  ]
});

if (process.env.NODE_ENV !== 'production') {
  logger.add(new winston.transports.Console({
    format: winston.format.simple()
  }));
}

module.exports = logger;

Step 8: Creating Basic Middleware

const Joi = require('joi');

const validate = (schema) => {
  return (req, res, next) => {
    const { error } = schema.validate(req.body);
    if (error) {
      return res.status(400).json({
        success: false,
        message: 'Validation error',
        details: error.details.map(detail => detail.message)
      });
    }
    next();
  };
};

// Common validation schemas
const schemas = {
  instance: Joi.object({
    type: Joi.string().valid('postgresql', 'mysql', 'mariadb').required(),
    name: Joi.string().alphanum().min(3).max(50).required(),
    config: Joi.object({
      password: Joi.string().min(8).required(),
      storage: Joi.string().pattern(/^\d+[KMGTPEZYkmgtpezy]i?$/).required(),
      database: Joi.string().alphanum().optional(),
      memory: Joi.string().pattern(/^\d+[KMGTPEZYkmgtpezy]i?$/).optional(),
      cpu: Joi.string().pattern(/^\d+m?$/).optional()
    }).required()
  })
};

module.exports = { validate, schemas };

Step 9: Testing Everything Together

Let's create a simple test script:

#!/bin/bash

echo "🧪 Testing Mini DBaaS Basic Setup"

# Test 1: Check if server starts
echo "📡 Testing server startup..."
cd backend
npm start &
SERVER_PID=$!
sleep 3

# Test 2: Health check
echo "💓 Testing health endpoint..."
HEALTH_RESPONSE=$(curl -s http://localhost:3000/health)
if echo "$HEALTH_RESPONSE" | grep -q "healthy"; then
    echo "✅ Health check passed"
else
    echo "❌ Health check failed"
    echo "Response: $HEALTH_RESPONSE"
fi

# Test 3: API info
echo "📚 Testing API info endpoint..."
API_RESPONSE=$(curl -s http://localhost:3000/)
if echo "$API_RESPONSE" | grep -q "Mini DBaaS API"; then
    echo "✅ API info endpoint passed"
else
    echo "❌ API info endpoint failed"
    echo "Response: $API_RESPONSE"
fi

# Cleanup
kill $SERVER_PID
echo "🎉 Basic setup test completed!"

Make it executable and run:

chmod +x scripts/test-basic.sh
./scripts/test-basic.sh

What We've Accomplished

Today we've successfully:

✅ Set up Kubernetes environment with minikube and necessary addons
✅ Configured Helm repositories for database charts
✅ Created project structure following best practices
✅ Built basic Node.js API server with Express
✅ Implemented health checks and basic routing
✅ Added utility functions for consistent responses
✅ Created validation middleware for API requests
✅ Set up logging with Winston
✅ Tested the basic setup with automated scripts

Next Steps

In Part 3, we'll integrate Kubernetes functionality into our API server and create our first Helm charts for database deployment. We'll learn about:

Kubernetes client integration
Helm chart creation and deployment
Basic database instance management
Error handling for Kubernetes operations

Troubleshooting

Common Issues

1. minikube won't start

# Check Docker is running
docker ps

# Reset minikube if needed
minikube delete
minikube start --cpus=4 --memory=8192

2. Port 3000 already in use

# Find and kill the process
lsof -ti:3000 | xargs kill -9

3. Helm repository issues

# Clear and re-add repositories
helm repo remove bitnami
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

Summary

We now have a solid foundation for our mini DBaaS platform! Our API server is running, our Kubernetes environment is ready, and we have a clean project structure to build upon.

In the next part, we'll dive into Kubernetes integration and start deploying actual database instances. Get ready to see your first PostgreSQL instance running in Kubernetes! 🚀

Series Navigation:

Part 1: Architecture Overview
Part 2: Environment Setup & Basic API Server (this post)
Part 3: Kubernetes Integration & Helm Charts
Part 4: Database Instance Management
Part 5: Backup & Recovery with CSI VolumeSnapshots
Part 6: High Availability with PostgreSQL Operator
Part 7: Multi-Tenant Features & Final Testing

kubernetes #nodejs #express #api #development #tutorial #series #docker #minikube #helm

Building a Mini DBaaS with Kubernetes in One Week - Part 1: Architecture Overview

프링글리스 — Sun, 20 Jul 2025 07:48:54 +0000

Building a Mini DBaaS with Kubernetes in One Week - Part 1: Architecture Overview

Introduction

Ever wondered how cloud database services like AWS RDS or Google Cloud SQL work under the hood? What if you could build your own Database-as-a-Service (DBaaS) platform using Kubernetes? In this series, I'll show you how to create a fully functional mini DBaaS platform in just one week using Node.js and Kubernetes.

By the end of this series, you'll have a working DBaaS that can:

Create and manage PostgreSQL, MySQL, and MariaDB instances
Provide high-availability PostgreSQL clusters with automatic failover
Offer Aurora-style backup and recovery using CSI VolumeSnapshots
Support multi-tenant isolation with namespaces
Monitor database health and performance

Why Build Your Own DBaaS?

Building a DBaaS might seem like overkill for small projects, but it's an excellent way to:

Learn Kubernetes deeply: Understand StatefulSets, PVCs, Operators, and more
Master cloud-native patterns: Experience real-world distributed systems
Gain DevOps skills: Practice infrastructure as code with Helm charts
Understand database operations: Learn about backup strategies, high availability, and monitoring

Our Architecture Overview

Our mini DBaaS follows a clean, layered architecture:

┌─────────────────────────────────────┐
│         User Requests (CLI/API)     │
└──────────────┬──────────────────────┘
               ↓
┌─────────────────────────────────────┐
│      Node.js API Server (Express)   │
│  • Instance CRUD operations         │
│  • Helm chart deployment            │
│  • Kubernetes resource management   │
└──────────────┬──────────────────────┘
               ↓
┌─────────────────────────────────────┐
│        Kubernetes (minikube)        │
│  ┌────────────────────────────────┐ │
│  │  Namespace per instance        │ │
│  │  StatefulSet + PVC (DB Pod)    │ │
│  │  Secret, ConfigMap             │ │
│  └────────────────────────────────┘ │
└──────────────┬──────────────────────┘
               ↓
┌─────────────────────────────────────┐
│    Local Storage (PVC, HostPath)    │
│  • MySQL/MariaDB/PostgreSQL data    │ │
└──────────────┬──────────────────────┘
               ↓
┌─────────────────────────────────────┐
│    CSI VolumeSnapshot Backup        │
│  • Aurora-style point-in-time       │ │
│  • 5-10 second snapshot creation    │ │
│  • 30-second recovery time          │ │
└──────────────┬──────────────────────┘
               ↓
┌─────────────────────────────────────┐
│  PostgreSQL HA (Zalando Operator)   │
│  • Automatic failover               │ │
│  • Read/write load balancing        │ │
│  • 3-5 node clusters                │ │
└─────────────────────────────────────┘

Technology Stack

We'll use a modern, cloud-native stack:

Backend: Node.js with Express (lightweight and fast)
Orchestration: Kubernetes with Helm (industry standard)
Databases: PostgreSQL, MySQL, MariaDB via Bitnami charts
High Availability: Zalando PostgreSQL Operator
Backup/Recovery: CSI VolumeSnapshots (like AWS Aurora)
Storage: PVC with hostPath (minikube) / cloud storage (production)
Monitoring: Real-time pod/helm status tracking

Key Features We'll Implement

1. Multi-Database Support

PostgreSQL, MySQL, and MariaDB instances
Custom Helm charts for each database type
Automatic configuration and secret management

2. High Availability Clusters

PostgreSQL HA using Zalando PostgreSQL Operator
Automatic failover and load balancing
Master/Replica service separation

3. Aurora-Style Backup System

CSI VolumeSnapshot for storage-level backups
Point-in-time recovery
Cross-instance backup restoration

4. Multi-Tenant Isolation

Namespace-based resource isolation
Independent storage volumes per tenant
Resource quotas and limits

5. RESTful API

Complete CRUD operations for instances
Real-time status monitoring
Connection information retrieval

Development Timeline

Here's our one-week development plan:

Day	Focus	Deliverables
Day 1	Environment Setup & Basic API	Node.js server, basic routes
Day 2	Kubernetes Integration	Helm charts, basic deployment
Day 3	Database Instance Management	Create/delete/status APIs
Day 4	Backup & Recovery System	CSI VolumeSnapshots
Day 5	High Availability Clusters	Zalando PostgreSQL Operator
Day 6	Multi-Tenant Features	Namespace isolation, quotas
Day 7	Testing & Documentation	API testing, deployment guide

What You'll Learn

Throughout this series, you'll gain hands-on experience with:

Kubernetes StatefulSets: Managing stateful applications
Persistent Volume Claims: Database storage management
Helm Charts: Package and deploy applications
Kubernetes Operators: Advanced application management
CSI VolumeSnapshots: Storage-level backup strategies
Multi-tenancy: Resource isolation and management
API Design: RESTful service architecture

Prerequisites

Before we start, make sure you have:

Docker Desktop installed and running
minikube for local Kubernetes cluster
Helm for package management
Node.js (v18+) for the API server
kubectl for Kubernetes interaction

Next Steps

In the next post, we'll set up our development environment and create the basic Node.js API server structure. We'll start with the foundation and gradually build up to a fully functional DBaaS platform.

Are you ready to build your own cloud database service? Let's dive in! 🚀

Series Navigation:

Part 1: Architecture Overview (this post)
Part 2: Environment Setup & Basic API Server
Part 3: Kubernetes Integration & Helm Charts
Part 4: Database Instance Management
Part 5: Backup & Recovery with CSI VolumeSnapshots
Part 6: High Availability with PostgreSQL Operator
Part 7: Multi-Tenant Features & Final Testing

Follow along and build your own mini DBaaS platform!

kubernetes #database #devops #nodejs #helm #postgresql #mysql #mariadb #cloud-native #tutorial #series

DEV Community: 프링글리스

Building a Mini DBaaS with Kubernetes: A DBA's Cloud-Native Engineering Journey

Why Did I Build This?

🎯 Development Motivation and Goals

💡 Development Tool Investment

The Challenges

Major Problems I Encountered

1. Complexity of Kubernetes StatefulSets

2. Multi-Database Support and High Availability Implementation

3. AWS Aurora-style Backup and Recovery System

4. High Availability Clustering (PostgreSQL vs MySQL)

What I Built

✅ Completed Features

🚧 Current Limitations

📊 Performance Metrics

Technical Architecture

Technology Stack

Key Learnings

1. Advanced Kubernetes Learning

2. Database Operations in Kubernetes

3. Cloud-Native Patterns

4. Importance of Development Tools

Results

Next Steps

Phase 1 (1-2 weeks)

Phase 2 (3-4 weeks)

Phase 3 (5-8 weeks)

Why This Matters

Check It Out

Quick Start

Conclusion

Tags

kubernetes #database #dba #cloud-native #nodejs #postgresql #mysql #mariadb #side-project #engineering #devops #cursor-ide #aws-aurora #ha-clustering

Kubernetes로 Mini DBaaS 구축하기: DBA의 클라우드 네이티브 엔지니어링 도전기

왜 만들게 되었나요?

💡 개발 동기와 목표

💡 개발 도구 투자

도전 과제

마주친 주요 문제들

1. Kubernetes StatefulSet의 복잡성

2. 다중 데이터베이스 지원과 고가용성 구현

3. AWS Aurora 스타일 백업 및 복구 시스템

4. 고가용성 클러스터링 (PostgreSQL vs MySQL)

무엇을 만들었나요

✅ 완성된 기능들

🚧 현재 한계점들

📊 성능 지표

기술적 아키텍처

기술 스택

주요 학습 내용

1. Kubernetes 심화 학습

2. Kubernetes에서의 데이터베이스 운영

3. 클라우드 네이티브 패턴

4. 개발 도구의 중요성

결과

다음 단계

Phase 1 (1-2주)

Phase 2 (3-4주)

Phase 3 (5-8주)

왜 이것이 중요한가

확인해보세요

빠른 시작

결론

태그

kubernetes #database #dba #cloud-native #nodejs #postgresql #mysql #mariadb #side-project #engineering #devops #cursor-ide #aws-aurora #ha-clustering

Building a Mini DBaaS with Kubernetes in One Week - Part 3: Kubernetes Integration & Helm Charts

Introduction

What We'll Build Today

Step 1: Adding Kubernetes Dependencies

Step 2: Creating Kubernetes Service

Step 3: Creating Helm Chart Service

Step 4: Creating Database Instance Controller

Step 5: Creating Routes

Step 6: Updating Main Server File

Step 7: Creating Basic Helm Charts

Step 8: Testing Our Kubernetes Integration

What We've Accomplished

Testing the API

Next Steps

Troubleshooting