DEV Community: Shaikh Al Amin

Install playwrite cli with claude code for any project with more accurate result

Shaikh Al Amin — Tue, 07 Apr 2026 19:43:12 +0000

First install playwright globally in PC

npm i -g @playwright/cli@latest

cd into any project directory and then install

playwright-cli install

and now install playwright skills under claude code

playwright-cli install --skills

Building a Reliable Job Scheduler with Cloudflare Durable Objects and Queues

Shaikh Al Amin — Sat, 07 Mar 2026 16:11:35 +0000

A beginner-friendly deep dive into scheduling repeatable jobs on the edge — no cron servers, no Redis, no infrastructure to manage.

The Problem: "I Need Something to Happen Later"

Imagine you're building an HR/Payroll SaaS app. Your users want things like:

"Send me a daily reminder at 9 AM to fill my timesheet"
"Generate a weekly report every Monday at noon"
"Remind me once on March 15th about the tax deadline"
"Check for new payslips every 30 minutes"

These are all scheduled jobs — things that need to happen at a specific time in the future, and sometimes repeatedly.

In a traditional setup, you'd spin up a cron server, or use something like Redis + BullMQ. But we're running on Cloudflare Workers — there are no long-running servers. Workers are stateless and short-lived. So how do we schedule things?

The answer: Durable Objects + Queues + KV Storage.

The Big Picture (Start Here)

Before diving into code, let's understand the system with a real-world analogy.

The Alarm Clock Analogy

Think of the system as a hotel wake-up call service:

Now map this to our system:

Analogy	Real System	What It Does
You (the guest)	User calling the API	Requests a schedule
Front Desk	Auth Worker (Notifications API)	Validates request, saves to database
Your Alarm Clock	Scheduler Worker (Durable Object)	Keeps time, fires at the right moment
Room Service	Cloudflare Queue + Notify Worker	Delivers the actual job (email, report, etc.)
Guest Registry	PostgreSQL Database	Permanent record of all schedules
Wake-up Call Logbook	KV Storage	Quick lookup of upcoming alarms

The key insight: each guest gets their own personal alarm clock. That's exactly how Durable Objects work — each schedule gets its own isolated instance with its own alarm.

Architecture Overview

Here's the complete system:

Let's break down each piece.

Part 1: The Three Types of Schedules

Our scheduler supports three ways to define "when":

1. Cron Schedule (Repeating Pattern)

Cron is a time expression format that describes recurring patterns. It has 5 fields:

 *    *    *    *    *
 |    |    |    |    |
 |    |    |    |    +--- Day of week (0=Sun, 1=Mon, ..., 6=Sat)
 |    |    |    +-------- Month (1-12)
 |    |    +------------- Day of month (1-31)
 |    +------------------ Hour (0-23)
 +----------------------- Minute (0-59)

Examples:

"0 9 * * *"      = Every day at 9:00 AM
"0 9 * * 1"      = Every Monday at 9:00 AM
"30 14 15 * *"   = 15th of every month at 2:30 PM
"*/30 * * * *"   = Every 30 minutes

In our system, users don't write raw cron. They provide friendly inputs:

{ "dailyTime": { "hour": 9, "minute": 0 } }

And we convert it:

// "0 9 * * *" means: at minute 0, hour 9, every day, every month, every weekday
function generateDailyCron(hour: number, minute: number): string {
  return `${minute} ${hour} * * *`
}

2. Interval Schedule (Every N Minutes)

Simpler than cron — just "run this every 30 minutes" or "run this every 2 hours":

{ "intervalMinutes": 30 }

The scheduler adds the interval to the current time each time it fires:

Now: 10:00 AM
  -> Next run: 10:30 AM
     -> Next run: 11:00 AM
        -> Next run: 11:30 AM
           -> ... forever until stopped

3. Once Schedule (Fire and Forget)

Run exactly one time at a specific moment:

{ "onceAt": "2026-03-15T14:00:00Z" }

After it fires once, it marks itself as inactive. Done.

Part 2: The API Layer (Auth Worker — Notifications Module)

This is where users interact with the system. It's a standard REST API built with Hono.

What Happens When a User Creates a Schedule

POST /notifications/schedules
{
  "type": "profile_reminder",
  "timezone": "America/New_York",
  "dailyTime": { "hour": 14, "minute": 0 }
}

Here's the step-by-step:

Step 1: Validate the Input

The Zod schema enforces that the user provides exactly one timing option:

// You must provide ONE of these — not zero, not two
const options = [
  data.intervalMinutes,  // every N minutes
  data.dailyTime,        // daily at specific time
  data.weeklyTime,       // weekly on specific day/time
  data.monthlyTime,      // monthly on specific day/time
  data.onceAt            // one-time at specific datetime
]
// Exactly one must be provided

Why? Because it doesn't make sense to say "run daily at 9 AM AND every 30 minutes." Pick one.

Step 2: Convert to Internal Format

The service layer (NotificationsService) translates the user-friendly input into the internal format:

// User sends:  { "dailyTime": { "hour": 14, "minute": 0 } }
// We convert:  scheduleType = "cron", cronExpression = "0 14 * * *"

// User sends:  { "intervalMinutes": 30 }
// We convert:  scheduleType = "interval", intervalMinutes = 30

// User sends:  { "onceAt": "2026-03-15T14:00:00Z" }
// We convert:  scheduleType = "once", onceAt = "2026-03-15T14:00:00Z"

Step 3: Save to Database

The schedule is saved to PostgreSQL. This is the permanent record — it tracks who owns the schedule, what type it is, and its current state.

const schedule = await this.repository.createSchedule({
  userId: currentUser.id,
  type: "profile_reminder",
  scheduleType: "cron",
  cronExpression: "0 14 * * *",
  timezone: "America/New_York",
  isActive: true,
  nextRun: /* calculated */,
  lastRun: null,
})

Step 4: Register with the Scheduler Worker

This is the critical bridge. The auth-worker tells the scheduler-worker: "Hey, I need you to fire a job on this schedule."

await this.schedulerPublisher.createSchedule({
  id: schedule.id,           // same UUID from the database
  targetQueue: "notification-queue",   // which queue should receive the job
  jobPayload: {              // the actual message to deliver
    type: "notification",
    scheduleId: schedule.id,
    userId: currentUser.id,
    userEmail: currentUser.email,
    notificationType: "profile_reminder",
    title: "profile_reminder notification",
    message: "Scheduled profile_reminder for user@example.com",
  },
  scheduleType: "cron",
  cronExpression: "0 14 * * *",
  timezone: "America/New_York",
})

Notice: the jobPayload is the exact message that will be delivered to the queue every time the alarm fires. It's stored once and sent repeatedly.

Part 3: The Scheduler Publisher (The Bridge)

How does auth-worker talk to scheduler-worker? Through a Cloudflare Service Binding.

What is a Service Binding?

Think of it like a private internal phone line between two workers. No internet, no public URL, no latency of a network call. It's a direct in-memory connection.

auth-worker  ---[Service Binding / Fetcher]---> scheduler-worker
             (private, fast, no public URL)

The SchedulerPublisher is a tiny HTTP client that uses this binding:

class SchedulerPublisher {
  constructor(private fetcher: Fetcher) {}

  async createSchedule(config) {
    // This looks like an HTTP call, but it's actually a direct
    // worker-to-worker call through Cloudflare's internal network.
    // The "https://scheduler" URL is fake — it's just required by the Fetch API.
    const res = await this.fetcher.fetch("https://scheduler/schedules", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(config),
    })
  }
}

The URL https://scheduler/schedules might look confusing. The hostname scheduler doesn't actually matter — Cloudflare ignores it because the Fetcher already knows which worker to talk to. Only the path /schedules matters for routing.

Part 4: The Scheduler Worker (The Brain)

This is where the magic happens. The scheduler-worker has two parts:

Hono routes — receives requests and delegates to Durable Objects
SchedulerDO — the Durable Object that actually manages timers

What is a Durable Object?

A normal Cloudflare Worker is stateless — it handles a request and forgets everything. A Durable Object is different:

It has persistent storage (survives restarts, deploys, crashes)
It can set alarms (wake me up at a specific time)
Each instance is unique and single-threaded (no race conditions)
It lives as long as it has data or a pending alarm

Think of it as a tiny, immortal server dedicated to one specific task.

One Durable Object Per Schedule

This is the most important design decision. When you create a schedule with ID abc-123, the system creates a Durable Object instance just for that schedule:

// In routes.ts — the Hono API layer
app.post("/schedules", async (c) => {
  const config = await c.req.json()

  // Create a unique Durable Object for this schedule ID
  const doId = c.env.SCHEDULER_DO.idFromName(config.id)  // "abc-123" -> unique DO ID
  const stub = c.env.SCHEDULER_DO.get(doId)               // get the DO instance

  // Forward the request to the DO
  const res = await stub.fetch(new Request("https://do/init", {
    method: "POST",
    body: JSON.stringify(config),
  }))
})

Why one DO per schedule?

Schedule "abc-123" -> DO instance A (has its own alarm, its own storage)
Schedule "def-456" -> DO instance B (completely independent)
Schedule "ghi-789" -> DO instance C (completely independent)

If schedule A fails, B and C are completely unaffected. If schedule A needs to change its timing, only DO instance A is touched. Total isolation.

Part 5: Inside the Durable Object (The Timer Engine)

This is the heart of the system. Let's trace exactly what happens inside a DO.

Initialization: Setting the First Alarm

private async initSchedule(input) {
  // 1. Build the schedule data
  const data = {
    id: input.id,
    targetQueue: "hr-queue",
    jobPayload: { type: "notification", ... },
    scheduleType: "cron",
    cronExpression: "0 14 * * *",
    timezone: "America/New_York",
    isActive: true,
    createdAt: "2026-03-07T...",
  }

  // 2. Save to DO's own persistent storage
  await this.state.storage.put("schedule", data)

  // 3. Calculate when to fire next
  const nextRun = this.calculateNextRun(data)  // e.g., today at 2:00 PM + random offset

  // 4. SET THE ALARM -- this is the key line
  await this.state.storage.setAlarm(nextRun.getTime())

  // 5. Write to KV for external queries
  await this.writeKvRecord(data, nextRun.toISOString())
}

The setAlarm() call is the magic. You're telling Cloudflare: "Wake this Durable Object up at exactly this timestamp." Cloudflare guarantees it will call the alarm() method at that time (within a few seconds of accuracy).

How `calculateNextRun` Works

private calculateNextRun(schedule) {
  const now = new Date()

  // ONCE: fire at the specified time (or immediately if it's in the past)
  if (schedule.scheduleType === "once" && schedule.onceAt) {
    const onceTime = new Date(schedule.onceAt)
    return onceTime > now ? onceTime : new Date(now.getTime() + 1000)
  }

  // INTERVAL: fire after N minutes + random 30-50s offset
  if (schedule.scheduleType === "interval" && schedule.intervalMinutes) {
    return new Date(now.getTime() + schedule.intervalMinutes * 60_000 + randomOffset())
  }

  // CRON: use the croner library to compute the next matching time
  if (schedule.scheduleType === "cron" && schedule.cronExpression) {
    const nextCron = getNextCronRun(schedule.cronExpression, schedule.timezone, now)
    return new Date(nextCron.getTime() + randomOffset())
  }

  // Fallback: 1 minute from now
  return new Date(now.getTime() + 60_000)
}

Why the random offset of 30-50 seconds?

Imagine 5,000 users all set a "daily at 9 AM" schedule. Without the offset, all 5,000 alarms fire at exactly 9:00:00 AM, hitting the queue simultaneously. The random offset spreads them between 9:00:30 and 9:00:50, preventing a stampede. This is called jitter and it's a common pattern in distributed systems.

The Alarm Fires: Delivering the Job

When the alarm goes off, Cloudflare calls the alarm() method:

async alarm() {
  // 1. Load the schedule from storage
  const schedule = await this.state.storage.get("schedule")
  if (!schedule || !schedule.isActive) return  // bail if deleted/paused

  // 2. Pick the right queue
  const queue = this.getTargetQueue(schedule.targetQueue)
  //    "hr-queue"      -> this.env.HR_QUEUE
  //    "auth-queue"    -> this.env.AUTH_QUEUE
  //    "payroll-queue" -> this.env.PAYROLL_QUEUE

  // 3. SEND THE JOB TO THE QUEUE
  await queue.send(schedule.jobPayload)
  // This delivers exactly the jobPayload that was stored during init:
  // { type: "notification", scheduleId: "abc-123", userId: "...", ... }

  // 4. Handle what comes next...
}

After the Job is Sent: The Repeat Loop

This is how repeatable schedules work:

async alarm() {
  // ... (job sent successfully) ...

  // ONE-TIME: deactivate and stop
  if (schedule.scheduleType === "once") {
    schedule.isActive = false
    await this.state.storage.put("schedule", schedule)
    // No new alarm set — this DO is done forever
    return
  }

  // CRON or INTERVAL: calculate next time and set a new alarm
  const nextRun = this.calculateNextRun(schedule)
  await this.state.storage.setAlarm(nextRun.getTime())
  await this.writeKvRecord(schedule, nextRun.toISOString())
  // The cycle continues...
}

Visualizing the repeating cycle:

CRON SCHEDULE: "0 14 * * *" (daily at 2 PM)

Day 1:
  init() -> setAlarm(Day 1, 2:00:37 PM)
                |
                v
  alarm() fires at 2:00:37 PM -> sends job to queue
                                -> calculateNextRun() = Day 2, 2:00:42 PM
                                -> setAlarm(Day 2, 2:00:42 PM)
                                            |
                                            v
  Day 2: alarm() fires at 2:00:42 PM -> sends job to queue
                                      -> calculateNextRun() = Day 3, 2:00:35 PM
                                      -> setAlarm(Day 3, 2:00:35 PM)
                                                  |
                                                  v
  Day 3: ... and so on, forever, until the user stops it

Each alarm sets the next alarm. It's a self-perpetuating chain.

Retry Logic: What If the Queue Send Fails?

Network issues happen. The scheduler handles this gracefully:

try {
  await queue.send(schedule.jobPayload)
  // Success! Reset retry counter
} catch (error) {
  const retryCount = (schedule.retryCount ?? 0) + 1

  if (retryCount <= 3) {
    // Try again in 60 seconds
    schedule.retryCount = retryCount
    await this.state.storage.setAlarm(Date.now() + 60_000)
    return  // don't schedule the next regular run yet
  }

  // 3 retries failed — give up, mark as inactive
  schedule.isActive = false
  // The schedule is now dead. A human needs to investigate.
}

Visualized:

alarm() fires -> queue.send() FAILS
                   |
                   v
              retry 1 (wait 60s) -> queue.send() FAILS
                                       |
                                       v
                                  retry 2 (wait 60s) -> queue.send() FAILS
                                                           |
                                                           v
                                      retry 3 (wait 60s) -> queue.send() FAILS
                                                               |
                                                               v
                                                          GIVE UP
                                                          isActive = false

Part 6: The Three Storage Layers (Why Three?)

The system uses three different storage mechanisms. Each serves a distinct purpose:

Storage	What it stores	Who reads it
PostgreSQL (DB)	User-facing schedule records with ownership (userId), notification type, active status, next/last run times	Auth Worker API (user CRUD operations)
DO Storage	The "live" schedule state: job payload, cron expression, retry count, active flag (private to each DO instance)	The Durable Object itself (timer engine)
KV Storage	Read-only index of all schedules with their next run times (queryable, fast)	Admin/debug queries, listing

Why can't we just use one?

PostgreSQL is in the auth-worker. The scheduler-worker can't access it (different worker, no DB connection).
DO Storage is private to each DO instance. You can't query across all DOs ("show me all active schedules"). That's by design — DOs are isolated.
KV fills the gap: it's globally readable, fast, and lets you list/filter schedules without waking up every single DO.

Think of it this way:

PostgreSQL = "What schedules does USER X have?"       (user-facing)
DO Storage = "What should I do when my alarm rings?"  (execution engine)
KV         = "What are ALL the upcoming schedules?"   (admin overview)

Part 7: Complete End-to-End Example

Let's trace a complete flow from user request to job delivery.

Scenario: "Send me a daily stock price notification at 2 PM New York time"

TIME: March 7, 2026 at 10:00 AM

STEP 1: User calls the API
=========================================================
POST /notifications/schedules
Authorization: Bearer <jwt-token>
{
  "type": "stock_price",
  "timezone": "America/New_York",
  "dailyTime": { "hour": 14, "minute": 0 }
}

STEP 2: Auth Worker processes
=========================================================
  a) Zod validates: type is valid, exactly one timing option provided  [OK]
  b) Convert: dailyTime {14, 0} -> cronExpression "0 14 * * *"
  c) Calculate next run: March 7, 2:00 PM ET (today! it's before 2 PM)
  d) Save to PostgreSQL:
     {
       id: "schedule-uuid-abc",
       userId: "user-uuid-123",
       type: "stock_price",
       scheduleType: "cron",
       cronExpression: "0 14 * * *",
       timezone: "America/New_York",
       isActive: true,
       nextRun: "2026-03-07T19:00:00Z"    // 2 PM ET = 7 PM UTC
     }

STEP 3: Auth Worker -> Scheduler Worker (via service binding)
=========================================================
  SchedulerPublisher sends:
  POST https://scheduler/schedules
  {
    id: "schedule-uuid-abc",
    targetQueue: "notification-queue",
    jobPayload: {
      type: "notification",
      scheduleId: "schedule-uuid-abc",
      userId: "user-uuid-123",
      userEmail: "alice@company.com",
      notificationType: "stock_price",
      title: "stock_price notification",
      message: "Scheduled stock_price for alice@company.com"
    },
    scheduleType: "cron",
    cronExpression: "0 14 * * *",
    timezone: "America/New_York"
  }

STEP 4: Scheduler Worker routes to Durable Object
=========================================================
  a) doId = SCHEDULER_DO.idFromName("schedule-uuid-abc")
     -> A unique DO instance is created (or resumed) for this ID
  b) stub.fetch("https://do/init", { body: config })

STEP 5: Durable Object initializes
=========================================================
  a) Saves full schedule data to DO storage
  b) Calculates next run: March 7, 2:00:37 PM ET (with 37s random jitter)
  c) Sets alarm: setAlarm(1741374037000)  // Unix timestamp in milliseconds
  d) Writes to KV:
     Key: "schedule:schedule-uuid-abc"
     Value: { ...full record, nextRun: "2026-03-07T19:00:37Z" }

STEP 6: Response flows back to user
=========================================================
  201 Created
  {
    "id": "schedule-uuid-abc",
    "type": "stock_price",
    "scheduleType": "cron",
    "cronExpression": "0 14 * * *",
    "timezone": "America/New_York",
    "isActive": true,
    "nextRun": "2026-03-07T19:00:37Z",
    "lastRun": null,
    "createdAt": "2026-03-07T15:00:00Z",
    "updatedAt": "2026-03-07T15:00:00Z"
  }

  === TIME PASSES... it's now March 7, 2:00:37 PM ET ===

STEP 7: Cloudflare wakes up the Durable Object
=========================================================
  alarm() method is called automatically by Cloudflare's runtime.

  a) Reads schedule from DO storage
  b) Checks isActive === true  [OK]
  c) Resolves target queue: "hr-queue" -> this.env.HR_QUEUE
  d) Sends message to queue:
     HR_QUEUE.send({
       type: "notification",
       scheduleId: "schedule-uuid-abc",
       userId: "user-uuid-123",
       userEmail: "alice@company.com",
       notificationType: "stock_price",
       title: "stock_price notification",
       message: "Scheduled stock_price for alice@company.com"
     })

STEP 8: Schedule the next run (THE REPEAT)
=========================================================
  a) scheduleType is "cron", not "once" -> keep going!
  b) calculateNextRun("0 14 * * *", "America/New_York")
     -> March 8, 2:00:42 PM ET (tomorrow, with new random jitter)
  c) setAlarm(March 8, 2:00:42 PM ET)
  d) Update KV with new nextRun

STEP 9: Queue consumer processes the job
=========================================================
  HR Worker receives the message from hr-queue.
  It sees type: "notification" and handles it
  (e.g., sends an email, pushes to websocket, etc.)

  === NEXT DAY: March 8, 2:00:42 PM ET ===

STEP 10: The cycle repeats (back to Step 7)
=========================================================
  alarm() fires again -> sends job -> calculates March 9 -> sets alarm
  ... and so on, every day, until the user deletes or pauses the schedule.

Part 8: Update and Delete Flows

Updating a Schedule

User wants to change from daily 2 PM to weekly Monday 9 AM:

PUT /notifications/schedules/schedule-uuid-abc
{ "weeklyTime": { "dayOfWeek": 1, "hour": 9, "minute": 0 } }

What happens:

Auth Worker:
  1. Load from DB, verify ownership
  2. Convert weeklyTime -> cronExpression "0 9 * * 1"
  3. Update PostgreSQL record
  4. Call schedulerPublisher.updateSchedule("schedule-uuid-abc", {
       scheduleType: "cron",
       cronExpression: "0 9 * * 1",
       timezone: "America/New_York"
     })

Scheduler Worker -> Durable Object:
  1. Load current schedule from DO storage
  2. Merge new fields over old fields
  3. Cancel old alarm (implicitly — setAlarm overwrites)
  4. Calculate new next run (next Monday at 9 AM)
  5. Set new alarm
  6. Update KV

Deleting a Schedule

DELETE /notifications/schedules/schedule-uuid-abc

What happens:

Auth Worker:
  1. Verify ownership
  2. Call schedulerPublisher.stopSchedule("schedule-uuid-abc")
  3. Delete from PostgreSQL

Scheduler Worker -> Durable Object:
  1. Delete the alarm (no more wake-ups)
  2. Delete schedule from DO storage
  3. Mark as inactive in KV
  4. Clean up secondary KV index

The DO effectively becomes empty. Cloudflare will garbage-collect it eventually.

Part 9: The Queue System (How Jobs Get Delivered)

Cloudflare Queues work like a conveyor belt:

The scheduler-worker has three queue bindings — it can send jobs to any of the three workers:

// In the Durable Object
private getTargetQueue(targetQueue: TargetQueue): Queue {
  switch (targetQueue) {
    case "auth-queue":    return this.env.AUTH_QUEUE     // -> auth-worker
    case "notification-queue":      return this.env.NOTIFICATION_QUEUE       // -> notification-worker
  }
}

Each queue has typed messages. The notification-queue accepts NotificationJob which includes:

type NotificationJob =
  | NotificationJob       // { type: "notification", scheduleId, userId, ... }
  | ReportGenerationJob   // { type: "report_generation", reportType, ... }
  | DataSyncJob           // { type: "data_sync", source, entityType, ... }
  | OnboardingReminderJob // { type: "onboarding_reminder", employeeId, ... }

The consumer uses the type field to decide what to do:

// Conceptual queue consumer in hr-worker
async queue(batch, env) {
  for (const message of batch.messages) {
    switch (message.body.type) {
      case "notification":
        await handleNotification(message.body)
        break
      case "report_generation":
        await generateReport(message.body)
        break
      // ...
    }
    message.ack()
  }
}

Part 10: Why This Architecture Works Well

Reliability

Durable Object alarms survive crashes and deploys. If Cloudflare restarts your DO, the alarm is still set.
Queue delivery has at-least-once guarantees. If the consumer crashes, the message is redelivered.
Retry logic in the DO handles transient queue failures (3 retries with 60s backoff).

Scalability

Each schedule is its own DO — no shared state, no database locks, no contention.
100,000 schedules = 100,000 independent DOs, each with its own alarm. Cloudflare handles the distribution.

Simplicity

No cron servers to manage. No Redis. No "is the scheduler process still running?" anxiety.
The entire scheduler-worker is ~260 lines of TypeScript.

Cost

DOs only consume resources when they're active (during alarm handling). A DO sitting idle waiting for its alarm costs nothing.
Queues charge per message, not per connection.

Summary

User -> Auth Worker API -> validates & saves to PostgreSQL
                        -> tells Scheduler Worker via service binding
                              -> creates a Durable Object for this schedule
                              -> DO sets an alarm (Cloudflare's built-in timer)
                              -> DO writes to KV for queryability

         ... time passes ...

Cloudflare wakes the DO -> alarm() fires
                        -> DO sends jobPayload to the target queue
                        -> Queue delivers to the consumer worker
                        -> Consumer processes the job (sends email, etc.)

         ... for repeating schedules ...

                        -> DO calculates next run time
                        -> DO sets a new alarm
                        -> The cycle repeats forever

That's it. An alarm clock that never forgets, never crashes, and scales to millions of schedules — all without a single server to manage.

Building Resilient, Scalable Apps with Bun, Hono, and Cloudflare

Shaikh Al Amin — Fri, 27 Feb 2026 21:31:47 +0000

When building modern web applications, handling background jobs, managing state, and ensuring fast database access are critical. Recently, I migrated my Bun + Hono project to Cloudflare’s ecosystem, and the combination of Cloudflare Queues, Durable Objects, KV, R2, D1, and Hyperdrive with Planetscale turned out to be a game-changer—especially for managing repeatable jobs and dead-letter queues (DLQ).

The Stack at a Glance:

Bun + Hono: Lightweight, fast runtime and web framework running on Cloudflare Workers.

Cloudflare Queues: Reliable message passing for background tasks.
Durable Objects + D1: Stateful processing with persistent SQL storage (via D1) for job metadata and retry logic.

KV & R2: Ultra-low-latency key-value cache and object storage for files.
Planetscale + Hyperdrive: MySQL-compatible database with Hyperdrive’s connection pooling and query caching, accessed through Drizzle ORM.

How It All Flows:

Request Handling: An incoming HTTP request hits the Hono app. For simple reads/writes, the app uses Drizzle ORM to query Planetscale through Hyperdrive. Hyperdrive caches frequent queries, dramatically reducing latency and database load.

Job Orchestration: For tasks that need background processing (e.g., sending emails, image processing), the app enqueues a message into Cloudflare Queue. This guarantees delivery and decouples the request from heavy work.

Durable Object Processing: A Durable Object consumes messages from the queue. It maintains job state in its attached D1 database (for persistence across failures) and uses KV for ephemeral data like rate-limit counters. If the job involves files, it streams them to/from R2.

Database Interactions: The Durable Object also updates the Planetscale database via Drizzle + Hyperdrive. Since Hyperdrive pools connections and caches read results, repeated lookups (e.g., checking user quotas) are lightning fast.

Resilience & DLQ Handling: Failed jobs are retried according to logic inside the Durable Object. If all retries fail, the job can be moved to a dead-letter queue (simulated using D1 or KV), allowing manual inspection without losing data.

Why This Rocks:

Performance: Hyperdrive’s cache reduces database round trips; KV gives microsecond access to hot data; R2 handles large files without bloating the DB.

Reliability: Queues and Durable Objects ensure exactly-once processing semantics and state persistence—perfect for financial transactions or idempotent jobs.

Developer Experience: Bun’s speed, Hono’s simplicity, and Drizzle’s type-safe SQL make development a joy. Everything deploys seamlessly with Wrangler.

Build Production-Ready GCP Infrastructure from Scratch Part 04

Shaikh Al Amin — Wed, 04 Feb 2026 12:49:58 +0000

Build Production-Ready GCP Infrastructure from Scratch: A Complete Console Guide

A 4-Part Series for Complete Beginners

Part 4: Observability & Load Balancer

Overview

In this final part, you'll complete your infrastructure with observability and external access. We'll create Prometheus for metrics, Loki for logs, Grafana for dashboards, and an Application Load Balancer for external traffic.

What you'll build:

Prometheus VM for metrics collection (7-day retention)
Loki VM for log aggregation with Grafana
External Application Load Balancer with SSL
End-to-end health checks and monitoring

Estimated time: 45-60 minutes

Estimated cost: ~$64/month

Final cumulative cost: ~$301/month

Prerequisites

Before continuing, ensure you've completed Parts 1-3:

[ ] VPC and 5 subnets exist (including private-obs subnet)
[ ] Cloud SQL with private IP is running
[ ] Backend MIG has 2+ healthy VMs
[ ] Cache VM with Redis/PgBouncer is running
[ ] Firewall rules allow health check IPs (35.191.0.0/16, 130.211.0.0/22)

If you missed Parts 1-3: Start with Part 1: Foundation →

Step 1: Create Static Internal IPs for Observability

What are Static Internal IPs?

Static IPs ensure observability VMs have predictable IP addresses. This makes configuration easier (no need to update configs if VMs are recreated).

Prometheus Static IP

Navigation Path

Navigate to VPC networks → Internal IP addresses
Click "Reserve static internal IP address"

IP Configuration

Field	Value	Notes
Name	`dev-prometheus-ip`	Descriptive
Network	`dev-network`	Our VPC
Subnetwork	`private-obs`	Observability subnet
IP address	`10.0.5.10`	Manual assignment

Click "Reserve".

Loki Static IP

Repeat the process:

Field	Value
Name	`dev-loki-ip`
Network	`dev-network`
Subnetwork	`private-obs`
IP address	`10.0.5.11`

Verify IPs

You should see 2 reserved IPs:

Name	IP Address	Subnetwork
dev-prometheus-ip	10.0.5.10	private-obs
dev-loki-ip	10.0.5.11	private-obs

Step 2: Create Prometheus VM

What is Prometheus?

Prometheus is a metrics collection and storage system:

Scrapes metrics from Node Exporter (every VM)
Stores 7 days of metrics data
Provides query API for Grafana

Why self-hosted: Full control, no vendor lock-in, predictable costs.

Navigation Path

Navigate to Compute Engine → VM instances
Click "Create instance"

VM Configuration

Basic Settings

Field	Value	Notes
Name	`dev-prometheus`	Descriptive
Region	`europe-west1`	Same as VPC
Zone	`europe-west1-b`	Zone b

Machine Type

Field	Value	Notes
Machine type	`e2-medium`	2 vCPU, 4GB RAM

Boot Disk

Field	Value
OS	`Ubuntu 22.04 LTS Minimal`
Disk type	`pd-balanced`
Size	`50 GB`

Why 50GB: 7 days of metrics at 15s interval requires ~30-40GB. 50GB provides headroom.

Network Interface

Field	Value	Notes
Network	`dev-network`	Our VPC
Subnetwork	`private-obs`	Observability subnet
Primary internal IP	`Static` → `dev-prometheus-ip`	Use static IP
External IPv4 address	None	No public IP needed

Service Account

Field	Value
Service account	`observability-dev-sa`

Metadata - Startup Script

Key: startup-script

Value:

#!/bin/bash
# Prometheus Startup Script

set -e

echo "=== Prometheus Startup Script Begin $(date) ==="

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

# Install Node Exporter for self-monitoring
NODE_EXPORTER_VERSION="1.6.1"

echo "Installing Node Exporter ${NODE_EXPORTER_VERSION}..."

useradd --no-create-home --shell /bin/false node_exporter || true

wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz" -O /tmp/node_exporter.tar.gz
tar xzf /tmp/node_exporter.tar.gz -C /tmp
cp /tmp/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter /usr/local/bin/
chown node_exporter:node_exporter /usr/local/bin/node_exporter
chmod +x /usr/local/bin/node_exporter

cat > /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
Type=simple
User=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9100
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable node_exporter
systemctl start node_exporter

# Create Prometheus configuration
cat > /opt/prometheus.yml <<'EOF'
global:
  scrape_interval: 15s
  retention: 7d

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['localhost:9100']

  # Backend VMs (update IPs after MIG creation)
  - job_name: 'backend'
    static_configs:
      - targets: ['10.0.2.2:9100', '10.0.2.3:9100']
        labels:
          tier: backend

  # Cache VM
  - job_name: 'cache'
    static_configs:
      - targets: ['10.0.4.2:9100']
        labels:
          tier: cache
EOF

# Run Prometheus
docker run -d \
  --name prometheus \
  --restart unless-stopped \
  -p 9090:9090 \
  -v /opt/prometheus.yml:/etc/prometheus/prometheus.yml \
  prom/prometheus:latest

echo "=== Prometheus Startup Script Complete $(date) ==="
echo "Prometheus running on port 9090"
echo "Node Exporter running on port 9100"

Create the VM

Click "Create".

VM Creation Time: 2-3 minutes

Verify Prometheus

You should see:

Name: dev-prometheus
Status: Running
Internal IP: 10.0.5.10
External IP: None

Step 3: Create Loki VM (with Grafana)

What is Loki and Grafana?

Loki: Log aggregation system (like Prometheus, but for logs)
Grafana: Visualization dashboard for metrics and logs

Why combined: Cost optimization. Single VM runs both services (~$23/month).

Navigation Path

Navigate to Compute Engine → VM instances
Click "Create instance"

VM Configuration

Basic Settings

Field	Value	Notes
Name	`dev-loki`	Descriptive
Region	`europe-west1`	Same as VPC
Zone	`europe-west1-b`	Zone b

Machine Type

Field	Value	Notes
Machine type	`e2-medium`	2 vCPU, 4GB RAM

Boot Disk

Field	Value
OS	`Ubuntu 22.04 LTS Minimal`
Disk type	`pd-balanced`
Size	`50 GB`

Network Interface

Field	Value	Notes
Network	`dev-network`	Our VPC
Subnetwork	`private-obs`	Observability subnet
Primary internal IP	`Static` → `dev-loki-ip`	Use static IP
External IPv4 address	`Ephemeral`	Enable for Grafana access

Why public IP: Grafana needs to be accessible from your browser. In production, use IAP instead of public IP.

Service Account

Field	Value
Service account	`observability-dev-sa`

Metadata - Startup Script

Key: startup-script

Value:

#!/bin/bash
# Loki and Grafana Startup Script

set -e

echo "=== Loki/Grafana Startup Script Begin $(date) ==="

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

# Install Docker Compose
apt-get update
apt-get install -y docker-compose

# Create docker-compose.yml
cat > /opt/docker-compose.yml <<'EOF'
version: '3.8'

services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    volumes:
      - /opt/loki-config.yml:/etc/loki/local-config.yaml
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin123
      - GF_USERS_ALLOW_SIGN_UP=false
      - GF_SERVER_ROOT_URL=http://localhost:3000
    volumes:
      - grafana-storage:/var/lib/grafana
      - /opt/grafana-provisioning:/etc/grafana/provisioning
    restart: unless-stopped

volumes:
  loki-data:
  grafana-storage:
EOF

# Create Loki config
cat > /opt/loki-config.yml <<'EOF'
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 1h
  max_chunk_age: 1h
  chunk_target_size: 1048576
  chunk_retain_period: 30s

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/cache
    shared_store: filesystem
  filesystem:
    directory: /loki/chunks

chunk_store_config:
  max_look_back_period: 168h

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s
EOF

# Create Grafana provisioning
mkdir -p /opt/grafana-provisioning/datasources

cat > /opt/grafana-provisioning/datasources/loki.yml <<'EOF'
apiVersion: 1

datasources:
  - name: Loki
    type: loki
    access: proxy
    url: http://loki:3100
    isDefault: false
    editable: false
EOF

cat > /opt/grafana-provisioning/datasources/prometheus.yml <<'EOF'
apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://10.0.5.10:9090
    isDefault: true
    editable: false
EOF

# Start services
cd /opt
docker-compose up -d

echo "=== Loki/Grafana Startup Script Complete $(date) ==="
echo "Loki running on port 3100"
echo "Grafana running on port 3000 (http://EXTERNAL_IP:3000)"
echo "Login: admin / admin123"

Create the VM

Click "Create".

VM Creation Time: 2-3 minutes

Verify Loki VM

You should see:

Name: dev-loki
Status: Running
Internal IP: 10.0.5.11
External IP: (Assigned IP)

Copy the external IP - we'll need it for Grafana access.

Step 4: Access Grafana Dashboard

Get Loki VM External IP

Navigate to Compute Engine → VM instances
Find dev-loki VM
Copy the External IP address

Access Grafana

Open your browser and navigate to:

http://[LOKI_EXTERNAL_IP]:3000

Grafana Login

Field	Value
Email or username	`admin`
Password	`admin123`

Click "Log in".

Verify Datasources

Click "Configuration" (gear icon) → Data sources
Verify Prometheus shows "Healthy" (green checkmark)
Verify Loki shows "Healthy"

Troubleshooting: If datasources are unhealthy:

Check Prometheus is running: docker ps on Loki VM

Verify network connectivity: curl http://10.0.5.10:9090/-/healthy

Check Docker logs: docker logs loki or docker logs grafana

Step 5: Update Prometheus Targets

What are Prometheus Targets?

Prometheus "scrapes" metrics from targets. We need to add our backend and cache VMs.

Update Prometheus Configuration

SSH to Prometheus VM via bastion:

# From your local machine
gcloud compute ssh dev-bastion --tunnel-through-iap

# From bastion, SSH to Prometheus
ssh 10.0.5.10

# Edit Prometheus config
sudo nano /opt/prometheus.yml

Update the targets with actual backend VM IPs:

scrape_configs:
  - job_name: 'backend'
    static_configs:
      - targets: ['10.0.2.2:9100', '10.0.2.3:9100']  # Update with actual IPs
        labels:
          tier: backend

  - job_name: 'cache'
    static_configs:
      - targets: ['10.0.4.2:9100']  # Update with actual IP
        labels:
          tier: cache

Restart Prometheus:

docker restart prometheus

Verify Targets

Navigate to http://[LOKI_EXTERNAL_IP]:3000
Go to Explore → Select Prometheus datasource
Query: up{job="backend"}
You should see metrics from backend VMs

Step 6: Create External Application Load Balancer

What is an ALB?

Application Load Balancer (ALB):

Distributes traffic across backend VMs
Provides single public IP for external access
Handles SSL termination
Health checks for backend availability

Step 6a: Create Health Check

Navigation Path

Navigate to Compute Engine → Health checks
Click "Create health check"

Health Check Configuration

Field	Value	Notes
Name	`dev-lb-health-check`	Descriptive
Protocol	`HTTP`	HTTP health check
Port	`3000`	NestJS app port
Request path	`/api/health`	App must implement this
Check interval	`20` seconds	How often to check
Timeout	`5` seconds	Response timeout
Healthy threshold	`2`	2 successes = healthy
Unhealthy threshold	`5`	5 failures = unhealthy

Click "Create".

Step 6b: Add Named Port to MIG

What is a Named Port?

Named ports link a name (like "http") to a port number (3000). The load balancer uses named ports for routing.

Navigation Path

Navigate to Compute Engine → Instance groups
Click on dev-backend-mig
Click "Edit group"

Edit MIG

Scroll to "Named ports" section:

Click "Add item":

Field	Value
Name	`http`
Port	`3000`

Click "Save".

Step 6c: Create Load Balancer

Navigation Path

Navigate to Network Services → Load balancing
Click "Create load balancer"

Select LB Type

Click "Application Load Balancer (HTTP/S)" → Click "Configure".

Basic Configuration

Field	Value	Notes
Name	`dev-lb`	Environment-prefixed
Region	`europe-west1`	Same as VPC
Network	`dev-network`	Our VPC

Backend Configuration

Backend type: Instance group

Backend configuration:

Field	Value	Notes
Region	`europe-west1`	Same region
Backend	`dev-backend-mig`	Our MIG
Balancing mode	`Rate`	Rate-based balancing
Maximum RPS	`100`	Per instance

Health check:

Field	Value
Health check	`dev-lb-health-check`

Session affinity:

Field	Value	Notes
Session affinity	`None`	Stateless app

Advanced settings:

Field	Value	Notes
Timeout	`30` seconds	Connection timeout
Enable Cloud CDN	Off	Not needed for now

Click "Done".

Frontend Configuration

Protocol: HTTPS (we'll add HTTP redirect)

HTTPS Frontend:

Click "Add frontend IP and port":

Field	Value	Notes
Protocol	`HTTPS`	Secure traffic
IP address	`Reserve new static IP`	Create new IP
IP address name	`dev-lb-ip`	Descriptive
Port	`443`	HTTPS port

Certificate:

Field	Value	Notes
Certificate	`Create a new certificate`	For SSL

Create Certificate:

Field	Value	Notes
Name	`dev-lb-cert`	Descriptive
Type	`Google-managed certificate`	Auto-renew

Domains:

Field	Value
Domains	(Skip for now)

Note: For testing, you can use HTTP only (skip certificate). For production, add your domain and create Google-managed certificate.

Click "Create".

HTTP Frontend (Redirect):

Click "Add frontend IP and port":

Field	Value
Protocol	`HTTP`
IP address	`dev-lb-ip`
Port	`80`
Enable redirect to HTTPS	✓ Enable

Routing Rules

Host rules: Leave default (all hosts)

Path matcher: Leave default (all paths)

Backend service: Select the backend created above

Click "Done".

Create the Load Balancer

Review configuration and click "Create load balancer".

Creation Time: 5-10 minutes

Verify Load Balancer

You should see:

Name: dev-lb
Status: (Checkmark) - Active
IP address: (Reserved IP)
Backends: dev-backend-mig (healthy)

Step 7: Test End-to-End Connectivity

Test 1: Load Balancer Health Check

# Get LB IP
gcloud compute forwarding-rules list --filter="name=dev-lb*"

# Test health endpoint
curl http://[LB_IP]/api/health

Expected: HTTP 200 with response like {"status":"ok"}

Test 2: Full Request Flow

# Test through load balancer
curl http://[LB_IP]/api/test

Expected: Response from backend application

Test 3: Prometheus Metrics

Open Grafana: http://[LOKI_EXTERNAL_IP]:3000
Go to Explore → Prometheus
Query: rate(http_requests_total[5m])
You should see request metrics

Test 4: Loki Logs

Open Grafana: http://[LOKI_EXTERNAL_IP]:3000
Go to Explore → Loki
Query: {job="nestjs"}
You should see application logs

Part 4 Verification Checklist

Final Verification

[ ] Prometheus VM running at 10.0.5.10:9090
[ ] Loki VM running with external IP assigned
[ ] Grafana accessible at http://[LOKI_EXTERNAL_IP]:3000
[ ] Prometheus datasource shows "Healthy"
[ ] Loki datasource shows "Healthy"
[ ] Prometheus scraping backend and cache VMs
[ ] Load balancer has reserved static IP
[ ] Backend service shows healthy instances
[ ] curl http://[LB_IP]/api/health returns 200
[ ] Full request flow works (LB → MIG → App)

Cost Summary - Part 4

Component	Monthly Cost	Notes
Prometheus VM	~$23	e2-medium, 50GB disk
Loki VM	~$23	e2-medium, 50GB disk
Load Balancer	~$18	ALB forwarding rules
Total Part 4	~$64	~$64/month

Final Infrastructure Cost

Component	Monthly Cost
Part 1 (VPC, NAT, Firewall)	~$42
Part 2 (Bastion)	~$11
Part 3 (Cloud SQL, MIG, Cache)	~$184
Part 4 (Observability, LB)	~$64
Total	~$301/month

Cost Optimization Tips:

Use preemptible VMs for non-critical workloads (save 80%)

Reduce Cloud SQL tier to db-g1-small for dev (save ~$70/month)

Scale MIG to 0 during off-hours (save ~$46/month)

Disable Flow Logs on non-critical subnets (save ~$5-10/month)

Comprehensive Troubleshooting

Issue: Grafana Cannot Connect to Datasources

Symptom: Datasource shows "Could not connect"

Diagnosis:

Check Prometheus is running
Verify network connectivity
Check firewall rules

Solution:

# From Loki VM
docker ps  # Check containers running

# Test Prometheus
curl http://10.0.5.10:9090/-/healthy

# Test Loki
curl http://localhost:3100/ready

Issue: Load Balancer Shows 0/0 Healthy

Symptom: No healthy backend instances

Diagnosis:

Check health check configuration
Verify app is running on port 3000
Check firewall for health check IPs

Solution:

# From backend VM
curl localhost:3000/health
netstat -tlnp | grep 3000

Common fixes:

Increase health check unhealthy threshold to 5
Increase initial delay for MIG autohealing to 300
Verify firewall rule allows 35.191.0.0/16 and 130.211.0.0/22

Issue: High CPU on Backend VMs

Symptom: VMs constantly at 90%+ CPU

Diagnosis:

Check application logs
Verify autoscaling thresholds
Profile application performance

Solution:

# Check CPU usage
top -bn1 | head -20

# Check Node.js process
pm2 monit

# Check autoscaler status
gcloud compute instance-groups managed describe dev-backend-mig \
  --region europe-west1

Fixes:

Increase machine type (e2-medium → e2-highcpu-4)
Optimize application code
Increase max replicas to 6 or 8

Issue: Secrets Not Accessible from VMs

Symptom: "Permission denied" accessing secrets

Diagnosis:

Verify service account has Secret Accessor role
Check secret exists and has versions

Solution:

# From VM with correct SA
gcloud secrets versions list db-credentials-dev

# Add IAM role if missing
gcloud secrets add-iam-policy-binding db-credentials-dev \
  --member='serviceAccount:backend-dev-sa@PROJECT_ID.iam.gserviceaccount.com' \
  --role='roles/secretmanager.secretAccessor'

Next Steps Beyond This Guide

1. Domain & SSL

Purchase domain from registrar
Configure DNS to point to LB IP
Update Load Balancer certificate with your domain
Enable Google-managed certificate

2. CI/CD Pipeline

Setup GitHub Actions
Auto-deploy on push
Run tests in container
Automated rollback on failure

3. Monitoring Enhancements

Add alert rules in Prometheus
Configure PagerDuty/Slack integration
Create custom Grafana dashboards
Set up uptime monitoring

4. Security Hardening

Enable VPC Service Controls
Configure Organization Policies
Setup Security Command Center
Implement workload identity

5. Multi-Environment

Create staging environment
Use Shared VPC
Implement environment isolation
Setup service directory

End-to-End Verification Test

Test 1: Full Request Flow

# 1. Get Load Balancer IP
gcloud compute forwarding-rules list --filter="name=dev-lb*"

# 2. Send test request
curl http://[LB_IP]/api/health

# Expected: {"status":"ok","timestamp":"..."}

Test 2: Database Connectivity

# From backend VM (via bastion)
gcloud compute ssh dev-bastion --tunnel-through-iap
ssh 10.0.2.2  # Backend VM IP

# Test Cloud SQL connection
psql -h 10.100.0.2 -U backend-dev-sa -d appdb

Test 3: Cache Connectivity

# From backend VM
redis-cli -h 10.0.4.2 PING
# Expected: PONG

# Test PgBouncer
psql -h 10.0.4.2 -p 6432 -U app_admin -d appdb

Test 4: Monitoring Pipeline

Generate traffic: ab -n 1000 http://[LB_IP]/api/health
Open Grafana
Check dashboards for metrics
Verify Loki has logs

Test 5: Disaster Recovery

# Simulate instance failure
gcloud compute instances delete [ONE_BACKEND_INSTANCE] --quiet

# Verify:
# - MIG auto-heals (new instance appears)
# - Load balancer continues serving
# - No data loss

Congratulations!

You've built a complete, production-ready GCP infrastructure:

✅ Network: VPC with 5 subnets, Cloud NAT, firewall rules
✅ Security: Secret Manager, bastion with IAP, OS Login
✅ Data: Cloud SQL PostgreSQL with regional HA
✅ Compute: Managed Instance Group with autoscaling
✅ Cache: Redis + PgBouncer
✅ Observability: Prometheus, Loki, Grafana
✅ Access: External Application Load Balancer

Your infrastructure is ready for production deployment!

References

Series Complete! You now have a fully functional, production-ready GCP infrastructure. From here, you can deploy your NestJS application and scale as needed.

Build Production-Ready GCP Infrastructure from Scratch Part 03

Shaikh Al Amin — Wed, 04 Feb 2026 12:49:25 +0000

Build Production-Ready GCP Infrastructure from Scratch: A Complete Console Guide

A 4-Part Series for Complete Beginners

Part 3: Database & Compute Resources

Overview

In this part, you'll build the data and compute layers of your infrastructure. We'll create a Cloud SQL PostgreSQL instance with high availability, set up a Managed Instance Group (MIG) for backend VMs, and deploy a cache VM with Redis and PgBouncer.

What you'll build:

Private Service Connection for Cloud SQL private IP
Cloud SQL PostgreSQL with regional HA
Managed Instance Group (MIG) with 2-4 backend VMs
Cache VM with Redis and PgBouncer

Estimated time: 60-90 minutes (includes Cloud SQL provisioning time)

Estimated cost: ~$184/month

Cumulative cost: ~$237/month

Prerequisites

Before continuing, ensure you've completed Parts 1 & 2:

[ ] VPC and 5 subnets exist (including private-data subnet)
[ ] Cloud NAT gateway is running
[ ] Service accounts created (including backend-dev-sa, cache-dev-sa)
[ ] Firewall rules created (health check, backend-to-cache)
[ ] Bastion host is running
[ ] Secrets created in Secret Manager

If you missed Parts 1-2: Start with Part 1: Foundation →

Step 1: Setup Private Service Connection

What is Private Service Connection?

Cloud SQL requires a private IP address for secure access. Private Service Connection allocates a CIDR range (10.100.0.0/16) that Google-managed services (like Cloud SQL) use for private IPs.

How it works:

You allocate an IP range (10.100.0.0/16)
Google creates a VPC peering connection
Cloud SQL gets a private IP from this range

Why private IP: More secure than public IP. No exposure to internet. Lower latency.

Navigation Path

Navigate to VPC networks → Private service connection
Click "Set up private service connection"

Allocation Settings

Allocate an IP range:

Field	Value	Notes
Name	`google-managed-services-dev-network`	Descriptive
IP range	`10.100.0.0/16`	Standard range for PSA
Purpose	`VPC peering`	Required for Cloud SQL

Why 10.100.0.0/16: This range doesn't overlap with our VPC (10.0.0.0/16). Google-managed services will allocate IPs from this range.

Create Connection

Click "Connect" and wait 1-2 minutes.

Verify Connection

You should see:

Status: Connected
IP range: 10.100.0.0/16
Peering: servicenetworking-googleapis-com

Step 2: Create Cloud SQL PostgreSQL Instance

What is Cloud SQL?

Cloud SQL is Google's managed PostgreSQL service:

Automatic backups
High availability (regional)
Point-in-time recovery
Automatic patching
99.99% SLA (regional)

Cost Alert: Cloud SQL is the most expensive component (~$115/month for regional HA). Consider db-g1-small (~$45/month) for non-critical workloads.

Navigation Path

Navigate to SQL → SQL
Click "Create Instance"
Click "Choose PostgreSQL"

Instance Settings

Basic Information

Field	Value	Notes
Instance ID	`app-dev-db`	Must be globally unique
Password	(Generate strong password)	Save this password!
PostgreSQL version	`PostgreSQL 16`	Latest stable
Region	`europe-west1`	Same as VPC

Password Generation: Use a password manager or generate with: openssl rand -base64 32

Zone selection:

Field	Value	Notes
Zone	`Automatically select high availability zone`	Let GCP choose for HA

Why automatic zone: GCP selects the zone with best availability. Regional HA will create a standby in another zone.

Database Configuration

Machine Type

Click "Change" to customize:

Field	Value	Notes
Instance type	`db-n1-standard-2`	2 vCPU, 8GB RAM
Availability	`Regional (High availability)`	Recommended for production

Cost Impact: Regional HA adds ~$70/month over single zone. Critical for production (99.99% vs 99.95% SLA).

Storage

Field	Value	Notes
Storage type	`SSD`	Fast I/O
Storage capacity	`100 GB`	Sufficient for most apps
Automatic storage increases	✓ Enable	Prevent disk full issues
Storage autoincrease limit	`1000 GB`	Maximum auto-increase

Connectivity

Private IP

Field	Value	Notes
Private IP	✓ Enable	Critical - no public IP
Network	`dev-network`	Our VPC
IP allocation	`Use automatically allocated IP`	Easier than manual

Associated allocation:

Field	Value
Allocation	`google-managed-services-dev-network`

CRITICAL - No Public IP: Do NOT enable public IP. Private IP with PSA is more secure.

Ensure: Public IP checkbox is unchecked.

Security

Automatic Backups

Field	Value	Notes
Automated backups	✓ Enable	Required for production
Time	`3:00 AM`	Low-traffic window
Retention	`7 days`	Standard retention

Point-in-Time Recovery

Field	Value	Notes
Point-in-time recovery	✓ Enable	Allows restore to any point in time

Why PITR: If you accidentally delete data, you can restore to any second within the retention period.

Deletion Protection

Field	Value	Notes
Deletion protection	✓ Enable	CRITICAL - prevents accidental deletion

Why Deletion Protection: Prevents accidental data loss. You must disable this before deleting the instance.

IAM Authentication

Field	Value	Notes
IAM authentication	✓ Enable	Passwordless auth via IAM

Why IAM Auth: Backend VMs can connect using service account (no passwords in secrets).

Force SSL

Field	Value	Notes
Force SSL	✓ Enable	Encrypt database connections

Note: SSL is optional for private IP (traffic stays within Google's network), but recommended for defense in depth.

Customize Your Instance

Database Flags

Click "Add flag" to add performance tuning flags:

Flag 1: Max Connections

Field	Value
Flag name	`max_connections`
Value	`100`

Flag 2: Shared Buffers

Field	Value
Flag name	`shared_buffers`
Value	`256MB`

Flag 3: Effective Cache Size

Field	Value
Flag name	`effective_cache_size`
Value	`1GB`

Flag 4: IAM Authentication

Field	Value
Flag name	`cloudsql.iam_authentication`
Value	`on`

Maintenance

Field	Value	Notes
Preferred window	`Sunday 3:00 AM`	Low-traffic window
Maintenance channel	`Preview channel`	Get new features early (optional)

Create the Instance

Review all settings and click "Create instance".

Provisioning Time: 10-15 minutes

You can monitor progress in the SQL instances list.

Verify Cloud SQL Creation

Once ready, you should see:

Instance ID: app-dev-db
Status: (Checkmark) - Runnable
Region: europe-west1
IP addresses: Private IP assigned (e.g., 10.100.0.2)

Copy the private IP - we'll need it for the secret update in Step 5.

Step 3: Create Database

What is a Database?

A Cloud SQL instance can host multiple databases. We'll create one for our application.

Navigation Path

Click on app-dev-db instance
Click the "Databases" tab
Click "Create database"

Database Configuration

Field	Value	Notes
Database name	`appdb`	Our application database
Charset	(Default)	UTF-8
Collation	(Default)	Default collation

Click "Create".

Verify Database

You should see appdb in the databases list:

Database name: appdb
Size: ~8MB (empty database)

Step 4: Create IAM Database User

What is IAM Authentication?

IAM authentication allows VMs to connect to Cloud SQL using their service account (no passwords needed).

Navigation Path

Still on app-dev-db instance page
Click the "Users" tab
Click "Add user account"

User Configuration

Field	Value	Notes
User type	`IAM service account`	Use IAM, not password
Service account	`backend-dev-sa@PROJECT_ID.iam.gserviceaccount.com`	Our backend SA

Click "Add".

Verify IAM User

You should see:

Username: backend-dev-sa@PROJECT_ID
Type: IAM service account

Why IAM User: Backend VMs can now connect without storing passwords in secrets. They use their service account identity.

Step 5: Update Secret with Cloud SQL Connection Details

Why Update the Secret?

Now that Cloud SQL has a private IP, we need to update the db-credentials-dev secret with the correct connection details.

Navigation Path

Navigate to Security → Secret Manager
Click on db-credentials-dev
Click "Create new version" at the top

Update Secret Value

Replace the secret value with the updated JSON:

{
  "username": "backend-dev-sa",
  "password": "use-iam-authentication",
  "host": "10.100.0.2",
  "database": "appdb",
  "port": "5432"
}

Replace host value: Use the private IP you copied from Cloud SQL (Step 2).

Note on credentials:

username: Uses IAM service account name
password: Not needed with IAM auth (placeholder)
host: Cloud SQL private IP from PSA range

Click "Create new version".

Verify Secret Update

You should see 2 versions:

Version 1: Original (with placeholder host)
Version 2: Updated (with Cloud SQL private IP)

Step 6: Create Backend Instance Template

What is an Instance Template?

An instance template defines VM configuration for a Managed Instance Group (MIG). It includes:

Machine type
Boot disk image
Startup script
Service account
Network settings

Why Templates: Ensure all VMs in the MIG have identical configuration.

Navigation Path

Navigate to Compute Engine → Instance templates
Click "Create instance template"

Template Configuration

Basic Settings

Field	Value	Notes
Name	`dev-backend-template`	Descriptive
Machine type	`e2-medium`	2 vCPU, 4GB RAM
Boot disk type	`pd-balanced`	Balance cost/performance
Boot disk size	`20 GB`	Sufficient for app

Boot Disk

Click "Change":

Field	Value
OS	`Ubuntu`
Version	`Ubuntu 22.04 LTS Minimal`
Disk type	`pd-balanced`
Size	`20 GB`

Network Interface

Field	Value	Notes
Network	`dev-network`	Our VPC
Subnetwork	`private-backend`	Private subnet
Network interface type	`IPv4`	IPv4 only
External IPv4 address	None	No public IP!

CRITICAL - No Public IP: Backend VMs must NOT have public IP. They access internet via Cloud NAT.

Service Account

Field	Value
Service account	`backend-dev-sa`

Metadata - Startup Script

This script runs automatically when the VM starts. It:

Installs dependencies (nginx, Node.js)
Clones your application
Fetches secrets from Secret Manager
Starts the application with PM2

Add Metadata Item

Expand "Metadata" section:

Click "Add item":

Key	Value
`startup-script`	(See script below)

Startup Script:

#!/bin/bash
# Backend VM Startup Script

set -e  # Exit on error

# Logging
exec > >(tee /var/log/startup-script.log)
exec 2>&1

echo "=== Backend Startup Script Begin $(date) ==="

# Install dependencies
apt-get update
apt-get install -y nginx git curl wget

# Install Node Exporter for metrics
NODE_EXPORTER_VERSION="1.6.1"
echo "Installing Node Exporter ${NODE_EXPORTER_VERSION}..."

useradd --no-create-home --shell /bin/false node_exporter || true

wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz" -O /tmp/node_exporter.tar.gz
tar xzf /tmp/node_exporter.tar.gz -C /tmp
cp /tmp/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter /usr/local/bin/
chown node_exporter:node_exporter /usr/local/bin/node_exporter
chmod +x /usr/local/bin/node_exporter
rm -rf /tmp/node_exporter*

# Create systemd service for Node Exporter
cat > /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
Type=simple
User=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9100
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable node_exporter
systemctl start node_exporter

echo "Node Exporter running on port 9100"

# Install nvm and Node.js
if [ ! -d "$HOME/.nvm" ]; then
  curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
fi

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

nvm install 20
nvm use 20
nvm alias default 20

# Clone application (replace with your repo)
if [ ! -d "/opt/app" ]; then
  echo "Cloning application..."
  git clone https://github.com/your-org/your-repo.git /opt/app
  cd /opt/app
  npm install
else
  cd /opt/app
  git pull
  npm install
fi

# Fetch secrets from Secret Manager
echo "Fetching secrets..."
DB_CREDENTIALS=$(gcloud secrets versions access latest --secret="db-credentials-dev")
API_KEY=$(gcloud secrets versions access latest --secret="api-key-dev")

# Create .env file
cat > /opt/app/.env <<'EOF'
DATABASE_URL=${DB_CREDENTIALS}
API_KEY=${API_KEY}
PORT=3000
NODE_ENV=production
EOF

# Configure nginx as reverse proxy
cat > /etc/nginx/sites-available/default <<'EOF'
server {
  listen 80;
  server_name _;

  location / {
    proxy_pass http://localhost:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_cache_bypass $http_upgrade;
  }
}
EOF

systemctl restart nginx

# Install and start PM2
npm install -g pm2
cd /opt/app
pm2 delete nestjs-app 2>/dev/null || true
pm2 start npm --name "nestjs-app" -- start
pm2 save
pm2 startup systemd

echo "=== Startup Script Complete $(date) ==="
echo "Application running on port 3000"

Modify the script: Replace git clone URL with your actual repository.

Management - Automation

Field	Value	Notes
Enable autohealing	(Configure in MIG)	Health check based

Create the Template

Click "Create".

Creation Time: 1-2 minutes

Step 7: Create Managed Instance Group (MIG)

What is a MIG?

A Managed Instance Group (MIG):

Ensures a specified number of VMs are running
Auto-heals unhealthy VMs
Auto-scales based on CPU/load
Performs rolling updates

Navigation Path

Navigate to Compute Engine → Instance groups
Click "Create instance group"

Group Configuration

Group Type

Field	Value	Notes
Group type	`Regional MIG`	High availability across zones
Name	`dev-backend-mig`	Descriptive
Region	`europe-west1`	Same as VPC

Why Regional MIG: Spreads VMs across multiple zones for HA. If one zone fails, VMs in other zones continue serving traffic.

Instance Template

Field	Value
Instance template	`dev-backend-template`

Group Size

Field	Value	Notes
Group size	`2`	Minimum for HA

Autoscaling

Expand "Autoscaling policy":

Field	Value	Notes
Autoscaling mode	`On`	Enable autoscaling
Minimum instances	`2`	Always have 2 VMs
Maximum instances	`4`	Scale up to 4 VMs
Autoscaling metric	`CPU utilization`	Scale based on CPU
Target CPU utilization	`70%`	Scale out when CPU > 70%
Cooldown period	`300` seconds	Wait 5 min between scaling

Autohealing

Expand "Autohealing policy":

Field	Value	Notes
Health check	`Create health check`	Click link to create

Health Check Configuration

Field	Value	Notes
Name	`dev-backend-http-health-check`	Descriptive
Protocol	`HTTP`	HTTP health check
Port	`3000`	NestJS app port
Request path	`/health`	Your app must implement this
Check interval	`30` seconds	How often to check
Timeout	`10` seconds	Response timeout
Healthy threshold	`1` consecutive success	Mark healthy after 1 success
Unhealthy threshold	`3` consecutive failures	Mark unhealthy after 3 failures

CRITICAL - /health endpoint: Your NestJS application MUST implement a /health endpoint that returns HTTP 200. Without this, health checks will fail and VMs will be constantly recreated.

Back to MIG Autohealing:

Field	Value
Initial delay	`300` seconds

Why 5 min delay: The startup script takes time (nvm install, npm install can take 3-5 minutes). If health checks start too early, VMs will be marked unhealthy and recreated (infinite loop).

Create the MIG

Click "Create".

VM Creation Time: 5-10 minutes per VM

Verify MIG Creation

You should see:

Group name: dev-backend-mig
Status: Running
Instances: 2/2 (healthy)
Autoscaling: Enabled (2-4 VMs)

Click on the MIG to see individual instances:

dev-backend-xxxxx (europe-west1-b)
dev-backend-yyyyy (europe-west1-c)

Zones: Regional MIG spreads VMs across zones for HA.

Step 8: Create Cache VM (Redis + PgBouncer)

What is the Cache VM?

A single VM running:

Redis: In-memory cache for session data, query results
PgBouncer: Connection pooler for Cloud SQL (reduces connections)

Why combine: Cost optimization. A single VM handles both services (~$23/month vs ~$46/month for separate VMs).

Navigation Path

Navigate to Compute Engine → VM instances
Click "Create instance"

VM Configuration

Basic Settings

Field	Value	Notes
Name	`dev-cache-vm`	Descriptive
Region	`europe-west1`	Same as VPC
Zone	`europe-west1-b`	Zone b

Machine Type

Field	Value	Notes
Machine type	`e2-medium`	2 vCPU, 4GB RAM

Boot Disk

Field	Value
OS	`Ubuntu 22.04 LTS Minimal`
Disk type	`pd-balanced`
Size	`20 GB`

Network Interface

Field	Value	Notes
Network	`dev-network`	Our VPC
Subnetwork	`private-cache`	Cache subnet
External IPv4 address	None	No public IP

Service Account

Field	Value
Service account	`cache-dev-sa`

Metadata - Startup Scripts

We'll add 3 metadata items for Redis, PgBouncer, and observability agents.

Metadata Item 1: Redis Startup Script

Key: redis-startup-script

Value:

#!/bin/bash
# Redis Startup Script

set -e

echo "Configuring Redis..."

# Install Redis
apt-get update
apt-get install -y redis-server

# Configure Redis to listen on all interfaces
sed -i 's/^bind 127.0.0.1/bind 0.0.0.0/' /etc/redis/redis.conf

# Set maxmemory policy
sed -i 's/^# maxmemory/maxmemory/' /etc/redis/redis.conf
sed -i 's/^maxmemory .*/maxmemory 512mb/' /etc/redis/redis.conf

# Set eviction policy
sed -i 's/^# maxmemory-policy/maxmemory-policy/' /etc/redis/redis.conf
sed -i 's/^maxmemory-policy .*/maxmemory-policy allkeys-lru/' /etc/redis/redis.conf

# Restart Redis
systemctl restart redis-server

# Enable Redis on boot
systemctl enable redis-server

echo "Redis configured and running on port 6379"

Metadata Item 2: PgBouncer Startup Script

Key: pgbouncer-startup-script

Value:

#!/bin/bash
# PgBouncer Startup Script

set -e

# Replace with your Cloud SQL private IP
CLOUD_SQL_IP="10.100.0.2"  # From Step 2

echo "Configuring PgBouncer for Cloud SQL at ${CLOUD_SQL_IP}..."

# Install PgBouncer
apt-get update
apt-get install -y pgbouncer

# Configure PgBouncer
cat > /etc/pgbouncer/pgbouncer.ini << EOF
[databases]
appdb = host=${CLOUD_SQL_IP} port=5432 dbname=appdb

[pgbouncer]
pool_mode = transaction
max_client_conn = 100
default_pool_size = 50
reserve_pool = 5
reserve_pool_timeout = 3
listen_port = 6432
listen_addr = 0.0.0.0
auth_type = any
EOF

# Create user list (for auth_type=any)
# In production, use proper authentication
echo "\"app_admin\" \"any_password\"" > /etc/pgbouncer/userlist.txt

# Restart PgBouncer
systemctl restart pgbouncer

# Enable PgBouncer on boot
systemctl enable pgbouncer

echo "PgBouncer configured and running on port 6432"

Replace CLOUD_SQL_IP: Use the private IP from Step 2.

Metadata Item 3: Observability Agents

Key: observability-agents-script

Value:

#!/bin/bash
# Observability Agents (Node Exporter + Promtail)

set -e

echo "Installing observability agents..."

# Install Node Exporter
NODE_EXPORTER_VERSION="1.6.1"

useradd --no-create-home --shell /bin/false node_exporter || true

wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz" -O /tmp/node_exporter.tar.gz
tar xzf /tmp/node_exporter.tar.gz -C /tmp
cp /tmp/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter /usr/local/bin/
chown node_exporter:node_exporter /usr/local/bin/node_exporter
chmod +x /usr/local/bin/node_exporter

cat > /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
Type=simple
User=node_exporter
ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9100
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable node_exporter
systemctl start node_exporter

echo "Node Exporter installed on port 9100"

# Install Promtail ( Loki URL will be updated in Part 4)
PROMTAIL_VERSION="2.9.0"
LOKI_URL="http://10.0.5.11:3100"  # Loki VM IP

useradd --no-create-home --shell /bin/false promtail || true

wget -q "https://github.com/grafana/loki/releases/download/v${PROMTAIL_VERSION}/promtail-linux-amd64.zip" -O /tmp/promtail.zip
unzip -o /tmp/promtail.zip -d /tmp
mv /tmp/promtail-linux-amd64/promtail /usr/local/bin/
chown promtail:promtail /usr/local/bin/promtail
chmod +x /usr/local/bin/promtail

mkdir -p /etc/promtail
cat > /etc/promtail/config.yml << EOF
server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: ${LOKI_URL}/loki/api/v1/push

scrape_configs:
  - job_name: cache
    static_configs:
      - targets:
          - localhost
        labels:
          job: cache
          env: dev
          host: $(hostname)
EOF

chown promtail:promtail /etc/promtail/config.yml

cat > /etc/systemd/system/promtail.service <<'EOF'
[Unit]
Description=Promtail Log Agent
After=network.target

[Service]
Type=simple
User=promtail
ExecStart=/usr/local/bin/promtail -config.file /etc/promtail/config.yml
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable promtail
systemctl start promtail

echo "Promtail installed and shipping logs to ${LOKI_URL}"

Create the VM

Click "Create".

VM Creation Time: 2-3 minutes

Verify Cache VM

You should see:

Name: dev-cache-vm
Status: Running
Internal IP: 10.0.4.x
Zone: europe-west1-b

Test Redis Connectivity

From the bastion host:

# SSH to bastion
gcloud compute ssh dev-bastion --tunnel-through-iap

# From bastion, SSH to cache VM (via internal IP)
ssh 10.0.4.2

# Test Redis
redis-cli PING
# Expected: PONG

# Test PgBouncer
psql -h 127.0.0.1 -p 6432 -U app_admin -d appdb
# Expected: psql (16.x) connection

Part 3 Verification Checklist

Before moving to Part 4, verify:

[ ] Private Service Connection is active (10.100.0.0/16)
[ ] Cloud SQL instance is running with private IP
[ ] Database "appdb" exists
[ ] IAM user created for backend-dev-sa
[ ] Secret updated with Cloud SQL private IP
[ ] Backend MIG has 2 running VMs (healthy)
[ ] Autohealing health check is configured
[ ] Cache VM is running
[ ] Redis is listening on port 6379
[ ] PgBouncer is listening on port 6432
[ ] Firewall rule allows backend to cache (ports 6379, 6432)

Cost Summary - Part 3

Component	Monthly Cost	Notes
Cloud SQL (regional HA)	~$115	db-n1-standard-2
Backend MIG (2-4 VMs)	~$46-92	e2-medium, autoscaling 2-4
Cache VM	~$23	e2-medium
Total Part 3	~$184-230	~$184/month (2 VMs)

Cumulative Cost (Part 1 + 2 + 3):

Component	Monthly Cost
Part 1 (VPC, NAT, etc.)	~$42
Part 2 (Bastion)	~$11
Part 3 (Cloud SQL, MIG, Cache)	~$184
Total	~$237/month

Troubleshooting - Part 3

Issue: Private Service Connection Fails

Symptom: "Allocation failed" error

Solution:

Verify 10.100.0.0/16 doesn't overlap with VPC
Check VPC peering status
Ensure you're in the correct project

# Check PSA status
gcloud compute networks peerings list \
  --network=dev-network

Issue: Cloud SQL Creation Fails

Symptom: "Instance creation failed"

Solution:

Verify Private Service Connection is active
Check subnet has Private Google Access enabled
Ensure sufficient quota (SQL instances)

Issue: Backend MIG VMs Unhealthy

Symptom: 0/2 VMs healthy

Solution:

Check health check configuration (/health endpoint)
Increase initial delay to 300+ seconds
Increase unhealthy threshold to 5
Check VM serial port logs for startup script errors

# View VM logs
gcloud compute instances get-serial-port-output INSTANCE_NAME \
  --zone=europe-west1-b --port=1

Issue: Cache VM Cannot Connect to Cloud SQL

Symptom: PgBouncer connection timeout

Solution:

Verify Cloud SQL private IP is correct
Check firewall allows cache VM to Cloud SQL
Ensure Private Service Connection is active
Verify PgBouncer configuration

# From cache VM, test Cloud SQL connectivity
ping 10.100.0.2
telnet 10.100.0.2 5432

What's Next - Part 4?

In Part 4: Observability & Load Balancer, you'll build:

Prometheus VM for metrics collection
Loki VM for log aggregation
Grafana dashboards for visualization
External Application Load Balancer
End-to-end testing

Continue to Part 4: Observability & Load Balancer →

References

Data and Compute Layers Complete! Your Cloud SQL database, backend MIG, and cache VM are ready. Next, we'll add observability (Prometheus, Loki, Grafana) and the load balancer for external access.

Build Production-Ready GCP Infrastructure from Scratch Part 02

Shaikh Al Amin — Wed, 04 Feb 2026 12:45:14 +0000

Build Production-Ready GCP Infrastructure from Scratch: A Complete Console Guide

A 4-Part Series for Complete Beginners

Part 2: Security Services - Secrets, Bastion & IAM

Overview

In this part, you'll build the security layer for your infrastructure. We'll create secrets for sensitive data, set up a bastion host for secure SSH access, configure IAP (Identity-Aware Proxy) for zero-trust access, and establish IAM permissions.

What you'll build:

Secret Manager secrets for database credentials and API keys
Bastion host with IAP tunneling
OS Login for IAM-managed SSH authentication
IAM roles for service account and user access

Estimated time: 30-45 minutes

Estimated cost: ~$11/month (Bastion VM only)

Cumulative cost: ~$53/month

Prerequisites

Before continuing, ensure you've completed Part 1:

[ ] VPC dev-network exists
[ ] 5 subnets created (including public-subnet)
[ ] Cloud NAT gateway is running
[ ] 4 service accounts created (including bastion-dev-sa)
[ ] Firewall rules created

If you missed Part 1: Start with Part 1: Foundation →

Step 1: Create Secret Manager Secrets

What is Secret Manager?

Secret Manager is Google Cloud's secure storage for sensitive data:

Database credentials
API keys
Certificates
Tokens

Secrets are encrypted at rest and accessed via IAM permissions.

Why Not Hardcode Secrets?

Security Risk: Hardcoding secrets in:

Configuration files (committed to Git)
Startup scripts (visible to anyone with VM access)
Environment variables (visible in process lists)

Best Practice: Store secrets in Secret Manager and fetch at runtime.

Cost: Secret Manager is free for active secrets. Storage costs ~$0.03/GB after the free tier.

Secret 1: Database Credentials

Navigation Path

Navigate to Security → Secret Manager
Click "Create Secret"

Secret Configuration

Basic Settings:

Field	Value	Notes
Name	`db-credentials-dev`	Environment-suffixed
Secret value	(See JSON below)	Click "Create or upload"

Secret Value (JSON format):

{
  "username": "app_admin",
  "password": "REPLACE_WITH_SECURE_PASSWORD",
  "host": "10.100.0.2",
  "database": "appdb",
  "port": "5432"
}

Note: The host IP will be updated after we create Cloud SQL in Part 3. For now, use the placeholder.

Password Generation:

# Generate a secure password
openssl rand -base64 32

Replication:

Field	Value	Notes
Replication	`Automatic`	Replicates across regions

Rotation:

Field	Value	Notes
Rotation period	Leave blank	No auto-rotation for now

Version Access:

Field	Value	Notes
Version access	`Enable secret version`	Access immediately

Click "Create secret".

Secret 2: API Key

Create API Key Secret

In Secret Manager, click "Create Secret"

Field	Value
Name	`api-key-dev`
Secret value	`your-api-key-here`

Replace with actual value: Use your actual API key or generate a placeholder for testing.

Replication: Automatic (same as above)

Click "Create secret".

Verify Secrets Created

You should see 2 secrets in Secret Manager:

Secret Name	Created	Versions
db-credentials-dev	(timestamp)	1
api-key-dev	(timestamp)	1

Step 2: Grant Secret Access to Service Accounts

What is IAM Access to Secrets?

Service accounts need permission to access secrets. Without this, VMs cannot fetch secrets at runtime.

Grant Access to Backend SA

Click on db-credentials-dev secret
Click the "Permissions" tab
Click "Grant access"

Add Principal:

Field	Value
New principals	`backend-dev-sa@PROJECT_ID.iam.gserviceaccount.com`

Select Role:

Role	Purpose
`Secret Manager Secret Accessor`	Can read secret values

Click "Save".

Grant Access to Cache and Observability SAs

Repeat for:

api-key-dev secret → Grant access to backend-dev-sa
db-credentials-dev secret → Grant access to cache-dev-sa (for PgBouncer)
api-key-dev secret → Grant access to observability-dev-sa (if needed)

Verify IAM Bindings

For db-credentials-dev, you should see:

Principal	Role
backend-dev-sa@PROJECT_ID	Secret Manager Secret Accessor
cache-dev-sa@PROJECT_ID	Secret Manager Secret Accessor

Step 3: Enable IAP (Identity-Aware Proxy)

What is IAP?

IAP provides zero-trust access to your VMs without:

Public IPs
VPN connections
SSH keys in metadata

How it works:

User authenticates with Google Cloud
IAP establishes a secure tunnel
Traffic flows through Google's network (not public internet)

Why IAP is more secure: SSH traffic never traverses the public internet. Access is controlled by IAM, not SSH keys.

Navigation Path

Navigate to Security → Identity-Aware Proxy
Click "Go to Identity-Aware Proxy"
Click "Tick the checkbox" to enable IAP

Configure IAP for SSH

Click "Configure SSH and TCP Resources"
Click "Enable IAP"

SSH and TCP forwarding: ✓ Enable

Note: IAP for TCP forwarding is required for SSH tunneling.

Verify IAP Enabled

You should see:

Status: Enabled
Resources: (None yet - bastion will be added)

Step 4: Create IAP Firewall Rule

What is the IAP Firewall Rule?

This rule allows IAP's infrastructure to connect to your VM. IAP uses IP range 35.235.240.0/20 for tunneling.

Note: If using the Terraform bastion module, this rule is created automatically. For console setup, we create it manually.

Navigation Path

Navigate to VPC networks → Firewall
Click "Create firewall rule"

Firewall Configuration

Field	Value	Notes
Name	`dev-firewall-allow-iap`	Descriptive
Network	`dev-network`	Our VPC
Priority	`1000`	Standard allow priority
Direction	`Ingress`	Inbound traffic
Action	`Allow`	Allow traffic
Target service accounts	`bastion-dev-sa`	Only bastion host

Source filter:

Field	Value
Source IPv4 ranges	`35.235.240.0/20`

Why this range: This is Google's IAP forwarding range. Without this rule, IAP cannot tunnel to your VM.

Protocols and ports:

Field	Value
TCP	✓ Checked
Ports	`22`

Logging:

Field	Value
Log config	`On`

Click "Create".

Step 5: Create Bastion Host VM

What is a Bastion Host?

A bastion host is a secure entry point to your private network. It's the only VM with:

Public IP (optional)
SSH access enabled
Access to private subnets

Security Model:

Users SSH to bastion via IAP
From bastion, SSH to backend VMs
Backend VMs have no public IP

Cost Alert: Bastion VM costs ~$11/month (e2-small). You can stop it when not in use to save costs.

Navigation Path

Navigate to Compute Engine → VM instances
Click "Create instance"

Basic Settings

Field	Value	Notes
Name	`dev-bastion`	Environment-prefixed
Region	`europe-west1`	Belgium
Zone	`europe-west1-b`	Zone b

Machine Configuration

Field	Value	Notes
Machine type	`e2-small`	2 vCPU, 2GB RAM
CPU platform	`Intel/AMD`	Default (Automatic)

Cost: e2-small ≈ $11/month. Sufficient for bastion (just forwards SSH).

Boot Disk

Click "Change" to configure:

Field	Value	Notes
OS	`Ubuntu`	Popular Linux distribution
Version	`Ubuntu 22.04 LTS Minimal`	Long-term support
Disk type	`pd-balanced`	Balance cost/performance
Size	`20 GB`	Sufficient for bastion

Click "Select".

Network Interface

Network settings:

Field	Value	Notes
Network	`dev-network`	Our VPC
Subnetwork	`public-subnet`	Must be public subnet
Network interface type	`IPv4`	IPv4 only

External IPv4 address:

Field	Value	Notes
Network Service Tiers	`Premium`	Default (recommended)
External IPv4 address	`Ephemeral`	Do NOT create static IP

Why ephemeral IP: Bastion uses IAP for access, so public IP is not directly accessed. Save money by using ephemeral IP.

Network Tags

Add network tag:

Field	Value
Network tags	`bastion-host`

Note: Network tags are used for the IP whitelist backup firewall rule (IAP is primary).

Identity and API Access

Service account:

Field	Value	Notes
Service account	`bastion-dev-sa`	Our bastion SA

Access scopes:

Field	Value
Access scopes	`Allow full access to all Cloud APIs`

Why full access: Bastion needs to access Secret Manager (for credentials) and other services. In production, restrict to specific scopes.

Security - OS Login

Expand "Security" section:

Field	Value	Notes
Enable OS Login	✓ Enable	Critical for IAM-based SSH

What is OS Login: OS Login allows you to manage SSH access via IAM. No SSH key distribution needed. Users log in with their Google account.

Advanced Options - Metadata

No custom metadata needed for bastion (OS Login handles authentication).

Create the VM

Review all settings and click "Create".

Wait 2-3 minutes for VM creation. You should see:

Instance 'dev-bastion' is RUNNING

Step 6: Grant IAP Access to Users

What IAM Roles Are Needed?

Users need two roles to SSH via IAP:

IAP-secured Tunnel User - Permission to use IAP tunnel
Compute OS Login - Permission to log in to VMs

Note: These roles are granted at the project level, not per VM.

Grant IAP Tunnel Access

Navigate to IAM & Admin → IAM
Click "Grant access"

Add principal:

Field	Value
New principals	`your-email@example.com`

Select role:

Role	Path
`IAP-secured Tunnel User`	Identity-Aware Proxy → IAP-secured Tunnel User

Click "Save".

Grant OS Login Access

Click "Grant access" again:

Add principal: Same email as above

Select role:

Role	Path
`Compute OS Login`	Compute Engine → Compute OS Login

Click "Save".

Verify IAM Bindings

You should see your user with these roles:

Principal	Role
your-email@example.com	IAP-secured Tunnel User
your-email@example.com	Compute OS Login

Step 7: Add SSH Key for OS Login (Optional)

What is OS Login SSH Key?

OS Login can use IAM-managed SSH keys. These are:

Stored in Google Cloud
Automatically rotated
Managed via IAM

Alternative: You can add your own SSH key to VM metadata.

Best Practice: Use OS Login with IAM for production. No SSH key management overhead.

Add SSH Key to Project (Optional Backup)

If OS Login fails, you can use SSH keys:

Navigate to Compute Engine → Metadata
Click "SSH Keys" tab
Click "Edit"
Click "Add item"

Generate SSH key locally:

ssh-keygen -t rsa -f ~/.ssh/gcp_bastion -C your-email@example.com

Copy public key:

cat ~/.ssh/gcp_bastion.pub

Paste the public key into the SSH Keys field:

Field	Value
SSH key	`ssh-rsa AAAA... your-email@example.com`

Click "Save".

Security Note: Metadata SSH keys are less secure than OS Login. Use only as backup.

Part 2 Verification Checklist

Before moving to Part 3, verify:

[ ] 2 secrets created (db-credentials-dev, api-key-dev)
[ ] Backend, cache, and observability SAs have Secret Accessor role
[ ] IAP is enabled for TCP forwarding
[ ] IAP firewall rule created (allows 35.235.240.0/20)
[ ] Bastion VM is RUNNING
[ ] Bastion has bastion-dev-sa attached
[ ] OS Login is enabled on bastion
[ ] Your user has IAP-secured Tunnel User role
[ ] Your user has Compute OS Login role

Test SSH via IAP

Verify IAP Access

Test SSH connection to bastion:

gcloud compute ssh dev-bastion \
  --project=PROJECT_ID \
  --zone=europe-west1-b \
  --tunnel-through-iap

Expected output:

Welcome to Ubuntu 22.04 LTS
your-email@dev-bastion:~$

Troubleshooting: If SSH fails:

Verify IAP is enabled

Check IAP firewall rule exists

Verify your user has IAP-secured Tunnel User role

Check OS Login is enabled on bastion

Cost Summary - Part 2

Component	Monthly Cost	Notes
Bastion Host (e2-small)	~$11	24/7 operation
Secret Manager	Free	Within free tier
IAP	Free	No additional cost
Total Part 2	~$11	~$11/month

Cumulative Cost (Part 1 + 2):

Component	Monthly Cost
Part 1 (VPC, NAT, etc.)	~$42
Part 2 (Bastion)	~$11
Total	~$53/month

Cost Optimization: Stop bastion VM when not in use to save ~$11/month. Start it only when needed for SSH access.

Troubleshooting - Part 2

Issue: Cannot Access Secret

Symptom: "Permission denied" when accessing secret

Solution:

Check service account has Secret Manager Secret Accessor role
Verify secret name matches exactly (case-sensitive)
Check secret has at least 1 version

# List secret versions
gcloud secrets versions list db-credentials-dev

# Access secret
gcloud secrets versions access latest --secret=db-credentials-dev

Issue: IAP Connection Fails

Symptom: "IAP does not have permission" error

Solution:

Verify IAP is enabled
Check your user has IAP-secured Tunnel User role
Verify IAP firewall rule exists (allows 35.235.240.0/20)

# Check IAP status
gcloud compute ssh dev-bastion --tunnel-through-iap --dry-run

Issue: OS Login Not Working

Symptom: "OS Login is not enabled" error

Solution:

Verify OS Login is enabled on bastion VM
Check your user has Compute OS Login role
Ensure no conflicting SSH keys in metadata

# Enable OS Login on existing VM
gcloud compute instances add-metadata dev-bastion \
  --metadata enable-oslogin=TRUE

Issue: Bastion Cannot Access Secrets

Symptom: Secret access denied from bastion

Solution:

Verify bastion-dev-sa has Secret Manager Secret Accessor role
Check service account is attached to VM
Ensure VM has access scopes for Secret Manager

What's Next - Part 3?

In Part 3: Database & Compute, you'll build:

Private Service Connection for Cloud SQL
Cloud SQL PostgreSQL instance with HA
Managed Instance Group (MIG) for backend VMs
Cache VM with Redis and PgBouncer

Continue to Part 3: Database & Compute →

References

Security Layer Complete! Your secrets are securely stored, bastion host is ready for secure SSH access, and IAM permissions are configured. Next, we'll add the data and compute layers (Cloud SQL, MIG, Cache).

Build Production-Ready GCP Infrastructure from Scratch Part 01

Shaikh Al Amin — Wed, 04 Feb 2026 12:42:35 +0000

Build Production-Ready GCP Infrastructure from Scratch: A Complete Console Guide

A 4-Part Series for Complete Beginners

Part 1: Foundation - Project Setup, VPC & Networking

Overview

In this first part of our series, you'll build the networking foundation for a production-ready GCP infrastructure. We'll create a custom VPC with 5 subnets, configure Cloud NAT for internet access, set up service accounts, and implement service account-targeted firewall rules.

What you'll build:

Custom VPC (10.0.0.0/16) in europe-west1 region
5 subnets for different application tiers
Cloud NAT Gateway for outbound internet access
4 Service Accounts for IAM-based security
Firewall rules with service account targeting

Estimated time: 45-60 minutes

Estimated cost: ~$32/month (Cloud NAT only)

Prerequisites

Before we begin, ensure you have:

GCP Account - Sign up at console.cloud.google.com
Billing Account - Required for creating resources
Free Tier Credit - New accounts get $300 credit for 90 days
Required Permissions - Project Owner or Editor role

Cost Alert: This tutorial will incur charges. With the free tier credit, you can follow along for free. After that, expect ~$350/month total for all infrastructure.

Why GCP for Your Infrastructure?

EU Data Residency - Host in europe-west1 (Belgium) for compliance
Private Network - No public IPs needed for database and application servers
Managed Services - Focus on code, not infrastructure management
Built-in Security - IAM-based access control, VPC Service Controls

Step 1: Create or Select GCP Project

Navigation Path

Go to console.cloud.google.com
Click the project dropdown at the top (next to "Google Cloud Platform")
Click "New Project"

Project Configuration

Fill in the form:

Field	Value	Notes
Project name	`My GCP Infrastructure`	Display name
Project ID	`my-gcp-project-id`	Must be globally unique
Organization	(Your organization)	Leave blank for personal
Location	`No organization`	For personal projects

Pro Tip: Project IDs cannot be changed after creation. Choose carefully and avoid spaces.

Click "Create" and wait 1-2 minutes for project creation.

Verify Project Creation

You should see a notification: "Project 'My GCP Infrastructure' is ready."

Click "Select project" to switch to your new project.

Enable Required APIs

Some resources require specific APIs to be enabled:

Navigate to APIs & Services → Library
Search for and enable these APIs:
- Compute Engine API
- Cloud SQL API
- Secret Manager API
- Identity-Aware Proxy API

Why: Enabling APIs early prevents "API not enabled" errors during resource creation.

Step 2: Create Custom VPC (10.0.0.0/16)

What is a VPC?

A Virtual Private Cloud (VPC) is your isolated network in Google Cloud. Think of it as your own data center network in the cloud.

Navigation Path

From the main menu (hamburger icon ≡), navigate to: Networking → VPC networks

Click "Create VPC network"

VPC Configuration

Fill in the form:

Field	Value	Notes
Name	`dev-network`	Environment-prefixed for clarity
Description	`Development environment VPC`	Optional but recommended
Custom VPC creation	✓ Enable	CRITICAL: Uncheck "Automatic subnet creation"
Routing mode	`Regional`	Recommended for most cases
MTU	`1460`	Default (leave unchanged)

VPC Flow Logs Configuration

Scroll down to "VPC flow logs" section:

Field	Value	Notes
Off/On	On	Enable for security monitoring
Sampling	`50%`	Balance cost vs visibility
Aggregation interval	`5 sec`	Most granular option
Metadata	`Include all`	Full logging for forensics

Cost Alert: VPC Flow Logs cost ~$0.005/GB. With 50% sampling, expect ~$5-10/month for moderate traffic.

Why Flow Logs: Essential for troubleshooting network issues and security audits. They show all traffic flows through your VPC.

Dynamic Routing Mode

Leave as "Regional" (default).

Create the VPC

Click "Create" at the bottom.

Wait 30-60 seconds for creation. You should see:

VPC network "dev-network" was created successfully.

Verify VPC Creation

You should see dev-network in your VPC networks list with:

Subnets: 0 (we'll add these next)
Routing mode: Regional
Flow logs: On

Step 3: Create 5 Subnets (Sequential /24)

What are Subnets?

Subnets divide your VPC into smaller networks. Each subnet hosts a specific tier of your application:

Public subnet (load balancer, bastion)
Private subnets (backend, database, cache, observability)

Why europe-west1?

We're using europe-west1 (Belgium) for:

EU Data Residency - Data stays within Europe
Low Latency - Good for European users
Cost - Competitive pricing compared to other regions

Pro Tip: For production, consider multi-region deployment for disaster recovery.

Subnet Overview

Subnet Name	CIDR	Purpose
`public-subnet`	10.0.1.0/24	Bastion, Load Balancer
`private-backend`	10.0.2.0/24	Backend VMs (NestJS app)
`private-data`	10.0.3.0/24	Cloud SQL PostgreSQL
`private-cache`	10.0.4.0/24	Redis + PgBouncer
`private-obs`	10.0.5.0/24	Prometheus, Loki, Grafana

Why /24: Provides 251 usable IP addresses per subnet (sufficient for most applications).

Create Each Subnet

We'll create all 5 subnets. The process is identical for each, just with different values.

Subnet 1: public-subnet (10.0.1.0/24)

Click on dev-network in your VPC list
Click the "SUBNETS" tab
Click "Create subnet"

Fill in the form:

Field	Value	Notes
Name	`public-subnet`	Descriptive name
Description	`Public subnet for bastion and load balancer`	Documentation
Region	`europe-west1`	Belgium
IP address range	`10.0.1.0/24`	Manual CIDR
Private Google Access	✓ Enable	CRITICAL - see below

CRITICAL - Private Google Access: This allows VMs without public IPs to reach Google services (Cloud SQL, Secret Manager, etc.). Without this, private VMs cannot access GCP services!

Flow Logs for Subnet 1

Scroll down to "Flow logs" section:

Field	Value	Notes
Off/On	On	Same as VPC
Sampling	`50%`	Consistent with VPC
Aggregation interval	`5 sec`	Consistent with VPC
Metadata	`Include all`	Consistent with VPC

Click "Add" to create the subnet.

Subnet 2: private-backend (10.0.2.0/24)

Repeat the process with these values:

Field	Value
Name	`private-backend`
Region	`europe-west1`
IP address range	`10.0.2.0/24`
Private Google Access	✓ Enable
Flow logs	On (same settings)

Subnet 3: private-data (10.0.3.0/24)

Field	Value
Name	`private-data`
Region	`europe-west1`
IP address range	`10.0.3.0/24`
Private Google Access	✓ Enable
Flow logs	On

Why separate subnet: Isolates database layer for security. Database VMs have different firewall rules than backend VMs.

Subnet 4: private-cache (10.0.4.0/24)

Field	Value
Name	`private-cache`
Region	`europe-west1`
IP address range	`10.0.4.0/24`
Private Google Access	✓ Enable
Flow logs	On

Subnet 5: private-obs (10.0.5.0/24)

Field	Value
Name	`private-obs`
Region	`europe-west1`
IP address range	`10.0.5.0/24`
Private Google Access	✓ Enable
Flow logs	On

Verify All Subnets

You should now see 5 subnets under dev-network:

Subnet	Region	CIDR	Private IP Access	Flow Logs
public-subnet	europe-west1	10.0.1.0/24	On	On
private-backend	europe-west1	10.0.2.0/24	On	On
private-data	europe-west1	10.0.3.0/24	On	On
private-cache	europe-west1	10.0.4.0/24	On	On
private-obs	europe-west1	10.0.5.0/24	On	On

Step 4: Create Cloud Router

What is a Cloud Router?

Cloud Router manages dynamic routing for your Cloud NAT Gateway. It's required for NAT functionality.

Navigation Path

Navigate to Hybrid Connectivity → Cloud Routers
Click "Create Router"

Router Configuration

Field	Value	Notes
Name	`dev-nat-router`	Descriptive name
Network	`dev-network`	Our VPC
Region	`europe-west1`	Same as subnets
Google ASN	`65000`	Default (leave unchanged)

Why ASN: Autonomous System Number uniquely identifies your router. 65000 is the default for private networks.

Click "Create" and wait for creation (should be instant).

Step 5: Create Cloud NAT Gateway

What is Cloud NAT?

Cloud NAT allows VMs without public IPs to access the internet. This is essential for:

Downloading packages (apt, npm)
Calling external APIs
Software updates

Navigation Path

Navigate to Network Services → Cloud NAT
Click "Get started" or "Create Cloud NAT Gateway"

Gateway Configuration

Basic Settings

Field	Value	Notes
Gateway name	`dev-nat-gateway`	Descriptive
Cloud Router	`dev-nat-router`	Router we created
Region	`europe-west1`	Same region

NAT Mapping (Critical Section)

Field	Value	Notes
Custom single region NAT	Select	For single region
Cloud Router	`dev-nat-router`	Already populated
Source subnetworks	✓ Select all 5 subnets	All subnets need internet
NAT IP allocation	Manual	Critical for HA

Why Manual IP Allocation: Gives us control over outbound IPs. We'll create 2 IPs for high availability.

NAT IP Allocation

Click "Add reserved IPs" twice:

IP 1:
| Field | Value |
|-------|-------|
| Name | dev-nat-ip-1 |
| Static IP address | (Leave blank - auto-allocate) |

IP 2:
| Field | Value |
|-------|-------|
| Name | dev-nat-ip-2 |
| Static IP address | (Leave blank - auto-allocate) |

Why 2 IPs: High availability. If one IP fails, the other handles traffic.

Cost Alert: ~$0.045/hr per NAT gateway = ~$32/month. Static IPs are free when attached to NAT.

Logging

Leave "Log and monitor" disabled (not needed for NAT).

Create the NAT Gateway

Click "Create" and wait 1-2 minutes.

Verify Cloud NAT

You should see:

Status: Running
Cloud Router: dev-nat-router
External IPs: 2 IPs assigned
Subnets: All 5 subnets mapped

Step 6: Create Service Accounts

What are Service Accounts?

Service accounts are identities for applications (not humans). They allow VMs to authenticate with Google Cloud services.

Why Service Account Targeting?

Security Best Practice: Instead of using network tags (which can be spoofed), we use service accounts to target firewall rules. This provides IAM-based security.

Why SA-targeted is more secure: If someone compromises your VM, they can't change network tags to bypass firewall rules. Service account targeting is enforced at the IAM level.

Create Service Accounts

We'll create 4 service accounts for different tiers.

Service Account 1: Backend SA

Navigate to IAM & Admin → Service Accounts
Click "Create Service Account"

Fill in the form:

Field	Value	Notes
Service account name	`backend-dev-sa`	Tier-environment pattern
Service account description	`Service account for backend VMs`	Documentation
Service account ID	(Auto-filled)	`backend-dev-sa@PROJECT_ID.iam`

Click "Create and continue".

Skip the "Grant this service account access to project" step (we'll add roles later).

Click "Done".

Service Account 2: Cache SA

Repeat the process:

Field	Value
Name	`cache-dev-sa`
Description	`Service account for Redis/PgBouncer VM`

Service Account 3: Observability SA

Field	Value
Name	`observability-dev-sa`
Description	`Service account for monitoring tools`

Service Account 4: Bastion SA

Field	Value
Name	`bastion-dev-sa`
Description	`Service account for bastion host`

Add IAM Roles to Service Accounts

Now we'll add roles to allow VMs to write logs and metrics.

For Backend, Cache, and Observability SAs:

Click on the service account (e.g., backend-dev-sa)
Click the "Permissions" tab
Click "Grant access"

Add these roles:

Role	Purpose
`Logs Writer`	Allow VMs to write to Cloud Logging
`Monitoring Metric Writer`	Allow VMs to send metrics to Cloud Monitoring

Add Principal: backend-dev-sa@PROJECT_ID.iam.gserviceaccount.com

Select Role: Logs Writer → Monitoring Metric Writer

Repeat for cache-dev-sa and observability-dev-sa.

For Bastion SA:

Bastion only needs Logs Writer role.

Step 7: Create Firewall Rules (Service Account Targeted)

Firewall Rule Priority

GCP evaluates firewall rules by priority number:

Lower numbers = Higher priority (evaluated first)
Deny rules typically use priority 500
Allow rules typically use priority 1000+

Why this order: Deny rules (higher priority) are evaluated before allow rules. This ensures we can block specific traffic before allowing general traffic.

Rule 1: Deny SMTP Egress (Priority 500)

Purpose: Prevent compromised VMs from sending spam.

Navigate to VPC networks → Firewall
Click "Create firewall rule"

Fill in the form:

Field	Value	Notes
Name	`dev-firewall-deny-smtp-egress`	Descriptive
Network	`dev-network`	Our VPC
Priority	`500`	High priority (deny)
Direction of traffic	`Egress`	Outbound
Action on match	`Deny`	Block traffic
Target service accounts	Select all 4 SAs	Apply to all private VMs

Filters section:
| Field | Value |
|-------|-------|
| Destination filter | IPv4 range |
| IPv4 range | 0.0.0.0/0 | All destinations |

Protocols and ports:
| Field | Value |
|-------|-------|
| Allow protocols | Specified protocols and ports |
| TCP | ✓ Checked |
| Ports | 25 | SMTP port |

Logging:
| Field | Value |
|-------|-------|
| Log config | On | Enable for security |
| Metadata | Include all | Full logging |

Click "Create".

Why log deny rules: Detect potential compromise. If a VM tries to send SMTP traffic, it's suspicious.

Rule 2: Allow SSH to Backend (Priority 1000)

Purpose: Allow SSH access to backend VMs for troubleshooting.

Field	Value
Name	`dev-firewall-allow-ssh-backend`
Network	`dev-network`
Priority	`1000`
Direction	`Ingress`
Action	`Allow`
Target service accounts	`backend-dev-sa`

Source filter:
| Field | Value |
|-------|-------|
| Source IPv4 ranges | 0.0.0.0/0 | All sources (restrict in prod!) |

Security Warning: In production, restrict SSH to specific IPs or VPN ranges.

Protocols and ports:
| Field | Value |
|-------|-------|
| TCP | ✓ Checked |
| Ports | 22 | SSH |

Logging:
| Field | Value |
|-------|-------|
| Log config | On | Critical - log SSH access |
| Metadata | Include all | Full metadata |

Click "Create".

Rule 3: Allow Internal HTTP (Priority 1001)

Purpose: Allow internal HTTP/HTTPS traffic between services (load balancer → backend).

Field	Value
Name	`dev-firewall-allow-http-backend`
Network	`dev-network`
Priority	`1001`
Direction	`Ingress`
Action	`Allow`
Target service accounts	`backend-dev-sa`

Source filter:
| Field | Value |
|-------|-------|
| Source IPv4 ranges | 10.0.0.0/8 | Internal VPC traffic only |

Why 10.0.0.0/8: Covers all our subnets (10.0.1.0-10.0.5.0). More secure than 0.0.0.0/0.

Protocols and ports:
| Field | Value |
|-------|-------|
| TCP | ✓ Checked |
| Ports | 80, 443 | HTTP and HTTPS |

Logging:
| Field | Value |
|-------|-------|
| Log config | Off | Too noisy for production |

Click "Create".

Rule 4: Allow Health Checks (Priority 1002)

Purpose: Allow GCP health checkers to probe backend VMs.

Field	Value
Name	`dev-firewall-allow-health-checks`
Network	`dev-network`
Priority	`1002`
Direction	`Ingress`
Action	`Allow`
Target service accounts	`backend-dev-sa`

Source filter:
| Field | Value |
|-------|-------|
| Source IPv4 ranges | 35.191.0.0/16, 130.211.0.0/22 | GCP health check ranges |

Why these IPs: These are GCP's health checker IP ranges. Without this rule, health checks fail and load balancer won't route traffic.

Protocols and ports:
| Field | Value |
|-------|-------|
| TCP | ✓ Checked |
| Ports | 80, 443 | HTTP and HTTPS |

Logging:
| Field | Value |
|-------|-------|
| Log config | Off | Health checks are too noisy |

Click "Create".

Rule 5: Allow Backend to Cache (Priority 1003)

Purpose: Allow backend VMs to connect to Redis (6379) and PgBouncer (6432).

Field	Value
Name	`dev-allow-backend-to-cache`
Network	`dev-network`
Priority	`1003`
Direction	`Ingress`
Action	`Allow`
Source service accounts	`backend-dev-sa`
Target service accounts	`cache-dev-sa`

Service Account Source: This is the key to SA-targeted firewall. We're allowing traffic FROM backend SA TO cache SA.

Protocols and ports:
| Field | Value |
|-------|-------|
| TCP | ✓ Checked |
| Ports | 6379, 6432 | Redis and PgBouncer |

Logging:
| Field | Value |
|-------|-------|
| Log config | On | Useful for debugging |

Click "Create".

Verify All Firewall Rules

You should see 5 firewall rules:

Rule Name	Priority	Direction	Action	Target
dev-firewall-deny-smtp-egress	500	Egress	Deny	All SAs
dev-firewall-allow-ssh-backend	1000	Ingress	Allow	backend-dev-sa
dev-firewall-allow-http-backend	1001	Ingress	Allow	backend-dev-sa
dev-firewall-allow-health-checks	1002	Ingress	Allow	backend-dev-sa
dev-allow-backend-to-cache	1003	Ingress	Allow	cache-dev-sa

Part 1 Verification Checklist

Before moving to Part 2, verify:

[ ] VPC dev-network exists in europe-west1
[ ] 5 subnets created with correct CIDRs (10.0.1.0/24 - 10.0.5.0/24)
[ ] Private Google Access enabled on all subnets
[ ] Cloud NAT dev-nat-gateway has 2 external IPs
[ ] Cloud Router dev-nat-router status is "Ready"
[ ] 4 service accounts created (backend, cache, observability, bastion)
[ ] Service accounts have Logs Writer role
[ ] All 5 firewall rules visible in VPC → Firewall

Cost Summary - Part 1

Component	Monthly Cost	Notes
Cloud NAT Gateway	~$32	2 static IPs + NAT gateway fee
VPC Flow Logs	~$5-10	Depends on traffic volume
Total Part 1	~$37-42	~$42/month

Cost Optimization: In production, consider:

Reducing Flow Logs sampling to 25%

Using 1 NAT IP instead of 2 (lower HA)

Disabling Flow Logs on non-critical subnets

Troubleshooting - Part 1

Issue: VPC Creation Fails

Symptom: "API not enabled" error

Solution:

Navigate to APIs & Services → Dashboard
Search for "Compute Engine API"
Click "Enable"

Issue: Subnet Creation Fails

Symptom: "IP range overlaps with existing subnet"

Solution:

Check existing subnet CIDRs in VPC details
Ensure each subnet has unique /24 range
Verify you're using 10.0.1.0 through 10.0.5.0 (not overlapping)

Issue: Cloud NAT Shows "Failed"

Symptom: NAT gateway status is "Failed"

Solution:

Verify Cloud Router exists and is in same region
Check NAT IP allocation (should show 2 IPs)
Ensure subnets have no external IP (Cloud NAT only for private IPs)

Issue: Firewall Rule Not Working

Symptom: Traffic blocked unexpectedly

Solution:

Check rule priority (lower = higher priority)
Verify service account email matches exactly
Check implicit deny: If no rule allows, traffic is blocked by default

What's Next - Part 2?

In Part 2: Security Services, you'll build:

Secret Manager for secure credential storage
Bastion host with IAP tunneling for secure SSH
IAM permissions for access control
OS Login for keyless SSH authentication

Continue to Part 2: Security Services →

References

Foundational Infrastructure Complete! Your VPC, subnets, NAT gateway, service accounts, and firewall rules are ready. Next, we'll add security services (Secrets, Bastion, IAM).

How to fix Ubuntu 24.04 NVIDIA RTX 4050 graphics driver issue in ASUS TUF A15

Shaikh Al Amin — Sat, 25 Oct 2025 14:57:35 +0000

Make sure you have secure boot off from BIOS

sudo apt purge 'nvidia-*' -y
sudo apt autoremove -y
sudo apt update
sudo apt install nvidia-driver-580 -y
sudo update-initramfs -u
sudo update-grub
sudo reboot

Verify after reboot:

nvidia-smi
xrandr

How to set up nats with docker and docker compose with token

Shaikh Al Amin — Wed, 23 Jul 2025 10:36:23 +0000

Docker compose file:

version: "3.9"

services:
  nats:
    image: nats:latest
    container_name: nats_main
    restart: always
    ports:
      - "4222:4222"
      - "8222:8222"
    volumes:
      - ./nats.conf:/etc/nats/nats.conf
    command: ["-c", "/etc/nats/nats.conf"]

  nats-client:
    image: natsio/nats-box
    container_name: nats_client
    restart: always
    depends_on:
      - nats
    entrypoint: /bin/sh
    tty: true

Config file:


listen: 0.0.0.0:4222
http: 8222

jetstream {
  store_dir: /data/jetstream
  max_mem_store: 1Gb
  max_file_store: 10Gb
}

authorization {
  token: dfji348934jdd0i24uhjd29834ijrr0345jo0r3j034n
}

Run:

docker compose up --build -d

Test:

docker compose exec nats-client nats sub -s nats://dfji348934jdd0i24uhjd29834ijrr0345jo0r3j034n@nats:4222 test

Fron another terminal:

docker compose exec nats-client nats pub -s nats://dfji348934jdd0i24uhjd29834ijrr0345jo0r3j034n@nats:4222 test "Hello via Docker Compose!"

Setup JetStream:

Verify JetStream is running:

docker compose exec nats-client nats server check jetstream -s nats://dfji348934jdd0i24uhjd29834ijrr0345jo0r3j034n@nats:4222

Create the TIXIO Stream (Manual CLI) and follow default command

docker compose exec nats-client nats stream add TIXIO \
  --subjects "tixio.>" \
  --storage file \
  --retention limits \
  -s nats://dfji348934jdd0i24uhjd29834ijrr0345jo0r3j034n@nats:4222

How to deploy Node.js Nest.js project on AWS lamda serverless

Shaikh Al Amin — Thu, 23 Jan 2025 13:40:33 +0000

N:B Setup necessary VPC configurations, public and private subnet, provide necessary permission of your AWS access and secret key and more importantly the lambda security group outbound rules:

443 Outbound rules should be forwarded to 0.0.0.0/0
Any 5432/Postgres Outbound rules should be forwarded be to postgres-security-group
For Postgres security group inbound rule accepts incoming from Lamda security groups

Create Serverless yml in project root:

service: backend-api

frameworkVersion: '^3.0.0'

useDotenv: true
configValidationMode: error

provider:
  name: aws
  region: us-east-2
  runtime: nodejs20.x
  stage: ${opt:stage, 'dev'}
  deploymentMethod: direct
  logRetentionInDays: 7

  vpc:
    securityGroupIds:
      - sg-13f39e24a9d
      - sg-87037c0f81
    subnetIds:
      - subnet-d40bc57
      - subnet-45gf564g3454

  ecr:
    scanOnPush: true
    images:
      backend-api-image:
        path: .
        file: Dockerfile

  environment:
    NODE_ENV: ${env:NODE_ENV}
    DATABASE_URL: ${env:DATABASE_URL}
    JWT_TOKEN_SECRET: ${env:JWT_TOKEN_SECRET}
    API_BACKEND_AWS_REGION: ${env:STUDY_BUDS_AWS_REGION}
    API_BACKEND_AWS_ACCESS_KEY: ${env:STUDY_BUDS_AWS_ACCESS_KEY}
    API_BACKEND_AWS_SECRETE_KEY: ${env:STUDY_BUDS_AWS_SECRETE_KEY}
    AWS_BUCKET_NAME: ${env:AWS_BUCKET_NAME}
    AWS_CLOUD_FRONT_URL: ${env:AWS_CLOUD_FRONT_URL}
    AWS_SMTP_HOST: ${env:AWS_SMTP_HOST}
    AWS_SMTP_USER: ${env:AWS_SMTP_USER}
    AWS_SMTP_PASS: ${env:AWS_SMTP_PASS}
    AWS_FROM_EMAIL: ${env:AWS_FROM_EMAIL}
    AI_BACKEND_URL: ${env:AI_BACKEND_URL}
    CORS_ALLOWED_HOSTS: ${env:CORS_ALLOWED_HOSTS}
    FRONTEND_BASE_URL: ${env:FRONTEND_BASE_URL}
    MAIL_HOST: ${env:MAIL_HOST}
    MAIL_USER: ${ env:MAIL_USER }
    MAIL_PASSWORD: ${env:MAIL_PASSWORD}
    MAIL_FROM_NAME: ${env:MAIL_FROM_NAME}
    MAIL_FROM_ADDRESS: ${env:MAIL_FROM_ADDRESS}

  apiGateway:
    binaryMediaTypes:
      - '*/*'
custom:
  prune:
    automatic: true
    number: 2

functions:
  main:
    image:
      name: backend-api-image
      command:
        - dist/src/serverless.handler
      entryPoint:
        - '/lambda-entrypoint.sh'
    memorySize: 1024
    timeout: 330
    url: true
    provisionedConcurrency: 0
    events:
      - http:
          method: ANY
          path: /
      - http:
          method: ANY
          path: '{proxy+}'

Install below two packages:

npm i @codegenie/serverless-express
npm i -D @types/aws-lambda

Create serverless.ts inside src/serverless.ts

import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';
import serverlessExpress from '@codegenie/serverless-express';
import { Callback, Context, Handler } from 'aws-lambda';
import { RequestMethod, ValidationPipe } from '@nestjs/common';
import helmet from 'helmet';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
import { ResponseTransformInterceptor } from './common/interceptor/global-response-interceptor';
import { RolePermissionsSeederService } from './modules/v1/user/seed/role-permissions.seeder.service';
import { SeedService } from './modules/v1/character/seeder/seeder.service';
let server: Handler;

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  const allowedHosts = (process.env.CORS_ALLOWED_HOSTS as string) || '*';

  app.enableCors({
    origin: allowedHosts.split(','),
    methods: 'GET,HEAD,PUT,PATCH,POST,DELETE,OPTIONS',
    credentials: true,
  });

  app.setGlobalPrefix('api/v1', {
    exclude: [{ path: '/', method: RequestMethod.GET }],
  });

  app.use(helmet());

  app.useGlobalPipes(
    new ValidationPipe({
      transform: true,
      whitelist: true,
    }),
  );

  const reflector = app.get(Reflector);
  app.useGlobalInterceptors(new ResponseTransformInterceptor(reflector));

  //const rolePermissionsService = app.get(RolePermissionsSeederService);

  //await Promise.all([rolePermissionsService.insertRolePermissions()]);

  const rolePermissionsService = app.get(RolePermissionsSeederService);
  const seedsService = app.get(SeedService);

  const config = new DocumentBuilder()
    .setTitle('Backend api')
    .setDescription('Swagger docs for backend apis')
    .setVersion('1.0')
    .addBearerAuth()
    .build();

  const document = SwaggerModule.createDocument(app, config);
  SwaggerModule.setup('/api-docs', app, document);

  await app.init();

  const expressApp = app.getHttpAdapter().getInstance();
  return serverlessExpress({
    app: expressApp,
  });
}

export const handler: Handler = async (
  event: any,
  context: Context,
  callback: Callback,
) => {
  try {
    server = server ?? (await bootstrap());
    return server(event, context, callback);
  } catch (error) {
    console.error('Error during Lambda execution:', error);
    return {
      statusCode: 500,
      body: JSON.stringify({
        error: 'Internal Server Error',
        message: error.message,
      }),
    };
  }
};

Create Dockerfile inside root directory :

# Use the AWS Lambda base image for Node.js
FROM public.ecr.aws/lambda/nodejs:22

# Set the working directory in the container
WORKDIR ${LAMBDA_TASK_ROOT}

# Copy only package files first for dependency installation
COPY package*.json ${LAMBDA_TASK_ROOT}/

# Install dependencies (both production and development for the build phase)
RUN npm install

# Copy the rest of the application code (excluding files in .dockerignore)
COPY . ${LAMBDA_TASK_ROOT}/

# Build the NestJS application
RUN npm run build

# Set the Lambda function handler
CMD ["dist/src/serverless.handler"]

Create .env.dev in project root

NODE_ENV=development
PORT=8035
DATABASE_URL=postgresql://postgres:postgres@database_container:5432/nestjs
REDIS_HOST=redis_container
REDIS_PORT=6379
JWT_TOKEN_SECRET=dkfjkjdfkjdfkdfjkjdfkjkdkdfjk
AWS_BUCKET_NAME=store-backend
API_BACKEND_AWS_REGION=us-east-2
API_BACKEND_AWS_ACCESS_KEY=JKDIUENDIUEKNIUEIJKIWJKWJOUIW
API_BACKEND_AWS_SECRETE_KEY=LKFLKORJKIJKUFKJFKJKFJKFKFJ
AWS_CLOUD_FRONT_URL=https://your.cloudfront.net
AWS_SMTP_HOST=email-smtp.ap-southeast-1.amazonaws.com
AWS_SMTP_USER=AJKDIUENSDUJEHUSDJBUE
AWS_SMTP_PASS=fkdjkjrtiu93j934jkjf98493ikjkjrjii4ju5
AWS_FROM_EMAIL=alamin.cse15@gmail.com
AI_BACKEND_URL=https://aibackend.io/api/v2
CORS_ALLOWED_HOSTS='http://localhost:3000'
FRONTEND_BASE_URL="http://localhost:3000"

MAIL_HOST=smtp.gmail.com
MAIL_USER=alamin.cse15@gmail.com
MAIL_PASSWORD='sdjkdfjkdfkdfhjdfhdjfhj'
MAIL_FROM_NAME=Shaikh
MAIL_FROM_ADDRESS=alamin.cse15@gmail.com

*Install serverless framework globally: *

npm i -g serverless@3.39.0

Setup AWS access and secret key using AWS CLI (setup in ubuntu):

**
Now navigate to project root and run **

sls deploy --stage dev

Make sure you have .env.dev exists in your project root:

**CI/CD: Deploy from github actions:**

name: Deploy Serverless App to Dev Environment

on:
  push:
    branches:
      - dev
    pull_request:
      types: [closed]
      branches:
        - dev

jobs:
  deploy:
    if: github.event.pull_request.merged == true || github.event_name == 'push'
    runs-on: ubuntu-latest

    steps:
      # 1. Checkout code
      - name: Checkout code
        uses: actions/checkout@v3

      # 3. Configure AWS Credentials
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v3
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-2

      # 4. Setup Node.js
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 20.x

      - name: Clear Serverless Cache
        run: |
          rm -rf .serverless

      # 5. Map environment variables manually
      - name: Set Environment Variables
        run: |
          rm -f .env.dev
          touch .env.dev
          echo "NODE_ENV=${{ secrets.NODE_ENV }}" >> .env.dev
          echo "DATABASE_URL=${{ secrets.DEV_DATABASE_URL }}" >> .env.dev
          echo "JWT_TOKEN_SECRET=${{ secrets.DEV_JWT_TOKEN_SECRET }}" >> .env.dev
          echo "AWS_BUCKET_NAME=${{ secrets.DEV_AWS_BUCKET_NAME }}" >> .env.dev
          echo "BACKEND_AWS_REGION=${{ secrets.DEV_BACKEND_AWS_REGION }}" >> .env.dev
          echo "BACKEND_AWS_ACCESS_KEY=${{ secrets.BACKEND_AWS_ACCESS_KEY }}" >> .env.dev
          echo "BACKEND_AWS_SECRETE_KEY=${{ secrets.DEV_BACKEND_AWS_SECRETE_KEY }}" >> .env.dev
          echo "AWS_CLOUD_FRONT_URL=${{ secrets.DEV_AWS_CLOUD_FRONT_URL }}" >> .env.dev
          echo "AWS_SMTP_HOST=${{ secrets.AWS_SMTP_HOST }}" >> .env.dev
          echo "AWS_SMTP_USER=${{ secrets.AWS_SMTP_USER }}" >> .env.dev
          echo "AWS_SMTP_PASS=${{ secrets.AWS_SMTP_PASS }}" >> .env.dev
          echo "AWS_FROM_EMAIL=${{ secrets.AWS_FROM_EMAIL }}" >> .env.dev
          echo "AI_BACKEND_URL=${{ secrets.DEV_AI_BACKEND_URL }}" >> .env.dev
          echo "CORS_ALLOWED_HOSTS=${{ secrets.DEV_CORS_ALLOWED_HOSTS }}" >> .env.dev
          echo "FRONTEND_BASE_URL=${{ secrets.DEV_FRONTEND_BASE_URL }}" >> .env.dev
          echo "MAIL_HOST=${{ secrets.MAIL_HOST }}" >> .env.dev
          echo "MAIL_USER=${{ secrets.MAIL_USER }}" >> .env.dev
          echo "MAIL_PASSWORD=${{ secrets.MAIL_PASSWORD }}" >> .env.dev
          echo "MAIL_FROM_NAME=${{ secrets.MAIL_FROM_NAME }}" >> .env.dev
          echo "MAIL_FROM_ADDRESS=${{ secrets.MAIL_FROM_ADDRESS }}" >> .env.dev


      # 6. Install Serverless Framework v3
      - name: Install Serverless Framework v3
        run: npm i -g serverless@3.39.0

        # 7. Deploy to the specified stage
      - name: Deploy to dev
        run: sls deploy --stage dev

How to deploy fastAPI app with postgreSQL database in AWS EC2

Shaikh Al Amin — Thu, 07 Nov 2024 03:42:13 +0000

SSH into your AWS cli

System update and install Postgres, Nginx, python3 pip, and venv

sudo apt update
sudo apt upgrade
sudo apt install curl ca-certificates
sudo install -d /usr/share/postgresql-common/pgdg
sudo curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
sudo install -d /usr/share/postgresql-common/pgdg
sudo sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
sudo apt install postgresql-14 postgresql-client-14
sudo apt install python3-pip python3-venv 
sudo apt install nginx

Complete Postgres setup and create a database

sudo -i -u postgres
psql -U postgres -h localhost

postgres@shaikh:~$ psql
postgres=# create database local_test;
CREATE DATABASE
postgres=# grant all privileges on database local_test to postgres;
GRANT
postgres=# ALTER USER postgres WITH PASSWORD 'postgres';
ALTER ROLE
postgres=# exit()

Generate ssh profile to pull the project from github

cd ~/.ssh/
ssh-keygen -t ed25519 -C "amin@test.com"

Copy the public key and put inside your github setting SSH keys with a valid name

cat id_ed25519.pub

Create a folder for git projects and clone the repo

cd ~/
mkdir projects
cd projects/
git clone git@github.com:shaikhalamin/backend-project.git

Create a virtual environment and activate the venv

python3.10 -m venv venv
source venv/bin/activate

Now CD into the project directory and install the requirements along with Gunicorn server

cd backend-project/
pip install -r requirements.txt
pip install gunicorn

Setup your .env and modify accordingly

cp .env.example .env
nano .env

Test the server run using unicorn

uvicorn main:app --host 0.0.0.0 --port 8000
cat .env
nano configs/setting.py 
uvicorn main:app --host 0.0.0.0 --port 8000

Now Setup gunicorn service for running your application as a service

Get the Gunicorn installed path and project path by the below command

which gunicorn
pwd

Create below service file and put the below following content in there

sudo nano /etc/systemd/system/devapi_gunicorn.service


[Unit]
Description=Gunicorn instance to serve fast API application
After=network.target

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu/backend-project/
ExecStart=/home/ubuntu/venv/bin/gunicorn -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8080 main:app

[Install]
WantedBy=multi-user.target

Now enable Gunicorn service

sudo systemctl enable devapi_gunicorn
sudo systemctl start devapi_gunicorn
sudo systemctl status devapi_gunicorn

Now create an nginx config file and change it according to your domain name

cd /etc/nginx/sites-available/
sudo nano orangedev.com


   server {

    server_name orangedev.com;

    location / {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

  }

Move the nginx config file to the sites-enabled

sudo ln -s /etc/nginx/sites-available/orangedev.com /etc/nginx/sites-enabled/

Check the Nginx config and reload the Nginx

sudo nginx -t
sudo systemctl reload nginx
sudo systemctl restart nginx

Allow fire all for SSH, HTTP, and HTTPS only

sudo ufw status
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
# For posgres from another server
sudo ufw allow 5432/tcp
sudo ufw enable
sudo ufw status

If everything goes well install certbot for SSL certificated

sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
[N:B]
# Setup domain name and point to the correct IP and check from DNS checker then run the below command
sudo certbot --nginx

How to detect your Python library path from docker in your VS Code

Shaikh Al Amin — Tue, 05 Nov 2024 16:28:07 +0000

Vs Code plugins dependencies:

1. Install python plugin from Microsoft
2. Install pylance plugin from Microsoft
3. Install Black Formatter plugin from Microsoft

Create a virtual env inside your project root.

python3.10 -m venv venv

Activate the venv.

source venv/bin/activate

Install the dependency from requirments.txt

pip install -r requirments.txt

Select python env from venv for VS Code interpreter

You can set the interpreter by pressing

Ctrl+Shift+P, searching for Python: Select Interpreter,

and then choosing the virtual environment in your project root

(.venv/bin/python or similar).

DEV Community: Shaikh Al Amin

Install playwrite cli with claude code for any project with more accurate result

Building a Reliable Job Scheduler with Cloudflare Durable Objects and Queues

The Problem: "I Need Something to Happen Later"

The Big Picture (Start Here)

The Alarm Clock Analogy

Architecture Overview

Part 1: The Three Types of Schedules

1. Cron Schedule (Repeating Pattern)

2. Interval Schedule (Every N Minutes)

3. Once Schedule (Fire and Forget)

Part 2: The API Layer (Auth Worker — Notifications Module)

What Happens When a User Creates a Schedule

Step 1: Validate the Input

Step 2: Convert to Internal Format

Step 3: Save to Database

Step 4: Register with the Scheduler Worker

Part 3: The Scheduler Publisher (The Bridge)

What is a Service Binding?

Part 4: The Scheduler Worker (The Brain)

What is a Durable Object?

One Durable Object Per Schedule

Part 5: Inside the Durable Object (The Timer Engine)

Initialization: Setting the First Alarm

How calculateNextRun Works

The Alarm Fires: Delivering the Job

After the Job is Sent: The Repeat Loop

Retry Logic: What If the Queue Send Fails?

Part 6: The Three Storage Layers (Why Three?)

Part 7: Complete End-to-End Example

Scenario: "Send me a daily stock price notification at 2 PM New York time"

Part 8: Update and Delete Flows

Updating a Schedule

Deleting a Schedule

Part 9: The Queue System (How Jobs Get Delivered)

Part 10: Why This Architecture Works Well

Reliability

Scalability

Simplicity

Cost

Summary

Building Resilient, Scalable Apps with Bun, Hono, and Cloudflare

Build Production-Ready GCP Infrastructure from Scratch Part 04

Build Production-Ready GCP Infrastructure from Scratch: A Complete Console Guide

Table of Contents

Part 4: Observability & Load Balancer

Overview

Prerequisites

Step 1: Create Static Internal IPs for Observability

What are Static Internal IPs?

Prometheus Static IP

Navigation Path

IP Configuration

Loki Static IP

Verify IPs

Step 2: Create Prometheus VM

What is Prometheus?

Navigation Path

VM Configuration

Basic Settings

Machine Type

Boot Disk

Network Interface

Service Account

Metadata - Startup Script

Create the VM

Verify Prometheus

Step 3: Create Loki VM (with Grafana)

What is Loki and Grafana?

Navigation Path

VM Configuration

Basic Settings

Machine Type

Boot Disk

Network Interface

Service Account

Metadata - Startup Script

Create the VM

Verify Loki VM

Step 4: Access Grafana Dashboard

How `calculateNextRun` Works