DEV Community: Linh Truong Cong Hong

Battle-Testing an IoT Telemetry Pipeline: Load Testing 60x Storm Spikes & Chaos Engineering with Grafana k6, KEDA, and Opossum

Linh Truong Cong Hong — Mon, 26 Oct 2026 15:34:55 +0000

In our previous article, we walked through the architecture of the Cloud-Native Weather Telemetry Monitoring System—a distributed microservices pipeline built with NestJS, gRPC, RabbitMQ, TimescaleDB, and Redis.

Designing an architecture on paper that claims to handle 60x storm traffic surges and downstream database outages is easy. Proving that it survives those conditions without dropping sensor data or paging engineers at 3:00 AM requires a rigorous verification strategy.

In this post, we’ll dive deep into how we test the three pillars of production readiness across our pipeline:

Scalability & Performance Testing (simulating 60x dynamic frequency bursts and 5,000-station fleet growth with Grafana k6, KEDA, and HPA).
High Availability Under Dependency Outages (verifying Opossum circuit breakers and Redis cache-aside fallbacks when downstream services die).
Resilience & Fault Isolation (injecting physical anomalies and malformed "poison pill" payloads to verify RabbitMQ Dead Letter Queues and consumer backpressure).

1. Why Standard Load Tests Fail for Weather Telemetry

Most web applications experience gradual, predictable traffic curves that follow human activity (morning ramp-up, afternoon peak, evening cool-down). Weather monitoring networks behave completely differently:

Calm Weather Baseline: Thousands of stations report once every 60 seconds (1 req/min).
The Storm Surge (60x Instantaneous Spike): When processing-service detects a severe anomaly—such as a rapid barometric pressure drop ($\Delta P > 3\text{ hPa}$ over 3 hours) or wind gusts exceeding $28.0\text{ m/s}$—alerting-service issues a gRPC UpdateReportingFrequency command that shifts stations into 1-second emergency sampling mode.
The Double Jeopardy: Traffic spikes by 6000% at the exact moment when meteorological data is most critical and when storm conditions may simultaneously degrade network links or infrastructure.

To validate our system against this reality, we defined strict Service Level Objectives (SLOs) before writing our test suites:

Metric	Target SLO	Why It Matters
Ingestion Latency (`p95`)	`< 50 ms`	Edge stations on unstable satellite/cellular links hold open sockets briefly; slow ACKs drain battery and cause retry storms.
Ingestion Latency (`p99`)	`< 150 ms`	Tail latency must stay bounded even during GC pauses or Redis cache misses.
Ingestion Error Rate	`< 0.1%` (`rate < 0.001`)	High-resolution storm telemetry cannot be lost at the gateway.
Outage Availability	`100%` degraded ingestion	If `station-service` crashes, `ingestion-service` must trip its circuit breaker within `2000ms` and continue accepting valid telemetry.
Poison-Pill Recovery	`0` consumer restarts	Corrupted packets must be isolated to `weather.telemetry.dlq` without blocking healthy messages in `weather.raw.telemetry`.

2. Scalability Testing with Grafana k6

We use Grafana k6 for performance and load testing because its Go-based runtime generates massive concurrent load from a single machine while allowing us to write expressive test scenarios in JavaScript.

Scenario A: The 60x Dynamic Frequency Spike (`ramping-arrival-rate`)

A common mistake in load testing is using closed-model Virtual Users (VUs) where each VU waits for the server to respond before sending the next request. If your server slows down under load, closed-model VUs naturally throttle themselves—masking real-world saturation!

Real IoT weather stations follow an open workload model: when the clock ticks 1 second, 1,200 stations transmit their packets whether your server finished the previous batch or not. To model this accurately in test/k6/scenarios/dynamic-frequency-spike.js, we use k6’s ramping-arrival-rate executor:

// test/k6/scenarios/dynamic-frequency-spike.js
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  scenarios: {
    baseline_and_spike: {
      executor: 'ramping-arrival-rate',
      startRate: 20, // Baseline: ~20 req/s across fleet (60s reporting mode)
      timeUnit: '1s',
      preAllocatedVUs: 200,
      maxVUs: 1500,
      stages: [
        { duration: '30s', target: 20 },   // Stage 1: Warmup / Calm Weather Baseline
        { duration: '15s', target: 600 },  // Stage 2: 30x Surge as storm front approaches
        { duration: '1m',  target: 1200 }, // Stage 3: 60x Burst — 1,200 req/sec sustained
        { duration: '30s', target: 20 },   // Stage 4: Storm passes, return to 60s interval
      ],
    },
  },
  thresholds: {
    http_req_duration: ['p(95)<50', 'p(99)<150'], // Sub-50ms p95 SLO
    http_req_failed: ['rate<0.001'],               // <0.1% failure tolerance
  },
};

Inside the test function, 20% of payloads simulate active storm readings (pressure: 998.0 hPa, wind_speed: 32.5 m/s, precipitation: 15.0 mm). This ensures our load test doesn't just exercise the HTTP gateway—it simultaneously triggers the sliding-window AnomalyDetector in processing-service and fires real anomaly events through alerting-service!

export default function () {
  const stationId = STATIONS[Math.floor(Math.random() * STATIONS.length)];
  const isStorm = Math.random() < 0.2; // 20% probability of storm readings

  const payload = JSON.stringify({
    station_id: stationId,
    timestamp: Date.now(),
    temperature: 18.5 + (Math.random() * 5 - 2.5),
    humidity: 65.0 + (Math.random() * 10 - 5),
    pressure: isStorm ? 998.0 : 1013.2 + (Math.random() * 4 - 2), // Sudden drop when storm
    wind_speed: isStorm ? 32.5 : 5.0 + Math.random() * 8,          // High gusts when storm
    wind_direction: Math.floor(Math.random() * 360),
    precipitation: isStorm ? 15.0 : 0.0,
    battery_voltage: 12.6,
    api_key: 'sec_token_alps_001',
  });

  const res = http.post(`${BASE_URL}/api/v1/telemetry`, payload, {
    headers: { 'Content-Type': 'application/json' },
  });

  check(res, {
    'status is 200': (r) => r.status === 200,
    'ingestion code SUCCESS': (r) => JSON.parse(r.body).code === 1,
  });
}

Scenario B: Horizontal Fleet Growth (`500` to `5,000` Concurrent Stations)

While dynamic-frequency-spike.js tests temporal burstiness across existing stations, test/k6/scenarios/fleet-growth.js tests high-cardinality station onboarding—ramping from 500 to 5,000 concurrent stations (ST-REGIONAL-0001 through ST-REGIONAL-1000).

// test/k6/scenarios/fleet-growth.js
export const options = {
  stages: [
    { duration: '30s', target: 500 },  // 500 concurrent stations
    { duration: '1m',  target: 2000 }, // Ramp to 2,000 stations
    { duration: '1m',  target: 5000 }, // Peak at 5,000 stations
    { duration: '30s', target: 500 },  // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<100'],
    http_req_failed: ['rate<0.01'],
  },
};

Why is high station cardinality important to test separately?

Redis Key Cardinality: Every new station_id triggers an initial cache miss on station:<id>:valid, testing our gRPC connection pool between ingestion-service and station-service.
In-Memory Sliding Windows: AnomalyDetector in processing-service maintains a per-station sliding window of historical readings. Testing 1,000+ distinct station IDs verifies that our sliding-window pruning prevents memory leaks.

3. How the Architecture Absorbs the Load: Autoscaling & Micro-Batching

When you run dynamic-frequency-spike.js and traffic jumps from 20 req/s to 1,200 req/s in 15 seconds, four architectural mechanisms work together to keep p95 latency under 50ms:

1. Stateless Ingress Scaling via Kubernetes HPA

At the edge, ingestion-service does zero disk I/O. It validates the payload, checks Redis (~0.5ms), and publishes to RabbitMQ. In Kubernetes (k8s/autoscaling/hpa-ingestion.yaml), a standard Horizontal Pod Autoscaler scales ingestion-service from 3 to 25 replicas whenever CPU crosses 70% or memory crosses 80%.

2. Event-Driven Consumer Scaling via KEDA

CPU utilization is a terrible metric for scaling message queue consumers—an I/O-bound worker waiting on database flushes might sit at 25% CPU while the RabbitMQ queue builds a backlog of 100,000 messages.

Instead, we use KEDA (Kubernetes Event-driven Autoscaling) in k8s/autoscaling/keda-processing-scaledobject.yaml to monitor the exact depth of the weather.raw.telemetry queue every 5 seconds:

# k8s/autoscaling/keda-processing-scaledobject.yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: processing-service-scaler
  namespace: weather-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: processing-service
  minReplicaCount: 2
  maxReplicaCount: 20
  cooldownPeriod: 30
  pollingInterval: 5
  triggers:
    - type: rabbitmq
      metadata:
        protocol: amqp
        queueName: weather.raw.telemetry
        mode: QueueLength
        value: "500" # Scale up 1 replica for every 500 queued messages

3. Bounded Consumer Backpressure (`prefetch: 200`)

To prevent a sudden surge of messages in RabbitMQ from flooding a single processing-service pod's V8 heap and causing an OutOfMemory crash, every consumer channel enforces channel.prefetch(200) with manual acknowledgments (noAck: false). RabbitMQ will never push more than 200 unacknowledged messages to a worker at once.

4. Multi-Row Micro-Batching into TimescaleDB

Even if KEDA spins up 20 processing-service pods, executing 1,200 individual INSERT statements per second would exhaust PostgreSQL connection pools and lock indexes.

Instead, ProcessingService buffers incoming messages in memory until either 500 records accumulate OR a 250ms timer expires, then flushes them in a single parameterized multi-row INSERT via pg.Pool (TimescaleRepository.insertBatch). At 1,200 req/s, 1,200 network round-trips to TimescaleDB collapse into just 2.4 bulk writes per second!

4. Availability & Chaos Engineering: Breaking Things on Purpose

High throughput in ideal conditions is only half the story. In distributed systems, dependencies will fail. We built an automated chaos engineering suite under test/chaos/ (runnable via npm run test:chaos) to prove our system degrades gracefully under partial outages.

Experiment 1: Station Service Outage & Circuit Breaker Verification

The Hypothesis: When a station sends telemetry and its auth token isn't cached in Redis, ingestion-service makes a synchronous gRPC call (ValidateStation) to station-service. What happens if station-service (or its PostgreSQL database) crashes or hangs during a storm?

Without protection: Every incoming HTTP/gRPC request hangs waiting for station-service, exhausting socket pools and causing a total ingestion outage (cascading failure).
With our Opossum Circuit Breaker: After 50% of requests fail or exceed the 2000ms timeout, the circuit breaker trips to OPEN, fails fast in < 1ms, and invokes a degraded fallback handler that validates the station ID against a strict format regex (/ST-[A-Z0-9-]+$/). Valid stations continue having their telemetry queued to RabbitMQ!

Let's look at the chaos test in test/chaos/test-circuit-breaker.js:

// test/chaos/test-circuit-breaker.js
const payload = {
  station_id: 'ST-ALPS-001',
  timestamp: Date.now(),
  temperature: 21.0,
  humidity: 55.0,
  pressure: 1013.0,
  wind_speed: 3.2,
  wind_direction: 120,
  precipitation: 0.0,
  battery_voltage: 12.5,
  api_key: 'sec_token_alps_001',
};

for (let i = 1; i <= 20; i++) {
  const start = Date.now();
  const res = await fetch(`${ingestionUrl}/api/v1/telemetry`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(payload),
  });

  const data = await res.json();
  const latency = Date.now() - start;

  if (res.ok && data.code === 1) {
    console.log(`[Req ${i}] ✅ SUCCESS in ${latency}ms`);
  } else if (data.message && data.message.includes('fallback')) {
    console.log(`[Req ${i}] 🛡️ DEGRADED FALLBACK in ${latency}ms: ${data.message}`);
  }
}

What Happens During the Chaos Run:

We stop station-service (and flush Redis) while keeping ingestion-service and rabbitmq running.
We fire 20 sequential telemetry requests via node test/chaos/test-circuit-breaker.js.
Requests 1–2: Experience the gRPC connection refusal/timeout, trigger the circuit breaker fallback, and still return HTTP 200 (code: 1) because the fallback permits format-valid station ST-ALPS-001.
Circuit Breaker Trips (CLOSED -> OPEN): AppLogger logs CircuitBreaker[StationService.ValidateStation] OPENED — failing fast and using fallback.
Requests 3–20: Bypass the dead gRPC channel completely! Latency drops to 2ms–4ms, and 100% of the 20 telemetry packets are safely persisted into RabbitMQ.
Once station-service recovers, after resetTimeout: 5000ms, the breaker enters HALF_OPEN, probes a single gRPC call, and snaps back to CLOSED.

Experiment 2: Poison-Pill Injection & Dead Letter Queue (DLQ) Isolation

The Hypothesis: Edge IoT sensors exposed to lightning, freezing rain, or failing batteries occasionally transmit corrupted JSON or physically impossible readings (e.g., -500°C temperature or 150% relative humidity).

The Danger (Head-of-Line Blocking): If a consumer fails to process a corrupted message and requeues it (channel.nack(msg, false, true)), RabbitMQ will redeliver that same poison pill in an infinite loop—blocking all valid telemetry behind it and pinning CPU at 100%.

We defend against poison pills at two layers, verified by test/chaos/test-dlq-poison-pill.js:

// test/chaos/test-dlq-poison-pill.js
const poisonPills = [
  {
    description: 'Physical impossibility: Temperature -500°C',
    payload: { station_id: 'ST-ALPS-001', temperature: -500, humidity: 50, pressure: 1013, wind_speed: 5, wind_direction: 0, precipitation: 0 },
  },
  {
    description: 'Physical impossibility: Humidity 150%',
    payload: { station_id: 'ST-ALPS-001', temperature: 25, humidity: 150, pressure: 1013, wind_speed: 5, wind_direction: 0, precipitation: 0 },
  },
  {
    description: 'Missing required station_id',
    payload: { temperature: 20, humidity: 50, pressure: 1013, wind_speed: 5, wind_direction: 0, precipitation: 0 },
  },
];

Layer 1 — Perimeter Sanity Check (ingestion-service): Before touching Redis or RabbitMQ, IngestionService.validateSensorRanges() enforces physical meteorological boundaries:
- Temperature: -90°C to +65°C (Absolute zero is -273.15°C; -500°C is rejected immediately with 400 Bad Request).
- Humidity: 0% to 100%.
- Barometric Pressure: 850 hPa to 1100 hPa.
- Wind Speed: 0 m/s to 150 m/s.
Layer 2 — Dead Letter Exchange Isolation (processing-service): If a malformed or unparseable binary packet gets published directly onto weather.raw.telemetry by a legacy producer, ProcessingService catches the parse error and executes:

   channel.nack(msg, false, false); // requeue = false!

Because weather.raw.telemetry is configured with x-dead-letter-exchange: weather.telemetry.dlx, RabbitMQ automatically routes the poison pill into weather.telemetry.dlq for forensic inspection while the worker immediately moves on to the next batch.

5. Run the Verification Suite Yourself!

Everything described in this post is 100% open-source (MIT Licensed) and reproducible on your local machine in minutes.

Step 1: Clone & Start the Stack

git clone https://github.com/linhtch90/weather-monitoring-system.git
cd weather-monitoring-system
npm install

# Start RabbitMQ, TimescaleDB, PostgreSQL, Redis, and Prometheus
docker compose up -d

Step 2: Run Unit & Contract Verification

npm test

Step 3: Run the Chaos & Resilience Suite

With the microservices running, execute our automated chaos runner:

npm run test:chaos

Output:

==================================================
⚡ RUNNING WEATHER SYSTEM CHAOS & RESILIENCE SUITE
==================================================

1. Testing Ingestion Circuit Breaker...
🧪 Starting Chaos Experiment: Station Service Outage & Circuit Breaker Verification
[Req 1] ✅ SUCCESS in 8ms
...
[Req 20] ✅ SUCCESS in 3ms

--- Experiment Summary ---
Total Requests: 20
Successful: 20
Graceful Fallbacks: 0
Zero unhandled crash exceptions observed.

2. Testing Poison Pill Rejection & DLQ...
🧪 Starting Chaos Experiment: Poison Pill Injection & DLQ Isolation
[Test: Physical impossiblity: Temperature -500°C] -> Status: 400 (Caught and rejected without crashing)
[Test: Physical impossibility: Humidity 150%] -> Status: 400 (Caught and rejected without crashing)
[Test: Missing required station_id] -> Status: 400 (Caught and rejected without crashing)

✅ Poison pills rejected at ingress; corrupted queue packets routed to weather.telemetry.dlq.
✅ All resilience experiments completed successfully.

Step 4: Trigger the 60x Storm Load Test with Grafana k6

# Run the 20 req/s -> 1,200 req/s storm burst test
k6 run test/k6/scenarios/dynamic-frequency-spike.js

# Or run the 500 -> 5,000 station fleet growth test
k6 run test/k6/scenarios/fleet-growth.js

Key Takeaways

Test Open Workloads with Arrival Rates: When testing IoT or telemetry pipelines, use ramping-arrival-rate in Grafana k6 rather than closed-loop VUs so your load generator doesn't politely slow down when your server struggles.
Scale Consumers on Queue Depth, Not CPU: Pairing RabbitMQ with KEDA (QueueLength: 500) and consumer prefetch: 200 gives you elastic horizontal scaling while protecting worker memory.
Never Trust a Synchronous Dependency on a Hot Path: Wrapping gRPC calls in Opossum circuit breakers with Redis caching and a deterministic fallback turns a catastrophic downstream outage into a non-event.
Automate Chaos Checks: Treat circuit-breaker trips and poison-pill DLQ isolation like unit tests—script them (npm run test:chaos) and run them continuously.

Check out the full repository, load test scripts, and Kubernetes autoscaling manifests on GitHub:
👉 github.com/linhtch90/weather-monitoring-system

Designing a Cloud-Native Weather Telemetry System: Handling 60x Storm Traffic Spikes with NestJS, gRPC, RabbitMQ & TimescaleDB

Linh Truong Cong Hong — Sat, 24 Oct 2026 13:22:11 +0000

Most IoT tutorials assume a flat, predictable stream of sensor data: 1,000 devices sending 1 payload per minute. In the real world, weather telemetry behaves nothing like that.

During calm weather, thousands of remote monitoring stations report once every 60 seconds. The moment a storm front or rapid barometric pressure drop hits ( $Δ P > 3 hPa /3 h$ ), scientists and automated control loops command those stations to switch to 1-second high-resolution sampling. Overnight, your ingestion pipeline faces an immediate 60x traffic explosion—right when losing data is least acceptable.

In this post, I’ll walk through how we designed and built the Cloud-Native Weather Telemetry Monitoring System—a 100% open-source microservices platform built with Node.js, TypeScript, NestJS, gRPC, RabbitMQ, TimescaleDB, and Redis—and the engineering trade-offs we made to guarantee scalability, availability, and chaos resilience.

1. The Core Requirements vs. Architectural Decisions

Before writing a single line of code, we mapped out five non-negotiable domain requirements and paired each with a specific architectural pattern:

Domain Requirement	The Engineering Challenge	Our Architectural Solution
Dynamic 60x Traffic Surges	Stations shift from `1 req/60s` to `1 req/1s` during severe weather, overwhelming synchronous DB writes.	Queue-Decoupled Ingestion + Micro-Batching: Gateway offloads to RabbitMQ in `< 15ms`; workers flush 500-row batches every `250ms` to TimescaleDB.
Dual Communication Needs	Some calls need instant validation (`ValidateStation`), while telemetry streams need buffered throughput.	gRPC + RabbitMQ Hybrid: Direct HTTP/2 binary Protobuf (gRPC) for synchronous RPCs; RabbitMQ topic exchanges for async telemetry streams.
Diverse Edge Hardware	Modern stations support HTTP/2 gRPC streams; constrained cellular/satellite IoT devices only speak HTTP/1.1 JSON.	Multi-Protocol Hybrid Gateway: A single NestJS `IngestionController` exposing both `:50052` (gRPC) and `:3000` (REST) backed by the same domain service.
Zero Cascading Failures	If `station-service` or Postgres slows down during a storm, the ingestion gateway must not drop incoming station packets.	Opossum Circuit Breakers + Redis Cache-Aside: 5-minute TTL Redis token cache + circuit breaker fallback that accepts format-valid station payloads in degraded mode.
100% Open-Source Stack	Zero vendor lock-in; runnable on any laptop via Docker Compose or scaled on Kubernetes.	NestJS, RabbitMQ, TimescaleDB (PostgreSQL 16), Redis, Prometheus, KEDA, and Grafana k6.

2. High-Level System Architecture

Tip — Diagram as Code: Rather than drawing static diagrams in external tools that quickly go stale, we wrote a zero-dependency Python generator (prompt/generate_diagram.py) that renders our architecture SVG directly from code. Whenever a queue or service changes, we update the script and regenerate the vector diagram in Git.

The system is structured as a NestJS Monorepo divided into four strictly isolated microservices (apps/*) and three shared libraries (libs/*):

ingestion-service (:3000 HTTP / :50052 gRPC): The ultra-fast front door. Validates physical sensor ranges, checks station credentials via Redis/gRPC, and publishes to RabbitMQ with Publisher Confirms.
station-service (:50051 gRPC): The source of truth for station metadata, coordinates, and active sampling intervals stored in PostgreSQL.
processing-service (RabbitMQ Worker): Consumes raw telemetry with prefetch: 200, evaluates sliding-window meteorological anomalies, and bulk-inserts micro-batches into TimescaleDB hypertables.
alerting-service (RabbitMQ Worker + gRPC Client): Listens for weather.anomaly.detected events and dispatches direct gRPC UpdateReportingFrequency(1s) commands back to station-service.

3. Deep Dive: Key Engineering Best Practices

3.1 Dual Communication: Why Both gRPC and RabbitMQ?

A common microservice anti-pattern is forcing all inter-service communication through a single hammer—either making everything an HTTP/gRPC call (creating brittle synchronous chains) or making everything an async event (turning simple queries into complex correlation-ID state machines).

We split communication by intent:

Direct Synchronous (gRPC with Proto3): When ingestion-service experiences a Redis cache miss for station:ST-ALPS-001:valid, it needs an immediate answer from station-service. Using gRPC over HTTP/2 gives us strict contract enforcement (station.proto) and low-latency binary serialization.
Indirect Asynchronous (RabbitMQ AMQP 0-9-1): Once a telemetry payload is validated, the station shouldn't wait for analytical processing or disk indexing. ingestion-service publishes to weather.telemetry.topic with routing key telemetry.raw.<region>.<stationId> and immediately returns an IngestionAck.

// Transport-Agnostic Controller: Exposing both gRPC and REST on the same logic
@Controller()
@UseInterceptors(LoggingInterceptor)
export class IngestionController {
  constructor(private readonly ingestionService: IngestionService) {}

  // 1. gRPC Unary Endpoint (:50052)
  @GrpcMethod('WeatherIngestionService', 'SubmitTelemetry')
  async submitTelemetryGrpc(data: TelemetryPayload): Promise<IngestionAck> {
    return this.ingestionService.processTelemetry(data);
  }

  // 2. HTTP REST Gateway Fallback (:3000)
  @Post('api/v1/telemetry')
  @HttpCode(HttpStatus.OK)
  async submitTelemetryHttp(@Body() body: TelemetryPayload): Promise<IngestionAck> {
    return this.ingestionService.processTelemetry(body);
  }
}

3.2 The Hybrid Persistence Strategy: TypeORM vs. Raw `pg.Pool`

One of the most impactful technical choices we made was refusing to use a single database access pattern everywhere:

Relational Metadata (station-service) $\rightarrow$ TypeORM: Station registry queries (findById, updateInterval) are low-frequency relational operations. Here, TypeORM (DataSource and Repository<StationRecord>) gives us clean @Entity('stations') decorators, type safety, and maintainable schema evolution.
Time-Series Telemetry (processing-service) $\rightarrow$ Raw pg.Pool Micro-Batches: During a 60x storm surge, processing-service commits 500 rows every 250ms into our TimescaleDB hypertable (weather_metrics). Using an ORM on this hot path would instantiate 500 class instances per batch, run reflection metadata, and trigger heavy V8 garbage collection pauses. Instead, we build a single parameterized multi-row SQL INSERT and stream it directly over the PostgreSQL wire protocol:

// High-throughput parameterized multi-row INSERT in TimescaleRepository
const valueClauses: string[] = [];
const values: any[] = [];
let paramIndex = 1;

for (const item of batch) {
  const date = new Date(item.timestamp || Date.now()).toISOString();
  valueClauses.push(
    `($${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++}, $${paramIndex++})`
  );
  values.push(
    date, item.station_id, item.temperature, item.humidity,
    item.pressure, item.wind_speed, item.wind_direction,
    item.precipitation, item.battery_voltage || 12.0
  );
}

const query = `
  INSERT INTO weather_metrics 
    (time, station_id, temperature, humidity, pressure, wind_speed, wind_direction, precipitation, battery_voltage)
  VALUES ${valueClauses.join(',\n')}
`;
await this.pool.query(query, values);

Combined with TimescaleDB 1-day hypertable chunks and Continuous Aggregates (weather_hourly_summary), historical queries over millions of rows return in milliseconds.

3.3 The Closed-Loop Dynamic Frequency Controller

How does the system react automatically when extreme weather hits?

[Station: 1005 hPa] ──► [ProcessingConsumer: AnomalyDetector (ΔP >= 3.0 hPa)]
                                │
                                ▼ (Publishes `weather.anomaly.detected`)
                        [AlertingConsumer]
                                │
                                ▼ (Direct gRPC: UpdateReportingFrequency)
                        [StationService: Sets interval = 1s]

Inside processing-service, the AnomalyDetector tracks a sliding window of barometric pressure and wind gusts per station. If pressure drops by $\ge 3.0\text{ hPa}$ or wind speed exceeds $30\text{ m/s}$, it emits a critical event to RabbitMQ. alerting-service consumes that event and invokes station-service via gRPC to shift the station's reporting interval from 60s down to 1s.

As the 60x wave of 1-second packets hits RabbitMQ, KEDA (Kubernetes Event-driven Autoscaling) monitors the weather.raw.telemetry queue depth (queueLength > 500) and horizontally scales processing-service pods from 2 to 20 replicas.

3.4 Designing for Failure: Circuit Breakers, Backpressure & Poison Pills

A cloud-native system is only as good as its worst failure mode. We baked in four layers of defense:

Opossum Circuit Breakers on gRPC: Every gRPC call from ingestion-service to station-service is wrapped in an Opossum circuit breaker (timeout: 2000ms, errorThresholdPercentage: 50%, resetTimeout: 5000ms). If station-service crashes, the circuit trips to OPEN and immediately executes a fallback handler that validates the station ID format (/^ST-[A-Z0-9-]+$/) and accepts the telemetry in degraded mode—resulting in zero client timeouts or HTTP 500 errors.
RabbitMQ Publisher Confirms & Consumer Prefetch: ingestion-service uses AMQP ConfirmChannel so a station only receives an ACK once the broker confirms persistence. Downstream, processing-service sets channel.prefetch(200) and noAck: false, guaranteeing that a slow database never causes out-of-memory crashes in Node.js workers.
Dead Letter Exchanges (DLX / DLQ): If a malfunctioning sensor sends corrupted JSON or unprocessable frames ("poison pills"), processing-service rejects the message with channel.nack(msg, false, false), routing it straight to weather.telemetry.dlq so it never blocks healthy telemetry.
Power-Aware Telemetry (battery_voltage): Remote alpine and buoy stations run on solar-charged batteries. Because 1-second burst mode drastically increases radio transmission power draw, every telemetry frame includes battery_voltage so operators can monitor discharge curves and prevent stations from going dark mid-storm.

4. Proving It Works: Load Testing (k6) & Chaos Engineering

Claims about scalability and resilience mean nothing without automated verification. We built two dedicated test suites into the repository:

1. 60x Dynamic Spike Simulation (`test/k6/scenarios/dynamic-frequency-spike.js`)

Using Grafana k6's ramping-arrival-rate executor, we simulate calm baseline traffic (20 req/s), ramp to a 600 req/s storm front, sustain a 60x burst (1,200 req/s), and recover—enforcing strict SLO thresholds:

p(95) < 50ms and p(99) < 150ms ingestion latency
< 0.1% error rate

2. Automated Chaos Suite (`test/chaos/run-chaos-suite.sh`)

Circuit Breaker Chaos Test: Floods the gateway while station-service is offline, verifying seamless fallback responses.
Poison Pill DLQ Test: Injects impossible physical readings (e.g., -500°C temperature, 150% humidity, missing station_id) and verifies ingress rejection and DLQ isolation.

5. Bonus: AI-Native Repository Ergonomics (`AGENTS.md` & `.agents/skills`)

Because we built and evolved this system alongside AI coding agents, we structured the repository with first-class AI Agent Customizations:

AGENTS.md: Always-on architectural guardrails (enforcing monorepo isolation between apps/*, mandatory circuit breakers on gRPC, and hybrid persistence rules).
.agents/skills/: On-demand runbooks (update-grpc-contract, scaffold-microservice, run-verification-suite) so any developer or AI assistant can extend .proto contracts or run the chaos suite with zero guesswork.

Try It Yourself!

The entire project is open-source and boots locally in minutes with Docker Compose:

git clone git@github.com:linhtch90/weather-monitoring-system.git
cd weather-monitoring-system
npm install
docker compose up -d

# Start the 4 microservices in watch mode:
npm run start:station:dev
npm run start:ingestion:dev
npm run start:processing:dev
npm run start:alerting:dev

🔗 GitHub Repository: linhtch90/weather-monitoring-system

What patterns do you use to handle sudden IoT traffic spikes or hybrid SQL/Time-Series workloads? Let’s discuss in the comments!

Build a microservices system with Java Spring Boot

Linh Truong Cong Hong — Sun, 06 Feb 2022 14:36:19 +0000

Github repo for this project: (Please give me a Github star ⭐ for this effort! Thank you!)

https://github.com/linhtch90/microservices-java-demo-zoo-devto

Introduction

Microservice is the concept of decomposing a large monolith application into small, independent, distributed services. This could improve the flexibility, scalability and resiliency of the whole system.

The introduction of Spring Cloud framework makes it easier for developers to build and deploy microservices to private and public clouds. This framework has wrapped a wide varieties of popular cloud-management microservices frameworks under a well-organized framework.

In this blog post, I want to help you have an overview about Spring Cloud framework by building a simple microservices system with Spring Cloud step by step. This system includes the following components:

Configuration server
Service discovery with Eureka
2 demo microservices called Tiger and Panther
Gateway server
RabbitMQ for messaging between microservices

The relationships between those services are described as the following figure.

Let's begin building this system!

Configuration server

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooconfsvr
Artifact ID: microservice-zoo-config-server
Name: microservice-zoo-config-server
Description: zoo microservices config server
JDK: I am using JDK 17 for this tutorial. I have not tested this project with older JDKs yet!
Dependencies: Config Server (So other services can get their configuration information from this service)

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
In folder src/main/resources/, add a folder called config/ and create 4 configuration files eureka-server.properties, gateway-server.properties, panther-service.properties, tiger-service.properties
The content of those .properties files could be found in my Github repo I put the link on top of this blog post)

Add @EnableConfigServer annotation to the main entrance class MicroserviceZooConfigServerApplication.java.

Service discovery (with Eureka)

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooeurekasvr
Artifact ID: microservice-zoo-services-discovery
Name: microservice-zoo-services-discovery
Description: zoo microservices services discovery
Dependencies: Eureka server, config client

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooServicesDiscoveryApplication.java

Gateway server

Gateway server is the service that handles all request from client. Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zoogateway
Artifact ID: microservice-zoo-gateway
Name: microservice-zoo-gateway
Description: microservice gateway for the zoo demo
Dependencies: Eureka client, config client, gateway

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-gateway</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooGatewayApplication.java

Tiger service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.tigerservice
Artifact ID: microservice-tiger
Name: microservice-tiger
Description: microservice tiger
Dependencies: Eureka client, config client, web, client load balancer, open feign (also for load balancer), RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-openfeign -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>3.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableFeignClients annotation to MicroserviceTigerApplication.java

Create com.microservices.tigerservice.service package and add 2 files ClientToPantherService.java for directly call Panther service and RabitMQSender.java for sending a string message to Panther via RabbitMQ.
Create com.microservices.tigerservice.controller package and add TigerController.java, specifying controllers for calling Panther service.
Create com.microservices.tigerservice.component package and add RabitMQSenderComponent.java for creating Beans related to messaging service.
Please refer to my Github repo for the content of these files.

Panther service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.pantherservice
Artifact ID: microservice-panther
Name: microservice-panther
Description: microservice panther
Dependencies: Eureka client, config client, web, client load balancer, RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Create controller and other components for the services as follow.

Create com.microservices.pantherservice.controller package and add TigerCallPanther.java for directly respond to calls from Tiger service in the form of a string.
Create com.microservices.pantherservice.component package and add 2 files RabbitMQComponent.java and RabbitMQReceiver.java for settings related to listening and responding to messages sent from Tiger service. Right after receiving the message from Tiger service, Panther service print the message into the Log.
Please refer to my Github repo for the content of these files.

RabbitMQ

Please refer to this tutorial for using RabbitMQ with Docker.

Running the whole system

Please follow this order:

Start RabbitMQ with Docker (see above section)
Start Configuration server
Start Discovery service
Start Tiger service
Start Panther service
Start Gateway server

Testing microservices with Postman

Direct call from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/callpantherservice

The response from Panther service is a string with content.

PANTHER RECEIVED CALL FROM TIGER

Send message from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/sendmessagetopanther

There is not any response in Postman. It is okay. Please go to the terminal screen where you are running Panther service. You will see a log message saying:

Received message is: Message from Tiger to Panther via RabbitMQ

And that is all everybody!

If you feel this project is useful for you, please give my project a Github star ⭐ as an encouragement!

Thank you so much!

Microservices with Java Spring Boot

Linh Truong Cong Hong — Sat, 05 Feb 2022 15:04:06 +0000

Github repo for this project:

https://github.com/linhtch90/microservices-java-demo-zoo-devto

Introduction

Configuration server
Service discovery with Eureka
2 demo microservices called Tiger and Panther
Gateway server
RabbitMQ for messaging between microservices

The relationships between those services are described as the following figure.

Let's begin building this system!

Configuration server

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooconfsvr
Artifact ID: microservice-zoo-config-server
Name: microservice-zoo-config-server
Description: zoo microservices config server
JDK: I am using JDK 17 for this tutorial. I have not tested this project with older JDKs yet!
Dependencies: Config Server (So other services can get their configuration information from this service)

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
In folder src/main/resources/, add a folder called config/ and create 4 configuration files eureka-server.properties, gateway-server.properties, panther-service.properties, tiger-service.properties
The content of those .properties files could be found in my Github repo I put the link on top of this blog post)

Add @EnableConfigServer annotation to the main entrance class MicroserviceZooConfigServerApplication.java.

Service discovery (with Eureka)

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooeurekasvr
Artifact ID: microservice-zoo-services-discovery
Name: microservice-zoo-services-discovery
Description: zoo microservices services discovery
Dependencies: Eureka server, config client

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooServicesDiscoveryApplication.java

Gateway server

Gateway server is the service that handles all request from client. Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zoogateway
Artifact ID: microservice-zoo-gateway
Name: microservice-zoo-gateway
Description: microservice gateway for the zoo demo
Dependencies: Eureka client, config client, gateway

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-gateway</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooGatewayApplication.java

Tiger service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.tigerservice
Artifact ID: microservice-tiger
Name: microservice-tiger
Description: microservice tiger
Dependencies: Eureka client, config client, web, client load balancer, open feign (also for load balancer), RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-openfeign -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>3.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableFeignClients annotation to MicroserviceTigerApplication.java

Create com.microservices.tigerservice.service package and add 2 files ClientToPantherService.java for directly call Panther service and RabitMQSender.java for sending a string message to Panther via RabbitMQ.
Create com.microservices.tigerservice.controller package and add TigerController.java, specifying controllers for calling Panther service.
Create com.microservices.tigerservice.component package and add RabitMQSenderComponent.java for creating Beans related to messaging service.
Please refer to my Github repo for the content of these files.

Panther service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.pantherservice
Artifact ID: microservice-panther
Name: microservice-panther
Description: microservice panther
Dependencies: Eureka client, config client, web, client load balancer, RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Create controller and other components for the services as follow.

Create com.microservices.pantherservice.controller package and add TigerCallPanther.java for directly respond to calls from Tiger service in the form of a string.
Create com.microservices.pantherservice.component package and add 2 files RabbitMQComponent.java and RabbitMQReceiver.java for settings related to listening and responding to messages sent from Tiger service. Right after receiving the message from Tiger service, Panther service print the message into the Log.
Please refer to my Github repo for the content of these files.

RabbitMQ

Please refer to this tutorial for using RabbitMQ with Docker.

Running the whole system

Please follow this order:

Start RabbitMQ with Docker (see above section)
Start Configuration server
Start Discovery service
Start Tiger service
Start Panther service
Start Gateway server

Testing microservices with Postman

Direct call from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/callpantherservice

The result should look like the following figure.

Send message from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/sendmessagetopanther

There is not any response in Postman. It is okay. Please go to the terminal screen where you are running Panther service. You will see a log message saying:

Received message is: Message from Tiger to Panther via RabbitMQ

And that is all everybody!

If you feel this project is useful for you, please give my project a Github star ⭐ as an encouragement!

Thank you so much!

Build a microservices system with Java

Linh Truong Cong Hong — Sat, 05 Feb 2022 11:47:12 +0000

Github repo for this project: (Please give me a Github star ⭐ for this effort! Thank you!)
https://github.com/linhtch90/microservices-java-demo-zoo-devto

Introduction

Configuration server
Service discovery with Eureka
2 demo microservices called Tiger and Panther
Gateway server
RabbitMQ for messaging between microservices

The relationships between those services are described as the following figure.

Let's begin building this system!

Configuration server

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooconfsvr
Artifact ID: microservice-zoo-config-server
Name: microservice-zoo-config-server
Description: zoo microservices config server
JDK: I am using JDK 17 for this tutorial. I have not tested this project with older JDKs yet!
Dependencies: Config Server (So other services can get their configuration information from this service)

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
In folder src/main/resources/, add a folder called config/ and create 4 configuration files eureka-server.properties, gateway-server.properties, panther-service.properties, tiger-service.properties
The content of those .properties files could be found in my Github repo I put the link on top of this blog post)

Add @EnableConfigServer annotation to the main entrance class MicroserviceZooConfigServerApplication.java.

Service discovery (with Eureka)

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zooeurekasvr
Artifact ID: microservice-zoo-services-discovery
Name: microservice-zoo-services-discovery
Description: zoo microservices services discovery
Dependencies: Eureka server, config client

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooServicesDiscoveryApplication.java

Gateway server

Gateway server is the service that handles all request from client. Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.zoogateway
Artifact ID: microservice-zoo-gateway
Name: microservice-zoo-gateway
Description: microservice gateway for the zoo demo
Dependencies: Eureka client, config client, gateway

Your pom.xml should look like this:

<properties>
        <java.version>17</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-gateway</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableEurekaServer annotation to MicroserviceZooGatewayApplication.java

Tiger service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.tigerservice
Artifact ID: microservice-tiger
Name: microservice-tiger
Description: microservice tiger
Dependencies: Eureka client, config client, web, client load balancer, open feign (also for load balancer), RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-openfeign -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>3.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Add @EnableFeignClients annotation to MicroserviceTigerApplication.java

Create com.microservices.tigerservice.service package and add 2 files ClientToPantherService.java for directly call Panther service and RabitMQSender.java for sending a string message to Panther via RabbitMQ.
Create com.microservices.tigerservice.controller package and add TigerController.java, specifying controllers for calling Panther service.
Create com.microservices.tigerservice.component package and add RabitMQSenderComponent.java for creating Beans related to messaging service.
Please refer to my Github repo for the content of these files.

Panther service

Go to start.spring.io and generate a new project with the following information:

Group ID: com.microservices.pantherservice
Artifact ID: microservice-panther
Name: microservice-panther
Description: microservice panther
Dependencies: Eureka client, config client, web, client load balancer, RabbitMQ

Your pom.xml should look like this:

<dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

In folder src/main/resources/, add a file called bootstrap.properties
The content of those .properties files could be found in my Github repo (I put the link on top of this blog post)

Create controller and other components for the services as follow.

Create com.microservices.pantherservice.controller package and add TigerCallPanther.java for directly respond to calls from Tiger service in the form of a string.
Create com.microservices.pantherservice.component package and add 2 files RabbitMQComponent.java and RabbitMQReceiver.java for settings related to listening and responding to messages sent from Tiger service. Right after receiving the message from Tiger service, Panther service print the message into the Log.
Please refer to my Github repo for the content of these files.

RabbitMQ

Please refer to this tutorial for using RabbitMQ with Docker.

Running the whole system

Please follow this order:

Start RabbitMQ with Docker (see above section)
Start Configuration server
Start Discovery service
Start Tiger service
Start Panther service
Start Gateway server

Testing microservices with Postman

Direct call from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/callpantherservice

The result should look like the following figure.

Send message from Tiger service to Panther service: Open Postman and send a request to

http://localhost:8074/tiger-service/tiger/sendmessagetopanther

There is not any response in Postman. It is okay. Please go to the terminal screen where you are running Panther service. You will see a log message saying:

Received message is: Message from Tiger to Panther via RabbitMQ

And that is all everybody!

If you feel this project is useful for you, please give my project a Github star ⭐ as an encouragement!

Thank you so much!

How to get start understanding source code of open source projects?

Linh Truong Cong Hong — Sat, 29 Jan 2022 09:33:57 +0000

Hello,

Open source software is awesome and important for the industry. However, whenever I try to read some code for an open source project with the intent to contribute, I get scared. I get this weird anxiety and for some reason my brain freezes. I can't understand anything what the code is doing and how it's working, it looks too complicated even if I kinda understand what things are supposed to do.

I can't describe the feeling. Even if I know the language and I've learned a lot.

How do I start?

Clone Twitter functions with ReactJS

Linh Truong Cong Hong — Wed, 26 Jan 2022 08:38:04 +0000

I have developed a small social network web app for educational purpose and want to share this with the community. This app is built with ReactJS for frontend, Java Spring Boot for backend and MongoDB as the database.

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height) User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

For source code and demo, please visit my Github repo:

https://github.com/linhtch90/psn-social-network-public-source

Please give me a ⭐ Github star ⭐ for my effort if you think this project is useful for you!

Develop a Twitter clone with ReactJS

Linh Truong Cong Hong — Fri, 21 Jan 2022 17:17:25 +0000

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height) User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

Please give me a ⭐ Github star ⭐ for my effort if you think this project is useful for you!

Live Demo and Source Code:

https://github.com/linhtch90/psn-social-network-public-source

Develope a twitter clone with ReactJS

Linh Truong Cong Hong — Tue, 18 Jan 2022 05:03:14 +0000

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height) User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

Hope you enjoy it and give me a ⭐ Github star ⭐ for my effort!

Live Demo and Source Code:

https://github.com/linhtch90/psn-social-network-public-source

Code a social network with ReactJS

Linh Truong Cong Hong — Mon, 17 Jan 2022 08:55:12 +0000

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height) User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

Hope you enjoy it and give me a ⭐ Github star ⭐ for my effort!

Live Demo:
https://psn-social-network.herokuapp.com/#/

Source Code:
https://github.com/linhtch90/psn-social-network-public-source

Build a social network app with ReactJS, Spring Boot and MongoDB

Linh Truong Cong Hong — Fri, 14 Jan 2022 02:07:30 +0000

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height)
User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

If this project is useful for you , please give me a Github star ⭐ for my effort!

Source Code:
https://github.com/linhtch90/psn-social-network-public-source

Live Demo:
https://psn-social-network.herokuapp.com/#/

How to build a social network app with ReactJS

Linh Truong Cong Hong — Thu, 13 Jan 2022 07:51:38 +0000

The app tries to mimic a few functions of Twitter, including:

Create post with text and image
The number of characters in each post is limited to 200 characters
Like, comment and re-post a post
View the number of likes, comments and shares of a post
Comments are toggle
The number of character in each comment is limited to 100 characters
Follow, un-follow someone
See who are following us and follow them back
Check who we are following
View all posts of an individual
View our own posts
List all user accounts in the entire social network (due to the number of users are still small, I did not develop search user function)

Other functions:

Due to limited storage space, client app compresses the uploaded image to the maximum of 250px (width or height)
User avatar is created by hashing user id and convert that hash code into a hexagonal image
Authentication and authorization use JWT
Responsive frontend for desktop and mobile devices

Technical stack:

Frontend: ReactJS, Redux Toolkit, React Router, React Bootstrap
Backend: Java Spring Boot
Database: MongoDB

If this project is useful for you , please give me a ⭐ Github star ⭐ for my effort!

Source Code:
https://github.com/linhtch90/psn-social-network-public-source

DEV Community: Linh Truong Cong Hong

Battle-Testing an IoT Telemetry Pipeline: Load Testing 60x Storm Spikes & Chaos Engineering with Grafana k6, KEDA, and Opossum

1. Why Standard Load Tests Fail for Weather Telemetry

2. Scalability Testing with Grafana k6

Scenario A: The 60x Dynamic Frequency Spike (ramping-arrival-rate)

Scenario B: Horizontal Fleet Growth (500 to 5,000 Concurrent Stations)

3. How the Architecture Absorbs the Load: Autoscaling & Micro-Batching

1. Stateless Ingress Scaling via Kubernetes HPA

2. Event-Driven Consumer Scaling via KEDA

3. Bounded Consumer Backpressure (prefetch: 200)

4. Multi-Row Micro-Batching into TimescaleDB

4. Availability & Chaos Engineering: Breaking Things on Purpose

Experiment 1: Station Service Outage & Circuit Breaker Verification

What Happens During the Chaos Run:

Experiment 2: Poison-Pill Injection & Dead Letter Queue (DLQ) Isolation

5. Run the Verification Suite Yourself!

Step 1: Clone & Start the Stack

Step 2: Run Unit & Contract Verification

Step 3: Run the Chaos & Resilience Suite

Step 4: Trigger the 60x Storm Load Test with Grafana k6

Key Takeaways

Designing a Cloud-Native Weather Telemetry System: Handling 60x Storm Traffic Spikes with NestJS, gRPC, RabbitMQ & TimescaleDB

1. The Core Requirements vs. Architectural Decisions

2. High-Level System Architecture

3. Deep Dive: Key Engineering Best Practices

3.1 Dual Communication: Why Both gRPC and RabbitMQ?

3.2 The Hybrid Persistence Strategy: TypeORM vs. Raw pg.Pool

3.3 The Closed-Loop Dynamic Frequency Controller

3.4 Designing for Failure: Circuit Breakers, Backpressure & Poison Pills

4. Proving It Works: Load Testing (k6) & Chaos Engineering

1. 60x Dynamic Spike Simulation (test/k6/scenarios/dynamic-frequency-spike.js)

2. Automated Chaos Suite (test/chaos/run-chaos-suite.sh)

5. Bonus: AI-Native Repository Ergonomics (AGENTS.md & .agents/skills)

Try It Yourself!

Build a microservices system with Java Spring Boot

Introduction

Configuration server

Service discovery (with Eureka)

Gateway server

Tiger service

Panther service

RabbitMQ

Running the whole system

Testing microservices with Postman

Microservices with Java Spring Boot

Introduction

Configuration server

Service discovery (with Eureka)

Gateway server

Tiger service

Panther service

RabbitMQ

Running the whole system

Testing microservices with Postman

Build a microservices system with Java

Introduction

Configuration server

Service discovery (with Eureka)

Gateway server

Tiger service

Panther service

RabbitMQ

Running the whole system

Testing microservices with Postman

How to get start understanding source code of open source projects?

Clone Twitter functions with ReactJS

Develop a Twitter clone with ReactJS

Develope a twitter clone with ReactJS

Code a social network with ReactJS

Build a social network app with ReactJS, Spring Boot and MongoDB

How to build a social network app with ReactJS

Scenario A: The 60x Dynamic Frequency Spike (`ramping-arrival-rate`)

Scenario B: Horizontal Fleet Growth (`500` to `5,000` Concurrent Stations)

3. Bounded Consumer Backpressure (`prefetch: 200`)

3.2 The Hybrid Persistence Strategy: TypeORM vs. Raw `pg.Pool`

1. 60x Dynamic Spike Simulation (`test/k6/scenarios/dynamic-frequency-spike.js`)

2. Automated Chaos Suite (`test/chaos/run-chaos-suite.sh`)

5. Bonus: AI-Native Repository Ergonomics (`AGENTS.md` & `.agents/skills`)