DEV Community: Manash Jyoti Baruah

Find Your Optimal PostgreSQL Connection Pool Size with k6 Load Testing

Manash Jyoti Baruah — Sat, 08 Nov 2025 21:50:00 +0000

A step-by-step guide to identifying and fixing connection pool bottlenecks in Node.js applications

How I reduced API response time from 1,221ms to 280ms using systematic load testing and pool optimization

What You'll Learn

In this guide, you'll discover how to:

Identify connection pool bottlenecks in your Node.js application
Calculate the optimal pool size using load testing with k6
Monitor pool health with simple metrics
Fix performance issues that cause 1000ms+ response times

Let's start with a real production issue I encountered and how I solved it systematically.

The Problem: Slow Response Times with Low CPU Usage

During a routine load test of my Node.js API with 150 concurrent users, I observed these concerning metrics:

Node.js CPU: 45%
RAM: 60%
PostgreSQL CPU: 15%
Average Response Time: 1,221ms
P95 Latency: 2,890ms

The symptom was clear: Database CPU at 15% while response times exceeded 1 second indicated a bottleneck between the application and database layers, not a database performance issue.

Step 1: Add Connection Pool Monitoring

First, I added monitoring to understand what's happening inside the connection pool. This single line of code revealed everything:

// Monitor pool health every 5 seconds
setInterval(() => {
  console.log(
    `[POOL] Total: ${pool.totalCount} | Idle: ${pool.idleCount} | Waiting: ${pool.waitingCount}`
  );
}, 5000);

Key Metrics Explained:

Total: Number of connections in the pool
Idle: Available connections ready to use
Waiting: Requests queued because all connections are busy

Basic Server Setup with Monitoring

import dotenv from "dotenv";
import express from "express";
import { Pool } from "pg";

dotenv.config();

const app = express();
const PORT = process.env.PORT || 3000;

const poolConfig = {
  host: process.env.DB_HOST,
  port: process.env.DB_PORT,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,

  // These are what we're testing
  min: parseInt(process.env.POOL_MIN) || 2,
  max: parseInt(process.env.POOL_MAX) || 10,

  // Important timeouts
  idleTimeoutMillis: parseInt(process.env.POOL_IDLE_TIMEOUT) || 30000,
  connectionTimeoutMillis: parseInt(process.env.POOL_CONNECTION_TIMEOUT) || 2000,
};

const pool = new Pool(poolConfig);
const WORKER_ID = process.pid;

console.log("=================================");
console.log(`[Worker-${WORKER_ID}]`);
console.log(`  Host: ${poolConfig.host}`);
console.log(`  Database: ${poolConfig.database}`);
console.log(`  Pool Min: ${poolConfig.min}`);
console.log(`  Pool Max: ${poolConfig.max}`);
console.log("=================================\n");

// Test database connection
pool.query("SELECT NOW()", (err, res) => {
  if (err) {
    console.error(`[Worker-${WORKER_ID}] Database connection failed:`, err.message);
    process.exit(1);
  }
  console.log(`[Worker-${WORKER_ID}] Database connected at:`, res.rows[0].now);
});

// API endpoint for testing
app.get("/users", async (req, res) => {
  const startTime = Date.now();

  try {
    const result = await pool.query("SELECT * FROM users LIMIT 100");
    const duration = Date.now() - startTime;

    res.json({
      success: true,
      count: result.rows.length,
      data: result.rows,
      query_time_ms: duration,
    });
  } catch (err) {
    const duration = Date.now() - startTime;
    res.status(500).json({
      success: false,
      error: err.message,
      query_time_ms: duration,
    });
  }
});

// The critical monitoring - logs every 5 seconds
setInterval(() => {
  console.log(
    `[Worker-${WORKER_ID}] [POOL] Total: ${pool.totalCount} | Idle: ${pool.idleCount} | Waiting: ${pool.waitingCount}`
  );
}, 5000);

// Graceful shutdown
process.on("SIGTERM", async () => {
  console.log("\nSIGTERM received, closing pool...");
  await pool.end();
  process.exit(0);
});

process.on("SIGINT", async () => {
  console.log("\nSIGINT received, closing pool...");
  await pool.end();
  process.exit(0);
});

app.listen(PORT, () => {
  console.log(`\n[Worker-${WORKER_ID}] Server running on http://localhost:${PORT}`);
  console.log(`[Worker-${WORKER_ID}] Users endpoint: http://localhost:${PORT}/users\n`);
});

Step 2: Create k6 Load Test

Create a realistic load test that simulates production traffic:

// load-test.js
import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  stages: [
    { duration: "10s", target: 10 },   // Warm up
    { duration: "20s", target: 30 },   // Ramp up
    { duration: "30s", target: 60 },   // Increase load
    { duration: "10s", target: 100 },  // Push harder
    { duration: "15s", target: 150 },  // Peak traffic
    { duration: "10s", target: 0 },    // Cool down
  ],
  thresholds: {
    http_req_duration: ["p(95)<2000"], // 95% of requests under 2s
    http_req_failed: ["rate<0.1"],     // Less than 10% errors
  },
};

export default function () {
  const res = http.get("http://localhost:3000/users");

  check(res, {
    "status is 200": (r) => r.status === 200,
    "response time < 500ms": (r) => r.timings.duration < 500,
  });

  sleep(0.1); // Each user waits 100ms between requests
}

export function handleSummary(data) {
  const metrics = data.metrics;
  console.log("\n========================================");
  console.log("LOAD TEST RESULTS");
  console.log("========================================");
  console.log(`Total Requests: ${metrics.http_reqs.values.count}`);
  console.log(`Average Response: ${metrics.http_req_duration.values.avg.toFixed(2)}ms`);
  console.log(`P95 Latency: ${metrics.http_req_duration.values["p(95)"].toFixed(2)}ms`);
  console.log(`Failed Requests: ${(metrics.http_req_failed.values.rate * 100).toFixed(2)}%`);
  console.log("========================================\n");
  return { stdout: "" };
}

Step 3: Test Different Pool Sizes Systematically

I tested each pool size by editing the .env file and restarting the server:

`.env` Configuration

# Database configuration
DB_HOST=localhost
DB_PORT=5432
DB_NAME=testdb
DB_USER=postgres
DB_PASSWORD=password

# Pool configuration - Edit POOL_MAX for each test
POOL_MIN=1
POOL_MAX=10  # Change to: 10, 15, 20, 40
POOL_IDLE_TIMEOUT=30000
POOL_CONNECTION_TIMEOUT=10000

Testing Workflow

For each pool size test:

Edit .env file - Change POOL_MAX value
Start the server - npm run dev
Run load test - k6 run load-tests/test.js (in another terminal)
Collect results - Watch the console logs and k6 output
Stop server - Ctrl+C and repeat with next pool size

# Step 1: Set POOL_MAX=10 in .env file
# Step 2: Start server
npm run dev

# Step 3: In another terminal, run k6
k6 run load-tests/test.js

# Step 4: Note the results
# Step 5: Stop server (Ctrl+C)

# Repeat: Edit .env to POOL_MAX=15
npm run dev
k6 run load-tests/test.js

# Repeat: Edit .env to POOL_MAX=20
npm run dev
k6 run load-tests/test.js

# Repeat: Edit .env to POOL_MAX=40
npm run dev
k6 run load-tests/test.js

Step 4: Analyse Test Results

Test #1: Pool Size 10 (Baseline)

Server Logs During Load:

[Worker-24064] [POOL] Total: 1 | Idle: 0 | Waiting: 0
[Worker-24064] [POOL] Total: 6 | Idle: 1 | Waiting: 0
[Worker-24064] [POOL] Total: 10 | Idle: 3 | Waiting: 0
[Worker-24064] [POOL] Total: 10 | Idle: 1 | Waiting: 0
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 11
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 14
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 22
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 24
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 36
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 37
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 58
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 53
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 69
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 72
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 105
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 123
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 130   PEAK QUEUE
[Worker-24064] [POOL] Total: 10 | Idle: 0 | Waiting: 54
[Worker-24064] [POOL] Total: 10 | Idle: 6 | Waiting: 0
[Worker-24064] [POOL] Total: 10 | Idle: 10 | Waiting: 0
[Worker-24064] [POOL] Total: 0 | Idle: 0 | Waiting: 0

k6 Results:

Test #2: Pool Size 15

Server Logs:

[Worker-29532] [POOL] Total: 1 | Idle: 1 | Waiting: 0
[Worker-29532] [POOL] Total: 5 | Idle: 1 | Waiting: 0
[Worker-29532] [POOL] Total: 8 | Idle: 3 | Waiting: 0
[Worker-29532] [POOL] Total: 11 | Idle: 1 | Waiting: 0
[Worker-29532] [POOL] Total: 15 | Idle: 4 | Waiting: 0
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 0
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 3
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 11
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 16
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 20
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 24
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 30
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 34
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 55
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 72
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 89
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 104
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 120   PEAK QUEUE
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 81
[Worker-29532] [POOL] Total: 15 | Idle: 0 | Waiting: 44
[Worker-29532] [POOL] Total: 15 | Idle: 15 | Waiting: 0
[Worker-29532] [POOL] Total: 14 | Idle: 14 | Waiting: 0
[Worker-29532] [POOL] Total: 0 | Idle: 0 | Waiting: 0

k6 Results:

Test #3: Pool Size 20

Server Logs:

[Worker-31608] [POOL] Total: 1 | Idle: 1 | Waiting: 0
[Worker-31608] [POOL] Total: 4 | Idle: 2 | Waiting: 0
[Worker-31608] [POOL] Total: 8 | Idle: 4 | Waiting: 0
[Worker-31608] [POOL] Total: 14 | Idle: 2 | Waiting: 0
[Worker-31608] [POOL] Total: 18 | Idle: 1 | Waiting: 0
[Worker-31608] [POOL] Total: 20 | Idle: 8 | Waiting: 0
[Worker-31608] [POOL] Total: 20 | Idle: 3 | Waiting: 0
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 2
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 10
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 18
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 24
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 36
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 48
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 66
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 86
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 99
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 114   PEAK
[Worker-31608] [POOL] Total: 20 | Idle: 0 | Waiting: 60
[Worker-31608] [POOL] Total: 20 | Idle: 8 | Waiting: 0
[Worker-31608] [POOL] Total: 20 | Idle: 20 | Waiting: 0    RECOVERED
[Worker-31608] [POOL] Total: 8 | Idle: 8 | Waiting: 0
[Worker-31608] [POOL] Total: 0 | Idle: 0 | Waiting: 0

k6 Results:

Test #4: Pool Size 40

Server Logs:

[Worker-31644] [POOL] Total: 1 | Idle: 1 | Waiting: 0
[Worker-31644] [POOL] Total: 4 | Idle: 0 | Waiting: 0
[Worker-31644] [POOL] Total: 9 | Idle: 0 | Waiting: 0
[Worker-31644] [POOL] Total: 13 | Idle: 2 | Waiting: 0
[Worker-31644] [POOL] Total: 21 | Idle: 4 | Waiting: 0
[Worker-31644] [POOL] Total: 27 | Idle: 8 | Waiting: 0
[Worker-31644] [POOL] Total: 32 | Idle: 12 | Waiting: 0
[Worker-31644] [POOL] Total: 38 | Idle: 1 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 4 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 6 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 13 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 12
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 36
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 51
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 65
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 88   PEAK (still queuing!)
[Worker-31644] [POOL] Total: 40 | Idle: 0 | Waiting: 19
[Worker-31644] [POOL] Total: 40 | Idle: 36 | Waiting: 0
[Worker-31644] [POOL] Total: 40 | Idle: 40 | Waiting: 0   RECOVERED
[Worker-31644] [POOL] Total: 7 | Idle: 7 | Waiting: 0
[Worker-31644] [POOL] Total: 0 | Idle: 0 | Waiting: 0

k6 Results:

Analysis:

Best response times: 280ms average (77% improvement from baseline)
Highest throughput: 13,680 requests (239% increase)
Still experienced queuing: Peak of 88 waiting despite 40 connections
Diminishing returns: Only 59% improvement over Pool 20

Step 5: Performance Analysis

Results Summary

Pool Size	Total Requests	Avg Response	P95 Latency	Max Queue	Improvement
10	4,040	1,221ms	2,890ms	130	Baseline
15	6,969	654ms	1,545ms	120	+72.5%
20	8,605	508ms	1,171ms	114	+113% (Optimal)
40	13,680	280ms	529ms	88	+239%

Key Insights

10 → 15: Significant improvement (72.5% throughput increase, 46% latency reduction)
15 → 20: Strong continued gains (23.5% more throughput, 22% latency reduction)
20 → 40: Diminishing returns (59% more throughput but 2x the connections)

VERDICT: Pool Size 20 is Optimal

While Pool Size 40 delivers the best raw performance, Pool Size 20 provides the best balance:

Pool 20 advantages:
- 113% throughput improvement with just 20 connections
- Clears queue completely (recovers to 0)
- Uses 50% fewer database resources than Pool 40
- Already achieves sub-600ms response times
Pool 40 considerations:
- Only 59% better than Pool 20 despite 100% more connections
- Still experiences queuing (88 peak) during high load
- Higher database resource consumption
- Better suited if you expect >200 concurrent users

Step 6: Calculate Your Optimal Pool Size

Based on the testing, here's a formula to calculate your starting point:

function calculateOptimalPoolSize(concurrentUsers, avgQueryTimeMs) {
  // Base calculation
  const baseSize = (concurrentUsers * avgQueryTimeMs) / 1000;

  // Add 20% safety buffer
  const withBuffer = baseSize * 1.2;

  return Math.ceil(withBuffer);
}

// Example for our case
const poolSize = calculateOptimalPoolSize(150, 150);
console.log(poolSize); // Returns 27

// Testing showed 20 was sufficient because actual query time was faster

Important Constraints

// Check database limits
const dbMaxConnections = 100;  // from SHOW max_connections;
const reserveForAdmin = 10;
const availableForApp = dbMaxConnections - reserveForAdmin;

// If using multiple workers
const workers = 4;  // PM2 cluster mode
const poolPerWorker = Math.floor(availableForApp / workers);
console.log(`Each worker gets ${poolPerWorker} connections`);

Step 7: Production Implementation

Enhanced Monitoring Setup

const ALERT_THRESHOLDS = {
  waitingQueue: 20,
  idlePercentage: 10,
};

setInterval(() => {
  const total = pool.totalCount;
  const idle = pool.idleCount;
  const waiting = pool.waitingCount;
  const idlePercentage = total > 0 ? (idle / total) * 100 : 0;

  // Log metrics
  console.log(`[POOL] Total: ${total} | Idle: ${idle} | Waiting: ${waiting}`);

  // Alert conditions
  if (waiting > ALERT_THRESHOLDS.waitingQueue) {
    console.error(` [ALERT] High queue: ${waiting} requests waiting`);
    // Send to monitoring service
  }

  if (idlePercentage < ALERT_THRESHOLDS.idlePercentage && total > 0) {
    console.warn(` [ALERT] Low idle connections: ${idlePercentage.toFixed(1)}%`);
  }
}, 5000);

Database Monitoring Queries

-- Check current connections
SELECT count(*) as total,
       count(*) FILTER (WHERE state = 'active') as active,
       count(*) FILTER (WHERE state = 'idle') as idle
FROM pg_stat_activity
WHERE datname = current_database();

-- Find connection hogs
SELECT pid, usename, application_name, state,
       NOW() - query_start as duration,
       substring(query, 1, 50) as query_preview
FROM pg_stat_activity
WHERE state != 'idle'
  AND query_start < NOW() - INTERVAL '1 minute'
ORDER BY query_start;

Common Pitfalls to Avoid

1. Not Releasing Connections

//  BAD: Connection leak
const client = await pool.connect();
const result = await client.query("SELECT * FROM users");
res.json(result.rows);
// Forgot to release!

//  GOOD: Always release
const client = await pool.connect();
try {
  const result = await client.query("SELECT * FROM users");
  res.json(result.rows);
} finally {
  client.release();
}

//  BETTER: Use pool.query() for simple queries
const result = await pool.query("SELECT * FROM users");
res.json(result.rows);

2. Holding Connections During External Calls

//  BAD: Blocks connection for 2+ seconds
const client = await pool.connect();
const user = await client.query("SELECT * FROM users WHERE id = $1", [id]);
const apiData = await fetch("https://slow-api.com/data"); // 2 seconds!
await client.query("UPDATE users SET data = $1", [apiData]);
client.release();

//  GOOD: Release before external call
const user = await pool.query("SELECT * FROM users WHERE id = $1", [id]);
const apiData = await fetch("https://slow-api.com/data");
await pool.query("UPDATE users SET data = $1", [apiData]);

Quick Reference

Healthy Pool Indicators

Waiting < 10 consistently
Idle > 0 (at least 10% of total)
Response time P95 < 1000ms
No connection timeouts

When to Increase Pool Size

Waiting > 30 consistently
Idle always 0
Response times > 1000ms
Database CPU < 50% under load

Useful Commands

# Start server (uses .env file)
npm run dev

# Run load test (in another terminal)
k6 run load-tests/test.js

# PostgreSQL connections check
psql -c "SELECT count(*) FROM pg_stat_activity"

Conclusion

Connection pool optimization reduced our response time from 1,221ms to 280ms - a 77% improvement - through systematic testing with k6 load tests.

The Complete Performance Picture:

Metric	Pool 10	Pool 20 (Recommended)	Pool 40
Avg Response	1,221ms	508ms (-58%)	280ms (-77%)
P95 Latency	2,890ms	1,171ms (-59%)	529ms (-82%)
Throughput	4,040 req	8,605 req (+113%)	13,680 req (+239%)
Max Queue	130	114	88
Efficiency	Poor	Excellent	Good

Final Verdict: Choose Pool Size 20

Why Pool 20 over Pool 40?

Resource efficiency: 113% performance gain with just 20 connections
Diminishing returns: Pool 40 only adds 59% more throughput for 100% more connections
Cost-benefit: Pool 20 achieves the "good enough" threshold (<600ms response)
Database overhead: Fewer connections = less memory and CPU on PostgreSQL

Key Takeaways:

Default pool (10) is severely undersized - caused 1.2s response times
Pool 20 hits the sweet spot - doubles throughput, halves latency
Pool 40 shows diminishing returns - consider only for >200 concurrent users
Queue monitoring is critical - even Pool 40 had 88 requests waiting at peak

Remember: The default pool size of 10 resulted in 130 requests queuing. Simply changing to Pool 20 delivered a 2x throughput improvement with sub-second response times.

Resources:

📊 k6 documentation

Questions? Drop them in the comments. What pool size works for your application?

💡 Found this helpful?

Drop a ❤️ and 🦄 if this helped you. Follow for more practical guides and real-world dev experiences.

🤝 Let's connect:

💼 LinkedIn → linkedin.com/in/manashjb

📺 YouTube → youtube.com/@CoderInSpace

🚀 Portfolio → manash.dev

Stop Trying to Learn Everything -Focus on These 5 Key Skills Every Developer Needs

Manash Jyoti Baruah — Sat, 21 Dec 2024 17:47:18 +0000

You’re probably thinking, ‘But as a developer, don’t I have to know everything to be relevant?’ No, you do not. In reality, focussing on these five abilities will not only enhance your confidence but also help you advance your career.

Introduction

As a developer, I’ve learnt the hard way that trying to understand everything isn’t the same as improving professionally.

Early in my career, I wanted to know everything: every framework, backend language, and new JavaScript tool. I believed knowing everything would make me irreplaceable. But instead, I felt as if I was going in circles. I was familiar with everything, but it was getting me nowhere.

It became worse when I realised I lacked depth in a certain aspect of my work. I suffered from imposter syndrome and often doubted my talents. That’s when I changed my approach. I stopped attempting to do everything and began focussing on the truly important skills. And I’m going to share those abilities with you because I believe they’ll help you on your journey.

Prioritize What Matters

Developers have an endless supply of tools and frameworks to learn. Each one has its own learning curve. However, you don’t need them all. Ask yourself, what solves difficulties in my current role?

For example, I used to switch between React and Vue in order to remain relevant. However, most of my projects required React. So I worked on improving my React skills. Tasks were easier, and I achieved better results. Stop juggling and start prioritising; it will shift your approach to development.

Master the Art of Learning How to Learn

The technology world moves quickly, and it’s impossible to keep up with everything. The key is not to learn everything, but to learn how to learn.

Instead of spending hours watching tutorials, I now focus on applying what I’ve learned. For example, when I was learning Docker, I didn’t just watch videos; I also created a containerized app from scratch. The best way to learn is to get your hands dirty. If you want to stay relevant as a developer, concentrate on improving your ability to adapt and learn quickly.

Communication Skills Are Just as Important as Code

Let me ask you something: have you ever written brilliant code, only for the team to misunderstand its purpose or not use it? That’s where communication comes in.

I once had to explain to my team why the existing code for a feature needed to be rewritten instead of patched again. It kept producing bugs, and while patching appeared like a simple remedy, it didn’t address the underlying problem. After rewriting it, the reoccurring difficulties were resolved, and no new issues occurred due to that functionality. Being a successful developer is more than simply writing code; it’s about making judgements that avoid future hassles and communicating the “why” behind those decisions.

Consistency Over Intensity in Learning

Ever tried to learn an entire framework over a weekend, only to forget half of it by the next day? Been there, done that. Here’s what I’ve learned: consistent, small steps are more effective than rushing through everything.

For example, instead of trying to master TypeScript in one week, I decided to take it a little at a time and practice it every day. Before I knew it, I started feeling more confident and actually got a good grip on it! Consistency not only makes learning easier but also sticks with you longer.

Focus on Problem-Solving

No matter what framework or tool you use, your main goal should be to solve the problem. That’s what makes you a great developer. This is a skill that goes beyond trends and technologies.

For example, I once worked on a project, in which the existing codebase was an absolute mess. Fixing one issue often leads to another problem. So, instead of focusing on the trending tools I wanted to use, I approached the task by breaking the problem into smaller, manageable parts. It wasn’t anything fancy, but it worked and that’s what made the project a success.

Conclusion

Being a great developer isn’t about knowing everything. But it's important to know what matters and start focusing on that. You don’t need to remember everything. You just need to know how and where to find the right answers when you need them. Isn’t Google there for a reason? So, use it wisely!

The most important thing is knowing how to tackle a task and find effective solutions to the issues. A lasting foundation is established by prioritising the appropriate abilities, learning with intention, and maintaining consistency.

So, let me ask you: are you ready to stop chasing trends and work on what really matters? Believe me, the outcomes will be worth it.

P.S. I wrote this article while I was on the train, so it’s a bit of a brain dump, but sometimes those are the most honest thoughts, right? 😅I’ve always felt the need to approach learning this way and wanted to share it with you guys. Hope it helps!

If you find this article helpful, feel free to highlight, clap, leave a comment, or even reach out to me on Twitter/X and LinkedInas it’s very appreciated and helps keeps content like this free!

By the way, if you’re curious about JavaScript promises and want to dive deeper, I’ve got a detailed article on that too.

Check it out here: https://dev.to/manashjb/everything-you-need-to-know-about-javascript-promises-and-how-they-work-1apg.

Stop Trying to Learn Everything -Focus on These 5 Key Skills Every Developer Needs

Manash Jyoti Baruah — Sat, 21 Dec 2024 17:47:18 +0000

You’re probably thinking, ‘But as a developer, don’t I have to know everything to be relevant?’ No, you do not. In reality, focussing on these five abilities will not only enhance your confidence but also help you advance your career.

Introduction

As a developer, I’ve learnt the hard way that trying to understand everything isn’t the same as improving professionally.

Prioritize What Matters

Developers have an endless supply of tools and frameworks to learn. Each one has its own learning curve. However, you don’t need them all. Ask yourself, what solves difficulties in my current role?

Master the Art of Learning How to Learn

The technology world moves quickly, and it’s impossible to keep up with everything. The key is not to learn everything, but to learn how to learn.

Communication Skills Are Just as Important as Code

Let me ask you something: have you ever written brilliant code, only for the team to misunderstand its purpose or not use it? That’s where communication comes in.

Consistency Over Intensity in Learning

Focus on Problem-Solving

No matter what framework or tool you use, your main goal should be to solve the problem. That’s what makes you a great developer. This is a skill that goes beyond trends and technologies.

Conclusion

So, let me ask you: are you ready to stop chasing trends and work on what really matters? Believe me, the outcomes will be worth it.

By the way, if you’re curious about JavaScript promises and want to dive deeper, I’ve got a detailed article on that too.

Check it out here: https://dev.to/manashjb/everything-you-need-to-know-about-javascript-promises-and-how-they-work-1apg.

Everything You Need to Know About JavaScript Promises and How They Work

Manash Jyoti Baruah — Sat, 07 Dec 2024 09:00:07 +0000

Modern web development relies heavily on asynchronous activities to enable responsive, interactive applications. Whether it’s retrieving data from an API, reading files, or running timers, these processes must run in the background without freezing the interface. JavaScript offers you a reliable way to handle these jobs. This article covers all you need to know about promises, including basic ideas and advanced features, to develop error-free asynchronous programs.

In this article you will learn about —

What is a Promise?
Why Use Promises?
How Promises Work?
Handling Promises
Chaining Promises
Error Handling in Promises
Advanced Promise Features
JavaScript Execution Flow with Promises (Important)
Converting Promise Chains to Async/Await
Best Practices and Common Mistakes

What is a Promise?

A Promise in JavaScript is equivalent to making a “promise” to do something in the future. When you make a promise, you are saying, “I promise to give you the results later.” This outcome could be success or failure.

In other words, a promise is an object that reflects the ultimate success (or failure) of an asynchronous operation and its resultant value. It lets you to correlate handlers with the success or failure of an asynchronous action, making your code easier to read and maintainable.

Why Use Promises?

In JavaScript, for instance, time-consuming operations-like retrieving data from a server-were generally accomplished with callbacks. A callback is just a function passed to another function to execute after the task is completed. You might use a callback, for example, to process data when it arrives from a server.

However, when there are complex operations, the use of callbacks get pretty messy. This mess is known as “callback hell,” where one can have a callback within another, and this makes the code unreadable and unmanageable.

Callback Hell Example:

    fetchData((data) => {
      processData(data, (processedData) => {
        saveData(processedData, (result) => {
          console.log(result);
        });
      });
    });

As shown above, such code becomes increasingly difficult to read and maintain in larger codebases due to its deeply nested structure, often referred to as “callback hell.”

Promises were introduced to address this problem, offering a cleaner and more organized way to handle asynchronous tasks by allowing chaining in a more readable manner.

Promise-Based Approach:

fetchData()
  .then(processData)
  .then(saveData)
  .then(console.log)
  .catch(console.error);

This approach flattens the structure and makes the code more readable and maintainable.

How Promises Work?

Promises in JavaScript can be in one of three states:

Pending: This is the initial step. The promise is still yet to be fulfilled.
Fulfilled: The promise has completed successfully which means it is resolved and has a value.
Rejected: The promise did not complete successfully, and it carries an error message.

Basic Syntax

const myPromise = new Promise((resolve, reject) => {
  setTimeout(() => {
    resolve("Promise resolved!");
  }, 1000);
});

myPromise.then(result => console.log(result));

In this example, the promise resolves after 1 second with the message “Promise resolved!”. The.then() method is used to handle the resolved value.

Handling Promises

Using .then() for Success Handling

The.then()method is used to handle what happens when a promise is successfully completed. It registers functions (callbacks) to run when the promise is fulfilled.

myPromise.then(data => {
  console.log("Data received:", data);
});

Using .catch() for Error Handling

The.catch()method is used to handle what happens when a promise fails. It registers a function (callback) to run when the promise is rejected.

myPromise.catch(error => {
  console.error("Error:", error);
});

Using .finally() for Cleanup

The.finally()method lets you run some code after the promise is done, whether it was successful or not.

myPromise.finally(() => {
  console.log("Cleanup tasks");
});

Chaining Promises

Chaining allows you to perform tasks sequentially by passing the outcome of the previous one. Then proceed to the next.then(). This allows you to handle several asynchronous tasks sequentially.

Example of Chaining:

fetch('https://api.example.com/user')
  .then(response => response.json())
  .then(data => {
    console.log("Processed data:", data);
    return processData(data);
  })
  .then(finalResult => {
    console.log("Final result:", finalResult);
  })
  .catch(error => console.error("Error:", error));

This example uses each.then()to handle each step in the process, allowing for clear data flow. This allows you to see how the result of one stage is transferred to the next.

Error Handling in Promises

Promises simplify error handling by allowing them to pass down the chain to the.catch()method for resolution. This eliminates the need to handle failures at each phase, keeping your code clearer and easier to manage.

Example with Error Propagation:

fetchData()
  .then(processData)
  .then(saveData)
  .catch(error => console.error("An error occurred:", error));

If any step in the promise chain fails, the error will be caught by the.catch()block. This makes it easy to handle issues and keep your code running smoothly.

Advanced Promise Features

1. Promise.all() for Parallel Execution

The Promise.all()method allows you to run several promises simultaneously and wait for them all to complete. If all of the promises are fulfilled, you will receive the results of each one. If any promise fails, it detects the mistake.

Promise.all([fetchData1(), fetchData2(), fetchData3()])
  .then(results => {
    console.log("All data fetched:", results);
  })
  .catch(error => console.error("Error fetching data:", error));

In this example, if any promise fails, the entire Promise.all() fails.

2. Promise.race() for Fastest Promise

The Promise.race() method returns the result of the first promise that finishes, whether it succeeds or fails.

Promise.race([fetchData1(), fetchData2()])
  .then(result => console.log("First to finish:", result));

In this example, whichever promise (fetchData1 or fetchData2) completes first will have its result logged to the console.

3. Promise.allSettled() for Handling All Outcomes

The Promise.allSettled()method waits for all the promises you give it to be in a successful or failed state and then finish. An array is then returned that has the results of each promise.

Promise.allSettled([fetchData1(), fetchData2()])
  .then(results => {
    results.forEach(result => console.log(result.status, result.value || result.reason));
  });

In this example, Promise.allSettled() waits for both fetchData1() and fetchData2() to complete. It then logs the status and result (or error) of each promise. This way, you can see what happened with each promise, regardless of whether they succeeded or failed.

4.Promise.any() for Resolving with the First Successful Promise

The Promise.any() method waits for the first promise to be resolved correctly from a list of promises. In case at least one promise is resolved, the value will be returned by the Promise.any() method. If all promises are refused, this method will throw an error.

const promise1 = new Promise((resolve, reject) => setTimeout(reject, 100, 'Error 1'));
const promise2 = new Promise((resolve) => setTimeout(resolve, 200, 'Success A'));
const promise3 = new Promise((resolve, reject) => setTimeout(reject, 300, 'Error 3'));

Promise.any([promise1, promise2, promise3])
  .then((value) => console.log(value)) // logs: 'Success A'
  .catch((error) => console.error('All promises were rejected', error));

In this example, Promise.any() waits for the first promise to be resolved successfully. The procedure returns the outcome of the first successful promise, in this case promise2with the value ‘Success A’. If all promises are refused, the.catch() block is executed, logging the error message. This strategy is beneficial when you want to receive the result of the first successful promise without having to wait for the rest.

JavaScript Execution Flow with Promises (Important)

1. Promises in JavaScript run in the microtask queue, which gets priority over macrotasks like setTimeout.

Here’s an example to illustrate this:

console.log(2);

setTimeout(() => console.log(4), 0);

Promise.resolve().then(() => console.log(3));

console.log(6);

// 2
// 6
// 3
// 4

In this example:

console.log(2) runs first because it's a regular synchronous operation.
console.log (6) runs next because it's also synchronous.
The promise’s.then()runs before the setTimeout callback because promises are microtasks, which have higher priority, hence prints 3.
Finally, the setTimeout callback runs, as it's a macrotask and prints 4.

So always remember, the promise’s.then()executes before the setTimeout callback due to the microtask queue's priority.

2. Promise Execution Order and the Microtask Queue with Multiple .then() Calls

In JavaScript, code runs in a specific order: first the synchronous code, then microtasks (like promises), and finally, macrotasks (likesetTimeout).

Here’s one example to explain this:

console.log(3)
const promise = new Promise((resolve) => {
  console.log(6)
  resolve()
  console.log(2)
})

console.log(7)

promise.then(() => {
  console.log(1)
}).then(() => {
  console.log(9)
})

console.log(8)

setTimeout(() => {
  console.log(13)
}, 10)

setTimeout(() => {
  console.log(21)
}, 0)

// 3
// 6
// 2
// 7
// 8
// 1
// 9
// 21
// 13

In this example, synchronous code runs first, logging 3, 6, 2, 7, and 8. Once the synchronous code finishes, microtasks (the.then() callbacks) are processed, logging 1 and 9. Finally, macrotasks (from setTimeout) execute in order of their delays, logging 21 (0ms) and 13 (10ms). This highlights JavaScript's execution order: synchronous code > microtasks > macrotasks.

3. Multiple Resolve and Reject Calls in a Promise: Only the First One Matters

When you create a promise, the first call to resolve or reject is the only one that counts. All the other calls are dismissed.

Here’s an example to illustrate this:

new Promise((resolve, reject) => {
  resolve(1); // This will be the only one that counts
  resolve(2); // Ignored
  reject('error'); // Ignored
}).then((value) => {
  console.log(value); // Logs: 1
}, (error) => {
  console.log('error'); // This won't run
});

In this example, the promise is resolved with the value 1. The second resolve and the reject calls are ignored because the promise has already been settled with the first resolve.

4. Chaining Promises and Handling Values in Sequential .then() Calls

When you chain promises, each.then() handles a step in the process.

Promise.resolve(1) // 1
.then(() => 2) // 2 (here 1 goes to the .then method but isn't used)
.then(3) // skip
.then((value) => value * 3) // 2 * 3 = 6
.then(Promise.resolve(4)) // creates a Pending promise
.then(console.log) // console.log will display 6

In this example, Promise.resolve(1) starts with a value of 1, but the first .then(() => 2) returns 2 instead. The next .then(3) is ignored, and the value 2 is passed on. The .then((value) => value * 3) multiplies the value by 3, resulting in 6. The .then(Promise.resolve(4)) doesn’t change the value, and finally, .then(console.log) logs 6. This demonstrates how values are passed through the chain, with non-function values being ignored.

5. Promise Chain with .catch() and .finally() Handling

Promise.resolve(1)
.then((val) => {
  console.log(val) // resolve with value 1
  return val + 1  //  return 2  
}).then((val) => {
  console.log(val) // 2
  // return undefined
}).then((val) => {
  console.log(val)  // undefined   
  return Promise.resolve(3)
    .then((val) => {
      console.log(val) // 3
      // return undefined
    })
}).then((val) => {
  console.log(val)  // undefined 
  return Promise.reject(4)  // return 4    
}).catch((val) => {
  console.log(val)  // 4
  // return undefined
}).finally((val) => {
  console.log(val)  // undefined: finally has no arguments
  return 10   // no effect on promise object
}).then((val) => {
  console.log(val)  // undefined: because recent 'catch()' handled the promise object with 'undefined'
})

In this example, we’re chaining multiple.then(),.catch(), and.finally() methods together to show how different stages of promise resolution are handled. Let's break it down:

finally() does not receive an argument:
The finally() block executes clean-up code but doesn't take or pass any values. It’s used to ensure certain code runs regardless of the promise’s outcome.
Returning a value in finally() doesn't affect the promise:
If you return a value in the finally() block, it doesn't affect the promise chain or the final value. It's executed after the promise resolution/rejection but doesn't modify the result.
Throwing an error in finally() causes rejection:
If you throw an error or return a rejected promise in finally(), it will cause the promise chain to reject with the error or rejection reason.

   Promise.reject(1)
    .finally(() => { throw new Error(2); });

   Promise.reject(1)
    .finally(() => { return Promise.reject(2); });

The order of then() and catch() matters The .then() and .catch() can be invoked in any order, but they will always return the promise’s final state. When a promise is handled by .catch(), any subsequent .then() will receive the final value.

Example:

 Promise.reject(1)
    .catch((val) => { console.log(val); })  // 1
    .then((val) => { console.log(val); });  // undefined

Converting Promise Chains to Async/Await

Async/await is a method for using promises that causes the code to become more like the code written in synchronous mode. The term often used is “syntactic sugar” because it gives a more straightforward and cleaner path of doing asynchronous code.

    async function fetchData() {
      try {
        const response = await fetch('https://api.example.com/user');
        const data = await response.json();
        console.log("Data:", data);
      } catch (error) {
        console.error("Error:", error);
      }
    }

Combining Promises with Async/Await

You can combine promises with async/await for parallel execution using Promise.all().

 async function getAllData() {
      try {
        const [data1, data2] = await Promise.all([fetchData1(), fetchData2()]);
        console.log("Data 1:", data1);
        console.log("Data 2:", data2);
      } catch (error) {
        console.error("Error fetching data:", error);
      }
    }

Best Practices and Common Mistakes

Avoid Deep Nesting: Use chaining or async/await to keep code flat and readable.
Always Handle Errors: Make sure every promise chain has a.catch() or a try/catch block.
Use Parallel Execution Wisely: Only use Promise.all() when tasks are independent but need to finish together.

Conclusion

JavaScript promises are one of the best ways to deal with your time-consuming operations, such as, retrieving data on a server. They even help you write cleaner easier-to-maintain code, and the practice of what you have learned will equip you to take full advantage of asynchronous coding. Once you get some hands-on experience and begin handling errors elegantly, promises will become such a huge part of JavaScript.

Thank you for reading! If you find this article helpful, feel free to highlight, clap, leave a comment, or even reach out to me on Twitter/X and LinkedIn as it’s very appreciated and helps keeps content like this free!

DEV Community: Manash Jyoti Baruah

Find Your Optimal PostgreSQL Connection Pool Size with k6 Load Testing

A step-by-step guide to identifying and fixing connection pool bottlenecks in Node.js applications

What You'll Learn

The Problem: Slow Response Times with Low CPU Usage

Step 1: Add Connection Pool Monitoring

Basic Server Setup with Monitoring

Step 2: Create k6 Load Test

Step 3: Test Different Pool Sizes Systematically

.env Configuration

Testing Workflow

Step 4: Analyse Test Results

Test #1: Pool Size 10 (Baseline)

Test #2: Pool Size 15

Test #3: Pool Size 20

Test #4: Pool Size 40

Step 5: Performance Analysis

Results Summary

Key Insights

Step 6: Calculate Your Optimal Pool Size

Important Constraints

Step 7: Production Implementation

Enhanced Monitoring Setup

Database Monitoring Queries

Common Pitfalls to Avoid

1. Not Releasing Connections

2. Holding Connections During External Calls

Quick Reference

Healthy Pool Indicators

When to Increase Pool Size

Useful Commands

Conclusion

The Complete Performance Picture:

Final Verdict: Choose Pool Size 20

Key Takeaways:

💡 Found this helpful?

Stop Trying to Learn Everything -Focus on These 5 Key Skills Every Developer Needs

Introduction

Prioritize What Matters

Master the Art of Learning How to Learn

Communication Skills Are Just as Important as Code

Consistency Over Intensity in Learning

Focus on Problem-Solving

Conclusion

Stop Trying to Learn Everything -Focus on These 5 Key Skills Every Developer Needs

Introduction

Prioritize What Matters

Master the Art of Learning How to Learn

Communication Skills Are Just as Important as Code

Consistency Over Intensity in Learning

Focus on Problem-Solving

Conclusion

Everything You Need to Know About JavaScript Promises and How They Work

What is a Promise?

Why Use Promises?

Promise-Based Approach:

How Promises Work?

Handling Promises

Using .then() for Success Handling

Using .catch() for Error Handling

Using .finally() for Cleanup

Chaining Promises

Error Handling in Promises

Advanced Promise Features

1. Promise.all() for Parallel Execution

2. Promise.race() for Fastest Promise

3. Promise.allSettled() for Handling All Outcomes

4.Promise.any() for Resolving with the First Successful Promise

JavaScript Execution Flow with Promises (Important)

1. Promises in JavaScript run in the microtask queue, which gets priority over macrotasks like setTimeout.

2. Promise Execution Order and the Microtask Queue with Multiple .then() Calls

3. Multiple Resolve and Reject Calls in a Promise: Only the First One Matters

4. Chaining Promises and Handling Values in Sequential .then() Calls

5. Promise Chain with .catch() and .finally() Handling

Converting Promise Chains to Async/Await

Combining Promises with Async/Await

Best Practices and Common Mistakes

Conclusion

`.env` Configuration