DEV Community: Shubham Kansal

DevOps Best Practices From the Trenches: What Actually Moves the Needle

Shubham Kansal — Wed, 10 Jun 2026 04:48:22 +0000

TL;DR: I've run production infrastructure for 14 years, founded DietGhar and NyayX, and cut a client's cloud bill 40% in one sprint. This is not a tool checklist. These are the specific practices — with the numbers — that move real needles.

Most DevOps content is a rehashed checklist. Use CI/CD. Write tests. Monitor things. Helpful in the same way "eat less, move more" is helpful for weight loss — technically correct and completely insufficient.

I've been building and running production infrastructure for 14 years. I've founded DietGhar (healthtech, live at dietghar.com) and NyayX (legaltech, live at nyayx.com), and I've consulted for teams running everything from two-person startups to platforms handling 2M-product catalogues under Black Friday load. What follows are the practices that actually moved needles, with specific numbers.

CI/CD is only as good as your slowest feedback loop

A pipeline that takes 40 minutes to run is a pipeline nobody trusts. Developers will start merging without waiting. For DietGhar I use GitHub Actions with a staged pipeline:

# .github/workflows/ci.yml
jobs:
  lint-typecheck:
    runs-on: ubuntu-latest
    timeout-minutes: 2
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci
      - run: npm run lint && npm run typecheck

  unit-tests:
    runs-on: ubuntu-latest
    needs: lint-typecheck
    timeout-minutes: 5
    strategy:
      matrix:
        shard: [1, 2, 3, 4]   # run in parallel
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test -- --shard=${{ matrix.shard }}/4

  integration-tests:
    runs-on: ubuntu-latest
    needs: unit-tests
    timeout-minutes: 10
    services:
      postgres:
        image: postgres:16
        env: { POSTGRES_PASSWORD: test }
      redis:
        image: redis:7
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:integration

Total: 12 minutes from push to green. That speed is intentional. If I let it creep to 30 minutes, the pipeline becomes ceremonial.

The second CI/CD failure mode is treating deployment as a one-way door. Every pipeline I build includes an automated rollback trigger: if the health check endpoint returns non-200 for 60 seconds post-deploy, the previous ECS task definition is reactivated automatically:

# In the deploy step — simplified ECS blue/green with health-check rollback
NEW_TASK_DEF_ARN=$(aws ecs register-task-definition \
  --cli-input-json file://task-def.json \
  --query 'taskDefinition.taskDefinitionArn' --output text)

aws ecs update-service \
  --cluster production \
  --service api \
  --task-definition "$NEW_TASK_DEF_ARN"

# Wait and verify — roll back if health check fails
if ! aws ecs wait services-stable --cluster production --services api; then
  echo "Deploy failed health check, rolling back..."
  aws ecs update-service \
    --cluster production \
    --service api \
    --task-definition "$PREVIOUS_TASK_DEF_ARN"
  exit 1
fi

A bad release never touches more than the seconds between deploy and health check failure.

Infrastructure as code is your documentation

When I inherited a client's AWS account for a cost review, I found: 23 EC2 instances, 11 RDS instances, 6 load balancers. No Terraform, no CDK, no CloudFormation. Nobody knew what half of them did. Three were serving zero traffic. One had been running since 2019.

That engagement paid for itself in the first sprint. I wrote Terraform to model what was actually in use, deleted the orphaned resources, rightsized the rest against CloudWatch metrics, and moved batch workloads to Spot. The bill dropped 40% before I touched a single line of application code.

My rule: if I cannot recreate the entire environment from the repo in under 30 minutes, the infrastructure is undocumented. A minimal Terraform layout for a typical service:

infra/
  modules/
    ecs-service/    # reusable ECS service + ALB target group + autoscaling
    rds-postgres/   # RDS + PgBouncer sidecar + parameter group
    sqs-lambda/     # SQS queue + Lambda + IAM role + DLQ
  envs/
    staging/        # tfvars + state config
    production/     # tfvars + state config

One terraform apply per environment. Infrastructure you can't recreate is infrastructure you can't recover.

Observability before you ship, not after something breaks

With NyayX I instrumented the application before I wrote the first feature. Error rates, p95 latency, conversion funnel drop-off — all dashboarded before a single real user hit the system.

The payoff came in week two of beta. Onboarding had a 30% drop-off rate at step three. Without instrumentation I would have assumed users weren't interested. With it, I could see the drop-off happened on a specific form where validation errors weren't surfacing. A two-line fix.

For the Black Friday platform: three weeks before the event, we ran EXPLAIN ANALYZE on every query over 50ms. Found three queries doing full sequential scans on 2M+ row tables. Added composite indexes. Query times: 800ms to 12ms. No new servers. Observability data led to the right fix before it mattered.

Key metrics to track from day one:

Application layer:
  - p50 / p95 / p99 response time per route
  - Error rate (4xx vs 5xx, separated)
  - Apdex score

Infrastructure layer:
  - Postgres connection count (alert at 80% of max_connections)
  - Cache hit rate (alert if Redis hit rate drops below 90%)
  - Lambda error rate + duration p95

Business layer:
  - Conversion funnel drop-off per step
  - API error rate per endpoint (for externally-facing APIs)

Database connections are infrastructure, not an afterthought

At 12,000 concurrent users, your application will attempt thousands of simultaneous database connections. PostgreSQL degrades sharply above 200. The solution: PgBouncer in transaction pooling mode, deployed as a sidecar or separate service between your app tier and Postgres.

# pgbouncer.ini — the config that saved Black Friday
[pgbouncer]
pool_mode = transaction
max_client_conn = 10000
default_pool_size = 150

Before: 8,000 app-side connections → Postgres under severe load.
After: 8,000 app-side connections → 150 Postgres connections → 4x throughput.

This is a provisioning concern, not a code concern. The right moment to configure connection pooling is during infrastructure setup — it should be in your Terraform module alongside your RDS resource, not discovered during an incident.

Cloud cost is a product decision

Teams that treat cloud cost as a finance problem consistently overspend. Every architecture decision has a cost consequence:

Synchronous fan-out to 10 services vs an event queue? You're paying for the over-provisioning required to absorb spikes.
Polling an API every minute vs webhooks? You're paying for compute that generates mostly empty responses.

For the SP-API platform: the inventory sync originally polled the Catalog Items API for 50,000 SKUs on a 15-minute schedule. The right architecture was subscribing to ITEM_INVENTORY_UPDATE events via the Notifications API, processing changes as they arrived via SQS + Lambda. Sync latency: 24 hours → under 15 minutes. API call volume: down 90%. Compute cost dropped proportionally.

Review your cloud bill monthly. Flag any service growing faster than your user base for architectural review. That growth-rate discrepancy is almost always a design problem.

Environment parity prevents bugs that only appear in production

For NyayX, every environment — local, staging, production — runs the same Docker image built from the same Dockerfile:

# docker-compose.yml (local development)
services:
  api:
    build: .               # same Dockerfile as production
    image: nyayx-api:dev
    environment:
      - NODE_ENV=development
      - DATABASE_URL=postgres://postgres:postgres@postgres:5432/nyayx
    depends_on: [postgres, redis]

  postgres:
    image: postgres:16

  redis:
    image: redis:7-alpine

# ECS task definition (production) — same image, different env vars via Secrets Manager
container_definitions = jsonencode([{
  name  = "api"
  image = "${aws_ecr_repository.api.repository_url}:${var.image_tag}"
  environment = [
    { name = "NODE_ENV", value = "production" }
  ]
  secrets = [
    { name = "DATABASE_URL", valueFrom = aws_secretsmanager_secret.db_url.arn }
  ]
}])

The application code does not know which environment it is in beyond what environment variables tell it. This sounds obvious until you see a team discover that Node.js 16 in production and Node.js 20 in development produce different behaviour — found during a customer demo.

Security is not a phase at the end

NyayX is a legaltech platform handling sensitive legal documents. Security controls that are in production today were in the first pull request that touched each relevant subsystem:

-- Row-level security: tenants cannot query each other's data
-- regardless of application logic
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

-- Audit log: append-only — no UPDATE or DELETE permission
REVOKE UPDATE, DELETE ON audit_log FROM api_role;

// S3 objects private by default — pre-signed URLs for time-limited access
const url = await getSignedUrl(s3Client, new GetObjectCommand({
  Bucket: process.env.DOCUMENTS_BUCKET,
  Key: documentKey,
}), { expiresIn: 900 }); // 15 minutes

Retrofitting security into an existing system costs roughly 10x what it costs to design it in. This is the practice most frequently deferred and most frequently regretted.

Backups are a promise you haven't tested

Everyone has backups. Almost nobody has restores. For NyayX, which stores legal documents clients are legally required to retain, an untested backup is not a backup — it is a liability with good intentions.

I automate the restore, not just the backup:

#!/bin/bash
# Runs weekly via EventBridge + Lambda
set -e

# Restore latest snapshot into throwaway DB
SNAPSHOT_ID=$(aws rds describe-db-snapshots \
  --db-instance-identifier nyayx-production \
  --query 'DBSnapshots[-1].DBSnapshotIdentifier' \
  --output text)

aws rds restore-db-instance-from-db-snapshot \
  --db-instance-identifier nyayx-restore-test \
  --db-snapshot-identifier "$SNAPSHOT_ID"

aws rds wait db-instance-available \
  --db-instance-identifier nyayx-restore-test

# Run integrity checks
ROW_COUNT=$(psql "$RESTORE_URL" -t -c \
  "SELECT COUNT(*) FROM documents WHERE created_at > NOW() - INTERVAL '7 days'")

if [ "$ROW_COUNT" -lt 100 ]; then
  # Page me — something is wrong
  aws sns publish --topic-arn "$ALERT_TOPIC" \
    --message "Restore check FAILED: only $ROW_COUNT recent documents"
  exit 1
fi

# Tear down
aws rds delete-db-instance \
  --db-instance-identifier nyayx-restore-test \
  --skip-final-snapshot

A backup job succeeding tells me almost nothing. A restore succeeding tells me everything. I learned this the hard way: nightly backups that had been succeeding for months while silently writing zero-byte files — a rotated credential had quietly broken the export. The dashboard stayed green the entire time.

If you do one thing after reading this article, schedule a restore test before you schedule another backup.

Incident response needs a script before the incident

For every production system I run, there is a written runbook for the five most likely failure modes:

Database connection exhaustion
High error rate on the main API
Deployment rollback
Third-party API degradation
Certificate expiry

Each runbook has a decision tree, the commands to run, and the escalation path. When the SP-API platform went down during a peak sales window because Amazon's Notifications API was delayed, the runbook said: check SQS queue depth → check Lambda error rate → check SP-API status page → switch to polling fallback if SQS depth exceeds threshold. Resolution time: 11 minutes. Without the runbook, I estimate 45.

After every incident, write a short post-incident note: what broke, the signal, detection time, resolution time, and the single change that would have prevented it. That SP-API incident produced one action item: alert on SQS queue depth. That alert has fired twice since, both times early enough that nobody downstream noticed.

An incident you learn nothing from is just downtime. An incident that produces one concrete prevention is cheap insurance.

The compounding return

Each of these practices compounds:

CI/CD with automated rollbacks → ship confidently at any time
Infrastructure as code → rebuild from disaster in 30 minutes
Observability before launch → find problems before customers do
Connection pooling + cost discipline → scale without re-architecture
Security by design → sell to enterprise without a security audit sprint
Tested restores → a bad day is an inconvenience, not a closure
Runbooks → incidents are learning exercises, not emergencies

I run DietGhar and NyayX as a solo founder-engineer. The only reason that is viable is that the infrastructure runs itself 95% of the time because each of these practices is in place. DevOps best practices are not a tax on your time. They are how you get your time back.

Originally published at https://shubhamkansal.com/blog/devops-best-practices. I'm Shubham Kansal, a freelance Full Stack & DevOps engineer — more at https://shubhamkansal.com.

From 500 to 50,000 Concurrent Users: Architecture Decisions That Actually Scale

Shubham Kansal — Wed, 10 Jun 2026 04:47:54 +0000

TL;DR: Black Friday on a 2M-product platform. Before adding a single server, EXPLAIN ANALYZE found three queries doing sequential scans — fixing the indexes dropped query times from 800ms to 12ms. PgBouncer multiplexed 8,000 connections down to 150 and gave us 4x throughput. Redis cached the right three things and nothing else. Here's the full picture.

Black Friday at 12,000 concurrent users on a platform you migrated from Magento six months ago is a different kind of stress test. The team that ran it before me had a sensible instinct when load spiked: add app servers. That instinct is almost always wrong, and I spent the three weeks before the event proving it.

Start with the database, not the app servers

Every scaling conversation jumps to horizontal app server scaling. The database is almost always the first bottleneck. Before touching anything else, run:

-- Find slow queries in production (pg_stat_statements must be enabled)
SELECT
  query,
  calls,
  mean_exec_time,
  total_exec_time,
  rows
FROM pg_stat_statements
WHERE mean_exec_time > 50   -- anything over 50ms is a candidate
ORDER BY mean_exec_time DESC
LIMIT 20;

We found three queries. One of them, a product listing query joining categories and inventory:

-- Before: full sequential scan, 800ms at scale
EXPLAIN ANALYZE
SELECT p.*, c.name AS category_name, i.quantity
FROM products p
JOIN categories c ON c.id = p.category_id
JOIN inventory i ON i.product_id = p.id
WHERE p.status = 'active'
  AND c.slug = 'electronics'
ORDER BY p.created_at DESC
LIMIT 50;

-- Result: Seq Scan on products (cost=0.00..48234.12 rows=2089432)
--         Execution Time: 847.392 ms

The fix was a composite index that matched the filter and sort together:

-- Covering index for the status + category_id + created_at access pattern
CREATE INDEX CONCURRENTLY idx_products_status_category_created
  ON products (status, category_id, created_at DESC)
  INCLUDE (id, title, price, slug);

-- Separate index for the categories join on slug
CREATE INDEX CONCURRENTLY idx_categories_slug ON categories (slug);

-- After: index scan, 12ms
-- Result: Index Scan using idx_products_status_category_created
--         Execution Time: 11.843 ms

800ms to 12ms. No new servers, no code changes, no cost increase. Index analysis before Black Friday is table stakes.

Connection pooling is not optional

At 12,000 concurrent users, your Node.js app servers will each maintain a connection pool. Ten ECS tasks, 100 connections each: you're already at 1,000 connections against Postgres. Scale to 20 tasks under load and PostgreSQL starts refusing connections entirely — it degrades sharply above roughly 200 active connections.

The fix is a connection pooler sitting between your app tier and Postgres. We used PgBouncer in transaction pooling mode:

# pgbouncer.ini
[databases]
production = host=your-rds-endpoint port=5432 dbname=production

[pgbouncer]
listen_addr = 0.0.0.0
listen_port = 5432
pool_mode = transaction         # key: connection released after each transaction
max_client_conn = 10000         # app-side connections (generous ceiling)
default_pool_size = 150         # actual Postgres connections (stay under 200)
reserve_pool_size = 10
reserve_pool_timeout = 5
server_idle_timeout = 600
log_connections = 0
log_disconnections = 0

Before PgBouncer: 8,000 app-side connections → Postgres melting.
After PgBouncer: 8,000 app-side connections → 150 Postgres connections → 4x throughput increase.

One caveat with transaction pooling mode: you cannot use SET statements, advisory locks, or LISTEN/NOTIFY across transactions — the connection you get back may not be the same one. Keep that in mind if you rely on session-level state.

Redis: cache the right data and nothing else

Redis is not a general caching layer. The wrong data in Redis creates distributed consistency problems that are much harder to debug than slow queries.

We cached exactly three things:

1. Product page data          TTL: 5 minutes
2. User session data          TTL: 30 minutes (sliding)
3. Rate limit counters        TTL: 1 minute (sliding window)

We deliberately did not cache inventory counts. Here is why that matters:

// Wrong: cached inventory leads to overselling
async function getProductPage(productId) {
  const cached = await redis.get(`product:${productId}`);
  if (cached) return JSON.parse(cached); // inventory count may be stale

  const product = await db.query(
    `SELECT p.*, i.quantity FROM products p
     JOIN inventory i ON i.product_id = p.id
     WHERE p.id = $1`,
    [productId]
  );

  await redis.setex(`product:${productId}`, 300, JSON.stringify(product.rows[0]));
  return product.rows[0];
}

// Right: cache everything except the inventory count
async function getProductPage(productId) {
  const [cachedProduct, liveInventory] = await Promise.all([
    redis.get(`product:${productId}`),
    db.query(`SELECT quantity FROM inventory WHERE product_id = $1`, [productId])
  ]);

  const product = cachedProduct
    ? JSON.parse(cachedProduct)
    : await fetchAndCacheProduct(productId); // separate function

  return { ...product, quantity: liveInventory.rows[0].quantity };
}

Stale stock counts cost conversions and cause overselling. Read inventory directly from the database, optimised by index. The indexed inventory read takes 3ms — that is a fine trade for correctness.

Serverless for spiky, stateful for baseline

Not everything needs to survive sustained load. We split the workload by its traffic shape:

Kept on ECS (pre-warmed instances):

Checkout pipeline — latency-sensitive, stateful, must not cold-start
Product search — sustained load, stateful Elasticsearch client
Session API — low latency requirement, high call frequency

Moved to Lambda:

Email notifications — spiky (order confirmation bursts), fully async
PDF receipt generation — CPU-intensive but async, nobody waits for it
Inventory update event processing — event-driven, SQS-triggered

ECS (checkout, search, session)      ← baseline traffic, pre-warmed
Lambda (email, PDF, inventory)       ← burst traffic, event-driven

This cut our baseline EC2 spend by 35% while giving us effectively unlimited burst capacity for async workloads. The checkout service never competes for resources with a PDF generation spike.

Load test before you need to

Three weeks before Black Friday we ran k6 against a production-sized staging environment:

// k6 load test script (simplified)
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '5m', target: 5000 },    // ramp up
    { duration: '10m', target: 15000 },  // hold at 15k (above our target)
    { duration: '5m', target: 0 },       // ramp down
  ],
  thresholds: {
    http_req_duration: ['p95<500'],       // 95th percentile under 500ms
    http_req_failed: ['rate<0.01'],       // less than 1% errors
  },
};

export default function () {
  const res = http.get(`https://staging.yourdomain.com/products/${randomProductId()}`);
  check(res, { 'status 200': r => r.status === 200 });
  sleep(1);
}

The test revealed one non-obvious problem: our CloudFront cache behaviour headers were wrong. Browsers were re-fetching product images on every page load because Cache-Control: no-cache had been set during development and never changed.

Before fix: ~40% of image requests reached origin
After fix:  CDN cache hit rate → 94%

One CloudFront distribution setting. Under actual Black Friday load, the CDN absorbed 94% of all traffic. Origin never saw the full load.

What actually happened on Black Friday

Peak concurrent users: ~12,000
p95 response time on product pages: 210ms
Error rate: 0.3%
Database connections at peak: 148 (PgBouncer keeping us safely under 150)
Zero rollbacks, zero incidents

The database index changes, PgBouncer, and the CDN cache fix together did more work than any amount of horizontal scaling would have. We added zero new ECS tasks during the event.

The checklist before your next traffic event

[ ] Run pg_stat_statements analysis — fix any query over 50ms
[ ] Deploy PgBouncer in transaction pooling mode between app and Postgres
[ ] Audit Redis cache keys — evict anything that must be consistent
[ ] Split traffic-shaped workloads: stateful/latency-sensitive on ECS, async/spiky on Lambda
[ ] Run a k6 load test at 125% of your peak estimate, three weeks out
[ ] Verify CDN cache headers with curl -I on key asset URLs
[ ] Set a CloudWatch alarm on Postgres connection count (alert at 80% of max_connections)

Originally published at https://shubhamkansal.com/blog/scaling-from-500-to-50000-concurrent-users. I'm Shubham Kansal, a freelance Full Stack & DevOps engineer — more at https://shubhamkansal.com.

Amazon SP-API Tutorial: Building Real Marketplace Automation (What No One Tells You)

Shubham Kansal — Wed, 10 Jun 2026 04:47:09 +0000

TL;DR: I built a platform syncing 50k+ SKUs across 5 Amazon marketplaces. Polling the Catalog API nearly killed it. Switching to event-driven inventory sync (Notifications API → SQS → Lambda) cut sync latency from 24 hours to under 15 minutes and dropped API call volume by 90%. Here's everything I wish I'd known going in.

Amazon's Selling Partner API is powerful and genuinely complex. The official docs are long and the gotchas are buried. After building a production platform managing 50,000+ SKUs across 5 marketplaces, this is my ground-level guide — auth, rate limits, sync architecture, and a repricing engine that holds up under real load.

The auth story nobody explains clearly

SP-API does not use IAM credentials the way most AWS services do. The flow is:

Register your app in Seller Central and get an LWA (Login with Amazon) client ID and client secret.
Each seller authorises your app and hands you a refresh token scoped to their account.
Your app exchanges the refresh token for a short-lived access token (valid 1 hour).
You attach that access token to every API call via the x-amz-access-token header alongside a standard AWS Signature V4 header for your IAM role.

// Minimal LWA token refresh (Node.js)
const axios = require('axios');

async function refreshAccessToken(refreshToken) {
  const { data } = await axios.post('https://api.amazon.com/auth/o2/token', {
    grant_type: 'refresh_token',
    refresh_token: refreshToken,
    client_id: process.env.LWA_CLIENT_ID,
    client_secret: process.env.LWA_CLIENT_SECRET,
  });
  return {
    accessToken: data.access_token,
    expiresAt: Date.now() + (data.expires_in - 60) * 1000, // refresh 60s early
  };
}

Critical: refresh proactively, not reactively. If you retry a failed request on token expiry you burn your rate limit quota on the retry. Cache the token, check expiresAt before every call, and refresh in the background.

Rate limits are per-plan, per-endpoint

This trips almost everyone. SP-API is not a single rate limit bucket. Every operation has its own burst rate and restore rate:

Endpoint	Burst	Restore rate
`getListingsItem`	5 req/s	1 req/s
`getCompetitivePricing`	10 req/s	0.1 req/s
`getInventorySummaries`	2 req/s	2 req/s
`getOrders`	0.0167 req/s	0.0167 req/s

Model each endpoint's rate limit independently using a token bucket:

class TokenBucket {
  constructor({ burst, restoreRate }) {
    this.tokens = burst;
    this.burst = burst;
    this.restoreRate = restoreRate; // tokens per second
    this.lastRefill = Date.now();
  }

  consume() {
    this._refill();
    if (this.tokens < 1) return false;
    this.tokens -= 1;
    return true;
  }

  _refill() {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.burst, this.tokens + elapsed * this.restoreRate);
    this.lastRefill = now;
  }
}

// One bucket per endpoint
const buckets = {
  getListingsItem: new TokenBucket({ burst: 5, restoreRate: 1 }),
  getCompetitivePricing: new TokenBucket({ burst: 10, restoreRate: 0.1 }),
};

HTTP 429 counts against you — throttled calls still consume some quota on Amazon's side. Shape your traffic before it leaves your servers.

The sandbox is not production

This cost me two days. The SP-API sandbox:

Uses static test data that doesn't reflect real catalogue behaviour
Accepts certain call patterns that production will reject (e.g. bulk listing submissions with edge-case Unicode in titles)
Does not simulate webhook delivery realistically

Always test auth flows, webhook delivery, and edge-case SKU data against a real production test seller account before going anywhere near live inventory. The sandbox is only useful for verifying your request/response shapes.

Inventory sync: don't poll, stream

Polling the Catalog Items API for 50,000 SKUs every 15 minutes is how you get throttled into oblivion. Our original architecture:

Cron (every 15 min) → Lambda → SP-API Catalog Items (50k calls) → RDS

This hit rate limits constantly, had a sync latency of up to 24 hours under throttling, and cost more than it should. The right architecture:

SP-API Notifications API
        ↓
   Amazon SQS
        ↓
   Lambda (batch processor)
        ↓
       RDS

Step-by-step:

// 1. Create an SQS destination in SP-API (one-time setup)
const spApiClient = new SellingPartnerAPI({ /* config */ });

await spApiClient.notifications.createDestination({
  name: 'inventory-updates-queue',
  resourceSpecification: {
    sqs: { arn: process.env.SQS_ARN }
  }
});

// 2. Subscribe to inventory events per marketplace
await spApiClient.notifications.createSubscription({
  notificationType: 'ITEM_INVENTORY_UPDATE',
  destinationId: destinationId,
  payloadVersion: '1.0',
});

// 3. Lambda processes SQS batches — only changed SKUs
exports.handler = async (event) => {
  const updates = event.Records.map(r => JSON.parse(r.body));

  for (const update of updates) {
    await db.query(
      `UPDATE inventory SET quantity = $1, updated_at = NOW()
       WHERE seller_sku = $2 AND marketplace_id = $3`,
      [update.quantity, update.sellerSku, update.marketplaceId]
    );
  }
};

Results after the migration:

Sync latency: 24 hours → under 15 minutes
API call volume: down 90%
Lambda cost: down proportionally (you only process what changed)

The repricing engine

The Buy Box is determined by price, fulfilment method (FBA beats FBM in most categories), and seller health metrics — roughly in that order. A minimal repricing loop:

// EventBridge triggers this Lambda every 15 minutes
exports.repricerHandler = async () => {
  const activeSKUs = await db.query(
    `SELECT seller_sku, floor_price, ceiling_price, current_price
     FROM listings WHERE repricing_enabled = true`
  );

  for (const batch of chunk(activeSKUs.rows, 10)) {
    // 1. Fetch competitor prices
    const competitiveData = await spApi.getCompetitivePricing({
      marketplaceId: process.env.MARKETPLACE_ID,
      asins: batch.map(s => s.asin),
    });

    for (const sku of batch) {
      const lowestCompetitor = getLowestOffer(competitiveData, sku.asin);

      // 2. Apply floor/ceiling rules
      const newPrice = clamp(
        lowestCompetitor - 0.01,   // beat by 1 cent
        sku.floor_price,
        sku.ceiling_price
      );

      if (newPrice === sku.current_price) continue; // no update needed

      // 3. Submit price change
      await spApi.patchListingsItem({
        sellerId: process.env.SELLER_ID,
        sku: sku.seller_sku,
        marketplaceIds: [process.env.MARKETPLACE_ID],
        body: {
          productType: 'PRODUCT',
          patches: [{ op: 'replace', path: '/attributes/purchasable_offer', value: [{ our_price: [{ schedule: [{ value_with_tax: newPrice }] }] }] }]
        }
      });
    }
  }
};

Run repricing every 15 minutes. Faster than that and you risk pattern-matching Amazon's bot detection. Slower and you lose the Buy Box to competitors who reprice more aggressively.

Lessons in short

Build your LWA auth layer around proactive token refresh, not reactive retry
Model every endpoint's rate limit as a separate token bucket
Never use the sandbox as a proxy for production behaviour
Subscribe to Notifications API events instead of polling — the API was designed for this
Batch Listings API writes (max 10 per request); batch Competitive Pricing reads similarly
Keep your repricer on a 15-minute EventBridge schedule; it's frequent enough without triggering fraud signals

The SP-API ecosystem rewards patience and the event-driven mindset. Once the infrastructure is right, the business logic is the fun part.

Originally published at https://shubhamkansal.com/blog/amazon-sp-api-marketplace-automation. I'm Shubham Kansal, a freelance Full Stack & DevOps engineer — more at https://shubhamkansal.com.