DEV Community: Thi Nguyen (T)

Hackers Love Lazy Developers — 19 Security Mistakes You’re Making Right Now

Thi Nguyen (T) — Tue, 21 Apr 2026 03:42:00 +0000

Security Handbook for Developers

"Hackers don't break in — they walk through the door you left open."

Part I — Developer Identity & Workstation

Threats that start at the endpoint: your laptop, your keys, your credentials. If an attacker owns the developer, the rest of the defenses barely matter.

1. SSH Keys: Hackers' Easiest Target & Prime Hotspots

Let's be honest—every developer, at least once, has raced through ssh-keygen in a caffeine-fueled haze, hammered Enter three times to skip the passphrase, and thought, "I'll add it later… maybe." But guess what? That heroic shortcut is like leaving your car unlocked with the keys in the ignition and a "Please Steal Me" bumper sticker.

If some sneaky hacker nabs your unencrypted private SSH key, they don't need to knock—they walk right in. They can masquerade as you on your servers or GitHub, push malicious code, raid private repos, and throw a root-level rave in your infrastructure.

There have even been exploits—like the SSH-key-scan honeypot attack—that prey on keys without passphrases.

Don't be that person who leaves the passphrase blank. Instead, channel your inner secret-agent and pick a passphrase like C0ffee#Mug@3xtraShot (words + numbers + special chars). Your future self (and your servers) will thank you! And remember: rotate those keys every month, because even the best locks need fresh keys now and then.

If you provide an empty passphrase when generating an SSH key, the private key will not be encrypted. This means:

Anyone who gains access to your private key file (e.g., via a compromised machine, malware, or accidental exposure) can use it without any restriction to authenticate as you.
There will be no additional layer of security beyond file system permissions.
Attackers can use stolen private keys immediately, making them a prime target in exploits.

By setting a strong passphrase, your private key is encrypted with that passphrase, meaning:

Even if someone steals your private key file, they cannot use it without knowing the passphrase.
It adds an extra layer of security, making it much harder for attackers to exploit.

Maximum Security: Store Your SSH Key on a YubiKey (FIDO2)

Even a passphrase-protected SSH key still lives on disk — and anything on disk can be copied. If your laptop gets popped by malware, info-stealer, or a malicious postinstall script, the attacker walks away with ~/.ssh/id_ed25519 and just needs to crack (or keylog) the passphrase.

The fix: don't store the private key on disk at all. Generate a FIDO2/resident SSH key directly on a hardware token like a YubiKey. The private key material never leaves the device, and every git push requires a physical touch on the key. No touch = no auth. Steal the laptop, still can't push.

Why this is the gold standard for pushing to GitHub:

Private key is non-exportable — malware literally cannot exfiltrate it
Physical presence check (touch-to-authenticate) stops remote abuse even if the machine is fully compromised
Works with OpenSSH 8.2+ natively — no extra agents, no GPG dance
Same key can sign commits (gpg.format ssh) so auth and commit signing are hardware-backed

Generate a resident SSH key on the YubiKey:

ssh-keygen -t ed25519-sk -O resident -O application=ssh:github -O verify-required -C "github-yubikey"

-t ed25519-sk → hardware-backed Ed25519 key
-O resident → key can be pulled back onto any machine with ssh-keygen -K (no backup file needed)
-O verify-required → requires PIN and touch on every use (defense in depth)

Pin the key for GitHub pushes only:

git config --global core.sshCommand 'ssh -i ~/.ssh/id_ed25519_sk -o IdentitiesOnly=yes'

The IdentitiesOnly=yes flag prevents SSH from leaking other identities to the server. Add the public key (id_ed25519_sk.pub) to GitHub → Settings → SSH keys, and every push now demands a physical touch.

Recovery: Buy two YubiKeys and enroll both up front. If you lose one, you're not locked out of your own repos at 2 AM on a Friday.

2. Don't Let Hackers Impersonate You, Lock Down Commits with GPG

Why would a hacker forge your commits? How do they pull it off?

Sneak in malicious code: By masquerading as a trusted contributor, they slip dangerous payloads into your codebase.
Corrupt your audit trail: Fake author metadata clouds the "blame" history, making investigations a nightmare.
Exploit CI/CD: A forged commit can trigger automated pipelines, provisioning backdoors or leaking secrets.

How they hack through forged commits — spoofing Git metadata:

git commit --author="You " -m "Awesome new feature"

Signed commits add an extra layer of security by proving the authenticity of code contributions. This prevents commit forgery and ensures that only verified developers contribute to the codebase.

A signed commit with verified badge:

When you sign a commit, you must enter the password for the GPG key:

Additionally, to be able to push code to Github, you must enter the passphrase for the SSH key. This creates 2 layers of protection similar to multi factor authentication.

When generating GPG key, should generate with an expiration. Recommendation is 3 months.

Adding GPG key in Github Account Setting:

3. Enforce Strong Password Practices & Secure Credential Management

Ensuring strong password hygiene is critical to protecting developer accounts, repositories, and infrastructure from unauthorized access. Weak or reused passwords are a major attack vector, leading to account takeovers and data breaches.

Best Practices for Strong Password Security

Use a Password Manager — Developers should never store passwords in plaintext. Use a trusted password manager such as Bitwarden, 1Password, LastPass, or KeePassXC. Enable 2FA for the password manager itself.
Use Strong, Unique Passwords for Each Service — Generate randomized, long passwords (at least 16-24 characters with uppercase, lowercase, numbers, and special characters).
Rotate Passwords for Critical Applications Regularly — Critical applications include GitHub, social accounts, Google Account, crypto wallets, database credentials, cloud infrastructure, and financial accounts. Rotate every 3-6 months.
Enable Two-Factor Authentication (2FA) on All Apps — MFA is non-negotiable. Prefer hardware security keys (YubiKey, NitroKey) over SMS or authenticator apps.
Use Passkeys Where Possible — Passkeys (WebAuthn) are a more secure alternative to passwords. Prevents phishing attacks by eliminating password-based logins.

4. Secrets Aren't for Sharing, Not Even with Your Team

Never share private keys, API secrets, or credentials with anyone—including coworkers.

Do Not Share Credentials: Use role-based access controls (RBAC) and secure secret management tools.
Rotate Credentials Regularly: Implement automated credential rotation policies.
Compromised Secrets Lead to Untraceable Security Breaches: If credentials are leaked, attackers could move laterally within the system, making it harder to identify the entry point.

5. Don't Store Secrets in .env Files – Use a Secure Secret Vault

Why .env Files Are a Security Risk

.env files can be accidentally committed to Git, exposing secrets in repositories.
Attackers with access to a developer's machine can extract credentials from .env files.
.env files lack access controls, making it impossible to enforce least privilege access.
Secrets in .env files are not automatically rotated.

Use HashiCorp Vault Instead

Secrets are encrypted on the disk level
Provides dynamic secret injection without storing credentials in local files
Supports RBAC for secure access management
Allows automatic secret rotation

# Inject secrets dynamically
vault exec -wrap-env my_command

# Example: Injecting Database Credentials Securely
vault exec --env DB_USERNAME --env DB_PASSWORD my_command

Secrets are never stored in local files or exposed in shell history.

6. Secure Firewall Policy for Developer Desktops

A firewall is critical for developer desktops, ensuring unauthorized network traffic is properly controlled.

Key reasons:

Block unauthorized outbound traffic — Prevents malware from leaking sensitive data.
Restrict inbound access — Ensures only necessary services are accessible.
Mitigate attack risks — Reduces exposure to port scanning and unauthorized access.

Best Practices:

Least Privilege: Allow only necessary services.
Deny by Default: Block all and allow only required traffic.
Logging & Monitoring: Enable logging for audit trails.
Regular Review: Periodically check rules for outdated permissions.
Use IP Whitelisting: Restrict access to trusted IPs.

Suggested tools: UFW, OpenSnitch.

7. Monitor for Compromised Developer Devices

Even if your backend is locked down, all it takes is one developer's infected laptop to sink the ship.

Use Endpoint Detection & Response (EDR) tools like CrowdStrike, SentinelOne, or open-source Wazuh.
Disable automatic token caching in CLI tools like GitHub CLI, AWS CLI.
Limit SSH agent forwarding, especially on jump boxes and bastion hosts.
Restrict privileged developer laptops from using public Wi-Fi or enforce VPN-only policies.

Part II — Supply Chain & Dependencies

Everything that enters your machine from the outside world: packages, images, downloads, attachments. The inbound trust boundary is where modern attacks land first.

8. No Blind Installs: Treat Each Package Like a Potential Trojan

The open-source package ecosystems (npm, PyPI, Go modules and beyond) are prime real-estate for supply-chain attacks. One malicious dependency can turn your entire codebase into a backdoor.

Real-World Supply-Chain Ambushes

Common Attack Patterns

Typosquatting & Look-Alikes — Packages with names almost identical to popular ones sneak past casual installs.
Compromised Maintainers — If a trusted owner's account is hijacked, legitimate libraries can be laced with malware.
Dependency Confusion — Publishing internal package names publicly tricks CI/CD pipelines into fetching malicious versions.
Obfuscated Malware — Minified or compiled code conceals data-stealing routines.

Pre-Installation Security Checklist

Authenticate the Source — Install only from official registries and verify the maintainer's identity. Run npm audit, pip audit, or use tools like Socket.dev.
Pin & Inspect Your Dependencies — Commit lockfiles. Preview dependency trees with npm ls, go mod graph, or pipdeptree.
Spot Typosquats — Double-check package names against the registry and GitHub repo links.

9. Don't Let Docker Be Your Trojan Horse: Lock Down Your Containers

Containers are like magic boxes: lightweight, portable, and fast. But if you treat them like black boxes and skip security, they'll behave like Pandora's Box.

A container isn't a sandbox unless you make it one.

Major Docker Security Breaches

Tesla Kubernetes Hack (2018) — Attackers exploited an open Kubernetes dashboard to deploy cryptominers.
Docker Hub Data Breach (2019) — 190,000 user credentials exposed.
TeamTNT Docker API Attacks (2020–2021) — Exploited unauthenticated Docker daemons.
Graboid Docker Worm (2019) — First Docker worm spread through unsecured daemons.
Dero Miner Targeting Docker (2025) — Cryptomining worm exploiting misconfigured Docker daemons.

Best Practices for Secure Docker Usage

1. Never Run Containers as Root

# Bad
USER root

# Good - Use a non-root user
RUN useradd -m appuser
USER appuser

2. Use Minimal Base Images

# Small and secure base image
FROM alpine:3.20

3. Scan Images for Vulnerabilities — Use Trivy or Dockle.

4. Sign and Verify Docker Images — Use Docker Content Trust (DCT) or Cosign.

5. Avoid Leaking Secrets in Docker Images

# Bad
COPY .env /app/.env

# Better
ARG API_KEY
ENV API_KEY=$API_KEY

6. Use Docker Networks + Firewalls Wisely

# Limit to localhost
-p 127.0.0.1:8080:80

7. Monitor and Limit Container Resources

docker run --memory=256m --cpus=0.5 myapp

8. Keep Docker & Host Up-To-Date — Consider distros like Bottlerocket or Flatcar designed for containers.

9. Runtime Security with eBPF + Sandboxing — Use Cilium Tetragon for eBPF-based monitoring. Use gVisor or Kata Containers for sandboxing.

10. Verify Website URLs Carefully Before Downloading Anything

The Download Trap: When "Official" Isn't Actually Official

You're rushing to install a new tool, you Google it, click the first result, and BAM — you just downloaded malware disguised as legitimate software. Even Homebrew has been targeted by sophisticated phishing campaigns.

How Attackers Hook Developers

The Google Ads Hijack — Malicious ads appear above legitimate search results.
Typosquatting — https://node-js.org instead of https://nodejs.org, https://g1thub.com instead of https://github.com.
SEO Poisoning — Fake sites optimize for search rankings.

The Developer's Download Defense Playbook

Step 1: Source Authentication — Never trust search engines for software downloads. Bookmark trusted sources.

Step 2: URL Forensics — HTTPS is non-negotiable. Inspect certificates. Read URLs character by character.

Step 3: Prefer Package Managers

# Secure: Use official package managers
brew install node          # macOS
apt install nodejs         # Ubuntu/Debian
npm install -g typescript  # Node.js packages

# Risky: Manual downloads from random websites
curl https://suspicious-site.com/nodejs.tar.gz

Step 4: Pre-Installation Scanning — Upload to VirusTotal, verify checksums, sandbox test suspicious downloads.

Attack	Year	Impact	Method
PyPI Package Attack	2022	Thousands infected	Mimicking `pytorch-nightly`
Google Ads Malware	2023	Mass infections	Fake ads for VLC, Notepad++
NPM Supply Chain	2021	Worldwide impact	Malware in `ua-parser-js`

11. Weaponized PDFs & File Traps: Don't Get Baited

Cybercriminals love when you open random files, especially PDFs via Telegram, email, or Discord. From Radiant to Ronin, multimillion-dollar exploits began with a single careless click.

Radiant hack (2024) — $50M lost from malware in a PDF
Ronin Bridge hack (2022) — $615M lost from a similar vector

Guidelines for Handling PDFs:

Do NOT download and open PDFs directly — Open them in your browser using Google Drive or other secure preview options.
Be wary of unexpected attachments — Verify legitimacy before opening.
Never enable macros or scripts — Some PDFs prompt you to enable features that execute malicious code.
Report suspicious files — If you receive a suspicious PDF, report it to your security team.

Use application sandboxing if you must open a PDF locally:

Windows: Windows Sandbox (built-in)
Linux: Firejail
macOS: App Sandbox

Part III — DevSecOps & Shift-Left

Security baked into the build pipeline and the code itself. Find bugs before attackers do.

12. Secure Your CI/CD Pipelines: Attackers Love Automated Trust

Your CI/CD pipeline has the keys to the kingdom: access to source code, secrets, cloud credentials, and deployment mechanisms.

Major CI/CD Security Breaches

SolarWinds Orion (2020) — Nation-state attackers inserted a backdoor, impacting 18,000+ customers.
Codecov (2021) — Attackers tampered with the Bash uploader, stealing secrets from thousands of environments.
CircleCI (2023) — Compromise led to theft of environment variables, secrets, and keys.
Travis CI (2022) — Leaked secrets in build logs exposed thousands of credentials.
PHP Git Server (2021) — CI pipeline was hijacked to insert malicious code into official PHP source.

Best Practices:

Use ephemeral environments: Rebuild containers and runners on every job.
Restrict secrets exposure: Never expose all secrets to every job; use secret-scoping.
Require commit signature verification before running deploy jobs.
Use OpenID Connect (OIDC) for temporary cloud permissions via short-lived tokens.
Audit your pipeline dependencies: Install only trusted tools and lock versions.
Pipeline Isolation & Segmentation: Separate pipelines by environment.

13. Secure Coding: Write Code That Resists Attacks

Developers must write code that is resilient to common attacks. Key practices from the OWASP Top 10:

Input Validation: Treat all user input as untrusted. Validate and sanitize on the server side.
Output Encoding: Encode data output to prevent XSS.
Parameterized Queries: Prevent SQL injection.
Authentication and Session Management: Implement strong auth, MFA, and secure cookies.
Error Handling: Avoid leaking sensitive info in error messages.
Least Privilege: Never run applications as root.

# Vulnerable to SQL injection
cursor.execute("SELECT * FROM users WHERE username = '" + username + "' AND password = '" + password + "'")

# Secure: Parameterized query
cursor.execute("SELECT * FROM users WHERE username = ? AND password = ?", (username, password))

14. Security Testing: Shift Left and Catch Issues Early

Shifting security left means integrating security early in the SDLC:

SAST (Static Application Security Testing): Analyze source code during development. Tools: SonarQube, Bandit, ESLint with security plugins.
DAST (Dynamic Application Security Testing): Test running applications. Tools: OWASP ZAP, Burp Suite.
Dependency Scanning: Use Dependabot, Renovate, or Snyk.
Secret Scanning: Use GitGuardian or TruffleHog to prevent secrets from being committed.

Part IV — Access & Infrastructure

Shrink the blast radius: no standing production access, no flat networks, no permanent cloud admins.

15. VPN Connection Required for Critical Resources

VPN authentication must require Two-Factor Authentication (2FA).
All employees and contractors must enable 2FA on cloud accounts, code repositories, and internal dashboards.

16. Implement Just-In-Time (JIT) Privileges

Developers often have standing access to sensitive infrastructure. JIT access ensures elevated permissions are granted temporarily, on-demand, with approval, and revoked automatically.

How to Implement:

AWS IAM + SSO + Identity Center — Short-duration assume-role sessions
Azure PIM — Temporary admin roles with approval and MFA
Teleport Access Requests — JIT SSH/Kubernetes/database access via Slack or web UI
StrongDM or Boundary (HashiCorp) — Identity-based JIT session brokering

17. Cloud Security: Shared Responsibility Model

Key practices:

IAM: Assign minimal permissions. Use groups and roles. Enable MFA.
Network Security: Use VPCs, security groups, and network ACLs.
Encryption: Encrypt data at rest and in transit.
Logging and Monitoring: Enable cloud provider logs and set up alerts.
IaC Security: Scan Terraform/CloudFormation templates with Checkov or tfsec.

// Bad: Public read access
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"

// Good: Restrict to specific IAM role
"Effect": "Allow",
"Principal": {
  "AWS": "arn:aws:iam::123456789012:role/my-role"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"

Part V — Detect & Respond

Assume breach. The question isn't if something slips through — it's how fast you see it and how cleanly you contain it.

18. Security Logging & Alerting: Detecting Threats Before They Strike

"You can't defend what you can't see."

Centralizing developer logs using SIEM tools is essential. A well-configured SIEM can help you:

Detect lateral movement across developer machines
Correlate developer activity with alerts from cloud systems
Set up anomaly detection alerts for suspicious behavior

Tools:

Wazuh — Open-source SIEM + XDR
Falco (CNCF) — Cloud-native runtime threat detection
Osquery — Query endpoints like a database
MITRE ATT&CK Framework — Map attacker behavior to detection patterns

19. Incident Response: Be Prepared for the Worst

Following the NIST model:

Detection and Reporting: Recognize breaches through unusual behavior and monitoring alerts. Report immediately.
Containment: Isolate affected systems. Revoke compromised credentials.
Eradication: Patch vulnerabilities. Remove malware.
Recovery: Restore from clean backups. Monitor for recurrence.
Post-Incident Analysis: Conduct retrospectives to improve defenses.

Frameworks:

Open Source Toolkits:

GRR Rapid Response — Remote forensics
TheHive + Cortex — Collaborative incident handling
Zeek — Network traffic analysis

Originally published at fystack/developer-security-handbook

Secure crypto infrastructure: SLSA L3 Provenance for Docker Images - How We Made Our Builds Verifiable

Thi Nguyen (T) — Wed, 15 Apr 2026 05:54:01 +0000

Introduction

In March 2026, the axios package was compromised through a hijacked npm maintainer account. A malicious version was published containing a credential-stealing backdoor that exfiltrated cloud keys before self-destructing. This incident exemplified a persistent problem: downstream users must trust artifacts blindly without cryptographic verification of build provenance.

At Fystack, a crypto custody infrastructure provider, supply chain integrity is critical—"a tampered image in our environment doesn't mean a bug, it means compromised keys and drained wallets." The company addressed this challenge through SLSA L3 provenance implementation.

The Verification Gap Nobody Talks About

When pulling a Docker image, users receive a SHA-256 digest proving content integrity. However, this reveals nothing about production methods. Signatures verify identity but not whether the build environment was compromised or if code matched the source commit.

The gap exists between "signed" and "provable." A malicious actor accessing CI runners can build and sign anything with legitimate credentials. Dependency poisoning during builds produces validly-signed images containing malicious code.

This gap requires a third guarantee: provenance—a cryptographically signed record of the exact source commit, build environment, workflow, and parameters, generated by a process isolated from the build itself.

What SLSA Actually Means (Beyond the Acronym)

Supply-chain Levels for Software Artifacts was developed by Google and is governed by the OpenSSF. The framework defines graduated security guarantees for build pipelines:

L1: Untrusted provenance document; useful for auditing but easily forgeable
L2: Cryptographically signed provenance by CI platform; prevents post-build tampering but the same job generating the build creates the proof
L3: Provenance generated in a completely isolated process with separate identity and credentials; the build job cannot influence it

In practical terms:

No SLSA/L1: "Trust me, I built this"
SLSA L2: "Our CI built this, but if compromised, proof can be faked"
SLSA L3: "This image was built by our CI in a locked-down environment, and we can prove it, even if the build job is fully compromised"

SLSA L3 is now baseline for serious infrastructure providers. Google ships official container images at SLSA L3; GitHub builds Artifact Attestations on SLSA L3; Bitnami's Secure Images are SLSA L3 with publicly published verification keys.

The Critical Difference: BuildKit vs. SLSA Generator

Most implementations conflate these approaches. Docker's build-push-action with provenance: true provides L2 guarantees, not L3.

BuildKit Provenance (provenance: true):

Generated in the same job as the build
Forgeable if the build is compromised
SLSA L1–L2

SLSA L3 Provenance (via slsa-github-generator):

Generated in a separate isolated job
Receives only the image digest
Not forgeable even if the build is compromised
Signed by the SLSA generator's identity

How Fystack Applied SLSA

Fystack's implementation uses three sequential jobs:

Job 1: Build & Push

BuildKit compiles and pushes the image; native provenance is disabled, keeping only the immutable image digest.

Job 2: SLSA L3 Provenance

Uses slsa-framework/slsa-github-generator in an isolated job. It receives only the final image digest, independently verifies the source repository, commit, and workflow via GitHub APIs, and generates non-forgeable SLSA Level 3 attestation. Even if Job 1 is compromised, this provenance cannot be faked.

Job 3: Signing & Attestation

Signs the image using cosign with keyless signing (GitHub OIDC + Fulcio); no private keys touch the repository or runner
Generates full SPDX SBOM with Syft and attests it to the image
Stores signatures and attestations using OCI 1.1 referrers mode to avoid tag pollution

The final image carries five artifacts on the same digest:

Image layers
BuildKit SBOM (for Docker Scout)
Cosign signature
SLSA L3 provenance
Sigstore-attested SPDX SBOM

Fystack open sourced this pipeline as a reusable GitHub Actions workflow at fystack/slsa-workflows.

What Verification Looks Like

Downstream consumers verify independently with two commands:

# 1. Verify the cosign signature
cosign verify \
  --certificate-identity-regexp='https://github.com/fystack/slsa-workflows/.github/workflows/docker-build-slsa.yml@.*' \
  --certificate-oidc-issuer='https://token.actions.githubusercontent.com' \
  fystacklabs/apex:v1.0.54

# 2. Verify SLSA L3 provenance
slsa-verifier verify-image \
  --source-uri github.com/yourorg/your-repo \
  fystacklabs/apex@sha256:...

If the build was tampered with, provenance checks fail even if the signature remains valid.

All signatures and attestations are recorded in Rekor, Sigstore's public transparency log, creating an immutable, independently auditable record.

What This Still Doesn't Solve

SLSA L3 is powerful but not comprehensive. Dependency attacks remain real threats. Malicious packages can be legitimate at build time while the image passes all verification. SBOMs provide visibility, not prevention.

Base image risks are inherited. Backdoors like XZ Utils hid in official Debian images for over a year. Provenance records the base used but cannot detect compromise. Pinning by digest helps but doesn't eliminate risk.

Malicious maintainers bypass everything. If trusted committers merge backdoored code, SLSA L3 still passes. It proves the pipeline ran correctly, not that code was safe. Code review and branch protection remain essential.

Language registries such as npm and PyPI still lag behind container registry tooling.

The Standard Is Moving

Three years ago, SLSA L3 was considered advanced security theater. Today it's becoming baseline expectation for serious projects.

The question has shifted from "Should we adopt SLSA L3?" to "Why haven't we done it yet?"

Tools are free, mature, and maintained by Google and the OpenSSF. The effort is real but measured in hours, not weeks.

"Your users run your code in production. They deserve more than 'trust me.' They deserve cryptographic proof."

How to build a Crypto Payment Gateway with Fystack Programmable Wallet Infrastructure (Part 1)

Thi Nguyen (T) — Wed, 15 Apr 2026 05:41:40 +0000

Overview

Fystack enables developers to create production-ready cryptocurrency payment gateways without managing private keys or blockchain infrastructure. The platform handles wallet creation, deposit address generation across multiple chains, automatic fund consolidation, and transaction monitoring through a unified SDK.

Architecture Overview

The payment flow consists of six steps:

Applications create wallets per user via the Fystack SDK
Fystack generates deposit addresses across Ethereum, Solana, Tron, and Bitcoin
Users deposit funds to their assigned addresses
Sweep tasks automatically consolidate funds into a central hot wallet when thresholds are met
Consolidated funds reach the hot wallet for settlement
Withdrawals and payouts distribute funds to merchants or external addresses

The system uses Hyper wallets (HD-derived for high-volume user wallets) and MPC wallets (threshold signatures for treasury storage), eliminating direct private key exposure.

Prerequisites

Node.js 22 or later
Fystack account with API credentials
Workspace ID from the Fystack dashboard

Copy workspaceID from API Key section

Setup Steps

Step 1: Install and Configure SDK

npm install @fystack/sdk

Create a .env file with credentials:

FYSTACK_API_KEY=your-api-key
FYSTACK_API_SECRET=your-api-secret
FYSTACK_WORKSPACE_ID=your-workspace-uuid
HOT_WALLET_ID=your-hot-wallet-uuid

Initialize the SDK:

import {
  FystackSDK,
  Environment,
  WalletType,
  WalletPurpose,
  AddressType,
} from "@fystack/sdk";

const sdk = new FystackSDK({
  credentials: {
    apiKey: process.env.FYSTACK_API_KEY!,
    apiSecret: process.env.FYSTACK_API_SECRET!,
  },
  environment: Environment.Production,
  workspaceId: process.env.FYSTACK_WORKSPACE_ID!
});

Step 2: Create a Hot Wallet

const hotWallet = await sdk.createWallet(
  {
    name: "Treasury Hot Wallet",
    walletType: WalletType.MPC,
    walletPurpose: WalletPurpose.General,
  },
  true // Wait for creation to complete
);

console.log("Hot wallet ID:", hotWallet.wallet_id);

Create a wallet through Fystack portal

MPC wallet creation is asynchronous; the distributed key generation protocol requires 10-30 seconds across multiple nodes.

Step 3: Create a Sweep Task

Visit the Fystack portal → Automation → Add sweep task. Configure:

Task name
Destination wallet (the MPC hot wallet)
Minimum trigger value in USD

Create a sweep task

The sweep task ID appears as the last segment of the URL.

Step 4: Create Per-User Wallets

async function createUserWallet(userId: string) {
  const wallet = await sdk.createWallet(
    {
      name: `user_${userId}`,
      walletType: WalletType.Hyper,
      walletPurpose: WalletPurpose.OneTimeUse,
      sweepTaskID: '6bdc8e47-dc1a-4416-9158-b99cdc23205a'
    },
    true
  );

  return wallet;
}

OneTimeUse wallets attempt to empty completely during sweeps rather than preserving dust amounts for future gas fees, preventing stranded balances across thousands of deposit wallets.

Step 5: Get Multi-Chain Deposit Addresses

async function getDepositAddresses(walletId: string) {
  const evm = await sdk.getDepositAddress(walletId, AddressType.Evm);
  const sol = await sdk.getDepositAddress(walletId, AddressType.Solana);

  return { address, qr_code };
}

Supported chains by address type:

Type	Chains	Format
`evm`	Ethereum, Polygon, Arbitrum, Optimism, Base	`0x...`
`sol`	Solana	Base58 public key
`tron`	Tron	`T...`
`btc`	Bitcoin	`bc1q...`

Display endpoint example:

app.get("/api/payment/:userId/address", async (req, res) => {
  const { userId } = req.params;
  const { chain } = req.query;

  const wallet = await getOrCreateWallet(userId);
  const addressType = chain === "solana" ? AddressType.Solana : AddressType.Evm;
  const deposit = await sdk.getDepositAddress(wallet.wallet_id, addressType);

  res.json({
    address: deposit.address,
    qrCode: deposit.qr_code,
    chain,
  });
});

Step 6: Listen for Deposits with Webhooks

Fystack sends webhook events for transaction state changes:

Event	Trigger	Purpose
`deposit.pending`	On-chain detection, unconfirmed	Show pending UI
`deposit.confirmed`	Blockchain confirmation	Credit account
`withdrawal.pending`	Withdrawal request created	Update records
`withdrawal.confirmed`	Transaction confirmed	Mark complete
`withdrawal.failed`	Transaction failed	Alert team

Verify webhook signatures using Ed25519:

import crypto from "crypto";

const { public_key } = await sdk.getWebhookPublicKey(
  process.env.FYSTACK_WORKSPACE_ID!
);

app.post("/webhook/fystack", async (req, res) => {
  const signature = req.headers["x-webhook-signature"] as string;
  const event = req.headers["x-webhook-event"] as string;
  const payload = req.body;

  const canonical = JSON.stringify(sortKeysDeep(payload));

  const isValid = crypto.verify(
    null,
    Buffer.from(canonical),
    {
      key: Buffer.from(public_key, "base64"),
      format: "der",
      type: "spki",
    },
    Buffer.from(signature, "base64")
  );

  if (!isValid) {
    return res.status(401).json({ error: "Invalid signature" });
  }

  switch (event) {
    case "deposit.confirmed":
      await handleDepositConfirmed(payload);
      break;
    case "withdrawal.confirmed":
      await handleWithdrawalConfirmed(payload);
      break;
    case "withdrawal.failed":
      await handleWithdrawalFailed(payload);
      break;
  }

  res.status(200).json({ received: true });
});

function sortKeysDeep(obj: any): any {
  if (Array.isArray(obj)) return obj.map(sortKeysDeep);
  if (obj && typeof obj === "object") {
    return Object.keys(obj)
      .sort()
      .reduce((acc: any, key) => {
        acc[key] = sortKeysDeep(obj[key]);
        return acc;
      }, {});
  }
  return obj;
}

Summary

This part covers wallet infrastructure, deposit address generation, automatic consolidation via sweep tasks, and webhook-based deposit detection. Part 2 will explore withdrawals, treasury management, and production architecture patterns.

For questions about custody setup, visit the inquiry form or join the Telegram community.

When CPUs Hit 100%: Hard-Earned Lessons from a Multichain Indexer Outage, with Bloom Filters as the Rescue.

Thi Nguyen (T) — Tue, 30 Sep 2025 15:30:53 +0000

# When CPUs Hit 100%: Hard-Earned Lessons from a Multichain Indexer Outage | Fystack Blog
At Fystack, where we build blockchain wallet infrastructure, our transaction indexer needs to process over 100 million transactions per day across multiple blockchains, including Ethereum, BNB, Polygon, Tron, and Solana.

Our initial indexing logic was straightforward:

We maintain a database of wallet addresses generated for our users. For each block on the blockchain, we parse the raw transaction data and check if the destination address exists in our database. If it does, we process the transaction; otherwise, we ignore it.

Indexing flow of Fytack indexer

Handling EVM chains alone was manageable, but everything changed when we integrated Solana, one of the most active chains, with huge transaction volumes driven by meme coin crazes and projects like Bump.fun, averaging around 42 million transactions per day (~3,000 TPS).

Our servers, particularly the CPU, were constantly under stress, with utilization hovering between 80–90%. On peak days, CPU usage reached 100%, causing the server to become unresponsive and our customers repeatedly encounter 504 Gateway Timeout errors and started complaining.

The team had to temporarily suspend Solana processing and restart the server just to restore normal operations.

CPU is under high stress

We knew it was time to optimize.

Our team quickly identified that Consul where we persist user wallet addresses, was becoming a bottleneck. Storing and querying every address there doesn’t scale well when transaction volume is high.

To handle this efficiently, we adopted a Bloom filter, a well-known probabilistic data structure also used by the Ethereum blockchain for log filtering. A Bloom filter answers a simple question: Could this item be in the set?

If it says no, the item is definitely absent and can be skipped.
If it says yes, the item might be present, and we perform a full check against our source of truth.

Because most transactions are unrelated to our platform, this lets us skip the vast majority of checks. Properly tuned, a Bloom filter can store millions of wallet addresses while consuming only a few MBs of memory.

Bloom Filter Process

To ensure efficiency and scalability, we utilize the Redis Bloom filter module. When a transaction arrives, we check if the destination address exists in the Bloom filter. If it does not, we skip the transaction; if it does, we proceed to verify it in persistent storage.

The team successfully refactored the system, added new logic, tested, and deployed the solution in under 2 days, allowing operations on Solana to resume as usual.

func (s *publicKeyStore) Exist(addressType enum.AddressType, publicKey string) bool {
    // First check the bloom filter.
    // If the bloom filter returns false, the key definitely doesn't exist.
    if !s.bloomFilter.Contains(publicKey, addressType) {
        return false
    }

    // Since bloom filters may have false positives, check the underlying KV store.
    v, err := s.kvstore.GetWithOptions(composeKey(addressType, publicKey), &infra.DefaultCacheOptions)
    if v == "" || err != nil {
        return false
    }

    return true
}

To make it works seamlessly, we periodically synchronize addresses from our PostgreSQL database to a Redis Bloom filter through a simple and efficient process.

Recent optimizations have significantly enhanced our indexer’s performance and resilience. It now maintains CPU utilization between 40-60%, even under high load, rarely reaching 100%.

Fystack indexer with Bloom filter

We’ve open-sourced our multichain indexer, supporting Ethereum, EVM chains, and Tron. Check it out at github.com/fystack/multichain-indexer. Bitcoin and Solana support are planned for future development.

At Fystack, we are building open source scallable wallet infrastructure that users can self host and doens't need to rely on expensive custodians.

Check out our Github : https://github.com/fystack/