DEV Community

Cover image for SSH Bastion Hosts and Jump Servers: Architecture, ProxyJump, and Zero-Trust Patterns
Mahafuzur Rahaman
Mahafuzur Rahaman

Posted on

SSH Bastion Hosts and Jump Servers: Architecture, ProxyJump, and Zero-Trust Patterns

#architecture #infrastructure #networking #security

Exposing every server directly to the internet is how breaches happen. Here's how to build a hardened single entry point — and then evolve beyond it.

Picture your infrastructure as a building. You wouldn't install a door on every wall and hand out individual keys to every room. You'd build a lobby with a security desk, control who enters, and log every visit.

A bastion host is that lobby.

It's the single, hardened, publicly accessible server through which all SSH access flows. Every other server in your infrastructure sits behind it — no public IPs, no direct access, no exceptions.

This article covers why bastions exist, how to architect them properly, how ProxyJump makes them transparent to developers, and how modern zero-trust thinking is changing the model.

Why Direct SSH Access to Every Server Is a Problem

Consider what you're exposing when a server has a public IP and port 22 open:

  • Attack surface: Every public SSH endpoint is a target for brute-force attacks, credential stuffing, and vulnerability exploitation. The moment a new CVE drops for OpenSSH, every exposed server is at risk.
  • No audit trail: With direct access, you have no centralized record of who connected to what, when, and from where.
  • Key sprawl: Every engineer needs their key on every server they might access. Revocation means touching every machine individually.
  • Lateral movement: If an attacker compromises any server, they're already inside your network.

The bastion model addresses all of these by collapsing the public attack surface to a single point.

The Architecture

Basic Bastion Pattern 

Internet
    │
    ▼
┌─────────────────────────────┐
│  Bastion Host               │
│  - Public IP                │
│  - Port 22 open (or 2222)   │
│  - Hardened OS              │
│  - No other services        │
└─────────────┬───────────────┘
              │  Private network only
    ┌─────────┼─────────┐
    ▼         ▼         ▼
 web-01    db-01     cache-01
(no public IP)  (no public IP)  (no public IP)
Enter fullscreen mode Exit fullscreen mode

The bastion is the only server with a public IP and open SSH port. All other servers:

  • Live in private subnets
  • Have no public IP
  • Have security groups / firewall rules that only accept SSH from the bastion's private IP

What a Bastion Is Not

A bastion host is not a general-purpose server. It should not run your application, databases, or any other services. It exists solely to proxy SSH connections. The smaller its attack surface, the better.

A correctly configured bastion runs:

  • OpenSSH (sshd)
  • Fail2ban or equivalent
  • Audit logging
  • Nothing else

Building the Bastion: Server Configuration

OS and Package Hardening 

# Start with a minimal OS install — Ubuntu Server minimal or Debian
# Update immediately
sudo apt update && sudo apt upgrade -y

# Install only what's needed
sudo apt install -y openssh-server fail2ban auditd ufw

# Remove unnecessary packages
sudo apt autoremove -y
Enter fullscreen mode Exit fullscreen mode

Firewall Rules 

# Allow SSH only from known IP ranges (your office, VPN exit nodes)
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow from 203.0.113.0/24 to any port 22 comment "Office IP range"
sudo ufw allow from 198.51.100.5 to any port 22 comment "VPN exit node"
sudo ufw enable
Enter fullscreen mode Exit fullscreen mode

Restricting SSH to known source IPs is one of the highest-value security controls available. If your team uses a VPN or has static office IPs, there is no reason port 22 should be open to 0.0.0.0/0.

Hardened sshd_config 

# /etc/ssh/sshd_config on the bastion

Port 22
Protocol 2

# Authentication
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
MaxAuthTries 3
MaxSessions 5
LoginGraceTime 30

# No interactive shell needed for pure jump host use
# Remove or comment this if engineers need bastion shell access
# AllowTcpForwarding yes is required for ProxyJump
AllowTcpForwarding yes
X11Forwarding no
AllowAgentForwarding no          # Disable agent forwarding (use ProxyJump instead)
PermitTunnel no                  # No VPN-mode tunneling

# Restrict to specific users or groups
AllowGroups ssh-users

# Logging
LogLevel VERBOSE
SyslogFacility AUTH

# Timeouts
ClientAliveInterval 300
ClientAliveCountMax 2
TCPKeepAlive no                  # Use SSH-level keepalives, not TCP-level
Enter fullscreen mode Exit fullscreen mode 
sudo systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Fail2ban Configuration 

# /etc/fail2ban/jail.local
[sshd]
enabled = true
port = 22
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600     # 1 hour
findtime = 600     # Within 10 minutes
Enter fullscreen mode Exit fullscreen mode 
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
Enter fullscreen mode Exit fullscreen mode

Audit Logging 

# Enable auditd for comprehensive session logging
sudo systemctl enable auditd
sudo systemctl start auditd

# Log all commands run via SSH
echo 'session required pam_tty_audit.so enable=*' | sudo tee -a /etc/pam.d/sshd
Enter fullscreen mode Exit fullscreen mode

For production environments, ship bastion logs to a centralized, tamper-resistant log store (CloudWatch Logs, Splunk, Elastic) immediately. Local logs can be deleted by a compromised account; remote logs cannot.

ProxyJump: Making the Bastion Transparent

The traditional way to use a bastion required two steps: SSH to the bastion, then SSH to the target from there. This was clunky and left engineers with shells on the bastion they didn't need.

ProxyJump (introduced in OpenSSH 7.3) eliminates this entirely.

How ProxyJump Works 

Your machine  ──SSH──►  Bastion  ──SSH──►  Target server
              (encrypted)        (encrypted, new connection)
Enter fullscreen mode Exit fullscreen mode

SSH connects to the bastion, then creates a second encrypted connection through that channel to the target. From your perspective, you're connected directly to the target. The bastion is transparent — you never get a shell on it.

Command Line 

# Single jump
ssh -J ubuntu@bastion.example.com ubuntu@db.internal

# Multiple hops
ssh -J ubuntu@bastion.example.com,ubuntu@internal-gateway.example.com ubuntu@deep.internal
Enter fullscreen mode Exit fullscreen mode

~/.ssh/config (the right way) 

Host bastion
    HostName bastion.example.com
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_prod
    IdentitiesOnly yes
    ServerAliveInterval 60

Host *.internal
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_prod
    IdentitiesOnly yes
    ProxyJump bastion

Host *.prod.example.com
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_prod
    IdentitiesOnly yes
    ProxyJump bastion
Enter fullscreen mode Exit fullscreen mode

Now ssh db.internal automatically routes through the bastion. Same for scp, rsync, sftp — every SSH-based tool respects the config.

ProxyJump vs. the Old ProxyCommand

You'll encounter older configs using ProxyCommand:

# Old pattern — still works, but ProxyJump is cleaner
Host *.internal
    ProxyCommand ssh -W %h:%p bastion.example.com
Enter fullscreen mode Exit fullscreen mode

ProxyJump is preferred for new configs. It's cleaner syntax, it's purpose-built for this use case, and it handles edge cases (like exit codes) more correctly.

Access Control: Who Can Jump Through What

A bastion isn't useful if every engineer can jump to every server. Access control belongs at multiple layers.

Layer 1: OS-Level User Groups 

# On the bastion
sudo groupadd ssh-users
sudo groupadd prod-access
sudo groupadd dev-access

sudo usermod -aG ssh-users alice
sudo usermod -aG ssh-users bob
sudo usermod -aG prod-access alice    # Alice can reach prod
# Bob is not in prod-access
Enter fullscreen mode Exit fullscreen mode

Restrict which users can connect via sshd_config:

AllowGroups ssh-users
Enter fullscreen mode Exit fullscreen mode

Layer 2: authorized_keys Restrictions on Target Servers

The bastion controls who gets in. Target servers control what they can do:

# On db.internal's authorized_keys
from="10.0.1.5" ssh-ed25519 AAAA... alice@example.com
Enter fullscreen mode Exit fullscreen mode

from="10.0.1.5" — Alice's key only works when the connection originates from the bastion's private IP. Her key cannot be used from any other source, even if it's been copied elsewhere.

Layer 3: Separate Bastions per Environment

One bastion for prod, one for dev/staging:

Host bastion-prod
    HostName bastion.prod.example.com
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_prod

Host bastion-dev
    HostName bastion.dev.example.com
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_dev

Host *.prod.internal
    ProxyJump bastion-prod

Host *.dev.internal
    ProxyJump bastion-dev
Enter fullscreen mode Exit fullscreen mode

A dev credential compromised cannot touch prod infrastructure.

High Availability: Two Bastions

A single bastion is a single point of failure. If it goes down, no one can SSH anywhere.

Active-Active Pattern 

Internet
    │
    ├──► bastion-1.example.com (us-east-1a)
    │
    └──► bastion-2.example.com (us-east-1b)
Enter fullscreen mode Exit fullscreen mode

Both bastions are functional. Engineers connect to either. DNS round-robin or a simple config alias handles selection:

Host bastion
    HostName bastion-1.example.com   # Primary
    HostName bastion-2.example.com   # Fallback (use HostName list or scripts)
Enter fullscreen mode Exit fullscreen mode

More robustly, put both behind a Network Load Balancer (NLP) that passes through TCP connections — the NLB handles failover automatically.

Both bastions must have:

  • Identical sshd_config
  • The same authorized public keys (managed by Ansible/Terraform)
  • Logs shipping to the same centralized store

Bastion Patterns in Cloud Environments

AWS 

VPC
├── Public subnet
│   └── Bastion EC2 instance
│       Security group: inbound 22 from office IPs only
│
└── Private subnets
    ├── Web tier (SG: inbound 22 from bastion SG only)
    ├── App tier (SG: inbound 22 from bastion SG only)
    └── DB tier  (SG: inbound 22 from bastion SG only)
Enter fullscreen mode Exit fullscreen mode

Use Security Group references rather than IP-based rules for internal traffic. The database security group allows port 22 from the bastion's security group — not a specific IP. This survives instance replacement.

AWS also offers EC2 Instance Connect and AWS Systems Manager Session Manager as alternatives that eliminate the need for a bastion entirely. SSM Session Manager is particularly powerful — it uses IAM for authentication, requires no open ports, and logs every session.

GCP

Cloud IAP (Identity-Aware Proxy) serves a similar purpose — it proxies SSH through Google's infrastructure, authenticated via Google identity. No bastion needed, no public IPs.

gcloud compute ssh instance-name --tunnel-through-iap
Enter fullscreen mode Exit fullscreen mode

Zero-Trust Evolution: Beyond the Traditional Bastion

The traditional bastion model has a fundamental weakness: it's perimeter-based. Once you're through the bastion, you're "trusted" and can reach any internal server your keys permit. If the bastion is compromised, or if a legitimate user's session is hijacked, the attacker moves freely inside the perimeter.

Zero-trust security applies the principle of "never trust, always verify" to every connection — not just the perimeter.

Zero-Trust SSH Characteristics

Identity-based authentication, not network-location-based
Access is granted based on who you are (verified identity), not where you are (inside the bastion perimeter). Even internal servers verify your identity independently.

Short-lived credentials
Instead of static SSH keys that never expire, zero-trust SSH uses short-lived certificates. You authenticate to an identity provider, receive a certificate valid for (say) 8 hours, and use it to access servers. When you leave for the day, your access literally expires.

Continuous verification
Some zero-trust platforms verify identity continuously during a session, not just at connection time.

Full session audit
Every command, every keystroke is logged and attributable to a specific verified identity.

Implementing Zero-Trust SSH with Certificates

The foundation is an SSH Certificate Authority (CA):

# Generate the CA key pair (store the private key very securely)
ssh-keygen -t ed25519 -f /etc/ssh/ca_key -C "infrastructure-ca"
Enter fullscreen mode Exit fullscreen mode

Configure every server to trust the CA:

# /etc/ssh/sshd_config on every server
TrustedUserCAKeys /etc/ssh/ca.pub
Enter fullscreen mode Exit fullscreen mode

Configure every server's host key to be signed by the CA (solves known_hosts at scale):

# Sign the server's host key
ssh-keygen -s /etc/ssh/ca_key -I "hostname" -h -n "hostname,10.0.0.1" /etc/ssh/ssh_host_ed25519_key.pub
Enter fullscreen mode Exit fullscreen mode 
# Client's ~/.ssh/known_hosts
@cert-authority *.internal ssh-ed25519 AAAA...ca-public-key...
Enter fullscreen mode Exit fullscreen mode

Now clients trust any host presenting a CA-signed certificate. No more known_hosts sprawl.

Issue short-lived user certificates:

# Engineer authenticates to the CA (via Vault, Step CA, etc.)
# Receives a certificate valid for 8 hours
ssh-add ~/.ssh/id_ed25519-cert.pub

# Connect — no static keys needed on the server
ssh db.internal
Enter fullscreen mode Exit fullscreen mode

Tools for Zero-Trust SSH

HashiCorp Vault SSH Secrets Engine
Vault acts as the CA. Engineers authenticate to Vault (using any of Vault's auth methods — LDAP, Okta, GitHub, etc.), request an SSH certificate, and use it. Vault enforces policies: Alice can get certificates for db.internal, Bob cannot.

vault ssh -role=db-access -mode=ca ubuntu@db.internal
Enter fullscreen mode Exit fullscreen mode

Teleport
Purpose-built open-source zero-trust access platform. Handles SSH, Kubernetes, databases, and web apps through a unified proxy. Session recording, live session monitoring, and access requests built in.

Cloudflare Access + WARP
Cloudflare's zero-trust platform. SSH sessions proxy through Cloudflare's network, authenticated via your identity provider. No bastion, no public IPs, full audit trail.

Smallstep
Open-source CA with SSH certificate support. Easier to self-host than Vault if you only need certificates.

The Migration Path

You don't have to go from "SSH to every server" to "full zero-trust" in one step. A practical progression:

Stage 1 — Bastion (start here)
Single hardened entry point, all servers private, ProxyJump for transparency, fail2ban, centralized logging.

Stage 2 — Tightened bastion
Source IP restrictions on the bastion, separate bastions per environment, authorized_keys restrictions on target servers (from= directives), key inventory managed in Git.

Stage 3 — Certificate-based auth
Replace static keys with short-lived certificates. Use Vault or Smallstep as CA. Eliminates key sprawl and rotation overhead.

Stage 4 — Zero-trust
Remove the bastion as a network perimeter concept. Replace with identity-aware proxy (Teleport, Cloudflare Access, AWS SSM). Every connection is verified, logged, and time-limited. No implicit trust based on network location.

Most teams stop at Stage 2 and are significantly better off than they were. Stage 3 is the goal for anything handling sensitive data. Stage 4 is where regulated industries and security-first organizations operate.

Quick Reference: Bastion Checklist 

Infrastructure
[ ] Bastion has public IP, all other servers private
[ ] Security groups: target servers allow SSH from bastion SG only
[ ] Bastion security group: allow SSH from known IPs only
[ ] Separate bastions for prod and non-prod

Bastion hardening
[ ] PasswordAuthentication no
[ ] PermitRootLogin no
[ ] AllowAgentForwarding no
[ ] Fail2ban installed and configured
[ ] Logs shipped to centralized store
[ ] No unnecessary services running

Client configuration
[ ] ProxyJump configured in ~/.ssh/config
[ ] Separate keys for prod and non-prod
[ ] IdentitiesOnly yes on all host blocks

Access control
[ ] Key inventory exists (Git repo or equivalent)
[ ] Offboarding process removes keys from bastion
[ ] authorized_keys on target servers use from= restrictions
Enter fullscreen mode Exit fullscreen mode

The Bottom Line

The bastion host is one of the highest-leverage security controls in SSH infrastructure. It collapses your public attack surface to a single point, centralizes audit logging, and gives you a single place to enforce access policy.

ProxyJump makes it invisible to the people using it. Zero-trust patterns make it resilient even when the perimeter model isn't enough.

Start with a bastion. Tighten it over time. The architecture scales from a three-server startup to a thousand-server enterprise — the principles don't change.

Follow for more infrastructure security deep-dives. Next up: SSH certificates and why they replace static keys at scale.

Top comments (0)