Python-T Point

Posted on Apr 27 • Originally published at pythontpoint.in

🐧 AWS EC2 SSH permission denied fix Ubuntu — common mistakes and how to resolve them

#googlecloud #cloud #devops #linux

47 seconds. That’s all it takes.

One minute you’re confidently typing ssh -i …, the next you’re sweating over a blinking cursor and that cold, unfeeling line:

"Permission denied (publickey). "

Yeah. I’ve been there. Multiple times. Once during a Friday night deploy — team waiting, stakeholders pinging, and me Googling “aws ec2 ssh permission denied fix ubuntu” like my job depended on it. (Spoiler: it kinda did.)

Look — this error sucks. But here’s the thing: it’s rarely some catastrophic AWS failure. More often? A tiny misstep. A permission bit off by one. A file in the wrong place. Something small — but maddening if you don’t know where to look.

So let’s walk through it. Like I’m sitting next to you, coffee in hand, helping you debug this before the weekend slips away.

🔐 Key Pair — The Foundation of SSH Access

First things first: EC2 doesn’t care about passwords. At all. It only trusts your SSH key pair. If you're hitting “Permission denied (publickey)”, the key itself is usually the first suspect.

📑 Table of Contents

🔐 Key Pair — The Foundation of SSH Access
🔑 Lost or Mismatched Key?
🪪 Using EC2 Instance Connect?
⚙️ SSH Configuration — The Silent Gatekeeper
🔍 Debug Mode: Your Best Friend
🌐 Security Groups — The Traffic Filter
🌐 IPv4 vs IPv6 — Are You Connecting to the Right Address?
🧠 User and Home Directory — The Hidden Gotchas
🟩 Final Thoughts
❓ Frequently Asked Questions
Can I regenerate the .pem file for my EC2 instance?
Why does SSH say “Permission denied” even with the correct key?
How do I check SSH daemon logs on Ubuntu?
📚 References & Further Reading

You need two pieces:

The private key (.pem file) on your laptop.
The public key inside the instance’s ~/.ssh/authorized_keys.

And yes, both have to match. But here’s the kicker — SSH is paranoid about security. If your .pem file is too open — like, say, 644 — it’ll flat-out refuse to use it.

I learned this the hard way.

On my first AWS gig, I copied a .pem file across machines using a shared drive. Didn’t think twice. Later, SSH kept rejecting it. I triple-checked the IP, the user, even re-downloaded the key. Nothing.

Turns out: the file ended up with 644 permissions. World-readable. SSH saw that and said, “nope — not touching this.” One chmod later, everything worked.

So — always lock it down:

chmod 400 your-key.pem

That gives read-only access to the owner. Nothing else. That’s what SSH expects.

And hey — if your key is in ~/Downloads, be extra careful. That folder is usually world-readable. SSH hates that. Move the key to ~/.ssh/, fix the permissions, then try again.

Trust me, I’ve seen this one line waste hours.

🔑 Lost or Mismatched Key?

Okay. What if you lost the .pem file? Like — it’s gone. Trashed. Forgotten in some old laptop backup.

Bad news: AWS doesn’t store private keys. You can’t regenerate them.

But — and this is important — you don’t have to destroy and rebuild the instance.

Here’s the move: attach the instance’s EBS volume to another EC2 machine (call it a "rescue instance"). Mount the volume, navigate to /home/ubuntu/.ssh/authorized_keys, and inject a new public key.

Once that’s done, unmount, reattach, reboot — and boom. You’re back in.

Not glamorous. But it works.

I used this trick last year when a junior on my team accidentally deleted their key and had no backup. We saved 6 hours of rebuild time. (He still owes me coffee.)

🪪 Using EC2 Instance Connect?

Some folks skip .pem files entirely and use AWS Console → Connect → SSH. That’s fine — it uses EC2 Instance Connect under the hood.

But it doesn’t use your .pem file. It pushes a temporary key via the AWS API.

For that to work:

The instance must have the ec2-instance-connect package installed.
Your IAM user needs ec2:SendSSHPublicKey permission.
And — this trips people up — you must connect as ubuntu, not ec2-user.

Wait — what?

Yeah. Ubuntu AMIs use ubuntu as the default user. Amazon Linux uses ec2-user. Mix them up? SSH fails. Not because of the key — just because the user doesn’t exist.

So if you’re typing ssh ec2-user@… and your instance is Ubuntu — stop. Try ubuntu@… instead.

I’ve seen this misconfig in three different bootcamps. Every. Single. Time.

⚙️ SSH Configuration — The Silent Gatekeeper

🔍 Debug Mode: Your Best Friend

When in doubt — go verbose. (Also read: 🚀 The Day I Broke SSH on AWS… and Learned Something Awesome)

🌐 Security Groups — The Traffic Filter

Yeah, this feels obvious. But — from what I’ve seen on real projects — like, 30% of “permission denied” cases are actually network issues.

SSH can’t even start if the network blocks it.

Your EC2 instance needs a Security Group that allows inbound traffic on port 22 from your IP (or 0.0.0.0/0 — but don’t leave it like that).

Check:

Is the right Security Group attached?
Does it allow TCP 22 from your current IP?
Is the instance in a public subnet with an Internet Gateway?

And yes — if you’re on a corporate network, your IP might’ve changed. Or your office might block outbound SSH (shoutout to overzealous firewalls).

I had a dev call me from a co-working space once. His IP had changed, and the SG only allowed his old one. Five minutes to fix. Two hours of panic before that.

Use telnet to test the connection:

telnet your-instance-ip 22

If it hangs or says “Connection refused” — it’s not SSH. It’s the network.

Fix that first. Then go back to keys.

🌐 IPv4 vs IPv6 — Are You Connecting to the Right Address?

Some instances have both IPv4 and IPv6.

If you’re using the IPv6 address — make sure:

Your local network supports IPv6.
The Security Group has an IPv6 rule (like ::/0 or your specific IPv6).

Otherwise, you’ll get a timeout — which the client sometimes misreads as “permission denied.”

Not the end of the world. But confusing.

So — if you’re not sure, stick to IPv4. Easier to debug.

🧠 User and Home Directory — The Hidden Gotchas

Key? Good. Network? Fine. SSH config? Proper.

And still — “Permission denied.”

Now we go deeper.

Is the ubuntu user actually set up?

I had this happen once — launched an Ubuntu AMI, but a custom setup script ran before the user was fully initialized. Result? No /home/ubuntu/.ssh. No shell. Nothing.

The user existed in /etc/passwd — but the home directory was missing. SSH tried to log in, failed silently, and said “permission denied.”

Classic.

Check:

getent passwd ubuntu

You should see:

ubuntu:x:1000:1000:Ubuntu:/home/ubuntu:/bin/bash

If the home directory is missing — create it:

sudo mkdir -p /home/ubuntu/.ssh
sudo chown ubuntu:ubuntu /home/ubuntu
sudo chmod 700 /home/ubuntu/.ssh

And if the last part is /usr/sbin/nologin or something — fix it:

sudo usermod -s /bin/bash ubuntu

Also — double-check the AMI. Are you sure it’s Ubuntu? Because if it’s Amazon Linux, the default user is ec2-user, not ubuntu.

Yeah. That happened. To me. On a production box. (Won’t lie — that was a long Monday.)

🟩 Final Thoughts

Here’s the truth: “Permission denied” is never just one thing.

It’s a chain — key, network, config, user, permissions — and one broken link kills the whole thing.

The frustrating part? The error message doesn’t tell you which link failed. It just says “no.”

But — here’s the good news: every fix is local. You don’t need to nuke and rebuild. You just need to test each layer.

Start with the key. Then the network. Then the server config. Then the user.

One by one.

The real skill isn’t memorizing commands. It’s knowing where to look. Developing that instinct — that’s what turns a junior into someone people call when things go sideways.

So next time you see that red text — don’t panic.

Take a breath.

And start debugging.

You’ve got this.

❓ Frequently Asked Questions

Can I regenerate the .pem file for my EC2 instance?

No. AWS doesn’t store private keys. If you lose the original .pem file, you can’t regenerate it. But you don’t have to start over — just mount the EBS volume on another instance and update ~/.ssh/authorized_keys with a new key. (It’s janky, but it works.)

Why does SSH say “Permission denied” even with the correct key?

In my experience? Usually one of four things: (1) wrong .pem permissions (400 is mandatory), (2) wrong username (ubuntu vs ec2-user), (3) sshd_config disabling pubkey auth, or (4) missing/empty authorized_keys file. Always check the logs: sudo tail -f /var/log/auth.log.

How do I check SSH daemon logs on Ubuntu?

Run sudo tail -f /var/log/auth.log. During login attempts, look for lines with “sshd” — they’ll tell you if the key was rejected, the user doesn’t exist, or the file permissions are wrong. Logs don’t lie. (Unlike my junior who swore he’d set chmod 400… he hadn’t.)

📚 References & Further Reading

Official AWS EC2 Linux SSH troubleshooting guide: docs.aws.amazon.com
Ubuntu SSH server configuration documentation: ubuntu.com

(And if you’re reading this at 2 a.m. — hang in there. It’ll click. It always does.)

DEV Community