DEV Community

janak0ff
janak0ff

Posted on

Managing EC2 Access to S3 with IAM Role-based Permissions (No Hardcoded Credentials)

#aws #cloud #iam #kodekloud

If you've ever put an access key and secret key inside ~/.aws/credentials on an EC2 instance just to let it talk to S3, you're not alone — it's the path of least resistance, and it's also one of the most common AWS security mistakes.

In this post, I'll walk through the right way to do it: using an IAM role so the instance gets temporary, auto-rotating credentials with zero secrets stored on disk. By the end, an EC2 instance will be able to upload, list, and download objects from a private S3 bucket without a single key ever touching it.

What you'll need: an AWS account, basic familiarity with the EC2 and IAM consoles, and an SSH client.

Project Architecture

project

Why Not Just Use Access Keys?

The "quick" way to let an EC2 instance talk to S3 is to generate an IAM user's access key/secret key and drop them into ~/.aws/credentials on the instance. It works, but it's a bad habit for a few reasons:

  • Long-lived credentials sitting on disk — anyone who compromises the instance (or finds the keys in an AMI snapshot, a backup, or a leaked .bash_history) gets standing access to your AWS account.
  • No automatic rotation — you have to manually rotate and redistribute keys, and in practice this rarely happens, so the same key can live for years.
  • Hard to scope tightly per-instance — the same key often gets reused across multiple servers, so a single leak has a wide blast radius.
  • Easy to accidentally commit — hardcoded keys have a way of ending up in git history, Docker images, or CI logs.

IAM roles solve all of this using temporary, auto-rotating credentials delivered to the instance via the EC2 Instance Metadata Service (IMDS). The AWS CLI and SDKs check IMDS automatically, so nothing needs to be hardcoded — that's why the first upload attempt below fails with Unable to locate credentials, and why it starts working the moment a role is attached, with zero config changes on the instance itself.

Step 1: Create a Private S3 Bucket

Create a new S3 bucket with default settings, and make sure Block all public access stays checked. This bucket will hold the test file we upload later.

Step 2: Launch an EC2 Instance

Launch an EC2 instance with a public IP, your key pair, and a security group that allows inbound SSH (port 22).

Step 3: SSH In and Prepare the Instance 

ssh -i /path/to/your-key-pair.pem <username>@ec2-public-ip
Enter fullscreen mode Exit fullscreen mode

Install the AWS CLI on the instance:

sudo apt install awscli -y
# or, on Amazon Linux / RHEL-based systems:
sudo dnf install awscli -y
Enter fullscreen mode Exit fullscreen mode

Create a test file to upload to the private S3 bucket:

echo "welcome to private s3 bucket" > test.txt
Enter fullscreen mode Exit fullscreen mode

Now try uploading it — this should fail:

root@ip-172-31-22-21:~# aws s3 cp test.txt s3://my-s3-janak
upload failed: ./test.txt to s3://my-s3-janak/test.txt Unable to locate credentials
root@ip-172-31-22-21:~#
Enter fullscreen mode Exit fullscreen mode

This error is expected — and it's the whole point of the demo. The instance has no credentials configured anywhere (no aws configure, no environment variables, no access keys), which proves that whatever access it gets later is coming purely from the IAM role, not from something typed in manually.

Step 4: Create an IAM Policy

Create an IAM Policy

  • In the IAM console, click Policies in the left sidebar
  • Click Create policy
  • Select the JSON tab
  • Clear the existing content and paste the following (replace my-s3-janak with your bucket name): 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::my-s3-janak"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::my-s3-janak/*"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode
  • Click Next, give it a policy name (e.g. my-s3-janak), then click Create policy

Why two statements? s3:ListBucket is a bucket-level action, so its Resource points to the bucket ARN itself (arn:aws:s3:::my-s3-janak). s3:GetObject and s3:PutObject are object-level actions, so they need the /* suffix to apply to objects inside the bucket. Mixing these up — say, granting GetObject on the bare bucket ARN without /* — is one of the most common IAM policy mistakes, and AWS won't warn you about it. It just silently returns Access Denied at runtime.

This policy also follows the principle of least privilege: it grants access to exactly one bucket, and only the three actions actually needed (list, read, write) — not blanket s3:* access.

Step 5: Create an IAM Role

Create an IAM Role

  • In IAM, click Roles → Create role
  • Trusted entity type: AWS service → Use case: EC2 → click Next
  • Attach permissions: search for and select the my-s3-janak policy → click Next
  • Name and review:
    • Role name: my-s3-janak
    • Description: IAM role for EC2 to access S3
    • Click Create role

An IAM role is different from an IAM user — a role isn't tied to a specific person; it's an identity that something (in this case, the EC2 service) can assume temporarily. When you pick "AWS service → EC2" as the trusted entity, AWS attaches a trust policy behind the scenes that allows ec2.amazonaws.com to assume this role. That trust relationship is what lets EC2 automatically request and refresh temporary credentials on the role's behalf via IMDS — no manual credential handling required.

Step 6: Attach the Role to the EC2 Instance

Attach the Role to the EC2 Instance

  1. Go to EC2 Dashboard → Instances
  2. Select your instance
  3. Actions → Security → Modify IAM role
  4. IAM role: select my-s3-janak
  5. Click Update IAM role

Behind the scenes, attaching the role creates an instance profile — a lightweight container that associates the IAM role with the EC2 instance. This is why the console sometimes shows an instance profile with the same name as the role; it's created and linked automatically, so you rarely need to think about it separately.

No reboot is needed. The instance immediately starts receiving temporary security credentials (access key, secret key, and session token) via IMDS, which AWS rotates automatically roughly every few hours — which is exactly why the CLI just works on the very next command, with zero configuration changes.

Step 7: Upload txt file from EC2 instance to S3 private bucket. 

# Upload to S3
aws s3 cp test.txt s3://my-s3-janak/
# upload: ./test.txt to s3://my-s3-janak/test.txt

# List bucket contents
aws s3 ls s3://my-s3-janak/
# 2026-08-12 15:30:00        5 test.txt

# Download the file
aws s3 cp s3://my-s3-janak/test.txt downloaded.txt
# download: s3://my-s3-janak/test.txt to ./downloaded.txt
Enter fullscreen mode Exit fullscreen mode

Upload file to s3

The exact same aws s3 cp command that failed earlier with Unable to locate credentials now works — without installing new packages, without running aws configure, and without a single key ever touching the instance's disk. The AWS CLI transparently discovered the role's temporary credentials through IMDS and used them for the request.

Key Takeaways

  • No hardcoded credentials — the instance never stores an access key or secret key anywhere.
  • Temporary, auto-rotated credentials — reduces the blast radius if the instance is ever compromised.
  • Least-privilege IAM policy — the role can only touch this one bucket, and only list/get/put — nothing else.
  • This pattern scales — the same approach (IAM role → instance profile → least-privilege policy) is exactly how Lambda functions, ECS tasks, and other AWS compute services are meant to access resources like S3, DynamoDB, or Secrets Manager — instead of embedding credentials in code or environment variables.

If you're setting up anything on AWS that needs to talk to another AWS service, reach for an IAM role first. Access keys should be the exception, not the default.

10 Real-Life Projects Using EC2 + S3 with IAM Roles (No Hardcoded Credentials)

🎯 Project 1: Web Application Log Aggregation

Scenario: A web application running on EC2 instances generates logs that need to be centralized in S3 for analysis and monitoring.

How it works: Each EC2 instance has an IAM role allowing s3:PutObject to a centralized log bucket . A cron job or log shipper (like Fluentd) uploads logs directly to S3.

Web Servers (EC2) → Upload Logs → S3 Central Logs Bucket
        ↓
    IAM Role (PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: A SaaS platform collecting access logs from multiple EC2 instances for security auditing .

🎯 Project 2: CI/CD Artifact Storage

Scenario: Build servers (EC2) create artifacts (ZIP, JAR, Docker images) that need to be stored for deployment.

How it works: EC2 build agents assume an IAM role with s3:PutObject permission to upload artifacts to an S3 bucket .

GitHub → EC2 Build Agent → Build Artifacts → S3 Artifacts Bucket
                ↓
            IAM Role (PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: Jenkins or GitLab CI runners on EC2 storing build outputs in S3 for deployment pipelines .

🎯 Project 3: Database Backup Automation

Scenario: A database server needs automated backups to S3 for disaster recovery.

How it works: A cron job on the EC2 database server dumps the database, compresses it, and uploads it to S3 using IAM role credentials .

EC2 Database → Dump → Compress → Upload → S3 Backup Bucket
        ↓
    IAM Role (PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: Daily MySQL/PostgreSQL backups from application database servers .

🎯 Project 4: Media/Image Processing Pipeline

Scenario: User-uploaded images need to be processed (resized, optimized) and stored in S3.

How it works: A web application stores raw images in S3 (source bucket). An EC2 processing instance reads raw images, processes them, and uploads processed versions to a destination bucket .

S3 (Raw) → EC2 Processor → Process → S3 (Processed)
        ↓
    IAM Role (GetObject + PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: E-commerce product image optimization pipeline .

🎯 Project 5: Cross-Account Log Storage

Scenario: Application logs from Account A need to be stored in an S3 bucket in Account B for centralization.

How it works: EC2 in Account A assumes an IAM role in Account B that allows s3:PutObject to the destination bucket .

Account A: EC2 → STS AssumeRole → Account B: S3 Bucket
        ↓
    Cross-Account IAM Role
Enter fullscreen mode Exit fullscreen mode

Real-world example: Centralized logging across multiple AWS accounts .

🎯 Project 6: Machine Learning Model Artifacts

Scenario: ML training jobs on EC2 (GPU instances) save trained models to S3.

How it works: EC2 training instances assume an IAM role that allows s3:PutObject to a model artifacts bucket .

EC2 (ML Training) → Train Model → Save → S3 Model Artifacts
        ↓
    IAM Role (PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: TensorFlow/PyTorch training jobs saving model weights to S3.

🎯 Project 7: Terraform State Management

Scenario: Terraform state files need to be stored securely in S3 (remote backend).

How it works: EC2 instances running Terraform assume an IAM role allowing access to the Terraform state bucket.

EC2 (Terraform) → Apply → State File → S3 State Bucket
        ↓
    IAM Role (GetObject + PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: Terraform remote backend with S3 and DynamoDB state locking .

🎯 Project 8: Static Website File Mount

Scenario: An EC2 web server needs to serve static files from an S3 bucket (mounted via s3fs).

How it works: EC2 instance has an IAM role with s3:GetObject permission, and s3fs is used to mount the S3 bucket as a filesystem .

EC2 (s3fs) → Mount → S3 Static Files Bucket
        ↓
    IAM Role (GetObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: WordPress or Nginx serving static assets from S3 .

🎯 Project 9: Event-Driven Processing Job

Scenario: A new file uploaded to S3 triggers an EC2 job to process it (unzip, transform, etc.).

How it works: Lambda receives S3 event notification and triggers an EC2 instance via Systems Manager (SSM). The EC2 instance has an IAM role to read from the source bucket and write to the destination bucket .

S3 Upload → Lambda → SSM → EC2 Job → Process → S3 Result
        ↓
    IAM Role (GetObject + PutObject)
Enter fullscreen mode Exit fullscreen mode

Real-world example: Unzipping uploaded files on EC2 .

🎯 Project 10: Static Website Hosting

Scenario: A static website hosted on S3 needs content updates from an EC2 build server.

How it works: EC2 instance builds the static site and syncs it to S3 using aws s3 sync with IAM role permissions .

EC2 (Build) → Generate → Static Assets → S3 (Website Bucket)
        ↓
    IAM Role (PutObject + ListBucket)
Enter fullscreen mode Exit fullscreen mode

Real-world example: Static site deployment pipeline with Hugo/Jekyll .

Top comments (0)