<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Amira Abidi</title>
    <description>The latest articles on DEV Community by Amira Abidi (@amira_abidi).</description>
    <link>https://dev.to/amira_abidi</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4071897%2F77d0ea5b-1d71-495f-8327-cf783cccdea0.jpg</url>
      <title>DEV Community: Amira Abidi</title>
      <link>https://dev.to/amira_abidi</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/amira_abidi"/>
    <language>en</language>
    <item>
      <title>How I Removed AWS Access Keys from GitLab CI/CD with OIDC</title>
      <dc:creator>Amira Abidi</dc:creator>
      <pubDate>Wed, 12 Aug 2026 09:05:09 +0000</pubDate>
      <link>https://dev.to/amira_abidi/how-i-removed-aws-access-keys-from-gitlab-cicd-with-oidc-4fn6</link>
      <guid>https://dev.to/amira_abidi/how-i-removed-aws-access-keys-from-gitlab-cicd-with-oidc-4fn6</guid>
      <description>&lt;p&gt;When I first connected my GitLab CI/CD pipelines to AWS, I used the simplest solution: an IAM user with an Access Key and Secret Access Key stored as GitLab CI/CD variables.&lt;/p&gt;

&lt;p&gt;It worked.&lt;/p&gt;

&lt;p&gt;But there was one problem: &lt;strong&gt;those credentials were permanent.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;They had to be stored, protected and eventually rotated. If they were accidentally exposed in logs or compromised, they could remain valid until manually revoked.&lt;/p&gt;

&lt;p&gt;I wanted a cleaner solution.&lt;/p&gt;

&lt;p&gt;So I replaced permanent AWS credentials with &lt;strong&gt;OIDC federation between GitLab and AWS&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The result is simple: GitLab pipelines can access AWS without storing any permanent AWS credentials.&lt;/p&gt;

&lt;p&gt;In this post, I'll explain how I implemented it, how the authentication flow works, and one important issue I faced when using it with EKS and Terraform.&lt;/p&gt;




&lt;h2&gt;
  
  
  The architecture
&lt;/h2&gt;

&lt;p&gt;The authentication flow looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;┌──────────────┐
│  GitLab CI   │
└──────┬───────┘
       │
       │ OIDC token
       ▼
┌──────────────┐
│   AWS STS    │
└──────┬───────┘
       │
       │ Temporary credentials
       ▼
┌──────────────┐
│   IAM Role   │
└──────┬───────┘
       │
       ├──────────► Terraform
       │
       ├──────────► ECR
       │
       └──────────► EKS
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Instead of GitLab storing an AWS Access Key, it proves its identity to AWS using a short-lived OIDC token.&lt;/p&gt;

&lt;p&gt;AWS verifies the token and returns temporary credentials.&lt;/p&gt;




&lt;h2&gt;
  
  
  How does OIDC authentication work?
&lt;/h2&gt;

&lt;p&gt;The process can be summarized in five steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;GitLab creates an OIDC token for the CI/CD job.&lt;/li&gt;
&lt;li&gt;The pipeline sends this token to AWS.&lt;/li&gt;
&lt;li&gt;AWS verifies that the token really comes from GitLab.&lt;/li&gt;
&lt;li&gt;AWS checks that the project is allowed to assume the requested IAM role.&lt;/li&gt;
&lt;li&gt;AWS STS returns temporary credentials.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;These credentials expire automatically.&lt;/p&gt;

&lt;p&gt;So there is nothing permanent to store or rotate inside GitLab.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 1 — Register GitLab as an OIDC provider
&lt;/h2&gt;

&lt;p&gt;AWS first needs to trust GitLab as an identity provider.&lt;/p&gt;

&lt;p&gt;I configured the OIDC provider using Terraform:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;data&lt;/span&gt; &lt;span class="s2"&gt;"tls_certificate"&lt;/span&gt; &lt;span class="s2"&gt;"gitlab"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;url&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"${var.gitlab_url}/.well-known/openid-configuration"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"aws_iam_openid_connect_provider"&lt;/span&gt; &lt;span class="s2"&gt;"gitlab"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;url&lt;/span&gt;             &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;gitlab_url&lt;/span&gt;
  &lt;span class="nx"&gt;client_id_list&lt;/span&gt;  &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;gitlab_url&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="nx"&gt;thumbprint_list&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="nx"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tls_certificate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;gitlab&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;certificates&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nx"&gt;sha1_fingerprint&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This tells AWS that tokens issued by GitLab can be used for authentication.&lt;/p&gt;

&lt;p&gt;But trusting GitLab itself is not enough.&lt;/p&gt;

&lt;p&gt;AWS also needs to know &lt;strong&gt;which GitLab projects are allowed to access which IAM roles.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2 — Restrict access with the IAM Trust Policy
&lt;/h2&gt;

&lt;p&gt;This is one of the most important parts of the configuration.&lt;/p&gt;

&lt;p&gt;We don't want any GitLab project to be able to assume our AWS roles.&lt;/p&gt;

&lt;p&gt;The IAM Trust Policy checks information contained in the GitLab token.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;condition&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;test&lt;/span&gt;     &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"StringEquals"&lt;/span&gt;
  &lt;span class="nx"&gt;variable&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"gitlab.com:aud"&lt;/span&gt;
  &lt;span class="nx"&gt;values&lt;/span&gt;   &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;gitlab_url&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;condition&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;test&lt;/span&gt;     &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"StringLike"&lt;/span&gt;
  &lt;span class="nx"&gt;variable&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"gitlab.com:sub"&lt;/span&gt;
  &lt;span class="nx"&gt;values&lt;/span&gt;   &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="s2"&gt;"project_path:${each.value}:ref_type:branch:ref:*"&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;condition&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;test&lt;/span&gt;     &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"StringEquals"&lt;/span&gt;
  &lt;span class="nx"&gt;variable&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"gitlab.com:namespace_id"&lt;/span&gt;
  &lt;span class="nx"&gt;values&lt;/span&gt;   &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;gitlab_namespace_id&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important values here are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;aud&lt;/code&gt;: checks the expected audience.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;sub&lt;/code&gt;: identifies the GitLab project and branch.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;namespace_id&lt;/code&gt;: identifies the GitLab namespace.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I especially like using &lt;code&gt;namespace_id&lt;/code&gt; because it is a stable numeric identifier.&lt;/p&gt;

&lt;p&gt;GitLab project and group names can change, but the namespace ID provides an additional way to make sure the token comes from the expected namespace.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 3 — One IAM role per repository
&lt;/h2&gt;

&lt;p&gt;My project uses four GitLab repositories, so I created a separate IAM role for each one.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Repository&lt;/th&gt;
&lt;th&gt;Main AWS permissions&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Backend&lt;/td&gt;
&lt;td&gt;Push Docker images to ECR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Frontend&lt;/td&gt;
&lt;td&gt;Push Docker images to ECR&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Infrastructure&lt;/td&gt;
&lt;td&gt;Manage AWS infrastructure with Terraform&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deploy&lt;/td&gt;
&lt;td&gt;Check ECR images and deploy to EKS&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Why separate them?&lt;/p&gt;

&lt;p&gt;Because the backend pipeline doesn't need permission to modify the EKS cluster.&lt;/p&gt;

&lt;p&gt;And the deployment pipeline doesn't need permission to push Docker images.&lt;/p&gt;

&lt;p&gt;Each pipeline gets only the permissions required for its job.&lt;/p&gt;

&lt;p&gt;For example, the deployment role only needs to check whether an image exists:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;actions&lt;/span&gt; &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
  &lt;span class="s2"&gt;"ecr:DescribeImages"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"ecr:DescribeRepositories"&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It cannot push a new image.&lt;/p&gt;

&lt;p&gt;This gives me a much cleaner separation of responsibilities.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 4 — Request the OIDC token from GitLab
&lt;/h2&gt;

&lt;p&gt;On the GitLab side, the configuration is surprisingly small.&lt;/p&gt;

&lt;p&gt;GitLab can generate the OIDC token directly inside the CI/CD job:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;id_tokens&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;GITLAB_OIDC_TOKEN&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;aud&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://gitlab.com&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then I define the AWS role and the location of the token:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;variables&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;AWS_REGION&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;us-east-1"&lt;/span&gt;
  &lt;span class="na"&gt;AWS_ROLE_ARN&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;arn:aws:iam::123456789012:role/gitlab-ci-book-infra"&lt;/span&gt;
  &lt;span class="na"&gt;AWS_WEB_IDENTITY_TOKEN_FILE&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;${CI_PROJECT_DIR}/.gitlab-oidc-token"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And write the token to the file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;before_script&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;echo "$GITLAB_OIDC_TOKEN" &amp;gt; "$AWS_WEB_IDENTITY_TOKEN_FILE"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's basically it.&lt;/p&gt;

&lt;p&gt;Terraform and AWS SDKs understand the web identity authentication flow automatically.&lt;/p&gt;

&lt;p&gt;There is no need to run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws configure
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And more importantly, I don't need to store permanent values such as:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;in GitLab CI/CD variables anymore.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 5 — EKS needs an extra configuration
&lt;/h2&gt;

&lt;p&gt;This was an important thing I learned while implementing the solution.&lt;/p&gt;

&lt;p&gt;Successfully authenticating with AWS does &lt;strong&gt;not&lt;/strong&gt; automatically mean that the pipeline can access Kubernetes.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws eks update-kubeconfig
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;may work correctly while:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl get pods
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;returns:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Unauthorized
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Why?&lt;/p&gt;

&lt;p&gt;Because there are two different authorization levels:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;AWS IAM
   │
   │ Can this role access the AWS/EKS API?
   ▼
EKS Cluster
   │
   │ Is this IAM role allowed inside Kubernetes?
   ▼
Kubernetes resources
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The IAM role therefore also needs access to the EKS cluster.&lt;/p&gt;

&lt;p&gt;I configured this with an EKS access entry:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"aws_eks_access_entry"&lt;/span&gt; &lt;span class="s2"&gt;"ci_deploy"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;cluster_name&lt;/span&gt;  &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cluster_name&lt;/span&gt;
  &lt;span class="nx"&gt;principal_arn&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ci_deploy_role_arn&lt;/span&gt;
  &lt;span class="nx"&gt;type&lt;/span&gt;          &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"STANDARD"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And associated the required EKS access policy with it.&lt;/p&gt;

&lt;p&gt;Once this was configured, the deployment pipeline could authenticate correctly to the cluster.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Terraform + EKS issue I ran into
&lt;/h2&gt;

&lt;p&gt;This was probably the most interesting problem I encountered.&lt;/p&gt;

&lt;p&gt;EKS authentication tokens are short-lived.&lt;/p&gt;

&lt;p&gt;Initially, I used:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;data&lt;/span&gt; &lt;span class="s2"&gt;"aws_eks_cluster_auth"&lt;/span&gt; &lt;span class="s2"&gt;"this"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cluster_name&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This works when &lt;code&gt;terraform plan&lt;/code&gt; and &lt;code&gt;terraform apply&lt;/code&gt; happen immediately.&lt;/p&gt;

&lt;p&gt;But my pipeline separates them.&lt;/p&gt;

&lt;p&gt;The flow is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;terraform plan
      │
      ▼
Manual approval
      │
      ▼
terraform apply
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The problem is that the EKS token can be generated during &lt;code&gt;terraform plan&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;If I wait before approving the &lt;code&gt;apply&lt;/code&gt;, the saved token may already be expired.&lt;/p&gt;

&lt;p&gt;Terraform then fails when trying to communicate with Kubernetes.&lt;/p&gt;




&lt;h2&gt;
  
  
  The solution: generate the token when it is needed
&lt;/h2&gt;

&lt;p&gt;Instead of storing the token during the plan, I use an &lt;code&gt;exec&lt;/code&gt; block:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="nx"&gt;provider&lt;/span&gt; &lt;span class="s2"&gt;"kubernetes"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;host&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cluster_endpoint&lt;/span&gt;

  &lt;span class="nx"&gt;cluster_ca_certificate&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;base64decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cluster_certificate_authority_data&lt;/span&gt;
  &lt;span class="p"&gt;)&lt;/span&gt;

  &lt;span class="nx"&gt;exec&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;api_version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"client.authentication.k8s.io/v1beta1"&lt;/span&gt;
    &lt;span class="nx"&gt;command&lt;/span&gt;     &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"aws-iam-authenticator"&lt;/span&gt;
    &lt;span class="nx"&gt;args&lt;/span&gt;        &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="s2"&gt;"token"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="s2"&gt;"-i"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;eks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;cluster_name&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now the token is generated when Terraform actually needs to connect to Kubernetes.&lt;/p&gt;

&lt;p&gt;So even if I wait before approving &lt;code&gt;terraform apply&lt;/code&gt;, Terraform gets a fresh token.&lt;/p&gt;




&lt;h2&gt;
  
  
  Another small GitLab CI lesson
&lt;/h2&gt;

&lt;p&gt;Manual GitLab jobs can allow failures by default depending on how they are configured.&lt;/p&gt;

&lt;p&gt;For an infrastructure deployment, that's something I definitely don't want.&lt;/p&gt;

&lt;p&gt;If &lt;code&gt;terraform apply&lt;/code&gt; fails, the pipeline should fail too.&lt;/p&gt;

&lt;p&gt;So I explicitly use:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apply&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;rules&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;if&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;$CI_COMMIT_BRANCH'&lt;/span&gt;
      &lt;span class="na"&gt;when&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;manual&lt;/span&gt;
      &lt;span class="na"&gt;allow_failure&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's a small configuration detail, but an important one for infrastructure pipelines.&lt;/p&gt;




&lt;h2&gt;
  
  
  Debugging OIDC authentication
&lt;/h2&gt;

&lt;p&gt;When authentication fails, I usually check three things first:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;aud
sub
namespace_id
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Typical errors include:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Error&lt;/th&gt;
&lt;th&gt;What I check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;AssumeRoleWithWebIdentity&lt;/code&gt; denied&lt;/td&gt;
&lt;td&gt;Project path / &lt;code&gt;sub&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Incorrect token audience&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;aud&lt;/code&gt; configuration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;InvalidIdentityToken&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Token expiration / OIDC provider&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;Unauthorized&lt;/code&gt; from &lt;code&gt;kubectl&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;EKS access entry&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;I also keep this command in the pipeline:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws sts get-caller-identity
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It's very useful because it immediately shows which AWS identity the pipeline is actually using.&lt;/p&gt;




&lt;h2&gt;
  
  
  What I learned
&lt;/h2&gt;

&lt;p&gt;The biggest lesson from this implementation is that OIDC is not as complicated as it first looks.&lt;/p&gt;

&lt;p&gt;The basic idea is simply:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;GitLab proves its identity
        ↓
AWS verifies the GitLab project
        ↓
AWS STS provides temporary credentials
        ↓
The pipeline accesses AWS
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The main advantages for me are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;No permanent AWS credentials in GitLab.&lt;/li&gt;
&lt;li&gt;No Access Keys to rotate.&lt;/li&gt;
&lt;li&gt;Temporary credentials expire automatically.&lt;/li&gt;
&lt;li&gt;Each repository has its own IAM role.&lt;/li&gt;
&lt;li&gt;Permissions can be limited per pipeline.&lt;/li&gt;
&lt;li&gt;AWS activity can be traced through CloudTrail.&lt;/li&gt;
&lt;li&gt;GitLab can securely deploy to EKS.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;There are a few extra details when EKS is involved, especially around cluster access and short-lived Kubernetes tokens, but once those pieces are understood, the overall architecture is quite simple.&lt;/p&gt;

&lt;p&gt;For CI/CD pipelines, I now prefer this approach over storing permanent AWS credentials whenever OIDC federation is available.&lt;/p&gt;

</description>
      <category>aws</category>
      <category>terraform</category>
      <category>gitlab</category>
      <category>devops</category>
    </item>
  </channel>
</rss>
