DEV Community: Mohamed Radwan

How "Vibe Coding" Accidentally Turned My EC2 Instance Into a Cryptominer

Mohamed Radwan — Wed, 01 Jul 2026 11:37:43 +0000

A few days ago I got an email I did not expect: an abuse report from AWS Trust & Safety, saying my EC2 instance in my personal AWS account had been caught scanning other hosts on the internet.
My first thought was the obvious one someone hacked my box.
What I actually found was more interesting (and honestly, kind of embarrassing): a single line in a package.json file was all it took.

Here's the full story of how it happened, how I tracked it down, and what actually fixed it.

The abuse report
The email included a forwarded report from a third party network. They'd logged multiple completed TCP handshakes from my instance's public IP, hitting a handful of destination IPs on ports like 9200 (commonly Elasticsearch), 443, and 80.
The "completed handshake" detail matters — it rules out IP spoofing. Both sides have to respond for a handshake to finish, so this wasn't some rando forging my IP. My instance really was reaching out and probing other hosts.
Time to figure out why.

First theory: someone broke in
Outbound scanning traffic screams "compromised box," so I started where anyone would: how did they get in?

SSH brute force?
Nope. SSH was key only, and every entry in auth.log was Accepted publickey from IPs I recognized. No password attempts, no unfamiliar sources.

Accepted publickey for ubuntu from <trusted-ip> port 57322 ssh2: RSA SHA256:...

An exposed service?

Everything else was completely unreachable from the internet.

The security group sat in front of all of it. Dead end.
So: no brute force, no exposed service. Whatever this was, it didn't come in through the front door.

Finding the malware
A quick find / -mtime -2 (files modified in the last 2 days) turned up something ugly, sitting right in my frontend app's directory:

scanner_linux
xmrig.tar.gz
scanner_deployed.log
exploited.log
failed.log
monitor.log
data.log

xmrig is a legitimate (but heavily abused) Monero mining tool. scanner_linux was a binary I'd never seen before — and ps aux confirmed it was actively running, as root, chewing through CPU:

root  285679  27.7% CPU  ./scanner_linux -t 1000

There it was. That process was almost certainly what generated the traffic AWS flagged.

Both files were owned by root, written within about a minute of each other, at a time that didn't line up with any of my own SSH sessions. Whatever dropped them had root level filesystem access but never touched SSH to get it.

Following the trail outward
If nothing got in from the outside, something on the inside must have reached out. I checked the security group's outbound rule:
Outbound: All traffic → 0.0.0.0/0

The actual root cause
Digging through the frontend app's package.json, one line jumped out immediately:

"dependencies": {
    "zod": "latest",
    "child_process": "latest",
    "aws-amplify": "latest",
    "axios": "latest"
}

child_process is not a real npm package. It's a built-in Node.js core module, no install required, ever.
There is zero legitimate reason for it to show up as an external dependency.
But somebody had published a package under that exact name on the public npm registry. This is a known attack pattern: squat on a name that looks like a trusted standard-library module, and wait for someone (a developer copy-pasting a snippet, or these days, an AI coding assistant hallucinating a dependency) to install it by mistake.

These packages almost always ship a postinstall script, which npm runs automatically the second npm install finishes — silently, with whatever permissions the install process has. In my case, that was root, inside a container with a bind-mounted host directory. Every rebuild of that container had been quietly re-running it.

The fix: remove the child_process line from package.json, wipe node_modules and the lockfile, and rebuild from scratch with docker compose up -d --build — the --build flag matters, since a plain restart just resumes the already-infected image.

What I'm taking away from this

A locked-down security group only stops external attackers. It does nothing once malicious code is already executing locally with your own privileges.
Wide-open outbound rules are underrated as a risk. 0.0.0.0/0 outbound is a common default, but it's exactly what let a tiny postinstall script fetch a cryptominer. Restricting egress would've stopped this cold even after the bad package installed.
Any dependency with the same name as a Node built-in is an instant red flag. child_process, fs, http, etc. should never appear in your dependencies.
Stop pinning things to "latest". Half my dependency list was unpinned, which makes it way harder to spot when something new and unwanted sneaks in.
npm audit and a manual skim of package.json should be routine, not just a post incident activity.

[Boost]

Mohamed Radwan — Sat, 19 Jul 2025 20:39:04 +0000

Mohamed Radwan for AWS Community Builders

Jun 21 '25

Enriching Keycloak with LinkedIn VanityName, Headline & Profile Picture via Custom SPI

#keycloak #opensource

3 min read

[Boost]

Mohamed Radwan — Sun, 13 Jul 2025 12:39:14 +0000

Mohamed Radwan for AWS Community Builders

Jul 13 '25

Extract Invoice Data Automatically Using LangChain

#ai #langchain #aws #cognito

1 min read

Extract Invoice Data Automatically Using LangChain

Mohamed Radwan — Sun, 13 Jul 2025 12:01:54 +0000

In this article, I’m sharing an app I built to automate invoice processing using image recognition and language models. The goal is simple: take scanned or photographed invoices (in JPG, PNG, or PDF format) and extract structured data in JSON.

Under the hood, the system uses OpenAI’s GPT-4o (GPT-4 Vision) model via LangChain, and it’s wrapped in a lightweight FastAPI backend built with Python. The app can batch process files, run locally or in a container, and outputs clean JSON ready for downstream systems like accounting tools or CRMs.

Tech Stack & Architecture

The app is structured with a modern, API-first architecture:

Backend: Python with FastAPI, using LangChain + GPT-4o for invoice processing

Authentication: AWS Cognito for secure, scalable user auth

Database: MongoDB for storing processed invoice data and metadata

Frontend: A Next.js app handles the UI and connects to the backend via API

Backend Code

The authentication layer with AWS Cognito makes it easy to manage user sign-ups, login access. Invoice data and any synced product info are stored in MongoDB, which works well with JSON-like structures.
On the frontend, Next.js provides a fast and reactive UI that lets users upload invoice images, view extracted data, and manage syncing.

Sync Invoice Items with Product Barcodes

Once the app extracts line items from the invoice such as product names, you can take it a step further by syncing these items with your internal product database. This allows you to:

Match items by name
Automatically assign or update barcodes
Link products to existing inventory systems
Detect mismatches or missing items

Enriching Keycloak with LinkedIn VanityName, Headline & Profile Picture via Custom SPI

Mohamed Radwan — Sat, 21 Jun 2025 10:57:36 +0000

When integrating LinkedIn as an Identity Provider in Keycloak, you may quickly discover a limitation: by default, LinkedIn only provides basic user information such as email, first name, and last name.

However, in many real-world applications, you may also want to retrieve and store additional profile details, including:

Vanity name (public LinkedIn profile ID)

High-resolution profile picture

Professional headline (user's job title or tagline)

To achieve this, you'll need to create a custom Identity Provider Mapper SPI (Service Provider Interface) in Keycloak. This extension allows you to fetch additional data from LinkedIn's API and map it into the Keycloak user model during the authentication process.

Step 1: Set Up Your LinkedIn App Permissions

Before you begin implementing the custom SPI, make sure your LinkedIn developer app has the correct permissions. You'll need to request access to the following

These scopes allow your Keycloak instance to retrieve detailed user information such as their profile picture and professional headline.
Configuring LinkedIn as an Identity Provider in Keycloak (Tested on Keycloak 26.2.5)

After you've created a LinkedIn App and obtained the required credentials, follow these steps to configure LinkedIn in your example Keycloak realm:

Navigate to your Keycloak admin console.
Go to:
Identity Providers → Select LinkedIn.
Fill in the following fields:

Client ID: (from your LinkedIn app)
Client Secret: (from your LinkedIn app)

In the Scopes field, add the following permissions:

openid profile email r_basicprofile

These scopes are necessary to:

Authenticate the user (openid)
Retrieve profile information (profile, r_basicprofile)
Fetch the user’s verified email address (email)

Enable the following options:
Store Tokens (This is required for the SPI to extract the LinkedIn access token and make additional API calls.)
Trust Email (Assumes the email provided by LinkedIn is verified.)

Click Save.

To use this LinkedIn SPI extension in your Keycloak setup:

Download the JAR file
Either build it yourself using Maven, or download a prebuilt version from the GitHub Releases page.
Copy the JAR to Keycloak's provider directory

cp linkedin-profile-mapper-1.0.0.jar /opt/keycloak/providers/

Rebuild Keycloak to recognize the new provider

/opt/keycloak/bin/kc.sh build

Restart Keycloak

    /opt/keycloak/bin/kc.sh start

Source Code & Releases
You can find the full source code and latest releases at:

https://github.com/maradwan/keycloak-linkedin-profile-mapper

Then you need to add Identity Provider Mapper

Then you need to add attributes for vanityName, profilePicture, and headline in the User profile that is in the Realm settings of your example realm

Create an example-client in Keycloak

In your Keycloak Admin Console:

Go to your realm (e.g., example), then navigate to Clients → click Create client.
Enter example-client as the Client ID, choose OpenID Connect, and click Next.
Set Root URL to http://localhost:3000, and Valid Redirect URI to http://localhost:3000/callback, then click Save.
Go to the Credentials tab → copy the Client Secret.
Under Settings, enable Standard Flow, optionally enable Direct Access Grants.
Use these details in your app:

Test the Integration (Optional FastAPI Example)

You can test the LinkedIn login flow and see the enriched tokens by using a simple FastAPI application:

Install the required packages:

pip install fastapi uvicorn requests

# app.py
from fastapi import FastAPI, Request
from fastapi.responses import RedirectResponse
import requests

app = FastAPI()

# === CONFIG ===
KEYCLOAK_BASE_URL = "https://idp.example.com"
REALM = "example"
CLIENT_ID = "example-client"
CLIENT_SECRET = "XXXXXXXXX"  # Only needed for confidential clients
REDIRECT_URI = "http://localhost:3000/callback"  # Your app's redirect URI

# === ROUTES ===

@app.get("/login")
def login():
    return RedirectResponse(
        f"{KEYCLOAK_BASE_URL}/realms/{REALM}/protocol/openid-connect/auth"
        f"?client_id={CLIENT_ID}"
        f"&redirect_uri={REDIRECT_URI}"
        f"&response_type=code"
        f"&scope=openid"
    )

@app.get("/callback")
def callback(request: Request):
    code = request.query_params.get("code")
    if not code:
        return {"error": "No authorization code provided"}

    # Exchange code with Keycloak (not LinkedIn)
    token_response = requests.post(
        f"{KEYCLOAK_BASE_URL}/realms/{REALM}/protocol/openid-connect/token",
        data={
            "grant_type": "authorization_code",
            "code": code,
            "redirect_uri": REDIRECT_URI,
            "client_id": CLIENT_ID,
            "client_secret": CLIENT_SECRET,
        },
        headers={"Content-Type": "application/x-www-form-urlencoded"},
    )

    if token_response.status_code != 200:
        return {"error": "Token exchange failed", "details": token_response.json()}

    token_data = token_response.json()

    # Decode the token to inspect (optional)
    id_token = token_data.get("id_token")
    access_token = token_data.get("access_token")
    return {
        "access_token": access_token,
        "id_token": id_token,
        "message": "Use this token to call secured APIs or decode for LinkedIn data",
    }

Run the App

uvicorn app:app --reload --port 3000

Sources:
https://learn.microsoft.com/en-us/linkedin/marketing/quick-start?view=li-lms-2025-03#step-1-apply-for-api-access

[Boost]

Mohamed Radwan — Mon, 13 Jan 2025 09:23:36 +0000

Tetiana Mostova for AWS Community Builders

Jan 12 '25

How I automated Certificate expiration alerts with AWS

#aws #security #automation #devops

5 min read

[Boost]

Mohamed Radwan — Wed, 25 Dec 2024 12:34:15 +0000

Ravindra Singh for AWS Community Builders

Sep 27 '24

Installing ArgoCD and Securing Access Using Amazon Cognito

#kubernetes #eks #devops #cognito

6 min read

Optimize AWS Cloud Costs

Mohamed Radwan — Mon, 11 Nov 2024 18:01:21 +0000

Optimize AWS cloud costs by implementing the following strategies:

VPC Optimization:

Manage Data Transfer Costs: NAT gateways incur costs; in most cases, two subnets with 99.9% availability are sufficient. For 99.99% availability, use three subnets with three NAT gateways. Two subnets are generally sufficient for most scenarios.
Remove IP v4 addresses that are not needed or associated with Network Interface Card (NIC).
Use VPC endpoints to privately connect to supported AWS services and VPC endpoint services, rather than relying on NAT gateways, to reduce costs. Additionally, AWS PrivateLink is often required for other purposes, such as compliance, irrespective of cost considerations.

EKS/EC2/Auto Scaling Group (ASG) Optimization:

Implement Instance Auto scaling: Configure autoscaling with Spot instances for worker nodes by using Karpenter to adjust resources based on demand dynamically.
Use Reserved Instances and Savings Plans: Optimize for a single node type, such as m5a.large or m5a.xlarge , for most applications to gain cost efficiencies through reserved capacity.
Rightsize Resources: Regularly reassess applications and nodes to evaluate utilization and rightsize resources for optimized performance and cost.
Use scheduled actions in the Auto Scaling Group ASG for test and development environments to automatically stop instances at specified times when compute nodes are not in use and start them when needed.
CPU Type: ARM-based CPUs (such as AWS Graviton) are more cost-effective but may have compatibility issues with certain applications. Developers should ensure application compatibility to support multiple CPU types.
Upgrades: Cloud providers may impose fees for clusters that are not upgraded to a supported version. For EKS, using outdated Kubernetes Cluster requires Extended Support, resulting in additional expenses.

EBS Optimization:

Ensure that EBS volumes are configured to use the GP3 storage type, which is 20% more cost-effective than GP2.
Delete unused EBS volumes and snapshots that are no longer in use.

S3 Optimization:

Optimize Storage Costs for Object Storage: Configure lifecycle policies, especially when using versioning, and choose the appropriate storage class (e.g., Standard or Infrequent Access). Consider S3 Intelligent Tiering for automated cost-efficient storage management.

Database Optimization:

For RDS, consider rightsizing, using Graviton-based instances, and performing mass engine version upgrades if applications support the new version.
Switching to Amazon Aurora I/O-Optimized increases instance costs by approximately 25% to include I/O operations, making it an ideal choice for high I/O workloads. This option offers improved cost predictability and can potentially reduce total costs when I/O demand is significant.
Upgrades: Cloud providers may impose fees for clusters that are not upgraded to a supported version. For RDS, using outdated database engines may require Extended Support, resulting in additional expenses.

Take IT Shipping with Serverless Technology

Mohamed Radwan — Thu, 21 Mar 2024 12:35:59 +0000

In this article, I'll take you through the journey of creating a mobile application utilizing serverless architecture.

The app idea is a platform that facilitates and supports peer-to-peer shipping services, connecting requesters and travelers.

FrontEnd Flutter Code
Backend Lambda Code

The App Architecture:

Frontend Framework (Flutter):

Flutter serves as the frontend framework for developing the mobile application.
Flutter allows for cross-platform development, enabling the app to run on both iOS and Android devices from a single codebase.

Backend Services (Lambda):

The backend service is built using Lambda, a serverless computing platform.
Lambda functions are deployed in response to events triggered by user actions, ensuring scalability and cost-efficiency.
"The Take IT" app utilizes Python Flask within the Lambda functions

Authentication (Cognito):

Cognito, a serverless authentication service, manages the sign-up/sign-in process in the Take IT app. Additionally, it seamlessly integrates with identity providers such as Google or Facebook. Authentication is typically managed using JWT (JSON Web Tokens) for secure and efficient user authentication.

API Gateway:

Integrating AWS API Gateway with Cognito offers enhanced security and scalability for the Take IT app.
API Gateway ensures only authenticated users can access backend services.

Database (DynamoDB):

DynamoDB serves as the serverless database utilized to store all data in the Take IT app. It is leveraged to manage user preferences, trip details, contacts, as well as the status of sent and received requests (including accepted, pending, and declined requests).

Adding a Trip

The function looks like this:

def add_trip(item):
    return query_table.put_item(Item=item)

Below is an example of how a trip is saved into DynamoDB in JSON format:

{
 "username": "4fb1dce7-6a50-41a8-8d7d",
 "created": "2024-10-04-19-34-18-736061",
 "acceptfrom": "2024-10-05",
 "acceptto": "2024-10-30",
 "allowed": {
  "Clothes": {
   "cost": "3.0",
   "kg": "10.0"
  },
  "Electronics": {
   "cost": "20.0",
   "kg": "5.0"
  }
 },
 "currency": "EUR",
 "fromcity": "Berlin-Germany",
 "fromto": "Berlin-Germany_Cairo-Egypt",
 "tocity": "Cairo-Egypt",
 "trdate": "2024-10-31",
 "tstamp": 1732961762
}

By utilizing the TTL (Time to Live) feature in DynamoDB, you can automatically delete records after a specified time period. For instance, in the trip record, there is an attribute called "tstamp" that determines the deletion time of the record.

Find available trips

The attributes in the trip record, such as "fromCity," "toCity," or "fromTo," are utilized for search functionality when users seek trips. I employ a global secondary index in DynamoDB to retrieve trips based on the originating city, destination city, or the combination of both.

The function looks like this:

Index: This refers to the name of the global secondary index.
Key: This represents the DynamoDB attribute used for indexing, typically referring to "fromCity" or "toCity."
City: Denotes the name of the city being referenced.
Limit: Specifies the maximum number of records to retrieve.
Today_date: This indicates the current date, used to filter and display only trips available from today onwards.
LastKey: Utilized for pagination purposes, facilitating the retrieval of subsequent sets of records beyond the initial limit.

def get_global_index(index,key,city,limit,today_date,lastkey=None):

    if lastkey:
        return query_table.query(
        IndexName=index,KeyConditionExpression=Key(
            key).eq(city),Limit=int(limit),FilterExpression=Attr('acceptto').gte(today_date),ExclusiveStartKey=json.loads(lastkey))

    return query_table.query(
        IndexName=index,KeyConditionExpression=Key(
            key).eq(city),Limit=int(limit),FilterExpression=Attr('acceptto').gte(today_date))

Users Sent/Received Requests
DynamoDB supports querying data with a key that begins with a specific value. When users send or receive requests in the Take IT app, these requests are stored in DynamoDB with statuses such as "pending," "accepted," "request," or "declined." This allows for efficient querying and retrieval of requests based on their status, enabling seamless management and tracking of request statuses within the application.

The function looks like this:

item: This refers to "pending," "accepted," "request," or "declined."

query_table.query(KeyConditionExpression=Key("username").eq(username) & Key("created").begins_with(item))

Here's an example of how a user request is saved into DynamoDB in JSON format:

{
 "username": "user2-466b-a4b9-94f90",
 "created":  "request_2022-10-15-22-35-18-213147_user1-4fb1dce7-6a50",
 "dtime": "2022-10-15-22-42-41-599518",
 "tripid": "2022-10-15-22-35-18-213147",
 "tstamp": 1669766400
},
{
 "username": "user1-4fb1dce7-6a50",
 "created": "pending_2022-10-15-22-35-18-213147_user2-466b-a4b9-94f90",
 "dtime": "2022-10-15-22-42-41-599518",
 "tripid": "2022-10-15-22-35-18-213147",
 "tstamp": 1669766400
}

Remove Account
When a user decides to delete their account in the Take IT app, several steps are initiated:

Querying User Data: All data associated with the user is queried from DynamoDB, including trip history, pending requests, and any other relevant information.

Deleting User Data: Each record associated with the user's account is deleted from DynamoDB.
This includes trip records, request records (both sent and received), and any other user-specific data stored in the database.

Removing User from Cognito: The user is removed from the Cognito user pool, deleting their account and associated authentication tokens.

The Cognito delete function looks like this:

cognito = boto3.client('cognito-idp',region_name = region_name, verify=True)
cognito.admin_delete_user(UserPoolId= userpoolid, Username= username)

The delete function looks like this:

def delete_records(username):
   items= query_table.query(
       KeyConditionExpression=Key("username").eq(username)
   )
   for i in range(len(items['Items'])):
       query_table.delete_item(
       Key={
           'username': username,
           'created' : items['Items'][i]['created']
           }
           )
   return True

Configure a privately hosted Git repository for EMR Studio

Mohamed Radwan — Thu, 12 Oct 2023 09:13:02 +0000

By default, you can access GitHub and GitLab from the studio workspace. If you are using a private Git repository, follow these steps:

Create a Studio: You should use a VPC with private subnets that have a NAT gateway to enable communication with the internet.
Ensure that the security group for the default workspace has an outbound rule allowing HTTPS traffic on port 443 to the destination 0.0.0.0/0
Create VPC Endpoint:

Make sure the endpoint uses private subnets.
Confirm that the security group attached to the endpoint allows inbound rules for HTTPS traffic on port 443 with the source set to either "Your VPC Address" or "0.0.0.0/0".
Upload the following configuration file into your Amazon S3 storage location that is used for your Studio in a folder called life-cycle-configuration:
s3://BUCKET-NAME/life-cycle-configuration/configuration.json

GitServerDnsName - The DNS name of your Git server. For example "git.example.com".

GitServerIpV4List - A list of IPv4 addresses that belong to your Git servers, the example VPC CIDR is 10.0.0.0/16, DNS is 10.0.0.2

[
    {
        "Type": "PrivatelyHostedGitConfig",
        "Value": [
            {
                "DnsServerIpV4": "10.0.0.2",
                "GitServerDnsName": "git.example.com",
                "GitServerIpV4List": [
                    "1.2.3.4"
                ]
            }
        ]
    }
]

"If you are facing issues, you may need to stop the workspace and start it again."

Fix Cert-Manager Conflict with EKS

Mohamed Radwan — Wed, 29 Mar 2023 21:45:03 +0000

I was facing issue with multiple managed worker nodes running on EKS clusters.

The issue was appearing randomly in different nodes, I cannot access the pods or get the logs by kubectl.

x509: cannot validate certificate for 10.0.83.153 because it doesn’t contain any IP SANs

Kube API in the CloudWatch showing the following errors:

E0327 08:54:17.406029 11 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"error dialing backend: x509: cannot validate certificate for 10.0.83.153 because it doesn't contain any IP SANs"}: error dialing backend: x509: cannot validate certificate for 10.0.83.153 because it doesn't contain any IP SANs

After investigating the issue with the AWS EKS support team, we found that cert-manager-webhook is causing the issue.
Kubelet certificate chain is being used from cert-manager-webhook-ca.

Run the following command on the non-working node:

openssl s_client -connect localhost:10250

CONNECTED(00000003)
---
Certificate chain
 0 s:
   i:/CN=cert-manager-webhook-ca
---
Server certificate
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
subject=
issuer=/CN=cert-manager-webhook-ca
---

Run the following command on the working healthy node:

openssl s_client -connect localhost:10250

CONNECTED(00000003)
---
Certificate chain
 0 s:/O=system:nodes/CN=system:node:ip-10-0-31-151.eu-west-1.compute.internal
   i:/CN=kubernetes
---
Server certificate
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
subject=/O=system:nodes/CN=system:node:ip-10-0-31-151.eu-west-1.compute.internal
issuer=/CN=kubernetes
---

The cert-manager-webhook deployment uses port 10250 which is also used for kubelet.

The solution is change the port of cert-manager-webhook to 10260.

By setting webhook.securePort to 10260

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.10.0 \
  --set webhook.securePort=10260

Sources:

https://cert-manager.io/docs/concepts/webhook/
https://cert-manager.io/docs/installation/compatibility/#aws-eks

Integrate Keycloak and Kubeapps on AWS EKS

Mohamed Radwan — Sun, 25 Sep 2022 09:47:15 +0000

In this article, I am going to show you how to integrate keycloak with kubeapps on AWS EKS.

Steps:

Create EKS Cluster
Register a domain in route53 or create a subdomain, ex: example.com
Request a certificate from ACM
Install Keycloak

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/keycloak --version 9.8.1

No needs to use the service Loadbalancer for keycloak as we will use the Nginx ingress controller.
Let's switch the service of keycloak to ClusterIP.

kubectl patch svc my-release-keycloak  -p '{"spec": {"type": "ClusterIP"}}'

Note the admin password of keycloak

kubectl get secrets/my-release-keycloak -o jsonpath='{.data.admin-password}' | base64 --decode

Install Nginx Ingress Controller Update the AWS ACM certificate on the below installation

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install nginx ingress-nginx/ingress-nginx --set controller.service.type=LoadBalancer  --set controller.service.targetPorts.https=http --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-ssl-cert"="arn:aws:acm:eu-west-1:XXXX:certificate/XXXX-XXX-XXX-XXXX-XXXXX"  --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-backend-protocol"=http --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-ssl-ports"=443   --set-string controller.config.use-forwarded-headers="true" --version 4.2.3

Create ingress for keycloak

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-buffer-size: 10k
  name: keycloak
  namespace: default
spec:
  rules:
  - host: auth.example.com
    http:
      paths:
      - backend:
          service:
            name: my-release-keycloak
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific

Login to the keycloak console https://auth.example.com, with username: user and password in the keycloak installation step.

Create a new realm, name it kubeapps

Groups Claim
By default, there is no "groups" scope/claim. We will create a global client scope for groups.

In the admin console:

Click "Client Scopes" from the left navigator menu
Click on "Create" from the table (top right corner)
Provide a name, ensure the protocol is set to "openid-connect" and that the option "Include in Token Scope" is on.

Once the client scope is created, you should be redirected to a page with several tabs. Navigate to the "Mappers" tab as we need to create a mapper to populate the value of the associated claim:

Click on the "Mappers" tab
Click on "Create" from the table to create a new mapper
Configure:
Enter a name
Select "Group Membership" as the claim type
Enter "groups" as the token claim name
Ensure the "Full group path" is OFF
Keep the other knobs as ON
Click ‘Save'

Kubeapps Client on Keycloak

Click "Clients" from the left navigator
Click "Create" from the table
Enter an "id" and Save (e.g. kubeapps)
Once created, configure the authentication as follows:
Ensure the protocol is set to "openid-connect"
Configure the "Access Type" to be "confidential". This will add a new "Credentials" tab from which you can get the client secret
Ensure "Standard Flow Enabled" is enabled, this is required for the login screen.
"Direct Access Grants Enabled" can be disabled.
In the "Valid Redirect URIs" field, enter "https://kubeapps.example.com/*" as a placeholder.
Save

Note: take copy of the secret in Credentials

As for the cluster clients, we need to configure the client scopes:

Click the "Client Scopes" tab
Ensure the "email" scope is available either in the "Assigned Default Client Scopes" list or the "Assigned Optional Client Scopes" list
The "groups" client scope should be available in the lists on the left.
Add it either to the "Assigned Default Client Scopes" list or the "Assigned Optional Client Scopes" list.

Note the issuer URL from keycloak Realm

Realm Settings → Endpoints → OpenID Endpoint Configuration then copy the issuer URL
ex: https://auth.example.com/realms/kubeapps

On the EKS Cluster:
To the EKS cluster and then to Authentication

Add the identity provider like the following:
Issuer URL: https://auth.example.com/realms/kubeapps
Client ID: kubernetes
Username claim: email
Groups claim: groups

the process takes around 20 minutes.

Back to Keycloak

Create a new Client

Click "Clients" from the left navigator
Click "Create" from the table
Enter an "id" kubernetes and Save

Once created, configure the authentication as follows:

Ensure the protocol is set to "openid-connect"
Configure the "Access Type" to be "confidential"
turn on the "Standard Flow Enabled"
Ensure "Direct Access Grants Enabled" is enabled, as this is how we can get the tokens via API
Save

On Kubeapps client

Clients → Mappers → Create

Create a group
The group name is admin
Groups --> New

Add users to join the group
Users --> Your User --> Groups --> choose admin --> join

Install Kubeapps

Update the following:

clientSecret gets it from the above step Kubeapps Client on keycloak
issuer-url and redirect-url for your domain.

helm repo add bitnami https://charts.bitnami.com/bitnami

helm install kubeapps bitnami/kubeapps \
  --set authProxy.enabled=true \
  --set authProxy.provider=oidc \
  --set authProxy.clientID=kubeapps \
  --set authProxy.clientSecret=<secret client credentials in the kubeapps client> \
  --set authProxy.cookieSecret=$(echo "not-good-secret" | base64) \
  --set authProxy.extraFlags="{--cookie-secure=false,--oidc-issuer-url=https://auth.example.com/realms/kubeapps,--redirect-url=https://kubeapps.example.com/oauth2/callback}" \

Create an ingress for kubeapps

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffer-size: 10k
  name: kubeapps
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: kubeapps.example.com
    http:
      paths:
      - backend:
          service:
            name: kubeapps
            port:
              number: 80
        pathType: ImplementationSpecific

Important to add proxy-buffer-size on the ingress controller otherwise you will get an error that says "upstream sent too big header while reading response header from upstream"

Create a new user on the keycloak, make sure you add email, and Email Verified is on.

The last step we need to create a role binding to access the Kubernetes cluster, this group has the same name admin as the keycloak group

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  managedFields:
  - apiVersion: rbac.authorization.k8s.io/v1
  name: keycloak-admin-group
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: admin

Finally, log in to Kubeapps at https://kubeapps.example.com

If you want to connect to EKS cluster by using OIDC

Install kubelogin:

Go to keycloak and then go back to the Kubernetes client we created. You will see a tab called Credentials, grab the client secret.
Run the below command in your terminal to verify authentication to keycloak:

kubectl oidc-login setup --oidc-issuer-url=https://auth.example.com/realms/kubeapps --oidc-client-id=kubernetes --oidc-client-secret=XXXXXXXXXXX

Bind a cluster role

kubectl create clusterrolebinding oidc-cluster-admin --clusterrole=cluster-admin --user='https://auth.example.com/realms/kubeapps#f5092396-d3b8-452c-9d1a-2b1dfbd58718'

Set up the kubeconfig
Use the same secret of Kubernetes client

kubectl config set-credentials oidc \
      --exec-api-version=client.authentication.k8s.io/v1beta1 \
      --exec-command=kubectl \
      --exec-arg=oidc-login \
      --exec-arg=get-token \
      --exec-arg=--oidc-issuer-url=https://auth.example.com/realms/kubeapps \
      --exec-arg=--oidc-client-id=kubernetes \
      --exec-arg=--oidc-client-secret=XXXXXXX \
          --exec-arg=--oidc-extra-scope=groups

Verify cluster access

kubectl --user=oidc get nodes

Switch the default context to oidc

kubectl config set-context --current --user=oidc

In case you have issues, you need to check if your token has added admin group:

"groups": [
"admin"
]

curl -L -X POST 'https://auth.example.com/realms/kubeapps/protocol/openid-connect/token' \
-H 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=kubernetes' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'client_secret=XXXXXXX' \
--data-urlencode 'scope=openid email groups' \
--data-urlencode 'username=mohamed.radwan' \
--data-urlencode 'password=XXXXX' \
| jq -r .id_token

check your token by jwt.io

Sources:
https://docs.aws.amazon.com/eks/latest/userguide/authenticate-oidc-identity-provider.html
https://github.com/vmware-tanzu/kubeapps/blob/main/site/content/docs/latest/howto/OIDC/OAuth2OIDC-keycloak.md