DEV Community: Afzal Ansari

Building a resilient, scalable AWS Lambda + S3 architecture

Afzal Ansari — Sat, 17 Jan 2026 11:55:23 +0000

I’ve built and reviewed several serverless systems where AWS Lambda and Amazon S3 form the backbone—file ingestion pipelines, media processing platforms, and event-driven APIs. Over time, I noticed a recurring challenge: people either see Lambda + S3 as too simple ("just trigger a function on upload") or too abstract when diagrams become overwhelming.

In this article, I’ll walk you through how I think about designing a complex yet easy-to-understand Lambda + S3 architecture, using real-world patterns and the latest AWS capabilities. I’ll also show you how I draw this architecture in Lucidchart, so I can explain it clearly and broadly.

What I’m trying to solve

When I design serverless systems, I usually want three things:

Simple entry points for users and clients
Asynchronous, resilient processing behind the scenes
Strong cost and operational control as the system scales

Instead of thinking in terms of individual AWS services, I divide the architecture into layers. This mental model also maps very well to a Lucidchart diagram.

Edge / Client layer

This is where requests originate:

Web or mobile clients
CLI tools or third‑party webhooks

Most of the time, clients never talk to Lambda directly. I prefer to keep a clean boundary.

API & ingress layer

Here I typically use:

API Gateway for REST or HTTP APIs
Lambda Function URLs for very focused, internal endpoints
CloudFront + WAF when the API is public and needs protection

This layer is responsible only for authentication, validation, and routing—not heavy logic.

Compute layer (Lambda)

This is where Lambda shines.

I usually split responsibilities across multiple functions:

Request Lambdas – fast, synchronous (auth, presigned URLs)
Processor Lambdas – async workers (image resize, metadata extraction)
Indexer Lambdas – background tasks (search indexing, embeddings)

Storage & data plane (S3‑centric)

This is the heart of the system.

I usually work with multiple buckets, each with a clear purpose:

ingest-bucket – raw uploads
processed-bucket – derived artifacts
archive-bucket – long‑term retention

Key S3 features I actively use:

Direct uploads with presigned URLs (clients bypass Lambda)
S3 Intelligent‑Tiering to control costs automatically
S3 Object Lambda when I want on‑the‑fly transformations
S3 Batch Operations + Lambda for large‑scale reprocessing jobs

Below is the infra diagram for the above structure:

Final thoughts

Lambda and S3 are often introduced as simple services, but together they can power extremely sophisticated systems. The key is to:

Keep synchronous paths small
Push everything else to events
Let S3 do what it does best: scale and store cheaply

End to end argo-workflow for CI/CD

Afzal Ansari — Wed, 14 May 2025 14:24:40 +0000

If you're just getting started with GitOps or CI/CD pipelines in Kubernetes, Argo Workflows offers a powerful and Kubernetes-native way to automate your build pipelines.

In this blog, we’ll walk through a complete Continuous Integration (CI) and Continuous Deployment (CD) pipeline using Argo Workflows that performs the following steps:

🔁 The Pipeline:

✅ Clone a GitHub repository
🔧 Build and push a Docker image using BuildKit
🔐 Scan the Docker image for vulnerabilities
🔍 Scan Kubernetes manifests for misconfigurations
⏭ Deploy Kubernetes manifests to your cluster

Let’s break down each part of the workflow. But, before doing so, let's first understand basics of my argo workflow template.

⚙️ Understanding the Argo Workflow Structure
Argo Workflows is a Kubernetes-native workflow engine where each CI/CD process is defined as a custom resource (Workflow or WorkflowTemplate). In my case, I used a WorkflowTemplate named ci-build-workflow, which can be triggered manually or automatically through a CronWorkflow.

The workflow is defined using Argo’s CRD (Custom Resource Definition), and it includes metadata such as name or generateName. This value is used as a prefix for naming the pods in which each workflow step (or template) runs.

📦 Volume Handling
Just like regular Kubernetes pods, you can declare volumes in Argo workflows. These volumes—like work or buildkitd—are mounted across different steps, enabling shared storage between containers. This is especially useful for tasks that rely on common directories (like cloning a repo and building from it).

🔁 Parameterized Arguments
Argo allows the use of parameters in workflows to make them dynamic. In my CI workflow, I’ve defined parameters such as the GitHub repo owner, repo name, Docker image tag, registry name, etc. These parameters can be passed into the workflow during runtime or set statically in a CronWorkflow.

🧩 Workflow Entry Point: DAG Template

- name: main
  dag:
    tasks:
      - name: clone-repo
        ...
      - name: build-image
        ...
        depends: clone-repo
      - name: scan-image
        ...
        depends: build-image
      - name: scan-k8s
        ...
        depends: clone-repo && build-image
     - name: deploy-kubernetes
       ...
       depends: build-image && scan-k8s && clone-repo

The pipeline uses a DAG (Directed Acyclic Graph) to run tasks in order:

scan-k8s and build-image both run after cloning
scan-image runs after the image is built
deploy-kubernetes runs after the image is built, k8s manifests scanned and cloning a repo.

This setup ensures tasks are run in the right order without unnecessary blocking.

📂 Volumes & Storage
The workflow uses a shared Persistent Volume Claim (PVC) to exchange files between steps:

volumeClaimTemplates:
- metadata:
    name: work
  spec:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 64Mi

Additionally, BuildKit uses an emptyDir volume for its internal state:

volumes:
  - name: buildkitd
    emptyDir: {}

Before we can build and push container images to a Docker registry like Docker Hub, we need to authenticate. Argo Workflows uses Kubernetes secrets to securely store and access these credentials during execution.

In our workflow, the build and push steps rely on credentials to authenticate with Docker Hub:

volumes:
- name: docker-creds
  secret:
    secretName: '{{inputs.parameters.docker_secret_name}}'
    items:
    - key: .dockerconfigjson
      path: config.json

This volume mount makes your Docker config file available inside the container at runtime, which is required for tools like BuildKit to push images securely.

How to Create the Docker Secret:
To generate the .dockerconfigjson file, you can create the Kubernetes secret using:

kubectl create secret generic docker-config \
  --from-file=.dockerconfigjson=$HOME/.docker/config.json \
  --type=kubernetes.io/dockerconfigjson

In the workflow, we refer to this secret using:

- name: docker_secret_name
  value: docker-config

With the secret in place, your workflow can securely authenticate and push the built image to Docker Hub, completing the CI loop without exposing any sensitive information.

Now, let's understand every template involved in this case.

📦 Step 1: Cloning the Repository

- name: clone-repo
  template: clone
  arguments:
    parameters:
    - name: owner
      value: cloud-hacks
    - name: repo
      value: argocd-io
    - name: ref
      value: main
    - name: clone_path
      value: /work

We're using the official alpine/git image to shallow-clone a GitHub repository. It stores the code in a shared volume (/work) so later steps like build and scan can use it.

🛠️ Step 2: Building and Pushing the Docker Image

- name: build-image
  template: build-image
  arguments:
    parameters:
    - name: image
      value: example/nginx
    - name: path
      value: .
    - name: version
      value: v4
    - name: registry
      value: docker.io
    - name: docker_secret_name
      value: docker-config
    - name: insecure
      value: "false"
  depends: clone-repo

We're using the rootless BuildKit container (moby/buildkit) to build the Docker image. It reads the source from /work, builds the image, and is set up to push the image (when push=true is enabled in --output). We also provide:

The Docker image name and version
Docker registry credentials via a Kubernetes secret
Secure or insecure registry flag
Note: In this template, push=false is set — if you want to push, change it to push=true.

🐛 Step 3: Scanning the Image with Trivy

- name: scan-image
  template: scan-image
  arguments:
    parameters:
    - name: image
      value: example/nginx:v4
    - name: severity
      value: CRITICAL,HIGH
    - name: exit-code
      value: "0"
  depends: build-image

Here, we use Trivy to scan the Docker image (example/nginx:v4) for known vulnerabilities. This helps catch issues before the image is deployed.

We scan for CRITICAL and HIGH severity vulnerabilities
exit-code: 0 ensures the workflow doesn't fail even if vulnerabilities are found (customize this as needed)
Trivy pulls the image from Docker Hub, so make sure the build-image step pushes the image first.

🛡 Step 4: Scanning Kubernetes Manifests with Kubescape

- name: scan-k8s
  template: scan-k8s
  arguments:
    parameters:
    - name: path
      value: /work/dev
    - name: verbose
      value: "true"
  depends: clone-repo && build-image

We’re using kubescape to scan the Kubernetes YAML files located in /work/dev for misconfigurations, policy violations, and security issues. It helps ensure that the manifests follow best practices.

⚙️ Step 5: Deploy Kubernetes manifests using Kubectl

- name: deploy-kubernetes
  inputs:
    parameters:
      - name: path
      - name: namespace
  container:
    image: bitnami/kubectl:latest
    command: [sh, -c]
    args:
      - |
        echo "Deploying Kubernetes resources from {{inputs.parameters.path}}..."
        kubectl apply -f {{inputs.parameters.path}} -n {{inputs.parameters.namespace}}
    volumeMounts:
      - mountPath: /work
        name: work

We use official bitnami/kubectl image. It expects a path (like /work/dev) where my Kubernetes manifests are located.
The container mounts the shared work volume that was used in the clone step, ensuring that the files are accessible.
Once executed, it runs kubectl apply on the provided path to deploy the resources.

🔒 Note

Make sure that your Argo Workflow controller has the correct RBAC permissions to interact with Kubernetes resources (like pods, deployments, services, etc.) in the namespace where you intend to deploy.

Here's what we achieved so far using Argo Workflows:

Stage             Tool          Purpose
Clone Repo    alpine/git    Fetch source code
Build Image   BuildKit  Build Docker image in a
                                secure way
Scan Image    Trivy         Identify vulnerabilities in           
                         Docker image
Scan Manifests  Kubescape   Catch Kubernetes YAML issues
Deploy Kubernetes  Kubectl      Deploy kubernetes resources

🕒 Automating CI with CronWorkflow
To make the CI process completely hands-off, I added a CronWorkflow that runs every Tuesday at 9 AM UTC. This means our CI pipeline automatically triggers once a week without needing any manual input.

This is particularly useful for:

Automatically building and scanning your base images weekly.
Ensuring your Kubernetes manifests stay compliant.
Catching vulnerabilities on a routine basis, even if there are no recent code changes.

Here's what the CronWorkflow spec looks like:

spec:
  schedule: "0 9 * * 2"  # Every Tuesday at 9 AM UTC
  timezone: "UTC"
  concurrencyPolicy: "Replace"  # If the previous run is still running, replace it
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  workflowSpec:
    workflowTemplateRef:
      name: ci-build-workflow

With this in place, the entire CI process—from cloning the repo, building and scanning the image, to pushing it—is performed weekly without requiring any developer to trigger the pipeline.

As with all emerging tools out there, everything has to start somewhere and this is also the case with the Argo dashboard. It is minimal but does what it needs to do. Argo shows all the workflows and their steps, it updates automatically and all progress and logs can be viewed from here. This makes it very easy to monitor how everything is going.

Find the complete code and configuration for this setup on GitHub:
GitHub Repository Link CI-build-Workflow: https://github.com/Cloud-Hacks/argo-wf/blob/main/quick-start/wf-ci-workflow.yaml

GitHub Repository Link CronJob Example: https://github.com/Cloud-Hacks/argo-wf/blob/main/quick-start/wf-cronjob-ci.yaml

References:
https://argo-workflows.readthedocs.io/en/latest/walk-through/dag/

🌟 Let’s Connect!
I love sharing insights about DevOps, Kubernetes, and GitOps tools like ArgoCD. If you found this article helpful or have questions, let’s continue the conversation on LinkedIn!
👉 Connect with me on LinkedIn

How to Configure Pods to Enable IAM Roles for Service Accounts

Afzal Ansari — Mon, 13 Jan 2025 18:28:22 +0000

In this blog, we will dive into configuring Kubernetes Pods to use IAM Roles for Service Accounts (IRSA), enabling applications running in your Pods to securely access AWS services without embedding AWS credentials. This approach is secure, scalable, and aligns well with modern DevOps best practices.

Prerequisites
Before getting started, ensure you have the following ready:

An existing Amazon EKS cluster: If you don’t have one, follow the guide in Get started with Amazon EKS.

IAM OpenID Connect (OIDC) provider configured: Learn to create one or verify its existence by following the Create an IAM OIDC provider for your cluster guide.

AWS CLI installed: Ensure version 2.12.3 or later or version 1.27.160 or later is installed and configured. Check your version with:

aws --version | cut -d / -f2 | cut -d ' ' -f1
Update it if needed, following the Installing AWS CLI guide.

kubectl installed: Ensure it matches your Kubernetes version (within ±1 minor version). Follow the Set up kubectl and eksctl guide if necessary.

Step-by-Step Guide to Enable IAM Roles for Service Accounts

Create an IAM Policy

First, let's create an IAM policy that defines the permissions required by your application. For example, if your application needs read-only access to an S3 bucket, create a policy file s3-readonly-policy.json with the following content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

Then create the policy using the AWS CLI:
aws iam create-policy --policy-name S3ReadOnlyPolicy --policy-document file://s3-readonly-policy.json

Create an IAM Role for the Service Account

Using the IAM OIDC provider, let's create an IAM role that trusts the OIDC provider associated with your cluster.

First, retrieve the OIDC provider URL:
aws eks describe-cluster --name my-cluster1 --query "cluster.identity.oidc.issuer" --output text
Then, create a trust policy trust-policy.json for your service account:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::your-account-id:oidc-provider/oidc-provider-url"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc-provider-url:sub": "system:serviceaccount:namespace:service-account-name"
        }
      }
    }
  ]
}

Replace your-account-id, oidc-provider-url, namespace, and service-account-name with the appropriate values.

Let's create the IAM role as follows:
aws iam create-role --role-name S3ReadOnlyRole --assume-role-policy-document file://trust-policy.json
Attach the policy to the role:
aws iam attach-role-policy --role-name S3ReadOnlyRole --policy-arn arn:aws:iam::your-account-id:policy/S3ReadOnlyPolicy

Create a Kubernetes Service Account Let's create a Kubernetes service account and annotate it with the IAM role ARN:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: s3-access-sa
  namespace: default
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::your-account-id:role/S3ReadOnlyRole

Configure Your Pod to Use the Service Account

Let's update your Pod's YAML configuration to use the newly created service account:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
  namespace: default
spec:
  serviceAccountName: s3-access-sa
  containers:
  - name: app
    image: public.ecr.aws/nginx/nginx:1.12.0

Let's deploy the Pod now,
kubectl apply -f pod.yaml

Verify the Configuration

Once the Pod is running, we will need to check the ARN of the IAM role that the Pod is using.

kubectl describe pod my-app | grep AWS_ROLE_ARN:

We will find the output as follows.

AWS_ROLE_ARN: arn:aws:iam::111122223333:role/my-role

Conclusion

Configuring IAM roles for service accounts is a powerful way to grant AWS permissions to Kubernetes workloads securely. This approach eliminates the need for hardcoding credentials and simplifies permissions management. By following these steps, you’ve enabled a secure and scalable method to integrate AWS services with your Kubernetes applications.

Feel free to reach out in the comments below if you encounter any issues or have questions! 🚀

Implementing Authorization with OpenFGA | Part 2

Afzal Ansari — Thu, 05 Dec 2024 18:09:07 +0000

If you haven't followed up with my last post, i.e., Part 1, you can definitely grab the necessary concepts for this following post.

In this blog, I will focus on various aspects, from what you can experiment with to a practical workaround example.

We will understand the workflow of authorization model using a use case and its implementation on solving the issue of permissions access.

Modeling SaaS Project Permissions with OpenFGA

To understand our Fine-Grained Authorization (FGA) model, we will take a SaaS project use case into account. We will model a project organization permission model using OpenFGA. Our goal is to build a service that enables users to develop and collaborate on features efficiently.

We will implement a subset of the feature permission model using the OpenFGA Go SDK and validate the model through a few access control scenarios.

Requirements Recap

Users can be admins or members of services.
Each role inherits the permissions of the lower level (i.e., admins inherit member access).
Teams and organizations can have members.
Organizations can own services.
Organization admins have admin access to all services under that organization.

We will configure OpenFGA locally where we will use docker to run the tool and step further on my toes

Setting Up OpenFGA

There are multiple ways to set up OpenFGA, but we will leverage the Docker setup, as it is quite easy and traceable.

Running OpenFGA Locally

If you want to run OpenFGA locally as a Docker container, follow these steps:

Install Docker (if not already installed).

Pull the latest OpenFGA Docker image:

docker pull openfga/openfga

Run OpenFGA as a container:

docker run -p 8080:8080 -p 8081:8081 -p 3000:3000 openfga/openfga run

This will start:

An HTTP server on port 8080.

A gRPC server on port 8081.

The Playground on port 3000.

Running OpenFGA with Postgres

To run OpenFGA and Postgres in containers, follow these steps:

Create a Docker network to simplify communication between containers:

docker network create openfga

Start a Postgres container in the created network:

docker run -d --name postgres --network=openfga -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=password postgres:14

This will start Postgres in the openfga network.

Run the database migration to set up necessary tables (Very important, don't miss this step):

docker run --rm --network=openfga openfga/openfga migrate \ --datastore-engine postgres \ --datastore-uri "postgres://postgres:password@postgres:5432/postgres?sslmode=disable"

Start OpenFGA and connect it to Postgres:

docker run --name openfga --network=openfga -p 3000:3000 -p 8080:8080 -p 8081:8081 openfga/openfga run \ --datastore-engine postgres \ --datastore-uri 'postgres://postgres:password@postgres:5432/postgres?sslmode=disable'

This setup ensures:

The Postgres database is running.

The OpenFGA migration process is completed.

The OpenFGA server is running and connected to Postgres.

OpenFGA Playground

The Playground facilitates rapid development by allowing you to visualize and model your application's authorization models and manage relationship tuples with a locally running OpenFGA instance.

It is enabled on port 3000 by default and accessible at http://localhost:3000/playground.

Let's understand OpenFGA Model for SaaS Project and visualise it using OpenFGA playground

model
  schema 1.1

type user

type team
  relations
    define member: [user]

type organization
  relations
    define admin: [user, team#member] # Org admins
    define member: [user, team#member] # Org members
    define owner: [user] # Org owners

type service
  relations
    define admin: [user, organization#admin] # Admins include org admins
    define member: [user, team] # Members inherit from team
    define owner: [organization] # Services belong to an org

How This Model Works

User Roles
organization#admin → service#admin (Org admins inherit service admin access)
admin → member (Admins inherit all member permissions)
Team-Based Access
A team can include multiple users.
Teams can be granted organization#admin or organization#member permissions.
Service Ownership
Services are owned by an organization (service#owner).
Organization admins automatically get admin access to all services.

Example Use Cases

1️⃣ User is an Admin of an Organization
If a user is an admin of organization:org-1, they automatically have admin access to all services under that organization.

{
"user": "alice",
"relation": "admin",
"object": "organization:org-1"
}
✔ Alice has admin access to all services in org-1.

2️⃣ A Team Manages Multiple Services
If a team is assigned as an organization#admin, all its members automatically get admin access to services.

{
"user": "team-devops",
"relation": "admin",
"object": "organization:org-1"
}
✔ All members of team-devops inherit service admin access.

3️⃣ A User is a Member of a Specific Service
If a user is a service#member, they only have access to that specific service.

{
"user": "bob",
"relation": "member",
"object": "service:feature-x"
}
✔ Bob can collaborate on feature-x but does not have admin access.

Authorization Model with OpenFGA (Using Go SDK)

Let’s implement this using the OpenFGA Go SDK.

Install the SDK

go get github.com/openfga/go-sdk

Initialise the client and create the store After running OpenFGA locally, we will setup the envs as follows

Make sure to avoid creating store id multiple times once we get the id after one run

export FGA_API_URL=http://localhost:8080
export FGA_STORE_ID=01JR3M255RHWEE4KXGHE71H3F3 // example id

func main() {
    // Initialize OpenFGA client
    fgaClient, err := NewSdkClient(&ClientConfiguration{
        ApiUrl:  os.Getenv("FGA_API_URL"),  // e.g., "https://api.fga.example"
        StoreId: os.Getenv("FGA_STORE_ID"), // Required after store creation
    })

    if err != nil {
        log.Fatalf("Failed to create OpenFGA client: %v", err)
    }

    // Create OpenFGA store
    resp, err := fgaClient.CreateStore(context.Background()).
        Body(ClientCreateStoreRequest{Name: "Akuity Org"}).
        Execute()

    if err != nil {
        log.Fatalf("Failed to create OpenFGA store: %v", err)
    }

    fmt.Println("Store created:", resp.GetId())
}

Define and Write the Authorization Model This model defines roles and permissions for organizations, services, teams, and users.

func configureAuthorizationModel(fgaClient *OpenFgaClient) {
    var writeAuthorizationModelRequestString = `{
    "schema_version": "1.1",
    "type_definitions": [
      {
        "type": "user"
      },
      {
        "type": "team",
        "relations": {
          "member": {
            "this": {}
          }
        },
        "metadata": {
          "relations": {
            "member": {
              "directly_related_user_types": [
                { "type": "user" }
              ]
            }
          }
        }
      },
      {
        "type": "organization",
        "relations": {
          "admin": {
            "union": {
              "child": [
                { "this": {} },
                { "computedUserset": { "relation": "member", "userset": "team#member" } }
              ]
            }
          },
          "member": {
            "this": {}
          },
          "owner": {
            "this": {}
          }
        },
        "metadata": {
          "relations": {
            "admin": {
              "directly_related_user_types": [
                { "type": "user" },
                { "type": "team", "relation": "member" }
              ]
            },
            "member": {
              "directly_related_user_types": [
                { "type": "user" }
              ]
            },
            "owner": {
              "directly_related_user_types": [
                { "type": "user" }
              ]
            }
          }
        }
      },
      {
        "type": "service",
        "relations": {
          "owner": {
            "this": {}
          },
          "admin": {
            "union": {
              "child": [
                { "this": {} },
                {
                  "tupleToUserset": {
                    "tupleset": { "relation": "owner" },
                    "computedUserset": { "relation": "admin" }
                  }
                }
              ]
            }
          },
          "member": {
            "union": {
              "child": [
                { "this": {} },
                {
                  "computedUserset": {
                    "relation": "admin",
                    "userset": "service#admin"
                  }
                }
              ]
            }
          }
        },
        "metadata": {
          "relations": {
            "owner": {
              "directly_related_user_types": [
                { "type": "organization" }
              ]
            },
            "admin": {
              "directly_related_user_types": [
                { "type": "user" }
              ]
            },
            "member": {
              "directly_related_user_types": [
                { "type": "user" }
              ]
            }
          }
        }
      }
    ]
  }`

    var body ClientWriteAuthorizationModelRequest
    if err := json.Unmarshal([]byte(writeAuthorizationModelRequestString), &body); err != nil {
        log.Fatalf("Failed to parse model JSON: %v", err)
    }

    // Write the model to OpenFGA
    response, err := fgaClient.WriteAuthorizationModel(context.Background()).Body(body).Execute()
    if err != nil {
        log.Fatalf("Failed to write authorization model: %v", err)
    }

    fmt.Println("Authorization model configured, Model ID:", response.GetAuthorizationModelId())
}

Assign roles(Set Relationships) This function assigns users to specific roles in organizations or services.

func assignRole(fgaClient *OpenFgaClient, user, object, relation string) {
    _, err := fgaClient.Write(context.Background()).Body(ClientWriteRequest{
        Writes: []ClientTupleKey{
            {
                User:     "user:" + user,
                Relation: relation,
                Object:   object,
            },
        },
    }).Execute()
    if err != nil {
        log.Fatalf("Failed to assign role: %v", err)
    }

    fmt.Printf("Assigned %s as %s to %s\n", user, relation, object)
}

e.g.

assignRole(fgaClient, "james", "organization:akuity", "admin")
assignRole(fgaClient, "alice", "organization:akuity", "member")

Assigned james as admin to organization:akuity

Check permissions This function verifies if a user has a specific permission.

func checkPermission(fgaClient *OpenFgaClient, user, object, relation, modelID string) {
    options := ClientCheckOptions{
        AuthorizationModelId: PtrString(modelID),
    }

    body := ClientCheckRequest{
        User:     "user:" + user,
        Relation: relation,
        Object:   object,
    }

    data, err := (*fgaClient).Check(context.Background()).
        Body(body).
        Options(options).
        Execute()

    if err != nil {
        log.Fatalf("Failed to check permission: %v", err)
    }

    fmt.Printf("User %s has %s access to %s: %v\n", user, relation, object, data.Allowed)
}

e.g.

checkPermission(fgaClient, "alice", "member:org1", "admin", "01JR4FY69VK3G33EFTT6A1372E")

User alice has admin access to organization:org1: 0x1400021e117

Conclusion

In this second part of the series, we took a hands-on approach to implementing fine-grained authorization using OpenFGA in the context of a SaaS project. From setting up OpenFGA with Docker and Postgres to defining an extensible authorization model with the Go SDK, we've covered the foundational steps to get your access control logic up and running.

By modeling roles like admin and member, introducing hierarchical permission inheritance, and incorporating organizational ownership, we've laid the groundwork for a scalable and maintainable permissions system.

This model not only supports flexibility but also aligns with real-world requirements of multi-tenant SaaS platforms, writing relationship tuples, validating permissions via API calls,
and showing real-world scenarios to test access logic.

Until then, feel free to experiment with the Playground, tweak the model, and explore how OpenFGA can be tailored to your specific authorization needs. 🔐

Let me know if you'd like me to add this directly to the doc or tweak the tone!

Enhancing Application Security: Implementing Authorization with OpenFGA

Afzal Ansari — Wed, 04 Dec 2024 18:55:00 +0000

Introduction to OpenFGA

OpenFGA, an open-source authorization system under the Cloud Native Computing Foundation (CNCF), empowers developers to implement robust authorization mechanisms for any application. Inspired by Google’s Zanzibar, OpenFGA adopts a Relationship-Based Access Control (ReBAC) model, enabling developers to seamlessly integrate Role-Based Access Control (RBAC) and extend into Attribute-Based Access Control (ABAC). This system not only supports evolving complexity but also ensures scalability, making it ideal for modern applications.

Why Choose OpenFGA for Authorization?

Authorization is a cornerstone of application security. It ensures that users can only access resources and perform actions that they are permitted to, enhancing both security and compliance. Implementing authorization using OpenFGA provides:

Centralized and Externalized Authorization
- Decouple authorization logic from application code.
- Simplify policy management, changes, and audits.
Standardization and Velocity
- Use a unified authorization system to accelerate development.
Simplified Compliance
- Centralized decision-making and audit logs help with security and compliance requirements.
Ease of Evolution
- Dynamically adapt authorization policies to evolving application needs.

Key Features of OpenFGA

OpenFGA’s robust feature set empowers developers to implement fine-grained authorization effectively:

Multi-Environment Support: Manage authorization across production, testing, and development environments.
ABAC Scenarios Support: Utilize Contextual Tuples and Conditional Relationship Tuples for dynamic access control.
Extensive SDK Support: Available in Java, .NET, JavaScript, Go, and Python.
Flexible APIs: Includes both HTTP and gRPC APIs.
Versatile Deployment Options: Supports Postgres, MySQL, SQLite, and in-memory datastores.
Developer Tooling: Includes CLI tools, GitHub Actions, VS Code Extensions, and Helm Charts for Kubernetes deployments.
Monitoring Support: Integrate seamlessly with OpenTelemetry.

Understanding Authorization Concepts with OpenFGA

Authentication vs. Authorization

Authentication: Verifies identity.
Authorization: Determines access permissions.

Fine-Grained Authorization (FGA)

Allows specific actions on defined resources, scaling with millions of users and objects.
Example: Google Drive's granular sharing features.

Access Control Models

RBAC: Role-based permissions (e.g., Editors can edit content).
ABAC: Attribute-driven permissions (e.g., Marketing Managers can publish marketing content).
PBAC: Centralized policy management for access control.
ReBAC: Relationship-driven permissions, supporting complex hierarchies like document owners or folder relationships.

With OpenFGA, developers can create scalable, flexible, and compliant authorization systems while focusing on application logic. Its features make it the go-to choice for modern applications requiring fine-grained, dynamic access controls.

Delving into ReBAC with OpenFGA

Let's focus on ReBAC which is quite necessary for the OpenFGA implementation

Relationship-Based Access Control
(ReBAC) goes beyond traditional access control models by basing user access decisions on relationships between users, objects, and other entities. This model provides powerful flexibility, enabling applications to define fine-grained, dynamic access policies that naturally reflect real-world relationships.

How ReBAC Works
In ReBAC, access rules are defined based on relationships such as:

A user's relationship with an object (e.g., a user is an owner of a document).
An object's relationship with other objects (e.g., a document belongs to a specific folder).
Conditions that combine user and object relationships (e.g., a user’s manager can view team documents). With OpenFGA, relationships are stored as object-relation-user tuples, which act as the foundation for making authorization decisions. Applications can call the OpenFGA check endpoint to determine if a user has a specific relationship with an object.

ReBAC in Action: A Real-World Example

Let’s explore a document management system where access to documents is governed by ReBAC:

Scenario:
- Users can only view documents if they have access to the parent folder.
- Some users (like managers) can view all documents within their department.
Data Representation in OpenFGA: OpenFGA models these relationships as tuples:
- folder:finance#viewer@alice → Alice has viewer access to the "finance" folder.
- document:budget#parent@folder:finance → The "budget" document belongs to the "finance" folder.
Check Request Example: To determine if Alice can view the "budget" document:

   json{
     "tuple_key": {
       "object": "document:budget",
       "relation": "viewer",
       "user": "alice"
     }
   }

OpenFGA's Response:
The service verifies the relationships:

Alice has viewer access to folder:finance.
The budget document's parent is folder:finance. Result: true (Alice can view the "budget" document).

ReBAC with OpenFGA empowers developers to implement flexible, scalable, and secure access control tailored to the needs of modern applications. Start exploring OpenFGA to implement relationship-driven policies in your system today!

Let's explore configuration language required for your application

OpenFGA’s Configuration Language defines the relationships and authorization rules in a system. This configuration acts as the blueprint for determining access rights within your application, enabling the OpenFGA API to enforce those rules dynamically. Let's delve deeper into its structure and expand the example provided.

How the Configuration Language Works

Schema Definition:
- Specifies the schema version (e.g., 1.1) to ensure compatibility.
Type Definitions:
- Defines the object types (e.g., user, folder, document) and their possible relationships.
Relation Definitions:
- Describes relationships (e.g., viewer, writer) and their inheritance or conditions for validity.

Let's explore a scenario i.e. Managing Access in a Simple Document System

In this example:

Users can view or edit documents.
Documents can belong to folders, and access can be inherited from the parent folder.
A user can also be the owner of a document or folder, granting full control.

Model Definition

model
  schema 1.1

type user

type folder
  relations
    define owner: [user]
    define viewer: [user] or viewer from parent_folder
    define editor: [user] or owner or editor from parent_folder
    define parent_folder: [folder]

type document
  relations
    define owner: [user]
    define viewer: [user] or viewer from parent_folder
    define editor: [user] or owner or editor from parent_folder
    define parent_folder: [folder]

Explanation of the Example
Key Elements

Users:
- Represents individuals in the system.
Folders and Documents:
- Folders can contain other folders or documents.
- Access rules for documents can be inherited from the parent folder. Relations:

owner:
- Direct ownership grants all permissions for a folder or document.
viewer:
- A user can view a folder or document if they are:
- Explicitly added as a viewer.
- A viewer of the parent folder (inheritance).
editor:
- A user can edit a folder or document if they are:
- Explicitly added as an editor.
- The owner.
- An editor of the parent folder (inheritance).

How OpenFGA Handles These Scenarios

When checking if Bob can view document:budget: OpenFGA evaluates:
- Is Bob explicitly a viewer of document:budget?
- Is Bob a viewer of folder:project (the parent folder)?
When checking if Alice can edit document:proposal: OpenFGA evaluates:
- Is Alice the owner or editor of document:proposal?
- Does Alice inherit edit permissions from the parent folder?

Conclusion:

Authorization is a crucial aspect of application security, ensuring that users can only access resources they are entitled to. OpenFGA, inspired by Google's Zanzibar, provides a powerful, scalable solution to implement fine-grained, relationship-based authorization. By decoupling authorization logic from application code, OpenFGA offers a flexible and centralized approach to managing access control in a way that adapts to evolving requirements.

Key Takeaways:

Simplicity: OpenFGA's Configuration Language allows developers to easily define authorization models using intuitive relationships.
Scalability: OpenFGA’s ability to handle millions of objects and permissions makes it ideal for modern applications.
Flexibility: Support for Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), and Relationship-Based Access Control (ReBAC) ensures diverse authorization needs can be met.
Efficiency: Inherited and conditional relationships streamline access control across hierarchical structures.
Integration: With SDKs, APIs, and CLI tools, OpenFGA fits seamlessly into existing development workflows.

Harness Authentication Capabilities

Afzal Ansari — Mon, 21 Oct 2024 11:24:32 +0000

An Overview of Harness Platform

Harness is an Enterprise DevOps platform that provides a comprehensive set of tools and services to help organizations deliver software faster and more securely. Harness Capabilities authentication process is designed to ensure that only authorized users have access to the platform and its resources.

Harness Authentication Overview

Harness authentication provides a secure way for users to access and interact with Harness capabilities. It offers various authentication mechanisms to use for a wide range of customer needs. This guide will walk you through Harness' authentication capabilities, explain the options available, and help you decide which method is best for your organization

Supported Authentication Mechanisms

Login via a Harness Account or Public OAuth Providers: Harness allows users to authenticate using either a Harness account (email and password) or a range of single sign-on with public OAuth 2.0 providers like Google, GitHub, and GitLab. This method is ideal for organizations that either want a Harness-managed user account system or OAuth providers for authentication

e.g. if an user (registered with Harness) John logs into a Harness account using a OAuth 2.0 provider i.e. GitHub, then the user must also be registered with GitHub using JohnOAuth20@outlook.com.

SAML Provider: Harness supports Single Sign-On (SSO) via Security Assertion Markup Language (SAML), which allows organizations to integrate with an identity provider (IdP) such as Okta, Azure AD, or Google Workspace. This option is best for organizations that already have a centralized IdP and want to provide a seamless, secure login experience across their applications.

e.g. if an user (registered with Harness) John logs into a Harness account using a SAML provider i.e. Okta, then the user must also be registered with an Okta account using the same email address.

LDAP Provider: Harness also supports Lightweight Directory Access Protocol (LDAP), which allows organizations to integrate with internal directory services like Microsoft Active Directory (AD) or OpenLDAP. This method is ideal for organizations with an established LDAP infrastructure that want to manage users centrally through their directory service.

How to Set Up Authentication in Harness:

Setting Up Login via a Harness Account or OAuth Providers

Navigate to Account Settings → Authentication.
Select either Enable Harness Account Authentication or choose an OAuth Provider (e.g., Google, GitHub, GitLab).
Enforce password policies: Ensure that users create and use strong passwords that meet complexity requirements, and periodically expire passwords
Enforce lockout after failed logins: Implement a lockout mechanism that automatically blocks access after a specified number of consecutive failed login attempts.
Enforce two factor authentication: Implement two-factor authentication (2FA) for all Harness users to add an extra layer of security.
Follow the provider’s instructions to configure OAuth credentials, such as Client ID and Client Secret.
Once configured, users can log in via Harness-managed credentials or their OAuth provider’s login flow.

Setting Up SAML Authentication

Navigate to Account Settings → Authentication.
Select Enable SAML.
Provide the required SAML metadata from your IdP (such as Okta or Azure AD), including the SAML Metadata in XML format.
Configure the required SAML attributes (e.g., Entity Id).
Save the configuration and test logging in via your IdP.
Note: To do this, you should first disable any configured public OAuth providers. And to use SAML SSO, Harness Users must use the same email addresses to register in Harness and the SAML provider as mentioned in the example above.

Set up vanity URL

You can access app.harness.io using your own unique subdomain URL in the following format; https://\{company}.harness.io where {company}is the name of your account.
To set up this, you have to contact Harness Support.

Restrict email domains

You can allow (whitelist) only certain domains as usable in login credentials, e.g. gmail.com

Set inactive and absolute session timeout

You can set a session timeout (in minutes) for auto logout if there has been no activity. Similarly, you have an option to set an absolute Session Timeout (in minutes) for any user’s logout, regardless of any activity.

Conclusion:

Harness offers a range of authentication options to meet the needs of organizations of all sizes, from small teams leveraging public OAuth providers to large enterprises managing users with SAML or LDAP providers. Choosing the right method depends on your existing infrastructure, security requirements, and user management needs. By following the setup guides, you can ensure that your organization’s authentication is secure, scalable, and easy to manage, that’s where Harness stands out and makes a difference.

Reference:

https://developer.harness.io/docs/platform/authentication/authentication-overview

Optimising Performance: A Deep Dive into Caching Strategies with AWS Services

Afzal Ansari — Sun, 28 Jan 2024 11:14:36 +0000

Introduction

In the ever-evolving landscape of cloud computing, optimizing performance is a top priority for businesses leveraging AWS. One crucial aspect of achieving this optimization is the implementation of efficient caching strategies. In this blog post, we will explore the benefits, use cases, and scenarios surrounding popular AWS caching services such as DynamoDB Accelerator (DAX), Amazon ElastiCache, etc. Additionally, we will delve into how these caching strategies can be integrated with AWS CloudFront and API Gateway for enhanced performance.

Benefits of Caching Strategies:

Efficient caching is the key to reducing latency and improving the overall responsiveness of applications. By strategically implementing caching strategies, businesses can achieve the following benefits:
Improved Response Time: Caching allows frequently requested data to be stored closer to the application, significantly reducing the time it takes to retrieve information.
Cost Optimization: Caching minimizes the need for repeated requests to backend services, resulting in lower compute and data transfer costs.
Scalability: Caching services enable applications to scale more effectively by offloading the backend infrastructure and distributing the load across multiple nodes.

Let's explore some common use cases for caching strategies

DynamoDB Accelerator (DAX) with CloudFront:

Use Case:
DynamoDB is a highly scalable and managed NoSQL database service, but frequent read operations can impact response times. By integrating DynamoDB Accelerator (DAX) with CloudFront, you can create a powerful combination for enhanced read performance.

Diagram depicting the integration of DAX with DynamoDB for accelerated read performance.

Scenario:

Read Acceleration: DAX sits between your application and DynamoDB, caching frequently accessed items. CloudFront then distributes this cached data globally through its content delivery network.
Reduced Latency: As CloudFront caches and serves data from edge locations, users experience significantly reduced latency when accessing frequently requested DynamoDB items.
Cost Savings: By minimising the load on DynamoDB, CloudFront helps reduce read capacity units, leading to cost savings.

ElastiCache with CloudFront:

Use Case:
Amazon ElastiCache is a fully managed, in-memory caching service. When combined with CloudFront, it creates a robust solution for both static and dynamic content delivery.

Illustration showcasing ElastiCache as a caching layer for frequently executed database queries.

Scenario:

Static Content Caching: CloudFront can be configured to cache and distribute static content globally, reducing latency for end-users.
Dynamic Content Acceleration: ElastiCache, integrated with CloudFront, serves as a caching layer for frequently executed database queries, reducing the load on backend databases.
Global Distribution: CloudFront ensures that cached content is available at edge locations worldwide, providing a faster and more responsive experience for users.

Diagram illustrating the integration of CloudFront with ElastiCache for dynamic content caching.

Summary:
In conclusion, adopting effective caching strategies with AWS services is pivotal for achieving optimal performance in cloud-based applications. Whether leveraging DAX, ElastiCache, businesses can significantly enhance response times, reduce costs, and improve scalability. Integrating these caching strategies with AWS CloudFront and API Gateway further amplifies their impact, ensuring a seamless and efficient user experience. As you embark on your journey to optimise performance on AWS, understanding and implementing these caching strategies will undoubtedly be a game-changer for your applications.

Unlocking the Power of AWS WAF: Safeguarding Your Cloudfront and Load Balancer Services

Afzal Ansari — Sat, 20 Jan 2024 10:04:05 +0000

Protect your Web App from vulnerabilities by Unleashing AWS WAF for Cloudfront and Load Balancer Services

Introduction:

In the ever-evolving landscape of cloud computing, ensuring robust security measures is paramount. Amazon Web Services (AWS) offers a comprehensive solution with its Web Application Firewall (WAF), particularly when integrated seamlessly with Cloudfront and Load Balancer services. This blog explores the benefits, use cases, and scenarios of leveraging AWS WAF for enhanced security in three key setups.

Let’s speak about the benefits of AWS WAF:
Before diving into specific scenarios, let's highlight the overarching benefits of AWS WAF. This web application firewall provides a layer of protection against common web exploits, such as SQL injection and cross-site scripting (XSS). By seamlessly integrating with AWS services like Cloudfront and Load Balancer, AWS WAF ensures a centralised and effective approach to safeguarding your applications. Some other key benefits include:

Granular Control: Fine-tune security rules to suit specific application needs.
Threat Intelligence Integration: Leverage AWS Threat Intelligence feeds for proactive security.
Automated Protections: Automatically block common threats and respond to emerging attack patterns.
Scalability: Scale security measures seamlessly with growing application demands.

Let’s discuss a few uses cases:
Use Case 1: EC2 Instance with Network Access Control List (NACL):
In the first scenario, we examine the traditional setup of an EC2 instance protected by either a Network Access Control List (NACL). NACL offers basic network-level security but not protecting from a client vulnerable IP address where attack surface is higher. The diagram below illustrates this configuration:

In lucid architecture diagram as shown below:

Use Case 2: EC2 Instance Followed by Application Load Balancer (ALB) with NACL or WAF:
Moving to a more scalable architecture, the second scenario involves an EC2 instance behind an Application Load Balancer (ALB), further fortified by NACL or AWS WAF. This setup not only distributes traffic across multiple instances but also provides enhanced security at both the network and application layers. The following diagram visualizes this configuration:

In lucid architecture way:

Use Case 3: EC2 Instance, ALB, and Cloudfront with WAF:
For the most robust security and performance, the third scenario combines an EC2 instance, ALB, and Cloudfront, with AWS WAF ensuring protection at every level. Cloudfront, a content delivery network (CDN), accelerates content delivery while AWS WAF safeguards against web exploits filtering out IP addresses. The diagram below showcases this comprehensive setup:

In lucid architecture way:

Summary:
In conclusion, leveraging AWS WAF in conjunction with Cloudfront and Load Balancer services provides a powerful and flexible approach to securing your web applications. Whether opting for a basic EC2 instance with NACL or a sophisticated setup involving ALB and Cloudfront, AWS WAF ensures a robust defense against a variety of cyber threats. As you architect and refine your AWS infrastructure, consider these scenarios to enhance both the performance and security of your applications in the cloud.

My Experience as an LFX Mentee for Jaeger Project

Afzal Ansari — Mon, 04 Sep 2023 06:52:48 +0000

Introduction

I am thrilled to share my experience as an LFX Mentee for Jaeger, an open-source project that focuses on observability within the cloud native ecosystem. My passion for open source projects and emerging technologies led me to explore the world of DevOps, where monitoring and observability are paramount.
I started my impeccable journey by applying for the CNCF LFX Mentorship program. I highlighted my enthusiasm for DevOps and explained how it intertwined with monitoring and observability, aligning perfectly with Jaeger's objectives.

Being selected as a mentee for Jaeger was a moment of great excitement. It affirmed my commitment to diving deep into the observability ecosystem and experiencing the day-to-day tasks of managing infrastructure in a cloud-native environment.
My mentor and the Jaeger community provided an exceptional onboarding experience. I gained insights into the project's history, architecture, and its role in the CNCF landscape. Understanding the fundamentals was crucial for my contributions.
Throughout my mentorship, I actively contributed to Jaeger where my main task is to refactor code migrating open tracing pkg to OpenTelemetry pkg. These experiences allowed me to sharpen my coding skills while also understanding the complexities of a real-world, production-ready open-source project. These also gave me a deeper understanding of the Observability part. I learned how it handles microservices and how observability tools like OpenTelemetry can be integrated seamlessly. This knowledge was invaluable for my DevOps aspirations. I understood the importance of tracing and monitoring for troubleshooting and optimizing performance.

This mentorship significantly boosted my confidence as a developer and my understanding of open-source contributions. I honed my skills in coding, debugging, and collaboration, which are all crucial for a career in DevOps. I now have a clearer path towards my career goals and a network of professionals to learn from and collaborate with in the future.

Application Process

I discovered the Jaeger project at CNCF LFX Mentorship page. I thought this was a great opportunity to work on an open source project, which I had been dreaming about. I also had the right technology stack, so I submitted my resume right before the deadline where I highlighted my relevant experience, skills, and contributions to the tech community. In my cover letter, I went the extra mile by proposing concrete tasks to be undertaken during the mentorship period. This demonstrated my commitment and readiness to contribute meaningfully to the project. It also showcased my understanding of the project's needs and how my skills could address them.I think this is the crucial thing to give your proposal more chances of getting selected.

After submitting my application, there was a period of anticipation. Waiting for the results can be a nervous but exciting time, as you hope for the opportunity to work on a project you're passionate about. I was lucky enough to receive the good news that I got selected for the mentorship. This was a moment of great joy and validation of my efforts.

Project Process

The project I applied for was Upgrade internal use of tracing to OpenTelemetry under Jaeger category project, which aimed to upgrade the Jaeger backend to use the OpenTelemetry tracing API and SDK directly.

During the first two weeks of the project, I got familiar with the business process and some details of Jaeger code base. In the next two weeks, I started to contribute to the sub-part of the project introducing a new pkg which included passing a wrapper that contained a wrapper object. This wrapper object streamlined the migration from open tracing to OpenTelemetry by providing a convenient interface.

This phase of work required a deep understanding of both technologies and the project's architecture. During this time, I studied one of many blogs written by Yuri about open tracing and met with my next mentor Albert, to understand the clear understanding of the blog and some of the code logic.

The tasks related to hotROD application migration were strange to me in the first place. Therefore, I had to ask my mentors for technical support from time to time, and they were always very responsive. I was greatly impressed by my mentor’s extensive knowledge and experience in this field. Under the guidance from my mentors, I was finally able to put together all sets of milestones for my task. Later, in order to track my work, I created a tracking checklist on another doc as suggested.
However, during the subsequent coding work, I encountered various problems. Most challenging part was to enable OpenTelemetry SDK and APIs everywhere I saw use of opentracing pkg which I later was able to resolve through submitting small small PRs following the milestones created for my project gradually.

Conclusion

My project process involved a systematic journey of learning, contributing, and problem-solving. Hence, I was able to demonstrate adaptability by seeking help when needed and effectively collaborating with mentors and the community.
The experience of encountering challenges and overcoming them will undoubtedly contribute to my growth as a developer and open-source contributor. My commitment to documenting my work and reflecting on my experiences showcases a proactive approach to continuous improvement. This project not only contributed to the Jaeger project but also enriched my own skill set and understanding of tracing technologies.
If you’re interested in contributing to the LFX Mentorship projects, start your journey now by getting exposed to the community with small contributions before you apply for mentorship.

Thanks for reading by.

How I debugged through AWS ec2 instance creation when SSH failed to work

Afzal Ansari — Wed, 12 Jul 2023 11:05:03 +0000

Introduction:

Creating and connecting to an Amazon Web Services (AWS) EC2 instance is a fundamental skill for anyone working with cloud infrastructure. However, even experienced users can encounter challenges, such as failed SSH connections, when attempting to connect to newly created instances. In this blog post, we will explore the steps I took to debug an SSH failure while connecting to an EC2 instance, specifically when I created an instance without a key pair. By following these troubleshooting techniques, you can quickly identify and resolve similar issues.

When I started to create an ec2 instance, I came to this step where I needed to select key-pair(login) option. I chose without key-pair to create further as shown below.

By the time I got to start the instance and connect it through SSH after finishing all the steps, I grabbed the command and ran it in the terminal.

Next, I tried to setup as per the instruction given in the SSH connection without key-pair. I failed to do so.

After retrying several times, I realised that I am missing somewhere. Then I got to know I can't connect to the instance without key-pair.

I chose other option to connect the instance created.

Finally, I was able to connect through EC2 instance connect which opens you up a new tab to use the web based console.

If we want to connect ec2 instance using SSH and ignore the failures like me, we can follow the steps as such below

Step 1: Confirm SSH Configuration
The first step in troubleshooting an SSH connection issue is to ensure that the SSH configuration on the EC2 instance is correctly set up.

Step 2: Create and Associate a Key Pair
If you created the EC2 instance without a key pair, you won't be able to establish an SSH connection. In this case, you will need to create a new key pair and associate it with the instance:

We can also follow as such given following ways:
Generate a New Key Pair: In the EC2 Dashboard, navigate to the "Key Pairs" section and create a new key pair. Save the private key (.pem file) securely on your local machine.

Associate Key Pair with the Instance: Right-click on the instance in the EC2 Dashboard and select "Actions" > "Instance Settings" > "Attach/Replace IAM Role." Choose the newly created key pair, and AWS will associate it with the instance.

Conclusion:

Debugging SSH connection issues when creating an AWS EC2 instance can be a frustrating experience, but by following a systematic troubleshooting approach, you can identify and resolve the underlying problems. In this blog post, we explored the steps to confirm SSH configuration, create and associate a key pair, and establish an SSH connection. Remember to double-check security group rules, ensure the instance has a public IP, and verify the permissions and path of the private key. By mastering these techniques, you can effectively troubleshoot SSH connection failures and confidently manage your EC2 instances in AWS.

Developing a Web App with Gin framework, integrating it with psql DB and my GO HACK Hackathon Experience

Afzal Ansari — Thu, 26 May 2022 07:58:36 +0000

Introduction

Before I deep dive into the architecture of what I rebuilt an app during the hackathon, I would like to tell about my experiences in my words.
When I first came across #GO Hack Challenge on devpost, I was overwhelming to find related tracks which I could be fit for and tutorials to enrich my knowledge here and there to get into it. It was really a learning opportunity to someone who is newbie and wants to contribute or build any stuff under the hood.
I stepped into the open source projects under CERN Foundation, where I didn’t even know what the ecosystem around the project is. I was much curious then. Next, I delved into one of the projects for a few days and looked into a few issues and made a small contribution to it by fixing one of the bugs. I moved to another track where we needed to build any app based on golang. I came to know about use of generic function which is currently released and used in Go >= 18 Version. I managed to rebuild an app using Gin-Gonic framework and use the generic function usage in my app.

Monolithic services architecture enables developers to develop product simpler and deploy easier, which can further help them achieve the product's needs sharply. This has also better development productivity ultimately. With that, I am going to give you a short demonstration on how to build a web app using monolithic way.

Well, this looks scary when you never heard about this architecture. But when you start building things on top of that, you can gradually understand the scope of this architecture i.e. when and how to use it.

We take the resource management app as an example to build a monolithic service here. The app service is consists of multiple modules, the modules include add resource, get resource and login modules, etc. Each module will have its own independent business logic, and each module will also depend on some others. For example, the get resource (/resource/get-resc) module will depend on the login module. In the monolithic application this kind of dependency is usually accomplished by method calls between modules. Monolithic services generally share storage resources, such as MySQL but in our case, it is PSQL.

The overall architecture of monolithic services is relatively simple, which is also the advantage of monolithic services.

Development

This section describes how to quickly implement a resource management app monolithic service based on gin framework of golang.
With this framework, we can quickly build, configure and deploy resources within a few commands. We can create and store our API’s and configuration in a centralized repository as many as possible.

Getting Started

To get started, I would assume you have GOlang already installed and configured locally and the corresponding Go package of verion >=18 for this app. For more details, please follow the official docs. Once we set up the requirements, we will proceed with the next steps to play around GO gin pkg for apis and gorm pkg for database instance creations.

So, let’s install plugins and libraries step by step;

The below commands will clone basic templates for you to work with the API after you clone the repo. Plz, ignore the docker file for now.

After that install the dependencies as follows; this allows you to update the pkg installed.

go mod download

go mod tidy

Let’s dive into main.go file and create another file inside our routes dir.

User and Resource model API definition

type Resource struct {
    ID        int    `json:"id"`
    Title     string `json:"title"`
    Category  string `json:"category"`
    Status    string `json:"status"`
    Types     string `json:"types"`
    Content   string `json:"content"`
    FileLink  string `json:"file_link"`
    CreatedBy int    `json:"created_by"`
    CreatedAt int64  `json:"created_at"`
    UpdatedBy int    `json:"updated_by"`
    UpdatedAt int64  `json:"updated_at"`

type User struct {
    ID        int    `json:"id"`
    FirstName string `json:"first_name"`
    LastName  string `json:"last_name"`
    Password  string `json:"password"`
    Email     string `json:"email"`
    Role      string `json:"role"`
}

Resource module API definition

Add Resource details -> resource/add-resource
View Resource details -> resource/get-resc
Login User -> resource/login

Another way to handle http request through http.HandleFunc to support generic function here as shown below

To more about generic function, you can definitely check this link out.

Let's deep dive into gorm pkg and psql pkg a little bit now.

Since we are focusing here PostgreSql, we look into that sql query managed by that as shown below;
In order to execute the program, you need to have psql installed in your system:

Using Linux env and running on its terminal:

sudo apt install postgresql -> installs the psql

sudo -u postgres psql -> sudo into user name as postgress

psql --version -> checks version

export RESC_DB_DSN='postgres://USER_NAME:YOUR_PWD@localhost/DB_NAME' psql $RESC_DB_DSN

psql --host=localhost --dbname=DB_NAME --username=USER_NAME

I hope we are done with the psql creation and its usage through gorm pkg.

Let's run the App using go run main.go, then using POSTMAN API Client or curl command you can go ahead request the body.
curl localhost:8080/resource/get-resource

Using Postman

What’s next!

I am looking forward to adding more API endpoints to make use of HTTP requests persistently and integrating with kubernetes cluster

Summary

The above walk-through shows that it is very simple to use go-gin gonic framework to develop monolithic services. You only need to define the api file, and then the goctl tool can automatically generate the project code. We only need to fill in the business logic code in the logic package. In this article, I just demonstrated how to quickly develop monolithic services based on go-gin, gorm pkgs and generic function. Hope, you find it insightful and adopt the usage in building your app.