DEV Community: Vipin Vijaykumar

Bye-Bye Credentials! Automate BTP & Cloud Foundry Setup with Terraform using Github Actions and Github OIDC

Vipin Vijaykumar — Wed, 13 Aug 2025 15:38:36 +0000

Imagine this: You want to automate setting up SAP BTP Accounts and Cloud Foundry using Terraform, but without the pain of managing and tracking usernames, passwords, or rotating any of those credentials.

Well, that's no longer a dream! Most common CI/CD Solutions like Github/Gitlab actions, Jenkins, Azure DevOps etc and OIDC Provider integration, you can wave goodbye to maintaining long-lived credentials. Say hello to short-lived, secure, just-in-time tokens.

In this blog, we’ll set up GitHub’s OIDC provider to authenticate directly against SAP BTP using JWT Bearer Assertion flow, and then use Terraform to provision your BTP resources, Cloud Foundry spaces, and more. These steps work for both Terraform and OpenTofu.

What's a JWT Bearer Assertion Anyway?

Before we jump into Terraform and GitHub Actions, let’s talk JWT Bearer tokens. A JWT Bearer Assertion is a way for a client to prove its identity to an authorisation server using a signed JSON Web Token (JWT) instead of a password or traditional credentials. Since the tokens are short-lived and ephemeral, they bring somewhat better security and rotation.

In short: Instead of passing around long-term credentials, GitHub can request a short-lived token for your pipeline, signed and trusted via OpenID Connect. This token is accepted by an SAP Cloud Identity Service tenant, which then lets Terraform (on behalf of GitHub Actions) access BTP and Cloud Foundry APIs.

Event Sequence

Github Workflow is triggered
The Github OIDC Provider is called, and a JWT Token is generated for the workflow. This token is stored in environment variables or github variables
The JWT Bearer token is used by the BTP & CF terraform provider in the Terraform scripts
The calls for the respective providers are authenticated in BTP and CF (UAA) via the trusted SAP Cloud Identity Service tenant. The bearer token is forwarded to the SAP Cloud Identity Service
SAP Cloud Identity services validate the token against the Github OIDC Public Keys(JWKS)
If the key is successfully validated, the clients are authenticated, and subsequent calls can be made.

Let’s dig in!

This tutorial builds upon a community blog that showcases how the cf CLI can leverage GitHub OIDC for logging into and managing Cloud Foundry.

We'll reference the community blog for the initial setup steps as it's well-documented there.

Step 1: Establish Trust between SAP BTP & SAP Cloud Identity Services

Please refer to the official documentation for doing the same

Step 2: Create the Github Repo

Create the github repo with repository-name in the Organisation you need. This repository and name will be used in the next steps.

Step 3:Configure SAP Cloud Identity Services

In this step, configure the SAP Cloud Identity Services to

Add the Repo User for Github Action Workflows. Note:
- Name format: repo/<Organisation>/<repository-name>
- Email format: repo/<Organisation>/<repository-name>@<github-domain>
Add the Repo User to a group
Configure Github OIDC as a Corporate IdP
Map the repository name as NameId (Enriched Token Claims)
Configure Identity Federation.
Set up the BTP Application to use the GitHub IDP

Please refer to the community blog mentioned above for the detailed steps with regards to this.

Step 4: Add user to Global Account

Add the repo-user email, created in SAP Cloud Identity Services in BTP.

Since we will be working with creating a subaccount and managing resources at the Global Account level, assign the Global Account Administrator Role Collection.

Step 5: Add content to the Github Repository

The following will be the structure of the project and files.

<repository-root-directory>/
├── .github
│   └── workflows
│       └── deploy.yml
├── main.tf
├── provider.tf
├── terraform.tfvars
└── variables.tf

Here we create a Terraform configuration that uses the Terraform providers for SAP BTP & Cloud Foundry to:-
Add the configuration to initialise the provider.tf file

`main.tf`

The main.tf has the definition of the infrastructure that will be created. For this article, we take a simple scenario as:-

Create a Subaccount
Initialise a Cloud Foundry Runtime Environment in the subaccount
Create a space within Cloud Foundry

This would, of course vary depending on your setup and scenario. So we won't delve into this section much.

For details on the different resources & datasources present in the Terraform providers for SAP BTP & Cloud Foundry, please refer to the registry.

resource "btp_subaccount" "project" {
  name      = "sample project"
  subdomain = "sample.project"
  region    = lower(var.region)
}

resource "btp_subaccount_environment_instance" "cloudfoundry" {
  subaccount_id    = btp_subaccount.project.id
  name             = "sample-cf-instance"
  landscape_label  = "cf-ap10"
  environment_type = "cloudfoundry"
  service_name     = "cloudfoundry"
  plan_name        = "standard"
  parameters = jsonencode({
    instance_name = "sample-cf-instance"
  })
}

resource "cloudfoundry_space" "space" {
  name = "dev_space"
  org  = btp_subaccount_environment_instance.cloudfoundry.platform_id
}

`variables.tf` file

Here we declare the variables globalaccount and idp. You'd have more depending on your script.

variable "globalaccount" {
  type        = string
  description = "The subdomain of the SAP BTP global account."
}

variable "idp" {
  type        = string
  description = "Orgin key of Identity Provider"
  default     = null
}

Configure `provider.tf`

By convention, the provider.tf file usually represents the configuration to initialise the providers. Here we will configure the SAP BTP and Cloud Foundry providers.

terraform {
  required_providers {
    btp = {
      source  = "sap/btp"
      version = "~> 1.15.0"
    }
    cloudfoundry = {
      source  = "cloudfoundry/cloudfoundry"
      version = "~> 1.8.0"
    }
  }
}

provider "btp" {
  globalaccount = var.globalaccount
  idp           = var.idp
}
provider "cloudfoundry" {
  api_url = "https://api.cf.${var.region}.hana.ondemand.com"
  origin  = var.idp
}

As can be seen:-

btp provider is configured with the global account and the idp (Based on the trust configured in the above steps)
cloudfoundry is configured with the cf-api and the origin(uaa), which is the same trust configuration as created above
Since the credentials used here is the assertion token, we'll be using the environment variables BTP_ASSERTION and CF_ASSERTION_TOKEN.

Note:- CF API depends on the region where the subaccount is created. Please refer to the right API in the official document

Step 6: Add the Github Workflow

The Github workflow initialises Terraform, generates the token and stores it as Github Outputs. These tokens are then assigned as environment variables to the Terraform Providers. Thereafter, the Terraform executes the scripts just as any typical workflow.

name: Deploy
on:
  workflow_dispatch:
    inputs:
      region:
        description: 'BTP Subaccount Region'
        required: true
        type: string
      idp:
        description: 'Identity Provider'
        required: true
        type: string
      global_account_id:
        description: 'Global Account ID'
        required: true
        type: string
      tenantIssuerUri:
        description: 'Issuer URL'
        required: true
        type: string
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
        id-token: write # This is required for requesting the JWT
        contents: read
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          terraform_wrapper: false
          terraform_version: latest

      - name: Install OIDC Client from Core Package
        run: npm install @actions/core @actions/http-client

      - name: Get Id Token
        uses: actions/github-script@v7
        id: idtoken
        env:
          TENANT_URL: ${{inputs.tenantIssuerUri}}
        with:
          script: |
            const tenanturl = process.env.TENANT_URL;
            const coredemo = require('@actions/core')
            let id_token = await coredemo.getIDToken(tenanturl)
            coredemo.setOutput('githubJwt', id_token)

      - name: Apply Terraform configuration
        run: |
          export BTP_ASSERTION="${{ steps.idtoken.outputs.githubJwt }}"
          export CF_ASSERTION_TOKEN="${{ steps.idtoken.outputs.githubJwt }}"
          terraform apply -auto-approve -var "region=${{ inputs.region }}" -var "idp=${{ inputs.idp }}" -var "globalaccount=${{ inputs.global_account_id }}"

For this article, the workflow is triggered manually based on user inputs. It takes the following inputs:-

BTP Subaccount Region: Region where the Subaccount will be created
Identity Provider: The name of the identity provider (Origin Key (Cloud Foundry)) that was configured in the Trust Configuration in BTP from the above step.
Global Account ID: The Global Account Subdomain
Issuer URL: The URL of the SAP Cloud Identity Service Tenant. Github OIDC Provider issues tokens for this tenant as the audience.

The token is generated in the step Get Id Token. In this article, we use the core package for the GH Actions Toolkit. An alternative approach of using curl. This approach can be seen in the blog referred above.

Step 7: Trigger Workflow

Trigger the workflow from the Github Actions tab. Add the necessary details as seen below.

The Job should run successfully, whereby you can see that the Token is generated and used by the subsequent steps.

Conclusion

By federating GitHub Actions with SAP Cloud Identity and leveraging the JWT Bearer Assertion flow, you can automate provisioning of SAP BTP resources and Cloud Foundry environments without storing or rotating long-lived service credentials.

Some Best Practices

Use any supported remote, encrypted Terraform/OpenTofu backends like HCP, Vault, OpenBao, S3/GCS/Azure Blob, etc, with server-side encryption.
Work with least privilege & role collections: In BTP, assign the repo user (or group) only the role collections needed for the automation (For e.g Avoid Global Account Admin if the setup is for a specific subaccount).
Ensure GitHub Repo is protected and require reviewers for workflows that can create or alter infrastructure.
Test your workflow setup on dev/test environments first.

Setting up OTEL Collectors for mTLS

Vipin Vijaykumar — Mon, 17 Feb 2025 03:27:47 +0000

What's mTLS

mTLS (Mutual Transport Layer Security) is an enhanced version of TLS (Transport Layer Security) where the client and the server authenticate each other during the connection establishment.

A very Simple Flow for mTLS. (Note arrow direction)

+--------+ ---1--> +--------+
| Client | <--2--- | Server |
+--------+ <--3--> +--------+

1a) Client sends request to server along with its certificate.
1b) Server verifies the client certificate using client's public key.(Server has the public Key in its store/DB)
2a) Server sends response along with its certificate.
2b) Client verifies the server certificate with the server public key
3) After mutual authentication, encrypted communication between client and server.

Typically, the server and client certificates are signed by a CA. As the server can manage multiple clients, the server usually maintains the public key of the CA that is used to sign all clients.

How is this different from regular TLS?

Regular TLS is unidirectional. I.E Only the client verifies the certificates presented by the server. The server does not verify the client's certificates.

Using mTLS in OpenTelemetry(OTEL)

To use mTLS between 2 OTEL components, we need client and server certificates.

Scenario

2 OTEL Collectors. One as client (exporter) and the other as server (receiver). Metrics are sent/forwarded from the client to the server. Since this is a demo setup (or internal) we will be using Self Signed certificates.

+-------------+                       +--------------+
| Otel Client |-------GRPC----------->|  OTEL Server |
| (Exporter)  |                       |  (Receiver)  |
+-------------+                       +--------------+

We'll be running this inside a Kubernetes cluster in 2 different namespaces.

Prerequisites

A Kubernetes cluster
kubectl & helm
System with OpenSSL client installed

Demo

1) Generate certificates

a)Generate rootCA certificate.
Since we are using Self-signed certificates, we will create and use our own rootCA.

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:4096 -subj '/O=A Team/CN=root.com' -keyout rootCA.key -out rootCA.pem

b)Generate server certificate
We need to first create a certificate signing request (csr) for the server certificate: server.csr and the key server.key

openssl req -out server.csr -newkey rsa:4096 -nodes -keyout server.key -subj "/CN=test.com/O=PM" -config server.cnf -extensions req_ext

You will notice a server.cnf config file. This file consists of extensions for adding Subject Alternative Names (SAN), which are alternative host names that can identify the server.

[req]
default_bits = 4096
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
CN = 'test.com'
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1   = 'test.com'
DNS.2   = 'test.server.svc.cluster.local'
DNS.3   = 'localhost'
DNS.4   = 'test'
# add more for other names

We will then use the server.csr to create a signed server certificate: server.pem. The signing is done by the rootCA certificate that was generated in step a.

openssl x509 -req -sha256 -days 365 -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -in server.csr -out server.pem -extensions req_ext -extfile server.cnf

Notice: We use the rootCA certificate and private key to sign the certificate.

b)Generate client certificate
Similarly, we create client certificates & key: client.pem| client.key. For the ease of this demo , we use the same rootCA to sign the client certificates.

As before we use a client.cnf also maintained for SAN:

[req]
default_bits = 4096
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
CN = 'localhost'
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1   = 'localhost'

Creating the csr and signing it with the rootCA:

openssl req -out client.csr -newkey rsa:4096 -nodes -keyout client.key -subj "/CN=localhost/O=client" -extensions req_ext -config client.cnf

openssl x509 -req -sha256 -days 365 -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -in client.csr -out client.pem -extensions req_ext -extfile client.cnf

1) Deploy on Kubernetes

Prerequisite: Please install the opentelemetry-operator in the cluster.

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts

helm install otel-operator open-telemetry/opentelemetry-operator --namespace otel-collector-operator --create-namespace --set "manager.collectorImage.repository=otel/opentelemetry-collector-k8s" --set admissionWebhooks.certManager.enabled=false --set admissionWebhooks.autoGenerateCert.enabled=true

a)Create namespaces
Create the 2 namespaces

kubectl create ns server
kubectl create ns client

b)Create Kubernetes secrets.
The certificates that were generated in step 1 will be used to create the secrets. These secrets will be mounted into the OTEL pods in the next step.

k create secret generic server -n server  --from-file=tls.pem=server.pem --from-file=tls.key=server.key --from-file=ca.pem=rootCA.pem # server certificates

k create secret generic client -n client  --from-file=tls.pem=client.pem --from-file=tls.key=client.key --from-file=ca.pem=rootCA.pem # client certificates

c)Deploy the OTEL Receiver (server)
The OTEL receiver is the server that accepts the metrics. For our demo purpose the OTEL server will receive the metrics (Port 4317) and print it in the logs. The server is deployed in server namespace and the secrets are mounted as volume to be used by the otel-collector receiver.

We also create a service named test with port 4317. While this is optional for the demo, its a good practice. Additionally certain meshes..For E.g istio relies on the port name for the protocol. (grpc is the name of the port in the service here)

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: test
  namespace: server
  labels:
    app: otel-collector
spec:
  mode: deployment
  volumeMounts:
  - name: server
    mountPath: /etc/pki/ca-trust/source/server-ca
    readOnly: true
  volumes:
  - name: server
    secret:
      secretName: server
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            #endpoint: test.server.svc.cluster.local:4317
            tls:
              cert_file: /etc/pki/ca-trust/source/server-ca/tls.pem
              key_file: /etc/pki/ca-trust/source/server-ca/tls.key
              client_ca_file: /etc/pki/ca-trust/source/server-ca/ca.pem
    exporters:
      # NOTE: Prior to v0.86.0 use `logging` instead of `debug`.
      debug:
        verbosity: detailed
    service:
      pipelines:
        metrics:
          receivers: [otlp]
          processors: []
          exporters: [debug]
---
apiVersion: v1
kind: Service
metadata:
  name: test
  namespace: test
spec:
  selector:
    app.kubernetes.io/component: opentelemetry-collector
    app.kubernetes.io/instance: test.test
    app.kubernetes.io/name: test-collector
  type: ClusterIP
  ports:
  - name: grpc # important for istio!
    protocol: TCP
    port: 4317

Verify the otel-collector instance and service is created

kubectl get pods,svc -n server

b)Deploy the OTEL Exporter (client)
The OTEL Exporter is the client that generates (i.e source) the metrics. We use the hostmetrics receiver plugin. This will vary according to the needs.
The other receiver is deployed in the client namespace. The client certificates are mounted as volumeMounts just as before.

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: client
  namespace: client
  labels:
    app: client
spec:
  mode: deployment
  volumeMounts:
  - name: client
    mountPath: /etc/pki/ca-trust/source/client-ca
    readOnly: true
  volumes:
  - name: client
    secret:
      secretName: client
  config: |
    receivers:
      # Data sources: metrics
      hostmetrics:
        scrapers:
          cpu:
          disk:
    processors:
      # Data sources: traces
      attributes:
        actions:
          - key: environment
            value: external-test-3
            action: insert
      # Data sources: traces, metrics, logs
    exporters:
      otlp:
        endpoint: test.server.svc.cluster.local:4317
        tls:
          ca_file: /etc/pki/ca-trust/source/client-ca/ca.pem
          cert_file: /etc/pki/ca-trust/source/client-ca/tls.pem
          key_file: /etc/pki/ca-trust/source/client-ca/tls.key
      debug: #just to print metrics in logs of the client
    service:
      pipelines:
        metrics:
          receivers: [hostmetrics]
          processors: [attributes]
          exporters: [otlp,debug]

Verify the otelcollector instance is created

kubectl get pods -n client

c) Verify the setup
Check the logs of the server otel (receiver). You should see logs of metrics generated by the client otel (exporter).

kubectl logs -f <otel-receiver-pod> -n server

Accessing Privileged Containers in Cloud Foundry: A Guide

Vipin Vijaykumar — Thu, 19 Dec 2024 09:28:25 +0000

Usually, your productive apps don’t and shouldn't have SSH access in Cloud Foundry. However, you might need to SSH into an application for debugging purposes. Fortunately, this is achievable by enabling the SSH permission for both the space and the app and be enabled by the operator, after which you can use the cf ssh command. This approach suffices for most scenarios.

But sometimes, this level of access isn’t enough. You might require privileged access to the application container for advanced troubleshooting or debugging. Why might you need this? That’s a question only you can answer, but this blog will show you how it’s done.

Note: This process can only be executed if you are an operator of the Cloud Foundry deployment and can access the Diego cells.

Steps to Gain Privileged Access

Retrieve the app-guid for the application

cf app <app-name> --guid

bosh -d cf vms
bosh -d cf ssh <diego-cell>  #(any Diego cell)

Use the cfdot utility to locate application details

In Diego terms, applications are referred to as “lrps” (long-running processes). Using the app-guid, identify the lrp instance_id and the diego_cell where the application or process is running.

cfdot actual-lrps | grep -E '<app-guid>' | jq -r '. | select(.index == 0)' # Index 0 is for the 0th instance of the application.

This command will provide the instance_guid and cell_id of the application container.

SSH into the respective Diego Cell From the cell_id identified in the previous step, SSH into the corresponding Diego Cell.

bosh -d cf ssh <diego_cell>

Attach to the Garden container You can attach it to the container using one of two methods:

Approach 1: Using runc

sudo /var/vcap/packages/runc/bin/runc --root /run/containerd/runc/garden/ exec <instance_guid> --tty /bin/sh

Approach 2: Using ctr
This approach is not recommended as the ctr command line is not actively maintained and the commands may vary over time.

sudo /var/vcap/packages/containerd/bin/ctr -a /var/vcap/sys/run/containerd/containerd.sock -n garden tasks exec --exec-id my-shell --tty <instance_guid> /bin/sh

After running either of these commands, you will have privileged access to the application container.

! A Word of Caution !

With great power comes great responsibility.
Privileged access should be used sparingly and responsibly if possible NEVER. Always assess the necessity of these steps, and avoid this process unless required.
Things can break here if you are NOT careful!

I've messed up something what do I do

Fear not. Mostly deleting the app instance can reset and set things back to usual.

How to get into an app container manually in Cloud Foundry with Garden-Containerd Backend

Vipin Vijaykumar — Mon, 18 Nov 2019 09:54:26 +0000

Summary

A method to get into a container on the Diego Cell

Purpose

The requirement is to be able to access the application container in Cloud Foundry in scenarios where cf ssh would not be possible. Main reasons being troubleshooting inside the container. This write-up describes the procedures for the same. The original article for containers based on garden-runc is provided below.

Procedure

Obtain the app process-guid

cf curl /v2/apps/$(cf app <app-name> --guid)| jq -r '.metadata.guid + "-" + .entity.version'

Obtain the hosts in which the app is running. Choose one of the I

cf curl v2/apps/$(cf app <app-name> --guid)/stats | grep host

Login to the bosh director.
Identify the cell/VM Instance name associated to the IP
SSH into the VM

bosh -e <environment> -d <deployment> ssh <Instance-Name>
diego-cell/8cbe944d-d0cc-4158-802e-59426506faf7:~$ sudo -i
diego-cell/8cbe944d-d0cc-4158-802e-59426506faf7:~#

Obtain the "instance_guid" for the process-guid from step 1

cfdot actual-lrp-groups | grep d7c18647-9560-4327-8bcc-b7ab1ecdf295-2d6ebf81-7857-46d8-8bdc-f958795abf86

Attach into the container using ctr (containerd client)

export containerid=<Instance_Guid_>
/var/vcap/packages/containerd/bin/ctr -a /var/vcap/sys/run/containerd/containerd.sock -n garden tasks exec --exec-id my-shell --tty $containerd /bin/bash
root@9b5900f2-34f3-4317-63bb-863c:/#

You will now be in the container.