DEV Community: Goodluck Ekeoma Adiole

AKS DISASTER RECOVERY USING VELERO

Goodluck Ekeoma Adiole — Wed, 29 Oct 2025 08:47:54 +0000

This documentation teaches you Velero from the ground up and show you exactly how to install it and use it to back up an AKS cluster. I will assume you’re a beginner, so I created the documentation to explain what each step does and give you copy-paste commands. I’ll also point out important choices (CSI snapshots vs Restic) and recommend a simple path you can follow.

Notes before we start

These instructions show the common Azure/AKS flow: Velero stores backups in an Azure Storage account (blob container) and runs inside your AKS cluster. ([Velero][2])

1) Prerequisites (what you need first)

Make sure you have these on your laptop or a management machine:

kubectl configured to talk to the AKS cluster (you should be able to run kubectl get nodes).
Azure CLI (az) logged in as a user that can create resources and assign roles: az login.
helm (optional — we’ll use Velero CLI approach, not helm).
A shell (bash) and ability to create files locally.
The Velero CLI binary (we’ll install it below).
(Optional) jq helps with JSON parsing in examples.

If any of the above are missing, install them first. The Velero docs and AKS docs show these prerequisites. ([Velero][3])

2) High-level plan (what we’ll do)

Install Velero CLI locally.
Create an Azure Storage account + blob container to hold backups.
Create Azure credentials for Velero (service principal or storage key).
Install Velero server into AKS with the Azure plugin (and optionally Restic or CSI snapshot support).
Make a test backup of a namespace / sample app.
Restore from that backup.
(Optional) Set a schedule and check/monitor backups.

Key docs (used to craft commands): Velero install docs, Velero Azure config, Velero backup/restore docs, and AKS guidance. ([Velero][3])

3) Install the Velero CLI (local machine)

Pick the latest Velero release from the Velero releases page, then download & install. Replace vX.Y.Z with the release version (e.g. v1.16.0 — check releases page). Example for Linux x86_64:

# example: change v1.16.0 to the version you choose
VERSION="v1.16.0"
curl -LO https://github.com/vmware-tanzu/velero/releases/download/${VERSION}/velero-${VERSION}-linux-amd64.tar.gz
tar -xvf velero-${VERSION}-linux-amd64.tar.gz
sudo mv velero-${VERSION}-linux-amd64/velero /usr/local/bin/velero
velero version

On macOS you can use brew install velero. The official docs show the same options. ([Velero][3])

4) Create Azure storage (backup target)

Velero needs an object store. We’ll create a resource group, storage account, and a blob container.

Create a Storage Location for Backups

Velero needs a storage location — typically an Azure Blob Storage container.

Set environment variables:

export AZURE_RESOURCE_GROUP=<your-resource-group>
export AZURE_STORAGE_ACCOUNT_ID=<your-storage-account-name>
export BLOB_CONTAINER=<your-container-name>
export AZURE_SUBSCRIPTION_ID=<your-subscription-id>
export AZURE_TENANT_ID=<your-tenant-id>
export AZURE_CLIENT_ID=<your-app-client-id>
export AZURE_CLIENT_SECRET=<your-app-client-secret>

(These credentials come from your Azure AD app with permissions to the storage account.)

# variables - change values to your naming
SUBSCRIPTION_ID=$(az account show --query id -o tsv)
RESOURCE_GROUP="velero-backups-rg"
LOCATION="eastus"            # choose your Azure region
STORAGE_ACCOUNT="velerobackups$RANDOM"  # must be globally unique, lower case, numbers
CONTAINER="velero"

# create resource group
az group create -n $RESOURCE_GROUP -l $LOCATION

# create storage account
az storage account create \
  --name $STORAGE_ACCOUNT \
  --resource-group $RESOURCE_GROUP \
  --sku Standard_LRS \
  --encryption-services blob \
  --https-only true

# create blob container
az storage container create \
  --account-name $STORAGE_ACCOUNT \
  --name $CONTAINER

Velero requires the storage account and container to store backup tarballs and metadata. The Velero Azure plugin docs and Microsoft guidance explain this. ([GitHub][4])

5) Create Azure credentials for Velero (service principal recommended)

Velero needs permissions to write blobs and manage snapshots (if using CSI snapshots). We’ll create a Service Principal (SP) and give it the Storage Blob Data Contributor role on the storage account and (if you want snapshotting) appropriate snapshot permissions in the AKS infrastructure resource group (the MC_* resource group AKS created).

Create SP and give access to the storage account:

# create service principal (output will include appId, password, tenant)
AZURE_SP_NAME="velero-sp-$RANDOM"
az ad sp create-for-rbac --name $AZURE_SP_NAME --role "Contributor" --scopes "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP" -o json > velero-sp.json

Note: using the "Contributor" role on the resource group is broad — for production you should scope further and create a custom role that grants only the needed permissions (blob write/list/read, snapshots). Many guides assign Storage Blob Data Contributor on storage account and Contributor or a custom role on the AKS resource group for snapshotting. ([DEV Community][5])

Create a file in the format Velero expects (an example credentials-velero):

# Example using service principal credentials
# Replace values from velero-sp.json
AZ_CLIENT_ID=$(jq -r .appId velero-sp.json)
AZ_CLIENT_SECRET=$(jq -r .password velero-sp.json)
AZ_TENANT_ID=$(jq -r .tenant velero-sp.json)

cat > credentials-velero <<EOF
AZURE_SUBSCRIPTION_ID=${SUBSCRIPTION_ID}
AZURE_TENANT_ID=${AZ_TENANT_ID}
AZURE_CLIENT_ID=${AZ_CLIENT_ID}
AZURE_CLIENT_SECRET=${AZ_CLIENT_SECRET}
AZURE_RESOURCE_GROUP=${RESOURCE_GROUP}
AZURE_CLOUD_NAME=AzurePublicCloud
EOF

Velero can alternatively use a storage account key instead of SP; plugin docs show both options. ([GitHub][4])

6) Decide: CSI snapshots vs Restic

CSI Volume Snapshot: snapshot of the underlying cloud disk (fast restores). Requires the CSI snapshot driver and permissions; better for block volumes (e.g., Azure Disk). You must enable the EnableCSI feature flag when installing Velero. ([Velero][6])
Restic: Velero integrates restic to copy file-level data from pods to object storage. Simpler to enable (--use-restic) and works when snapshot support is not available, but it’s slower and requires restic to run as init/daemon pods. ([Velero][7])

For a beginner on AKS: if your AKS cluster has the Azure CSI drivers and supports snapshots, prefer CSI snapshots for PVs. If you’re unsure or want a simpler setup, enable Restic (it’s easier to get started). I’ll show the install command enabling Restic; I’ll also show how to enable CSI in case you want that.

7) Install Velero into AKS (server components)

Run this from your local machine (the velero CLI will push server components into the cluster). Replace placeholders:

# variables used in install
BUCKET="$CONTAINER"
BACKUP_RG="$RESOURCE_GROUP"
STORAGE_ACCOUNT="$STORAGE_ACCOUNT"
SECRET_FILE="./credentials-velero"

# Basic install with Azure plugin + restic
velero install \
  --provider azure \
  --plugins velero/velero-plugin-for-microsoft-azure:v1.11.1 \
  --bucket $BUCKET \
  --secret-file $SECRET_FILE \
  --backup-location-config resourceGroup=$BACKUP_RG,storageAccount=$STORAGE_ACCOUNT \
  --use-restic

Notes & options

--plugins points to the velero Azure plugin image. Use the plugin version that matches Velero best (check release notes). ([GitHub][4])
To enable CSI snapshotting instead of (or in addition to) restic, add --features=EnableCSI and appropriate --snapshot-location-config values for Azure (specify the resourceGroup where snapshots belong). Example:

# Example install with CSI support (instead of restic)
velero install \
  --provider azure \
  --plugins velero/velero-plugin-for-microsoft-azure:v1.11.1 \
  --bucket $BUCKET \
  --secret-file $SECRET_FILE \
  --backup-location-config resourceGroup=$BACKUP_RG,storageAccount=$STORAGE_ACCOUNT \
  --snapshot-location-config resourceGroup=MC_myAksRG_myAksCluster_westeurope \
  --features=EnableCSI

If you use both, Velero can snapshot volumes via CSI when possible and fall back to restic if needed.

The velero install guide and Azure config pages show these flags in detail. ([Velero][8])

8) Verify Velero server is running

kubectl get pods -n velero
kubectl get deployments -n velero

You should see velero pods (and restic pods if you used --use-restic). If pods are CrashLooping, check logs:

kubectl logs deploy/velero -n velero

If install succeeded, Velero is now watching your cluster and ready to back up resources.

9) Test: deploy a sample app (optional but recommended)

Deploy a small nginx deployment in namespace test-app:

kubectl create ns test-app
kubectl -n test-app apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - containerPort: 80
EOF

Wait until pod is Running:

kubectl -n test-app get pods -w

10) Create a backup with Velero

Simple namespace backup (this will back up Kubernetes resources; if you used --use-restic or CSI snapshotting, PVs will be saved accordingly):

# create a one-off backup of namespace test-app
velero backup create test-app-backup --include-namespaces test-app --wait

# check backup status
velero backup describe test-app-backup --details
velero backup logs test-app-backup

Important flags:

--snapshot-volumes can be added to ensure volumes are snapshotted (if CSI snapshotting is supported).
--wait makes the command wait until backup completes or fails.

Velero backup/restore docs explain scheduling, including/excluding resources, and common flags. ([Velero][9])

11) Simulate disaster & restore

Delete the namespace / app and restore:

# delete the namespace to simulate data loss
kubectl delete ns test-app

# restore from the backup
velero restore create --from-backup test-app-backup --wait

# watch the restore progress and logs
velero restore describe <restore-name> --details
velero restore logs <restore-name>

Replace <restore-name> with the name Velero gave to the restore (or use the name you supplied). After restore completes, check resources:

kubectl get all -n test-app

This demonstrates a full backup → delete → restore workflow. Velero restore docs explain API fields and options. ([Velero][10])

12) Useful Velero commands (cheat sheet)

velero backup create <name> --include-namespaces <ns> --wait — create backup.
velero backup get — list backups.
velero backup describe <name> — see details.
velero backup logs <name> — logs for that backup.
velero restore create --from-backup <name> --wait — restore backup.
velero restore logs <name> — restore logs.
velero schedule create <sched-name> --schedule "0 2 * * *" --include-namespaces <ns> — create scheduled backups.
kubectl -n velero get pods — see Velero pods. (Official docs contain many more options.) ([Velero][9])

13) Production considerations & tips

Credentials & least privilege: Create a minimal custom role for Velero rather than broad Contributor. Limit the scope to the storage account and the resource group that needs snapshot permissions. ([DEV Community][5])
Test restores regularly — a backup that hasn’t been restored is not proven.
Retention & lifecycle — configure retention policies and lifecycle on storage to avoid runaway costs.
Monitor & alerts — watch Velero backups and failures; integrate with monitoring/alerts.
Storage costs — backups in Azure Blob incur storage and egress costs; plan accordingly.
Snapshots vs restic — prefer CSI snapshots where possible (faster, cloud-native), but use restic when snapshots are unavailable or for file-level backups. ([Velero][6])

14) Troubleshooting quick hits

Velero pods CrashLoop? kubectl logs deploy/velero -n velero — common causes: bad credentials file, plugin image mismatch, or permission error.
Restic issues: ensure restic DaemonSet is present and pods run; check annotations to include/exclude volumes. ([Velero][7])
PVs not snapshotted: ensure CSI snapshot driver installed and EnableCSI feature is turned on in install. See CSI docs. ([Velero][6])

15) Short checklist you can copy & paste

az login
kubectl config current-context (should point to your AKS cluster)
Create storage account & container (step 4 commands)
Create SP & credentials file (step 5 commands)
velero install ... --provider azure --plugins velero/velero-plugin-for-microsoft-azure:... --bucket <name> --secret-file ./credentials-velero --backup-location-config resourceGroup=<rg>,storageAccount=<sa> --use-restic
kubectl get pods -n velero ✅
Deploy test app, velero backup create ... --wait
velero restore create --from-backup ... --wait ✅

Encrypt secrets stored in etcd with a Key Vault key

Goodluck Ekeoma Adiole — Mon, 01 Sep 2025 08:03:21 +0000

AKS uses the KMS (Key Management Service) plugin to encrypt secrets stored in etcd with a Key Vault key (your CMK). See Microsoft’s KMS docs. (Microsoft Learn)

KMS requires a user-assigned managed identity (system-assigned identities are not supported for KMS). You must grant that identity key permissions on the Key Vault before enabling KMS. (Microsoft Learn)

DO NOT delete the Key or Key Vault once KMS is turned on — doing so will make secrets unrecoverable. (Microsoft Learn)

Quick variable setup (copy/paste and edit)

# your cluster info (already provided)
RG="MiddleWare-Dev-RG"
CLUSTER="hil-middleware-staging-cluster"
LOCATION="<your-azure-location>"         # e.g. eastus2
# choose names
KV_NAME="<your-keyvault-name>"
KEY_NAME="<your-key-name>"
IDENTITY_NAME="<your-user-identity-name>"

Step 1 — Ensure Azure CLI is new enough

az --version   # need recent az (see docs), update if old

Step 2 — Create Key Vault + Key (public example)

(If you prefer a private Key Vault, I list the extra args afterwards.)

# Create key vault (public network access)
az keyvault create \
  --name $KV_NAME \
  --resource-group $RG \
  --location $LOCATION

# Create an RSA key in the vault
az keyvault key create \
  --vault-name $KV_NAME \
  --name $KEY_NAME \
  --kty RSA

Save the key id:

KEY_ID=$(az keyvault key show --vault-name $KV_NAME --name $KEY_NAME --query 'key.kid' -o tsv)
echo $KEY_ID   # you'll pass this to az aks update

(Private vault: az keyvault create --public-network-access Disabled and you’ll need API Server VNet Integration on AKS — see docs. (Microsoft Learn))

Step 3 — Create a user-assigned managed identity (required for KMS)

az identity create --resource-group $RG --name $IDENTITY_NAME --location $LOCATION

IDENTITY_PRINCIPAL_ID=$(az identity show --resource-group $RG --name $IDENTITY_NAME --query principalId -o tsv)
IDENTITY_RESOURCE_ID=$(az identity show --resource-group $RG --name $IDENTITY_NAME --query id -o tsv)

echo $IDENTITY_PRINCIPAL_ID
echo $IDENTITY_RESOURCE_ID

Step 4 — Grant the identity permissions on the Key Vault

Choose the correct method depending on whether your Key Vault uses Access Policies (classic) or RBAC.

If the vault uses access policies (default):

az keyvault set-policy \
  --name $KV_NAME \
  --object-id $IDENTITY_PRINCIPAL_ID \
  --key-permissions decrypt encrypt

If the vault uses RBAC (you created it with --enable-rbac-authorization true):

KEYVAULT_RESOURCE_ID=$(az keyvault show --name $KV_NAME --resource-group $RG --query id -o tsv)
az role assignment create \
  --assignee-object-id $IDENTITY_PRINCIPAL_ID \
  --assignee-principal-type ServicePrincipal \
  --role "Key Vault Crypto User" \
  --scope $KEYVAULT_RESOURCE_ID

(Documentation shows both approaches — pick the one matching your Key Vault config.) (Microsoft Learn)

Step 5 — Attach the user-assigned identity to the AKS cluster

KMS requires a user-assigned identity; attach it to the control plane:

az aks update \
  --resource-group $RG \
  --name $CLUSTER \
  --enable-managed-identity \
  --assign-identity $IDENTITY_RESOURCE_ID

You can confirm the identity was attached:

az aks show -g $RG -n $CLUSTER --query identity

(If your cluster already used system-assigned identity, switching to / adding user-assigned can cause the control plane to reconfigure; monitor the cluster resource state via az aks show.) (Microsoft Learn)

Step 6 — Enable KMS (turn on Azure KeyVault KMS for etcd)

For a public Key Vault:

az aks update \
  --resource-group $RG \
  --name $CLUSTER \
  --enable-azure-keyvault-kms \
  --azure-keyvault-kms-key-vault-network-access "Public" \
  --azure-keyvault-kms-key-id "$KEY_ID"

For a private Key Vault (requires API server VNet integration and vault resource id):

KEYVAULT_RESOURCE_ID=$(az keyvault show --name $KV_NAME --resource-group $RG --query id -o tsv)

az aks update \
  --resource-group $RG \
  --name $CLUSTER \
  --enable-azure-keyvault-kms \
  --azure-keyvault-kms-key-vault-network-access "Private" \
  --azure-keyvault-kms-key-id "$KEY_ID" \
  --azure-keyvault-kms-key-vault-resource-id "$KEYVAULT_RESOURCE_ID"

After this command completes, KMS will be enabled and AKS will use that Key Vault key for encrypting secrets in etcd. See the official KMS doc for the exact flags. (Microsoft Learn)

Step 7 — Re-encrypt existing secrets (important)

Secrets created before KMS was enabled remain encrypted with the old mechanism. Run this to update them (you can scope to namespaces if desired):

kubectl get secrets --all-namespaces -o json | kubectl replace -f -

If you have many secrets, consider doing namespace-by-namespace to reduce blast radius. The docs recommend re-replacing secrets after enabling KMS. (Microsoft Learn)

Step 8 — Verify status via CLI (recommended)

Check the AKS resource for the KMS block:

az aks show --resource-group $RG --name $CLUSTER --query "securityProfile.azureKeyVaultKms" -o json

Expected output when enabled (example):

{
  "keyId": "https://<vault>.vault.azure.net/keys/<key>/<version>",
  "networkAccess": "Public"
}

You can also inspect the whole cluster JSON:

az aks show -g $RG -n $CLUSTER -o json | jq '.securityProfile'

If the azureKeyVaultKms object appears, KMS is active. (This matches guidance in the Microsoft KMS doc.) (Microsoft Learn, GitHub)

Step 9 — Verify in Azure Portal (if you still want the Portal)

The Portal UI may not show a dedicated "Encryption" blade for older clusters. If a blade appears for newer clusters it will be under the cluster Settings → Security/Encryption area.
The reliable method is CLI/JSON as above, or open the cluster resource in the Portal, click “JSON View” (or “Export template”), and inspect properties.securityProfile.azureKeyVaultKms. If present, CMK/KMS is configured. (Microsoft Learn)

Safety & operational cautions (read these)

Do not delete the Key or Vault while KMS is enabled — you will break access to secrets and potentially make the cluster unusable. The docs warn strongly about this. (Microsoft Learn)
If you rotate the key (new version), update the cluster with az aks update --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $NEW_KEY_ID and re-replace secrets as noted. (Microsoft Learn)
If your Key Vault uses firewall rules / private endpoints, follow the API Server VNet Integration guidance in the docs before enabling KMS. (Microsoft Learn)

Helpful one-liners (copy/paste)

Get the current KMS block:

az aks show -g $RG -n $CLUSTER --query "securityProfile.azureKeyVaultKms" -o json

Re-encrypt secrets (all namespaces):

kubectl get secrets --all-namespaces -o json | kubectl replace -f -

If you want, I can:

produce the exact command sequence filled with specific names (I used your RG/cluster; tell me KV_NAME, KEY_NAME, IDENTITY_NAME, and LOCATION and I’ll print the full script), or
give the equivalent ARM template / Terraform snippet to bake this into infra-as-code.

Which one do you want me to produce next?

Azure DevOps Tutorial for Beginners

Goodluck Ekeoma Adiole — Fri, 29 Aug 2025 05:30:01 +0000

1. Introduction to Azure DevOps

Azure DevOps is a cloud-based DevOps platform by Microsoft that provides an end-to-end set of tools to plan, develop, test, deliver, and monitor applications. It supports collaboration between development and operations teams, enabling organizations to deliver software faster, more reliably, and with higher quality.

✅ By the end of this tutorial, beginners should understand the purpose and functionality of each Azure DevOps component, and how they work together to support modern software development and operations.

Why Azure DevOps?
- Supports Agile planning, CI/CD, source control, and testing in one place.
- Integrates with multiple programming languages and platforms (Java, Python, .NET, Node.js, etc.).
- Works well with cloud (Azure, AWS, GCP) and on-premises environments.
- Scalable for individuals, small teams, and enterprise organizations.
Main Components of Azure DevOps:

Azure Boards – Project planning and tracking.
Azure Repos – Source code management.
Azure Pipelines – CI/CD automation.
Azure Test Plans – Manual and exploratory testing.
Azure Artifacts – Package management and sharing.

2. Azure Boards

Azure Boards helps teams plan, track, and discuss work across the entire development cycle.

Key Features:
- Work Items – Units of work (tasks, bugs, features) that can be tracked.
- Boards (Kanban) – Visual boards to manage tasks in “To Do”, “Doing”, and “Done”.
- Backlogs – Organize work in product and sprint backlogs.
- Sprints – Agile iteration planning.
- Dashboards & Reports – Real-time progress tracking.
Why Use Azure Boards?
- Provides transparency and visibility into project progress.
- Supports Agile, Scrum, and Kanban methodologies.
- Integrates directly with Git commits and pipelines.
- Keeps development and project management in sync.

Example Workflow:

Product Owner creates a User Story in the backlog.
Developers break it into Tasks.
Work is tracked on the Kanban board.
Progress is visible in sprint reports and burndown charts.

3. Azure Repos

Azure Repos provides source code management for teams. It allows developers to store, version, and collaborate on code.

Two Types of Repos:

Git Repos – Distributed version control system (most popular).
TFVC (Team Foundation Version Control) – Centralized version control.

Key Features:
- Branching & Merging – Work on features in isolation and merge to main branch.
- Pull Requests (PRs) – Collaborate on code changes before merging.
- Code Reviews – Discuss and improve code quality.
- Policies – Require reviews, checks, or tests before merging.
- Integration – Linked directly with Boards (e.g., commit messages linked to work items).
Best Practices:
- Use feature branches for new work.
- Enforce PR reviews for quality control.
- Follow a branching strategy like GitFlow or Trunk-based development.

4. Azure Pipelines

Azure Pipelines provides Continuous Integration (CI) and Continuous Delivery (CD) automation.

Continuous Integration (CI):
- Automatically builds and tests code when changes are pushed.
- Ensures code always stays in a working state.
Continuous Delivery (CD):
- Deploys applications to different environments (Dev, QA, Production).
- Supports automated approvals and release gates.
Key Features:
- Pipeline as Code – Defined using YAML files.
- Multi-platform – Supports Windows, Linux, and macOS builds.
- Language Support – Works with .NET, Java, Python, Node.js, Go, etc.
- Integrations – Deploy to Azure, AWS, GCP, Kubernetes, or on-prem servers.
- Build Agents – Microsoft-hosted or self-hosted agents for executing pipelines.

Example CI/CD Flow:

Developer pushes code → triggers pipeline.
Pipeline builds code, runs unit tests, and creates artifacts.
Pipeline deploys application to Azure Web App (staging).
After approval, it promotes to production.

5. Azure Test Plans

Azure Test Plans is a testing solution integrated into Azure DevOps for manual, exploratory, and automated testing.

Key Features:
- Manual Testing – Step-by-step test execution with pass/fail tracking.
- Exploratory Testing – Ad-hoc testing without predefined scripts.
- Automated Testing Integration – Link with CI/CD pipelines.
- Bug Tracking – Report bugs directly linked to builds and code commits.
Benefits:
- Ensures quality before releases.
- Provides traceability between requirements, tests, and defects.
- Improves collaboration between testers and developers.

6. Azure Artifacts

Azure Artifacts provides a package management solution for teams. It allows developers to create, host, and share packages.

Supported Package Types:
- NuGet (for .NET)
- npm (for JavaScript/Node.js)
- Maven (for Java)
- Python packages
- Universal Packages
Key Features:
- Feeds – Store and distribute packages within the organization.
- Integration with Pipelines – Automate publishing and consuming packages.
- Versioning & Retention – Manage multiple versions and clean old ones.
Use Case Example:
- A team creates a shared library in .NET.
- Publishes it as a NuGet package in Azure Artifacts.
- Other projects consume it from the feed instead of duplicating code.

7. How the Components Work Together

Here’s how the pieces fit into a DevOps lifecycle:

Plan Work → Azure Boards (backlog, sprints, work items).
Develop Code → Azure Repos (Git source control).
Build & Test → Azure Pipelines (CI builds, automated tests).
Release → Azure Pipelines (CD deployment to environments).
Test Quality → Azure Test Plans (manual/automated tests).
Share Packages → Azure Artifacts (distribute dependencies).

This integration ensures traceability: from a feature request in Boards → commit in Repos → build in Pipelines → test in Test Plans → deployment in Pipelines → package distribution in Artifacts.

8. Benefits of Azure DevOps

Single integrated platform for the entire DevOps lifecycle.
Supports both Agile and DevOps methodologies.
Scalable for startups, mid-size companies, and enterprises.
Strong integration with Azure cloud and other external tools (Slack, Jira, GitHub, etc.).
Enhances collaboration, automation, and traceability.

9. Getting Started

Sign up at dev.azure.com.
Create an organization and a project.
Explore each service (Boards, Repos, Pipelines, etc.) step by step.
Start with a small project to learn before applying to enterprise-level projects.

Building and Deploying a Simple Banking Application Using Containerization and Cloud Automation

Goodluck Ekeoma Adiole — Wed, 27 Aug 2025 06:32:56 +0000

This project introduces participants to the fundamentals of modern application development and deployment by combining software development, containerization, and DevOps practices.

You will build a simple Banking API that allows basic operations such as creating accounts, deposits, withdrawals, and transfers. While the business logic is simple, the real learning comes from how the application is structured, containerized with Docker, tested with CI, and deployed automatically to an AWS EC2 instance using Bitbucket Pipelines.

Through this project, you will gain hands-on experience with:

Application development (Flask + SQLite)

Containerization (Docker, Gunicorn for production serving)

CI/CD pipelines (Bitbucket Pipelines building, testing, and pushing images)

Cloud deployment (running the container on AWS EC2)

By the end, you’ll understand how modern teams package and deploy applications reliably, a foundational skill for DevOps, Cloud, and Software Engineering careers.

1) Project structure

banking-app/
├─ banking_app/
│  ├─ __init__.py
│  ├─ app.py
│  ├─ models.py
│  ├─ db.py
│  └─ templates/
│     └─ index.html
├─ tests/
│  └─ test_api.py
├─ .env.example
├─ .gitignore
├─ Dockerfile
├─ gunicorn.conf.py
├─ Makefile
├─ README.md
├─ requirements.txt
└─ bitbucket-pipelines.yml

2) File contents (copy exactly)

banking_app/init.py

from flask import Flask
from .db import init_db, db_session, shutdown_session
from .app import api_blueprint

def create_app(config_override=None):
    app = Flask(__name__)
    app.config.from_mapping(
        SECRET_KEY="change-me",
        DATABASE_URL="sqlite:////app/data/banking.db",
        JSON_SORT_KEYS=False
    )
    if config_override:
        app.config.update(config_override)

    init_db(app.config["DATABASE_URL"])

    # Blueprints
    app.register_blueprint(api_blueprint)

    # DB session teardown
    app.teardown_appcontext(lambda exception: shutdown_session())

    # Health endpoint (for LB/containers)
    @app.get("/health")
    def health():
        return {"status": "ok"}, 200

    return app

banking_app/db.py

from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker, declarative_base

engine = None
db_session = scoped_session(sessionmaker(autocommit=False, autoflush=False))
Base = declarative_base()

def init_db(database_url):
    global engine
    engine = create_engine(database_url, connect_args={"check_same_thread": False} if database_url.startswith("sqlite") else {})
    db_session.configure(bind=engine)
    Base.query = db_session.query_property()
    from .models import Account, Transaction  # noqa
    Base.metadata.create_all(bind=engine)

def shutdown_session():
    db_session.remove()

banking_app/models.py

from datetime import datetime
from sqlalchemy import Column, Integer, String, DateTime, Float, ForeignKey
from sqlalchemy.orm import relationship
from .db import Base

class Account(Base):
    __tablename__ = "accounts"

    id = Column(Integer, primary_key=True)
    name = Column(String(100), nullable=False)
    balance = Column(Float, nullable=False, default=0.0)
    created_at = Column(DateTime, default=datetime.utcnow)

    transactions = relationship("Transaction", back_populates="account")

    def to_dict(self):
        return {
            "id": self.id,
            "name": self.name,
            "balance": round(self.balance, 2),
            "created_at": self.created_at.isoformat() + "Z",
        }

class Transaction(Base):
    __tablename__ = "transactions"

    id = Column(Integer, primary_key=True)
    type = Column(String(50), nullable=False)  # deposit, withdraw, transfer
    amount = Column(Float, nullable=False)
    note = Column(String(255))
    created_at = Column(DateTime, default=datetime.utcnow)

    account_id = Column(Integer, ForeignKey("accounts.id"), nullable=False)
    related_account_id = Column(Integer)  # for transfers

    account = relationship("Account", back_populates="transactions")

    def to_dict(self):
        return {
            "id": self.id,
            "type": self.type,
            "amount": round(self.amount, 2),
            "note": self.note,
            "created_at": self.created_at.isoformat() + "Z",
            "account_id": self.account_id,
            "related_account_id": self.related_account_id,
        }

banking_app/app.py

from flask import Blueprint, jsonify, request, render_template
from sqlalchemy.exc import IntegrityError
from .db import db_session
from .models import Account, Transaction

api_blueprint = Blueprint("api", __name__)

@api_blueprint.route("/", methods=["GET"])
def home():
    return render_template("index.html")

@api_blueprint.route("/api/accounts", methods=["POST"])
def create_account():
    data = request.get_json(silent=True) or {}
    name = (data.get("name") or "").strip()
    if not name:
        return jsonify({"error": "name is required"}), 400

    acct = Account(name=name, balance=0.0)
    db_session.add(acct)
    db_session.commit()
    return jsonify({"message": "account created", "account": acct.to_dict()}), 201

@api_blueprint.route("/api/accounts/<int:account_id>", methods=["GET"])
def get_account(account_id):
    acct = db_session.get(Account, account_id)
    if not acct:
        return jsonify({"error": "account not found"}), 404
    return jsonify(acct.to_dict()), 200

@api_blueprint.route("/api/accounts/<int:account_id>/deposit", methods=["POST"])
def deposit(account_id):
    acct = db_session.get(Account, account_id)
    if not acct:
        return jsonify({"error": "account not found"}), 404

    data = request.get_json(silent=True) or {}
    try:
        amount = float(data.get("amount"))
    except (TypeError, ValueError):
        return jsonify({"error": "valid amount is required"}), 400
    if amount <= 0:
        return jsonify({"error": "amount must be > 0"}), 400

    acct.balance += amount
    tx = Transaction(type="deposit", amount=amount, note=data.get("note"), account=acct)
    db_session.add(tx)
    db_session.commit()
    return jsonify({"message": "deposit successful", "account": acct.to_dict(), "transaction": tx.to_dict()}), 200

@api_blueprint.route("/api/accounts/<int:account_id>/withdraw", methods=["POST"])
def withdraw(account_id):
    acct = db_session.get(Account, account_id)
    if not acct:
        return jsonify({"error": "account not found"}), 404

    data = request.get_json(silent=True) or {}
    try:
        amount = float(data.get("amount"))
    except (TypeError, ValueError):
        return jsonify({"error": "valid amount is required"}), 400
    if amount <= 0:
        return jsonify({"error": "amount must be > 0"}), 400
    if acct.balance < amount:
        return jsonify({"error": "insufficient funds"}), 400

    acct.balance -= amount
    tx = Transaction(type="withdraw", amount=amount, note=data.get("note"), account=acct)
    db_session.add(tx)
    db_session.commit()
    return jsonify({"message": "withdrawal successful", "account": acct.to_dict(), "transaction": tx.to_dict()}), 200

@api_blueprint.route("/api/transfer", methods=["POST"])
def transfer():
    data = request.get_json(silent=True) or {}
    try:
        from_id = int(data.get("from_id"))
        to_id = int(data.get("to_id"))
        amount = float(data.get("amount"))
    except (TypeError, ValueError):
        return jsonify({"error": "from_id, to_id and valid amount are required"}), 400
    if amount <= 0:
        return jsonify({"error": "amount must be > 0"}), 400
    if from_id == to_id:
        return jsonify({"error": "cannot transfer to the same account"}), 400

    from_acct = db_session.get(Account, from_id)
    to_acct = db_session.get(Account, to_id)
    if not from_acct or not to_acct:
        return jsonify({"error": "one or both accounts not found"}), 404
    if from_acct.balance < amount:
        return jsonify({"error": "insufficient funds"}), 400

    # Perform transfer atomically
    try:
        from_acct.balance -= amount
        to_acct.balance += amount
        tx_out = Transaction(type="transfer", amount=-amount, note=f"to {to_acct.id}", account=from_acct, related_account_id=to_acct.id)
        tx_in = Transaction(type="transfer", amount=amount, note=f"from {from_acct.id}", account=to_acct, related_account_id=from_acct.id)
        db_session.add_all([tx_out, tx_in])
        db_session.commit()
    except IntegrityError:
        db_session.rollback()
        return jsonify({"error": "transfer failed"}), 500

    return jsonify({
        "message": "transfer successful",
        "from": from_acct.to_dict(),
        "to": to_acct.to_dict()
    }), 200

banking_app/templates/index.html

<!doctype html>
<html>
  <head>
    <meta charset="utf-8"/>
    <title>Banking API</title>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <style>
      body { font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; margin: 2rem; }
      code { background: #f5f5f5; padding: 0.15rem 0.3rem; border-radius: 6px; }
      pre { background: #f5f5f5; padding: 1rem; border-radius: 8px; overflow: auto; }
      a { color: #0a58ca; text-decoration: none; }
    </style>
  </head>
  <body>
    <h1>Banking API</h1>
    <p>This is a beginner-level banking API you can deploy with Docker and Bitbucket Pipelines.</p>

    <h2>Quick endpoints</h2>
    <ul>
      <li>GET <code>/health</code></li>
      <li>POST <code>/api/accounts</code> – body: <code>{"name":"Alice"}</code></li>
      <li>GET <code>/api/accounts/&lt;id&gt;</code></li>
      <li>POST <code>/api/accounts/&lt;id&gt;/deposit</code> – body: <code>{"amount":100}</code></li>
      <li>POST <code>/api/accounts/&lt;id&gt;/withdraw</code> – body: <code>{"amount":50}</code></li>
      <li>POST <code>/api/transfer</code> – body: <code>{"from_id":1,"to_id":2,"amount":25}</code></li>
    </ul>
  </body>
</html>

tests/test_api.py

import json
import os
import tempfile
import pytest
from banking_app import create_app
from banking_app.db import init_db

@pytest.fixture
def client():
    db_fd, db_path = tempfile.mkstemp()
    os.close(db_fd)
    app = create_app({
        "TESTING": True,
        "SECRET_KEY": "test",
        "DATABASE_URL": f"sqlite:///{db_path}"
    })
    with app.test_client() as client:
        yield client
    os.remove(db_path)

def test_health(client):
    rv = client.get("/health")
    assert rv.status_code == 200
    assert rv.get_json()["status"] == "ok"

def test_create_and_get_account(client):
    rv = client.post("/api/accounts", json={"name": "Alice"})
    assert rv.status_code == 201
    acct_id = rv.get_json()["account"]["id"]

    rv2 = client.get(f"/api/accounts/{acct_id}")
    assert rv2.status_code == 200
    assert rv2.get_json()["name"] == "Alice"

def test_deposit_withdraw_and_transfer(client):
    a = client.post("/api/accounts", json={"name": "A"}).get_json()["account"]["id"]
    b = client.post("/api/accounts", json={"name": "B"}).get_json()["account"]["id"]

    # Deposit to A
    rv = client.post(f"/api/accounts/{a}/deposit", json={"amount": 100})
    assert rv.status_code == 200
    assert rv.get_json()["account"]["balance"] == 100

    # Withdraw from A
    rv = client.post(f"/api/accounts/{a}/withdraw", json={"amount": 40})
    assert rv.status_code == 200
    assert rv.get_json()["account"]["balance"] == 60

    # Transfer 25 from A to B
    rv = client.post("/api/transfer", json={"from_id": a, "to_id": b, "amount": 25})
    assert rv.status_code == 200
    from_balance = rv.get_json()["from"]["balance"]
    to_balance = rv.get_json()["to"]["balance"]
    assert from_balance == 35
    assert to_balance == 25

.env.example

# Copy this to .env (for local dev) or /srv/banking-app/.env (on EC2)
FLASK_SECRET_KEY=change-this-in-production
DATABASE_URL=sqlite:////app/data/banking.db
LOG_LEVEL=INFO

.gitignore

__pycache__/
*.pyc
.env
.env.*
*.db
dist/
build/
*.egg-info
.pytest_cache/
data/

Dockerfile

# Simple, production-ready container
FROM python:3.11-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

# System deps
RUN apt-get update && apt-get install -y --no-install-recommends \
    tini curl ca-certificates && \
    rm -rf /var/lib/apt/lists/*

# App directory
WORKDIR /app

# Install deps first (better layer caching)
COPY requirements.txt .
RUN pip install -r requirements.txt

# Create a non-root user
RUN useradd -m appuser
RUN mkdir -p /app/data && chown -R appuser:appuser /app

# Copy app
COPY banking_app ./banking_app
COPY gunicorn.conf.py .
COPY .env.example ./.env.example

USER appuser

EXPOSE 8000

ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["gunicorn", "-c", "gunicorn.conf.py", "banking_app:create_app()"]

gunicorn.conf.py

bind = "0.0.0.0:8000"
workers = 2
threads = 2
timeout = 30
graceful_timeout = 30
keepalive = 5
accesslog = "-"
errorlog = "-"

Makefile

IMAGE ?= your-dockerhub-username/banking-app
TAG ?= local

.PHONY: run test build docker-run docker-push

run:
\tuvicorn

test:
\tpytest -q

build:
\tdocker build -t $(IMAGE):$(TAG) .

docker-run:
\tmkdir -p data
\tdocker run --rm -it -p 8000:8000 -v ${PWD}/data:/app/data --env-file .env $(IMAGE):$(TAG)

docker-push:
\tdocker push $(IMAGE):$(TAG)

requirements.txt

Flask==3.0.3
SQLAlchemy==2.0.31
gunicorn==22.0.0
pytest==8.2.2

bitbucket-pipelines.yml

image: atlassian/default-image:3

options:
  docker: true

pipelines:
  branches:
    main:
      - step:
          name: Build & Test
          caches:
            - docker
          services:
            - docker
          script:
            - echo "Running unit tests"
            - pip install -r requirements.txt
            - pytest -q
      - step:
          name: Build Docker image
          services:
            - docker
          script:
            - export DOCKER_IMAGE="${DOCKERHUB_USERNAME}/banking-app"
            - export TAG="${BITBUCKET_COMMIT:0:7}"
            - echo "$DOCKERHUB_PASSWORD" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
            - docker build -t "$DOCKER_IMAGE:$TAG" -t "$DOCKER_IMAGE:latest" .
            - docker push "$DOCKER_IMAGE:$TAG"
            - docker push "$DOCKER_IMAGE:latest"
          artifacts:
            - deployment/**
      - step:
          name: Deploy to EC2
          deployment: production
          script:
            - pipe: atlassian/ssh-run:0.7.0
              variables:
                SSH_USER: $EC2_USER
                SERVER: $EC2_HOST
                PORT: "22"
                MODE: "command"
                COMMAND: >
                  set -e;
                  sudo mkdir -p /srv/banking-app/data;
                  sudo chown -R $EC2_USER:$EC2_USER /srv/banking-app;
                  echo "$DOCKERHUB_PASSWORD" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin;
                  DOCKER_IMAGE="${DOCKERHUB_USERNAME}/banking-app";
                  TAG="${BITBUCKET_COMMIT:0:7}";
                  docker pull "$DOCKER_IMAGE:$TAG";
                  (docker rm -f banking-app || true);
                  docker run -d --name banking-app --restart always
                  -p 80:8000
                  --env-file /srv/banking-app/.env
                  -v /srv/banking-app/data:/app/data
                  "$DOCKER_IMAGE:$TAG";
definitions:
  services:
    docker:
      memory: 3072

Bitbucket Repository Variables you must add (Settings → Repository variables):

DOCKERHUB_USERNAME

DOCKERHUB_PASSWORD (use a Docker Hub access token)

EC2_HOST (e.g., 3.XX.XX.XX)

EC2_USER (e.g., ubuntu for Ubuntu, ec2-user for Amazon Linux)

SSH access: Add your Bitbucket Pipelines public key to the EC2 instance’s ~/.ssh/authorized_keys. In Bitbucket: Repository Settings → SSH keys → Generate keys, then copy the public key to the server.

3) README.md (instructions for completing the project)

Create this README.md in the repo root:

# Banking API — Docker + Bitbucket Pipelines → AWS EC2 (Beginner Friendly)

A tiny Flask + SQLite banking API you can build in Docker and deploy automatically to an AWS EC2 instance using Bitbucket Pipelines.

## Features

- Create accounts, deposit, withdraw, transfer
- SQLite database (file persisted on the server)
- Gunicorn production server
- Healthcheck `/health`
- Unit tests with pytest
- Bitbucket Pipelines: build → push Docker image → SSH deploy to EC2
- Zero-downtime style (container is replaced quickly)

---

## 1) Prerequisites

- Docker & Docker Compose (for local)
- A Docker Hub account (for pushing images)
- Bitbucket repository
- An EC2 instance (Ubuntu 22.04 LTS or Amazon Linux 2023 recommended)
- Security Group allows inbound TCP 80 (HTTP) from your IP/Internet

---

## 2) Run locally (optional, recommended)

```

bash
# Clone and enter
git clone <your-repo-url> banking-app
cd banking-app

# Create a local .env from example (you can keep defaults)
cp .env.example .env

# Build and run
docker build -t your-dockerhub-username/banking-app:local .
docker run --rm -it -p 8000:8000 \
  -v ${PWD}/data:/app/data \
  --env-file .env \
  your-dockerhub-username/banking-app:local

Test in another terminal:


bash
curl -s http://localhost:8000/health
curl -s -X POST http://localhost:8000/api/accounts -H "Content-Type: application/json" -d '{"name":"Alice"}'

3) Run tests locally


bash
pip install -r requirements.txt
pytest -q

4) Prepare AWS EC2 (one-time)

Launch EC2

AMI: Ubuntu 22.04 LTS (or Amazon Linux 2023)
Instance type: t3.micro is fine for demo
Security Group: allow inbound HTTP (80) and SSH (22) from your IP

SSH into the server


bash
   ssh -i your-key.pem ubuntu@EC2_PUBLIC_IP

Install Docker

Ubuntu:


bash
 sudo apt-get update
 sudo apt-get install -y ca-certificates curl
 sudo install -m 0755 -d /etc/apt/keyrings
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
 echo \
   "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
   $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
   sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 sudo apt-get update
 sudo apt-get install -y docker-ce docker-ce-cli containerd.io
 sudo usermod -aG docker $USER
 newgrp docker

Create app folder and .env


bash
   sudo mkdir -p /srv/banking-app/data
   sudo chown -R $USER:$USER /srv/banking-app

   cat << 'EOF' > /srv/banking-app/.env
   FLASK_SECRET_KEY=change-this-in-production
   DATABASE_URL=sqlite:////app/data/banking.db
   LOG_LEVEL=INFO
   EOF

Add Bitbucket Pipelines SSH key to server

In Bitbucket: Repository Settings → SSH keys → Generate key

Copy the public key shown there and append to the server:


bash
 mkdir -p ~/.ssh
 chmod 700 ~/.ssh
 echo "<PASTE_PUBLIC_KEY_HERE>" >> ~/.ssh/authorized_keys
 chmod 600 ~/.ssh/authorized_keys

(Optional) Open firewall for HTTP

If using Ubuntu UFW:


bash
 sudo ufw allow 22/tcp
 sudo ufw allow 80/tcp
 sudo ufw enable
 sudo ufw status

5) Configure Bitbucket Pipelines

Enable Pipelines: Repository → Pipelines → Enable
Repository Variables (Settings → Repository variables):

DOCKERHUB_USERNAME = <your-dockerhub-username>
DOCKERHUB_PASSWORD = <docker-hub-access-token> (create from Docker Hub → Security)
EC2_HOST = ec2-x-x-x-x.compute-1.amazonaws.com or the public IP
EC2_USER = ubuntu (or ec2-user on Amazon Linux)
1. SSH key: Repository Settings → SSH keys → Generate keys
This is the key Pipelines uses to SSH into your EC2.
Add the public key to your EC2 server as described above.

6) Deploy flow

Push to branch main.
Pipeline runs:

Install deps + tests
Build Docker image, tag with short commit SHA and latest, push to Docker Hub
SSH into EC2:

 * Login to Docker Hub
 * Pull the image
 * Stop and remove the running container (if any)
 * Run a new container on port 80 → 8000

Verify:
Visit http://<EC2_PUBLIC_IP>/health → should return {"status":"ok"}.

7) Example API usage


bash
# Create two accounts
curl -s -X POST http://EC2_PUBLIC_IP/api/accounts -H "Content-Type: application/json" -d '{"name":"Alice"}'
curl -s -X POST http://EC2_PUBLIC_IP/api/accounts -H "Content-Type: application/json" -d '{"name":"Bob"}'

# Deposit to Alice (id 1)
curl -s -X POST http://EC2_PUBLIC_IP/api/accounts/1/deposit -H "Content-Type: application/json" -d '{"amount": 100}'

# Transfer from Alice (1) to Bob (2)
curl -s -X POST http://EC2_PUBLIC_IP/api/transfer -H "Content-Type: application/json" -d '{"from_id":1,"to_id":2,"amount":25}'

8) Common troubleshooting

403/timeout reaching EC2: Check Security Group allows inbound TCP 80 from your IP/Internet.
Pipeline can’t SSH: Ensure Bitbucket Pipelines public key is in ~/.ssh/authorized_keys on EC2 and you used the correct EC2_USER.
Container fails to start: docker logs banking-app on EC2 to see errors.
DB not persisting: Ensure volume -v /srv/banking-app/data:/app/data is present in the docker run command (the pipeline does this).
Change port: Edit -p 80:8000 in the deploy step, and adjust security group accordingly.

9) Cleaning up / re-deploy manually


bash
# On EC2:
docker ps
docker stop banking-app && docker rm banking-app
docker pull your-dockerhub-username/banking-app:latest
docker run -d --name banking-app --restart always -p 80:8000 \
  --env-file /srv/banking-app/.env \
  -v /srv/banking-app/data:/app/data \
  your-dockerhub-username/banking-app:latest

10) Notes on security (basics)

Always use a Docker Hub access token (not your password) for DOCKERHUB_PASSWORD.
Keep FLASK_SECRET_KEY strong and private in /srv/banking-app/.env (don’t commit it).
Consider attaching an Elastic IP to the instance and setting up a domain + HTTPS with a reverse proxy later.

CI/CD With GitHub Actions — A Practical, End‑to‑End Tutorial

Goodluck Ekeoma Adiole — Sat, 16 Aug 2025 15:49:26 +0000

What Is CI/CD?

Continuous Integration (CI): automate build, lint, test, and packaging on every change.
Continuous Delivery/Deployment (CD): automate promotion to environments (staging → production), often with approvals, feature flags, and rollbacks.
Benefits: faster feedback, fewer regressions, reproducible releases, higher confidence.

CI/CD Tools at a Glance (Quick Compare)

Jenkins: highly extensible, self‑hosted; you manage controllers/agents and plugins.
GitLab CI: tightly integrated with GitLab; YAML pipelines, built‑in container registry.
Azure Pipelines: great for Microsoft stacks; Windows/macOS/Linux hosted pools.
CircleCI: cloud‑hosted, fast parallelism, orbs ecosystem.
Bitbucket Pipelines: simple pipelines for Bitbucket repos.
Buildkite: hybrid model; run builders on your infra.
Tekton: Kubernetes‑native pipelines (CRDs).
Argo CD / Flux: GitOps CD for Kubernetes (pull‑based).
Spinnaker / Harness: powerful multi‑cloud CD, canary/blue‑green baked in.

Why GitHub Actions? Native to GitHub, enormous marketplace, generous hosted runners, great DX, reusable workflows, and first‑class security integrations (OIDC, environments, approvals).

GitHub Actions Core Concepts

Workflow: YAML in .github/workflows/*.yml
Trigger (Event): push, pull_request, workflow_dispatch, schedule, release, etc.
Job: runs on a runner (runs-on: ubuntu-latest, windows-latest, etc.). Jobs can depend on others via needs.
Step: individual shell command or “action” (like actions/checkout).
Runners: GitHub‑hosted or self‑hosted (ephemeral or static).
Artifacts & Caching: persist build outputs; speed up installs.
Environments: dev, staging, prod with protection rules, approvals, and secrets.
Secrets & Variables: org/repo/environment scope; injected at runtime.
Permissions: least‑privilege via permissions: (and OIDC via id-token: write).

A Minimal CI Workflow (Node example)

Create .github/workflows/ci.yml:

name: CI

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Use Node
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install
        run: npm ci

      - name: Lint
        run: npm run lint --if-present

      - name: Test
        run: npm test -- --ci --reporters=default --reporters=jest-junit

Notes:

Runs on PRs and on pushes to main.
Caches node_modules automatically via setup-node@v4 + cache: npm.

Python & Java Variants

Python (.github/workflows/python-ci.yml):

name: Python CI
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.10, 3.11, 3.12]
      fail-fast: false
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pip install -r requirements.txt
      - run: pytest -q --maxfail=1 --disable-warnings --junitxml=report.xml
      - uses: actions/upload-artifact@v4
        with:
          name: pytest-report
          path: report.xml

Java (Gradle):

name: Java CI
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 21
          cache: gradle
      - run: ./gradlew build --stacktrace
      - uses: actions/upload-artifact@v4
        with:
          name: app-jar
          path: build/libs/*.jar

Service Containers for Integration Tests

Example: test against Postgres and Redis

name: Integration Tests
on: [push, pull_request]

jobs:
  itest:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_USER: app
          POSTGRES_PASSWORD: password
          POSTGRES_DB: appdb
        ports: ["5432:5432"]
        options: >-
          --health-cmd="pg_isready -U app -d appdb"
          --health-interval=10s --health-timeout=5s --health-retries=5
      redis:
        image: redis:7

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: 3.11 }
      - run: pip install -r requirements.txt
      - env:
          DATABASE_URL: postgresql://app:password@localhost:5432/appdb
          REDIS_URL: redis://localhost:6379
        run: pytest tests/integration -q

Caching & Artifacts (Speed and Traceability)

Use setup actions’ built‑in caching (cache: npm/pip/gradle).
For custom caches:

- name: Cache build
  uses: actions/cache@v4
  with:
    path: .m2/repository
    key: maven-${{ runner.os }}-${{ hashFiles('**/pom.xml') }}
    restore-keys: |
      maven-${{ runner.os }}-

Upload build outputs for later jobs or downloads:

- uses: actions/upload-artifact@v4
  with:
    name: dist
    path: dist/**
    if-no-files-found: error

CD Basics: Environments, Approvals, and Secrets

Create environments in GitHub: dev, staging, prod.
Add environment secrets/variables (e.g., PROD_DB_URL).
Configure protection rules (approvers, wait timers).

Sample CD job gated by an environment:

jobs:
  deploy_prod:
    needs: build
    runs-on: ubuntu-latest
    environment:
      name: prod
      url: https://app.example.com
    permissions:
      contents: read
      id-token: write  # for OIDC to cloud providers
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        run: ./scripts/deploy.sh

Approvers receive a prompt in the PR/Actions UI before the job runs.

Multi‑Stage CI/CD (Build → Test → Deploy)

name: Webapp CI/CD

on:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: read
  id-token: write

concurrency:
  group: app-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      image: ${{ steps.meta.outputs.tags }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - id: meta
        uses: docker/metadata-action@v5
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=ref,event=branch
            type=sha
            type=semver,pattern={{version}}
      - name: Build & Push
        uses: docker/build-push-action@v6
        with:
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

  deploy_staging:
    runs-on: ubuntu-latest
    needs: [test]
    environment: staging
    steps:
      - name: Deploy to Staging
        run: ./scripts/deploy_staging.sh ${{ needs.build.outputs.image }}

  deploy_prod:
    runs-on: ubuntu-latest
    needs: [deploy_staging]
    environment: prod
    steps:
      - name: Deploy to Production
        run: ./scripts/deploy_prod.sh ${{ needs.build.outputs.image }}

Cloud Deployments via OIDC (No Long‑Lived Secrets)

AWS (ECS/EKS/Lambda/etc.)

- uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::123456789012:role/GHActionsDeployRole
    aws-region: eu-west-1

- name: Deploy infra with Terraform
  run: |
    terraform init
    terraform apply -auto-approve

- name: Update ECS service
  run: aws ecs update-service --cluster app --service web --force-new-deployment

Prereq: set IAM role with a trust policy allowing GitHub’s OIDC provider and your repo/environment.

Azure (Web Apps/AKS)

- uses: azure/login@v2
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

- name: Deploy to Azure Web App
  uses: azure/webapps-deploy@v3
  with:
    app-name: my-webapp
    images: ghcr.io/${{ github.repository }}:sha-${{ github.sha }}

Use Azure Federated Credentials for your repo/environment to avoid client secret rotation.

GCP (Cloud Run/GKE)

- uses: google-github-actions/auth@v2
  with:
    workload_identity_provider: projects/123/locations/global/workloadIdentityPools/gh/providers/github
    service_account: gha-deployer@myproj.iam.gserviceaccount.com

- name: Deploy to Cloud Run
  run: |
    gcloud run deploy web --image=ghcr.io/${{ github.repository }}:${{ github.sha }} --region=europe-west1

Terraform in Actions (Infra as Code)

name: Terraform

on:
  pull_request:
    paths: ["infra/**.tf", ".github/workflows/terraform.yml"]
  push:
    branches: [main]
    paths: ["infra/**.tf"]

jobs:
  plan:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform -chdir=infra init
      - run: terraform -chdir=infra plan -out=plan.out
      - uses: actions/upload-artifact@v4
        with:
          name: tf-plan
          path: infra/plan.out

  apply:
    if: github.ref == 'refs/heads/main'
    needs: plan
    environment: prod
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - uses: actions/download-artifact@v4
        with: { name: tf-plan, path: infra }
      - run: terraform -chdir=infra apply -auto-approve plan.out

Secure Supply Chain (SCA, SAST, Code Scanning)

Dependency Review on PRs:

name: Dependency Review
on: [pull_request]
permissions:
  contents: read
  pull-requests: write
jobs:
  dep-review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/dependency-review-action@v4

CodeQL (static analysis):

name: CodeQL
on:
  push: { branches: [main] }
  pull_request:
  schedule: [{ cron: "35 1 * * 1" }] # weekly
permissions:
  contents: read
  security-events: write
jobs:
  analyze:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix: { language: [javascript, python, java] }
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with: { languages: ${{ matrix.language }} }
      - uses: github/codeql-action/analyze@v3

Optional: container provenance (SLSA‑style):

- name: Attest build provenance
  uses: actions/attest-build-provenance@v1
  with:
    subject-name: ghcr.io/${{ github.repository }}
    subject-digest: ${{ steps.meta.outputs.digest }}

Reusable Workflows & Composite Actions

Reusable workflow (publisher):

.github/workflows/reusable-test.yml

name: Reusable Test
on:
  workflow_call:
    inputs:
      node-version: { required: true, type: string }
jobs:
  run-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: ${{ inputs.node-version }}, cache: npm }
      - run: npm ci && npm test

Caller:

jobs:
  tests:
    uses: your-org/your-repo/.github/workflows/reusable-test.yml@main
    with:
      node-version: "20"

Composite action (encapsulate repeatable steps):

.github/actions/setup-app/action.yml

name: "Setup App"
runs:
  using: "composite"
  steps:
    - uses: actions/setup-node@v4
      with: { node-version: 20, cache: npm }
    - run: npm ci
      shell: bash

Use it:

- uses: ./.github/actions/setup-app

Monorepos & Path Filters

Run pipelines only when relevant code changes:

on:
  pull_request:
    paths:
      - "services/api/**"
      - ".github/workflows/api-*.yml"

jobs:
  api-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: make -C services/api test

Matrix by service:

strategy:
  matrix:
    service: [api, web, worker]
steps:
  - run: make -C services/${{ matrix.service }} test

Branching & Release Strategies

Trunk‑based (recommended): PRs → main; feature flags; short‑lived branches.
GitFlow: develop, release/*, hotfix/*; more ceremony.
Semver tags trigger releases:

on:
  push:
    tags: ["v*.*.*"]

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Changelog
        run: npx conventional-changelog -p angular -i CHANGELOG.md -s
      - name: Create GitHub Release
        uses: softprops/action-gh-release@v2
        with:
          files: |
            dist/**

Advanced Controls & Guardrails

concurrency: to prevent overlapping deploys.
environment: with required reviewers and wait timers.
timeout-minutes: per job.
if: conditions (e.g., only deploy on tags).
needs: to enforce stage order.
permissions: minimal scopes; add id-token: write only when needed.
Pin actions to major versions or SHAs (for maximum supply‑chain safety).
Secret scanning & push protection: keep secrets out of code.

Self‑Hosted Runners (When and How)

Why: custom tools, private networks, GPUs, cost control, long builds.

Basic setup steps:

Provision VM/container with network access to targets.
Create runner in repo/org (Settings → Actions → Runners).
Label runners (e.g., self-hosted, gpu, arm64).
Prefer ephemeral/auto‑scaled runners (clean state per job).

Use in workflow:

jobs:
  build:
    runs-on: [self-hosted, linux, x64, docker]

Security tips:

Lock runners to specific repos.
Rotate tokens; auto‑update runner.
Isolate with VM snapshots or ephemeral images.

Slide 18 — Observability, Test Reports, Coverage, and Artifacts

Upload test reports & coverage to keep PR feedback rich.
Publish HTML reports as artifacts or Pages in non‑prod.
Example (Jest coverage):

- run: npm run test -- --coverage
- uses: actions/upload-artifact@v4
  with:
    name: coverage
    path: coverage/**

Annotate PRs via problem matchers or actions (eslint, flake8, etc.).

Scheduling, Manual Runs, and Branch Protections

Schedules use UTC (not repository timezone):

on:
  schedule:
    - cron: "0 5 * * 1-5" # 05:00 UTC on weekdays
  workflow_dispatch:
    inputs:
      target:
        description: "Env to deploy"
        required: true
        default: "staging"

Combine with branch protection rules (required checks before merge).

End‑to‑End Example: Dockerized API → Staging/Prod (ECS)

.github/workflows/api-cicd.yml:

name: API CI/CD

on:
  pull_request:
    paths: ["api/**", ".github/workflows/api-cicd.yml"]
  push:
    branches: [main]
    paths: ["api/**", ".github/workflows/api-cicd.yml"]

permissions:
  contents: read
  id-token: write

env:
  IMAGE_NAME: ghcr.io/${{ github.repository }}/api

jobs:
  ci:
    runs-on: ubuntu-latest
    outputs:
      image_tag: ${{ steps.meta.outputs.version }}
    steps:
      - uses: actions/checkout@v4

      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.IMAGE_NAME }}
          tags: |
            type=sha,format=long
            type=ref,event=branch

      - name: Build & Push
        uses: docker/build-push-action@v6
        with:
          context: ./api
          push: true
          tags: ${{ steps.meta.outputs.tags }}

      - uses: actions/upload-artifact@v4
        with:
          name: image-tags
          path: |
            # emit a simple file with the final tag(s)
            /dev/stdout
        if: always()

  deploy_staging:
    needs: ci
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/gh-ecs-deployer
          aws-region: eu-west-1
      - name: Update ECS service (staging)
        run: |
          aws ecs update-service \
            --cluster app-staging \
            --service api \
            --force-new-deployment

  deploy_prod:
    if: startsWith(github.ref, 'refs/tags/v')
    needs: deploy_staging
    runs-on: ubuntu-latest
    environment: prod
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/gh-ecs-deployer
          aws-region: eu-west-1
      - name: Update ECS service (prod)
        run: |
          aws ecs update-service \
            --cluster app-prod \
            --service api \
            --force-new-deployment

Flow:

PR → CI only.
Push to main → build/push image + deploy to staging.
Create tag vX.Y.Z → deploy to prod (after staging job completes and environment approval, if configured).

Common Pitfalls & Troubleshooting

“Permission denied” on checkout/push: set permissions: contents: write when publishing tags or creating releases.
OIDC failing: verify correct audience/repo/environment in cloud role trust policy.
Slow builds: add caches, narrow paths, enable parallel matrix, use bigger runners (e.g., ubuntu-latest vs ubuntu-24.04 as available).
Flaky tests: add service health checks and retries; separate unit vs integration; use timeout-minutes.
Secrets not found: confirm secret scope (environment vs repo vs org) and name casing.

Security Best Practices Checklist

Least‑privilege permissions: per workflow/job.
Use environments for prod with required reviewers.
Prefer OIDC to cloud over static keys.
Pin actions to major versions or commit SHAs.
Keep runners ephemeral or routinely cleaned.
Enable branch protections + required status checks.
Turn on secret scanning, Dependabot alerts & updates.
Store sensitive config as environment secrets, not repo secrets if env‑specific.

Suggested Project Structure

.
├─ api/                      # your app(s)
├─ infra/                    # terraform/helm
├─ scripts/                  # deploy scripts
├─ .github/
│  ├─ actions/
│  │  └─ setup-app/          # composite action
│  └─ workflows/
│     ├─ ci.yml
│     ├─ codeql.yml
│     ├─ terraform.yml
│     └─ api-cicd.yml
└─ Dockerfile

Quick Start Checklist

Add .github/workflows/ci.yml and run a PR.
Add environments and secrets for staging and prod.
Add OIDC role/federated credentials in your cloud.
Implement build → test → deploy workflow with approvals.
Add CodeQL + Dependency Review.
Add path filters and caches.
Monitor, iterate, and keep pipelines as code.

Copy‑Paste Starters (Grab‑Bag)

Manual Deploy with Inputs

on:
  workflow_dispatch:
    inputs:
      env:
        type: choice
        options: [staging, prod]
        default: staging

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.env }}
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy_${{ inputs.env }}.sh

Blue‑Green (Feature Flag‑Style) Toggle

- name: Flip production traffic
  run: ./scripts/switch_traffic.sh --to blue

Conditional Job (Only on PRs from internal repo)

if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}

What to Implement Next

Tracing CI duration and break‑down per step; set goals for speed.
Parallelize test suites via matrix/shards.
Add canary/percentage rollouts (ECS, Cloud Run, AKS/GKE).
GitOps for Kubernetes (Argo CD/Flux) and make Actions only push manifests/images.
DORA metrics (deployment frequency, lead time, MTTR, change‑fail rate) from Actions runs.

Tutorial Presentation on CI/CD with GitHub Actions

Goodluck Ekeoma Adiole — Sat, 16 Aug 2025 15:43:26 +0000

1. Introduction to CI/CD

Continuous Integration (CI):
- A software development practice where developers frequently merge code changes into a shared repository.
- Automated builds and tests are run to detect integration issues early.
- Benefits: faster feedback, reduced integration problems, improved collaboration.
Continuous Delivery (CD):
- Extends CI by automatically preparing code for release to production.
- Ensures software can be deployed reliably at any time.
- Deployment can be manual but is always ready.
Continuous Deployment (CD - extended form):
- Goes a step further: every change that passes all pipeline stages is automatically deployed to production.
- Requires very high confidence in automated testing and monitoring.

2. Why CI/CD is Important

Speeds up software development lifecycle (SDLC).
Ensures high-quality code through automated testing.
Reduces deployment risks and downtime.
Promotes collaboration between developers, operations, and security teams (DevOps/DevSecOps).
Enables faster innovation while maintaining stability.

3. Popular CI/CD Tools (Brief Overview)

Jenkins: Open-source automation server, highly customizable with plugins.
GitLab CI/CD: Integrated with GitLab repositories, offers built-in pipelines.
CircleCI: Cloud-based CI/CD tool with container-native workflows.
Azure DevOps Pipelines: Microsoft solution with strong integration to Azure services.
Travis CI: Well-known CI/CD platform integrated with GitHub (mostly for open-source).
GitHub Actions: Native CI/CD for GitHub repositories, deeply integrated into the GitHub ecosystem.

(Next sections will focus deeply on GitHub Actions.)

4. Introduction to GitHub Actions

A CI/CD platform built directly into GitHub.
Automates workflows across software development lifecycle: build, test, deploy.
Event-driven: workflows can be triggered by repository events (push, pull request, release).
Uses YAML configuration files stored in .github/workflows/.

Key Benefits:

Native GitHub integration.
Free usage with GitHub-hosted runners (with limitations).
Support for self-hosted runners (for more control).
Large marketplace of prebuilt actions.

5. Core Concepts of GitHub Actions

Workflows

Automated processes defined in .yml files.
Example: "Run tests on every push."

Events

Triggers that start workflows.
Examples: push, pull_request, schedule, workflow_dispatch.

Jobs

A set of steps executed on the same runner.
Can run sequentially or in parallel.

Steps

Individual tasks inside a job.
Each step can run commands or actions.

Actions

Reusable units of code packaged for specific tasks.
Examples: checkout code, set up Node.js, deploy to AWS.

Runners

Virtual machines that execute workflows.
Options: GitHub-hosted (Linux, Windows, macOS) or self-hosted.

6. Workflow Lifecycle in GitHub Actions

Developer pushes code to GitHub.
Trigger event activates the workflow.
Workflow runner executes jobs as defined.
Steps run sequentially within each job.
Jobs may run in parallel or depend on each other.
Results (logs, artifacts, test results) are reported in GitHub UI.
(Optional) Deployment to staging/production occurs.

7. Typical CI/CD Workflow with GitHub Actions

Continuous Integration Workflow

Trigger: push or pull_request.
Jobs:
- Checkout code.
- Install dependencies.
- Run static analysis (linting, code quality checks).
- Run unit/integration tests.
- Build artifacts.

Continuous Delivery Workflow

Trigger: Merge into main or tagged release.
Jobs:
- Build Docker image or application package.
- Run security scans.
- Deploy to staging environment.
- (Optional) Await manual approval for production release.

Continuous Deployment Workflow

Same as delivery, but automatically deploys to production after successful tests.

8. Best Practices for GitHub Actions

Use caching (e.g., dependencies) to speed up workflows.
Separate workflows for CI (test/build) and CD (deployment).
Use secrets management (GitHub Secrets) for credentials.
Set branch protection rules to enforce CI checks before merging.
Reuse actions from the Marketplace to save time.
Keep workflows modular and readable.
Monitor logs and set up alerts for failed workflows.

9. Advantages of GitHub Actions over Other Tools

Direct integration with GitHub repositories (no external setup).
Easy YAML configuration and readability.
Access to GitHub ecosystem (Marketplace, APIs).
Free tier sufficient for many small to medium projects.
Supports both simple and complex workflows.
Good balance of flexibility and ease of use compared to Jenkins or GitLab CI.

10. Example Use Cases

CI Only: Run tests and lint checks for every pull request.
CD to Cloud: Deploy app to AWS, Azure, or GCP after successful tests.
Security Automation: Run vulnerability scans with Dependabot or custom security actions.
Scheduled Jobs: Nightly builds, automated backups, or dependency updates.
Custom DevOps Pipelines: Integrate with Docker, Kubernetes, or Terraform.

11. Challenges & Considerations

Limited free minutes and storage (for large projects, consider self-hosted runners).
YAML syntax errors can break workflows—needs careful review.
May require external integrations for advanced deployment strategies.
Security risks if secrets are not managed properly.

12. Conclusion

CI/CD is essential for modern software delivery, improving quality and speed.
GitHub Actions provides a powerful, integrated, and user-friendly platform for CI/CD directly within GitHub.
By understanding workflows, jobs, steps, and runners, teams can build robust automation pipelines.
Adoption of best practices ensures secure, efficient, and scalable software delivery pipelines.

Containerizing and Deploying a Simple FastAPI Application to AWS EC2.

Goodluck Ekeoma Adiole — Sun, 10 Aug 2025 16:27:05 +0000

This project demonstrates the end-to-end process of building, containerizing, and deploying a lightweight FastAPI application to an AWS EC2 instance. FastAPI, known for its high performance and intuitive API design, is paired with Docker to create a portable, consistent runtime environment. The containerized application is then hosted on AWS EC2, making it accessible over the internet.
The guide is intentionally simple, making it suitable for beginners who want hands-on experience with API development, containerization, and cloud deployment. By completing this project, you’ll understand how to:

Develop a basic FastAPI application.
Write a Dockerfile to containerize the app.
Build, tag, and push Docker images to a registry.
Configure and run the container on an AWS EC2 instance.

It’s a foundational DevOps exercise that blends software development with infrastructure management — essential for modern application delivery.

Project layout

fastapi-simple/
├─ main.py
├─ requirements.txt
├─ Dockerfile
├─ .dockerignore

1) Code — `main.py`

Create main.py:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title="Simple FastAPI App")

class Echo(BaseModel):
    message: str

@app.get("/")
def read_root():
    return {"message": "Hello from FastAPI in Docker!"}

@app.get("/health")
def health():
    return {"status": "ok"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: str | None = None):
    return {"item_id": item_id, "q": q}

@app.post("/echo")
def post_echo(payload: Echo):
    return {"you_sent": payload}

2) Dependencies — `requirements.txt`

fastapi
uvicorn[standard]

3) Dockerfile

Create Dockerfile (uses a small Python base, installs dependencies, runs uvicorn):

# Use an official Python runtime as a parent image
FROM python:3.11-slim

# Prevent Python from writing .pyc files and enable stdout buffering
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

# Copy only requirements first for better caching
COPY requirements.txt .

# Install pip dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy app source
COPY . .

# Expose the port uvicorn will listen on
EXPOSE 8000

# Run the app with uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

4) .dockerignore (optional but recommended)

__pycache__
*.pyc
.env
.venv
.git

5) Build & run locally

From the project root:

# build
docker build -t fastapi-simple:local .

# run (maps container port 8000 -> host 8000)
docker run --rm -p 8000:8000 fastapi-simple:local

Open http://localhost:8000/ and http://localhost:8000/docs (interactive API docs).

6) Push to Docker Hub (simple public flow)

Create a repo at hub.docker.com (or use an existing one).
Tag and push:

docker login                # enter Docker Hub credentials
docker tag fastapi-simple:local yourhubusername/fastapi-simple:latest
docker push yourhubusername/fastapi-simple:latest

If you want a private registry, consider AWS ECR (more secure for production).

7) Deploy on AWS EC2 (manual steps)

Use an Ubuntu 22.04 (or similar) instance. Minimal overview:

Create an EC2 instance

AMI: Ubuntu 22.04 LTS
Instance type: t3.micro (or as needed)
Key pair: create/download one (you’ll use .pem to SSH)
Security Group: allow inbound TCP ports 22 (SSH) and 80 (HTTP). If you plan to use 8000, open that too.

SSH to EC2

ssh -i path/to/key.pem ubuntu@YOUR_EC2_PUBLIC_IP

Install Docker (simple method)

sudo apt update
sudo apt install -y docker.io
sudo systemctl enable --now docker
# give your ubuntu user permission to use docker (you may need to logout/login)
sudo usermod -aG docker $USER

Pull the image and run it (map container 8000 -> host 80)

# (optional) docker login
docker pull yourhubusername/fastapi-simple:latest

# run as detached, restart automatically on reboot/crash
docker run -d --name fastapi \
  --restart unless-stopped \
  -p 80:8000 \
  yourhubusername/fastapi-simple:latest

Visit http://YOUR_EC2_PUBLIC_IP/ in your browser. API docs: http://YOUR_EC2_PUBLIC_IP/docs.

8) Optional: run container as a systemd service (survives reboots)

Create /etc/systemd/system/fastapi.service with:

[Unit]
Description=FastAPI Docker Container
After=docker.service
Requires=docker.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/docker stop fastapi
ExecStartPre=-/usr/bin/docker rm fastapi
ExecStart=/usr/bin/docker run --name fastapi --rm --restart unless-stopped -p 80:8000 yourhubusername/fastapi-simple:latest
ExecStop=/usr/bin/docker stop fastapi

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable --now fastapi.service
sudo journalctl -u fastapi -f

9) Optional: EC2 user-data (auto bootstrap on instance creation)

You can use this script as user-data when launching the instance to auto-install docker and run the container. Warning: do not embed Docker Hub credentials in user-data for security — use public image or ECR + IAM role.

Example user-data (Ubuntu):

#!/bin/bash
apt update
apt install -y docker.io
systemctl enable --now docker
# pull public image
docker pull yourhubusername/fastapi-simple:latest
docker run -d --restart unless-stopped --name fastapi -p 80:8000 yourhubusername/fastapi-simple:latest

10) Troubleshooting tips

If you get a blank page: check docker logs fastapi on EC2.
If EC2 public IP not serving: confirm Security Group inbound rules include port 80, and that the container is running (docker ps).
If docker commands require sudo: re-login after usermod -aG docker $USER or run with sudo.
For private images: use docker login on the EC2 instance before docker pull, or use AWS ECR with an instance role.

11) Next steps / production notes (short)

For production, use a process manager / orchestrator (ECS, EKS, or at least systemd + reverse proxy like Nginx).
Use HTTPS: put an nginx reverse proxy + certbot / or use a load balancer with TLS.
Use AWS ECR + instance profile or secrets manager instead of embedding Docker Hub credentials.

Enterprise-Grade FastAPI Application — Containerized and Deployed on AWS EC2

Goodluck Ekeoma Adiole — Sun, 10 Aug 2025 16:20:19 +0000

A compact enterprise-grade project layout and the key source files (ready-to-run).
A production Dockerfile (multi-stage, non-root).
A docker-compose for local dev (Postgres).
Step-by-step commands to build → push to ECR → deploy to EC2 (including systemd unit example).
Notes about security & production best-practices and citations for the most important platform commands.

I kept code pragmatic and copy-paste ready. Replace placeholders (YOUR_REGION, AWS_ACCOUNT_ID, DB creds, secret keys) with your values.

1. Project layout (suggested)

fastapi-enterprise/
├─ app/
│  ├─ main.py
│  ├─ core/
│  │  ├─ config.py
│  │  ├─ logging.py
│  │  └─ security.py
│  ├─ db/
│  │  ├─ session.py
│  │  └─ base.py
│  ├─ models/
│  │  └─ user.py
│  ├─ schemas/
│  │  └─ user.py
│  ├─ crud/
│  │  └─ user.py
│  └─ api/
│     ├─ deps.py
│     └─ v1/
│        └─ routers/
│           └─ users.py
├─ alembic/   (optional migrations)
├─ requirements.txt
├─ Dockerfile
├─ docker-compose.yml
├─ .env.example
└─ gunicorn_conf.py   (optional)

2. Key source files

All code below fits the project layout above. Keep files under app/.

app/core/config.py

from pydantic import BaseSettings, AnyUrl
from typing import Optional

class Settings(BaseSettings):
    APP_NAME: str = "FastAPI Enterprise"
    ENV: str = "production"
    DEBUG: bool = False

    # Database (asyncpg example)
    DATABASE_URL: AnyUrl = "postgresql+asyncpg://postgres:password@postgres:5432/appdb"

    # Security
    SECRET_KEY: str
    ACCESS_TOKEN_EXPIRE_MINUTES: int = 60 * 24

    # Gunicorn/worker tuning
    WORKERS: int = 4

    class Config:
        env_file = ".env"
        env_file_encoding = "utf-8"

settings = Settings()

app/core/logging.py

import logging
import sys

def setup_logging():
    fmt = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
    logging.basicConfig(stream=sys.stdout, level=logging.INFO, format=fmt)
    # Optionally integrate with json-loggers or structlog for structured logs

app/core/security.py

from passlib.context import CryptContext
from datetime import datetime, timedelta
import jwt
from app.core.config import settings

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

def hash_password(password: str) -> str:
    return pwd_context.hash(password)

def verify_password(plain: str, hashed: str) -> bool:
    return pwd_context.verify(plain, hashed)

def create_access_token(subject: str, expires_minutes: int = None):
    expire = datetime.utcnow() + timedelta(minutes=(expires_minutes or settings.ACCESS_TOKEN_EXPIRE_MINUTES))
    payload = {"sub": subject, "exp": expire}
    token = jwt.encode(payload, settings.SECRET_KEY, algorithm="HS256")
    return token

app/db/session.py

from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
from sqlalchemy.orm import sessionmaker
from app.core.config import settings

engine = create_async_engine(str(settings.DATABASE_URL), future=True, echo=False)
AsyncSessionLocal = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)

# Dependency
async def get_db():
    async with AsyncSessionLocal() as session:
        yield session

app/db/base.py

from sqlalchemy.orm import declarative_base
Base = declarative_base()

app/models/user.py

from sqlalchemy import Column, Integer, String, Boolean
from app.db.base import Base

class User(Base):
    __tablename__ = "users"
    id = Column(Integer, primary_key=True, index=True)
    email = Column(String(255), unique=True, index=True, nullable=False)
    full_name = Column(String(255), nullable=True)
    hashed_password = Column(String(255), nullable=False)
    is_active = Column(Boolean, default=True)

app/schemas/user.py

from pydantic import BaseModel, EmailStr
from typing import Optional

class UserCreate(BaseModel):
    email: EmailStr
    full_name: Optional[str]
    password: str

class UserRead(BaseModel):
    id: int
    email: EmailStr
    full_name: Optional[str]
    is_active: bool

    class Config:
        orm_mode = True

app/crud/user.py

from sqlalchemy.future import select
from sqlalchemy import insert
from sqlalchemy.exc import IntegrityError
from app.models.user import User
from app.core.security import hash_password

async def get_user_by_email(db, email: str):
    q = await db.execute(select(User).where(User.email == email))
    return q.scalars().first()

async def create_user(db, *, user_in):
    hashed = hash_password(user_in.password)
    db_user = User(email=user_in.email, full_name=user_in.full_name, hashed_password=hashed)
    db.add(db_user)
    try:
        await db.commit()
        await db.refresh(db_user)
        return db_user
    except IntegrityError:
        await db.rollback()
        raise

app/api/deps.py

from fastapi import Depends
from app.db.session import get_db

async def get_db_dep():
    async for s in get_db():
        yield s

app/api/v1/routers/users.py

from fastapi import APIRouter, Depends, HTTPException, status
from app.schemas.user import UserCreate, UserRead
from app.api.deps import get_db_dep
from app.crud.user import get_user_by_email, create_user

router = APIRouter(prefix="/users", tags=["users"])

@router.post("/", response_model=UserRead, status_code=status.HTTP_201_CREATED)
async def register(user_in: UserCreate, db=Depends(get_db_dep)):
    existing = await get_user_by_email(db, user_in.email)
    if existing:
        raise HTTPException(status_code=400, detail="User already exists")
    u = await create_user(db, user_in=user_in)
    return u

@router.get("/{user_id}", response_model=UserRead)
async def read_user(user_id: int, db=Depends(get_db_dep)):
    from sqlalchemy.future import select
    q = await db.execute(select(__import__("app.models.user", fromlist=["User"]).User).where(__import__("app.models.user", fromlist=["User"]).User.id == user_id))
    user = q.scalars().first()
    if not user:
        raise HTTPException(status_code=404, detail="Not found")
    return user

(note: read_user uses inline import to keep example short — in production import models at top.)

app/main.py

import uvicorn
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.core.config import settings
from app.core.logging import setup_logging
from app.api.v1.routers import users  # ensure package __init__ imports router path

def create_app():
    setup_logging()
    app = FastAPI(title=settings.APP_NAME, debug=settings.DEBUG)
    app.add_middleware(
        CORSMiddleware,
        allow_origins=["*"],  # restrict in prod
        allow_methods=["*"],
        allow_headers=["*"],
    )

    @app.get("/health", tags=["health"])
    async def health():
        return {"status": "ok"}

    app.include_router(users.router, prefix="/api/v1")
    return app

app = create_app()

if __name__ == "__main__":
    uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=False)

requirements.txt

fastapi==0.95.2
uvicorn[standard]==0.22.0
SQLAlchemy[asyncio]==1.4.52
asyncpg==0.27.0
pydantic==1.10.10
python-jose==3.3.0
passlib[bcrypt]==1.7.4
gunicorn==21.2.0
PyJWT==2.8.0

(Adjust versions as needed; pin to your policy.)

3. Dockerfile (production, multi-stage)

# Stage 1: build dependencies
FROM python:3.11-slim AS builder
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN apt-get update && \
    apt-get install -y --no-install-recommends build-essential libpq-dev gcc && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip wheel --no-cache-dir --wheel-dir /wheels -r requirements.txt

# Stage 2: final image
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

# create non-root user
RUN useradd -m appuser
WORKDIR /app

# copy wheels
COPY --from=builder /wheels /wheels
RUN pip install --no-cache /wheels/*

# copy app code
COPY . /app

# give non-root ownership
RUN chown -R appuser:appuser /app
USER appuser

EXPOSE 80

ENV PYTHONPATH=/app

# Use gunicorn + uvicorn workers
CMD ["gunicorn", "-k", "uvicorn.workers.UvicornWorker", "-w", "4", "app.main:app", "-b", "0.0.0.0:80", "--log-level", "info"]

Notes: the wheel stage avoids compiling at container runtime. Adjust worker count to available CPUs.

4. docker-compose.yml (local dev - postgres + app)

version: "3.8"
services:
  postgres:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  web:
    build: .
    restart: on-failure
    env_file:
      - .env
    ports:
      - "8000:80"
    depends_on:
      - postgres

volumes:
  pgdata:

5. .env.example

SECRET_KEY=supersecretreplace_me
DATABASE_URL=postgresql+asyncpg://postgres:password@postgres:5432/appdb
ENV=development
DEBUG=True
WORKERS=4

6. Build, test locally

# build
docker build -t myorg/fastapi-enterprise:latest .

# run with env
docker run --rm -it -p 8000:80 --env-file .env myorg/fastapi-enterprise:latest
# then hit http://localhost:8000/api/v1/users or http://localhost:8000/health

7. Deploy to AWS EC2 — step-by-step (recommended workflow)

We'll use Amazon ECR as image registry and a single EC2 instance that pulls & runs the container. Two approaches are shown: push to Docker Hub or to ECR. ECR is preferred for private images.

Important — key commands & notes:

Authenticate to ECR with aws ecr get-login-password | docker login --username AWS --password-stdin <account>.dkr.ecr.<region>.amazonaws.com. (AWS docs on get-login-password). ([AWS Documentation][1])
Install Docker on Amazon Linux 2: use sudo amazon-linux-extras install docker then sudo service docker start. (AWS docs). ([AWS Documentation][2])
Docker install on Ubuntu: follow Docker Engine install instructions from Docker docs. ([Docker Documentation][3])
Use Gunicorn with Uvicorn worker in production: gunicorn -k uvicorn.workers.UvicornWorker -w 4 app.main:app. ([Uvicorn][4], [FastAPI][5])

Below are the commands.

A — Prepare ECR & push image

(Assumes AWS CLI configured with credentials OR use an EC2 role that can create repos and push)

# 1. create repo (once)
aws ecr create-repository --repository-name fastapi-enterprise --region YOUR_REGION

# 2. build and tag
docker build -t fastapi-enterprise:latest .
AWS_ACCOUNT_ID=YOUR_AWS_ACCOUNT_ID
REGION=YOUR_REGION
ECR_URI=${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/fastapi-enterprise

docker tag fastapi-enterprise:latest ${ECR_URI}:latest

# 3. login to ECR (aws cli v2)
aws ecr get-login-password --region ${REGION} \
  | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com
# (source: AWS docs: pipe get-login-password to docker login). :contentReference[oaicite:4]{index=4}

# 4. push
docker push ${ECR_URI}:latest

B — Provision EC2 (quick guide)

Choose an instance (t3.small or larger). For production scale use autoscaling groups + ALB; below is a single-instance example.

From AWS Console or CLI create an EC2 instance (Amazon Linux 2 or Ubuntu). Give it an IAM instance profile with permission to pull from ECR (AmazonEC2ContainerRegistryReadOnly) — this avoids embedding keys.

Basic SSH in from your workstation:

ssh -i key.pem ec2-user@EC2_PUBLIC_IP

C — Install Docker on EC2 instance

If using Amazon Linux 2:

sudo yum update -y
sudo amazon-linux-extras install docker -y
sudo service docker start
sudo usermod -a -G docker ec2-user
# Log out and back in to apply group change or run docker with sudo

(Install instructions from AWS docs). ([AWS Documentation][2])

If using Ubuntu (example):

# follow Docker official steps - e.g., for ubuntu 22.04
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
  https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io
sudo usermod -aG docker $USER

(Use Docker docs for latest specific commands). ([Docker Documentation][3])

D — Pull & run image on EC2

If your instance has an IAM role that allows ECR read, you can pull directly. Otherwise authenticate using AWS CLI (or use the instance role recommended).

# login (if needed)
aws ecr get-login-password --region YOUR_REGION \
  | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com

# pull
docker pull ${ECR_URI}:latest

# run (env file transferred to instance beforehand)
docker stop fastapi || true && docker rm fastapi || true
docker run -d \
  --name fastapi \
  --restart unless-stopped \
  --env-file /home/ec2-user/app/.env \
  -p 80:80 \
  ${ECR_URI}:latest

E — Run as a systemd service (so it restarts on boot)

Create /etc/systemd/system/fastapi.service (run as root):

[Unit]
Description=FastAPI Container
After=docker.service
Requires=docker.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/docker stop fastapi
ExecStartPre=-/usr/bin/docker rm fastapi
# (Optional) pull latest image on start
ExecStartPre=/usr/bin/aws ecr get-login-password --region YOUR_REGION | /usr/bin/docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com
ExecStartPre=/usr/bin/docker pull ${ECR_URI}:latest
ExecStart=/usr/bin/docker run --rm --name fastapi --env-file /home/ec2-user/app/.env -p 80:80 ${ECR_URI}:latest
ExecStop=/usr/bin/docker stop fastapi

[Install]
WantedBy=multi-user.target

Then on the EC2 instance:

sudo systemctl daemon-reload
sudo systemctl enable fastapi
sudo systemctl start fastapi
sudo journalctl -u fastapi -f

8. Production best-practices & notes

Use an IAM instance profile (role) with least-privilege for ECR access instead of embedding AWS keys on the EC2 host.
In production use an ALB (Application Load Balancer) in front of EC2 instances, and an autoscaling group (ASG) for resilience.
Use separate environments (dev/staging/prod) and secure secrets with AWS Secrets Manager or Parameter Store rather than .env checked into source.
For database in production prefer RDS (managed Postgres) and grant network access via security groups.
Use health checks on the load balancer pointing to /health.
Set logging to stdout and ship logs to CloudWatch (or other log aggregator).
Use metrics & tracing (Prometheus + OpenTelemetry) for observability.
Set worker counts with a formula like workers = 2 * CPU_CORES + 1 (tune during load tests). ([Stack Overflow][6])
Use Gunicorn + uvicorn workers for multi-process handling of CPU-bound tasks (or use uvicorn directly with process-manager if preferred). ([Uvicorn][4], [FastAPI][5])

9. Quick checklist (deploy once)

Build & test locally with docker-compose up.
Create ECR repo and push image (see A).
Provision EC2 with IAM role (ECR read).
SSH to EC2, install Docker, configure env, pull image.
Create systemd unit to run container and allow auto-start.
Attach ALB & set security groups.

10. Useful references (sources)

ECR login (use aws ecr get-login-password | docker login ...). ([AWS Documentation][1], [AWS CLI Command Reference][7])
Install Docker on Amazon Linux 2 (amazon-linux-extras install docker). ([AWS Documentation][2])
Install Docker Engine on Ubuntu (official Docker docs). ([Docker Documentation][3])
Gunicorn + Uvicorn worker pattern for FastAPI deployment. ([Uvicorn][4], [FastAPI][5])
Worker-count guidance. ([Stack Overflow][6])

MongoDB/Oracle sources and MSSQL Managed Instance target

Goodluck Ekeoma Adiole — Thu, 07 Aug 2025 09:34:45 +0000

Based on the scripts, I'll create a detailed deployment plan for Azure Functions (v2 model) that addresses the MongoDB/Oracle sources and MSSQL Managed Instance target. The solution will include proper connection management, logging, and incremental sync logic.

Project Structure

oracle-to-mssql-sync/
├── .vscode/
│   └── settings.json
├── oracle/                     # Oracle Instant Client for Linux
│   ├── libclntsh.so
│   └── ... (other Oracle libs)
├── scripts/
│   ├── __init__.py
│   ├── accountstatementpremium.py
│   ├── agency.py
│   ├── agents.py
│   ├── allocatedpremiums.py
│   └── happilymaster.py
├── function_app.py
├── requirements.txt
├── host.json
├── local.settings.json
└── .gitignore

Step-by-Step Deployment Guide

1. Set Up Project Directory

mkdir oracle-to-mssql-sync
cd oracle-to-mssql-sync
mkdir .vscode scripts oracle

2. Install Oracle Instant Client

Download Basic Light Package from Oracle Instant Client
Unzip and place contents in oracle/ directory
Verify files: ls oracle should show libclntsh.so and other shared libraries

3. Create Environment File (local.settings.json)

{
  "IsEncrypted": false,
  "Values": {
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "ORACLE_CONN_STR": "oracle+oracledb://reporter:reporter@40.112.50.79:1521/HEIRSDB",
    "MONGODB_CONN_STR": "mongodb+srv://heirs-prod-retail-service:a0UQ3WLTD8J7sTng@production-middleware-d.3tprz.mongodb.net/",
    "MSSQL_HLA_CONN_STR": "DRIVER={ODBC Driver 18 for SQL Server};SERVER=hig-prod-sql-mi.public.ae890fe634a5.database.windows.net,3342;DATABASE=HLA;UID=heirsdata;PWD=P@ssw0rd;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;",
    "MSSQL_MONGODB_CONN_STR": "DRIVER={ODBC Driver 18 for SQL Server};SERVER=hig-prod-sql-mi.public.ae890fe634a5.database.windows.net,3342;DATABASE=MONGODB;UID=heirsdata;PWD=P@ssw0rd;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;",
    "LD_LIBRARY_PATH": "/home/site/wwwroot/oracle"
  }
}

4. requirements.txt

azure-functions==3.3.0
pandas==1.5.3
sqlalchemy==1.4.41
pyodbc==4.0.39
pymongo==4.3.3
cx_Oracle==8.3.0
python-dotenv==0.21.0

5. Refactored Scripts (common pattern)

scripts/init.py

import os
import logging
import cx_Oracle
from dotenv import load_dotenv

# Initialize environment
load_dotenv()

# Configure Oracle client
try:
    cx_Oracle.init_oracle_client(lib_dir=os.getenv("LD_LIBRARY_PATH"))
except Exception as e:
    logging.warning(f"Oracle client already initialized: {str(e)}")

# Common database functions
def get_mssql_engine(database_type="HLA"):
    conn_str = os.getenv(f"MSSQL_{database_type}_CONN_STR")
    quoted = urllib.parse.quote_plus(conn_str)
    return create_engine(f"mssql+pyodbc:///?odbc_connect={quoted}")

def get_max_value(engine, table, column):
    with engine.connect() as conn:
        result = conn.execute(text(f"SELECT MAX({column}) FROM {table}"))
        return result.scalar()

scripts/accountstatementpremium.py (example)

import pandas as pd
from . import get_mssql_engine, get_max_value
from pymongo import MongoClient
from sqlalchemy import text
import logging
import os

def sync():
    logging.info("Starting AccountStatementPremium sync")

    # Setup connections
    mongo_client = MongoClient(os.getenv("MONGODB_CONN_STR"))
    db = mongo_client['production-retail-db']
    collection = db['AccountStatementPremium']

    sql_engine = get_mssql_engine("MONGODB")

    # Get max date from target
    max_date = get_max_value(sql_engine, 'AccountStatementPremium1', 'date')

    # Fetch new data
    query = {"date": {"$gt": max_date}} if max_date else {}
    data = list(collection.find(query).sort("date", -1))

    if not data:
        logging.info("No new data found")
        return

    df = pd.DataFrame(data)
    # ... (rest of processing logic)

    # Batch insert
    batch_size = 10000
    for i in range(0, len(df), batch_size):
        batch = df.iloc[i:i+batch_size]
        batch.to_sql('AccountStatementPremium1', sql_engine, if_exists='append', index=False)
        logging.info(f"Inserted batch {i//batch_size + 1}/{(len(df)//batch_size)+1}")

    logging.info(f"Completed sync: {len(df)} records")

6. function_app.py

import azure.functions as func
import logging
from scripts import (
    accountstatementpremium,
    agency,
    agents,
    allocatedpremiums,
    happilymaster
)

app = func.FunctionApp()

@app.function_name(name="AccountStatementPremiumSync")
@app.schedule(schedule="*/2 * * * *", arg_name="timer", run_on_startup=True)
def account_sync(timer: func.TimerRequest):
    accountstatementpremium.sync()

@app.function_name(name="AgencySync")
@app.schedule(schedule="*/2 * * * *", arg_name="timer", run_on_startup=True)
def agency_sync(timer: func.TimerRequest):
    agency.sync()

@app.function_name(name="AgentsSync")
@app.schedule(schedule="*/2 * * * *", arg_name="timer", run_on_startup=True)
def agents_sync(timer: func.TimerRequest):
    agents.sync()

@app.function_name(name="AllocatedPremiumsSync")
@app.schedule(schedule="*/2 * * * *", arg_name="timer", run_on_startup=True)
def allocated_sync(timer: func.TimerRequest):
    allocatedpremiums.sync()

@app.function_name(name="HappylyMasterSync")
@app.schedule(schedule="*/2 * * * *", arg_name="timer", run_on_startup=True)
def happily_sync(timer: func.TimerRequest):
    happilymaster.sync()

7. host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[3.15.0, 4.0.0)"
  }
}

8. .vscode/settings.json

{
  "azureFunctions.deploySubpath": ".",
  "azureFunctions.projectLanguage": "Python",
  "azureFunctions.projectRuntime": "~3",
  "azureFunctions.scmDoBuildDuringDeployment": true,
  "azureFunctions.pythonVenv": ".venv",
  "azureFunctions.subscriptions": "<your-subscription-id>",
  "azureFunctions.resourceGroup": "sync-resources"
}

9. .gitignore

.venv
__pycache__
.vscode
local.settings.json
.env
*.log

Deployment to Azure

Create Azure Resources:

az group create --name sync-resources --location eastus
az storage account create --name syncstorageacc --location eastus --resource-group sync-resources --sku Standard_LRS
az functionapp create --resource-group sync-resources --consumption-plan-location eastus --runtime python --runtime-version 3.9 --functions-version 4 --name oracle-sync-app --storage-account syncstorageacc --os-type Linux

Set Environment Variables in Azure:

az functionapp config appsettings set --name oracle-sync-app --resource-group sync-resources --settings \
  "ORACLE_CONN_STR=oracle+oracledb://reporter:reporter@40.112.50.79:1521/HEIRSDB" \
  "MONGODB_CONN_STR=mongodb+srv://heirs-prod-retail-service:a0UQ3WLTD8J7sTng@production-middleware-d.3tprz.mongodb.net/" \
  "MSSQL_HLA_CONN_STR=DRIVER={...};SERVER=...;DATABASE=HLA;..." \
  "MSSQL_MONGODB_CONN_STR=DRIVER={...};SERVER=...;DATABASE=MONGODB;..." \
  "LD_LIBRARY_PATH=/home/site/wwwroot/oracle" \
  "WEBSITE_TIME_ZONE=Africa/Lagos"

Deploy from VS Code:
- Install Azure Functions Extension
- Press F1 → "Azure Functions: Deploy to Function App"
- Select your subscription and "oracle-sync-app"
- Confirm deployment

Post-Deployment Configuration

Enable Managed Identity for SQL:

In Azure Portal, go to SQL Managed Instance
Add function app's managed identity as login:

 CREATE USER [oracle-sync-app] FROM EXTERNAL PROVIDER
 ALTER ROLE db_datareader ADD MEMBER [oracle-sync-app]
 ALTER ROLE db_datawriter ADD MEMBER [oracle-sync-app]

Connection Security:
- Add Azure Function's outbound IPs to MongoDB Atlas IP whitelist
- Enable VNet integration for SQL Managed Instance connectivity
Monitoring:
- Configure Application Insights for logging
- Set up alerts for function timeouts (max duration: 10 minutes)

Key Improvements in Refactored Code

Secure Credential Management:
- All credentials moved to environment variables
- Azure Key Vault integration ready (use @Microsoft.KeyVault references)
Efficient Data Transfer:
- Batched inserts (10k records/batch)
- Incremental sync using MAX(date/id) checks
- Connection pooling with SQLAlchemy
Error Handling:
- Comprehensive logging to Application Insights
- Automatic retries built into Azure Functions
- Connection validation before data transfer
Oracle Client Management:
- Instant client included in deployment
- Proper Linux library path configuration
- Lazy initialization of connections

Troubleshooting Tips

Oracle Client Issues:

   # SSH into function app
   func azure functionapp fetch-app-settings oracle-sync-app
   func azure functionapp publish oracle-sync-app --build-native-deps

   # Check library path
   kudu console → ls -la oracle

Connection Problems:

Test connectivity from Kudu Console:

 python -c "import pyodbc; pyodbc.connect('DRIVER={ODBC Driver 18 for SQL Server};SERVER=...')"

Performance Optimization:
- Increase timeout in host.json:
```
 "functionTimeout": "00:10:00"
```

Use Premium plan for longer execution times

This implementation provides a production-ready solution that handles the hybrid data source scenario while following Azure Functions best practices and maintaining security compliance.

Deploying Python Scripts to Azure Functions as a Cron Job

Goodluck Ekeoma Adiole — Thu, 07 Aug 2025 06:35:36 +0000

Step-by-Step Guide: Deploying Python Scripts to Azure Functions as a Cron Job

Prerequisites

Azure Account (with an active subscription)
VS Code installed
Azure Functions Core Tools installed
Python 3.9+ installed
Azure Functions VS Code Extension (install from Extensions Marketplace)

Step 1: Set Up Project Structure

Create a root directory oracle-to-mssql-sync with this structure:

oracle-to-mssql-sync/
├── .vscode/                         # VS Code settings
│   └── settings.json                # Azure Functions runtime settings
├── functions/                       # Directory for all Azure Functions
│   ├── accountstatementpremium/     # Function 1
│   │   ├── __init__.py              # Your script content
│   │   └── function.json            # Trigger configuration
│   ├── agency/                      # Function 2
│   │   ├── __init__.py
│   │   └── function.json
│   ├── ...                          # Repeat for agents, allocatedpremiums, happilymaster
├── requirements.txt                 # Python dependencies
├── local.settings.json              # Local environment variables (NOT checked into Git)
└── host.json                        # Global Functions configuration

Step 2: Initialize Azure Functions Project

Open VS Code → Terminal → Run:

   mkdir oracle-to-mssql-sync
   cd oracle-to-mssql-sync
   func init --python

Create folders for each script:

   cd functions
   func new --name accountstatementpremium --template "timer trigger"
   func new --name agency --template "timer trigger"
   # Repeat for agents, allocatedpremiums, happilymaster

Step 3: Configure Timer Triggers (Cron Jobs)

Modify function.json for each function to run every 2 minutes:

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "name": "mytimer",
      "type": "timerTrigger",
      "direction": "in",
      "schedule": "0 */2 * * * *",  // Every 2 minutes
      "runOnStartup": false
    }
  ]
}

Step 4: Add Script Code

Copy your scripts into __init__.py for each function.

Example for accountstatementpremium/__init__.py:

import logging
import azure.functions as func
import cx_Oracle
import pyodbc

def main(mytimer: func.TimerRequest) -> None:
    # 1. Connect to Oracle
    oracle_conn = cx_Oracle.connect(os.environ["ORACLE_CONN_STRING"])
    oracle_cursor = oracle_conn.cursor()
    oracle_cursor.execute("SELECT * FROM your_table")

    # 2. Connect to MSSQL
    mssql_conn = pyodbc.connect(os.environ["MSSQL_CONN_STRING"])
    mssql_cursor = mssql_conn.cursor()

    # 3. Sync data (your existing logic)
    # ...

    logging.info("Sync completed at %s", utc_timestamp)

Step 5: Configure Dependencies

Populate requirements.txt:

azure-functions
cx_Oracle==8.3.0
pyodbc==4.0.39

Note: cx_Oracle requires Oracle Instant Client. Include it in your Azure Function App Settings (see Step 8).

Step 6: Local Environment Setup

Create local.settings.json (exclude from Git):

{
  "IsEncrypted": false,
  "Values": {
    "FUNCTIONS_WORKER_RUNTIME": "python",
    "ORACLE_CONN_STRING": "user/password@oracle-host:port/service",
    "MSSQL_CONN_STRING": "Driver={ODBC Driver 18 for SQL Server};Server=tcp:<server>.database.windows.net,1433;Database=<db>;Uid=<user>;Pwd=<password>;Encrypt=yes;TrustServerCertificate=no;"
  }
}

Step 7: Test Locally

Start the Functions runtime:

   func start

Verify logs show functions triggering every 2 minutes.

Step 8: Deploy to Azure

Sign in to Azure in VS Code (click the Azure icon → Sign in).
Create Function App:
- Azure Icon → Functions → Create New Function App
- Name: oracle-mssql-sync-app
- Runtime: Python 3.9
- OS: Linux
Deploy:
- Right-click your Function App → "Deploy to Function App"
- Select the oracle-to-mssql-sync folder.

Step 9: Configure Production Settings

In Azure Portal → Function App → Configuration → Application Settings: Add the same settings as local.settings.json:
- ORACLE_CONN_STRING
- MSSQL_CONN_STRING
Add Oracle Instant Client Path:

   Name  : LD_LIBRARY_PATH  
   Value : /home/site/wwwroot/oracle_instantclient

Install Oracle Instant Client (Kudu Console):

Open https://<app-name>.scm.azurewebsites.net
Run:

 mkdir oracle_instantclient
 cd oracle_instantclient
 wget https://download.oracle.com/otn_software/linux/instantclient/instantclient-basiclite-linuxx64.zip
 unzip instantclient-basiclite-linuxx64.zip

Step 10: Verify Deployment

In Azure Portal → Function App → Functions → Select a function → Monitor.
Check logs for sync activity every 2 minutes.

Troubleshooting Tips

Connection Issues:
- Use ping in Kudu Console to verify network access to Oracle/MSSQL.
- Enable Azure Application Insights for detailed logging.
Dependency Errors:
- Run pip install -r requirements.txt in Kudu Console.
Cron Not Triggering:
- Validate cron expression: 0 */2 * * * * = every 2 minutes.

Final Project Structure

oracle-to-mssql-sync/
├── .vscode/
│   └── settings.json                # { "azureFunctions.deploySubpath": "." }
├── functions/
│   ├── accountstatementpremium/
│   │   ├── __init__.py             # Your script from accountstatementpremium.py
│   │   └── function.json           # Timer config
│   ├── agency/
│   │   ├── __init__.py             # agency.py content
│   │   └── function.json
│   ├── ...                         # Other functions
├── host.json                       # { "version": "2.0" }
├── local.settings.json             # .gitignore this!
└── requirements.txt                # Dependencies

By following these steps, your Python scripts will run as scheduled Azure Functions, syncing data between Oracle and MSSQL every 2 minutes.

Deploying Multiple Python Scripts to Azure Functions as a Cron Job

Goodluck Ekeoma Adiole — Wed, 06 Aug 2025 15:38:04 +0000

Ultra-Detailed Step-by-Step Guide: Deploying 4 Python Scripts to Azure Functions as a Cron Job

For absolute beginners — no step skipped!

Before You Start: Prerequisites Checklist

Active Azure Account
- Create one free at azure.com
VS Code Installed
- Download from code.visualstudio.com
Install These VS Code Extensions
- Open VS Code → Extensions (Ctrl+Shift+X) → Search and install:
  - Azure Functions (by Microsoft)
  - Python (by Microsoft)
  - Azure Account (by Microsoft)
Python 3.8+ Installed
- Download from python.org
- Verify installation: Open terminal → Type python --version
Node.js (for Azure Tools)
- Download from nodejs.org
- Verify: Terminal → node --version

Step 1: Setup Project Folder in VS Code

Create a new folder on your computer (e.g., C:\db-sync-project)
Open VS Code → "File" → "Open Folder" → Select your new folder
Create 4 Python files in VS Code:
- Right-click in Explorer pane → "New File" → Create:
  - __init__.py (main function)
  - oracle_utils.py
  - mssql_utils.py
  - data_transformer.py
Copy your existing script code into these files

Step 2: Configure Azure Functions Project

Open Integrated Terminal
- VS Code → Terminal → New Terminal (Ctrl+Shift+`)
Install Azure Functions Tools: bash npm install -g azure-functions-core-tools@4
Create Virtual Environment: bash python -m venv .venv
- Activate Environment:
  - Windows: .venv\Scripts\activate
  - Mac/Linux: source .venv/bin/activate
Install Required Packages: bash pip install azure-functions cx_Oracle pyodbc python-dotenv
Generate Requirements File: bash pip freeze > requirements.txt

Step 3: Configure Timer Trigger

Create a new folder named SyncFunction in your project
Inside SyncFunction, create function.json with: json { "scriptFile": "../__init__.py", "bindings": [ { "name": "mytimer", "type": "timerTrigger", "direction": "in", "schedule": "*/2 * * * *" // Runs every 2 minutes } ] }
Edit __init__.py (main file): `python import azure.functions as func import logging from .oracle_utils import fetch_data from .data_transformer import transform_data from .mssql_utils import update_database

def main(mytimer: func.TimerRequest):
logging.info("Starting sync...")

   # Step 1: Get data from Oracle
   oracle_data = fetch_data()

   # Step 2: Transform data
   transformed_data = transform_data(oracle_data)

   # Step 3: Update MSSQL
   update_database(transformed_data)

   logging.info("Sync completed successfully!")

Step 4: Set Environment Variables

Create local.settings.json in your main project folder: json { "IsEncrypted": false, "Values": { "FUNCTIONS_WORKER_RUNTIME": "python", "AzureWebJobsStorage": "UseDevelopmentStorage=true", "ORACLE_CONN_STR": "your_oracle_connection_string", "MSSQL_CONN_STR": "your_mssql_connection_string" } }
Important! Add this file to .gitignore:
- Create .gitignore file → Add line: local.settings.json

Step 5: Configure Function Files

File 1: oracle_utils.py

`python
import cx_Oracle
import os
import logging

def fetch_data():
conn_str = os.environ["ORACLE_CONN_STR"]
try:
connection = cx_Oracle.connect(conn_str)
cursor = connection.cursor()
cursor.execute("SELECT * FROM your_table")
return cursor.fetchall()
except Exception as e:
logging.error(f"Oracle error: {str(e)}")
raise
finally:
cursor.close()
connection.close()
`

File 2: mssql_utils.py

`python
import pyodbc
import os
import logging

def update_database(data):
conn_str = os.environ["MSSQL_CONN_STR"]
try:
conn = pyodbc.connect(conn_str)
cursor = conn.cursor()
for row in data:
cursor.execute("UPDATE your_table SET col1=? WHERE id=?", (row[1], row[0]))
conn.commit()
except Exception as e:
logging.error(f"MSSQL error: {str(e)}")
raise
finally:
cursor.close()
conn.close()
`

File 3: data_transformer.py

python def transform_data(data): transformed = [] for row in data: # Add your transformation logic here transformed.append({ 'id': row[0], 'value': row[1].upper() # Example: Convert to uppercase }) return transformed

Step 6: Test Locally

Start storage emulator (for Windows):
- Install Azure Storage Emulator
- Run: AzureStorageEmulator.exe start
In VS Code terminal: bash func start
Verify output: plaintext Functions: SyncFunction: [timerTrigger] Invoked by schedule: */2 * * * *
Wait 2 minutes → Check terminal logs for sync status

Step 7: Deploy to Azure

Sign in to Azure in VS Code:
- Click Azure icon (left sidebar) → "Sign in to Azure..."
- Follow browser login prompts
Create Function App:
- In Azure sidebar → Click "Create Function App" icon (↑+☁️)
- Follow prompts:
  - Function App Name: db-sync-app-123 (must be globally unique)
  - Runtime: Python 3.10
  - Region: Select closest to you
  - Resource Group: Create new → db-sync-group
Deploy Code:
- Right-click your project folder in Explorer → "Deploy to Function App"
- Select your new Function App
- Wait for deployment (watch Output window)

Step 8: Configure Azure Settings

Go to Azure Portal
Find your Function App → "Configuration" → "Application settings"
Add new settings: | Name | Value | |---|---| | ORACLE_CONN_STR | Your Oracle connection string | | MSSQL_CONN_STR | Your MSSQL connection string | | AzureWebJobsStorage | (Auto-created, don't change) |
Click "Save"

Step 9: Verify Deployment

In Azure Portal → Function App → "Functions"
Select your SyncFunction → "Code + Test"
- Verify all 4 files appear in file list
Check logs:
- Go to "Monitor" → "Logs"
- Run query: kusto traces | order by timestamp desc
Wait 2 minutes → Check for sync logs

Step 10: Troubleshooting Checklist

✅ Deployment failed?

Run func azure functionapp publish <APP_NAME> --build remote in terminal
Check requirements.txt has all packages

✅ Script not running?

Verify timer schedule in function.json: "schedule": "*/2 * * * *"

✅ Database connection issues?

Allow Azure IPs in your database firewall:
- Oracle: Add Azure datacenter IP ranges
- MSSQL: In Azure SQL → Networking → Add firewall rule for Azure services

✅ Missing files?

All files must be in same directory as __init__.py
Redeploy entire project folder

Final Validation

Force-run the function:
- In Azure Portal → Function → "Code + Test" → "Test/Run"
- Click "Run"
Check "Logs" panel for output
Verify data sync in your MSSQL database

Congratulations! Your 4-script database sync is now running automatically every 2 minutes in Azure! 🎉

Pro Tip: For production, use Azure Key Vault for connection strings:

Create Key Vault in Azure

Store connection strings as secrets

In Function App → Configuration → Add setting: @Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/)

Deploying Oracle-to-MSSQL Sync Script to Azure Functions via VS Code.

Goodluck Ekeoma Adiole — Wed, 06 Aug 2025 09:53:16 +0000

Step-by-Step Guide: Deploying Oracle-to-MSSQL Sync Script to Azure Functions via VS Code

Detailed information on all steps.

Prerequisites

Azure Account (free tier suffices).
VS Code installed.
Python (3.8+ recommended).
Azure Functions Core Tools (for local testing).
Azure Functions VS Code Extension.
Python Extension for VS Code.

Step 1: Install Required Tools

Install Azure Functions Core Tools:

   npm install -g azure-functions-core-tools@4

(If you don’t have Node.js/npm, install Node.js).

Install VS Code Extensions:
- Open VS Code → Extensions (Ctrl+Shift+X) → Search and install:
  - Azure Functions
  - Python (by Microsoft)
  - Azure Account (for authentication)

Step 2: Set Up the Project Structure

Create a Project Folder:

   mkdir oracle-to-mssql-sync
   cd oracle-to-mssql-sync
   code .  # Opens VS Code in this folder

Create Virtual Environment (avoid dependency conflicts):
- In VS Code:
  - Open terminal (Terminal → New Terminal).
  - Run:
```
   python -m venv .venv
```

 - Activate the environment:  
   - **Windows**: `.venv\Scripts\activate`  
   - **Linux/macOS**: `source .venv/bin/activate`

Install Required Packages:

   pip install azure-functions pyodbc cx_Oracle

(cx_Oracle for Oracle, pyodbc for MSSQL)

Create Key Files:
- In VS Code, create:
  - __init__.py: Your main script (copy your existing script here).
  - requirements.txt: Dependencies (auto-generate with):
```
   pip freeze > requirements.txt
```

 - `local.settings.json`: For local environment variables.  
 - `function_app.py`: Azure Functions entry point (if not using `__init__.py`).

Step 3: Configure the Timer Trigger

Modify __init__.py to use a timer trigger:

   import azure.functions as func
   import datetime

   def main(mytimer: func.TimerRequest) -> None:
       utc_timestamp = datetime.datetime.utcnow().isoformat()
       if mytimer.past_due:
           print("Timer is running late!")
       # PASTE YOUR SCRIPT LOGIC HERE
       print("Oracle to MSSQL sync ran at:", utc_timestamp)

Add Timer Schedule:

Create a file named function.json in a new folder TimerTrigger (if auto-generated by Azure Tools, skip this).
Contents of function.json:

 {
   "scriptFile": "__init__.py",
   "bindings": [
     {
       "name": "mytimer",
       "type": "timerTrigger",
       "direction": "in",
       "schedule": "*/2 * * * *"  # Every 2 minutes
     }
   ]
 }

Step 4: Configure Environment Variables

For Local Testing (local.settings.json):

   {
     "IsEncrypted": false,
     "Values": {
       "FUNCTIONS_WORKER_RUNTIME": "python",
       "AzureWebJobsStorage": "UseDevelopmentStorage=true",  # For local storage
       "ORACLE_CONN_STR": "your-oracle-connection-string",
       "MSSQL_CONN_STR": "your-mssql-connection-string"
     }
   }

Replace placeholders with your actual DB connection strings.

Access Variables in Code:

   import os
   oracle_conn_str = os.environ["ORACLE_CONN_STR"]
   mssql_conn_str = os.environ["MSSQL_CONN_STR"]

Step 5: Test Locally

Start Azure Functions Emulator:
- In VS Code terminal:
```
 func start
```

Verify output:

 TimerTrigger: [GET,POST] http://localhost:7071/api/TimerTrigger

Check Logs:
- You’ll see cron-like executions every 2 minutes.

Step 6: Deploy to Azure Functions

Sign in to Azure in VS Code:
- Click the Azure logo in the sidebar → Sign in to Azure.
Create a Function App in Azure:
- In the Azure Functions sidebar:
  - Click Create Function App in Azure (↑ icon).
  - Follow prompts:
  - Function App Name: oracle-mssql-sync (globally unique).
  - Runtime: Python 3.9.
  - Region: Pick one near you.
Deploy Code:
- Right-click your Function App under Resources → Deploy to Function App.
- Select your project folder when prompted.
- Wait for deployment (check VS Code output terminal).

Step 7: Configure Azure Settings

Set Connection Strings in Azure Portal:
- Go to Azure Portal → Your Function App → Configuration.
- Under Application settings, add:
  - ORACLE_CONN_STR → Your Oracle connection string.
  - MSSQL_CONN_STR → Your MSSQL connection string.
- Save changes.
Verify Timer Schedule:
- In your Function App → Functions → select your timer function.
- Click Integration → Timer trigger → Confirm schedule is */2 * * * *.

Step 8: Test in Azure

Trigger Manually:
- In the Azure Portal → Function App → Functions → your function → Code + Test → Test/Run.
- Check Logs for output.
Monitor Executions:
- Go to Monitor → Logs → View real-time executions every 2 minutes.

Troubleshooting Tips

Deployment Failures:
- Check requirements.txt exists with correct dependencies.
- Run pip install -r requirements.txt locally to validate.
DB Connection Issues:
- Verify connection strings in Azure Application Settings.
- Allow Azure IPs in your Oracle/MSSQL firewall.
Timer Not Firing:
- Confirm cron expression */2 * * * * in function.json.
- Check timezone (Azure uses UTC by default).

Final Notes

Cost: Azure Functions has a generous free tier (1M executions/month).
Security: Use Azure Key Vault for sensitive connection strings in production.
Logs: Use Azure’s Application Insights for advanced monitoring.

By following these steps, your Oracle-to-MSSQL sync will run securely in Azure every 2 minutes! 🚀