Aisalkyn Aidarova

Posted on May 14

project #2: The Apex Platform

In a senior role at a major institution like Nexus Bank, you don't just "work in a team"—you operate within a Squad that is part of a larger Chapter or Platform Group.

Understanding this hierarchy is vital for an interview because it proves you’ve worked in an enterprise "at scale," not just a small startup.

1. Your Core Team: The "Platform Squad"

In a bank, the ideal team size follows the "Two-Pizza Rule" (usually 6 to 8 people). This ensures the team is small enough to be agile but large enough to handle high-stakes infrastructure.

The Squad Composition

1 Product Owner (PO): They don't write code. They manage the "Backlog" and decide which business features (like "Multi-Region DR") are the highest priority.
1 Scrum Master: They facilitate the Daily Stand-ups and remove "blockers" (e.g., if the Security team hasn't approved your PR yet).
1 Tech Lead / Architect: The most senior person who makes the final decision on high-level architecture.
3–4 Senior Cloud Platform Engineers (You): You are the engine. You write the Terraform, design the ECS tasks, and handle the "on-call" rotations.
1–2 Junior/Mid-level Engineers: You often mentor them, performing their code reviews and helping them understand Linux networking.

2. The Wider Organization: The Central Platform Group (CPG)

While your squad is 8 people, you are part of a Central Platform Group of about 30 to 50 engineers. At Nexus Bank, this group is usually split into specialized squads:

Compute Squad (Your Squad): Focused on ECS, EC2, and Scaling.
Data Platform Squad: Focused on RDS, DynamoDB, and Data Encryption.
Security & Identity Squad: Focused on IAM, Vault, and Compliance.
Observability Squad: Focused on the Prometheus/Grafana stack and logging.

3. How You "Manage" These Relationships

As a Senior Engineer, you don't "manage" people in terms of hiring/firing (that’s the Manager’s job), but you manage technical outcomes and stakeholders.

Managing Your Squad Peers

Code Reviews: You spend 1–2 hours a day reviewing Terraform code. You ensure no one is hardcoding secrets and that every resource has the correct banking tags (e.g., CostCenter, Environment).
Knowledge Sharing: You lead "Brown Bag" sessions where you teach the team a new trick in Linux or a better way to structure Terraform modules.

Managing Cross-Team Collaboration

The Network Team (The Gatekeepers): You don't "manage" them; you negotiate with them. When you need a new Transit Gateway attachment, you provide them with the technical CIDR ranges and architectural justification so they approve your request quickly.
The App-Dev Teams (The Customers): You treat the 500+ developers as your "customers." You manage this by creating Self-Service Templates.
Example: Instead of manually building an ECS service for every developer, you provide a "Golden Terraform Module" they can use. This reduces your workload and keeps the bank secure.

This is a classic "Senior Engineer" interview question: "Tell me about a time you had a conflict with another team."

In a bank, this conflict almost always involves the Security Team. They are paid to be paranoid, and you are paid to build a functional platform. At the 6-year level, you don't "fight" them—you negotiate with architecture.

The Conflict Scenario: "The Egress Deadlock"

The Situation

Your team is building the Apex Payment Gateway on ECS. To finish the sprint, the developers need the containers to reach out to a third-party Credit Scoring API and pull updated container images from a public registry.

The Conflict

Your Team’s Position: "We want to put a NAT Gateway in the VPC so our ECS tasks can reach the internet. It’s fast, standard, and we can finish the project by Friday."
The Security Team’s Position: "Absolutely not. Under PCI-DSS Compliance, no production resource handling credit card data can have a direct route to the internet. NAT Gateways are too 'open.' Request denied."

The Deadlock

The project stops. The developers are frustrated because they can’t test their code, and your manager is worried about the deadline.

How a Senior Engineer Manages the Resolution

A 6-year veteran doesn't just complain. They host a Technical Alignment Meeting and propose a "Middle Ground" architecture that satisfies both speed and safety.

1. The "Senior" Negotiation (The Meeting)

You invite the Security Architect to a 30-minute deep dive. Instead of asking for a NAT Gateway again, you say:

"I understand the risk of data exfiltration via a NAT Gateway. My goal is to ensure the Apex Gateway remains isolated while still allowing our services to function. What if we eliminate the NAT Gateway entirely and use a *'Deny-by-Default'** egress architecture?"*

2. The Technical Solution (The "Win-Win")

You propose and implement three layers of "Invisible" security using Terraform:

Layer 1: VPC Endpoints (PrivateLink): Instead of going over the internet to talk to AWS services (like S3 or Secrets Manager), you provision Interface Endpoints. Traffic never leaves the bank's network.
Layer 2: Centralized Egress Proxy: You suggest routing all remaining external traffic through a Squid Proxy or an AWS Network Firewall sitting in a separate "Inspection VPC."
Layer 3: FQDN Whitelisting: You tell Security: "We will only whitelist the specific domain of the Credit Scoring API. Everything else—literally the rest of the internet—will be hard-blocked at the firewall level."

3. The Narrative for the Interview

When the interviewer asks how you handle conflict, you use this story. It proves you understand Security, Networking, and Stakeholder Management.

The Response:

"During Project Apex, we hit a roadblock where the Security team blocked our deployment because our ECS tasks required internet access for third-party API calls. My team wanted a quick fix with a NAT Gateway, but Security flagged it as a PCI-DSS violation.
I took the lead on the negotiation. I realized that Security wasn't trying to be difficult—they were protecting the bank's license. I researched and proposed a **Zero-Trust Egress Architecture. I refactored our Terraform modules to use **AWS PrivateLink* for all internal AWS traffic and set up a centralized Network Firewall for whitelisted egress.*
By presenting a solution that addressed their specific fear (unauthorized data exfiltration) while still meeting our delivery date, I gained their approval. We actually ended up using this 'Locked-down VPC' as the **Golden Template* for all future projects at the bank."*

Why this "Invisible" part of the job is vital:

Empathy: You showed you understood the Security Team's job.
Authority: You used advanced AWS networking concepts (PrivateLink, FQDN filtering).
Leadership: You turned a "No" into a "Yes" without compromising the bank's safety.

Final Pro-Tip:

In an interview, never say "The Security team was wrong." Always say "We had different priorities, and my role was to find the architectural bridge between them."

As a Senior Cloud Platform Engineer, your value isn't just in the code you write, but in the mistakes you prevent others from making. When you review a Pull Request (PR) for Project Apex, you are looking for "smells"—small signs that the infrastructure might be insecure, expensive, or hard to recover.

Senior-Level Code Review Checklist specifically for a banking ECS/Terraform environment.

1. The Security & Identity Pillar

[ ] The "Wildcard" Hunt: Does any IAM policy contain Resource: "*" or Action: "s3:*"?
Senior Move: Reject it. Every role must be scoped to the specific bucket or API action needed.
[ ] Secret Leaks: Are there any hardcoded passwords, API keys, or database URIs?
Senior Move: Ensure they are using data "aws_secretsmanager_secret" or injecting them as environment variables via ECS Secret injection.
[ ] Encryption job-zero: Is encryption_at_rest enabled for every S3 bucket and RDS instance? Is a custom KMS Key used instead of the default AWS-managed key? (Banks require custom keys for better audit trails).

2. The Networking Pillar

[ ] Public IP Check: Are any ECS tasks or RDS instances assigned a public IP?
Senior Move: In a bank, nothing in a private subnet should have associate_public_ip_address = true.
[ ] Security Group "Laziness": Is there a rule allowing 0.0.0.0/0 on port 22 or 5432?
Senior Move: Only allow specific Security Group IDs (chaining). For example, the Database SG should only accept traffic from the ECS Task SG.
[ ] Missing VPC Endpoints: Is the code trying to reach S3 or ECR over the internet?
Senior Move: Ask the dev to add VPC Gateway Endpoints for S3 to keep traffic inside the AWS network.

3. The Reliability & Operations Pillar

[ ] The "Noisy Neighbor" Prevention: Are cpu and memory limits defined in the ECS Task Definition?
Senior Move: Without limits, one buggy container can starve the entire EC2 host.
[ ] Multi-AZ Deployment: Is the subnets variable for the ECS Service pulling from at least two Availability Zones?
Senior Move: Single-AZ deployments are a "fail" for Project Apex's 99.99% uptime goal.
[ ] Health Check Configuration: Is there a proper health_check defined for the Load Balancer? (e.g., /health instead of just checking if port 80 is open).

4. The "Invisible" Maintainability Pillar

[ ] Standardized Tagging: Are the mandatory bank tags present? (Owner, CostCenter, Environment, Project: Apex).
Senior Move: If it's not tagged, we can't track the $5,000 monthly bill.
[ ] Variable Validation: Does the code use Terraform validation blocks?
Example: If a variable is for environment, does it check that the input is only dev, uat, or prod?
[ ] The "Delete" Test: If I run terraform destroy on this module, will it accidentally delete the production database?
Senior Move: Ensure deletion_protection = true is set for all production RDS and S3 resources.

How to use this in an Interview

If an interviewer asks, "How do you ensure code quality in your team?", don't just say "we do code reviews." Say this:

"In Project Apex, I established a **Rigid Review Protocol. I personally focused on catching 'Architectural Drift.' For example, I'd look for any IAM roles that violated the **Principle of Least Privilege* or networking rules that lacked VPC Endpoints. I see code review as a mentorship opportunity; if I see a junior engineer hardcoding a subnet ID, I don't just fix it—I explain how using data sources makes our infrastructure region-agnostic and disaster-recovery ready."*

The "Invisible" Result:

By using this checklist, you aren't just a "Terraform guy." You are a Guardian of the Bank's Infrastructure. You prove that you think about cost, security, and the long-term health of the platform.

4. The "Invisible" Senior Skill: Managing Up

At 6 years of experience, you also manage your Product Owner’s expectations.

If a PO asks for a feature that is technically "dirty" or insecure to save time, a Senior Engineer says:

"I understand the deadline for the Payment API is Friday. However, skipping the VPC Endpoint setup creates a security risk that violates our PCI-DSS compliance. I suggest we deploy a 'Minimum Viable Environment' now and automate the full hardening by next sprint."

5. The Interview Narrative: Talking About Your Team

Interviewer: "Tell me about your team structure at the bank."

The Senior Response:

"I was part of a high-performing **Platform Squad of 7 people* within the Central Platform Group at Nexus Bank. My squad consisted of a Product Owner, a Scrum Master, and five engineers of varying seniority. We operated on 2-week sprints. While our squad's focus was the Apex ECS Platform, I frequently collaborated with the Security Chapter to ensure our IAM roles met the bank’s zero-trust standards. A big part of my daily role was also 'managing' our relationship with the Application Teams; I acted as a consultant to help them optimize their containerized workloads for our platform."*

Why this wins the interview:

Terminology: Using words like "Squad," "Chapter," and "Zero-Trust" shows you know the enterprise language.
Scale: It shows you understand how your small team fits into a 50-person group and a 500-person developer org.
Leadership: It highlights that you consult and negotiate, rather than just taking orders.

To move from a "strong candidate" to a "must-hire senior engineer," you need the Governance and Cultural layers. In a bank, the "invisible" work isn't just technical—it's about compliance, risk mitigation, and technical debt.

To make this project truly "interview-proof," "invisible" pillars is a curriculum.

1. The "Invisible" Governance: Change Management

In a bank, you never just run terraform apply from your laptop to Production. That is a firing offense.

The Workflow: you must describe a GitOps pipeline.
The Process: Code is pushed to a Feature Branch -> Pull Request (PR) is opened -> Automated Linting & Security Scanning (like tfsec or checkov) runs -> Peer Review by another Senior Engineer -> Approval -> Merge to Main -> Jenkins/GitHub Actions deploys to Staging -> CAB (Change Advisory Board) approval -> Deployment to Production.
Interview Tip: When asked about deployments, you should say: "We treat our infrastructure as code with a strict 4-eye principle. No change reaches the Apex environment without a peer-reviewed PR and a successful automated security scan."

2. The "Invisible" Safety Net: Disaster Recovery (DR)

A 6-year veteran knows that things will fail. They don't just build for uptime; they build for recovery.

The Technical Task: Add a "Cross-Region Backup" component to the project.
RDS Snapshots: Use Terraform to automate encrypted RDS snapshots being copied to a different AWS Region (e.g., from us-east-1 to us-west-2).
Route 53 Failover: Describe how they would use a Health Check to flip traffic to a "Maintenance Page" or a DR cluster if the main ECS service fails.
Interview Tip: If asked about reliability, say: "My priority for Project Apex was the **RTO (Recovery Time Objective). I architected the Terraform modules to be region-agnostic, allowing us to spin up the entire payment stack in a secondary region within 30 minutes if a provider outage occurs."

3. The "Invisible" Conflict: Technical Negotiation

Senior engineers spend a lot of time saying "No" in a way that helps the business.

The Scenario: A Developer wants "Admin" access to the ECS cluster to "debug faster."
The Senior Solution: Instead of giving Admin access, you provide them with CloudWatch Logs Insights and a Read-Only role.
Interview Tip: This demonstrates leadership. "I had a situation where the Dev team was frustrated by restrictive IAM roles. Instead of compromising the bank's security, I built a custom Grafana dashboard that gave them 100% visibility into their container logs and performance metrics, removing their need for direct cluster access."

The "Final Boss" Interview Scenario

The Question: "We noticed your Apex Project uses ECS on EC2. Why didn't you just use Fargate to reduce operational overhead?"

The "6-Year Veteran" Answer:

"That's a great question. While Fargate reduces the 'server management' burden, for a **Banking Payment Gateway, we chose **ECS on EC2* for three strategic reasons:

Compliance: We needed deep visibility into the OS level for our security agents and auditd logging to meet PCI-DSS requirements.

Performance: We required specific Linux kernel tuning (sysctl) to handle high-concurrency socket connections during peak transaction hours, which Fargate doesn't allow.

Cost at Scale: Given the predictable 24/7 load of a payment gateway, using Reserved Instances on EC2 saved the bank approximately 30% in cloud spend compared to Fargate's pricing model."*

Updated Final Checklist

To ensure you are "Cloud Platform Engineers," be able to produce:

[ ] A "Peer Review" Checklist: What do they look for when reviewing a teammate's Terraform code? (e.g., Are variables hardcoded? Is the state locked?).
[ ] A Security Posture Document: A 1-page PDF explaining how "Project Apex" protects customer data (Encryption at rest/transit, IAM roles, WAF).
[ ] A "Post-Mortem" Template: A document they filled out after their "Simulated Incident."

To truly give you the confidence of a 6-year veteran, you need to understand the "invisible" parts of the job: the complex environment and the constant communication required in a high-stakes banking setting.

Part 1: The Deep Environment (The "Nexus Bank" Ecosystem)

In a bank, you never work in a single AWS account. You work in a Multi-Account Landing Zone.

1. The Multi-Account Strategy

The project exists across four distinct AWS Accounts to ensure a small "blast radius":

Security Account: Centralized IAM, CloudTrail logs, and GuardDuty alerts.
Shared Services Account: Where the Prometheus/Grafana stack lives, along with the ECR (Elastic Container Registry) and CI/CD tools.
Non-Prod Account: For Dev and UAT (User Acceptance Testing) environments.
Production Account: The "Holy Grail." Access is extremely restricted (No manual changes allowed).

2. The Network Topology

Hybrid Cloud: The bank has "on-prem" data centers. You manage the AWS Direct Connect or Site-to-Site VPN that connects the ECS tasks to the bank's legacy mainframe databases.
Traffic Flow: External traffic hits an AWS WAF (Web Application Firewall), then a Public Application Load Balancer, which routes traffic through a Transit Gateway to the ECS Services sitting in private subnets.

3. Compliance & Governance

PCI-DSS: Every line of Terraform must be written with the Payment Card Industry Data Security Standard in mind (e.g., rotating secrets every 90 days, encrypting all S3 buckets).
Drift Detection: You use AWS Config to ensure that if someone manually changes a security group, you get an alert immediately.

Part 2: The Daily Collaboration (Who you meet and why)

A Senior Cloud Platform Engineer spends 40% of their time coding and 60% of their time aligning with other humans.

The Daily Rhythm

Time	Meeting / Activity	Who is there?	The Goal
09:30 AM	The Daily Stand-up	Platform Team, Scrum Master	Update on your Terraform PRs (Pull Requests). "I'm blocked on the firewall rules for the new RDS cluster."
11:00 AM	Security Review	Security Architect, Compliance Officer	Presenting your infrastructure design. You must prove that your ECS tasks are isolated and that logs are being shipped to the Security Account.
01:30 PM	App-Dev "Office Hours"	Software Engineers (Java/Node.js)	Helping developers who are struggling to get their containers to run in ECS. You explain how to optimize their `Dockerfile`.
03:00 PM	Sprint Refinement	Product Owner, SREs	Looking at the backlog. Deciding if the next priority is "Automated Scaling" or "Disaster Recovery Testing."
Ad-hoc	Incident War Room	Network Engineers, DBAs	(Only if something breaks) Troubleshooting why the latency between ECS and the on-prem database spiked.

The Key Stakeholders (The "Partners")

To sound like an expert, students must use these titles and understand these relationships:

1. The Security Engineer: Your "Best Friend/Worst Enemy." They will audit every IAM role you create. You collaborate with them to ensure the ECS Task Execution Role has the minimum permissions possible.
2. The Network Engineer: They manage the "pipes" between AWS and the Bank. You meet with them to request Route 53 changes or to open ports in the corporate firewall.
3. The Software Architect: They care about performance. You meet with them to discuss how Prometheus metrics can help them find bottlenecks in their Java code.
4. The Database Administrator (DBA): Since you are managing the RDS instances via Terraform, you collaborate with them on "Maintenance Windows" and "Storage Auto-scaling."

Part 3: "A Day in the Life" (The Interview Narrative)

If a student is asked, "What did you do yesterday?", a 6-year senior response sounds like this:

"Yesterday morning, I started by reviewing the **Grafana* dashboards for our Apex Payment Gateway; we noticed a slight increase in 5xx errors after the last deployment. I met with the Network Team to verify our Transit Gateway limits. After the Daily Stand-up, I spent the afternoon refactoring our Terraform modules to implement 'Blue/Green' deployments for our ECS services. This was a request from the Security Team to ensure we can roll back instantly if a compliance check fails. I finished the day by mentor-reviewing a junior engineer's Dockerfile to improve layer caching."*

Why this works:

It mentions the tools (Grafana, Transit Gateway, Terraform, ECS, Docker).
It mentions the people (Network Team, Security Team, Junior Engineer).
It shows the "Senior" mindset (Refactoring, Compliance, Mentoring, Monitoring).

This project is designed to simulate a high-level environment at a major financial institution. To represent six years of experience, the focus isn't just on "how to use a tool," but on architecture, security, governance, and cross-team collaboration.

The Persona: Senior Cloud Platform Engineer

In this project, you are a Senior Cloud Platform Engineer at Nexus Bank, a global retail and investment bank. You aren't just "doing DevOps"—you are building the foundation that allows 500+ developers to ship code safely.

Who you are: A technical leader who bridges the gap between software development and infrastructure. You treat infrastructure as a product.
Your Mission: To automate the lifecycle of the bank’s "Core Payment Gateway" to ensure 99.99% availability and strict regulatory compliance.

Company Profile: Nexus Bank

Sector: FinTech / Global Banking.
Infrastructure: 100% AWS (Multi-region for Disaster Recovery).
Engineering Scale: 25+ Product Teams (e.g., Credit Cards, Mortgages, Mobile App).
Platform Team: You are part of the Central Platform Group (CPG), consisting of 12 engineers divided into three sub-squads: Compute, Security/Identity, and Observability.

The Core Project: "Project Sentinel"

Objective: Architect and deploy a PCI-DSS compliant, multi-region payment processing platform using Infrastructure as Code (IaC).

1. The Technical Stack

Infrastructure: AWS (EKS, RDS, VPC, Transit Gateway, IAM, S3).
Provisioning: Terraform (using Terragrunt for multi-environment DRY code).
Operating System: Linux (Amazon Linux 2 / Ubuntu)—hardened for financial security.
Networking: Deep VPC peering, Private Link for third-party payment APIs, and AWS WAF.
Observability: Prometheus for metric collection and Grafana for executive and technical dashboards.

2. Your Responsibilities

Architecting Guardrails: Writing Terraform modules that prevent developers from creating "public" databases or unencrypted buckets.
Performance Tuning: Deep-diving into Linux kernel parameters and networking throughput to reduce payment latency.
Cost Optimization: Implementing "Scale-to-Zero" for non-prod environments to save cloud spend.
Security: Managing secrets via AWS Secrets Manager and ensuring encryption at rest/transit.

Day-to-Day Activities & Collaboration

A Typical Schedule

09:00 – 09:30: The Triage. Check Slack and PagerDuty. Review Grafana "Golden Signals" (Latency, Errors, Traffic, Saturation) for the Payment Gateway.
09:30 – 10:00: Daily Stand-up. Synchronize with your Platform squad. "Yesterday I finished the Terraform module for the new RDS cluster; today I'm troubleshooting a networking latency issue between the VPC and the On-prem data center."
10:00 – 12:30: Deep Work (The "Engineering"). Writing Terraform to deploy a new EKS cluster across two regions. Debugging a Linux networking issue where packets are dropping at the NAT Gateway.
13:30 – 14:30: Collaboration Meeting. Meet with the Security Team to review IAM policies and the App Dev Team to help them containerize a new Java-based microservice.
15:00 – 16:30: Observability Sprints. Configuring Prometheus Alertmanager to send high-priority alerts to the SRE team if payment success rates drop below 98%.

How You Solve Issues

When a "Severity 1" (Production Down) issue arises:

Detection: Grafana triggers an alert.
Isolation: You use Linux command-line tools (tcpdump, netstat, top) and AWS CloudWatch to see if it’s a network bottleneck or a code bug.
Collaboration: You open a "War Room" (Zoom/Teams) with the Database Admins and Network Engineers.
Resolution: You apply a fix via Terraform (never manually in the console!) to ensure the fix is permanent and documented.
Post-Mortem: You lead a meeting to discuss why it happened and how to automate the prevention of it.

Interview Simulation: Key Questions & Responses

Question	The "6-Year Experience" Answer
"How do you manage state in Terraform?"	"At Nexus Bank, we use S3 backends with DynamoDB for state locking. I've implemented a modular structure where 'State' is separated by environment and region to minimize the blast radius of any single change."
"Explain a complex Linux issue you solved."	"I once diagnosed a high 'iowait' issue on a production database node. By using `iostat` and `strace`, I found that a legacy logging script was saturating the disk I/O. I moved the logs to a separate EBS volume and adjusted the `sysctl` swappiness parameters."
"How do you handle security in AWS?"	"I follow the Principle of Least Privilege. I use IAM Roles for Service Accounts (IRSA) in EKS, ensuring that pods only have access to the specific S3 buckets or RDS instances they need, rather than using broad node-level permissions."

Project Implementation Steps

Network Setup: Use Terraform to create a multi-AZ VPC with private and public subnets.
Security Hardening: Provision an EC2 instance, log in via SSH, and harden the Linux OS (disable root login, setup fail2ban, optimize networking stack).
Infrastructure: Deploy an AWS EKS cluster using Terraform modules.
Monitoring: Install Prometheus and Grafana using Helm. Create a dashboard that visualizes CPU/Memory and custom application metrics.
The "Fix": Simulate a failure (e.g., shut down a subnet) and document how you would use your tools to find and fix it.

Expert Tip: In an interview, don't just talk about the tools. Talk about why you chose them. For a bank, the answer is always Security, Scalability, and Auditability.

Project Persona: Senior Cloud Platform Engineer (ECS Stack)

The Architecture: "Project Apex"

You are building a high-availability Payment Processing API. This isn't just a single container; it’s a distributed system with a focus on Security, Observability, and Network Isolation.

The Technical Stack

Orchestration: AWS ECS (using EC2 Launch Type for deep OS control).
Infrastructure: Terraform (Modularized for Multi-environment).
OS: Amazon Linux 2 (Heavily hardened).
Networking: VPC with Private Subnets, NAT Gateways, and Application Load Balancers (ALB).
Observability: Prometheus & Grafana (running as ECS Services).

Step-by-Step Implementation Plan

Phase 1: The Network Backbone (Terraform & Networking)

In a bank, you never put a database or an application server in a public subnet.

VPC Design: Use Terraform to create a VPC with 6 subnets (2 Public for ALBs, 2 Private for ECS Tasks, 2 Isolated for RDS).
Connectivity: Set up VPC Endpoints (Interface Endpoints) for ECS, ECR, and S3. This ensures your traffic stays within the AWS network and never touches the public internet—a major banking requirement.
Security Groups: Define strict "Chain of Trust" rules. Example: The Database SG only allows traffic from the ECS Task SG on port 5432.

Phase 2: The "Hardened" Host (Linux & Security)

Since you are using the EC2 launch type, you are responsible for the "Security of the Cloud."

Auto Scaling Group (ASG): Use Terraform to create an ASG that registers instances into your ECS Cluster.
Golden Image: Use User Data scripts in Terraform to harden the Linux OS on boot (e.g., updating packages, installing the CloudWatch agent, and setting sysctl parameters for networking optimization).
IAM Roles: Create an ECS Instance Role and a separate ECS Task Execution Role. This demonstrates knowledge of granular security.

Phase 3: Containerizing the Bank (ECS & Docker)

Task Definitions: Write JSON or Terraform-based Task Definitions.
Include Log Configuration (sending logs to CloudWatch).
Set Resource Limits (CPU/Memory) to prevent one container from crashing the whole EC2 host.
Service Discovery: Use AWS Cloud Map so your microservices can find each other via internal DNS (e.g., payment.service.local).
Secrets Management: Inject API keys and DB passwords directly from AWS Secrets Manager into environment variables securely.

Phase 4: The "Eyes" of the System (Prometheus & Grafana)

Sidecar Pattern: Deploy the AWS Distro for OpenTelemetry (ADOT) as a sidecar container in your ECS tasks to scrape metrics.
Prometheus Service: Run Prometheus as an ECS service with an EBS volume attached for data persistence.
Grafana Dashboards: Connect Grafana to Prometheus. Build a dashboard that monitors:
ECS Cluster Health: CPU/Memory Reservation vs. Utilization.
Networking: Active connections on the ALB.
Linux Stats: Disk I/O and Network TX/RX on the EC2 hosts.

Phase 5: Day-to-Day Operations & Collaboration

Activity	Senior Level Execution	Collaboration Partner
Capacity Planning	Analyzing Grafana trends to decide if the ASG needs to scale out before a big sale.	Finance/Product Team
Troubleshooting	Investigating "502 Bad Gateway" errors by checking ALB logs and using `tcpdump` on the EC2 host.	App Developers
Security Audit	Reviewing IAM policies to ensure no "wildcard" permissions exist.	Security/Compliance
Deployment	Updating the Terraform code to roll out a new version of the app using Blue/Green Deployment.	DevOps/Release Team

The "Interview Ready" Scenario: The Incident

The Problem: The "Mortgage Processing" service is running slow. Customers are complaining.

How the Senior Engineer Solves it:

Check Grafana: You notice "CPU Steal" is high on the EC2 hosts.
Linux Deep Dive: You SSH into the Bastion, then to the EC2 host. You run top and htop to see which process is hogging resources.
Discovery: You find a "noisy neighbor"—a non-critical logging container is consuming more CPU than allowed.
The Fix: You don't just kill the container. You go to Terraform, update the Task Definition to include strict cpu_shares, and run terraform apply.
The Report: You document how the resource limits prevented a total outage and suggest moving non-critical tasks to a separate ECS cluster.

Final Deliverable Checklist for Students

[ ] Terraform Code: Organized by modules/, environments/, and main.tf.
[ ] Network Diagram: Showing the flow from User -> ALB -> ECS Task -> RDS.
[ ] Monitoring Snapshot: A screenshot of their Grafana dashboard during a "Load Test."
[ ] CLI Competency: Ability to explain the difference between docker top and Linux top in the context of ECS.

This project covers every base: Infrastructure as Code (Terraform), Cloud Architecture (AWS), Deep Systems Knowledge (Linux/Networking), and Operational Excellence (Prometheus/Grafana).