DEV Community: Shuwen

Why We Do Indexing: Not Just for Speed, but for Cost and Hardware Reality

Shuwen — Tue, 30 Dec 2025 19:31:42 +0000

As software engineers, we are often asked a very common interview question:

Why do we use database indexing?

The standard answer is familiar to all of us:

To make queries faster
To reduce time complexity from O(n) to O(log n)
By using data structures such as B-trees or B+-trees
At the cost of extra storage and maintenance during insert, update, and delete operations

All of this is correct.

But this answer is incomplete.

There is a deeper reason behind indexing—one that becomes obvious only when you work with real production systems, cloud databases, and billing dashboards.

The Real Problem: Disk I/O, Not CPU

At the end of the day, databases store data on disk.

Even in modern systems with caching and memory layers:

Memory access happens in nanoseconds
Disk access happens in milliseconds

That difference is millions of times slower.

When a query does not use an index, the database often has no choice but to:

Scan a large number of rows
Read many data pages from disk
Perform repeated I/O operations

This is where performance collapses—not because of algorithms, but because of physical disk access.

Indexing Is an I/O Optimization

Indexes allow the database engine to:

Jump directly to relevant data pages
Read far fewer disk blocks
Avoid full table scans

This results in:

Fewer disk reads
Better cache utilization
Lower latency

In other words:

Indexing primarily reduces disk I/O, not just CPU work.

Speed is the symptom.
Reduced I/O is the cause.

Disk Has a Lifespan and a Cost

Every disk read:

Consumes I/O bandwidth
Adds wear to storage devices (especially SSD-based systems)
Competes with other workloads

In traditional on-prem systems, this means:

Hardware degradation
Capacity planning problems

In cloud systems, it means something very concrete:

Money.

Cloud Databases Charge for I/O

Managed databases such as Amazon Aurora charge based on:

Storage
Compute
I/O requests

Behind the scenes, every disk read is counted.

So when a poorly indexed query:

Scans millions of rows
Performs unnecessary disk reads
Runs repeatedly

You are not just slowing down the system—you are directly increasing your cloud bill.

A Realistic Example

Imagine a legacy system with:

Minimal or missing indexes
Heavy reporting queries
High read traffic

Suppose it generates:

100 billion I/O requests per month

At that scale, even a tiny per-request cost becomes massive.

Now introduce:

Proper indexing
Query optimization
Better access patterns

If indexing reduces disk reads by 80–90%, then:

Query latency drops
System stability improves
Cloud cost drops dramatically

This is not theoretical.
This happens all the time in real production systems.

Yes, Indexes Have a Cost — and That’s OK

Indexes do introduce trade-offs:

Extra storage
Slower writes
Maintenance overhead during INSERT, UPDATE, and DELETE

But in most production systems:

Reads vastly outnumber writes
Disk I/O is the dominant bottleneck
Cloud billing is tied to I/O

In these environments, indexing is almost always the right trade-off.

A Better Answer to “Why Do We Use Indexing?”

Instead of saying:

“Indexing makes queries faster.”

A more accurate answer is:

Indexing reduces disk I/O, which improves performance, extends hardware lifespan, and significantly lowers cloud costs.

Final Takeaway

Indexing exists because:

Disk access is slow
Disk access is expensive
Disk access is limited
Disk access is billed in the cloud

Speed is just the visible benefit.
Cost and hardware reality are the real reasons.

Credits & References

Concept inspiration and database fundamentals:
Berkeley CS186 – Introduction to Database Systems
University of California, Berkeley
https://cs186berkeley.net/

The Real Software Architecture Lives in the Mind

Shuwen — Tue, 30 Dec 2025 19:29:42 +0000

Why software problems are not code problems

Most software systems don’t fail. They survive.

They keep running, keep serving users, keep appearing “stable” — while quietly becoming more expensive, more fragile, and more feared every year.

Teams complain about legacy code, bad architecture, and messy systems, but rarely ask the harder question:

Why do we keep building software that works, yet nobody dares to change?

The answer is uncomfortable, because it has less to do with frameworks or patterns, and much more to do with how we think while building software in the first place.

When We Talk About Software Problems

When we talk about software problems, we usually talk about code.

Messy code
Legacy code
Bad architecture
Systems nobody dares to touch

We complain about:

projects that “work” but cost a lot to run
systems that are slow to change
services that survive for years without improving
teams that keep adding code but never make things better

These problems are everywhere.

And the question we rarely ask is not how to fix them, but:

Why do we keep creating systems like this in the first place?

A Real Story From a Real System (Story #1)

There was a system built more than ten years ago.

At the time:

the company owned its own data center
traffic was small
database access was cheap and unlimited
the team was small and under pressure to deliver fast

So the system grew naturally — but without structure.

To move faster:

logic went into one file
database queries were written freely
business rules, control flow, and data access mixed together

Over time:

that one file became over 8,000 lines
nested conditions were everywhere
class names only the original author understood

And still — it worked.

Years later, the company moved to AWS.

The database became Aurora.

Every query now had a price.

Nothing in the business logic changed.

But suddenly, the system started showing up on the monthly bill.

Hundreds of dollars.

Then thousands.

Everyone knew:

“We could reduce this cost a lot.”

But nobody touched the code.

It wasn’t because the team didn’t know SQL.

It wasn’t because they didn’t know optimization.

It was because the system was too risky to change.

The system survived — not because it was good,

but because it was hard to kill.

Another Real Story (Story #2)

I’ve also seen a different kind of system.

It looked “clean” on the surface.

There were:

controllers
services
repositories
interfaces

Each folder was organized.

Each layer had its place.

But when a new feature was requested, something strange happened.

Developers spent days:

tracing logic across files
jumping between layers
trying to understand where the real business rules lived

Nothing was obviously wrong.

But nothing was obviously clear either.

To add a feature, the easiest solution was always:

“Let’s just add another service file.”

Over time:

the number of files grew
duplication increased
behavior became implicit, not explicit

Again, the system worked.

Again, nobody dared to refactor deeply.

So we have to ask:

Why do systems that “work” so often become systems we fear?

The Easy Answers (And Why They Are Wrong)

People usually give simple explanations:

“They didn’t know design patterns.”
“They didn’t follow clean architecture.”
“They didn’t know SOLID principles.”
“They didn’t write tests.”

These answers are comfortable — and mostly wrong.

I’ve seen teams that:

knew SOLID
knew clean architecture
knew patterns
organized folders correctly

And still produced systems that were hard to change.

So the problem is not knowledge.

And it’s not a missing pattern.

The Real Root Cause: The Software Engineering Mind

At some point, I realized something important:

Architecture does not come first.
Design principles do not come first.
The engineering mind comes first.

Good or messy systems are not created by accident.

They are created by how people think while building them.

Two developers can face:

the same requirements
the same deadlines
the same pressure

One creates something that evolves.

The other creates something that freezes.

The difference is not skill.

It’s not experience.

It’s not tools.

It’s the mind behind the decisions.

Software Is Invisible — That Changes Everything

Unlike hardware, software is invisible.

You can’t:

touch it
weigh it
feel its quality immediately

A piece of software can:

run correctly
pass tests
serve users

And still:

cost too much to operate
require too many people to maintain
slow down every future change

The real cost appears late:

in cloud bills
in on-call pain
in slow delivery
in burned-out teams

Because the damage is invisible,

the engineering mind matters more than the code itself.

What Is the Software Engineering Mind? (In Practice)

This is not about being perfect.

It’s not about knowing all the answers.

From my experience, the software engineering mind shows up in a few very concrete ways.

1. Thinking Beyond “It Works”

A developer stops at:

“It works.”

An engineer keeps going:

How will this run at scale?
How will this change?
What will this cost — not just today, but later?

This is why some systems quietly become expensive,

while others stay manageable.

2. Treating Software as a Craft, Not a Task

With this mind:

shortcuts are taken consciously, not blindly
clarity is valued even when nobody asks for it
responsibility extends beyond the ticket

Like a craftsman:

you know shortcuts exist
but you also know what they cost

This mindset doesn’t slow teams down.

It prevents teams from getting stuck.

3. Respecting Invisibility

Engineers with this mind understand:

complexity hides
problems accumulate quietly
systems don’t fail immediately — they decay

So they design with humility.

They expect change.

They leave room for replacement.

This is why their systems are easier to refactor — or shut down.

4. Seeking Problems, Not Just Solving Tasks

Some people solve exactly what is asked.

Others ask:

“What will hurt later?”
“What are we afraid to touch?”
“Why is this part so fragile?”

Legacy systems are rarely created by one bad decision.

They are created when no one looks for problems early.

Rethinking “Good” and “Bad” Code

I don’t like labeling code as “good” or “bad”.

That’s too simple.

A better distinction is this:

Some code invites change
Some code resists change

Good systems are not immortal.

They are easy to replace.

Bad systems don’t survive because they are strong.

They survive because they cannot be changed safely.

This is the paradox:

The hardest systems to kill are often the worst ones.

From Developer to Engineer

Becoming a great engineer is not about memorizing patterns.

It’s not about copying architectures from books.

It’s about developing a mind that:

sees beyond today’s task
respects long-term cost
treats software as a living system
owns consequences, not just output

With this mind:

even imperfect designs improve over time
even legacy systems become better
even unknown problems find solutions

Without it:

no framework saves you
no architecture survives
no amount of knowledge prevents decay

Final Thought

Good software does not come from knowing the right answers.

It comes from having the mind that knows:

how to ask better questions
how to find solutions when none are obvious
how to build things that can change — or disappear — when needed

That is the difference between writing code

and practicing software engineering.

And that mind is what truly separates

a good developer from a great engineer.

How to Learn as a Software Engineer: Lessons from The Pragmatic Programmer

Shuwen — Sun, 21 Dec 2025 20:53:03 +0000

I’ve known about The Pragmatic Programmer for a long time, but I only read it recently. To be honest, it truly deserves its reputation as a classic. Even though the book was written years ago, many of its ideas still feel surprisingly relevant today—especially the parts about how software engineers learn and grow. Below is my own summary and reflection on what I learned from the book, with a focus on the learning mindset rather than specific languages or tools.

1. Treat Learning as a Core Professional Skill

The book’s central idea is simple but strict:

Your knowledge has a half-life.

If you stop learning, your value decays.

Software engineering isn’t about mastering one stack. It’s about continuously adapting as tools, paradigms, and expectations change.

Key mindset:

Learning is not optional
Learning is part of the job, not a hobby
Curiosity is a professional responsibility

2. Build a Personal Knowledge Portfolio

Just like financial investments, your skills need diversification.

The book recommends:

Don’t over-specialize too early
Avoid being “the X-only engineer”
Regularly invest in new areas

Examples of diversification:

One new programming language per year
Different paradigms (OO, FP, procedural, event-driven)
Domains outside your daily work (databases, networking, security, UX)

A good engineer is T-shaped:

deep in one area, broad across many.

3. Learn by Doing, Not by Collecting Information

Reading alone doesn’t count as learning.

The book emphasizes:

Experimentation
Prototyping
Hands-on practice

Instead of:

“I read about X”

Prefer:

“I built a small thing with X”

Concrete advice:

Write throwaway prototypes
Build toy versions of real systems
Re-implement small tools to understand them

This aligns strongly with your own habit of:

Building tracing demos
Refactoring real production services
Writing tests to understand behavior

You’re already practicing this philosophy well.

4. Learn How to Learn (Meta-Learning)

One of the most important points in the book:

Technology changes, learning skills don’t.

Good engineers:

Know how to break down unfamiliar systems
Can quickly form mental models
Ask better questions over time

Techniques encouraged:

Read source code
Debug other people’s systems
Trace data and control flow
Use logs, traces, and experiments as learning tools

This is exactly why observability (logs, metrics, tracing) is framed as a learning amplifier, not just a debugging tool.

5. Read Widely — Not Just Code

The book strongly discourages tunnel vision.

Recommended learning sources:

Code (yours and others’)
Technical books
Design papers
Postmortems
Non-technical books (psychology, systems thinking, communication)

Why?

Because software engineering is human systems + technical systems.

Your interest in:

Architecture
Leadership
Clean design
Organizational impact

fits perfectly with this advice.

6. Learn Continuously in Small, Regular Batches

Don’t wait for:

“Free time”
A new job
A crisis

Instead:

Learn a little every day
20–30 minutes consistently beats cramming

This mirrors the book’s preference for:

Sustainable habits
Incremental growth
Long-term thinking

7. Teach What You Learn

A subtle but powerful idea:

Teaching is a forcing function for understanding.

The book encourages:

Writing
Blogging
Explaining concepts to others
Reviewing code thoughtfully

Your habit of:

Writing Medium articles
Creating structured notes
Turning experience into documentation

is exactly what the book would recommend.

8. Be Humble, but Confident

Finally, the learning mindset is emotional as much as technical.

The book encourages engineers to:

Admit when they don’t know
Be comfortable being a beginner again
Avoid ego-driven attachment to tools

Growth comes from:

Curiosity > pride
Adaptability > certainty

One-Sentence Summary

A great software engineer grows by continuously learning, experimenting, teaching, and adapting—treating knowledge as a living system, not a fixed achievement.

Learning as a Software Engineer Is Not About Reading — It’s About Applying

Shuwen — Sun, 21 Dec 2025 20:50:07 +0000

I’ve read many software engineering books over the years, but one idea keeps repeating itself in different forms:

Learning only matters when it changes how you work.

This idea appears clearly in :contentReference[oaicite:0]{index=0}, and it shows up again—from a different angle—in :contentReference[oaicite:1]{index=1}.

Recently, while working on a real production system, I realized I wasn’t just reading these ideas anymore—I was actively applying them.

From Reading to Doing: What Changed in My Daily Work

Over the past months, I’ve been refactoring and evolving a system that spans multiple microservices. Instead of adding features blindly, I started applying what I had learned:

Refactoring services using Clean / Hexagonal Architecture
Separating domain logic from infrastructure
Introducing distributed tracing to make invisible dependencies visible
Designing around use cases, not frameworks
Treating observability as a learning tool, not just a debugging aid

None of these came from copying code snippets from a book.

They came from:

Reading
Reflecting
Applying
Observing results
Adjusting again

This is exactly what The Pragmatic Programmer emphasizes:

knowledge has value only when it is exercised.

Clean Architecture Was Not the Goal — Understanding Was

When I first read about Clean Architecture, it sounded elegant. But elegance alone doesn’t survive contact with production systems.

The real value appeared only when I applied it to:

Reduce coupling between services
Make refactoring safer
Improve testability
Clarify ownership of business rules

At that point, architecture stopped being a diagram.

It became a tool for thinking.

That shift—from concept to practice—is learning.

Tracing: Learning the System by Observing It

Another turning point was introducing distributed tracing.

Before tracing:

Debugging meant guessing
Logs were fragmented
Cross-service failures felt like finding a needle in a haystack

After tracing:

Service dependencies became visible
Execution paths were explicit
Failures told a story

Tracing didn’t just help debugging—it taught me how the system actually behaves, not how I assumed it behaved.

Observability became a way to learn the system, not just fix it.

Broad Knowledge Matters More Than We Think

This brings me to a personal experience that changed how I think about seniority.

The Plumber Story

Several months ago, we had a water-dripping issue in our house.

We hired a plumber.

He drilled holes, checked pipes, and looked everywhere inside the house—but found nothing. No solution.

Later, on a rainy night, we noticed water dripping along the bricks. The real issue was outside:

Rusted nails on the roof
Water entering from above
Gravity doing the rest

The plumber wasn’t incompetent.

He was too narrow in perspective.

An experienced plumber might have asked:

“Does this only happen when it rains?”
“Could the source be outside?”
“Let’s check the roof first.”

That moment immediately reminded me of Staff Engineer.

The Staff Engineer Mindset: Depth + Breadth

Staff Engineer makes a simple but powerful point:

A staff engineer is deep in one or two areas,

but broad enough to see the whole system.

That’s the difference between:

Fixing symptoms
Understanding systems

In software, this means:

Deep expertise in your core domain
Broad understanding of:
- Infrastructure
- Networking
- Observability
- Data flow
- Human and organizational workflows

Without breadth, we drill holes in the wrong place.

Continuous Learning Is About Expanding Vision

Looking back, I realized something important:

I didn’t become better by reading more.

I became better by connecting what I read to real problems.

Architecture books helped me refactor safely
Observability taught me how systems actually fail
Broader technical exposure helped me ask better questions

This is continuous learning:

Not chasing trends
Not memorizing tools
But expanding how you see problems

Final Thought

A strong software engineer is not defined by:

How many books they read
How many frameworks they know

But by:

How they adapt what they learn
How they apply it to real systems
How wide their perspective becomes over time

Just like an experienced plumber,

the best engineers don’t just know where to drill.

They know where not to.

Clean Architecture Design Flow: A Practical Guide to Diagrams That Actually Help

Shuwen — Sat, 20 Dec 2025 20:06:01 +0000

When I applied Clean Architecture in a new project, I treated it as an engineering process, not a refactor-after-the-fact exercise.

Instead of starting with frameworks or package structures, I began by explicitly identifying the system’s use cases and modeling their workflows at a high level. These early diagrams helped me reason about execution flow, failure paths, and—most importantly—where architectural boundaries should live.

Only after those boundaries were clear did I open IntelliJ and start coding.

This allowed architecture to guide implementation, instead of being constantly corrected by it.

What I Did (Concrete Steps)

Identified and named core application use cases (business workflows)
Drew use case and sequence diagrams to understand execution flow
Used diagrams to identify core vs. infrastructure boundaries
Created the project skeleton in IntelliJ based on those boundaries
Defined interfaces (ports) in the core
Implemented adapters (REST, persistence, messaging) against ports
Updated high-level diagrams and shared them with the team

When to Draw Diagrams, What to Draw, and Why

If you’ve ever felt overwhelmed by UML diagrams that look impressive but don’t help you build better software, this guide is for you.

This is a lightweight, practical design flow for projects using Clean Architecture (Hexagonal Architecture). The goal is to make system intent explicit, reduce cognitive load, and keep diagrams useful—not decorative.

The Guiding Principle

Diagrams are thinking tools, not documentation artifacts.
Draw them to make decisions.
Stop drawing when the code becomes clearer than the diagram.

Stage 0 — Problem Framing (No Diagrams Yet)

Goal

Understand why the system exists.

What You Do

Write a short paragraph answering:

Who uses the system?
What problem does it solve?
What outcomes matter?

Example

This system handles order checkout, payment retries, and order expiration across REST APIs, Kafka retries, and scheduled jobs.

📌 No UML yet.
If this is unclear, diagrams won’t help.

Stage 1 — Identify Use Cases (MANDATORY)

Goal

Make system intent explicit.

What You Produce

A list of use cases (verbs, not nouns)
Optionally, a use case diagram

Input

Business requirements
Product discussions
Real workflows

Output

Clear, named use cases.

Example Use Case List

Checkout Order
Retry Payment
Expire Unpaid Orders

Use Case Diagram

┌──────────┐
│   User   │ ────────> Checkout Order
└──────────┘

┌──────────┐
│  Kafka   │ ────────> Retry Payment
└──────────┘

┌──────────┐
│Scheduler │ ────────> Expire Unpaid Orders
└──────────┘

Rules

No entities
No methods
No databases
Only who triggers what

Stage 2 — Core Workflow (Sequence Diagram)

Goal

Understand flow and boundaries, not implementation.

When to Draw

A use case touches multiple systems
Failure handling matters

Input

One use case
Happy path (+ one failure path if needed)

Output

A simple sequence diagram.

Sequence Diagram: Checkout Order

Controller    CheckoutUseCase    InventoryPort    PaymentPort    OrderRepo    EventBus
    |                 |                |              |             |           |
    |-- execute(cmd)->|                |              |             |           |
    |                 |-- reserve() -->|              |             |           |
    |                 |<-- reserved ---|              |             |           |
    |                 |-- charge() ------------------>|             |           |
    |                 |<-- charged -------------------|             |           |
    |                 |-- save() ---------------------------------->|           |
    |                 |-- publish() ------------------------------------------>|
    |<-- result ------|                |              |             |           |

Rules

Show ports, not implementations
No frameworks
No DTOs
Stop once decisions are clear

Stage 3 — Domain Modeling (Optional but Powerful)

Goal

Define business language and invariants.

When to Draw

Complex domain
Multiple developers
State transitions matter

Domain Model Diagram

┌─────────────────────────────┐
│           Order             │
├─────────────────────────────┤
│ id                          │
│ status                      │
│ total                       │
├─────────────────────────────┤
│ markPaid()                  │
│ expire()                    │
└─────────────────────────────┘
              |
              | 1..*
              ▼
┌─────────────────────────────┐
│        OrderItem            │
├─────────────────────────────┤
│ productId                  │
│ quantity                   │
│ price                      │
└─────────────────────────────┘

Rules

No repositories
No controllers
No annotations
Only business meaning

Stage 4 — Architecture Skeleton (Folder Tree)

Purpose

Turn clear intent into enforceable structure before real code.

If someone opens your repo and only reads folder names,
they should understand what the system does.

Top-Level Boundaries

domain/
application/
adapter/
config/

One Folder Per Use Case

application/usecase/
├── checkout/
├── retry/
└── expire/

Use Case Placeholders

application/usecase/checkout/
├── CheckoutOrderUseCase.java
├── CheckoutOrderCommand.java
└── CheckoutOrderResult.java

Rules

Minimal logic
No frameworks
No infrastructure imports

Ports (Interfaces Only)

application/port/out/
├── OrderRepositoryPort.java
├── PaymentPort.java
├── InventoryPort.java
└── EventPublisherPort.java

Adapter Placeholders

adapter/
├── in/
└── out/

Do not implement yet—this enforces direction.

orders-service/
├── README.md
├── docs/
│   ├── architecture/
│   │   ├── use-cases.md
│   │   ├── sequence-checkout-order.md
│   │   └── domain-model.md
│   └── decisions/
│       └── ADR-001-use-case-driven-design.md
│
├── src/main/java/com/example/orders/
│
│   ├── domain/
│   │   ├── model/
│   │   │   ├── Order.java
│   │   │   ├── OrderItem.java
│   │   │   ├── OrderStatus.java
│   │   │   ├── Money.java
│   │   │   └── Discount.java
│   │   │
│   │   ├── rule/
│   │   │   ├── PricingRule.java
│   │   │   └── OrderInvariant.java
│   │   │
│   │   └── event/
│   │       ├── OrderPaidEvent.java
│   │       └── OrderExpiredEvent.java
│
│   ├── application/
│   │   ├── usecase/
│   │   │   ├── checkout/
│   │   │   │   ├── CheckoutOrderUseCase.java
│   │   │   │   ├── CheckoutOrderCommand.java
│   │   │   │   ├── CheckoutOrderResult.java
│   │   │   │   └── CheckoutOrderService.java
│   │   │   │
│   │   │   ├── retry/
│   │   │   │   ├── RetryPaymentUseCase.java
│   │   │   │   ├── RetryPaymentCommand.java
│   │   │   │   └── RetryPaymentService.java
│   │   │   │
│   │   │   └── expire/
│   │   │       ├── ExpireUnpaidOrdersUseCase.java
│   │   │       └── ExpireUnpaidOrdersService.java
│   │   │
│   │   └── port/
│   │       ├── out/
│   │       │   ├── OrderRepositoryPort.java
│   │       │   ├── PaymentPort.java
│   │       │   ├── InventoryPort.java
│   │       │   ├── FraudCheckPort.java
│   │       │   ├── EventPublisherPort.java
│   │       │   ├── IdempotencyPort.java
│   │       │   └── ClockPort.java
│   │       │
│   │       └── in/
│   │           └── (optional - often implicit via usecase)
│
│   ├── adapter/
│   │   ├── in/
│   │   │   ├── web/
│   │   │   │   ├── OrderController.java
│   │   │   │   └── OrderRequestMapper.java
│   │   │   │
│   │   │   ├── messaging/
│   │   │   │   ├── CheckoutRetryConsumer.java
│   │   │   │   └── RetryMessageMapper.java
│   │   │   │
│   │   │   └── scheduler/
│   │   │       └── ExpireUnpaidOrdersJob.java
│   │   │
│   │   └── out/
│   │       ├── persistence/
│   │       │   ├── JpaOrderRepositoryAdapter.java
│   │       │   ├── JpaOrderEntity.java
│   │       │   └── SpringDataOrderRepository.java
│   │       │
│   │       ├── payment/
│   │       │   ├── StripePaymentAdapter.java
│   │       │   └── PaymentMapper.java
│   │       │
│   │       ├── inventory/
│   │       │   └── InventoryHttpAdapter.java
│   │       │
│   │       ├── fraud/
│   │       │   └── FraudApiAdapter.java
│   │       │
│   │       ├── messaging/
│   │       │   └── KafkaEventPublisherAdapter.java
│   │       │
│   │       └── idempotency/
│   │           └── RedisIdempotencyAdapter.java
│
│   └── config/
│       ├── UseCaseConfig.java
│       ├── AdapterConfig.java
│       └── ApplicationConfig.java
│
└── src/test/java/com/example/orders/
    ├── domain/
    │   └── OrderTest.java
    │
    ├── application/
    │   ├── checkout/
    │   │   └── CheckoutOrderServiceTest.java
    │   └── retry/
    │       └── RetryPaymentServiceTest.java
    │
    └── adapter/
        ├── in/
        │   └── web/
        │       └── OrderControllerTest.java
        └── out/
            └── persistence/
                └── JpaOrderRepositoryAdapterTest.java

Stage 5 — Identify Ports

What Qualifies as a Port?

Anything that:

Touches a DB
Touches the network
Touches time
Sends messages

interface PaymentPort {
    PaymentResult charge(Money amount);
}

📌 Ports emerge naturally.
If you’re forcing them, you started too early.

Stage 6 — Implement Adapters (No Diagrams)

What You Do

REST controllers
Kafka consumers
JPA repositories
External API clients

Rule

Adapters depend on the core — never the other way around.

📌 No UML here.

Stage 7 — Post-Implementation Diagrams (Optional)

Purpose

Onboarding and communication.

Draw:

One hexagonal overview
One key sequence diagram

Design Flow Cheat Sheet

Stage	What to Draw	Purpose
0	Nothing	Clarify intent
1	Use cases	Make intent explicit
2	Sequence diagram	Find boundaries & flow
3	Domain model (optional)	Shared language
4	Folder structure	Lock architecture
5	Interfaces (ports)	Define dependencies
6	Nothing	Implement adapters
7	Overview diagrams	Onboarding

Final Rule to Remember

If a new developer understands the system by reading use case names,
your architecture is working.

If they must jump across controllers, services, and repositories to understand intent—it’s not.

Key Takeaways

Draw diagrams to make decisions, not to look professional
Start with use cases — they reveal system intent
Sequence diagrams expose boundaries early
Domain models create shared language
Stop diagramming when code becomes clearer
Ports emerge naturally from workflows
Post-implementation diagrams are for onboarding, not design

Clean Architecture isn’t about rigid rules.
It’s about clarity of intent, clear boundaries, and maintainable code.

Start simple.
Draw only what helps you think.
Let architecture emerge from real use cases, not theoretical purity.

Grant RBAC permission for IAM principle (allow it to view eks)

Shuwen — Wed, 05 Apr 2023 13:00:51 +0000

Grant RBAC permission for IAM principle (allow it to view eks)

Step One

method 1: aws cli

create policy to include the necessary permissions for a principal to view Kubernetes resources for all clusters in your account. replace the following 111122223333 with your aws account id

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "eks:ListFargateProfiles",
                "eks:DescribeNodegroup",
                "eks:ListNodegroups",
                "eks:ListUpdates",
                "eks:AccessKubernetesApi",
                "eks:ListAddons",
                "eks:DescribeCluster",
                "eks:DescribeAddonVersions",
                "eks:ListClusters",
                "eks:ListIdentityProviderConfigs",
                "iam:ListRoles"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "ssm:GetParameter",
            "Resource": "arn:aws:ssm:*:111122223333:parameter/*"
        }
    ]
}

create EKS connector IAM role with its policy. AmazonEKSConnectorAgentRole:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ssm.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

AmazonEKSConnectorAgentPolicy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "SsmControlChannel",
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateControlChannel"
            ],
            "Resource": "arn:aws:eks:*:*:cluster/*"
        },
        {
            "Sid": "ssmDataplaneOperations",
            "Effect": "Allow",
            "Action": [
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenDataChannel",
                "ssmmessages:OpenControlChannel"
            ],
            "Resource": "*"
        }
    ]
}

Create the Amazon EKS Connector agent role using the trust policy and policy you created in the previous list items.

aws iam create-role \
     --role-name AmazonEKSConnectorAgentRole \
     --assume-role-policy-document file://eks-connector-agent-trust-policy.json

Attach the policy to your Amazon EKS Connector agent role.

aws iam put-role-policy \
     --role-name AmazonEKSConnectorAgentRole \
     --policy-name AmazonEKSConnectorAgentPolicy \
     --policy-document file://eks-connector-agent-policy.json

method 2: terraform

//https://docs.aws.amazon.com/eks/latest/userguide/view-kubernetes-resources.html#view-kubernetes-resources-permissions
//create EKSViewResourcesPolicy
resource "aws_iam_policy" "eks_view_resources_policy" {
  name        = "EKSViewResourcesPolicy"
  description = "Policy to allow a principal to view Kubernetes resources for all clusters in the account"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "eks:ListFargateProfiles",
          "eks:DescribeNodegroup",
          "eks:ListNodegroups",
          "eks:ListUpdates",
          "eks:AccessKubernetesApi",
          "eks:ListAddons",
          "eks:DescribeCluster",
          "eks:DescribeAddonVersions",
          "eks:ListClusters",
          "eks:ListIdentityProviderConfigs",
          "iam:ListRoles"
        ]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = "ssm:GetParameter"
        Resource = "arn:aws:ssm:*:${var.aws_account_id}:parameter/*"
      }
    ]
  })
}


//https://docs.aws.amazon.com/eks/latest/userguide/connector_IAM_role.html
// create AmazonEKSConnectorAgentRole and AmazonEKSConnectorAgentPolicy
resource "aws_iam_role" "eks_connector_agent_role" {
  name = "AmazonEKSConnectorAgentRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "ssm.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_policy" "eks_connector_agent_policy" {
  name = "AmazonEKSConnectorAgentPolicy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "SsmControlChannel"
        Effect = "Allow"
        Action = [
          "ssmmessages:CreateControlChannel"
        ]
        Resource = "arn:aws:eks:*:*:cluster/*"
      },
      {
        Sid    = "ssmDataplaneOperations"
        Effect = "Allow"
        Action = [
          "ssmmessages:CreateDataChannel",
          "ssmmessages:OpenDataChannel",
          "ssmmessages:OpenControlChannel"
        ]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy_attachment" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_connector_agent_role.name
}

resource "aws_iam_role_policy_attachment" "eks_connector_agent_custom_policy_attachment" {
  policy_arn = aws_iam_policy.eks_connector_agent_policy.arn
  role       = aws_iam_role.eks_connector_agent_role.name
}

Step Two

update kubeconfig

aws eks update-kubeconfig --region us-east-1 --name education-eks-tf

Create a Kubernetes rolebinding or clusterrolebinding that is bound to a Kubernetes role or clusterrole that has the necessary permissions to view the Kubernetes resources. -- View Kubernetes resources in all namespaces

kubectl apply -f https://s3.us-west-2.amazonaws.com/amazon-eks/docs/eks-console-full-access.yaml

-- View Kubernetes resources in a specific namespace

kubectl apply -f https://s3.us-west-2.amazonaws.com/amazon-eks/docs/eks-console-restricted-access.yaml

or using customized by updating the downloaed file

curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/docs/eks-console-full-access.yaml

kubectl apply -f rbac.yaml

rbac.yaml:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: reader
rules:
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: reader
subjects:
  - kind: Group
    name: reader
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: reader
  apiGroup: rbac.authorization.k8s.io

Step Three

Map the IAM principal to the Kubernetes user or group in the aws-auth ConfigMap

kubectl edit -n kube-system configmap/aws-auth

add:

mapUsers: |
  - groups:
    - reader
    userarn: arn:aws:iam::4673623285:user/admin
    username: admin
mapRoles: |
  - groups:
    - reader
    rolearn: arn:aws:iam::4673623285:role/AmazonEKSConnectorAgentRole
    username: AmazonEKSConnectorAgentRole

references: https://docs.aws.amazon.com/eks/latest/userguide/view-kubernetes-resources.html#view-kubernetes-resources-permissions