DEV Community

Devam Parikh
Devam Parikh

Posted on

Exposing Internal Kubernetes Agent UIs: Lessons from VPC Lattice, DNS, and Private Access

#aws #kubernetes #devops #networking

A Kubernetes UI can be healthy and still unreachable from your browser.

That sounds contradictory until you debug private networking.

This post covers a common pattern: exposing an internal Kubernetes UI through AWS VPC Lattice, Route53, and private network access.

The examples are generic, but the lesson is practical:

A working backend does not guarantee a reachable human URL.

The symptom

The UI is deployed. The pod is running. The Kubernetes Service has endpoints. The route is accepted. Health checks are passing.

But the browser times out.

At this point, it is tempting to blame the application.

In many private-network cases, the application is fine. The problem is the access path.

Trace from the browser inward

Debug in this order:

  1. DNS resolution
  2. local route table
  3. VPN/private network routing
  4. Lattice or ingress listener
  5. route attachment
  6. target group health
  7. Kubernetes Service endpoints
  8. pod readiness

Do not start at the pod if the user cannot reach the hostname.

Start with:

dig internal-ui.example.com
Enter fullscreen mode Exit fullscreen mode

Then check what kind of IP the hostname resolves to.

Link-local addresses are a clue

If a private UI hostname resolves to something like:

169.254.x.x
fd00:...
Enter fullscreen mode Exit fullscreen mode

your laptop may not be able to route it, even if you are on VPN.

Many VPN clients route VPC CIDRs such as:

10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
Enter fullscreen mode Exit fullscreen mode

They may not route Lattice link-local addresses from your machine into AWS.

So the backend can be reachable from inside the VPC, while the browser on your laptop cannot reach the same DNS target.

Compare with a known working service

The fastest way to find the difference is to compare with another internal UI that already works.

Check:

  • What does its Route53 record point to?
  • Does it resolve to VPC-routable private IPs?
  • Does it use HTTPS?
  • Does it have an ACM certificate attached?
  • Does it point to a VPC endpoint association DNS name?
  • Does the non-working service point directly to a raw service DNS name?

In one common working pattern, Route53 points to a VPC endpoint association DNS name:

vpce-...-snsa-....vpc-lattice-svcs.region.on.aws
Enter fullscreen mode Exit fullscreen mode

That may resolve to private VPC IPs reachable over VPN.

A broken pattern may point directly to a Lattice service-assigned domain:

service-name-....vpc-lattice-svcs.region.on.aws
Enter fullscreen mode Exit fullscreen mode

That can resolve to link-local addresses that are not reachable from a user's laptop.

The fix is usually DNS target ownership

For human browser access, Route53 should point at the DNS target that matches the intended access path.

If users connect over VPN, the hostname should resolve to addresses their VPN routes.

That often means using the VPC endpoint association path rather than the raw service domain.

The exact implementation can be owned by Terraform, Crossplane, or another platform controller. The important part is not the tool. The important part is the contract:

Human UI hostname -> private routable endpoint -> Lattice/ingress -> Kubernetes Service
Enter fullscreen mode Exit fullscreen mode

Not:

Human UI hostname -> link-local service address unreachable from laptop
Enter fullscreen mode Exit fullscreen mode

Keep service-to-service and human access separate

The best endpoint for service-to-service traffic is not always the best endpoint for browser access.

For example:

  • MCP or internal service traffic may use one Lattice path.
  • Human UI access may need a different private DNS and endpoint association path.
  • Public access may be forbidden entirely.

That is fine.

Platform teams should make the access modes explicit:

agent-to-tool traffic: internal service path
human UI traffic: private browser-accessible path
Enter fullscreen mode Exit fullscreen mode

Verification checklist

Before declaring the UI broken, verify:

  1. Pod is running.
  2. Service has endpoints.
  3. Route is accepted.
  4. Lattice target is healthy.
  5. DNS resolves to the expected target.
  6. Resolved IPs are routable from the client network.
  7. HTTPS listener and certificate match the hostname.
  8. Security groups and network policy allow the path.

If steps 1-4 pass and step 6 fails, the application is not the problem.

Final thought

Private access bugs are often naming bugs in disguise.

The hostname exists. The service exists. The route exists. But the DNS target points to the wrong kind of network address for the client.

For internal Kubernetes UIs, always ask:

Does this hostname resolve to something the user can actually route to?

That question saves a surprising amount of time.

Top comments (0)