DEV Community: Deepu K Sasidharan

Introducing LlamaStash: a zero-overhead, terminal-native llama.cpp launcher

Deepu K Sasidharan — Tue, 02 Jun 2026 12:01:22 +0000

Originally published at deepu.tech.

In my recent post about my fully offline AI-assisted Linux development machine, I dropped a small detail near the bottom. I run my local model with an alias.

llamaServer

I described it as "a small script. It lets me pick a GGUF model, context size, and reasoning mode. It remembers the last choice, so most of the time I just start it and get going."

That script grew up. Today I'm releasing LlamaStash, the first public release of a fast, cross-platform, terminal-native launcher for llama.cpp with zero overhead.

It is a TUI. It is also a CLI. It is also a background daemon. It is also an OpenAI-compatible proxy. One small Rust binary (~5 MB download), three personas, same primitives everywhere.

Why does this exist?

Well: ADHD, insomnia and the below 😆

Local LLMs sit in an awkward gap.

On one side, raw llama-server is fast and honest. It is also tedious. You memorize flags. You write wrapper scripts. You remember which port a model is on. You guess context sizes that may or may not fit your VRAM. After a while you have a ~/scripts/ folder full of shell aliases that nobody else can read.

On the other side, Ollama and LM Studio wrap llama.cpp in friendlier shells. Ollama is opinionated about model storage, format, and config. LM Studio is GUI-first and not terminal native. Both pay a real performance cost compared to raw llama-server, and both hide the underlying primitives that I actually like working with.

I wanted something in the middle. A launcher that:

Stays out of llama.cpp's way (no fork, no patched copy, no opinions about its flags).
Is fast to invoke from a terminal and fast to drive from a script.
Is also good as a TUI, because I genuinely like terminal interfaces.
Treats agents and humans as equals. Anything a person can do in the TUI, an agent can do via --json.
Has a daemon underneath so models survive the TUI closing, and so multiple clients can hit the same model concurrently.
Exposes an OpenAI/Ollama-compatible proxy on loopback so any existing OpenAI client (your editor, your agent, your scripts) just works without per-model setup.

LlamaStash is that.

Why a TUI?

I love terminal UIs (see KDash, JWT-UI, and battleship-rs). I wrote KDash, a Kubernetes dashboard TUI in Rust. That was 2020. The Rust TUI ecosystem at the time was tui-rs and a lot of patience. Threading was DIY. Layouts were arithmetic. State management was you-figure-it-out.

Building LlamaStash brought me back to a lot of that, but the ground has shifted. ratatui (the maintained fork of tui-rs) is a real, polished framework now. tokio makes async daemons boring in a good way. hyper gives you a respectable HTTP server in a few hundred lines. crossterm handles the cross-platform terminal mess. sysinfo covers host metrics. The pieces are all there and you have LLMs to help you speed up everything to 10x.

I still believe what I wrote then. Rust gives you safety, speed, and a great UX without picking just one. LlamaStash is ~180 Rust files and not one production panic. It feels solid in a way that the JavaScript and Java tooling I shipped earlier in my career never quite did.

OK, enough nostalgia. Let me show you what the tool actually does.

Zero to chat in one command

llamastash init

This is the first-run wizard. It detects your hardware, installs llama-server for your OS/GPU combo, looks at your available VRAM, recommends a GGUF model that fits, downloads it, writes a tuned config, updates configs for your AI tools (OpenCode, Zed, etc.), and smoke-launches it to make sure the whole pipeline works end-to-end.

On my Strix Halo machine that means an automatic ROCm/HIP path with sensible defaults. On a MacBook it picks up Metal. On an NVIDIA Linux box it picks up Vulkan (CUDA coming soon). On a Windows 11 machine it picks the matching win-cpu / win-cuda / win-vulkan llama.cpp asset. On an old laptop with no GPU it picks up CPU and quietly recommends a smaller model.

If you already have a llama.cpp build you like, point at it with --llama-server. If you already have GGUFs in ~/.cache/huggingface/, ~/.ollama/models, or ~/.lmstudio/models, LlamaStash discovers them. It also watches your model paths live, so a new download shows up without a restart.

Already have a coding model on disk? Skip the wizard.

llamastash start qwen-coder --ctx 16384 --reasoning on

That's the whole command. Or use the TUI and pick from a list.

The TUI

This is what, I hope, most users will spend time in.

A few things worth knowing:

Vim-style navigation everywhere. hjkl, Ctrl+F/Ctrl+B, 0/$, gt/gT for tab cycling, / to filter, ? for help. The keybindings are not modal voodoo, they are real, documented, and rebindable.
The right pane is a smoke test. Logs, Chat, Embed, Rerank tabs that hit the same OpenAI-compatible endpoint external clients use. So when something works in the TUI, it works in your editor, your agent, your script.
In-TUI HuggingFace browser. Search, sort, paginate, see per-file hardware fit, download with cancel. No tab-switching to a website.
Five themes plus custom. Catppuccin Macchiato is the default because of course it is.
Adaptive layout. Works from 60 cells up. Tiny terminal? List columns and hint chips drop by priority rank so the model name stays readable.
Accessibility. Status is dual-encoded with both color and a glyph, so it reads on mono terminals too.

The mouse is off by default (except scrolling) so your terminal keeps native click-and-drag text selection. Opt in with --mouse-focus or a single line in config.yaml if you want it.

The CLI

Everything the TUI can do, the CLI can do. The CLI is a first-class surface, not an afterthought.

# List models, human-friendly
llamastash list

# List models, machine-friendly
llamastash list --json | jq

# Show, start, stop, status
llamastash show qwen-coder
llamastash start qwen-coder
llamastash stop qwen-coder
llamastash status --json

# Pull a model from HuggingFace
llamastash pull bartowski/Qwen3.6-27B-Instruct-GGUF

# Recommend something based on my hardware
llamastash recommend

A few things that matter for scripting:

--json is the stable agent contract. If we change the human-readable output, scripts using --json keep working.
Documented exit codes per failure class. Branch on numbers, not on message text.
TTY output is colored and padded. Piped output is byte-stable TSV, so your awk and column pipelines keep working.

There is also an Agent Skills bundle in the repo that teaches your coding agent to use the CLI properly. Drop it into your OpenCode, OpenClaw, or Claude Code skills directory and the agent learns to prefer --json, branch on exit codes, and read status --json before configuring an OpenAI-compatible client.

The proxy

Once a model is running, LlamaStash exposes an OpenAI-compatible endpoint on http://127.0.0.1:11435/v1 by default.

curl -s http://127.0.0.1:11435/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "qwen-coder", "messages": [{"role": "user", "content": "hi"}]}'

OpenCode, Pi, Cline, the OpenAI SDKs, llm-cli, anything that speaks OpenAI works as-is.

A few details worth knowing:

Auto-start on first request. If the requested model is not running yet, the proxy starts it and serves the request.
Fallback with audit headers. If launch fails for any reason, the proxy falls back to a Ready peer model and adds x-llamastash-served-by and x-llamastash-fallback-reason headers so the client can react. You can turn off this behavior via the fallback_enabled config option if you prefer a hard failure.
Ollama discovery surface. GET /api/tags, /api/version, /api/ps, POST /api/show. Tools that auto-detect Ollama via OLLAMA_HOST will recognize LlamaStash and fall through to the OpenAI-compat endpoints for actual inference.
Opt-in Ollama drop-in mode. With --ollama-compat (or LLAMASTASH_OLLAMA_COMPAT=1) the proxy claims port 11434 and answers the byte-exact "Ollama is running" handshake. The official ollama CLI talks to it. Most Ollama-Go-based clients talk to it. Useful when you want to replace Ollama in a workflow you didn't write.
Loopback only, no auth. Single-user local threat model. The proxy refuses LAN binds. LAN exposure plus auth plus TLS is on the roadmap, not in 0.0.2.

Zero overhead means zero overhead

The benchmark detail lives in its own post and docs/benchmarks.md, but here is the one-line version.

LlamaStash spawns the unmodified upstream llama-server. So the wrapper better not add measurable overhead vs running llama-server directly. We measure it. It does not. LlamaStash matches raw llama-server within ≤1% on every cell, across AMD APU, Apple Silicon, and NVIDIA. The proxy hop is +0.45 ms median TTFT and zero impact on decode.

The comparison against Ollama is more interesting, but I'll let the benchmark post do that talking. The methodology is published. The harness is reproducible. Run make bench-end-to-end and check for yourself.

Architecture, in one paragraph

One Rust binary. Invoking it with no arguments is the TUI. Invoking it with a subcommand is the CLI. Invoking it as llamastash daemon is the supervisor. The TUI and the CLI both talk to the daemon over a 127.0.0.1 HTTP control plane that's bearer-token-authed. The URL and per-restart token live in a runtime.json handshake file under the state dir, locked down to the file owner (mode 0600 on Unix, Protected DACL on Windows). Same trust model on every supported OS: same-machine, same-UID. The daemon spawns llama-server instances on demand, tracks them with a state machine, picks ports from a configurable range, and exposes the OpenAI-compatible proxy on loopback as a separate listener. State persists to atomic, mode-checked files. If your daemon crashes, the next invocation reads the state and re-adopts the running llama-server instances. Models survive a TUI close. They do not survive a reboot, by design.

For the deeper version, see docs/architecture.md. I had so much fun designing the architecture and a lot of it came from KDash obviously.

How carefully tested?

What CI gates before anything reaches crates.io, the Homebrew tap, or the AUR:

~2,000 test attributes across the workspace (1,700+ in Rust, the rest in pytest and bats). 86.5% line coverage tracked via grcov + Coveralls.
cargo audit --deny warnings for dependency vulnerabilities. Any new advisory fails CI.
bats integration tests for install.sh on Linux and macOS, plus shellcheck -s sh on the script itself.
pytest for the Python packager scripts that drive the Homebrew formula and the AUR PKGBUILDs.
MSRV pinned to Rust 1.95, verified with cargo check --locked against the pinned toolchain. Cross-compile check for aarch64-unknown-linux-gnu on every push.
A "release readiness" job runs cargo publish --dry-run --locked plus packager unit tests for both the Homebrew formula and the AUR PKGBUILDs, so the publish steps are green before the tag ever moves.

LlamaStash is a single-author project. Bugs will find me. The hygiene above is there to make that less painful when they do. The Windows and macOS builds are not as vigorously tested since I develop on Linux so there are gonna be rough edges there. If you have a Windows 11 machine or a Mac and want to help with testing, please do.

Install

Pick a channel. They all install the same binary. The download is ~5 MB across every supported target.

# macOS + Linux, one-shot installer
curl -fsSL https://llamastash.dev/install.sh | sh

# Homebrew (macOS + Linuxbrew)
brew install llamastash/llamastash/llamastash

# Arch Linux (AUR)
yay -S llamastash         # source build from the tagged GitHub release
yay -S llamastash-bin     # prebuilt x86_64 / aarch64 tarball
yay -S llamastash-git     # main-branch checkout, rebuilds on every -Syu

# Windows 11 (PowerShell, no admin elevation)
irm https://llamastash.dev/install.ps1 | iex

# Windows via Scoop bucket
scoop bucket add llamastash https://github.com/llamastash/scoop-llamastash && scoop install llamastash

# crates.io (any platform with a Rust toolchain)
cargo install llamastash

Then llamastash init and you are off.

Full install notes, including the non-interactive agent-friendly path, in INSTALL.md.

What's not in 0.0.2

A few things are explicitly on the next-release roadmap.

LAN exposure + auth + TLS. Today: loopback-only by design. If there's enough interest in safe LAN sharing, that needs auth and TLS done properly. Soon.
Anthropic /v1/messages compatibility shim. OpenAI-compatible is good enough for most agents, but a few prefer the Anthropic shape.
MCP server surface. The CLI is already agent-friendly, so I'm double minded about whether a Model Context Protocol server would make integration smoother. I'm personally not a fan of MCP and prefer skills and CLIs.
MLX and vLLM backends. llama.cpp is the right default, but I'd like LlamaStash to be backend-pluggable if possible.
NPU support. It would be nice if the same tool could run models on both GPUs and NPUs, so modern Copilot+ PCs and M-series Macs put their Neural Engines to work too.
Per-PID VRAM attribution. Today the supervisor reports host-level GPU memory. A per-process number is more useful.
CUDA on Linux. Vulkan is a fine GPU backend, but CUDA is the most popular GPU compute API on NVIDIA hardware. Today the Linux+NVIDIA route ships the Vulkan binary. CUDA on Linux is a bit sketchy to support (driver version matching, ABI stability), but I want to do it cleanly.
aarch64-pc-windows-msvc. 0.0.2 ships x86_64 Windows only. Snapdragon X / Surface Pro coverage on the roadmap.

Track or shape the roadmap on the TODO.md file in the repo.

Why local AI tooling matters

I wrote about this in the Linux machine post, and I'm not going to repeat it all here. The short version.

I use hosted models too. The best ones are still better at some things. But local-first tooling is a capability I want to have, not just for the privacy or the cost or the offline use, but for the ownership. It lets me change models, contexts, build flags, server parameters, and client config whenever I want. It lets me run experiments that I wouldn't think to run on a paid API. And in a world that is sliding fast toward a few companies owning the infrastructure that developers depend on, having my own stack feels less like a hobby and more like a small act of preservation.

LlamaStash is one piece of that. There will be others.

Try it, break it, tell me

If you have a Linux box, a Mac, or a Windows 11 machine sitting around with a half-decent GPU (or an Apple Silicon machine with enough unified memory), llamastash init should get you to a working local model in a few minutes. Bug reports, feature ideas, and "this didn't work for me" stories are all welcome at github.com/llamastash/llamastash/issues.

A star on the repo helps the project show up for other people looking for the same itch. It also makes me feel good about the effort that went into this 🤓. Not going to lie about that.

And finally, thanks to all the AI coding harnesses (OpenCode, Claude Code, Copilot) and LLMs (Claude, DeepSeek, Kimi, Qwen, Gemini, etc.) that turned what would have been a six-month, multi-person project into a one-person, couple-of-weeks project. I'm planning a deeper dive into the development process in a future post, especially the AI harness engineering around it and the vibe coding discipline that made it possible to keep the scope tight and the quality high. Stay tuned.

If you like this article, please leave a like or a comment.

You can follow me on Bluesky, Mastodon, and LinkedIn.

How fast is LlamaStash? Overhead, throughput, and a fair comparison with Ollama and LM Studio

Deepu K Sasidharan — Tue, 02 Jun 2026 11:34:05 +0000

Originally published at deepu.tech.

In my release post for LlamaStash I made a claim I need to back up. The wrapper adds zero overhead vs running llama-server directly. That is the kind of claim that should not exist in a blog post without numbers behind it. So here are the numbers.

LlamaStash spawns the unmodified upstream llama-server. So three different questions follow from that, and there is a benchmark suite for each.

Suite A: overhead regression. Does llamastash start <model> add any measurable overhead on top of raw llama-server when both run the same command line? This is the question the whole architecture depends on.
Suite B: cross-tool comparison. How does LlamaStash-as-shipped compare to Ollama and LM Studio on the same model, same hardware, through their OpenAI-compatible HTTP endpoints? This is the question users care about.
Suite C: proxy overhead. Does going through the LlamaStash OpenAI-compat proxy cost anything vs hitting llama-server directly? This is the smallest question, but it has to be asked because the proxy is the default surface.

All three suites live in docs/benchmarks/ in the repo. The harness is in scripts/bench/. The methodology page covers fairness, how I throw out noisy runs, and why outputs differ across backends. Read it before pulling any single number out of context. On its own, one cell can mislead.

The short version, if you are skimming. Two terms first: decode is tokens per second while the model generates, and TTFT is time to first token, the delay before anything comes back. And there are two ways to read every comparison below: matched-flags (called normalized in the harness, every tool runs the same knobs) and out-of-the-box (called defaults, what you get when you just run the tool). They tell different stories.

The wrapper adds no measurable overhead. With matched flags, LlamaStash runs the same speed as raw llama-server: within 1% on Apple Silicon and on three of four sizes on the AMD APU. The one exception is the 35B-A3B MoE on AMD, at -1.8% decode. That same cell flips to +1.1% the other way out of the box, so it is run-to-run timing noise on a fork-and-spawn, not real overhead. NVIDIA decode is -0.57%, a hair past my 0.5% warning line. Everything stays well inside my 2% hard-fail line.
Out of the box, LlamaStash is a touch faster than raw llama-server, because it sets good defaults for you: all GPU layers on every GPU backend, and flash-attention for Qwen and Llama models on Apple Metal and CUDA. On the Mac Qwen run that is +7.3% decode over raw llama-server's stock defaults. On the AMD APU the upstream defaults are already good, so the two match. On NVIDIA the gap is wider than those flags explain, which I leave as an open question below.
Ollama is slower, sometimes a lot slower. It is 38-72% behind raw llama-server on decode on the AMD APU. On the Mac its RAG prefill is 52× slower than the direct path, because it re-reads the whole document on every request instead of caching it. On NVIDIA Vulkan, the RAG prefill runs never finished.
LM Studio is a mixed bag. Its bundled ROCm runtime crashes on Strix Halo (gfx1151) for every model except the small one, so I could only compare it on Vulkan. There it loses decode to LlamaStash on three of four sizes (+8% small, +52% mid, +32% large MoE) and wins large_dense by 7%. And everywhere, it pays a 170-2300 ms first-token tax from its OpenAI shim and just-in-time model loading.
The proxy is free. Talking to LlamaStash's proxy instead of llama-server directly costs +0.45 ms to first token on AMD, -0.6 ms on the Mac, +0.57 ms on NVIDIA. Zero on decode. Sub-millisecond, which is nothing.

Now to the actual point of this post.

The setup

Same model bytes, same workloads, same hardware. Three platforms.

Platforms

Platform	CPU / SoC	GPU	RAM / VRAM	OS	llama.cpp build	Date
AMD APU	Ryzen AI Max+ 395 (Strix Halo)	Radeon 8060S iGPU (RDNA 3.5, `gfx1151`)	128 GiB unified, 70 W TDP	Arch Linux, ROCm 7.2.3	b9245 (`b39a7bf1b`)	2026-05-24
Apple Silicon	Apple M1	Integrated, Metal	16 GiB unified	macOS 26 (Darwin 25.4.0)	b9330 (`328874d05`, Homebrew Metal)	2026-05-27
NVIDIA	Intel i9-11900H laptop	RTX 3050 Ti Laptop (Ampere, `sm_86`)	4 GiB VRAM, 63 GiB host	Manjaro 6.6.141, driver 595.71.05, CUDA 13.2	b9360 (`6b4e4bd5`, CUDA + Vulkan lanes)	2026-05-28

This is just the hardware I had on hand, but the three machines sit in usefully different places. Strix Halo is high-end consumer with a unified-memory iGPU. The M1 with 16 GiB is the entry-level Apple Silicon most laptops actually have. The RTX 3050 Ti Laptop is a discrete NVIDIA card starved for VRAM. Not a full sweep of the hardware space, but three points different enough to stress different assumptions.

Models

The AMD APU run covered four points on the size and architecture axes.

ID	Model	Quant	Size	Why it's here
`small`	Gemma 4 E2B	Q4_K_M	1.6 GiB	Fast iteration, tests TTFT and per-request overhead
`mid`	Gemma 4 31B	Q4_K_M	17.4 GiB	The everyday workhorse size
`large_dense`	Qwen3.6 27B	Q8_0	26.6 GiB	High-fidelity dense, exposes memory bandwidth limits
`large_moe`	Qwen3.6 35B-A3B	Q8_0	34.4 GiB	Mixture of experts, ~3B active per token

The Mac and NVIDIA runs were constrained to a single model each: different models, different sizes, both intentional.

Platform	Model	Quant	Size	Why this one
Apple M1 16 GiB	Qwen2.5-0.5B-Instruct	Q4_K_M	~397 MB	The M1 16 GiB tier is the entry-level Apple Silicon class. It can load larger models, but the primary target for this hardware is fast, responsive edge inference of small models. M2 Pro / M3 Max runs at the mid/large tier are deferred.
RTX 3050 Ti Laptop 4 GiB	gemma-3-4b-it	Q3_K_M	2.1 GiB	The 4 GiB VRAM ceiling forces every dense model larger than `small` into a partial-offload regime that is not interesting for cross-tool comparison. Larger-VRAM NVIDIA hosts should re-run the full matrix.

Same GGUF bytes across tools. Where a tool needs its own pulled copy (Ollama has its own blob store), bit-identical content was verified via SHA-256 before benchmarking.

If you are interested in running this benchmarks yourself on your hardware, checkout scripts/run-uat-and-bench.md. Please consider sending a PR with the results if you do.

Workloads

Four. They model what a real coding session looks like.

chat_turn: 50-token prompt, 64-token decode. The conversational baseline.
agent_decode: short prompt, 1024-token decode. Measures sustained generation throughput.
rag_prefill: 4096-token prompt, 256-token decode. Measures prompt processing at realistic agent context sizes.
parallel_4: four chat_turn requests concurrently. Measures the supervisor's ability to interleave.

One warmup run, then three measured repetitions per cell. If those three spread by more than 10%, the cell is re-run; if it is still noisy on the second try, it gets flagged in the per-cell JSON.

Methodology rules

A few rules the harness enforces, because they are what make the result honest.

Suite A runs an identical command line. The harness drops --port, then checks that the rest of the llama-server command LlamaStash spawns is character-for-character identical to the one the raw run uses. There is nowhere for a hidden tweak to hide.
Suite B talks to every tool the same way. Every tool is driven through its OpenAI-compatible HTTP endpoint, not its CLI. Same client, same payloads.
Each platform runs in one sitting. All of a platform's rows (LlamaStash, raw, Ollama, LM Studio) run in the same session, no reboots in between.
Engine fallbacks are labelled. When a tool falls back to a different engine (for example LM Studio's ROCm runtime crashing on gfx1151, so the row uses its Vulkan runtime instead), the cell is annotated and the reason explained.

The full methodology is in docs/benchmarks/methodology.md. If you intend to argue with a number, read that page first.

Suite A: overhead regression

The headline question. If LlamaStash spawns the unmodified llama-server, the only way it can be slower is by adding overhead in the wrapper. We measure that overhead directly.

Each metric has two thresholds: a warning line (the harness calls it advisory, where the run passes but prints a banner) and a hard-fail line (catastrophic, where the run exits non-zero). That is the "warning line" and "hard-fail line" I refer to throughout.

Metric	Catastrophic (exits non-zero)	Advisory (exits zero with banner)
`ttft_ms` delta	≥ 200 ms	≥ 30 ms
`decode_tps` delta percentage	≥ 2.0% slower	≥ 0.5% slower
Daemon idle RSS	≥ 64 MiB extra	≥ 48 MiB extra

Results.

Top panel: decode delta % per cell, positive = LlamaStash faster. Bottom panel: TTFT delta ms, negative = LlamaStash faster. Yellow dashed lines are the harness advisory thresholds (±0.5% decode, ±30 ms TTFT); red dashed lines are the hard-fail thresholds (±2% decode; the ±200 ms TTFT hard-fail line is off-chart). Every cell sits inside the hard-fail box.

AMD APU

Suite A on Strix Halo used normalized mode against the matching self-built HIP llama-server (b9440).

Workload	LlamaStash decode tok/s	raw decode tok/s	Δ decode	LlamaStash TTFT ms	raw TTFT ms	Δ TTFT
`chat_turn` (small)	82.1	81.0	+1.3%	51	51	0
`chat_turn` (mid)	9.9	9.9	-0.0%	468	466	+3
`chat_turn` (large_dense)	7.5	7.5	+0.1%	406	406	0
`chat_turn` (large_moe)	42.3	43.1	-1.8%	178	185	-7

Three of the four sizes land within 0.3% on decode. The 35B-A3B MoE is the exception at -1.8%, the only AMD cell past my 1% warning line. The run-to-run spread was tiny on both sides (0.06% to 0.17%), so the gap is real measurement, not jitter. But out of the box, the same cell flips the other way, to +1.1% in LlamaStash's favor, and a real overhead would not change sign like that. So this is fork-and-spawn timing noise on a big MoE, not the wrapper costing you throughput. It is well inside my 2% hard-fail line, and the daemon's idle memory is well under budget too.

Apple Silicon

Suite A on M1 used normalized mode against the matching Homebrew Metal llama-server.

Metric	Delta	Tier
TTFT (mean)	`+2.3 ms`	OK (advisory 30 ms)
Decode	`+0.8%` LlamaStash faster (92.2 vs 91.5 tok/s)	OK (advisory 0.5%)

An earlier overhead pass showed LlamaStash -5.33% on decode, but that number mixed in the out-of-the-box defaults and was noisy on top of a real defaults difference. The matched-flags chat_turn cell is the clean Suite A comparison: +0.8% for LlamaStash, comfortably inside 1%. What the defaults add is broken out under Suite B below.

NVIDIA

Suite A on RTX 3050 Ti Laptop used normalized mode against the b9360 Vulkan llama-server.

Metric	Delta	Tier
TTFT (mean)	`+1.7 ms` (114.9 vs 113.2 ms)	OK (advisory 30 ms; well inside catastrophic 200 ms)
Decode	`−0.57%` (41.80 vs 42.04 tok/s)	ADVISORY (marginal miss on the 0.5% advisory floor)

Decode misses the 0.5% warning line by 0.07 of a point. A marginal miss, nowhere near the 2% hard-fail line.

Verdict: Suite A

Across four model sizes on the AMD APU and one model each on the Mac and the RTX 3050 Ti Laptop, LlamaStash matches raw llama-server within 1% on five of six matched-flags cells. Two cells miss that line: NVIDIA chat_turn at -0.57% decode (0.07 of a point past 0.5%), and the AMD MoE at -1.8% (which flips to +1.1% out of the box, so it reads as noise, not overhead). Both are well inside the 2% hard-fail line. The architecture (one binary, daemon on demand, a supervisor state machine, an HTTP control plane on loopback) does not cost throughput.

The zero-overhead claim holds.

Suite B: cross-tool comparison

Now the more interesting question. How does LlamaStash compare to Ollama and LM Studio in practice?

Same model bytes, same workload, OpenAI-compatible endpoint on every tool. Variance-gated runs. Decode tok/s and TTFT in milliseconds per cell.

AMD APU

Hardware: Ryzen AI Max+ 395, Radeon 8060S, 70 W. HIP/ROCm via system ROCm 7.2.3 + self-built llama-server (b9440). Dates: 2026-05-23 (small) and 2026-05-24 (mid, large_dense, large_moe).

Tool	small (E2B Q4)	mid (31B Q4)	large_dense (27B Q8)	large_moe (35B-A3B Q8)	Engine notes
LlamaStash	82.1 / 51	9.9 / 468	7.5 / 406	42.3 / 178	local HIP/ROCm
raw `llama-server`	81.0 / 51	9.9 / 466	7.5 / 406	43.1 / 185	local HIP/ROCm
LM Studio 2.18.0	91.1 / 187	— (crash¹)	— (crash¹)	— (crash¹)	bundled ROCm 6.4 vendor (see footnote)
Ollama 0.24.0	50.8 / 224	4.8 / 1096	2.6 / 1750	12.2 / 484	bundled

Each cell is decode tok/s / TTFT ms on chat_turn in normalized mode (ctx=4096, n_gpu_layers=999, flash_attn=on, batch=512, ubatch=512). Sourced from one bench run per row, no averaging across runs or modes. See source map at the bottom of this section. Defaults-mode numbers, agent-decode workload, and the wrapper-vs-raw delta are broken out in the charts.

Headline numbers from this matrix.

LlamaStash is the same speed as raw llama-server within 1% on three of four sizes (small +1.3%, mid -0.0%, large_dense +0.1%). The 35B-A3B MoE shows -1.8%, the same noise-not-overhead cell explained in Suite A above.
Ollama is 38-72% slower on decode than raw llama-server across all four sizes. Worst on large_dense (-65%), least bad on small (-38%).
LM Studio only loads the small model on ROCm on this hardware (gfx1151); mid, large_dense, and large_moe all crash at startup. Its full per-size view is in the Vulkan addendum below.

The first-token gap is the one to watch. Ollama and LM Studio both pay at least 200 ms, and over a second on the larger models, mostly in their HTTP shim layers. LlamaStash and raw llama-server stay under 200 ms even on the MoE. For an agent that fires a lot of short requests, that delay is most of the wall-clock you feel.

¹ LM Studio's bundled ROCm vendor libraries (v6.4 in linux-llama-rocm-vendor-v3) abort in ggml_cuda_error during backend initialization on gfx1151 (Strix Halo), across all three LMS-shipped runtime versions (amd-rocm-avx2@1.33.0, @2.16.0, @2.18.0). The system ROCm 7.2.3 loads the same models without issue via raw llama-server, so this is an LM Studio vendor-bundle limitation, not a hardware limitation. LM Studio numbers on AMD APU therefore use its Vulkan runtime exclusively, shown as a separate engine addendum below.

Vulkan addendum (LM Studio's only working AMD path)

Same-day side-by-side of LlamaStash and LM Studio on the Vulkan backend, since that is the only engine where LMS can load every model on this hardware. The other two cross-tool comparators are explicitly out of scope:

raw llama-server is omitted because the wrapper-overhead claim is already covered by the HIP/ROCm table above (and it repeats on Vulkan: LlamaStash matches raw within 1% across the measured cells).
Ollama is omitted because mainline Ollama does not support Vulkan. (Community forks exist; benchmarking a fork is not a fair representation of Ollama-as-shipped.)

Date: 2026-06-01. Backend: Vulkan (LlamaStash via self-built llama-server b9440 Vulkan; LM Studio via bundled vulkan-avx2@2.18.0 runtime). Same GGUF bytes across tools.

Tool	small (E2B Q4)	mid (31B Q4)	large_dense (27B Q8)	large_moe (35B-A3B Q8)
LlamaStash	101.2 / 55	10.8 / 671	7.5 / 196	50.7 / 72
LM Studio 2.18.0	93.6 / 191	7.1 / 2307	8.0 / 801	38.4 / 227

Each cell is decode tok/s / TTFT ms on chat_turn in normalized mode. Same-day rerun on a clean memory baseline.

Vulkan addendum read-out:

LlamaStash wins decode on three of four sizes: small (101.2 vs 93.6, +8%), mid (10.8 vs 7.1, +52%), large_moe (50.7 vs 38.4, +32%). LM Studio takes large_dense by 7% (8.0 vs 7.5); its defaults pick something better than LlamaStash's stock defaults for that one model. The harness logs every effective flag if you want to dig into why.
LM Studio's Vulkan mid number got worse since May. It used to be 11.6 tok/s decode and 1463 ms to first token; today's LM Studio with its bundled vulkan-avx2 2.18.0 runtime lands 7.1 and 2307. That is just what LM Studio ships now on this hardware, and it reproduced across two runs (33 GiB and 105 GiB free RAM). Worth flagging, because anyone on an older LM Studio version will see different numbers than this table.
First token is the wider gap. LlamaStash stays under 200 ms on small, large_dense, and large_moe. LM Studio sits at 191-2307 ms across the board, from its OpenAI shim and just-in-time model loading. On short-request agent work, that adds up fast.

Combined view (all tools × backends × modes)

For readers who want the whole comparison in one place, the next two charts overlay HIP/ROCm and Vulkan side by side, in both defaults and normalized mode, across all four model sizes and all four tools.

Light mauve / dark mauve = HIP/ROCm defaults / normalized. Light teal / dark teal = Vulkan defaults / normalized. Em-dashes mark cells not run (raw and Ollama on Vulkan) or backend-init crashes (LM Studio on HIP for mid, large_dense, large_moe). agent_decode is the longer-context workload (256-token target vs 64 for chat_turn); the relative ordering across tools holds.

Source map (AMD APU HIP cells)

Per-cell provenance, since this is the table the audience will pick apart first.

Tool	Size	Source JSON
LS / raw / Ollama	small	`runs/deepu-flowz13-arch/2026-05-23-335deaf47ccf.json`
LS / raw / Ollama	mid	`runs/deepu-flowz13-arch/2026-05-24-135416-0f23808f9b5f.json`
LS / raw / Ollama	large_dense	`runs/deepu-flowz13-arch-clean70w/2026-05-24-191126-0f23808f9b5f.json`
LS / raw / Ollama	large_moe	`runs/deepu-flowz13-arch/2026-05-24-150131-0f23808f9b5f.json`
LM Studio	small	`runs/deepu-flowz13-arch/2026-05-23-335deaf47ccf.json`
LM Studio	mid / large_dense / large_moe	crash: no HIP data
LS / LMS Vulkan addendum	all sizes	`runs/deepu-flowz13-arch-vulkan-clean-2026-06-01/`

Apple Silicon

Hardware: Apple M1, 16 GiB unified, Metal. Model: Qwen2.5-0.5B-Instruct Q4_K_M. Date: 2026-05-27.

chat_turn decode tok/s / TTFT ms, both modes shown side-by-side because the defaults-vs-normalized story matters here.

Tool	defaults	normalized	Engine notes
LlamaStash	99.0 / 15	92.2 / 21	local Metal (Homebrew b9330)
raw `llama-server`	92.3 / 19	91.5 / 21	local Metal (Homebrew b9330)
LM Studio	88.0 / 68	88.7 / 67	bundled Metal
Ollama 0.24.0	79.1 / 101	80.1 / 102	bundled

Here is the whole story in one paragraph. With matched flags (ctx 4096, all GPU layers, flash-attention on, batch and ubatch 512), LlamaStash and raw llama-server are within 1% of each other. The wrapper has nothing to add when both sides start from the same flags. Out of the box they pull apart, because LlamaStash's defaults table turns on all GPU layers and flash-attention for Qwen on Metal, two things upstream leaves off. That puts the chat_turn cell +7.3% ahead on decode (99.0 vs 92.3) and 4 ms faster to first token. You get that without knowing which flags matter on your hardware.

Ollama lands at 79.1 tok/s and 101 ms out of the box, about 14% slower decode than raw llama-server, and 5.3× the time to first token. The first-token gap is the one you actually feel in interactive use, and it compounds for an agent making many short requests.

The other workloads (agent_decode, rag_prefill, parallel_4) track the same shape. Two cells worth pulling out before they get buried.

Ollama's RAG prefill takes 2,849 ms to first token, against 40-86 ms for LlamaStash, raw llama-server, and LM Studio. That is 52× the direct path on the same hardware, for the same 8K-token document. The other three cache the document after the first request; Ollama re-reads it every time by default.
Ollama's four-way parallel run: 314.6 tok/s total, but 1,352 ms to first token per stream. Ollama posts the highest total throughput by queueing the four requests, but each user waits 35× longer for their first token than on LlamaStash (38 ms). Total throughput is the wrong thing to optimize for here.

Full per-workload tables and the LM Studio reasoning-parser caveat: Apple M1 final report.

NVIDIA

Hardware: NVIDIA RTX 3050 Ti Laptop (Ampere, 4 GiB VRAM), Intel i9-11900H, 63 GiB RAM. Model: gemma-3-4b-it Q3_K_M. Date: 2026-05-28.

The NVIDIA platform has two backends worth measuring side-by-side: CUDA (driver-provided) and Vulkan (vendor-neutral). Both come from the same upstream llama.cpp commit (b9360); the comparison is across engines, not across builds. chat_turn decode tok/s / TTFT ms, defaults mode.

Tool	CUDA	Vulkan	Engine notes
LlamaStash	41.1 / 74	42.0 / 113	b9360 CUDA self-built (`sm_86`) and b9360 Vulkan prebuilt
raw `llama-server`	36.6 / 110	37.5 / 148	same b9360 binaries, invoked directly
LM Studio 2.16.0	48.7 / 318	48.3 / 308	LMS-bundled CUDA / Vulkan runtimes
Ollama 0.24.0	40.7 / 120	42.0 / 115	bundled

Three findings from this matrix.

Out of the box, LlamaStash leads raw llama-server by 12-16% on decode, on both CUDA and Vulkan, same b9360 binary, same model, same hardware. But the visible defaults for gemma3 on NVIDIA only turn on all GPU layers (gemma is not on the flash-attention list, so the overlay does not enable it here). A 12-16% gap is wider than all-GPU-layers alone should buy. Maybe raw llama-server's CUDA build defaults to fewer GPU layers; maybe the auto-fit context picks a different size than raw's default; maybe there is another knob outside the visible command line. The gap is real in the data, but I have not pinned down the cause. With matched flags, LlamaStash and raw collapse to within 0.5 tok/s of each other, which confirms the wrapper itself adds nothing.

Vulkan decode beats CUDA decode in 26 of 28 comparable cells, median +5%, range -7% to +27%. This is a 4 GiB Ampere card running a Q3 4B model, a memory-bandwidth-bound job where Vulkan's kernels have caught up. The old "CUDA always wins on NVIDIA" rule is not universal. Suite A and Suite C here were Vulkan-only; a CUDA run of Suite A is a follow-up.

Two cross-tool data points from the broader matrix worth pulling forward.

Ollama's Vulkan RAG prefill never finished. Both runs together burned about 1.5 hours before I gave up. Ollama on Vulkan is simply not usable for document-RAG on this hardware. The CUDA rows at least finish, but slowly: 3,422 ms and 3,712 ms to first token, against 52-61 ms for LlamaStash and raw on the same document. That is an order-of-magnitude gap you cannot buy your way out of with faster hardware.
LM Studio wins on decode, loses on first token. It posts the highest decode in six of eight out-of-the-box cells. With matched flags everyone converges to the same engine baseline, so that lead is smarter defaults, not a faster engine. But its first token sits at 300-830 ms across every workload, an order of magnitude worse than the direct llama-server path (74-189 ms). On NVIDIA that first-token tax is even worse than on AMD.

Reproducing this on NVIDIA needs two local bench-harness patches that are not yet upstream (forwarding LLAMASTASH_LLAMA_SERVER to llamastash start and using Path.absolute() instead of Path.resolve() for the proxy model-id fallback). Both are documented in the NVIDIA report, which also carries the open question about the 12-16% out-of-the-box gap above.

Verdict: Suite B

Three findings pull through every platform.

With matched flags, LlamaStash and raw llama-server are indistinguishable. The wrapper is honest.
Out of the box, LlamaStash's defaults make it a little faster by turning on all GPU layers plus flash-attention where the model supports it (Qwen, Llama on Metal and CUDA). On the AMD APU the upstream ROCm defaults are already good, so the two match. On the Mac Qwen run the defaults buy +7.3% decode over raw, about what you would expect from turning on flash-attention. On NVIDIA the gap is wider than the visible flags explain, an open question, covered above.
Ollama and LM Studio each have a structural weakness that follows them across hardware. Ollama's problem is RAG prefill: fine on tiny inputs, catastrophic when it has to re-read a document (52× on the Mac, 60×+ on NVIDIA CUDA, never finishing on NVIDIA Vulkan). LM Studio pays a steady 300 ms to 1.5 s first-token tax from its OpenAI shim everywhere, with smart defaults that win some decode cells on small models and lose on bigger ones.

The picture is clearest if you want a simple takeaway. If you care about throughput, run raw llama-server or LlamaStash; they are the same engine. If you want raw llama-server plus a TUI, plus a CLI, plus a daemon, plus an OpenAI-compatible proxy, plus a script-friendly agent surface, run LlamaStash. There is no measurable cost to that bundle.

Suite C: proxy overhead

Last question. If you talk to the LlamaStash proxy at http://127.0.0.1:11435/v1 instead of llama-server directly, does it cost you anything?

The harness brings up one model, then sends the same chat request to both URLs, alternating one-for-one. Same llama-server behind both.

Platform	Direct TTFT	Proxy TTFT	Delta TTFT	Decode delta
AMD APU (`chat_turn`, small, 15 reps)	52.37 ms	52.82 ms	+0.45 ms	0%
Apple M1 (Qwen2.5-0.5B Q4, 5 reps)	31.4 ms	30.8 ms	−0.6 ms	+1.4%
NVIDIA RTX 3050 Ti Laptop (Vulkan, 4 reps)	111.0 ms	111.6 ms	+0.57 ms	−0.20%

Top: decode delta % (proxy − direct). Bottom: TTFT delta ms (proxy − direct). The advisory thresholds from Suite A (±0.5% decode, ±30 ms TTFT) are drawn for context; the TTFT panel is zoomed in to ±1.6 ms because the actual deltas are sub-millisecond.

The first-token difference is under a millisecond on every platform, inside run-to-run noise. The Mac row even shows the proxy 0.6 ms faster than direct, which is just noise. The loopback round-trip is sub-millisecond and disappears in the overall number. Decode is unchanged, because the proxy is not in the streaming path after the first byte.

Full numbers and method: docs/benchmarks/proxy/.

What this means in practice

Pulling the three suites together.

The wrapper is honest. LlamaStash spawns the same binary you would run by hand, with the same flags. With matched flags there is no measurable throughput cost on AMD APU, Apple Silicon, or NVIDIA, on every metric.
Out of the box you get good defaults for free. The defaults turn on all GPU layers plus flash-attention where the model supports it. On the Mac Qwen run that is +7.3% decode over raw llama-server's stock defaults. On AMD the upstream defaults are already good and the two match. On NVIDIA the gap is wider than the visible flags explain (open question above). The right way to read this is not "LlamaStash is faster than llama.cpp." It is "LlamaStash gives most people the tuned defaults they would otherwise have to discover themselves."
The proxy is free. Sub-millisecond on every platform. Nothing on the timescale an LLM works at.
Ollama is slower than raw llama.cpp on decode, and structurally worse on repeated-document RAG everywhere I measured. The gap is biggest on the AMD APU and the RTX 3050 Ti Laptop, smaller on the Mac.
LM Studio is fast where its bundled engine is well-tuned, slow where it falls back, and always pays a first-token tax that hurts short-request agent work more than long-form generation.
You can measure your own setup. Do not take my hardware as yours.

If you want a single sentence: LlamaStash gives you the speed of raw llama-server, plus defaults that pick the right GPU-layer and flash-attention settings for you, with a real tool wrapped around it.

Caveats and limits

This benchmark intentionally does not cover a few things. Calling them out so I'm not pretending they are settled.

Quality is not measured here. Decode throughput and TTFT are speed numbers, not quality numbers. Two tools can be at the same throughput and produce different outputs, especially on reasoning-mode prompts. A separate quality-first benchmark is in scope for a follow-up post.
Cross-backend determinism varies. Different llama.cpp builds on different platforms produce different outputs for the same input, even at temperature 0. The methodology page explains why and what we control. Don't quote a single number; quote the full row.
Mac and NVIDIA cover one model each. The M1 16 GiB run used a 0.5 B small model, sized for the entry-level Apple Silicon target. The RTX 3050 Ti Laptop run used a 4 B Q3 model, sized for the 4 GiB VRAM ceiling. Larger Apple Silicon (M2 Pro, M3 Max, M4) and larger-VRAM NVIDIA hardware should re-run the full matrix; the harness is published.
The NVIDIA 12-16% out-of-the-box gap is not fully explained. The visible defaults for gemma3 on NVIDIA only turn on all GPU layers (gemma is not flash-attention-eligible). The measured gap is bigger than that one flag should buy. A follow-up should log and compare the full effective command line on both sides before anyone quotes the number as marketing. The Suite A (overhead) and Suite C (proxy) results on NVIDIA do not depend on this.
NVIDIA CUDA Suite A and Suite C used Vulkan only. The overhead and proxy measurements on the RTX 3050 Ti Laptop used the Vulkan binary. The proxy is engine-agnostic and expected to give the same result on CUDA, but a CUDA-lane Suite A certification has not been run.
NVIDIA reproduction requires two local bench-harness patches that are not yet upstream. Both are documented at the bottom of the NVIDIA report.
CUDA vs Vulkan binaries were built differently on NVIDIA. CUDA was a self-built cmake -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 against b9360; Vulkan was the prebuilt b9360 release asset. Same upstream commit, but compiler flags and link-time options can differ. Interpret the CUDA vs Vulkan comparison as "these two specific builds on this hardware" rather than "CUDA vs Vulkan in general."
Ollama and LM Studio versions move. Ollama 0.24.0 and LM Studio 2.16.0 were the latest stable releases at run time. Newer versions will move the numbers. Rerun against current versions.
LM Studio version not captured on M1. The lms version CLI returned an ANSI banner instead of a semver string; the desktop app was current as of 2026-05-27.
The prompt-processing rate on rag_prefill is inflated when the cache hits, because it divides the prompt tokens by the cached ~50 ms instead of the real cold-prefill time. On rag_prefill, trust decode and first-token, not the prompt rate.
LM Studio's rag_prefill decode is blank on the small models. Its reasoning-mode parser eats the token count, and the benchmark's output cap leaves at most one real token to measure. First-token is still valid there; decode is dropped from those cells.
Single hardware points are not the population. Strix Halo, M1 16 GiB, and the RTX 3050 Ti Laptop are specific machines on specific power budgets. A different SoC, a different TDP, a different driver will move numbers. The publishable claim is "this is what these three machines do today." The harness lets you produce the same claim for your own machine.

Reproducing this

Both suites are maintainer-run. Nothing in CI fires them. To run them yourself:

git clone https://github.com/llamastash/llamastash
cd llamastash

# Suite B (cross-tool)
make bench-end-to-end -- --dry-run    # print the planned matrix first
make bench-end-to-end

# Suite A (overhead)
make bench-overhead

# Suite C (proxy)
make bench-proxy -- --model <path/to/model.gguf>

# Pivot existing JSONs into the headline summary
make bench-table

Prerequisites, per-backend gotchas, and honored environment variables: docs/benchmarks/methodology.md.

Drop new host directories into docs/benchmarks/runs/ and re-render. No central database, no schema migration dance.

Conclusion

The release post made a claim. This post backs it up, with one addition I did not see coming.

LlamaStash is a wrapper around llama-server, and the only honest kind of wrapper is one that adds no measurable overhead. I measured it. It does not. The whole architecture is built around staying out of llama-server's way, and the numbers say it works: with matched flags, the two are the same speed on every platform.

The defaults are a side note worth keeping. Turning on all GPU layers and flash-attention (where the model supports it) at startup is there to save you from discovering which flags matter on your hardware. The Mac Qwen run shows that buys +7.3% decode over raw llama-server's stock defaults, same hardware and same binary, about what flash-attention on Qwen Metal should give. NVIDIA shows an even wider gap that the visible flags do not fully explain; the NVIDIA section above is honest about it, and a follow-up will dig into the cause.

The cross-tool comparison turned out more interesting than the overhead claim. Ollama is consistently behind raw llama.cpp on decode, and its repeated-document RAG first-token is a structural problem across hardware. LM Studio's Vulkan path is well-tuned on some hardware and worth respecting, but its first-token tax is consistent and real for agent work. LlamaStash gives you raw-llama.cpp speed with matched flags, a little better out of the box, and a real tool around it that does not cost you throughput.

If you want to run this yourself on your own hardware, the harness is published, the methodology is published, the per-cell JSONs are checked in. The whole thing is reproducible.

The right tool for the right job. The right tool is whichever one you measured.

If you like this article, please leave a like or a comment.

You can follow me on Bluesky, Mastodon, and LinkedIn.

My fully offline AI-assisted Linux development machine

Deepu K Sasidharan — Mon, 11 May 2026 20:47:59 +0000

Originally published at deepu.tech.

One of my most popular posts of all time was when I wrote about my beautiful Linux development machine in 2019. I followed that up in 2021 with my sleek and modern Linux development machine. Since then, a lot has changed in my setup.

I moved from Fedora and KDE to a mostly vanilla Arch Linux setup. I moved from a traditional desktop environment to niri, a scrolling Wayland compositor. And of course, like every developer out there, my workflow now has AI in it. But this time, I wanted something a bit different: AI-assisted development that can run fully offline on my own machine.

Yes, local AI coding on Linux. What is not to love here?

Deepu K Sasidharan

Jun 2

Introducing LlamaStash: a zero-overhead, terminal-native llama.cpp launcher

#ai #llamacpp #localllm #rust

11 min read

This is not a tutorial on how to reproduce every single bit of my setup. My full personal configuration is private because it has too much machine-specific and personal stuff. But I'm making a stripped-down public version with the bare minimum needed for Arch, niri, DMS, OpenCode, and llama.cpp at deepu105/archdots.

This post is more about the current shape of my Linux development machine and why I ended up with this stack.

This is my primary machine for all of the below.

Rust, JavaScript, TypeScript, Java, Go, Python, Linux, and web development
Running multiple web applications locally
Running Docker containers and local Kubernetes clusters
Kubernetes, Terraform, and cloud CLI work
Writing, blogging, presentations, and demos
Heavy browser usage
E-mail, chat, and video conferencing
Screen recording, screenshots, video editing, and light media work
Running local LLMs for coding, refactoring, and code review
Testing AI tools without sending every prompt to a cloud provider

Machine configuration

The configuration of the machine is quite crucial for this setup. Running a browser, a few IDEs, Docker, terminals, and local LLMs is not exactly a light workload.

My current machine is an ASUS ROG Flow Z13 2025 model. It is a weird little beast. It is technically a tablet, but it has enough CPU, GPU, and memory to behave like a mobile workstation.

Here is the current setup.

Model: ASUS ROG Flow Z13 GZ302EA
Processor: AMD Ryzen AI Max+ 395, 16 cores and 32 threads
Graphics: AMD Radeon 8060S integrated GPU with 40 compute units
Memory: 128GB unified memory. I have assigned 64GB to the GPU and 64GB to the CPU. This is configurable in the BIOS.
Storage: 2TB NVMe SSD
Built-in display: 13-inch, 2560x1600, 180Hz
External displays: 34-inch 3440x1440 monitor and 27-inch 2560x1440 monitor
Camera: Razer Kiyo Pro
Keyboard and mouse: Keychron K2 and Logitech MX Vertical

The memory is the most interesting part here. For normal development work, 32GB is still fine and 64GB is great. But for local AI work, memory changes everything. A 27B quantized model, a large context window, Docker, Chrome, and an editor can happily eat memory like there is no tomorrow.

Having that much unified memory means the machine can run a useful local coding model without feeling like a science experiment. That is a big deal.

Operating system

I praised Fedora in the previous posts, and I still think Fedora is one of the best Linux distributions for most developers. Updates are smooth, new packages land often, and it mostly stays out of the way.

But this time I went with vanilla Arch Linux. So yes, I use Arch btw! 😉 I know, rolling release and all that. I have been using Linux long enough to know what I was signing up for.

The main reason was simple: I wanted the latest kernel, Mesa, ROCm-adjacent bits, Wayland tools, and desktop packages without waiting for the next distro release. New hardware like the Flow Z13 usually benefits from being closer to the bleeding edge. Arch gives me that. Well, OK, I also fell in love with the sexy new compositors like niri and Hyprland, and Arch is a great way to run those without waiting for backports. I started with Hyprland, but I ended up liking niri better for my workflow, and Arch made it easy to switch and experiment.

My installation is still fairly boring, and I mean that as a compliment.

Btrfs for root, home, cache, and log subvolumes
GRUB for boot
paru for pacman and AUR packages
Timeshift and grub-btrfs for snapshots
PipeWire for audio
NetworkManager for networking
Docker, Distrobox, Flatpak, and a bit of Homebrew where it makes sense

I also use Topgrade to keep the system updated. My private config even wires it into DankMaterialShell, so I can see available updates from the bar and trigger an update for everything on the system from pacman/AUR, brew, cargo, npm, VS Code plugins, Docker images, and so on in Kitty.

Again, quite simple, at least in my eyes.

Desktop environment, or lack of one

This is probably the biggest change from my previous setup. I no longer run GNOME or KDE as my main desktop. I use niri, which is a scrollable tiling Wayland compositor.

If you have not used niri, the workflow is quite different from a regular tiling window manager. Instead of forcing everything into a fixed grid, windows live in columns and you scroll horizontally across them. It sounds odd until it clicks. Once it clicks, it feels very natural on ultrawide monitors and laptop displays. I especially love the touchpad gestures for switching workspaces and moving windows around. It is a very fluid way to manage windows.

My current session looks like this.

Login manager: SDDM on Wayland
Window manager: niri 26.04 on Wayland
Shell: DankMaterialShell, usually called DMS
Terminal: Kitty
Theme: Catppuccin Macchiato everywhere. I just love this color theme.
Fonts: Inter Variable and JetBrainsMono Nerd Font

Niri and DMS

Niri gives me the compositor. DMS gives me the desktop shell pieces that I would otherwise have to stitch together myself.

DMS replaces a lot of the usual Wayland desktop plumbing:

Top bar
Application launcher, Control center, Media controls
Clipboard manager, Notification center
Process and system monitoring
Power menu, Lock screen
Screenshot and screen recording plugins
Wallpaper and theme switching

This is the kind of stuff where I do not want to maintain five different tools and a bunch of scripts if one project does the job well enough. DMS is still young, but it is already quite useful, especially with niri. It's also quite extensible, and I have already started adding tools that I want. For example, a locally saved TODO widget.

The Flow Z13 also needs some special handling. I have fixes for ASUS hotkeys, touchpad behavior, keyboard backlight, Thunderbolt rescans, and Wi-Fi quirks in my private config. The public archdots repo will only carry the reusable bits. This is Linux on new hardware, so of course there are quirks. What is a Linux experience without glitches, right?

Development tools

My development tools are still mostly boring, in a good way. These are subjective choices, and they do not matter as long as you are comfortable with your tools.

Shell: I use Zsh with zinit, Powerlevel10k, zoxide, and fzf. I still use a bunch of aliases for Git, Docker, package management, Jekyll, and local AI tools.

Terminal: I use Kitty. I have tabs, splits, clipboard bindings, quick access terminal, and a few custom keybindings. It is fast, it works well on Wayland, and it does not get in my way.

Editors: I use Neovim with LazyVim as my default editor. I still use Visual Studio Code depending on the project and what I am testing.

Toolchains: I use SDKMAN! for JDKs, NVM for Node.js, rustup for Rust, Bun, Go, Python, Deno, and the usual Linux build tools.

DevOps: Docker, Docker Compose, kubectl, kdash, Terraform, Distrobox, and so on. Some come from pacman or AUR, some from Homebrew, and some from language-specific installers.

Offline AI-assisted development

Now to the fun part.

I use cloud AI tools as well, and they are useful. But I also wanted a setup where I can code with an AI assistant without sending code, prompts, logs, or half-written ideas to a remote API. Not because every project is secret, but because local-first tooling is a good capability to have especially in a world that's heading towards techno oligarchy.

My current stack is:

OpenCode as the coding agent
A custom llama.cpp build with HIP support. This is much more performant than Ollama or LM Studio.
A local llama-server exposing an OpenAI-compatible API on 127.0.0.1:18080/v1
Qwen3.6 27B and Gemma 4 31B for local models, depending on the task. I use different quantization levels based on need, from 4-bit to 8-bit.
LM Studio for managing models and for offline chat.
ROCm/HIP acceleration on the Radeon 8060S
opencode-telegram-bot for managing OpenCode sessions remotely from Telegram

Here is my OpenCode provider config:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llama.cpp": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llama.cpp ROCm (local)",
      "options": {
        "baseURL": "http://127.0.0.1:18080/v1"
      },
      "models": {
        "qwen3-6-27b-q8-0": {
          "name": "Qwen3.6 27B Q8_0 (local ROCm)",
          "limit": {
            "context": 262144,
            "output": 16384
          }
        },
        "qwen3-6-27b-q6-k": ...,
        "qwen3-6-27b-q4-k-m": ...,
        "gemma-4-31b-it-q4-k-m": ...,
        "gemma-4-31b-it-q8-0": ...
      }
    },
    "openrouter": {
      "models": {
        "moonshotai/kimi-k2.6": {
          "name": "Kimi K2.6 (OpenRouter backup)",
          "limit": {
            "context": 262144,
            "output": 16384
          }
        },
        "deepseek/deepseek-v4-pro": {
          "name": "DeepSeek V4 Pro (OpenRouter backup)",
          "limit": {
            "context": 1048576,
            "output": 384000
          }
        }
      }
    }
  }
}

I start the local model server with an alias.

llamaServer

That points to a small script. It lets me pick a GGUF model, context size, and reasoning mode. It remembers the last choice, so most of the time I just start it and get going.

The default model and context right now are:

Qwen3.6-27B-Q8_0.gguf - 256k context

Here is a quick llama-bench comparison of the local models on my machine. The numbers are tokens per second with ROCm, full GPU offload, flash attention, f16 KV cache, a 4096-token prompt, a 256-token generation, and 3 repetitions.

Model	Quantization	Size	Prompt tokens/s	Generation tokens/s
Qwen3.6 27B	Q4_K_M	15.40 GiB	260.06	10.41
Qwen3.6 27B	Q6_K	20.56 GiB	279.37	8.70
Qwen3.6 27B	Q8_0	26.62 GiB	260.12	7.18
Gemma 4 31B IT	Q4_K_M	17.39 GiB	209.57	9.12
Gemma 4 31B IT	Q8_0	30.38 GiB	202.31	6.19

The full context is 256k tokens. Here is a benchmark with full context for the Qwen variants.

Model	Quantization	Size	Prompt+Generation tokens/s
Qwen3.6 27B	Q4_K_M	15.40 GiB	67.15
Qwen3.6 27B	Q6_K	20.56 GiB	65.77
Qwen3.6 27B	Q8_0	26.62 GiB	64.34

Running Qwen3.6 27B Q8_0 with 256k context in reasoning mode loads around 70% of the GPU memory in my setup and gives around 64 tokens/s for prompt+generation. That is quite good for a local model with that much context.

The llama.cpp build is also automated with a small script.

cmake -S /mnt/work/Workspace/llms/llama.cpp \
  -B /mnt/work/Workspace/llms/llama.cpp/build-hip \
  -G Ninja \
  -DGGML_HIP=ON \
  -DAMDGPU_TARGETS=gfx1151 \
  -DCMAKE_BUILD_TYPE=Release

cmake --build /mnt/work/Workspace/llms/llama.cpp/build-hip \
  --config Release \
  -j "$(nproc)" \
  --target llama-server llama-bench

The server runs like this under the hood.

ROCBLAS_USE_HIPBLASLT=1 llama-server \
  --model "$model" \
  --alias "$alias_name" \
  --host 127.0.0.1 \
  --port 18080 \
  --ctx-size "$ctx" \
  --n-gpu-layers 999 \
  --flash-attn on \
  --no-mmap \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --batch-size 4096 \
  --ubatch-size 512 \
  --reasoning "$reasoning"

Once the server is running, OpenCode talks to it like it would talk to any OpenAI-compatible provider. The difference is that the whole loop stays on my machine.

It's very elegant IMO!

I do not only use local models, though. For complex tasks, I also use frontier models through OpenRouter, mostly Kimi K2.6 and DeepSeek V4. Occasionally I use Copilot CLI and at work, I use Claude Code as well.

For the harness, I prefer OpenCode. I do not see any noticeable performance difference between Claude Code and OpenCode with Kimi or DeepSeek for the kind of coding tasks I do, which is mostly open source projects in Rust and TypeScript. That might vary for other people, of course, but for me OpenCode has been quite good and I especially prefer its UX over others. I'm trying Pi on the side as well to see if I keep it in the mix.

Why local AI coding matters to me

Local AI is not a replacement for everything. The best hosted models are still better for many tasks, especially when you need maximum reasoning quality or very fast responses. But local models have their own sweet spot.

For me, the advantages are clear.

Education/Fun: Running a local model is a great way to learn about how these models work, what kind of hardware they need, and how to optimize them. It is a fun hobby in itself. This has already paid off in terms of understanding the AI landscape better and being able to troubleshoot issues that come up in my work.
Privacy: I can use it on private code, half-baked ideas, local logs, and experiments without thinking about what leaves the machine.
Offline use: It works without internet. That is great while travelling or when the network is flaky.
Cost control: No token anxiety for long coding sessions and experiments. I can run it as long as I want without worrying about bills.
Hackability: I can change model, context, cache type, build flags, server parameters, and client config whenever I want.

But there are tradeoffs.

The model quality depends heavily on the model and quantization. Qwen3.6 27B, for example, has been good in most tasks I threw at it so far, but once context keeps growing the model compacts and starts hallucinating. So for long context tasks, I use Kimi or DeepSeek. I did some non-scientific benchmarks by giving Claude Opus 4.7, Qwen3.6 27B, and Kimi K2.6 the same set of prompts, and the difference in quality was noticeable but not too dramatic. The local model at times even did better than others, like catching stuff in review that Claude Opus missed.
Large context is useful, but it is not free since it affects tokens/second. Qwen3.6 27B at 256k context is noticeably slow compared to a hosted frontier model. Like maybe twice or three times slower.
ROCm on fresh AMD hardware can be a bit of a moving target but thanks to current models, a fix for any issue has been a prompt away.
Some agent workflows are slower locally than with hosted models.
You have to care about model storage, updates, server flags, GPU memory, and cooling.

So no, I do not think everyone should run a local coding model. But if you enjoy owning your stack and you have the hardware for it, it is a very satisfying setup.

The AI workflow

My usual workflow is quite simple.

Start the local model server with llamaServer.
Pick the model and context preset if I want to change it.
Start opencode in the repository and pick a model if I want to change it.
Ask it to inspect the codebase before making changes.
Let it edit, test, and iterate, while I review the changes using the opencode-telegram-bot remotely from Telegram.

For small tasks, I turn reasoning off because it makes tool-heavy work faster. For design questions, debugging, or code review, I turn reasoning on. The script makes that a prompt instead of forcing me to remember a long command.

This is the kind of boring automation I like. It removes friction without hiding what is actually happening.

Productivity and media tools

Most of my productivity stack did not change much.

Browser: Google Chrome is still my primary browser. I also keep Firefox around.

Password management: I use Bitwarden and a YubiKey.

Communication: Zoom, Signal, Telegram, and the usual suspects.

Screen capture: DMS screenshot plugin, screen recorder plugin, and OBS Studio when I need more control.

Images and video: Gimp, Inkscape, Kdenlive, and a few Flatpak utilities like Upscayl and Buzz.

File manager: Dolphin, because KDE apps are still excellent even when KDE is not my main desktop.

What is still not perfect

Of course, not everything is perfect. This is bleeding-edge Linux, on a new ASUS convertible, with a new AMD chip, a Wayland compositor, and a local AI stack. If everything worked perfectly on day one, I would be suspicious.

Some current rough edges are below.

Suspend and wake-up glitches at times when I have my dock connected. Usually, the monitors refuse to wake up and I have to reboot or rescan Thunderbolt devices. I have a workaround for this, but it is still a bit annoying.
Hibernate is not practical because of the large memory.
Screen recording with OBS can be a bit tricky to set up and get right on Wayland, especially with niri. The built-in DMS screen recorder is good for quick captures but not for longer sessions or when I need more control. I'm still tweaking my OBS setup for this.
The Flow Z13 has a cool transparent window on the back with RGB lights. It doesn't work under Linux. I'm planning to try and fix that with Qwen3.6 27B and OpenCode. It would be a complex hardware project that could test the model's ability.
ROCm/HIP support on new integrated GPUs needs patience. I haven't had any issues lately, so maybe it has stabilized.

None of these are deal breakers for me. Most are either already fixed in my private config or on my TODO list.

Conclusion

This is easily the most interesting Linux machine I have used so far. My 2019 setup was beautiful, my 2021 setup was sleek, and this one feels like a proper local-first AI development workstation.

Vanilla Arch gives me the latest bits. Niri gives me a workflow that fits both the tiny built-in screen and my ultrawide monitor. DMS gives me the desktop polish without a full desktop environment. And OpenCode plus llama.cpp gives me an AI coding assistant that can run without the cloud.

It is not the right setup for everyone. If you want a machine that never asks you to think about kernels, ROCm, compositor configs, or model files, this is probably not it. But for me, this is exactly the kind of developer machine that sparks joy.

The right tool for the right job.

If you like this article, please leave a like or a comment.

You can follow me on Bluesky and LinkedIn.

A Passwordless Future: Passkeys for Developers

Deepu K Sasidharan — Fri, 16 Feb 2024 15:53:42 +0000

Originally published at auth0.com

So, first things first, why do we even need to go passwordless? After all, passwords have been around for over 1000 years, right? And we were all happily sharing our Netflix passwords.

The Password Problem

The most important reason to go passwordless would be the password problem. According to Verizon's 2023 Data Breach Investigations Report, stolen credentials and phishing account for over 65% of all data breaches.

Most of the password problems are human problems in my opinion. Because passwords rely on us humans to remember them and to not share them and do a bunch of other stuff. History has repeatedly, a quadrillion times, shown us that we are not good at doing these things. So it is an us problem rather than the technology itself.

These are the main problems with passwords:

Knowledge-based:
- People can be socially engineered, quite easily, to divulge passwords, or other information that can be used to get to the password.
- In this day and age there are just too many passwords to remember. If passwords are easy to remember they are also easy to guess. Complex passwords are not easy to remember, so we end up reusing passwords.
Phishing: Phishing websites can easily harvest passwords from even the most tech-savvy.
Remote Replay: Accounts can be accessed remotely using harvested passwords.
Data Breach: Applications become a target for data breaches when they store passwords.
Share and Reuse: Sharing and reusing passwords makes them even more vulnerable.
Password management: Passwords are not just a hassle for the end users, they are a hassle on the server side as well. Because
- We need to build password recovery and reset flows.
- We need multi-factor authentication flows to secure them further.
- They need to be reset regularly in some use cases.

Did you know it could cost around 70$ to reset a password?

Of course, password managers help with some aspects of this and everyone should use one. But they are still an overhead and not very convenient for everyone, especially non-tech folks.

What is Passwordless?

The obvious solution for the password problem is to go passwordless. So what exactly is passwordless?

If you can verify a user's identity with something other than a password as the first factor of authentication, it is passwordless. We are doing this every day to unlock our phones and laptops using our fingerprints, faces, and so on.

There are a few passwordless methods that you might have seen here and there. Like:

Biometric authentication
Magic links
SMS/Email One-Time Password (OTP)
Push notifications

But most of these methods are not secure enough to replace a password + Multi-Factor Authentication (MFA) combination.

Passkeys

Passkeys are a password replacement that provide faster, easier, and more secure sign-ins to websites and apps across a user’s devices. Unlike passwords, passkeys are resistant to phishing, are always strong, and are designed so that there are no shared secrets.

— FIDO Alliance

This is where passkeys come into the picture. A secure passwordless future is the one offered by passkeys in my opinion. You probably already encountered passkeys since Google and GitHub have been rolling it out to all users recently. If you haven't set them up yet, you should!

A passkey is a unique cryptographic key pair that allows you to access online services without using passwords. It is based on asymmetric public-key cryptography.

Before we dive deep into passkeys let's look at some of the underlying technologies that make passkeys possible.

Public-key cryptography

Asymmetric public key cryptography involves a pair of mathematically linked keys: a public key, which is shared openly, and a private key, kept secret by the owner.

This key pair can be used for encryption. When a message is encrypted with the public key, only the corresponding private key can decrypt it, ensuring confidentiality.

The same key pair can also be used for digital signatures. A message signed with a private key can be verified with the public key, authenticating the sender's identity.

Passkeys use the signature verification mechanism. These keys are generated using a cryptographic algorithm, such as RSA or ECC.

Authenticator

An authenticator is a hardware or software entity that can create and store public-private key pairs which can be used for user registration and authentication. There are two types of authenticators:

Platform authenticators: An authenticator built into a user's device. For example, TouchID and FaceID from Apple, smartphone authenticators, Windows Hello, and so on.
Roaming authenticators: A removable authenticator usable with any device the user is trying to sign in from. They are attached using USB, NFC, and/or Bluetooth. For example, security keys like YubiKey, Google Titan and smartphones.

FIDO

FIDO stands for Fast IDentity Online. FIDO is a global authentication standard based on public key cryptography developed by the FIDO Alliance. It aims to solve all our password problems. FIDO Credentials are cryptographic key pairs that can be used for authentication.

Passkeys are made possible by the FIDO2 standard which is made up of Web Authentication (WebAuthn) and Client to Authenticator Protocol (CTAP).

Web Authentication

Web Authentication is a W3C recommendation that lets a webpage use a set of JavaScript APIs to talk to authenticators.

The WebAuthn architecture consists of three main entities:

Authenticator: Platform or roaming authenticators that let a user authenticate by confirming their presence.
Relying Party: A server (custom implementation or an Identity Provider like Auth0) that requires authentication. It issues challenges and stores public keys.
Client: A client consists of the user's browser. The client relays information between an authenticator and a relying party.

Client to Authenticator Protocol

The FIDO Client to Authenticator Protocol is used for communications with authenticators over a variety of transports like USB, NFC, and Bluetooth. It is used to send requests from WebAuthn to authenticators.

Passkeys are passwordless FIDO credentials implemented using WebAuthn.

Passkeys were originally called FIDO Multi-Device Credentials implemented with the WebAuthn. But recently that definition has evolved to mean any passwordless FIDO credentials that are discoverable by the browser. Passkeys are still evolving and hence this could change as well. But for simplicity let's stick to passkeys as that is the term used by the FIDO Alliance.

Types of passkeys

Passkeys have two variants. Synced passkeys (sometimes referred to as multi-device passkeys) and device-bound passkeys (sometimes referred to as hardware-bound passkeys).

Synced passkeys

Synced passkeys have a better user experience since the private keys are end-to-end encrypted and synced to the cloud. For example, on the Apple ecosystem, the private key is synced on your iCloud Keychain and you can register on one device and log in to any synced Apple device. The same goes for the Google ecosystem using the Chrome browser and Google Password Manager. Or you can use a password manager like BitWarden or 1Password to store your passkeys.
This kind of passkeys can be restored on new devices. But they are less secure than single device-bound passkeys since your private key is on the cloud and theoretically can be breached.

Device-bound passkeys

In the device-bound passkeys, the private key stays on the device itself and you need to authenticate using the same authenticator used for registration. It is slightly less convenient but more secure than synced passkeys. The relying party must support registering multiple credentials for a user so that backup keys can be registered, which is a best practice for device-bound passkeys.

For example, I use a YubiKey with a fingerprint reader since I'm on Linux, and my passkeys are device-bound to that YubiKey. I don't get any roaming or backup benefits like in the Apple or Google ecosystem. But it's not a big deal, in my opinion, since I can register multiple YubiKeys as backup and use them on any device, and it's more secure and way more convenient than password + MFA.

How does it work?

Let's see how user registration and authentication work with passkeys.

User registration

First, let's see how the registration flow works.

The user begins the registration flow. The relying party provides a randomly generated challenge string.
The navigator.credentials.create() method of the WebAuthn API is invoked and the user provides approval using their authenticator.
The authenticator creates a private-public key pair which is unique for the relying party's domain and the user. The private key is used to sign the challenge.
- The private key is stored on the authenticator.
- For synced passkeys, the private key is also synced to a cloud service for backup and roaming (This is the only place where synced passkeys differ from device-bound passkeys).
- An attestation object is created which contains the public key, signed challenge, credential ID, and certificate.
The attestation object and other metadata are then passed to the relying party by the client-side implementation. The relying party verifies the signed challenge using the public key and registers the user by storing the public key and credential ID along with the user details.

User authentication

Now, let's see how the login flow works, which is quite similar except for the third step.

The user begins the login flow. The relying party provides a randomly generated challenge string.
The navigator.credentials.get() method of the WebAuthn API is invoked and the user provides approval using their authenticator.
The authenticator retrieves the private keys for the relying party's domain name.
- For synced passkeys, if the device is new, the private key is synced from a cloud service if available (This is the only place where synced passkeys differ from device-bound passkeys).
- The user selects the private key for their username. The private key is used to sign the challenge.
- An assertion object is created which contains the signed challenge and credential ID.
The assertion object and other metadata are then passed to the relying party by the client-side implementation. The relying party verifies the signed challenge using the public key stored for the user and authenticates the user.

Why Passkeys?

Let's see why we need passkeys to replace passwords.

Passkeys are superior to password + traditional OTP MFA in terms of security and usability and they are as secure and more convenient than password + FIDO MFA. Most importantly, you don’t have to remember anything (unless you are like me and forget your YubiKeys all the time).

Discoverable: Passwords are knowledge-based but passkeys are discoverable credentials and the browser can autofill them for a service making it unnecessary for you to remember even usernames. It doesn’t rely on something you know, instead, it relies on something you have or something you are which is more secure from hacking and social engineering.
Phishing resistant: Passkeys cannot be phished as they rely on public key cryptography and are bound to the domain name of the website, making it impossible to work on a spoofed website.
Remote attack resistant: Passkeys rely on physical keys, like biometric sensors of platform authenticators or roaming authenticators like YubiKey, hence cannot be remotely breached.
Breach resistant: The website only stores the public key of a user which is useless to an attacker on a data breach on the server side. This makes the server less attractive to hackers.
Not reusable and shareable*: They cannot be reused as they are unique per service and user combination and cannot be shared.
- *except for Apple which lets you share a passkey by air-dropping it 🤷🏽.
Easier management: Passkeys are scalable. Synced passkeys are backed up and replicated across your devices by services like iCloud Keychain, Google Password Manager, Bitwarden, and so on. This makes recovery part of the platform rather than the application.
Cross-device authentication: Passkeys can also perform cross-device authentication regardless of ecosystem or platform. For example, you can simply use your Android phone as an authenticator for your Apple laptop.

The security of passkeys is way better than most other combinations. When it comes to user experience, though it is subjective, I think it outperforms all other combinations.

How Does Passkeys Differ from WebAuthn Multi-Factor Authentication?

Technically they are very similar since both are implemented with WebAuthn and in the case of device-bound passkeys they are even more similar. However, some differences set them apart.

Passkeys are Discoverable Credentials and are entirely stored on the authenticator. This means for hardware keys like YubiKey, the private key is stored on the key itself and hence can only hold what its memory allows. They are client-side discoverable during authentication ceremonies and can be used in the autofill UI of the browser. WebAuthn/FIDO-based MFA implementations are Non-Discoverable Credentials or Server-side Credentials and hence are not client-side discoverable. They are stored server-side and the private key is encrypted and sent to the relying party, hence there is no storage limitation on the hardware key.
WebAuthn MFA does not have a synced option, passkeys do.
In the case of synced passkeys there are even more differences in terms of usability and security. For example, enrollment needs to be done only once and the private keys are synced to a cloud.

The most important difference is that passkeys can be used as first-factor authentication whereas WebAuthn MFA can only be used as a second-factor after user registration with a password.

Challenges

There are still some challenges when it comes to passkeys:

OS/Browser support: It is dependent on the OS and Browser to implement the specs, and we all know how that turns out right? This means the support may not be uniform and can become fragmented and we might never have the same experience everywhere.
Cloud vendor reliance: It relies on companies like Apple, Google, and Microsoft to save the private keys in their cloud securely and protect them from breaches.
Enterprise use cases: Enterprise users might want more control and flexibility which could be a problem. For example, if an enterprise doesn’t allow iCloud or Google Chrome on their computer, synced passkeys will not work there, only device-bound passkeys will work.
Reset & Recovery: There are no default recovery flows for device-bound passkeys, and applications might still need to implement recovery & reset flows to accommodate all use cases.

Browser and OS compatibility is still catching up. As of writing, Chrome has the best support and the Apple ecosystem has the most seamless experience, especially for platform authenticators. Linux does not have any support for platform authenticators. Roaming authenticators have great support on all platforms.

Conclusion

Passkeys are the future of secure authentication. They are more secure and more convenient than passwords and traditional MFA. They are also more secure and more convenient than other passwordless methods like magic links, SMS/Email OTP, and push notifications. The wider adoption of passkeys could finally solve the password problem and make the internet a safer place.

Passkeys are not just for the future, they are here now. You can start using them today. If you are a developer, you can start implementing them in your applications using Auth0. If you are a user, you can start using them on services that support them.

Resources

Check out this follow-up blog post.

Deepu K Sasidharan for Okta

Jan 2 '24

A Passwordless Future: Passkeys for Java Developers

#java #spring #passkeys #webauthn

9 min read

I hope that you found this article helpful. Here are some additional resources to learn more about WebAuthn and passkeys.

If you like this article, please leave a like or a comment.

You can follow me on Mastodon and LinkedIn.

A Passwordless Future: Passkeys for Java Developers

Deepu K Sasidharan — Tue, 02 Jan 2024 15:39:21 +0000

Originally published at auth0.com

This blog post is a continuation of my previous blog post on passkeys. In this post, you will learn how to implement passkeys using Auth0 and the WebAuthn4j library on your Java applications.

Deepu K Sasidharan for Okta

Feb 16 '24

A Passwordless Future: Passkeys for Developers

#security #passkeys #webauthn

11 min read

If you have not read the previous blog post, I would highly recommend you read it first to understand the basics of passkeys and WebAuthn.

Passkeys

A passkey is a unique cryptographic key pair that allows you to access online services without using passwords. It is based on asymmetric public-key cryptography.

Passkeys are passwordless FIDO credentials implemented using WebAuthn.

Why Passkeys?

Let's See Passkeys in Action with Auth0

Let's build a Spring Boot web app and secure it using passkeys with the help of Auth0 by Okta. You can find a sample app on GitHub if you just want to try passkeys.

Before you get started, you will need the following:

Java 17 or higher. You can use SDKMAN! to install Java if you don't have it already.
A free Auth0 account. Sign up if you don't have one already.
The Auth0 CLI. Install the CLI if you don't have it and log in to your Auth0 account using the auth0 login command.

Create a Spring Boot application

Create a new Spring Boot application using the Spring Initializr. You can use the web version or the curl command below. Use the default for most of the options. For the dependencies, select web, and okta. For the build tool, select Gradle.

curl -G https://start.spring.io/starter.tgz \
  -d dependencies=web,okta \
  -d baseDir=passkey-demo \
 | tar -xzvf -

The web dependency provides Spring Web MVC with basic HTTP REST functionality.
The okta dependency provides the Okta Spring Boot Starter, which provides the required dependencies and configuration to add OIDC authentication to your application.

Add a web controller

Imports are omitted for brevity so make sure to import them using your IDE.

Open the created starter application in your favorite IDE. Add a simple web controller to the application. Create a new file src/main/java/com/example/demo/HomeController.java with the following content:

@RestController
class HomeController {
    @GetMapping("/")
    public String home(@AuthenticationPrincipal OidcUser user) {
        return "Hello, " + user.getFullName() + "!";
    }
}

This controller will handle requests to the / path.

If you run the application using ./gradlew bootRun, you will see a login page from the Okta Spring Boot starter instead of your home screen. This is OK, and you will be able to configure this soon. You can comment out the okta-spring-boot-starter dependency in the build.gradle file if you want to run the application at this point.

Enable passkeys on your Auth0 tenant

Log in to your Auth0 Dashboard and navigate to Authentication > Database > Username-Password-Authentication.
1. If the second tab says Authentication Methods, your tenant supports passkeys, proceed to the next step.
2. If the second tab says Password Policy, your tenant doesn't support passkeys, Create a new tenant and proceed to the next step.
Navigate to Authentication > Authentication Profile and select Identifier First. Save your changes.
Navigate to Authentication > Database > Username-Password-Authentication and select the Authentication Methods tab and enable Passkey.

Configure OIDC Authentication with Auth0

Configure the application to use Auth0 as the Identity Provider (IdP). You can use the Auth0 CLI to create a new authorization server application. Run the following command to create a new application:

auth0 apps create \
  --name "Spring Boot Passkeys" \
  --description "Spring Boot Example" \
  --type regular \
  --callbacks http://localhost:8080/login/oauth2/code/okta \
  --logout-urls http://localhost:8080 \
  --reveal-secrets

The --type option specifies that you use a regular web application.
The --callbacks option specifies the callback URL for the application.
The --logout-urls option specifies the logout URL for the application.
The --reveal-secrets option will display the client secret in the output.

You can also use the auth0 apps update command to update the application with the callback and logout URLs.

Note down the Auth0 issuer (for example, https://dev-12345678.us.auth0.com/), CLIENT ID, and CLIENT SECRET from the output. You will use these values in the next step.

Configure the Spring Boot application

Configure the application by creating an application.properties file in the applications root folder with the following content:

# trailing `/` is important for issuer URI
okta.oauth2.issuer=https://<AUTH0_domain>/
okta.oauth2.client-id=<AUTH0_clientId>
okta.oauth2.client-secret=<AUTH0_clientSecret>

Add the application.properties file to the .gitignore file to avoid committing the secrets to the repository.

Run the application

To run the application, execute the following command:

./gradlew bootRun

The application should start successfully. Navigate to http://localhost:8080 in your browser. You will be redirected to the Auth0 universal login page for authentication.

Click on the Sign up link to register a new user. Enter any email address and click Continue. You will now be prompted to register a passkey.

Create a passkey using your platform authenticator or roaming authenticator like YubiKey. Once you have registered a passkey, you should be redirected back to the application and see the welcome message.

Open a new incognito window and navigate to http://localhost:8080. You will be prompted to sign in using your passkey. Once you have signed in, you will see the welcome message.

Isn't that cool? You just implemented passkeys in your Spring Boot application with so little effort thanks to Auth0.

WebAuthn for Java

Though Web Authentication’s user experience is a client-side implementation using JavaScript, the backend or Relying party can be a Java server. Ideally using an IdP like Auth0 would be the best option since it takes care of all the heavy lifting for you. But if you want to implement it yourself and walk the harder path, you can use one of the below libraries.

WebAuthn4j: A 100% FIDO2 conformant library with support for all attestation formats and validation. It is used by Keycloak and Spring Security.
java-webauthn-server: A library from Yubico that supports many attestation format. But it is not 100% FIDO2 conformant.

WebAuthn4j with Spring Security in Action

Let's look at a simple Spring Boot application that uses passkeys for authentication without using an IdP. You can find the sample app on GitHub.

Clone and run the application

Start by cloning the application.

git clone https://github.com/deepu105/webauthn4j-spring-boot-passkeys-demo.git

cd webauthn4j-spring-boot-passkeys-demo
./gradlew bootRun

Visit http://localhost:8080/. You should see the below screen. Try registering a new user with passkeys and log in.

WebAuthn4j configuration

Let's look at some of the important parts of the application.

The webauthn4j-spring-security-core dependency, in build.gradle file, provides the Spring Security integration for WebAuthn4j.
The required beans for WebAuthn4j are configured in src/main/java/com/example/demo/config/WebSecurityBeanConfig.java.
The InMemoryWebAuthnAuthenticatorManager is used to keep things simple but it means authenticator data is lost on application restart. For production use, it is better to implement the WebAuthnAuthenticatorManager interface and persist credential IDs for users.
WebAuthn4j is configured using the standard Spring Security filter chain in src/main/java/com/example/demo/config/WebSecurityConfig.java.

@Configuration
@EnableWebSecurity
public class WebSecurityConfig {
    ...
    @Bean
    public SecurityFilterChain filterChain(HttpSecurity http, AuthenticationManager authenticationManager) throws Exception {
        // WebAuthn Login
        http.apply(WebAuthnLoginConfigurer.webAuthnLogin())
            .defaultSuccessUrl("/", true)
            .failureHandler((request, response, exception) -> {
                logger.error("Login error", exception);
                response.sendRedirect("/login?error=Login failed: " + exception.getMessage());
            })
            .attestationOptionsEndpoint()
            .rp()
            .name("WebAuthn4J Passkeys Demo")
            .and()
            .pubKeyCredParams(
                // supported algorithms for cryptography
                new PublicKeyCredentialParameters(PublicKeyCredentialType.PUBLIC_KEY, COSEAlgorithmIdentifier.ES256),
                new PublicKeyCredentialParameters(PublicKeyCredentialType.PUBLIC_KEY, COSEAlgorithmIdentifier.RS256)
            )
            .attestation(AttestationConveyancePreference.DIRECT)
            .extensions()
            .uvm(true)
            .credProps(true)
            .extensionProviders()
            .and()
            .assertionOptionsEndpoint()
            .extensions()
            .extensionProviders();

        http.headers(headers -> {
            // 'publickey-credentials-get *' allows getting WebAuthn credentials to all nested browsing contexts (iframes) regardless of their origin.
            headers.permissionsPolicy(config -> config.policy("publickey-credentials-get *"));
            // Disable "X-Frame-Options" to allow cross-origin iframe access
            headers.frameOptions(Customizer.withDefaults()).disable();
        });

        // Authorization
        http.authorizeHttpRequests(authz -> authz
            .requestMatchers(HttpMethod.GET, "/login").permitAll()
            .requestMatchers(HttpMethod.POST, "/signup").permitAll()
            .anyRequest().access(getWebExpressionAuthorizationManager("@webAuthnSecurityExpression.isWebAuthnAuthenticated(authentication)"))
        );

        http.exceptionHandling(eh -> eh.accessDeniedHandler((request, response, accessDeniedException) -> {
            logger.error("Access denied", accessDeniedException);
            response.sendRedirect("/login");
        }));

        http.authenticationManager(authenticationManager);

        // As WebAuthn has its own CSRF protection mechanism (challenge), CSRF token is disabled here
        http.csrf(csrf -> {
            csrf.csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse());
            csrf.ignoringRequestMatchers("/webauthn/**");
        });

        return http.build();
    }
    ...
}

The endpoints are configured in src/main/java/com/example/demo/web/WebAuthnSampleController.java. The / and /login endpoints are quite simple and self-explanatory. The /signup endpoint handles the WebAuthn registration request using WebAuthn4j. The request is first validated using WebAuthnRegistrationRequestValidator and then the authenticator is created using WebAuthnAuthenticatorManager.

@Controller
public class WebAuthnSampleController {
    ...
    @PostMapping(value = "/signup")
    public String create(HttpServletRequest request, @Valid @ModelAttribute("userForm") UserCreateForm userCreateForm, BindingResult result, Model model, RedirectAttributes redirectAttributes) {
        try {
            if (result.hasErrors()) {
                model.addAttribute("errorMessage", "Your input needs correction.");
                logger.error("User input validation failed.");
                return VIEW_LOGIN;
            }
            WebAuthnRegistrationRequestValidationResponse registrationRequestValidationResponse;
            try {
                registrationRequestValidationResponse = registrationRequestValidator.validate(
                    request,
                    userCreateForm.getClientDataJSON(),
                    userCreateForm.getAttestationObject(),
                    userCreateForm.getTransports(),
                    userCreateForm.getClientExtensions()
                );
            } catch (WebAuthnException | WebAuthnAuthenticationException e) {
                model.addAttribute("errorMessage", "Authenticator registration request validation failed. Please try again.");
                logger.error("WebAuthn registration request validation failed.", e);
                return VIEW_LOGIN;
            }
            var username = userCreateForm.getUsername();
            var authenticator = new WebAuthnAuthenticatorImpl(
                "authenticator",
                username,
                registrationRequestValidationResponse.getAttestationObject().getAuthenticatorData().getAttestedCredentialData(),
                registrationRequestValidationResponse.getAttestationObject().getAttestationStatement(),
                registrationRequestValidationResponse.getAttestationObject().getAuthenticatorData().getSignCount(),
                registrationRequestValidationResponse.getTransports(),
                registrationRequestValidationResponse.getRegistrationExtensionsClientOutputs(),
                registrationRequestValidationResponse.getAttestationObject().getAuthenticatorData().getExtensions()
            );
            try {
                webAuthnAuthenticatorManager.createAuthenticator(authenticator);
            } catch (IllegalArgumentException ex) {
                model.addAttribute("errorMessage", "Registration failed. The user may already be registered.");
                logger.error("Registration failed.", ex);
                return VIEW_LOGIN;
            }
        } catch (RuntimeException ex) {
            model.addAttribute("errorMessage", "Registration failed by unexpected error.");
            logger.error("Registration failed.", ex);
            return VIEW_LOGIN;
        }
        model.addAttribute("successMessage", "User registration successful. Please login.");
        return VIEW_LOGIN;
    }
}

Client-side configuration

The file src/main/resources/templates/login.html handles login and sign-up. The login button will invoke the navigator.credentials.get() API and the register button will invoke the navigator.credentials.create() API. The buttons submit the corresponding forms with the input data in them. All inputs except the username field are hidden as their data will be set using JavaScript.

WebAuthn4j exposes /webauthn/attestation/options endpoint in the application to fetch the registration options. Some of the option parameters need to be decoded from base64URL. The base64url-arraybuffer library is used for this. The options are then passed to the navigator.credentials.create() API. The response from the API is then updated to the form fields and submitted to the /signup endpoint.

document.getElementById('signup-form').addEventListener('submit', async (e) => {
    e.preventDefault();
    const userHandle = document.getElementById('userHandle').value;
    const username = document.getElementById('username').value;
    try {
        const optionsRes = await fetch('/webauthn/attestation/options');
        const options = await optionsRes.json();
        const publicKey = {
            ...options,
            challenge: base64url.decode(options.challenge, true),
            user: {
                id: base64url.decode(userHandle, true),
                name: username,
                displayName: username,
            },
            excludeCredentials: options.excludeCredentials.map((credential) => ({
                ...credential,
                id: base64url.decode(credential.id, true),
            })),
            authenticatorSelection: {
                requireResidentKey: true,
                userVerification: 'discouraged',
            },
        };
        const credential = await navigator.credentials.create({ publicKey });
        document.getElementById('clientDataJSON').value = base64url.encode(credential.response.clientDataJSON);
        document.getElementById('attestationObject').value = base64url.encode(credential.response.attestationObject);
        document.getElementById('clientExtensions').value = JSON.stringify(credential.getClientExtensionResults());
        document.getElementById('signup-form').submit();
    } catch (error) {
        console.error('Error:%s, Message:%s', error.name, error.message);
    }
});

WebAuthn4j exposes /webauthn/assertion/options endpoint in the application to fetch the authentication options. Some of the option parameters need to be decoded from base64URL. The options are then passed to the navigator.credentials.get() API. The response from the API is then updated to the form fields and submitted to the /login endpoint.

document.getElementById('login-form').addEventListener('submit', async (e) => {
    e.preventDefault();
    try {
        const optionsRes = await fetch('/webauthn/assertion/options');
        const options = await optionsRes.json();
        const publicKey = {
            ...options,
            challenge: base64url.decode(options.challenge, true),
            userVerification: 'preferred',
        };
        const credential = await navigator.credentials.get({ publicKey });
        document.getElementById('credentialId').value = credential.id;
        document.getElementById('loginClientDataJSON').value = base64url.encode(credential.response.clientDataJSON);
        document.getElementById('authenticatorData').value = base64url.encode(credential.response.authenticatorData);
        document.getElementById('signature').value = base64url.encode(credential.response.signature);
        document.getElementById('loginClientExtensions').value = JSON.stringify(credential.getClientExtensionResults());
        document.getElementById('login-form').submit();
    } catch (error) {
        console.error('Error:%s, Message:%s', error.name, error.message);
    }
});

Conclusion

You have now learned:

How to implement passkeys using an IdP like Auth0.
You also learned how to configure the application to use Auth0 as the Identity Provider and how to configure Auth0 for passkey support.
Roll your own passkey solution using WebAuthn4j and Spring Security.

Passkeys are the future of authentication. They are more secure and convenient than traditional passwords and OTPs. Though you could roll your own solution using WebAuthn4j, it is always better to use an IdP like Auth0 to handle the heavy lifting for you and take care of all the security best practices.

Resources

I hope that you found this article helpful. Here are some additional resources to learn more about WebAuthn and passkeys.

If you like this article, please leave a like or a comment.

You can follow me on Mastodon and LinkedIn.

Deploy Secure Spring Boot Microservices on Amazon EKS Using Terraform and Kubernetes

Deepu K Sasidharan — Thu, 23 Nov 2023 14:34:44 +0000

Originally published at auth0.com

When it comes to infrastructure, public clouds are the most popular choice these days, and Amazon Web Services (AWS) is the go-to option. If you are working with microservices, orchestrating their deployments becomes essential. Kubernetes is the de-facto choice for orchestrating microservices, and most public cloud providers offer managed Kubernetes services. For AWS, the managed Kubernetes service is Amazon Elastic Kubernetes Service (EKS).

Deploying and managing microservices on the public cloud is not without its challenges. Each cloud service has its own complexities. Among them, Amazon EKS is one of the most flexible but also one of the most difficult Kubernetes services to use. EKS utilizes clever orchestrations on top of other AWS services like EC2, EBS, and more.

To run a microservice stack on EKS, you will need to spend extra time and effort setting it up and managing it. This is where infrastructure as code (IaC) tools like Terraform come in handy.

Overview

So here is what you will learn to do today:

Scaffold a Java microservice stack using JHipster, Spring Boot, and Spring Cloud
Create an EKS cluster, Virtual Private Cloud (VPC), subnets, and required Kubernetes add-ons using Terraform on AWS
Set up Terraform scripts for OpenID Connect (OIDC) authentication for the microservice stack using Auth0 by Okta
Build and deploy the microservice stack to the cluster

Prerequisites:

AWS account with the IAM permissions to create EKS clusters
AWS CLI installed and configured
kubectl
Docker installed and configured
Terraform CLI
Java 11+
Auth0 CLI
[Optional] JHipster CLI
[Optional] KDash

Why Terraform, Why Not Cloud Formation?

At this point, the first question that might pop up in your mind would be, "Why not use CloudFormation?". It's a good question; after all, CloudFormation is built by AWS and hence sounds like an excellent solution to manage AWS resources. But anyone who has tried both CloudFormation and Terraform will probably tell you to forget that CloudFormation even exists. I think CloudFormation is far more complex and less developer-friendly than Terraform. You also need to write a lot more boilerplate with CloudFormation in YAML or JSON. Yikes! In contrast, Terraform is elegant and concise, and the syntax is easier to read and write. It's cross-platform, developer-friendly, and does not require a lot of ramp-up time.

Okay, now that we have that sorted, let's dive into the steps to deploy our microservice stack on EKS.

Scaffold a Java Microservice Stack Using JHipster

To start, you will scaffold a Java microservice stack using JHipster, Spring Boot, and Consul. JHipster is an excellent tool to generate a microservice stack with Spring Boot, Angular/React/Vue.js, and other modern frameworks. You can use another microservice stack if you want. If you prefer using the same application as in this demo, then you can either scaffold it using JHipster JDL or clone the sample repository from GitHub. Here is how you can scaffold your microservice stack using JHipster:

Option 1: Scaffold the microservice stack using JHipster

Use JHipster version 7.9.3 or 8.0.0

mkdir jhipster-microservice-stack
cd jhipster-microservice-stack
# download the JDL file.
jhipster download https://raw.githubusercontent.com/oktadev/auth0-jhipster-k8s-eks-microservices-example/main/apps.jdl
# Update the `dockerRepositoryName` property to use your Docker Repository URI/Name.
# scaffold the apps.
jhipster jdl apps.jdl

Option 2: Clone the sample repository

git clone https://github.com/oktadev/auth0-jhipster-k8s-eks-microservices-example

In either case, remember to change the Docker repository name in the JDL file and Kubernetes manifests to match your Docker repository.

The JHipster scaffolded sample application has a gateway application and two microservices. It uses Consul for service discovery and centralized configuration.

Create an EKS Cluster Using Terraform

Now let us move on to the important part of the tutorial. Creating an EKS cluster in AWS is not as straightforward as in other cloud platforms. You need to also create a lot more resources for everything to work correctly without surprises. You will be using a bunch of Terraform providers to help with this, and you will also use some prebuilt Terraform modules like AWS VPC Terraform module and Amazon EKS Blueprints for Terraform to reduce the amount of boilerplate you need to write.

These are the AWS resources and VPC architecture you will create:

Build the Terraform configuration

First, make sure you use a specific version of the providers as different versions might use different attributes and features. Create a versions.tf file:

mkdir terraform
cd terraform
touch versions.tf

Add the following to the file:

terraform {
  required_version = ">= 1.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.47"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.20"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.9"
    }
  }
}

Next, you need to define variables and configure the providers. Create a config.tf file:

touch config.tf

Add the following to the file:

# ##  To save state in s3. Update to suit your needs
# backend "s3" {
#   bucket = "create-an-s3-bucket-and-provide-name-here"
#   region = local.region
#   key    = "eks-cluster-with-new-vpc/terraform.tfstate"
# }

variable "region" {
  default     = "eu-west-1"
  description = "AWS region"
}

data "aws_availability_zones" "available" {}

locals {
  name            = "okta-auth0-jhipster-eks"
  region          = var.region
  cluster_version = "1.27"
  instance_types  = ["t2.large"] # can be multiple, comma separated

  vpc_cidr = "10.0.0.0/16"
  azs      = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    Blueprint  = local.name
    GitHubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
  }
}

provider "aws" {
  region = local.region
}

# Kubernetes provider
# You should **not** schedule deployments and services in this workspace.
# This keeps workspaces modular (one for provision EKS, another for scheduling
# Kubernetes resources) as per best practices.
provider "kubernetes" {
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)

  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    # This requires the awscli to be installed locally where Terraform is executed
    args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
  }
}

provider "helm" {
  kubernetes {
    host                   = module.eks.cluster_endpoint
    cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)

    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      command     = "aws"
      # This requires the awscli to be installed locally where Terraform is executed
      args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
    }
  }
}

You can uncomment the backend section above to save the state in AWS S3 instead of your local filesystem. This is recommended for production setup so that everyone in a team has the same state. This file defines configurable and local variables used across the workspace and configures some of the providers used. The Kubernetes provider is included in this file so the EKS module can complete successfully. Otherwise, it throws an error when creating kubernetes_config_map.aws_auth. The Helm provider is used to install Kubernetes add-ons to the cluster.

Build the VPC

Next up, you need a VPC, subnets, route tables, and other networking bits. You will use the vpc module from the terraform-aws-modules repository. This module is a wrapper around the AWS VPC module. It makes it easier to configure VPCs and all the other required networking resources. Create a vpc.tf file:

touch vpc.tf

Add the following to the file:

#---------------------------------------------------------------
# VPC, Subnets, Internet gateway, Route tables, etc.
#---------------------------------------------------------------
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k)]
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 10)]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  # Manage so we can name them
  manage_default_network_acl    = true
  default_network_acl_tags      = { Name = "${local.name}-default" }
  manage_default_route_table    = true
  default_route_table_tags      = { Name = "${local.name}-default" }
  manage_default_security_group = true
  default_security_group_tags   = { Name = "${local.name}-default" }

  public_subnet_tags = {
    "kubernetes.io/cluster/${local.name}" = "shared"
    "kubernetes.io/role/elb"              = 1
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${local.name}" = "shared"
    "kubernetes.io/role/internal-elb"     = 1
  }

  tags = local.tags
}

This will create:

A new VPC, three private subnets, and three public subnets,
An Internet gateway and NAT gateway for the public subnets,
and AWS routes for the gateways, public/private route tables, and route table associations.

Build the EKS cluster

Now that you have the networking part done, you can build configurations for the EKS cluster and its add-ons. You will use the terraform-aws-modules to create the EKS cluster and eks_blueprints module from terraform-aws-eks-blueprintsto configure EKS add-ons.

Create a eks-cluster.tf file:

touch eks-cluster.tf

Add the following to the file:

#---------------------------------------------------------------
# EKS cluster, worker nodes, security groups, IAM roles, K8s add-ons, etc.
#---------------------------------------------------------------
module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = local.cluster_version
  cluster_endpoint_public_access = true
  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = module.vpc.private_subnets

  # EKS Addons
  cluster_addons = {
    aws-ebs-csi-driver = {
      most_recent = true
    }
    coredns    = {}
    kube-proxy = {}
    vpc-cni    = {}
  }
  eks_managed_node_group_defaults = {
    # Needed by the aws-ebs-csi-driver
    iam_role_additional_policies = {
      AmazonEBSCSIDriverPolicy = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
    }
  }
  eks_managed_node_groups = {
    initial = {
      instance_types = local.instance_types
      min_size       = 2
      max_size       = 4
      desired_size   = 3
      subnet_ids     = module.vpc.private_subnets
    }
  }

  tags = local.tags
}

module "eks_blueprints_addons" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons?ref=v4.32.1"

  eks_cluster_id        = module.eks.cluster_name
  eks_cluster_endpoint  = module.eks.cluster_endpoint
  eks_cluster_version   = module.eks.cluster_version
  eks_oidc_provider     = module.eks.oidc_provider
  eks_oidc_provider_arn = module.eks.oidc_provider_arn

  # K8S Add-ons
  enable_aws_load_balancer_controller = true
  enable_metrics_server               = true
  enable_aws_cloudwatch_metrics       = false

  tags = local.tags
}

# To update local kubeconfig with new cluster details
resource "null_resource" "kubeconfig" {
  depends_on = [module.eks_blueprints_addons]
  provisioner "local-exec" {
    command = "aws eks --region ${local.region}  update-kubeconfig --name $AWS_CLUSTER_NAME"
    environment = {
      AWS_CLUSTER_NAME = local.name
    }
  }
}

The eks module definition creates:

EKS Cluster Control plane with one managed node group,
Cluster and node security groups and rules, IAM roles and policies required,
Amazon EKS add-ons ebs-csi-driver, vpc-cni, CoreDNS, and kube-proxy,
and AWS Key Management Service (KMS) configuration.

The eks_blueprints_addons module definition creates:

AWS Load Balancer Controller that provisions an AWS Network Load Balancer for distributing traffic,
and the Metrics Server.

The null_resource configuration updates your local kubeconfig file with the new cluster details. It's not a required step for provisioning but just a handy hack.

Finally, you can also define some outputs to be captured. Create a outputs.tf file:

touch outputs.tf

Add the following to the file:

output "vpc_private_subnet_cidr" {
  description = "VPC private subnet CIDR"
  value       = module.vpc.private_subnets_cidr_blocks
}

output "vpc_public_subnet_cidr" {
  description = "VPC public subnet CIDR"
  value       = module.vpc.public_subnets_cidr_blocks
}

output "vpc_cidr" {
  description = "VPC CIDR"
  value       = module.vpc.vpc_cidr_block
}

output "eks_cluster_id" {
  description = "EKS cluster ID"
  value       = module.eks.cluster_id
}

output "eks_cluster_name" {
  description = "EKS cluster Name"
  value       = module.eks.cluster_name
}

output "eks_managed_nodegroups" {
  description = "EKS managed node groups"
  value       = module.eks.eks_managed_node_groups
}

output "configure_kubectl" {
  description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
  value       = "aws eks update-kubeconfig --name ${module.eks.cluster_name} --alias ${module.eks.cluster_name}"
}

Provision the cluster

Our Terraform definitions are ready. Now you can provision the cluster.

First, ensure you have configured your AWS CLI to use the correct AWS account and a region that supports EKS. If not, run the following:

# Visit https://console.aws.amazon.com/iam/home?#/security_credentials for creating access keys
aws configure

Next, initialize Terraform workspace and plan the changes:

# Download modules and providers. Initialize state.
terraform init
# see a preview of what will be done
terraform plan

Review the plan and make sure everything is correct.

Now you can apply the changes:

terraform apply

Confirm by typing yes when prompted. This will take a while (15-20 minutes), so sit back and have a coffee or contemplate what led you to this point in life. 😉

Once the EKS cluster is ready, you will see the output variables printed to the console.

configure_kubectl = "aws eks update-kubeconfig --name okta-auth0-jhipster-eks --alias okta-auth0-jhipster-eks"
eks_cluster_name = "okta-auth0-jhipster-eks"
eks_managed_nodegroups = {
  "initial" = {
    "autoscaling_group_schedule_arns" = {}
    "iam_role_arn" = "arn:aws:iam::216713166862:role/initial-eks-node-group-20230628170956397500000007"
    "iam_role_name" = "initial-eks-node-group-20230628170956397500000007"
    "iam_role_unique_id" = "AROATE5I4QAHNFCURF5WI"
    "launch_template_arn" = "arn:aws:ec2:eu-west-1:216713166862:launch-template/lt-028654c46a256c879"
    "launch_template_id" = "lt-028654c46a256c879"
    "launch_template_latest_version" = 1
    "launch_template_name" = "initial-2023062817211846290000000e"
    "node_group_arn" = "arn:aws:eks:eu-west-1:216713166862:nodegroup/okta-auth0-jhipster-eks/initial-20230628172118695900000010/f2c48183-0cd0-f970-d405-0869ccddad37"
    "node_group_autoscaling_group_names" = [
      "eks-initial-20230628172118695900000010-f2c48183-0cd0-f970-d405-0869ccddad37",
    ]
    "node_group_id" = "okta-auth0-jhipster-eks:initial-20230628172118695900000010"
    "node_group_labels" = tomap(null) /* of string */
    "node_group_resources" = tolist([
      {
        "autoscaling_groups" = tolist([
          {
            "name" = "eks-initial-20230628172118695900000010-f2c48183-0cd0-f970-d405-0869ccddad37"
          },
        ])
        "remote_access_security_group_id" = ""
      },
    ])
    "node_group_status" = "ACTIVE"
    "node_group_taints" = toset([])
  }
}
vpc_cidr = "10.0.0.0/16"
vpc_private_subnet_cidr = tolist([
  "10.0.10.0/24",
  "10.0.11.0/24",
  "10.0.12.0/24",
])
vpc_public_subnet_cidr = tolist([
  "10.0.0.0/24",
  "10.0.1.0/24",
  "10.0.2.0/24",
])

You should see the cluster details if you run kdash or kubectl get nodes commands.

Note: The EKS cluster defined here will not come under AWS free tier; hence, running this will cost money, so delete the cluster as soon as you finish the tutorial to keep the cost within a few dollars.

Set Up OIDC Authentication Using Auth0 by Okta

You can proceed to deploy the sample application. You could skip this step if you used a sample that does not use Auth0 or OIDC for authentication.

Since you are using Terraform, you can set up the Auth0 application using the Auth0 Terraform provider. This will allow you to automate the setup of the Auth0 application and manage the addition of users, customizations, and such.

Create an Auth0 application

Open the versions.tf file in the terraform folder and add the following content to the required_providers section:

    auth0 = {
      source  = "auth0/auth0"
      version = "~> 0.49.0"
    }

Let's create a Terraform script that creates an Auth0 web application and required customizations. Create a file named auth0.tf in the terraform folder and add the following content:

provider "auth0" {
  domain        = "https://<your_auth0_domain_uri>"
  debug         = false
}

# Create a new Auth0 application for the JHipster app
resource "auth0_client" "java_ms_client" {
  name                = "JavaMicroservices"
  description         = "Java Microservices Client Created Through Terraform"
  app_type            = "regular_web"
  callbacks           = ["http://localhost:8080/login/oauth2/code/oidc"]
  allowed_logout_urls = ["http://localhost:8080"]
  oidc_conformant     = true

  jwt_configuration {
    alg = "RS256"
  }
}

# Configuring client_secret_post as an authentication method.
resource "auth0_client_credentials" "java_ms_client_creds" {
  client_id = auth0_client.java_ms_client.id

  authentication_method = "client_secret_post"
}

# Create roles for the JHipster app
resource "auth0_role" "admin" {
  name        = "ROLE_ADMIN"
  description = "Administrator"
}

resource "auth0_role" "user" {
  name        = "ROLE_USER"
  description = "User"
}

# Create an action to customize the authentication flow to add the roles and the username to the access token claims expected by JHipster applications.
resource "auth0_action" "jhipster_action" {
  name    = "jhipster_roles_claim"
  runtime = "node18"
  deploy  = true
  code    = <<-EOT
  /**
   * Handler that will be called during the execution of a PostLogin flow.
   *
   * @param {Event} event - Details about the user and the context in which they are logging in.
   * @param {PostLoginAPI} api - Interface whose methods can be used to change the behavior of the login.
   */
   exports.onExecutePostLogin = async (event, api) => {
     const namespace = 'https://www.jhipster.tech';
     if (event.authorization) {
         api.idToken.setCustomClaim(namespace + '/roles', event.authorization.roles);
         api.accessToken.setCustomClaim(namespace + '/roles', event.authorization.roles);
     }
   };
  EOT

  supported_triggers {
    id      = "post-login"
    version = "v3"
  }
}

# Attach the action to the login flow
resource "auth0_trigger_actions" "login_flow" {
  trigger = "post-login"

  actions {
    id           = auth0_action.jhipster_action.id
    display_name = auth0_action.jhipster_action.name
  }
}

# Create a test user. You can create more users here if needed
resource "auth0_user" "test_user" {
  connection_name = "Username-Password-Authentication"
  name            = "Jane Doe"
  email           = "jhipster@test.com"
  email_verified  = true
  password        = "passpass$12$12" # Don't set passwords like this in production! Use env variables instead.
  lifecycle {
    ignore_changes = [roles]
  }
}

resource "auth0_user_roles" "test_user_roles" {
  user_id = auth0_user.test_user.id
  roles   = [auth0_role.admin.id, auth0_role.user.id]
}

output "auth0_webapp_client_id" {
  description = "Auth0 JavaMicroservices Client ID"
  value       = auth0_client.java_ms_client.client_id
}

output "auth0_webapp_client_secret" {
  description = "Auth0 JavaMicroservices Client Secret"
  value       = auth0_client_credentials.java_ms_client_creds.client_secret
  sensitive   = true
}

You can find your Auth0 domain in the Auth0 dashboard or by running the auth0 tenants list command. The script above does the following:

The auth0_client resource definition creates an Auth0 Web application client conforming to the OIDC standard.
The auth0_client_credentials resource definition configures the client to use the client_secret_post authentication method.
The auth0_role resource definition creates two roles for the JHipster application.
The auth0_action resource definition creates an action that will be executed during the Auth0 post-login flow. This action will add the roles and the username to the ID and access token claims as expected by JHipster applications.
The auth0_user resource definition creates a test user.

Provision the Auth0 application

Now before you can run this script you need to create a machine-to-machine application in Auth0 so that Terraform can communicate with the Auth0 management API. This can be done using the Auth0 CLI. Please note that you also need to have jq installed to run the below commands. Run the following commands to create an application after logging into the CLI with the auth0 login command:

# Create a machine-to-machine application on Auth0
export AUTH0_M2M_APP=$(auth0 apps create \
  --name "Auth0 Terraform Provider" \
  --description "Auth0 Terraform Provider M2M" \
  --type m2m \
  --reveal-secrets \
  --json | jq -r '. | {client_id: .client_id, client_secret: .client_secret}')

# Extract the client ID and client secret from the output.
export AUTH0_CLIENT_ID=$(echo $AUTH0_M2M_APP | jq -r '.client_id')
export AUTH0_CLIENT_SECRET=$(echo $AUTH0_M2M_APP | jq -r '.client_secret')

This will create the application and set environment variables for the client ID and secret. This application needs to be authorized to use the Auth0 management API. This can be done using the commands below.

# Get the ID and IDENTIFIER fields of the Auth0 Management API
export AUTH0_MANAGEMENT_API_ID=$(auth0 apis list --json | jq -r 'map(select(.name == "Auth0 Management API"))[0].id')
export AUTH0_MANAGEMENT_API_IDENTIFIER=$(auth0 apis list --json | jq -r 'map(select(.name == "Auth0 Management API"))[0].identifier')
# Get the SCOPES to be authorized
export AUTH0_MANAGEMENT_API_SCOPES=$(auth0 apis scopes list $AUTH0_MANAGEMENT_API_ID --json | jq -r '.[].value' | jq -ncR '[inputs]')

# Authorize the Auth0 Terraform Provider application to use the Auth0 Management API
auth0 api post "client-grants" --data='{"client_id": "'$AUTH0_CLIENT_ID'", "audience": "'$AUTH0_MANAGEMENT_API_IDENTIFIER'", "scope":'$AUTH0_MANAGEMENT_API_SCOPES'}'

Now you can run the Terraform script to create the Auth0 application. Run the following commands to initialize the script and apply it.

# Upgrade the terraform script
terraform init -upgrade
terraform apply

This will create the Auth0 application and the test user. You can find the client ID and secret in the output of the terraform output command. You can also find the client ID and secret in the Auth0 dashboard.

# Client ID
terraform output --json | jq -r '.auth0_webapp_client_id.value'
# Client Secret
terraform output --json | jq -r '.auth0_webapp_client_secret.value'

Note the client ID and client secret from the output. You will need these values in the next section.

Configure the JHipster application to use Auth0

Update kubernetes/registry-k8s/application-configmap.yml with the Spring Security OIDC configuration using values from the previous step. This configuration is loaded into Consul, and it shares the values with the gateway and microservices.

apiVersion: v1
kind: ConfigMap
metadata:
  name: application-config
  namespace: jhipster
#common configuration shared between all applications
data:
  application.yml: |-
    configserver:
      ...
    jhipster:
      security:
        ...
        oauth2:
          audience:
            - https://<your-auth0-domain>/api/v2/
    spring:
      security:
        oauth2:
          client:
            provider:
              oidc:
                # make sure to include the trailing slash
                issuer-uri: https://<your-auth0-domain>/
            registration:
              oidc:
                client-id: <client-id>
                client-secret: <client-secret>
  # app specific configuration

The microservice applications are now ready.

Secure secrets

If you have noticed, you are setting secrets in plain text on the application-configmap.yml file, which is not ideal and is not a best practice for security. The best way to do this securely would be to use AWS Secrets Manager, an external service like HashiCorp Vault, or Sealed Secrets. To learn more about these methods see the blog post Shhhh... Kubernetes Secrets Are Not Really Secret!.

Deploy the Microservice Stack

You are ready to deploy to our shiny new EKS cluster, but first, you need to build and push the Docker images to a container registry. You can use Amazon Elastic Container Registry (ECR) or any other container registry.

Build the Docker images

You need to build Docker images for each app. This is specific to the JHipster application used in this tutorial which uses Jib to build the images. Make sure you are logged into Docker using docker login. Navigate to each app folder (store, invoice, product) and run the following command:

./gradlew bootJar -Pprod jib -Djib.to.image=<docker-repo-uri-or-name>/<image-name>

Image names should be store, invoice, and product.

Deploy the applications to EKS

Start the deployment using the handy script provided by JHipster. You could also manually apply deployments using kubectl apply -f <file> commands.

cd kubernetes
./kubectl-apply.sh -f

You can also run the following command to see the status of the deployments:

kubectl get pods -n jhipster

View the Consul registry using port-forwarding as follows, and you will be able to access the application at http://localhost:8500.

kubectl port-forward svc/consul-ui -n jhipster 8500

You can access the gateway application using port-forwarding as follows, and you will be able to access the application at http://localhost:8080.

kubectl port-forward svc/store -n jhipster 8080

Or, you can access the application via the load balancer exposed. Find the external IP of the store service by navigating to the service tab in KDash or by running the following:

kubectl get svc store -n jhipster

You need to add the external IP of the store to allowed callback URLs and allowed logout URLs in the Auth0 web application. You can do this by adding the values to the callbacks and allowed_logout_urls array in the java_ms_client resource in auth0.tf file like below:

resource "auth0_client" "java_ms_client" {
  name                = "JavaMicroservices"
  description         = "Java Microservices Client Created Through Terraform"
  app_type            = "regular_web"
  callbacks           = ["http://localhost:8080/login/oauth2/code/oidc", "http://aws-elb-id.region.elb.amazonaws.com:8080/login/oauth2/code/oidc"]
  allowed_logout_urls = ["http://localhost:8080", "http://aws-elb-id.region.elb.amazonaws.com:8080"]
  oidc_conformant     = true

  jwt_configuration {
    alg = "RS256"
  }
}

Run terraform apply -target="auth0_client.java_ms_client" to update the configuration in your Auth0 tenant.

Now you should be able to visit the external IP of the store service on port 8080 and see the application, and you should be able to log in using your Auth0 test user credentials.

If you encounter an issue where the Consul pods do not start, you might have issues with the AWS EBS addon for EKS. Run kubectl describe pvc -n jhipster to see if there are any errors. If you see could not create volume in EC2: UnauthorizedOperation in errors, then you need to troubleshoot by following the Managing the Amazon EBS CSI driver as an Amazon EKS add-on guide.

Tear Down the Cluster with Terraform

Once you are done with the tutorial, you can delete the cluster and all the resources created using Terraform by running the following commands:

cd terraform
# The commands below might take a while to finish.
terraform destroy -target="module.eks_blueprints_addons" -auto-approve
# If deleting the VPC fails, then manually delete the load balancers and the security groups
# for the load balancer associated with the VPC from the AWS EC2 console and try again.
# This is due to the fact that EKS creates a load balancer for Kubernetes service and it is not known to Terraform.
terraform destroy -target="module.eks" -auto-approve
terraform destroy -target="module.vpc" -auto-approve
# Clean up Auth0 apps and anything left over.
terraform destroy -auto-approve

You can find all the code from this example on GitHub.

If you like this article, please leave a like or a comment.

You can follow me on Mastodon and LinkedIn.

Mastodon for Developers: Everything You Need to Know

Deepu K Sasidharan — Mon, 06 Feb 2023 13:34:14 +0000

Originally published at auth0.com

What Is Mastodon?

Mastodon is an open-source, distributed micro-blogging platform that can host social networking sites. It was created by Eugen Rochko and was first released in 2016. It is similar to Twitter regarding features, target audience, and user experience. But unlike Twitter, it is run on a decentralized network of servers, each of which is called an instance. This does not mean that you have fragmented silos; instances are federated, which means you can talk to other instances, follow people, and see content from other instances, making Mastodon the ideal decentralized social network.

Any single company does not own the Mastodon network, and users can join any instance they wish. The federation allows people to connect and interact with each other across different instances, making Mastodon more open, secure, and free than something like Twitter. This means no single entity has absolute control over Mastodon, which is a big advantage over traditional social networks like Twitter, which is owned by a single company and is not open source.

After the recent Twitter saga, it's clear why this is a major benefit. However, this does not mean there is no moderation. Quite the contrary, actually. Each instance can have its own rules and can moderate content as they see fit. This means you can join an instance that is more aligned with your values and beliefs or even hosts your own if you have the necessary resources.

Why Should You Care about Mastodon?

Platforms like Twitter, Facebook, Instagram, and TikTok make most of their revenue from advertisements. This means they are selling our attention to advertisers.

How do you keep someones attention? Our brains are naturally wired to seek dopamine reward pathways and algorithms used by these social media often target dopamine pathways to create a feeling of reward and reinforcement that encourages people to keep using their services. This is done by providing users with a steady stream of new content and positive feedback, such as likes and comments, which stimulates dopamine production in the brain. This can lead to people becoming addicted to social media, as they constantly seek out the reward of dopamine that comes with using the platform.

In addition, many platforms use algorithms to personalize content and increase the chance of a positive outcome, further encouraging people to use the service. But that's not all; these algorithms also use tactics like amplifying content that is reactionary, sensational, or frightening, which grabs our attention.

Platforms like Mastodon, which does not operate for profit, do not have any incentives to follow similar tactics. This does not mean that it is not possible as a Mastodon instance can choose to serve advertisements. But since you have control as a user, you can choose to join an instance that does not serve advertisements. This means that algorithms are not manipulating you to keep you hooked on the platform. This is a big advantage over platforms like Twitter which are designed to keep you hooked on the platform, making it easier to sow division and hatred by nefarious actors.

Mastodon is an important platform for anyone who values their freedom, security, mental health, and privacy online. It is a great alternative to Twitter for those who are looking for a more open and secure platform to connect with others without having to worry about being harassed or bullied. It is quite customizable and provides greater control over your data. With its growing user base, Mastodon is quickly becoming the go-to platform for many people looking for a better way to connect with others and get out of the grasp of BigTech.

What Is Fediverse, and How Does It Work?

Fediverse is the term used to describe a network of interconnected servers that can communicate with each other using decentralized networking protocols. Fediverse is bigger than Mastodon and can include, among others:

Mastodon servers (social networking and microblogging)
Friendica servers (social networking and microblogging)
PeerTube servers (video hosting)
Pleroma servers (social networking and microblogging)

Fediverse networks can be used for social networks, file hosting services, and so on.

Fediverse works using several different communication protocols. The most important ones are ActivityPub, OStatus, and diaspora.

ActivityPub is a protocol that allows servers to communicate with each other. It is a decentralized protocol based on the ActivityStreams standard. Mastodon, PeerTube, and Pleroma use ActivityPub.

OStatus is a decentralized protocol based on the Atom Syndication Format. OStatus is a predecessor to ActivityPub and is used by older instances of Mastodon and Pleroma.

Diaspora is a decentralized protocol. Mastodon, Friendica, and Pleroma use it.

Any server that supports one of these protocols can communicate with other servers that support the same protocol.

It is difficult to estimate the number of users in Fediverse due to the distributed nature, but rough third-party estimates put it at around 8 million users. This is a small fraction of the number of users on Twitter, but it is growing rapidly, as well as the number of instances.

Choosing Servers

One of the biggest strengths of Mastodon, decentralization, is also its biggest hurdle when it comes to adoption. This means that there is no single place to sign up for Mastodon. Instead, you have to choose an instance to join. Choosing a server could be a daunting task, especially if you are new to Mastodon. There are some factors to consider when choosing a server. These include:

Quality and reliability of the server
Community and moderation
Rules and Policies
Non-profit status

Choosing a server based on this is important, but it's not as critical as it seems. This is because you can follow people from other servers and see their content in your timeline. This means you can join a server that is more aligned with your values and beliefs and still follow people from other servers. This is one of the biggest advantages of Mastodon over Twitter, where you are forced to follow people from the same server. Not just that, Mastodon lets you move from one server to another in case you choose a server that turns out to be unreliable, or you realize that you disagree with the server's policies. You can also easily import your data, like your followers and people you follow, to the new server. So choose a server you feel comfortable with and stick around to see how it goes. You can always move to another server if you don't like it.

As of this writing, there are over 17,000 instances of Mastodon, which is growing daily. This means that there is a Mastodon instance for everyone. You can find a list of Mastodon instances on instances.social.

Recommended servers

For technical folks like developers, you could consider joining one of these servers:

fosstodon.org: Ideal for developers, especially if you are an open-source enthusiast
mstdn.social: It's a general-purpose server with a good community and is quite reliable
mastodon.social: The official Mastodon server with a good community and is quite reliable
hachyderm.io: A server for tech industry professionals
techhub.social: A server for technology enthusiasts

Building a Timeline Based on Hashtags and Follows

Unlike Twitter, when you join a Mastodon server, you will not be greeted by a timeline with interesting posts and recommendations for people to follow. Instead, you are going to be greeted by an empty timeline. This is by design, as Mastodon does not have any algorithm or recommendation system, and you will not be following people from the same server. This means that you have to build your timeline by following people and hashtags. This is a good thing as it gives you more control over your timeline, and you will not be bombarded with content you are not interested in. You can find and follow people from other instances by searching for their usernames. For example, if you want to follow me, you can search for my username @deepu105 or @deepu105@mastodon.social

Similarly, you can also follow hashtags; look for the + button on the top right corner of the screen when you are on the hashtag page. This works on the web version and some mobile clients.

It is important to follow people and hashtags you are interested in to have interesting content on your timeline. You can mute people you are not interested in.

You can also filter hashtags or words you don't want to see on your timeline.

This is a great way to keep your timeline clean and free of unwanted content.

If you are migrating from Twitter, Movetodon.org is a great tool to help you find and follow people from Twitter on Mastodon.

Cross-Posting

If you prefer to keep your Twitter account and cross-post to Mastodon and vice versa, you can use some tools to do so, including writing your own scripts using Twitter and Mastodon APIs. My personal favorite is moa.party. It supports cross-posting to and from Twitter and Mastodon and is simple to setup and use. It can also post from Instagram to Mastodon. It is open-source, and you can host it yourself if you don't want to give the service access to your Twitter/Mastodon accounts using OAuth.

Securing the Account

As usual, it is important to keep your Mastodon account secure. This includes using a strong password, using a password manager, and enabling two-factor authentication (2FA). Mastodon supports 2FA using TOTP (Time-based One-time Password Algorithm) authenticator apps like Google Authenticator and FIDO security keys like Yubikey.

You can also verify your Mastodon account by linking your official website, GitHub profile, and so on. This is a great way to prove that you are the owner of the account and prevent impersonation.

To do this, go to Preferences -> Appearance -> Profile metadata and copy the verification URL and add it to your website as instructed. For the GitHub profile, add your Mastodon profile URL in the GitHub profile's website field. For example, my Mastodon profile URL is https://mastodon.social/@deepu105. Now add your website or GitHub profile URL to your Mastodon profile metadata and save.

Moving between Servers

If you decide to move from one Mastodon server to another, here are a few tips to make the process smooth.

First, create a profile on the new server you want to use. Note that this will be a new username, as Mastodon usernames include the server name, and you need to choose a username that is available on the new server.
Export your data from the old server by going to Preferences -> Import and export -> Data export. You will get CSV files for each item you export, like Follows, mutes, and so on.
Create an account alias in your new account by going to Preferences -> Account -> Account settings -> Moving from a different account. This will allow you to redirect your old account and move followers from your old account to the new account.
Now, from your old account, redirect to the new account by going to Preferences -> Account -> Account settings -> Moving to a different account. This will redirect your old account to the new account, and all your followers will be moved to the new account.
Now, from your new account, go to Preferences -> Import and export -> Import and import the CSV files that you exported from your old account. This will import all your follows, mutes, lists, and so on to the new account.

Conclusion

Mastodon is a great alternative to Twitter and is a great way to connect with people from around the world without worrying about a single entity dictating what you can and cannot do with your social media. It is a great way to build a community around your projects and share your thoughts and ideas. It is also a great way to connect with people from the open-source community. I hope this guide will help you get started with Mastodon.

If you like this article, please leave a like or a comment.

You can follow me on Mastodon and LinkedIn.

Shhhh... Kubernetes Secrets Are Not Really Secret!

Deepu K Sasidharan — Thu, 15 Dec 2022 11:46:30 +0000

Originally published at auth0.com

Kubernetes has become an inevitable part of the modern software infrastructure. Hence managing sensitive data on Kubernetes is also an essential aspect of modern software engineering so that you can put the security back into DevSecOps. Kubernetes offers a way to store sensitive data using the Secret object. While it's better than nothing, it is not really a secret, as it is just base64 encoded strings that anyone with access to the cluster or the code can decode.

Caution:
Kubernetes Secrets are, by default, stored unencrypted in the API server's underlying data store (etcd). Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd. Additionally, anyone authorized to create a Pod in a namespace can use that access to read any Secret in that namespace; this includes indirect access, such as the ability to create a Deployment.
— Kubernetes docs

The problem of reading secrets from the cluster can be fixed using proper RBAC configuration and by securing the API server, check out How to Secure Your Kubernetes Clusters With Best Practices to learn more about RBAC and cluster API security. Securing secrets on the source code is the bigger problem. Everyone who has access to the repositories containing those secret definitions can also decode them. This makes it quite tricky to manage Kubernetes secrets in Git, like every other resource.

Let's see how to setup more secure secrets using the;

Sealed Secrets,
External Secrets Operator,
Secrets Store CSI driver.

You would need a Kubernetes cluster to run the samples. I used k3d to create a local cluster. You can also use kind or minikube for this purpose.

Sealed Secrets

Sealed Secrets is an open-source Kubernetes controller and a client-side CLI tool from Bitnami that aims to solve the "storing secrets in Git" part of the problem, using asymmetric crypto encryption. Sealed Secrets with an RBAC configuration preventing non-admins from reading secrets is an excellent solution for the entire problem.

It works as below;

Encrypt the secret on the developer machine using a public key and the kubeseal CLI. This encodes the encrypted secret into a Kubernetes Custom Resource Definition (CRD)
Deploy the CRD to the target cluster
The Sealed Secret controller decrypts the secret using a private key on the target cluster to produce a standard Kubernetes secret.

The private key is only available to the Sealed Secrets controller on the cluster, and the public key is available to the developers. This way, only the cluster can decrypt the secrets, and the developers can only encrypt them.

Advantages

Supports template definition so that metadata can be added to the unsealed secrets. For example, you can add labels and annotations to the unsealed secrets using the template definition.
The unsealed secrets will be owned by the sealed secret CRD and updated when the sealed secrets are updated.
Certificates are rotated every 30 days by default, and this can be customized.
Secrets are encrypted using unique keys for each cluster, namespace, and secret combination (private key + namespace name + secret name), preventing any loopholes in decryption. This behavior is configurable using scopes strict, namespace-wide, and cluster-wide during the sealing process.
Can be used to manage existing secrets in the cluster.
Has a VSCode extension to make it easier to use.

Disadvantages

Since it unseals the sealed secrets into regular secrets, you can still decode them if you have access to the cluster and namespace.
Requires resealing for each cluster environment, as the key pair will be unique for each cluster.

Installation

Install the controller on the cluster and the CLI on the local machine.

Download the controller.yaml manifest file from the release page.
Deploy the controller using kubectl apply -f controller.yaml to your cluster. The controller will be installed on the kube-system namespace. The controller will start and be ready in a few moments.
Install the CLI on your local machine using brew install kubeseal (Linux & macOS) or using the pre-built binaries on the release page.

Usage

Let's create a sealed secret.

Create a secret using the kubectl create secret command or by hand coding a YAML file as follows:

echo -n secretvalue | kubectl create secret generic mysecret \
  --dry-run=client \
  --from-file=foo=/dev/stdin -o yaml > my-secret.yaml

This will produce a secret definition like the one below;

# my-secret.yaml

apiVersion: v1
data:
  foo: c2VjcmV0dmFsdWU=
kind: Secret
metadata:
  creationTimestamp: null
  name: mysecret

Seal the secret using the kubeseal CLI. This will encrypt the secret using the public key fetched from the server and produce a sealed secret definition. The my-secret.yaml file can be discarded now. You can also download the public key and use it locally in offline mode.

kubeseal --format yaml < my-secret.yaml > my-sealed-secret.yaml

This will produce a sealed secret definition, my-sealed-secret.yaml, like the one below;

# my-sealed-secret.yaml

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: default
spec:
  encryptedData:
    foo: AgA6a4AGzd7qzR8mTPqTPFNor8tTtT5...==
  template:
    metadata:
      creationTimestamp: null
      name: mysecret
      namespace: default

This file is safe to commit to Git or to share with other developers.

Finally, you can deploy this to the cluster to be unsealed.

kubectl apply -f my-sealed-secret.yaml

Now, you can see the unsealed secret in the cluster.

kubectl describe secret mysecret

You can use this secret in deployments like any other Kubernetes secret.

External Secrets Operator

Sealed Secrets are a great starting point for securing secrets, but there is an even better way. Using the External Secrets Operator (ESO) and an external secret management system like HashiCorp Vault, AWS Secrets Manager, Google Secrets Manager, or Azure Key Vault. While this is a bit more involved to set up, it is a better approach if you use a cloud provider to host your Kubernetes cluster. ESO supports many such secret managers and watches for changes to external secret stores, and keeps Kubernetes secrets in sync.

ESO provides four CRDs to manage secrets. The ExternalSecret and ClusterExternalSecret CRD define what data needs to be fetched and how it should be transformed. The SecretStore and ClusterSecretStore CRD define the connection details to the external secret stores. The Cluster variations can be used cluster-wide.

It works as below;

Create a SecretStore CRD to define the connection details to the external secret store.
Create secrets in the external secret store.
Create an ExternalSecret CRD to define what data needs to be fetched from the external secret store.
Deploy the CRDs to the target cluster.
The ESO controller will fetch the data from the external secret store and create a Kubernetes secret.

Advantages

Secrets are stored in a secure external secret manager, not the code repository.
Keeps secrets in sync with the external secret manager.
Works with many external secret managers.
Can use multiple secret stores in the same cluster.
Provides Prometheus metrics for monitoring.

Disadvantages

Needs an elaborate setup to use.
Creates a Kubernetes secret object which can be decoded if you have access to the cluster and namespace.
Relies on the external secret manager and its access policies to be secure.

Installation

ESO can be installed via Helm using the following commands:

helm repo add external-secrets https://charts.external-secrets.io

helm install external-secrets \
  external-secrets/external-secrets \
  --namespace external-secrets \
  --create-namespace

If you want to include ESO in your Helm releases, add the --set installCRDs=true flag to the above command.

Let's see how you can use ESO with different secret managers.

Using HashiCorp Vault

HashiCorp Vault is a popular secret manager providing different secret engines. ESO can only be used with the KV Secrets Engine offered by Vault. Vault provides a free and open-source version that you can self-manage and a managed version with a free tier on the HashiCorp Cloud Platform (HCP).

Make sure you have a Key-value secret store setup in your local Vault instance or on the HCP cloud. You can also deploy Vault to your Kubernetes cluster using the Vault Helm chart

Create a new SecretStore CRD, vault-backend.yaml, to define the connection details to Vault.

# vault-backend.yaml

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "YOUR_VAULT_ADDRESS"
      path: "secret"
      version: "v2"
      namespace: "admin" # required for HCP Vault
      auth:
        # points to a secret that contains a vault token
        # https://www.vaultproject.io/docs/auth/token
        tokenSecretRef:
          name: "vault-token"
          key: "token"

Create a secret resource to hold the Vault token. Use a token that has policies with read access to the secret/ path in the Vault KV store.

kubectl create secret generic vault-token \
  --dry-run=client \
  --from-literal=token=YOUR_VAULT_TOKEN

Create a secret in Vault. If you are using the Vault CLI, you can use the below command to create a secret. Make sure you are logged in to the vault instance from the CLI with appropriate policies.

vault kv put secret/mysecret my-value=supersecret

Create an ExternalSecret CRD to define what data needs to be fetched from Vault.

# vault-secret.yaml

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: vault-example
spec:
  refreshInterval: "15s"
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: vault-example-sync
  data:
    - secretKey: secret-from-vault
      remoteRef:
        key: secret/mysecret
        property: my-value

Apply the above CRDs to the cluster, and it should create a Kubernetes secret named vault-example-sync with the data fetched from Vault.

kubectl apply -f vault-backend.yaml
kubectl apply -f vault-secret.yaml

You can see the secret in the cluster using the kubectl describe command.

kubectl describe secret vault-example-sync

# output should have the below data
Name:         vault-example-sync
Namespace:    default
Labels:       <none>
Annotations:  reconcile.external-secrets.io/data-hash: ...

Type:  Opaque

Data
====
secret-from-vault:  16 bytes

If you have issues creating the secret, check the events section of the describe output of the ExternalSecret resource.

kubectl describe externalsecret vault-example

If you see permission errors, make sure you use tokens with the right policies.

Other secret managers

Setting up other secret managers is similar to the above steps. The only difference would be the SecretStore CRD and the remoteRef section in the ExternalSecret CRD. You can find official guides for different providers in the ESO documentation.

Secrets Store CSI Driver

The Secrets Store CSI Driver is a native upstream Kubernetes driver that can be used to abstract where the secret is stored from the workload. If you want to use a cloud provider's secret manager without exposing the secrets as Kubernetes Secret objects, you can use the CSI Driver to mount secrets as volumes in your pods. This is a great option if you use a cloud provider to host your Kubernetes cluster. The driver supports many cloud providers and can be used with different secret managers.

The Secrets Store CSI Driver is a daemonset that communicates with the secret provider to retrieve secrets specified in a SecretProviderClass custom resource.

It works as below;

Create a SecretProviderClass CRD to define the details of the secret to be fetched from the secret provider.
Create deployments and reference the SecretProviderClass in the pod's volume spec.
The driver will fetch the secret from the secret provider and mount it as a tmpfs volume in the pod during pod startup. This volume will be removed during pod deletion.

The driver can also sync changes to secrets. The driver currently supports Vault, AWS, Azure, and GCP providers. Secrets Store CSI Driver can also sync provider secrets as Kubernetes secrets; if required, this behavior needs to be explicitly enabled during installation.

Advantages

Secrets are stored in a secure external secret manager, not the code repository.
Keeps secrets in sync with the external secret manager. It also supports the rotation of secrets.
Works with all major external secret managers.
Mounts secrets as volumes in the pod so they are not exposed as Kubernetes secrets. It can be configured to create Kubernetes secrets as well.

Disadvantages

Needs an elaborate setup to use and is more complex than ESO.
Uses more resources than ESO as this needs to run in every node.
Relies on the external secret store and its access policies to be secure.

Using Google Secret Manager provider

Let us see how to configure the driver to use Google Secret Manager (GSM) as the secret provider.

Make sure you are using a Google Kubernetes Engine (GKE) cluster with the Workload Identity feature enabled. Workload Identity allows workloads in your GKE clusters to impersonate Identity and Access Management (IAM) service accounts to access Google Cloud services. You would also need to enable Kubernetes Engine API, Secret Manager API, and Billing for the project. The gcloud CLI should prompt you to enable these APIs if they are not enabled.

The below command can be used to create a new cluster with Workload Identity enabled using the gcloud CLI.

export PROJECT_ID=<your gcp project>
gcloud config set project $PROJECT_ID

gcloud container clusters create hello-hipster \
  --workload-pool=$PROJECT_ID.svc.id.goog

Install the Secrets Store CSI Driver

Secrets Store CSI Driver can be installed on the cluster with Helm using the following commands:

helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts

helm install csi-secrets-store \
    secrets-store-csi-driver/secrets-store-csi-driver \
    --namespace kube-system

This will install the driver and CRDs on the kube-system namespace. You also need to install the provider required into the cluster.

Install the GSM provider

Let us install the GSM provider into the cluster. The provider can be installed using the following command:

kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp/main/deploy/provider-gcp-plugin.yaml

Create a Secret

First, you need to setup a workload identity service account.

# Create a service account for workload identity
gcloud iam service-accounts create gke-workload

# Allow "default/mypod" to act as the new service account
gcloud iam service-accounts add-iam-policy-binding \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:$PROJECT_ID.svc.id.goog[default/mypodserviceaccount]" \
    gke-workload@$PROJECT_ID.iam.gserviceaccount.com

Now let's create a secret that this service account can access.

# Create a secret with 1 active version
echo "mysupersecret" > secret.data
gcloud secrets create testsecret --replication-policy=automatic --data-file=secret.data
rm secret.data

# grant the new service account permission to access the secret
gcloud secrets add-iam-policy-binding testsecret \
    --member=serviceAccount:gke-workload@$PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/secretmanager.secretAccessor

Now you can create a SecretProviderClass resource that will be used to fetch the secret from GSM. Remember to replace $PROJECT_ID with your GCP project ID.

# secret-provider-class.yaml

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: app-secrets
spec:
  provider: gcp
  parameters:
    secrets: |
      - resourceName: "projects/$PROJECT_ID/secrets/testsecret/versions/latest"
        path: "good1.txt"
      - resourceName: "projects/$PROJECT_ID/secrets/testsecret/versions/latest"
        path: "good2.txt"

Create a Pod

Now you can create a pod that will use the SecretProviderClass resource to fetch the secret from GSM. Remember to replace $PROJECT_ID with your GCP project ID.

# my-pod.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: mypodserviceaccount
  namespace: default
  annotations:
    iam.gke.io/gcp-service-account: gke-workload@$PROJECT_ID.iam.gserviceaccount.com
---
apiVersion: v1
kind: Pod
metadata:
  name: mypod
  namespace: default
spec:
  serviceAccountName: mypodserviceaccount
  containers:
    - image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
      imagePullPolicy: IfNotPresent
      name: mypod
      resources:
        requests:
          cpu: 100m
      stdin: true
      stdinOnce: true
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      tty: true
      volumeMounts:
        - mountPath: "/var/secrets"
          name: mysecret
  volumes:
    - name: mysecret
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "app-secrets"

Apply the above resources to the cluster.

kubectl apply -f secret-provider-class.yaml
kubectl apply -f my-pod.yaml

Wait for the pod to start, and then exec into the pod and check the contents of the mounted files.

kubectl exec -it mypod /bin/bash
# execute the below command in the pod to see the contents of the mounted secret file
root@mypod:/# cat /var/secrets/good1.txt

Other secret managers

You can find similar guides for the AWS CSI provider, Azure CSI provider and Vault CSI provider.

Conclusion

Sealed Secrets are a great solution for small teams and projects to secure secrets in Git. For larger teams and projects, the External Secrets Operator or the Secrets Store CSI Driver is a better solution to manage secrets securely. The External Secrets Operator can be used with many secret management systems and is not limited to the ones mentioned above. Of course, this should be used with RBAC to prevent non-admins from reading secrets in the cluster. The Secrets Store CSI Driver might be more involved than ESO, but it is a more native solution.

Cover image created using Midjourney under CC BY-NC 4.0.

If you like this article, please leave a like or a comment.

You can follow me on Mastodon and LinkedIn.

Rust Easy! Modern Cross-platform Command Line Tools to Supercharge Your Terminal

Deepu K Sasidharan — Mon, 07 Nov 2022 09:36:44 +0000

Originally published at deepu.tech.

Rust is taking over the terminal. Rust is a general-purpose programming language that is blazing fast and memory safe. It is the fastest-growing and most loved programming language in the world. It is used to build everything from operating systems to web servers to command-line tools. Recently there has been a surge of command line tools and utilities written in Rust, and many of them are intended to replace standard Unix commands. They are faster, more user-friendly, and have more features than their standard Unix counterparts. In this post, I will cover some of the best Rust command line tools I have used for a while. You can also use these to supercharge your terminal.

These tools are available for both GNU/Linux and macOS. I have not tested them on Windows, but most should also work on Windows. I recommend aliasing the commands to replace the standard commands based on your preferences. If you have Cargo, the rust package manager, you can install all these using Cargo.

Alacritty

Let us start with the terminal itself. Alacritty is a cross-platform modern terminal emulator with sensible defaults. It is GPU accelerated, super fast, and highly configurable. You can use it on Linux, macOS, and Windows. It doesn't have much in terms of a UI, and hence all configurations are done through YAML files. I don't use it as my primary terminal as I love Yakuake too much for all its cool features. We can get most of those features (tabs, split panes, dropdown mode) using tmux and tdrop if really needed. I use Alacrity when I need speed and GPU acceleration. There is an excellent tutorial on using Alacritty with tmux. You could also use Zellij, a modern terminal multiplexer written in Rust, with Alacritty.

There is also the Warp terminal, but it is not open source. It is a great terminal, but I prefer open source software. Thanks to Fran Sancisco for the suggestion.

Installation

# Arch Linux
yay -S alacritty
# Fedora/CentOS
dnf copr enable atim/alacritty
dnf install alacritty
# Debian/Ubuntu
add-apt-repository ppa:aslatter/ppa
apt install alacritty
# macOS Homebrew
brew install --cask alacritty
# Windows Scoop
scoop bucket add extras
scoop install alacritty
# Cargo on any
cargo install alacritty

Starship

Starship is the best terminal prompt I have ever used. Forget Oh My Zsh and stuff like that. Starship is fast, highly customizable, and has a great default theme and settings. I didn't even change most of the default settings, as things were perfect as it is. Starship works on shells like zsh, fish, and bash and can also work alongside other prompts like Oh My Zsh, in case you still want to use Oh My Zsh for other plugins like autosuggestions and so on. Starship works best with a Nerd Font as it can show icons and ligatures based on context. I used Oh My Zsh for many years with the powerlevel10k theme, but the prompt was a bit slow. Starship is blazing fast with more features and an excellent UX.

Installation

# Arch Linux
yay -S starship
# Fedora/CentOS
dnf install starship
# Debian/Ubuntu
curl -sS https://starship.rs/install.sh | sh
# macOS/Linux Homebrew
brew install starship
# macOS MacPorts
port install starship
# Windows Scoop
scoop install starship
# Cargo
cargo install starship --locked

bat

bat is one of my favorite tools from this list. It's a replacement for cat, and once you have used bat, you will never go back. It provides features like syntax highlight, line numbers, Git change highlight, shows special chars, paging, and so on. It is super fast and looks beautiful. I have aliased cat to bat immediately after trying it for the first time. By default, bat behaves similarly to less by paging large output, but that can be disabled to make it work precisely like cat. It can be used as a drop-in replacement for cat even in scripts. bat can also be used as a previewer for fzf. It can also be combined with many other commands and tools like tail, man, and git, among others, to add syntax highlighting to outputs. Syntax highlighting themes are configurable.

Installation

# Arch Linux
yay -S bat
# Fedora/CentOS
dnf install bat
# Debian/Ubuntu
apt install bat
# macOS/Linux Homebrew
brew install bat
# macOS MacPorts
port install bat
# Windows Scoop
scoop install bat
# Cargo
cargo install bat --locked

LSD and exa

Both LSD and exa are replacements for the ls command. They both look gorgeous with nice colors and icons and have features like headers, sorting, tree views, and so on. Exa is a bit faster than LSD for tree views and can show the Git status of files and folders. I prefer exa due to the Git support and faster tree views. I have set up my ls alias to use exa by default. Both can be configured to show custom columns and sorting behaviors.

exa Installation

# Arch Linux
yay -S exa
# Fedora/CentOS
dnf install exa
# Debian/Ubuntu
apt install exa
# macOS Homebrew
brew install exa
# Cargo
cargo install exa

# Alias ls to exa
alias ls='exa --git --icons --color=always --group-directories-first'

LSD Installation

# Arch Linux
yay -S lsd
# Fedora/CentOS
dnf install lsd
# Debian/Ubuntu
dpkg -i lsd_0.23.1_amd64.deb # get .deb file from https://github.com/Peltoche/lsd/releases
# macOS Homebrew
brew install lsd
# macOS MacPorts
port install lsd
# Windows Scoop
scop install lsd
# Cargo
cargo install lsd

# Alias ls to lsd
alias ls='lsd --header --color=always --group-directories-first'

rip

rip is an improved version of the rm command. It is faster, safer, and user-friendly. rip sends deleted files to a temp location so they can be recovered using rip -u. I really like the simplicity and the revert feature, as I don't have to worry about accidentally deleting something using rm. While rip can be aliased to replace rm, the creators advise not doing that as you might get used to it and do rm on other systems where you cannot revert the delete.

Installation

# Arch Linux
yay -S rm-improved
# Fedora/CentOS/Debian/Ubuntu
# Install from binary or build locally using Cargo
# macOS Homebrew
brew install rm-improved
# Cargo
cargo install rm-improved

xcp

xcp is a partial clone of the cp command. It is faster and more user-friendly with progress bars, parallel copying, .gitignore support, and so on. I like its simplicity and developer experience, especially the progress bars. I have aliased cp to xcp so I can use it everywhere.

Installation

# Arch Linux
yay -S xcp
# Fedora/CentOS/Debian/Ubuntu/macOS
# Install from binary or build locally using Cargo
# Cargo
cargo install xcp

# Alias cp to xcp
alias cp='xcp'

zoxide

zoxide is a smarter cd replacement. It remembers the directories you visit, and you can jump to them without providing a full path. You can provide partial paths or even a word from the path. When there are similar paths, zoxide offers an interactive selection using fzf. It is super fast and works with all major shells. I like how it works, and I have aliased cd to z so I can use it everywhere.

Installation

# Arch Linux
yay -S zoxide
# Fedora/CentOS
dnf install zoxide
# Debian/Ubuntu
apt install zoxide
# macOS/Linux Homebrew
brew install zoxide
# macOS MacPorts
port install zoxide
# Windows Scoop
scoop install zoxide
# Cargo
cargo install zoxide --locked

Once installed, you must add the following to your shell config file. For other shells, refer the docs

# bash (~/.bashrc)
eval "$(zoxide init bash)"
# zsh (~/.zshrc)
eval "$(zoxide init zsh)"
# fish (~/.config/fish/config.fish)
zoxide init fish | source

# Alias cd to z
alias cd='z'

dust

Dust is an alternative for the du command. It is fast and has a better UX with nice visualization for disk usage.

Installation

# Arch Linux
yay -S dust
# Fedora/CentOS
# Install binary from https://github.com/bootandy/dust/releases
# Debian/Ubuntu
deb-get install du-dust
# macOS Homebrew
brew install dust
# macOS MacPorts
port install dust
# Windows Scoop
scoop install dust
# Cargo
cargo install du-dust

ripgrep

ripgrep (rg) is a line-oriented search tool that recursively searches your current directory for a regex pattern. It is faster than grep and has many features like compressed files search, colorized output, smart case, file type filtering, multi-threading, and so on. It understands .gitignore files and skips hidden and ignored files. Here is a feature comparison with other similar tools, and yes, it is faster than all the other tools in the list.

Installation

# Arch Linux
yay -S ripgrep
# Fedora/CentOS
dnf install ripgrep
# Debian/Ubuntu
apt-get install ripgrep
# macOS/Linux Homebrew
brew install ripgrep
# macOS MacPorts
port install ripgrep
# Windows Scoop
scoop install ripgrep
# Cargo
cargo install ripgrep

fd

fd is a simpler alternative to find. It is more intuitive to use and comes with sensible defaults. It is extremely fast due to parallel traversing and shows a modern colorized output and supports patterns and regex, parallel commands, smart case, understands .gitignore files, and so on. I have aliased find to fd as I could never remember what options to pass to get a basic find command working.

Installation

# Arch Linux
yay -S fd
# Fedora/CentOS
dnf install fd-find
# Debian/Ubuntu
apt install fd-find
# macOS Homebrew
brew install fd
# macOS MacPorts
port install fd
# Windows Scoop
scoop install fd
# Cargo
cargo install fd-find

sd

sd is a find-and-replace CLI, and you can use it as a replacement for sed and awk. It is way more user-friendly and modern. It is also magnitudes faster than sed.

Installation

# Arch Linux
yay -S sd
# Fedora/CentOS
dnf install sd
# Debian/Ubuntu
# Install binary from the release page
# macOS Homebrew
brew install sd
# Windows Scoop
choco install sd-cli
# Cargo
cargo install sd

procs

procs is a ps replacement. It provides colorized human-readable output, multi-column search, more information than ps, docker support, paging, watch mode, and tree view. It is a much more user-friendly and modern alternative to ps. You can filter by name and PID and use logical and/or operators to combine multiple filters. It also has a tree view which is very useful for seeing the process hierarchy. It can also show docker container names for the process running docker containers.

Installation

# Arch Linux
yay -S procs
# Fedora/CentOS
dnf install procs
# Debian/Ubuntu
# Install binary from the release page
# macOS Homebrew
brew install procs
# macOS MacPorts
port install procs
# Windows Scoop
scoop install procs
# Cargo
cargo install procs

bottom

bottom is a top replacement with a nice terminal UI. It's quite feature-rich and customizable.

Installation

# Arch Linux
yay -S bottom
# Fedora/CentOS
dnf copr enable atim/bottom -y
dnf install bottom
# Debian/Ubuntu
dpkg -i bottom_0.6.8_amd64.deb
# macOS Homebrew
brew install bottom
# macOS MacPorts
port install bottom
# Windows Scoop
scoop install bottom
# Cargo
cargo install bottom --locked

Topgrade

Topgrade is a fantastic utility if you prefer to keep your system up-to-date, like me. It detects most of the package managers on your system and triggers updates. It is configurable, so you can configure it to ignore certain package managers. On my system, it detected pacman, SDKMAN, Flatpak, snap, Homebrew, rustup, Linux firmware, Pip, and so on. Topgrade is cross-platform; you can use it on Windows, macOS, and Linux.

Installation

# Arch Linux
yay -S topgrade
# Fedora/CentOS/Debian/Ubuntu/Windows
# Install binary from the release page
# macOS Homebrew
brew install topgrade
# macOS MacPorts
port install topgrade
# Cargo
cargo install topgrade --locked

Broot

Broot is a tree alternative with a better user experience, and you can use it to navigate a file structure. It's fast and respects .gitignore. You can cd into a directory from the tree view, open sub-directories in a panel, and even preview files. It has excellent keyboard navigation as well. It has many more features.

Installation

# Arch Linux
yay -S broot
# Fedora/CentOS/Debian/Ubuntu/Windows
# Install binary from release page https://dystroy.org/broot/install/
# macOS Homebrew
brew install broot
# macOS MacPorts
port install broot
# Cargo
cargo install broot --locked

Tokei

Tokei is a nice utility to count lines and stats of code. It is very fast, accurate, and has a nice output. It supports over 150 languages and can output in JSON, YAML, CBOR, and human-readable tables.

Installation

# Arch Linux
yay -S tokei
# Fedora/CentOS
dnf install tokei
# Debian/Ubuntu
# Install binary from the release page
# macOS Homebrew
brew install tokei
# macOS MacPorts
port install tokei
# Windows Scoop
scoop install tokei
# Cargo
cargo install tokei

Other notable tools

kdash: A fast and simple dashboard for Kubernetes. Its created by me :)
Zellij: A feature rich modern terminal multiplexer with batteries included.
Nushell: A modern shell written in Rust. Looks quite promising.
xh: A HTTPie alternative with better performance.
monolith: Convert any webpage into a single HTML file with all assets inlined.
delta: A syntax-highlighting pager for git, diff, and grep output.
ripsecrets: Find secret keys in your code before committing them to git.
eva: A CLI REPL calculator.
You can find a list of other Rust CLI tools here

If you like this article, please leave a like or a comment.

You can follow me on Twitter and LinkedIn.

What the Heck Is Project Loom for Java?

Deepu K Sasidharan — Sat, 27 Aug 2022 14:29:25 +0000

Java has had good multi-threading and concurrency capabilities from early on in its evolution and can effectively utilize multi-threaded and multi-core CPUs. Java Development Kit (JDK) 1.1 had basic support for platform threads (or Operating System (OS) threads), and JDK 1.5 had more utilities and updates to improve concurrency and multi-threading. JDK 8 brought asynchronous programming support and more concurrency improvements. While things have continued to improve over multiple versions, there has been nothing groundbreaking in Java for the last three decades, apart from support for concurrency and multi-threading using OS threads.

Though the concurrency model in Java is powerful and flexible as a feature, it was not the easiest to use, and the developer experience hasn't been great. This is primarily due to the shared state concurrency model used by default. One has to resort to synchronizing threads to avoid issues like data races and thread blocking. I wrote more about Java concurrency in my Concurrency in modern programming languages: Java post.

What is Project Loom?

Project Loom aims to drastically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications that make the best use of available hardware.

— Ron Pressler (Tech lead, Project Loom)

OS threads are at the core of Java's concurrency model and have a very mature ecosystem around them, but they also come with some drawbacks and are expensive computationally. Let's look at the two most common use cases for concurrency and the drawbacks of the current Java concurrency model in these cases.

One of the most common concurrency use cases is serving requests over the wire using a server. For this, the preferred approach is the thread-per-request model, where a separate thread handles each request. Throughput of such systems can be explained using Little's law, which states that in a stable system, the average concurrency (number of requests concurrently processed by the server), L, is equal to the throughput (average rate of requests), λ, times the latency (average duration of processing each request), W. With this, you can derive that throughput equals average concurrency divided by latency (λ = L/W).

So in a thread-per-request model, the throughput will be limited by the number of OS threads available, which depends on the number of physical cores/threads available on the hardware. To work around this, you have to use shared thread pools or asynchronous concurrency, both of which have their drawbacks. Thread pools have many limitations, like thread leaking, deadlocks, resource thrashing, etc. Asynchronous concurrency means you must adapt to a more complex programming style and handle data races carefully. There are also chances for memory leaks, thread locking, etc.

Another common use case is parallel processing or multi-threading, where you might split a task into subtasks across multiple threads. Here you have to write solutions to avoid data corruption and data races. In some cases, you must also ensure thread synchronization when executing a parallel task distributed over multiple threads. The implementation becomes even more fragile and puts a lot more responsibility on the developer to ensure there are no issues like thread leaks and cancellation delays.

Project Loom aims to fix these issues in the current concurrency model by introducing two new features: virtual threads and structured concurrency.

Virtual threads

Java 19 is scheduled to be released in September 2022, and Virtual threads will be a preview feature. Yayyy!

Virtual threads are lightweight threads that are not tied to OS threads but are managed by the JVM. They are suitable for thread-per-request programming styles without having the limitations of OS threads. You can create millions of virtual threads without affecting throughput. This is quite similar to coroutines, like goroutines, made famous by the Go programming language (Golang).

The new virtual threads in Java 19 will be pretty easy to use. Compare the below with Golang's goroutines or Kotlin's coroutines.

Virtual thread

Thread.startVirtualThread(() -> {
    System.out.println("Hello, Project Loom!");
});

Goroutine

go func() {
    println("Hello, Goroutines!")
}()

Kotlin coroutine

runBlocking {
    launch {
        println("Hello, Kotlin coroutines!")
    }
}

Fun fact: before JDK 1.1, Java had support for green threads (aka virtual threads), but the feature was removed in JDK 1.1 as that implementation was not any better than platform threads.

The new implementation of virtual threads is done in the JVM, where it maps multiple virtual threads to one or more OS threads, and the developer can use virtual threads or platform threads as per their needs. A few other important aspects of this implementation of virtual threads:

It is a Thread in code, runtime, debugger, and profiler
It's a Java entity and not a wrapper around a native thread
Creating and blocking them are cheap operations
They should not be pooled
Virtual threads use a work-stealing ForkJoinPool scheduler
Pluggable schedulers can be used for asynchronous programming
A virtual thread will have its own stack memory
The virtual threads API is very similar to platform threads and hence easier to adopt/migrate

Let's look at some examples that show the power of virtual threads.

Total number of threads

First, let's see how many platform threads vs. virtual threads we can create on a machine. My machine is Intel Core i9-11900H with 8 cores, 16 threads, and 64GB RAM running Fedora 36.

Platform threads

var counter = new AtomicInteger();
while (true) {
    new Thread(() -> {
        int count = counter.incrementAndGet();
        System.out.println("Thread count = " + count);
        LockSupport.park();
    }).start();
}

On my machine, the code crashed after 32_539 platform threads.

Virtual threads

var counter = new AtomicInteger();
while (true) {
    Thread.startVirtualThread(() -> {
        int count = counter.incrementAndGet();
        System.out.println("Thread count = " + count);
        LockSupport.park();
    });
}

On my machine, the process hung after 14_625_956 virtual threads but didn't crash, and as memory became available, it kept going slowly. You may be wondering why this behavior! It's due to the parked virtual threads being garbage collected, and the JVM is able to create more virtual threads and assign them to the underlying platform thread.

Task throughput

Let's try to run 100,000 tasks using platform threads.

try (var executor = Executors.newThreadPerTaskExecutor(Executors.defaultThreadFactory())) {
    IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> {
        Thread.sleep(Duration.ofSeconds(1));
        System.out.println(i);
        return i;
    }));
}

This uses the newThreadPerTaskExecutor with the default thread factory and thus uses a thread group. When I ran this code and timed it, I got the numbers shown here. I get better performance when I use a thread pool with Executors.newCachedThreadPool().

# 'newThreadPerTaskExecutor' with 'defaultThreadFactory'
0:18.77 real,   18.15 s user,   7.19 s sys,     135% 3891pu,    0 amem,         743584 mmem
# 'newCachedThreadPool' with 'defaultThreadFactory'
0:11.52 real,   13.21 s user,   4.91 s sys,     157% 6019pu,    0 amem,         2215972 mmem

Not so bad. Now, let's do the same using virtual threads.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> {
        Thread.sleep(Duration.ofSeconds(1));
        System.out.println(i);
        return i;
    }));
}

If I run and time it, I get the following numbers.

0:02.62 real,   6.83 s user,    1.46 s sys,     316% 14840pu,   0 amem,         350268 mmem

This is far more performant than using platform threads with thread pools. Of course, these are simple use cases; both thread pools and virtual thread implementations can be further optimized for better performance, but that's not the point of this post.

Running Java Microbenchmark Harness (JMH) with the same code gives the following results, and you can see that virtual threads outperform platform threads by a huge margin.

# Throughput
Benchmark                             Mode  Cnt  Score   Error  Units
LoomBenchmark.platformThreadPerTask  thrpt    5  0.362 ± 0.079  ops/s
LoomBenchmark.platformThreadPool     thrpt    5  0.528 ± 0.067  ops/s
LoomBenchmark.virtualThreadPerTask   thrpt    5  1.843 ± 0.093  ops/s

# Average time
Benchmark                             Mode  Cnt  Score   Error  Units
LoomBenchmark.platformThreadPerTask   avgt    5  5.600 ± 0.768   s/op
LoomBenchmark.platformThreadPool      avgt    5  3.887 ± 0.717   s/op
LoomBenchmark.virtualThreadPerTask    avgt    5  1.098 ± 0.020   s/op

You can find the benchmark source code on GitHub. Here are some other meaningful benchmarks for virtual threads:

An interesting benchmark using ApacheBench on GitHub by Elliot Barlas
A benchmark using Akka actors on Medium by Alexander Zakusylo
JMH benchmarks for I/O and non-I/O tasks on GitHub by Colin Cachia

Structured concurrency

Structured concurrency will be an incubator feature in Java 19.

Structured concurrency aims to simplify multi-threaded and parallel programming. It treats multiple tasks running in different threads as a single unit of work, streamlining error handling and cancellation while improving reliability and observability. This helps to avoid issues like thread leaking and cancellation delays. Being an incubator feature, this might go through further changes during stabilization.

Consider the following example using java.util.concurrent.ExecutorService.

void handleOrder() throws ExecutionException, InterruptedException {
    try (var esvc = new ScheduledThreadPoolExecutor(8)) {
        Future<Integer> inventory = esvc.submit(() -> updateInventory());
        Future<Integer> order = esvc.submit(() -> updateOrder());

        int theInventory = inventory.get();   // Join updateInventory
        int theOrder = order.get();           // Join updateOrder

        System.out.println("Inventory " + theInventory + " updated for order " + theOrder);
    }
}

We want updateInventory() and updateOrder() subtasks to be executed concurrently. Each of those can succeed or fail independently. Ideally, the handleOrder() method should fail if any subtask fails. However, if a failure occurs in one subtask, things get messy.

Imagine that updateInventory() fails and throws an exception. Then, the handleOrder() method throws an exception when calling inventory.get(). So far this is fine, but what about updateOrder()? Since it runs on its own thread, it can complete successfully. But now we have an issue with a mismatch in inventory and order. Suppose the updateOrder() is an expensive operation. In that case, we are just wasting the resources for nothing, and we will have to write some sort of guard logic to revert the updates done to order as our overall operation has failed.
Imagine that updateInventory() is an expensive long-running operation and updateOrder() throws an error. The handleOrder() task will be blocked on inventory.get() even though updateOrder() threw an error. Ideally, we would like the handleOrder() task to cancel updateInventory() when a failure occurs in updateOrder() so that we are not wasting time.
If the thread executing handleOrder() is interrupted, the interruption is not propagated to the subtasks. In this case updateInventory() and updateOrder() will leak and continue to run in the background.

For these situations, we would have to carefully write workarounds and failsafe, putting all the burden on the developer.

We can achieve the same functionality with structured concurrency using the code below.

void handleOrder() throws ExecutionException, InterruptedException {
    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
        Future<Integer> inventory = scope.fork(() -> updateInventory());
        Future<Integer> order = scope.fork(() -> updateOrder());

        scope.join();           // Join both forks
        scope.throwIfFailed();  // ... and propagate errors

        // Here, both forks have succeeded, so compose their results
        System.out.println("Inventory " + inventory.resultNow() + " updated for order " + order.resultNow());
    }
}

Unlike the previous sample using ExecutorService, we can now use StructuredTaskScope to achieve the same result while confining the lifetimes of the subtasks to the lexical scope, in this case, the body of the try-with-resources statement. The code is much more readable, and the intent is also clear. StructuredTaskScope also ensures the following behavior automatically.

Error handling with short-circuiting — If either the updateInventory() or updateOrder() fails, the other is canceled unless its already completed. This is managed by the cancellation policy implemented by ShutdownOnFailure(); other policies are possible.
Cancellation propagation — If the thread running handleOrder() is interrupted before or during the call to join(), both forks are canceled automatically when the thread exits the scope.
Observability — A thread dump would clearly display the task hierarchy, with the threads running updateInventory() and updateOrder() shown as children of the scope.

State of Project Loom

The Loom project started in 2017 and has undergone many changes and proposals. Virtual threads were initially called fibers, but later on they were renamed to avoid confusion. Today with Java 19 getting closer to release, the project has delivered the two features discussed above. One as a preview and another as an incubator. Hence the path to stabilization of the features should be more precise.

What does this mean to regular Java developers?

When these features are production ready, it should not affect regular Java developers much, as these developers may be using libraries for concurrency use cases. But it can be a big deal in those rare scenarios where you are doing a lot of multi-threading without using libraries. Virtual threads could be a no-brainer replacement for all use cases where you use thread pools today. This will increase performance and scalability in most cases based on the benchmarks out there. Structured concurrency can help simplify the multi-threading or parallel processing use cases and make them less fragile and more maintainable.

What does this mean to Java library developers?

When these features are production ready, it will be a big deal for libraries and frameworks that use threads or parallelism. Library authors will see huge performance and scalability improvements while simplifying the codebase and making it more maintainable. Most Java projects using thread pools and platform threads will benefit from switching to virtual threads. Candidates include Java server software like Tomcat, Undertow, and Netty; and web frameworks like Spring and Micronaut. I expect most Java web technologies to migrate to virtual threads from thread pools. Java web technologies and trendy reactive programming libraries like RxJava and Akka could also use structured concurrency effectively. This doesn't mean that virtual threads will be the one solution for all; there will still be use cases and benefits for asynchronous and reactive programming.

If you like this article, please leave a like or a comment.

You can follow me on Twitter and LinkedIn.

The cover image was created using a photo by Peter Herrmann on Unsplash

This post was originally published on the Okta Developer Blog.

How to Deploy JHipster Microservices on Amazon EKS Using Terraform and Kubernetes

Deepu K Sasidharan — Tue, 05 Jul 2022 13:50:34 +0000

When it comes to infrastructure, public clouds are the most popular choice these days, especially Amazon Web Services (AWS). If you are in one of those lucky or unlucky (depending on how you see it) teams running microservices, then you need a way to orchestrate their deployments. When it comes to orchestrating microservices, Kubernetes is the de-facto choice. Most public cloud providers also provide managed Kubernetes as a service; for example, Google provides Google Kubernetes Engine (GKE), Microsoft provides Azure Kubernetes Service (AKS), and Amazon provides Amazon Elastic Kubernetes Service (EKS).

This doesn't mean that deploying and managing microservices on the public cloud is easy; each cloud service comes with its own challenges and pain. This is especially true for Amazon EKS, which, in my opinion, is the most difficult Kubernetes service to use, but also one of the most flexible. This is because EKS consists of some clever orchestrations doing a complex dance on top of other AWS services like EC2, EBS, etc.

If you want to run a microservice stack on EKS, you will need to spend some extra time and effort setting it up and managing it. This is where infrastructure as code (IaC) tools like Terraform come in handy.

So here is what you will learn to do today:

Scaffold a Java microservice stack using JHipster, Spring Boot, and Spring Cloud
Create an EKS cluster, Virtual Private Cloud (VPC), subnets, and required Kubernetes add-ons using Terraform on AWS
Set up OIDC authentication for the microservice stack using Okta
Build and deploy the microservice stack to the cluster

Prerequisites

AWS account with the IAM permissions to create EKS clusters
AWS CLI installed and configured
AWS IAM Authenticator installed
kubectl installed
Docker installed and configured
Terraform installed
Java 11+ installed
Okta CLI installed
JHipster CLI installed
[Optional] KDash

Why Terraform, why not CloudFormation?

At this point, the first question that might pop up in your mind would be, "Why not use CloudFormation?". It's a good question; after all, CloudFormation is built by AWS and hence sounds like an excellent solution to manage AWS resources. But anyone who has tried both CloudFormation and Terraform will probably tell you to forget that CloudFormation even exists. I think CloudFormation is far more complex and less developer-friendly than Terraform. You also need to write a lot more boilerplate with CloudFormation in YAML or JSON. Yikes! And most importantly, Terraform is far more powerful and flexible than CloudFormation. It's cross-platform, which means you can take care of all your infrastructure management needs on any platform with one tool.

Scaffold a Java microservice stack using JHipster

You need a microservice stack to deploy to the cluster. I'm using a microservice stack scaffolded for demo purposes using JHipster. You can use another microservice stack if you want. If you prefer using the same application as in this demo, then you can either scaffold it using JHipster JDL or clone the sample repository from GitHub.

Option 1: Scaffold the microservice stack using JHipster

mkdir jhipster-microservice-stack
cd jhipster-microservice-stack
# download the JDL file.
jhipster download https://raw.githubusercontent.com/oktadev/okta-jhipster-k8s-eks-microservices-example/main/apps.jdl
# Update the `dockerRepositoryName` property to use your Docker Repository URI/Name.
# scaffold the apps.
jhipster jdl apps.jdl

Option 2: Clone the sample repository

git clone https://github.com/oktadev/okta-jhipster-k8s-eks-microservices-example

In either case, remember to change the Docker repository name to match your Docker repository.

The JHipster scaffolded sample application has a gateway application, two microservices, and uses JHipster Registry for service discovery and centralized configuration.

Create an EKS cluster using Terraform

Now let us move on to the important part of the tutorial. Creating an EKS cluster in AWS is not as straightforward as in Google Cloud Platform (GCP). You need to also create a lot more resources for everything to work correctly without surprises. You will be using a bunch of Terraform providers to help with this, and you will also use some prebuilt Terraform modules like AWS VPC Terraform module and Amazon EKS Blueprints for Terraform to reduce the amount of boilerplate you need to write.

These are the AWS resources and VPC architecture you will create:

Build the Terraform configuration

First, make sure you use a specific version of the providers as different versions might use different attributes and features. Create a versions.tf file:

mkdir terraform
cd terraform
touch versions.tf

Add the following to the file:

terraform {
  required_version = ">= 1.0.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.72"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.10"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.4.1"
    }
    random = {
      source  = "hashicorp/random"
      version = ">= 3.2.0"
    }
    nullres = {
      source  = "hashicorp/null"
      version = ">= 3.1"
    }
  }
}

Next, you need to define variables and configure the providers. Create a config.tf file:

touch config.tf

Add the following to the file:

# ##  To save state in s3. Update to suit your needs
# backend "s3" {
#   bucket = "create-an-s3-bucket-and-provide-name-here"
#   region = local.region
#   key    = "eks-cluster-with-new-vpc/terraform.tfstate"
# }

variable "region" {
  default     = "eu-west-1"
  description = "AWS region"
}

resource "random_string" "suffix" {
  length  = 8
  special = false
}

data "aws_availability_zones" "available" {}

locals {
  name            = "okta-jhipster-eks-${random_string.suffix.result}"
  region          = var.region
  cluster_version = "1.22"
  instance_types  = ["t2.large"] # can be multiple, comma separated

  vpc_cidr = "10.0.0.0/16"
  azs      = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    Blueprint  = local.name
    GitHubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
  }
}

provider "aws" {
  region = local.region
}

# Kubernetes provider
# You should **not** schedule deployments and services in this workspace.
# This keeps workspaces modular (one for provision EKS, another for scheduling
# Kubernetes resources) as per best practices.
provider "kubernetes" {
  host                   = module.eks_blueprints.eks_cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)

  exec {
    api_version = "client.authentication.k8s.io/v1alpha1"
    command     = "aws"
    # This requires the awscli to be installed locally where Terraform is executed
    args = ["eks", "get-token", "--cluster-name", module.eks_blueprints.eks_cluster_id]
  }
}

provider "helm" {
  kubernetes {
    host                   = module.eks_blueprints.eks_cluster_endpoint
    cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)

    exec {
      api_version = "client.authentication.k8s.io/v1alpha1"
      command     = "aws"
      # This requires the awscli to be installed locally where Terraform is executed
      args = ["eks", "get-token", "--cluster-name", module.eks_blueprints.eks_cluster_id]
    }
  }
}

You can uncomment the backend section above to save state in S3 instead of your local filesystem. This is recommended for production setup so that everyone in a team has the same state. This file defines configurable and local variables used across the workspace and configures some of the providers used. The Kubernetes provider is included in this file so the EKS module can complete successfully. Otherwise, it throws an error when creating kubernetes_config_map.aws_auth. The helm provider is used to install Kubernetes add-ons to the cluster.

Build the VPC

touch vpc.tf

Add the following to the file:

#---------------------------------------------------------------
# VPC, Subnets, Internet gateway, Route tables, etc.
#---------------------------------------------------------------
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.0"

  name = local.name
  cidr = local.vpc_cidr

  azs             = local.azs
  public_subnets  = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k)]
  private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 10)]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  # Manage so we can name
  manage_default_network_acl    = true
  default_network_acl_tags      = { Name = "${local.name}-default" }
  manage_default_route_table    = true
  default_route_table_tags      = { Name = "${local.name}-default" }
  manage_default_security_group = true
  default_security_group_tags   = { Name = "${local.name}-default" }

  public_subnet_tags = {
    "kubernetes.io/cluster/${local.name}" = "shared"
    "kubernetes.io/role/elb"              = 1
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/${local.name}" = "shared"
    "kubernetes.io/role/internal-elb"     = 1
  }

  tags = local.tags
}

This will create:

A new VPC, three private subnets, and three public subnets,
Internet gateway and NAT gateway for the public subnets,
and AWS routes for the gateways, public/private route tables, and route table associations.

Build the EKS cluster

Now that you have the networking part done, you can build configurations for the EKS cluster and its add-ons. You will use the eks_blueprints module from terraform-aws-eks-blueprints, which is a wrapper around the terraform-aws-modules and provides additional modules to configure EKS add-ons.

Create a eks-cluster.tf file:

touch eks-cluster.tf

Add the following to the file:

#---------------------------------------------------------------
# EKS cluster, worker nodes, security groups, IAM roles, K8s add-ons, etc.
#---------------------------------------------------------------
module "eks_blueprints" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.0.9"

  cluster_name    = local.name
  cluster_version = local.cluster_version

  vpc_id             = module.vpc.vpc_id
  private_subnet_ids = module.vpc.private_subnets

  managed_node_groups = {
    node = {
      node_group_name = "managed-ondemand"
      instance_types  = local.instance_types
      min_size        = 2
      subnet_ids      = module.vpc.private_subnets
    }
  }

  tags = local.tags
}

module "eks_blueprints_kubernetes_addons" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons?ref=v4.0.9"

  eks_cluster_id       = module.eks_blueprints.eks_cluster_id
  eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
  eks_oidc_provider    = module.eks_blueprints.oidc_provider
  eks_cluster_version  = module.eks_blueprints.eks_cluster_version

  # EKS Managed Add-ons
  enable_amazon_eks_vpc_cni    = true
  enable_amazon_eks_coredns    = true
  enable_amazon_eks_kube_proxy = true

  # K8S Add-ons
  enable_aws_load_balancer_controller = true
  enable_metrics_server               = true
  enable_cluster_autoscaler           = true
  enable_aws_cloudwatch_metrics       = false

  tags = local.tags

}

# To update local kubeconfig with new cluster details
resource "null_resource" "kubeconfig" {
  depends_on = [module.eks_blueprints_kubernetes_addons]
  provisioner "local-exec" {
    command = "aws eks --region ${local.region}  update-kubeconfig --name $AWS_CLUSTER_NAME"
    environment = {
      AWS_CLUSTER_NAME = local.name
    }
  }
}

The eks_blueprints module definition creates:

EKS Cluster Control plane with one managed node group and fargate profile,
Cluster and node security groups and rules, IAM roles and policies required,
and AWS Key Management Service (KMS) configuration.

The eks_blueprints_kubernetes_addons module definition creates:

Amazon EKS add-ons vpc-cni, CoreDNS, and kube-proxy,
AWS Load Balancer Controller that provisions an AWS Network Load Balancer for distributing traffic,
and Metrics Server, and Cluster Autoscaler for scaling your workloads.

The null_resource configuration updates your local kubeconfig with the new cluster details. It's not a required step for provisioning but just a handy hack.

Finally, you can also define some outputs to be captured. Create a outputs.tf file:

touch outputs.tf

Add the following to the file:

output "vpc_private_subnet_cidr" {
  description = "VPC private subnet CIDR"
  value       = module.vpc.private_subnets_cidr_blocks
}

output "vpc_public_subnet_cidr" {
  description = "VPC public subnet CIDR"
  value       = module.vpc.public_subnets_cidr_blocks
}

output "vpc_cidr" {
  description = "VPC CIDR"
  value       = module.vpc.vpc_cidr_block
}

output "eks_cluster_id" {
  description = "EKS cluster ID"
  value       = module.eks_blueprints.eks_cluster_id
}

output "eks_managed_nodegroups" {
  description = "EKS managed node groups"
  value       = module.eks_blueprints.managed_node_groups
}

output "eks_managed_nodegroup_ids" {
  description = "EKS managed node group ids"
  value       = module.eks_blueprints.managed_node_groups_id
}

output "eks_managed_nodegroup_arns" {
  description = "EKS managed node group arns"
  value       = module.eks_blueprints.managed_node_group_arn
}

output "eks_managed_nodegroup_role_name" {
  description = "EKS managed node group role name"
  value       = module.eks_blueprints.managed_node_group_iam_role_names
}

output "eks_managed_nodegroup_status" {
  description = "EKS managed node group status"
  value       = module.eks_blueprints.managed_node_groups_status
}

output "configure_kubectl" {
  description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
  value       = module.eks_blueprints.configure_kubectl
}

Provision the cluster

Our Terraform definitions are ready. Now you can provision the cluster. First, initialize Terraform workspace and plan the changes:

# download modules and providers. Initialize state.
terraform init
# see a preview of what will be done
terraform plan

Review the plan and make sure everything is correct. Ensure you have configured your AWS CLI and IAM Authenticator to use the correct AWS account. If not, run the following:

# Visit https://console.aws.amazon.com/iam/home?#/security_credentials for creating access keys
aws configure

Now you can apply the changes:

terraform apply

Confirm by typing yes when prompted. This will take a while (15-20 minutes), so sit back and have a coffee or contemplate what led you to this point in life. 😉

Once the EKS cluster is ready, you will see the output variables printed to the console.

configure_kubectl = "aws eks --region eu-west-1 update-kubeconfig --name okta-tf-demo"
eks_cluster_id = "okta-tf-demo"
eks_managed_nodegroup_arns = tolist([
  "arn:aws:eks:eu-west-1:216713166862:nodegroup/okta-tf-demo/managed-ondemand-20220610125341399700000010/f0c0a6d6-b8e1-cf91-3d21-522552d6bc2e",
])
eks_managed_nodegroup_ids = tolist([
  "okta-tf-demo:managed-ondemand-20220610125341399700000010",
])
eks_managed_nodegroup_role_name = tolist([
  "okta-tf-demo-managed-ondemand",
])
eks_managed_nodegroup_status = tolist([
  "ACTIVE",
])
eks_managed_nodegroups = tolist([
  {
    "node" = {
      "managed_nodegroup_arn" = [
        "arn:aws:eks:eu-west-1:216713166862:nodegroup/okta-tf-demo/managed-ondemand-20220610125341399700000010/f0c0a6d6-b8e1-cf91-3d21-522552d6bc2e",
      ]
      "managed_nodegroup_iam_instance_profile_arn" = [
        "arn:aws:iam::216713166862:instance-profile/okta-tf-demo-managed-ondemand",
      ]
      "managed_nodegroup_iam_instance_profile_id" = [
        "okta-tf-demo-managed-ondemand",
      ]
      "managed_nodegroup_iam_role_arn" = [
        "arn:aws:iam::216713166862:role/okta-tf-demo-managed-ondemand",
      ]
      "managed_nodegroup_iam_role_name" = [
        "okta-tf-demo-managed-ondemand",
      ]
      "managed_nodegroup_id" = [
        "okta-tf-demo:managed-ondemand-20220610125341399700000010",
      ]
      "managed_nodegroup_launch_template_arn" = []
      "managed_nodegroup_launch_template_id" = []
      "managed_nodegroup_launch_template_latest_version" = []
      "managed_nodegroup_status" = [
        "ACTIVE",
      ]
    }
  },
])
region = "eu-west-1"
vpc_cidr = "10.0.0.0/16"
vpc_private_subnet_cidr = [
  "10.0.10.0/24",
  "10.0.11.0/24",
  "10.0.12.0/24",
]
vpc_public_subnet_cidr = [
  "10.0.0.0/24",
  "10.0.1.0/24",
  "10.0.2.0/24",
]

You should see the cluster details if you run kdash or kubectl get nodes commands.

Note: The EKS cluster defined here will not come under AWS free tier; hence, running this will cost money, so delete the cluster as soon as you finish the tutorial to keep the cost within a few dollars.

Set up OIDC authentication using Okta

You can proceed to deploy the sample application. You could skip this step if you used a sample that does not use Okta or OIDC for authentication.

First, navigate to the store application folder.

Before you begin, you’ll need a free Okta developer account. Install the Okta CLI and run okta register to sign up for a new account. If you already have an account, run okta login. Then, run okta apps create jhipster. Select the default app name, or change it as you see fit. Accept the default Redirect URI values provided for you.

The Okta CLI streamlines configuring a JHipster app and does several things for you:

Creates an OIDC app with the correct redirect URIs:

login: http://localhost:8080/login/oauth2/code/oidc and http://localhost:8761/login/oauth2/code/oidc
logout: http://localhost:8080 and http://localhost:8761

Creates ROLE_ADMIN and ROLE_USER groups that JHipster expects
Adds your current user to the ROLE_ADMIN and ROLE_USER groups
Creates a groups claim in your default authorization server and adds the user’s groups to it

NOTE: The http://localhost:8761\* redirect URIs are for the JHipster Registry, which is often used when creating microservices with JHipster. The Okta CLI adds these by default.

The OIDC app configuration will be written to .okta.env file in the folder where you ran the command.

Note: Make sure to add the newly created .okta.env file to your .gitignore file so that you don't accidentally expose your credentials to the public.

Update kubernetes/registry-k8s/application-configmap.yml with the OIDC configuration from the .okta.env file. The Spring Cloud Config server reads from this file and shares the values with the gateway and microservices.

data:
  application.yml: |-
    ...
    spring:
      security:
        oauth2:
          client:
            provider:
              oidc:
                issuer-uri: https://<your-okta-domain>/oauth2/default
            registration:
              oidc:
                client-id: <client-id>
                client-secret: <client-secret>

Next, configure the JHipster Registry to use OIDC for authentication. Modify kubernetes/registry-k8s/jhipster-registry.yml to enable the oauth2 profile, which is preconfigured in the JHipster Registry application.

- name: SPRING_PROFILES_ACTIVE
  value: prod,k8s,oauth2

The application is now ready.

Secure secrets

If you have noticed, you are setting secrets in plain text on the application-configmap.yml file, which is not ideal and is not a best practice for security. For the specific JHipster application, you can use the encrypt functionality provided by the JHipster Registry to encrypt the secrets. See Encrypt Your Secrets with Spring Cloud Config to learn how to do this. But that would also rely on a base64 encoded encryption key added as a Kubernetes Secret, which still can be decoded. The best way to do this would be to use AWS Secrets Manager, an external service like HashiCorp Vault, or Sealed Secrets. To learn more about these methods see Encrypt Your Kubernetes Secrets.

Deploy the microservice stack

Build the Docker images

./gradlew bootJar -Pprod jib -Djib.to.image=<docker-repo-uri-or-name>/<image-name>

Image names should be store, invoice, and product.

Deploy the applications to EKS

Start the deployment using the handy script provided by JHipster. You could also manually apply deployments using kubectl apply -f <file> commands.

cd kubernetes
./kubectl-apply.sh -f

You can also run the following command to see the status of the deployments:

kubectl get pods -n jhipster

View the registry using port-forwarding as follows, and you will be able to access the application at http://localhost:8761.

kubectl port-forward svc/jhipster-registry -n jhipster 8761

You can access the gateway application using port-forwarding as follows, and you will be able to access the application at http://localhost:8080.

kubectl port-forward svc/store -n jhipster 8080

Or, you can access the application via the load balancer exposed. Find the external IP of the store service by navigating to the service tab in KDash or by running the following:

kubectl get svc store -n jhipster

Navigate to the Okta Admin Console and go to Applications > Applications from left-hand navigation. Find the application you created earlier with okta apps create jhipster and add the external IP from kubectl get svc command to the Sign-in redirect URIs and Sign-out redirect URIs. Make sure to use the same paths as the current localhost entries.

Now you should be able to visit the external IP of the store service on port 8080 and see the application, and you should be able to log in using your Okta credentials.

Tear down the cluster with Terraform

Once you are done with the tutorial, you can delete the cluster and all the resources created using Terraform by running the following commands:

cd terraform
# The commands below might take a while to finish.
terraform destroy -target="module.eks_blueprints_kubernetes_addons" -auto-approve
# If deleting VPC fails, then manually delete the load balancers and security groups
# for the load balancer associated with the VPC from AWS EC2 console and try again.
terraform destroy -target="module.eks_blueprints" -auto-approve
terraform destroy -target="module.vpc" -auto-approve
# cleanup anything left over.
terraform destroy -auto-approve

Learn more about Java Microservices, EKS, Kubernetes, and JHipster

If you want to learn more about Kubernetes, OIDC, or using OIDC with Kubernetes, and security in general, check out these additional resources.

You can find all the code from this example on GitHub.

If you liked this tutorial, chances are you'll enjoy the others we publish. Please follow @oktadev on Twitter and subscribe to our YouTube channel to get notified when we publish new developer tutorials.

Cloud Native Java Microservices with JHipster and Istio

Deepu K Sasidharan — Tue, 28 Jun 2022 10:24:09 +0000

Microservices are not everyone's cup of tea, and they shouldn't be. Not every problem can or should be solved by microservices. Sometimes building a simple monolith is a far better option. Microservices are solutions for use cases where scale and scalability are important. A few years ago, microservices were all the rage, made popular, especially by companies like Netflix, Spotify, Google, etc. While the hype has died down a bit, genuine use cases still exist. With ongoing advances in cloud computing technologies, building microservices as cloud-native services is the way to go due to many benefits.

Today we will look at building a cloud-native Java microservice stack that utilizes a service mesh to provide most of the distributed system needs and we'll deploy it to the cloud using Kubernetes.

So here is what we will do today:

Build a Java microservice stack using JHipster, Spring Boot, and Spring Cloud
Create a Google Kubernetes Engine (GKE) cluster
Deploy Istio service mesh to the cluster
Set up monitoring and observability
Deploy and monitor the microservices to the cluster

Let's get started!

Prerequisites

A Google Cloud Platform account
Docker installed on your machine
Node.js installed on your machine
JHipster installed on your machine
Google Cloud SDK installed and configured on your machine
kubectl or KDash
Basic understanding of Java, Spring, Containers, and Kubernetes

If you prefer to follow along by watching a video, see [How to Build Low-Code Microservices on the Cloud Using Istio, JHipster, and Kubernetes] on the OktaDev YouTube channel.

Why build cloud-native microservices using a service mesh?

Before we dive into building a cloud-native microservice stack, let's look at what a service mesh is and the benefits of using one.

A service mesh provides features to help with common distributed microservice challenges. Like service discovery, routing, load balancing, and so on. Today we will be using Istio, one of the most popular service mesh solutions available. Istio is tailored for distributed application architectures, especially those you might run in Kubernetes. Istio plays nicely with Kubernetes, so nicely that you might think that it's part of the Kubernetes platform itself. Istio isn't the only service mesh around; we also have platforms like Linkerd and Consul, which are also quite popular.

Istio specifically provides the following features.

Secure service-to-service communication over TLS. Of course, with support for identity-based authentication and authorization.
Service discovery, so that your microservices can discover each other.
Automatic load balancing for the services
Traffic control features like routing, circuit breaking, retries, fail-overs, and fault injection.
A pluggable policy layer that can enforce stuff like access control, rate limiting, A/B testing, traffic splits, quotas, etc.
It also provides automatic metrics, logs, and traces for all traffic within the cluster from Ingress to Egress and between pods.

What is Istio Service Mesh?

Let's take a quick look at Istio internals. The Istio architecture can be classified into two distinct planes.

Control plane: It consists of the istiod demon, and it manages and configures the envoy proxies to route traffic. The control plane also enforces policies and collects telemetry, and includes components like Pilot for traffic management, Citadel to manage security, and Galley to manage configurations.

Data plane: It's made of Envoy proxies deployed as sidecars to our application containers. Envoy is a high-performance, lightweight distributed proxy. It controls all the incoming and outgoing traffic to the container it is attached to.

We can use tools like Grafana, Prometheus, Kiali and Zipkin for monitoring and observability as they work well with the telemetry provided by Istio. You can use these or use your existing monitoring stack as well.

Build a Java Microservices Stack using JHipster

Before you proceed, ensure you have installed JHipster. If not, install it using the command npm -g install generator-jhipster. At the moment of writing, I'm using JHipster version 7.8.1.

We will be using the JHipster Domain Language (JDL) to define our microservices, entities, and deployment options. But first, let's take a look at the architecture we will be building today.

We have the Istio control plane taking care of policy, load balancing, etc. We also have the Istio Ingress gateway to route all external traffic to our applications. We have four microservices. First is a gateway application created by JHipster that acts as our React GUI and authentication layer. The remaining are services that provide APIs. Each of our containers will have an envoy proxy as an auto-injected sidecar. We hook up Grafana, Prometheus, Zipkin, and Kiali to the telemetry provided by Istio so that we have monitoring and observability for our cluster. Each microservice also has its own database.

If you would prefer not to build the application yourself, clone the example from GitHub.

git clone https://github.com/oktadev/okta-java-spring-k8s-istio-microservices-example.git

It's not an overly complex architecture, but it's also not that simple. First, let us define our microservice using JDL. Create a file called app.jdl and paste the following content into it.

application {
  config {
    baseName store
    applicationType gateway
    packageName com.okta.developer.store
    serviceDiscoveryType no
    authenticationType jwt
    prodDatabaseType postgresql
    cacheProvider hazelcast
    buildTool gradle
    clientFramework react
  }
  entities *
}

application {
  config {
    baseName product
    applicationType microservice
    packageName com.okta.developer.product
    serviceDiscoveryType no
    authenticationType jwt
    prodDatabaseType postgresql
    cacheProvider hazelcast
    buildTool gradle
    serverPort 8081
  }
  entities Product, ProductCategory, ProductOrder, OrderItem
}

application {
  config {
    baseName invoice
    applicationType microservice
    packageName com.okta.developer.invoice
    serviceDiscoveryType no
    authenticationType jwt
    prodDatabaseType postgresql
    buildTool gradle
    serverPort 8082
  }
  entities Invoice, Shipment
}

application {
  config {
    baseName notification
    applicationType microservice
    packageName com.okta.developer.notification
    serviceDiscoveryType no
    authenticationType jwt
    databaseType mongodb
    cacheProvider no
    enableHibernateCache false
    buildTool gradle
    serverPort 8083
  }
  entities Notification
}

Each application defines its name, package name, authentication type, database, etc. For all supported options and configurations, please refer to the JDL applications documentation. Each application also defines the applicationType and the entities it serves. Next, add the entity definitions to the app.jdl you just created.

/**
 * Entities for Store Gateway
 */
// Customer for the store
entity Customer {
  firstName String required
  lastName String required
  gender Gender required
  email String required pattern(/^[^@\s]+@[^@\s]+\.[^@\s]+$/)
  phone String required
  addressLine1 String required
  addressLine2 String
  city String required
  country String required
}

enum Gender {
  MALE, FEMALE, OTHER
}

relationship OneToOne {
  Customer{user(login) required} to User
}

service Customer with serviceClass
paginate Customer with pagination

/**
 * Entities for product microservice
 */
// Product sold by the Online store
entity Product {
  name String required
  description String
  price BigDecimal required min(0)
  itemSize Size required
  image ImageBlob
}

enum Size {
  S, M, L, XL, XXL
}

entity ProductCategory {
  name String required
  description String
}

entity ProductOrder {
  placedDate Instant required
  status OrderStatus required
  code String required
  invoiceId Long
  customer String required
}

enum OrderStatus {
  COMPLETED, PENDING, CANCELLED
}

entity OrderItem {
  quantity Integer required min(0)
  totalPrice BigDecimal required min(0)
  status OrderItemStatus required
}

enum OrderItemStatus {
  AVAILABLE, OUT_OF_STOCK, BACK_ORDER
}

relationship ManyToOne {
  OrderItem{product(name) required} to Product
}

relationship OneToMany {
  ProductOrder{orderItem} to OrderItem{order(code) required}
  ProductCategory{product} to Product{productCategory(name)}
}

service Product, ProductCategory, ProductOrder, OrderItem with serviceClass
paginate Product, ProductOrder, OrderItem with pagination
microservice Product, ProductOrder, ProductCategory, OrderItem with product

/**
 * Entities for Invoice microservice
 */
// Invoice for sales
entity Invoice {
  code String required
  date Instant required
  details String
  status InvoiceStatus required
  paymentMethod PaymentMethod required
  paymentDate Instant required
  paymentAmount BigDecimal required
}

enum InvoiceStatus {
  PAID, ISSUED, CANCELLED
}

entity Shipment {
  trackingCode String
  date Instant required
  details String
}

enum PaymentMethod {
  CREDIT_CARD, CASH_ON_DELIVERY, PAYPAL
}

relationship OneToMany {
  Invoice{shipment} to Shipment{invoice(code) required}
}

service Invoice, Shipment with serviceClass
paginate Invoice, Shipment with pagination
microservice Invoice, Shipment with invoice

/**
 * Entities for notification microservice
 */
entity Notification {
  date Instant required
  details String
  sentDate Instant required
  format NotificationType required
  userId Long required
  productId Long required
}

enum NotificationType {
  EMAIL, SMS, PARCEL
}

microservice Notification with notification

We define entities for each service and mark the entities as microservice entities. We also define relationships between entities, enums, and other options like pagination, service layer, etc. Please refer to JDL Entities and relationships documentation for more possibilities.

Now, we are ready to run JHipster. Open a terminal window on the folder where you saved the JDL and run the following command.

mkdir jhipster-istio
cd jhipster-istio

jhipster jdl app.jdl --fork

This will create the applications with all their entities and specified configurations. You should be able to see the gateway application in action by running the following command on the store folder.

./gradlew # starts the Spring Boot application

You can see everything that's generated in the example application on GitHub.

Create a GKE cluster and install Istio

To deploy the stack to Google Kubernetes Engine, we need to create a cluster and install Istio. So let's begin by creating a cluster using Google Cloud SDK.

Create a cluster

Ensure you are logged into the gcloud CLI from the command-line and run the following command to create a GKE cluster.

# set region and zone
gcloud config set compute/region europe-west1
gcloud config set compute/zone europe-west1-b
# Create a project and enable container APIs
gcloud projects create jhipster-demo-okta # You need to also enable billing via GUI
gcloud config set project jhipster-demo-okta
gcloud services enable container.googleapis.com

# Create GKE Cluster
gcloud container clusters create hello-hipster \
   --num-nodes 4 \
   --machine-type n1-standard-2

This could take anywhere from 5 to 15 minutes. --machine-type is important as we need more CPU than available in the default setup. Once the cluster is created, it should be set automatically as the current Kubernetes context. You can verify that by running kubectl config current-context. If the new cluster is not set as the current context, you can set it by running gcloud container clusters get-credentials hello-hipster.

Note: I'm using KDash to monitor the cluster; you can try it or use kubectl, k9s, and so on as you prefer.

Install Istio to cluster

As of writing this, I'm using Istio version 1.13.4. You can install istioctl by running the following command, preferably from your home directory.

export ISTIO_VERSION=1.13.4
curl -L https://istio.io/downloadIstio | sh -
cd istio-$ISTIO_VERSION
export PATH=$PWD/bin:$PATH

You should now be able to run istioctl from the command line. Now, we can use the CLI to Install Istio to the GKE cluster. Istio provides a few Helm profiles out of the box. We will use the demo profile for demo purposes. You can choose the production or dev profile as well. The command should install Istio and set up everything required on our cluster.

istioctl install --set profile=demo -y

Note: If you run into any trouble with firewall or user privilege issues, please refer to GKE setup guide from Istio.

Once the installation is complete, we need to fetch the External IP of the Istio Ingress Gateway. If you are using KDash, you can see it on the services tab, or you can run the following command to get it using kubectl.

kubectl get svc istio-ingressgateway -n istio-system

Install observability tools

Istio also provides addons for most of the popular monitoring and observability tools. Let's install Grafana, Prometheus, Kiali and Zipkin on our cluster. These are preconfigured to work with the telemetry data provided by Istio. Ensure you are in the folder where you installed Istio, like istio-1.13.4.

cd istio-$ISTIO_VERSION
kubectl apply -f samples/addons/grafana.yaml
kubectl apply -f samples/addons/prometheus.yaml
kubectl apply -f samples/addons/kiali.yaml
kubectl apply -f samples/addons/extras/zipkin.yaml

If we look at the istio-system namespace, we can see all the Istio components along with Grafana, Prometheus, Kiali, and Zipkin running. You can also see this by running the following command .

kubectl get pods -n istio-system

Deploy the microservice stack to GKE

Our cluster is ready, and we have Istio installed. Now, we can deploy our microservice stack to the cluster. First, we need to create Kubernetes manifests for our deployments and services and configurations for Istio. And once again, JHipster comes to the rescue. We can use the JDL deployment configurations to generate Kubernetes setup for our stack with one command easily.

Create Kubernetes manifests

Create a new JDL file, say deployment.jdl, and add the following content.

// will be created under 'kubernetes' folder
deployment {
  deploymentType kubernetes
  appsFolders [store, invoice, notification, product]
  dockerRepositoryName "<your-docker-repository-name>"
  serviceDiscoveryType no
  istio true
  kubernetesServiceType Ingress
  kubernetesNamespace jhipster
  ingressDomain "<istio-ingress-gateway-external-ip>.nip.io"
  ingressType gke
}

I hope it's self-explanatory. You can refer to the JDL deployment documentation for all the available options. We have enabled Istio and set the ingress domain to the Istio Ingress Gateway's external IP that we noted earlier. Now we need a DNS for our IP. For real use-cases, you should map a DNS for the IP using a service provided by your cloud vendor like Google Cloud DNS, but for testing and demo purposes, we can use a wildcard DNS service like nip.io to resolve our IP. Just append nip.io to our IP and use that as the ingressDomain. Make sure to use a docker repo where you have push rights.

Now run the following command from the root folder where you ran the previous jhipster jdl command.

jhipster jdl deployment.jdl

This will create a new folder, kubernetes, with all the required Kubernetes manifests like deployments, services, Istio virtual services, gateways, and so on, for all the applications, databases, and monitoring.

Each of the services will also have an Istio virtual service and destination rule. For example, the invoice service will have the following destination rule defining traffic policies.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: invoice-destinationrule
  namespace: jhipster
spec:
  host: invoice
  trafficPolicy:
    loadBalancer:
      simple: RANDOM
    connectionPool:
      tcp:
        maxConnections: 30
        connectTimeout: 100ms
      http:
        http1MaxPendingRequests: 10
        http2MaxRequests: 100
        maxRequestsPerConnection: 10
        maxRetries: 5
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 60s
  subsets:
    - name: v1
      labels:
        version: "v1"

It also includes the following virtual service that defines the route. You could also use virtual services to do traffic split between two versions of the same app, among other things.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: invoice-virtualservice
  namespace: jhipster
spec:
  hosts:
    - invoice
  http:
    - route:
        - destination:
            host: invoice
            subset: "v1"
          weight: 100
      retries:
        attempts: 3
        perTryTimeout: 2s

The gateway is defined for the store application as it is our GUI as well.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: store-gateway
  namespace: jhipster
  labels:
    gateway: store-gateway
    istio: ingressgateway
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - store.jhipster.34.76.233.160.nip.io
    - port:
        number: 80
        name: http2
        protocol: HTTP2
      hosts:
        - store.jhipster.34.76.233.160.nip.io
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: store-gw-virtualservice
  namespace: jhipster
  labels:
    service: store-gw-virtualservice
spec:
  hosts:
    - store.jhipster.34.76.233.160.nip.io
  gateways:
    - store-gateway
  http:
    - match:
        - uri:
            prefix: /services/invoice/
      rewrite:
        uri: /
      route:
        - destination:
            host: invoice
    - match:
        - uri:
            prefix: /services/notification/
      rewrite:
        uri: /
      route:
        - destination:
            host: notification
    - match:
        - uri:
            prefix: /services/product/
      rewrite:
        uri: /
      route:
        - destination:
            host: product
    - route:
        - destination:
            host: store

As you can see, there are also many useful commands printed on the console that you can use to do the deployment.

Deploy to GKE

We are ready to deploy now. First, we need to build and push the images to the registry. We can use the handy Jib commands provided by JHipster. Navigate to each of the microservice folders and run the commands below.

cd store && ./gradlew bootJar -Pprod jib -Djib.to.image=yourDockerRepository/store
cd invoice && ./gradlew bootJar -Pprod jib -Djib.to.image=yourDockerRepository/invoice
cd notification && ./gradlew bootJar -Pprod jib -Djib.to.image=yourDockerRepository/notification
cd product && ./gradlew bootJar -Pprod jib -Djib.to.image=yourDockerRepository/product

Once the images are pushed to the Docker registry, we can deploy the stack using the handy script provided by JHipster. Navigate to the kubernetes folder created by JHipster and run the following command.

cd kubernetes
./kubectl-apply.sh -f

Once the deployments are done, we must wait for the pods to be in RUNNING status. Useful links will be printed on the terminal; make a note of them.

You can now access the application at the given http://store.jhipster.<istio-ingress-gateway-external-ip>.nip.io URI and log in with the default credentials.

Note: Currently, the JHipster OIDC setup does not work with Istio and there is an open issue in JHipster issue tracker for this. Alternative solutions would be to use an external authorization server with something like Open Policy Agent. We will cover this in a later blog post.

Monitoring and observability

Since we deployed tools for observability, let's see what we have.

Grafana

First up are Grafana and Prometheus for metrics and dashboards. Click the URI for Grafana from the previous deployment step. Click General at the top left corner and click the istio folder. You should see multiple preconfigured dashboards. You can monitor the performance of the workloads and the istio system itself here. You can also create your own dashboards if you like. Prometheus provides the data visualized on Grafana.

Kiali

Kiali is a management console for Istio service mesh, and it provides a web interface for visualizing the network topology of your service mesh. You can use it to explore the network topology of your cluster and see the network traffic flowing through it. Click Graph on the left side menu to see the network topology.

Zipkin

Zipkin is a distributed tracing solution for distributed systems. It is a tool for capturing distributed traces and providing a centralized view of the traces. This is essential for a microservice setup where a request could span multiple services, and debugging would require tracing them. Click RUN QUERY on the home screen to fetch recent traces, and click SHOW on one of them.

Cleanup the GCP cluster

Once you are done with experiments, make sure to delete the cluster you created so that you don't end up with a big bill from Google. You can delete the cluster from the Google Cloud Console GUI or via the command line using the following command.

gcloud container clusters delete hello-hipster

Learn more about Java Microservices, Istio, Kubernetes, and JHipster

If you want to learn more about Kubernetes, OIDC, or using OIDC with Kubernetes, and security in general, check out these additional resources.

You can find all the code from this example on GitHub.