DEV Community: Karthik Unnikrishnan

I Tried Getting Closer to the GPU With Triton

Karthik Unnikrishnan — Sat, 29 Aug 2026 16:48:22 +0000

I've been trying to understand what actually happens when a piece of ML code reaches the GPU. Not the usual model.cuda() and GPU go brrrr kind of understanding, but the actual stuff underneath it — threads, warps, memory, kernel launches, and why some operations are ridiculously fast while others suddenly become slow.

That rabbit hole eventually led me to Triton.

At first, I was wondering why anyone would even bother writing custom GPU kernels when PyTorch already gives us highly optimized operations. But the more I looked into it, the more I realized that GPU performance isn't always about how much computation you're doing. A lot of the time, it's about how you're moving data around.

CUDA gives you an insane amount of control over the GPU, but that control comes with a lot of complexity. You have to think about threads, warps, synchronization, memory access, and a bunch of other low-level details. That's great if you're comfortable with CUDA, but if you're coming from Python and PyTorch, it can be a pretty steep jump.

PyTorch makes this much easier through its higher-level abstractions. You can just write something like torch.softmax() and let the framework handle everything underneath. The problem is that when you're executing operations eagerly, individual operations can result in separate kernel launches and repeated reads and writes to global GPU memory. The code is easy to write, but there can be a lot of unnecessary movement of data happening underneath.

And this is where Triton becomes interesting.

Triton is a Python-based language and compiler for writing GPU kernels. What I found interesting about it is that instead of forcing you to think about individual GPU threads, it lets you think in terms of blocks of data. You basically describe the work that one program instance should perform, and Triton takes care of launching many of those program instances across the GPU.

That changed the way I started thinking about GPU programming.

Before getting into Triton, I had to understand the GPU itself a little better. A GPU is built around massive parallelism. Instead of having a small number of powerful CPU cores, GPUs have huge numbers of lightweight threads that can execute work simultaneously. Those threads are organized into warps and blocks, and the actual hardware that executes these warps is organized into Streaming Multiprocessors.

But the part that really matters for performance is the memory hierarchy.

GPU memory isn't just one big pool where everything costs the same to access. You have registers, shared memory, caches, and global memory. Registers are extremely close to the computation and very fast, while global memory is much larger but significantly more expensive to access. So if your algorithm keeps loading data from global memory, performing a tiny amount of work, writing it back, and then loading the same data again, you're potentially wasting a huge amount of time.

The basic idea I started taking away was pretty simple: move data as little as possible and reuse it as much as possible.

My first Triton experiment was just vector addition. Nothing fancy. Given two arrays, add them together. But even this simple example helped me understand the programming model. Instead of launching a thread for every individual element, I could define a block size and let each Triton program instance handle a chunk of the input.

For example, if I have 1024 elements and a block size of 128, I can think of the workload as eight program instances. The first handles elements 0 to 127, the second handles 128 to 255, and so on. Triton gives each program instance an ID, and I can use that ID to calculate which section of memory it needs to work on.

The next thing that confused me initially was pointers. Triton kernels don't work with tensors in the same high-level way normal PyTorch code does. They work with pointers to GPU memory. You calculate offsets from those pointers, load the values you need, perform the computation, and then store the results back.

This is also where masks become important. If your input size isn't perfectly divisible by your block size, the final program instance might be assigned elements that don't actually exist. You obviously don't want the GPU trying to access memory outside the tensor, so you use a mask to make sure only valid elements are loaded and stored.

After vector addition, I moved on to something more interesting: softmax.

A straightforward softmax implementation involves several operations. You find the maximum value, subtract it for numerical stability, calculate the exponential, sum the results, and finally divide by the sum. Each of these operations can involve reading and writing data.

So conceptually, you can end up with something like: load data, calculate something, write it back, load it again, calculate something else, write it again, and repeat.

The data is constantly travelling between global memory and the compute units.

Instead of doing that, we can fuse the operations into one kernel. The idea is to load the data once, keep it on-chip while performing the different operations, and only write the final result back to global memory.

This is one of the things that made Triton click for me. The code isn't necessarily about doing more computation. It's about avoiding unnecessary memory traffic.

Then came matrix multiplication, which made the whole concept of tiling much more obvious.

When multiplying two matrices, a naive implementation can repeatedly load the same pieces of the input matrices from global memory. But if you're computing multiple nearby output values, you're often reusing the same data. So instead of calculating one output element at a time, you divide the matrices into smaller tiles.

You load a small tile of each input matrix into faster on-chip memory, perform as much computation as possible using those tiles, accumulate the result, and then move on to the next tile.

The fundamental idea is basically: load less often, reuse more.

That sounds ridiculously simple, but it is one of the core ideas behind high-performance matrix multiplication on GPUs.

Once I understood tiling, some of the other Triton concepts started making more sense too. Program IDs can be used to determine which output tile a program instance is responsible for. Strides let you calculate where elements actually live in memory. Masks handle boundaries. The accumulator keeps partial results while working through different chunks of the reduction dimension.

Then there is scheduling and cache locality.

If multiple output tiles reuse the same pieces of input data, it makes sense to execute those tiles close together so that the data has a better chance of remaining in cache. Triton provides mechanisms for grouping program instances to take advantage of this kind of locality.

And then you get to autotuning.

Choosing the perfect block size, number of warps, pipeline stages, and other parameters manually isn't always easy. Different workloads and different GPUs can behave differently. Triton's autotuning system lets you provide multiple configurations and benchmark them to find a better configuration for a particular workload.

This is another thing I found pretty interesting. Instead of assuming there is one perfect configuration, you can basically let the system search through several possibilities and keep the one that performs best.

Eventually I started benchmarking the kernels instead of just looking at the code and assuming the optimized version was faster. And that was probably another important lesson.

GPU optimization isn't about writing code that looks complicated.

It's about understanding what the hardware is actually spending time doing.

Sometimes you're compute-bound. Sometimes you're memory-bound. Sometimes kernel launch overhead matters. Sometimes you're repeatedly moving data that could have stayed on-chip. And sometimes the library implementation you're trying to beat is already so optimized that your custom kernel isn't going to magically win.

That's probably the biggest thing I took away from learning Triton.

Before this, I used to think about GPU optimization mostly as “How do I make the GPU perform more computation?”

Now I find myself asking a slightly different question:

“Why am I moving this data in the first place?”

That shift in perspective is probably more valuable than any individual Triton kernel I wrote.

I'm definitely not claiming to be a GPU programming expert after doing this. If anything, the more I learn, the more I realize how much more there is to understand — occupancy, register pressure, memory coalescing, Tensor Cores, persistent kernels, FlashAttention, compiler behavior, and a lot more.

But at least now those topics don't feel completely alien.

And that's honestly why I wanted to write this.

I didn't want this to be another post that throws a complicated Triton kernel at you and says, "look how fast this is." I wanted to understand why the kernel is written that way and what the GPU is actually doing underneath it.

Because once you start thinking about GPUs in terms of data movement, reuse, parallelism, and locality, GPU programming starts making a lot more sense.

My CUDA/GPU Journey: From "What Even Is a GPU?" to Actually Fascinated

Karthik Unnikrishnan — Sun, 23 Aug 2026 17:45:13 +0000

A few months ago, if you'd asked me what a GPU actually does, I would've mumbled something about "graphics" and changed the subject. I'm an undergrad, self-taught in most of what I know about programming, and until recently, GPUs were just... background noise. Something gamers cared about. Something that showed up in specs sheets I didn't understand.

The moment it stopped being background noise

It started the way a lot of rabbit holes start: I kept seeing people online talking about RTX this, RTX that, benchmarks, VRAM, tensor cores — and it all went completely over my head. I'd nod along in threads I didn't understand, half-following conversations about why one card was "better" than another, quietly bothered that I had no idea what any of it meant.

At some point that turned into an actual itch. I don't like not knowing things, especially when it feels like everyone else is fluent in something I can't even parse. So I decided I had to figure out what a GPU actually was — not just as a gaming accessory, but as a piece of hardware people were clearly doing serious computational work with.

That's when I started noticing something else: people weren't just gaming on these things. I kept running into posts and videos of people running genuinely heavy workloads on GPUs — training models, running simulations, crunching numbers at speeds that made no sense to me coming from a CPU-only mental model. That was the real hook. Not the marketing, not the specs — the fact that people were using GPUs as general-purpose computing beasts, and I had zero idea how that was even possible.

Falling into CUDA (and discovering it's not the only player)

Once I started digging, I learned that "using a GPU for computation" isn't some universal, plug-and-play thing — it depends entirely on who made your hardware. NVIDIA has its own ecosystem called CUDA, and AMD has its own competing thing called ROCm. That distinction genuinely fascinated me. Two of the biggest hardware companies in the world, each with their own language and toolchain for talking to their own silicon.

A bit of digging confirmed what I'd suspected: CUDA is NVIDIA's proprietary platform, built specifically for NVIDIA GPUs, and it's been around since 2006 — which is a big part of why it has such a mature ecosystem of libraries, tools, and community support today. ROCm, on the other hand, is AMD's open-source answer, built around HIP (Heterogeneous-Compute Interface for Portability), which is designed to closely resemble CUDA's syntax so that code can be ported between the two with less pain. There's even a translation layer called HIPIFY that converts CUDA code into HIP so it can run on AMD hardware.

What struck me most was that this isn't just a technical footnote — it's basically a walled-garden situation. CUDA's maturity and dominance, especially in AI and deep learning workloads, is a big reason NVIDIA has had such a strong grip on the space. AMD is playing catch-up with an open-source strategy, hoping that openness eventually wins over raw maturity. As someone who'd never thought about hardware ecosystems as competing programming languages before, that was a genuinely new way of seeing the tech industry.

Where I'm at now

I'll be honest — I'm still very early. Right now my "journey" is less about writing optimized kernels and more about building the mental model: understanding what a thread, block, and grid even mean in CUDA terms, why memory transfer between CPU (host) and GPU (device) is such a big deal, and why "just throw it on the GPU" is a lot more nuanced than it sounds from the outside.

But that initial confusion — the feeling of being completely lost in a conversation about RTX cards — has turned into something I actually look forward to learning more about. I went from not knowing what a GPU was for, to being genuinely curious about parallel computing as a discipline. That's a bigger shift than I expected.

Why I'm writing this

I'm writing this partly for accountability, and partly because I know there are other self-taught devs out there who've felt that same "everyone else understands this but me" feeling. If that's you — you don't need a CS degree or a research lab to start. You just need the willingness to feel lost for a while before things click.

If you're also early in your GPU programming journey, or you've been down this road already and have advice, tools, or resources that helped things click for you, I'd genuinely love to hear about it in the comments. This is very much a "learning in public" post, not an expert one.

More updates to come as I actually start writing CUDA kernels instead of just reading about them.

From 4,000+ Applications to 250 Builders: Building a Multi-Agent Video Pipeline at the Google DeepMind Bangalore Hackathon

Karthik Unnikrishnan — Mon, 03 Aug 2026 17:18:56 +0000

TL;DR

Got selected out of 4,000+ applicants for the Google DeepMind Bangalore Hackathon 2026 (250 builders made it in). My teammate and I merged two hackathon tracks — Conversational Video & Motion with Omni Flash and Local-First Agents with Gemma 4 — into one pipeline: a multi-turn conversational video orchestration agent that asks clarifying questions and iteratively refines its output, backed by on-device reasoning for state management. No trophy this time, but a working prototype and a lot of lessons. Here's the build.

The selection

Google DeepMind and Cerebral Valley ran this hackathon in Bangalore, and the numbers alone made it feel different from the college-circuit events I'd been doing: 4,000+ applications, 250 seats. Getting the acceptance email didn't fully register until I was actually in the room — this wasn't a campus event anymore, it was a room full of people who build for a living.

The idea: merging two tracks instead of picking one

The hackathon offered several tracks. Two stood out to me and Rohan:

Conversational Video & Motion with Omni Flash — generate and iterate on video through natural conversation
Local-First Agents with Gemma 4 — on-device reasoning and autonomous decision-making

Instead of choosing one, we asked: what if the video generation system was the agent? So we built a pipeline where:

Gemini Omni Flash + NB2 Lite handle multi-turn conversational video orchestration — the system doesn't just take a prompt and spit out a video. It asks clarifying questions, holds context across turns, and iteratively refines the output based on what the user actually meant.
Gemma 4 runs on-device for state management and decision-making — tracking where the conversation is, what's already been generated, what still needs clarifying, and deciding the next action locally instead of round-tripping everything through a heavier model.

The result was an agent that blends reasoning, local intelligence, and multimodal generation into a single loop, rather than three separate systems bolted together.

What building under a deadline like this actually felt like

Tight deadlines are a hackathon constant, but the density of the room changes the pressure. Every table around us was shipping something legitimately interesting, which is equal parts motivating and intimidating. Having a partner who's good at both brainstorming and debugging (thanks, Rohan) mattered more here than in any hackathon I'd done before — there wasn't time to context-switch between "figuring out the architecture" and "fixing why the state manager is out of sync."

The outcome

We didn't place. No trophy this round. But we left with:

A working end-to-end prototype of the conversational video + local agent pipeline
Direct exposure to how a team like DeepMind's frames the "agentic + multimodal" problem space
Genuinely useful conversations with other builders working on adjacent problems
Insights from Amit Vadi and Ray Del Vecchio, who spoke at the event

Why I'm writing this up

Hackathons are noisy — you build for 24-48 hours, present, and the artifact usually dies in a GitHub repo somewhere. Writing it up is partly documentation for myself (what worked, what I'd do differently) and partly a way of tracking how the kind of problems I'm choosing to work on has shifted — from single-model chatbots to multi-agent, multimodal, local-first systems.

If you're building anything similar — conversational video generation, on-device agent state management, or just curious about how Omni Flash and Gemma 4 play together — I'd genuinely like to compare notes. Drop a comment or find me on GitHub.

This is part of an ongoing hackathon log — you can find the earlier entries (including two first-place wins and a near-miss with a legal-aid kiosk built on a multi-agent swarm) on my blog.

I Had Never Touched a Mobile Core Network. Then I Shipped a PR to Magma's Production Codebase

Karthik Unnikrishnan — Sun, 05 Jul 2026 17:40:13 +0000

A few weeks ago I had never touched a mobile core network. I didn't know what an EPC was, what AGW stood for, or why anyone would run a 4G stack on their laptop.

Today I've deployed one, fixed a production bug in it, and shipped a PR that's now part of the codebase. Here's how that happened.

What is Magma?

Magma is a Linux Foundation Networking project — an open-source mobile core network platform that lets you deploy 4G and 5G infrastructure without expensive proprietary hardware. Think of it as the software that sits between your phone and the internet when you're on a cellular network.

I joined as a mentee with basically zero telecom knowledge. What followed was one of the most intense technical learning experiences I've had.

Deploying the stack

The first challenge was just getting the thing running. Magma's Access Gateway (AGW) runs inside Docker containers, and the setup involves coordinating a lot of moving parts — orchestration, subscriber databases, the gateway itself. Getting through that process from scratch forced me to actually understand what each component does, not just follow steps.

Finding the Ubuntu 24.04 breakage

Once the environment was running, I started auditing the Dockerfiles. The project had accumulated dependencies that silently broke on newer Ubuntu:

python3-distutils — removed in Ubuntu 24.04, so installs fail quietly
libssl1.1 — not available on 24.04, replaced by libssl3

These aren't the kind of errors that scream at you. You have to actually check whether what the Dockerfile is requesting still exists in the target OS.

The fix: 14 files, one PR

The fix wasn't just swapping package names:

Updated the subscriberdb Dockerfile to use Ubuntu 24.04 as base
Replaced deprecated packages with their 24.04 equivalents
Updated the GitHub Actions CI workflow to test against a matrix of ubuntu-20.04 and ubuntu-24.04
Traced the breakage across 14 files where OS assumptions were baked in

The CI matrix means future contributors will catch OS-specific breakage before it ships — that felt meaningful. Not just fixing something, but making it harder to break again.

What surprised me

Open-source maintenance is a lot of archaeology. Code accumulates assumptions — about the OS, about which packages exist, about the Python version installed — and nobody documents them because at the time they were obvious. Finding them means reading carefully and asking what the code expects, not just what it does.

The second surprise: telecom infra sounds intimidating, but it's just software running in containers, with logs you can read like anything else.

What's next

I recently graduated from the Magma mentorship program (spring 2026 cohort) — credential here. Continuing to contribute to Magma while going deeper on the ML side — agentic AI frameworks, research papers, and projects connecting both worlds.

Originally posted on my blog — I write about ML, systems, and open source there.