DEV Community: Kat Padilla

My Notes on LLMs

Kat Padilla — Mon, 22 Jun 2026 13:35:13 +0000

These notes are my attempt to capture the concepts, observations and references that have helped me make sense of LLMs.

What is an LLM

A Large Language Model is a machine learning model trained to predict the next token in a sequence.

The first thing I had to unlearn: it's not a search engine. It doesn't look anything up by default. It generates responses based on patterns from training data. That distinction sounds small but it changes how you think about everything else.

Key concepts

Token

The unit everything is measured in. Roughly 3-4 characters in English text, which loosely maps to about 3/4 of a word. It varies by language and model though. Every message, every reply, every system instruction are all tokens. Most models have a hard cap on how many they can handle at once.

Parameters

The weights a model learned during training. Bigger models tend to perform better but it's not a clean relationship. Architecture, training data quality and what happened after training all matter too. Parameter count alone stopped being a reliable proxy for capability a while back.

Training vs inference

Training is the model learning from data. It happens once and is genuinely expensive. Inference is me talking to it in real time. I only ever interact during inference. The model's weights don't change mid-conversation. Whatever I tell it lives in the current context window, not in the model itself.

Knowledge cutoff

Training stopped at some date. Anything after that, the model doesn't know unless it has a search tool attached.

Hallucination

When the model generates confident-sounding but false information. It's pattern-matching without a fact-checker. More likely with obscure topics, long conversations or vague prompts. The rule I follow: verify anything that actually matters.

System prompt

Hidden instructions loaded before my first message. Sets the model's persona, rules and constraints. I never see it in most interfaces.

Temperature

Controls how predictable vs adventurous the model's word choices are.

Low (0-0.3): consistent, predictable. Good for code and factual answers.

High (0.7-1.0): varied, creative. Good for brainstorming and writing.

If AI-generated code keeps being weird, temperature is probably too high. If brainstorming outputs all sound identical, too low. Most interfaces default around 0.7. In higher-level interfaces like ChatGPT or Claude.ai, you can't set it directly so prompt phrasing does the steering instead.

Turns

A turn is one exchange: I send a message, the model replies. That's it.

The model doesn't pick up where it left off. Every turn, it re-reads the entire conversation from the beginning: the system prompt, every message I've sent, every reply it's given, any files I've attached. All of it, every single time.

There's no memory carried over between turns. No state. The conversation history only exists because it gets passed back in as part of the input. If it's not in the context window, it's gone.

This is also why the model can feel inconsistent mid-conversation. It's not drifting. It's reading more and more tokens each turn and, past a certain point, the earlier parts of the conversation start getting less attention. That re-read behavior is exactly what makes the context window worth understanding.

The context window

Everything the model can "see" at once, measured in tokens. Not memory. More like a working scratchpad. Everything the model needs to know about the current task has to fit inside it.

The smart zone and the dumb zone

Despite huge advertised context windows, the model doesn't process all tokens equally. This framing comes from Dex Horthy at AI Engineer World's Fair and it's the most useful mental model I've found for it.

The smart zone is the earlier portion of the context window. Full attention, good reasoning, follows instructions well.

The dumb zone kicks in as the context fills up. Tokens are still technically there, nothing gets deleted, but coherence starts degrading. The model forgets things, gets repetitive, gets vague and starts agreeing with everything.

Degradation tends to kick in well before the model hits its actual context limit. The exact threshold varies by model.

The U-curve

LLMs pay the most attention to the beginning and end of context. The middle gets ignored. So as a conversation grows, the early stuff, the instructions I set at the start, those drift into exactly the part the model stops caring about. Stanford researchers found real performance drops when relevant information sat in the middle of long contexts.

Context rot

The gradual degradation of LLM performance as context grows. Chroma ran a study across 18 frontier models and found performance decreasing with longer inputs across all tested models, not just some of them.

💡 When the AI starts saying "you're absolutely right" to everything or forgets what I told it earlier, that's the dumb zone. The fix is starting fresh.

Warning signs:

Repeating suggestions it already made
"You're absolutely right" then ignores what I said
Forgets rules or constraints from earlier
Answers get generic and vague

What eats context fast:

Long conversations
Pasted files or big code blocks
Tool integrations (can eat a significant chunk before I type anything)
The model's own long replies, those go back in too

Context engineering

Managing what's in the context window deliberately so the model stays in the smart zone. A bit different from prompt engineering.

Prompt engineering: writing a better single prompt.

Context engineering: designing what information reaches the model at all, instructions, retrieved documents, tool outputs, conversation history.

What I do in practice to stay in the smart zone:

Keep prompts specific. (Vague in, vague out!)
Start fresh often. New topic means new conversation.
Front-load the important stuff. Given the U-curve, critical instructions buried mid-conversation get less attention.
When context gets full, prefer a manual handoff over letting the model auto-summarize. A model that's already degraded writing its own summary is not ideal. I use Matt Pocock's /handoff skill for this.
Don't trust large context windows blindly. 1M tokens doesn't mean 1M tokens of reliable reasoning.
Verify anything that matters. Hallucination is real.

💡On the auto-summarize thing: by the time the context needs summarizing, the model is already in the dumb zone. A degraded model writing its own summary is not a reliable summary!

What I'm still figuring out

RAG (Retrieval-Augmented Generation)

Instead of stuffing everything into the context window, the system retrieves only the relevant pieces when needed, pulling from a database, a document store or a knowledge base. The model never sees the full corpus, just what's relevant to the current query.

Embeddings

Numerical representations of text that capture meaning. Similar ideas end up close together in vector space, which makes semantic search possible.

Embeddings are the foundation of RAG, vector databases and many AI memory systems.

Training

I understand the basic idea: expose the model to enormous amounts of text and for each token it predicts, adjust the weights slightly based on whether it was right. Repeat billions of times. But I don't have a clear picture of what that actually looks like at scale yet.

References

It's Markdown's World And We're All Just Living In It

Kat Padilla — Sat, 06 Jun 2026 07:50:08 +0000

Funny how I randomly thought: how the heck did .md files end up basically running the show now? Not just in repos as READMEs. They're literally load-bearing walls in the AI ecosystem and I would've laughed if you told me this a few years back.

It's a weird arc for a plain text file.

The README Era

For most of my career, Markdown meant one thing: README files. You wrote one when you pushed a repo. It explained what the project was, how to run it, maybe a quick example. Nobody was precious about it. It was the file you wrote last, usually when you were already done with your project and slightly exhausted.

GitHub made it the default in 2008 and that was pretty much that. It lived in repos. It rendered nicely. That's it.

Nobody sat down and thought: this is going to matter.

And yet here I am, writing this very post in Markdown as this blog runs on Ghost, and Ghost is Markdown under the hood. Even my note-taking app is Obsidian, which stores every single note as a .md file. I've been living in Markdown this whole time without really registering it.

The Quiet Takeover

That thought made me search. And the history is almost too on-the-nose.

John Gruber made it in 2004 because he just didn't want to write HTML by hand. The entire origin story is "this was annoying, let me fix it for myself." Aaron Swartz helped. The first converter was a Perl script.

That's it. That's the founding mythology of what is now, without exaggeration, critical infrastructure.

Stack Overflow shipped with Markdown support in 2008. GitHub made it the face of every open source project. Static site generators ran on it. Note-taking apps adopted it. And then slowly, quietly, it became the format developers just used, not because someone mandated it, but because plain text that renders well is genuinely hard to beat.

It won because it was portable. Humans can read it even without rendering. Machines can parse it easily. Copy-pasting between systems just works. It's not perfect. Not elegant. Just compatible with almost everything.

And then AI showed up.

Plot Twist

AI models output Markdown by default now. Every response, every structured answer, every tool output. Headers, bullet points, bold text, code blocks. All of it formatted in a syntax that one blogger invented because he hated angle brackets.

Agents read it. Agents write it. Headings become context boundaries. Bullet points become instructions. Code fences become executable blocks. Systems are being built where .md files are the interface between humans and AI, the thing you edit to change how an agent thinks, what it knows, what it does.

SKILL.md. AGENT.md. README.md as configuration.

It's not a document format anymore. It's an instruction layer.

The format is simple enough for humans but structured enough for machines. That balance is surprisingly rare. And that's exactly why it stuck.

Kind of Wild

Gruber didn't design it for agents. He designed it to get out of his own way while writing. And now that's exactly why it works, because it carries meaning without ceremony. No tags. No schemas. No overhead.

The trillion-dollar AI industry is running on a plain text format someone made for their blog in 2004. Not because it was engineered for this. Because it was simple enough to survive everything.

I don't know what that says exactly. But I find it kind of wild.

Brain Dump: AgentCore Gateway

Kat Padilla — Fri, 05 Jun 2026 12:30:56 +0000

As you build AI agents, they need to call tools: APIs, databases, services. Traditionally you'd write an MCP server for each one. Server scaffolding, tool descriptions, error handling, auth, observability, repeated across every team, every API.

TIL: You don't always need to write an MCP server

If you already have REST APIs with OpenAPI specs, AgentCore Gateway can wrap them and expose them as MCP-compatible tools automatically. Just point it at your spec and your endpoints become tools agents can call.

It also handles inbound auth (who can call the gateway) and outbound auth (how the gateway calls your backend) which are all managed by AWS.

Targets

AgentCore Gateway supports multiple target types. You can wire in AWS Lambda functions, native MCP servers and prebuilt connectors (Slack, Jira, etc. with managed OAuth). One gateway can manage multiple targets, so you end up with a single MCP endpoint that fronts many different backends.

Tool descriptions quality = agent quality

The documentation in your OpenAPI spec becomes what the agent reads to decide which tool to call and how. Vague or incomplete descriptions directly hurt agent behavior. It may pick the wrong tool or misuse parameters. So apparently, good descriptions aren't just nice to have anymore, they're part of making the agent work correctly.

Size matters in response payloads

If your REST endpoints return large, noisy payloads, that gets passed straight to the agent. This is a real concern. Bloated responses eat context window and can confuse the agent. Something to think about before wrapping an API as-is.

Request and response interceptors

AgentCore Gateway supports interceptors on both sides of a call.

Request interceptors run before the call reaches your backend. Useful for injecting auth, adding headers or modifying the request payload on the way in.

Response interceptors run before the result goes back to the agent. The main use I have in mind is trimming down fat responses so they don't bloat the agent's context. That said, I'm not sure yet if response interceptors are actually the right tool for payload trimming or if it even works well in practice. Still to be validated.

tl;dr

Agent -> AgentCore Gateway -> your REST APIs / MCP Servers

Not a silver bullet but it cuts out the MCP server grunt work if you have existing REST endpoints.

Dev Journal: Jot (Chrome Extension)

Kat Padilla — Fri, 03 Apr 2026 07:45:00 +0000

Motivation

I am working on a 14" personal laptop and I needed a fast, minimal notepad directly in the browser because because context switching is a pain. Existing options required accounts or had unnecessary features that I do not want.

P.S. This is practically a side quest. I was working on a different project when I got bothered switching tabs and desktop spaces.

Goal/s

Learn how to build a simple Chrome Extension
Build a Chrome extension popup notepad

Dev Log

Browsed https://developer.chrome.com/docs/extensions/get-started/tutorial/hello-world as a primer.
Created a Chrome extension with manifest.json, popup.html, popup.css, and popup.js
Implemented a textarea in popup.html and wired it to localStorage in popup.js
Added a clear button and hooked it to reset both UI state and stored data
Added a character counter (just for the heck of it)
Structured the UI into header, body, and footer sections in popup.css
Added a light/dark mode toggle and persisted the selected theme in localStorage
Implemented a "Copy all" button using the Clipboard API
Wrote the Chrome Web Store listing fields:
- manifest.json description (105 chars)
- single-purpose statement
- privacy policy page (ikr?)
Packaged the extension into a zip
Submitted the extension through the Chrome Web Store Developer Dashboard

This blog is harder than popup.js itself because it simply does the following:

Load = restore note and theme from localStorage on popup open
Auto-save = persist note on every keystroke
Count = update character count
Copy = copy content to clipboard with feedback
Clear = reset note and storage

Result

Currently, the extension has been packaged and submitted to the Chrome Web Store. Internet says it usually takes 2-5 days.

I am already able to use it via Developer Tools. I just had to put it on Chrome Web Store and see what the process is like in case I want to build other extensions in the future.

Lessons

Chrome Web Store requires a privacy policy URL even if no data leaves the browser
localStorage in extensions is scoped to the extension origin, not the regular webpage context. So, inspecting extension storage requires opening DevTools on the popup, not the main page

Github Repo: https://github.com/katpadi/jot/

Update: it's now published https://chromewebstore.google.com/detail/jot/gogmghfcgficidobmgjbpfdejaaekgid. Only took less than 24h to get approval.

Handling Errors Gracefully

Kat Padilla — Thu, 05 Apr 2018 15:09:36 +0000

Sometimes it makes sense to treat errors as part of an application's "valid" flow of events. Meaning, expected errors should not necessarily make your code go crazy when random errors are raised.

With that in mind, I created a way to implement some kind of "Result Object" by rescuing StandardError instances and handling these errors gracefully.

class Gracefully
  class << self
    alias_method :handle, :new
  end

  attr_reader :error, :value

  def initialize
    @value = nil
    @error = nil
    begin
      @value = yield
    rescue => e
      @error = e
    end
  end

  def success?
    error.nil?
  end

  def on_success
    return self unless success?
    result = yield(@value)
    return result if result.is_a?(Gracefully)
    Gracefully.handle { result }
  end

  def on_failure
    return self if success?
    result = yield(@error)
    return result if result.is_a?(Gracefully)
    Gracefully.handle { result }
  end
end

gracefully.rb view raw

In short, _ Gracefully _ is a wrapper for a result object– it’s either a success value or an error object.

result = Gracefully.handle { Order.find 999 }

In the example above, if order with id 999 does not exist, result.success? method will return false. Otherwise, true.

If result.success?is true, result.value will contain the successful value. Otherwise, result.error will hold the error object itself.

Whenever Gracefully.handle is called, it will return a Gracefully object. If you want to get the raw value, you can use result.value.

More Usage

result = Gracefully.handle { ComplicatedService.new(foo, bar).process }
if result.success?
  # do success stuff
  use_value(result.value)
else
  # do failure stuff
  log_error(result.error)
end

Chaining Usage

Gracefully.handle { SomeComplicatedService.new.call }.on_success do |val|
  val.update!(column_name: 'stuff')
end

Gracefully.handle { SomeComplicatedService.new.call }.on_failure do |e|
  do_whatever_you_wish_with_the_error(e)
end

By using this, I can be resilient in handling errors and be able to recover from failures in the process flow.

The post Handling Errors Gracefully appeared first on KatPadi's Point.

--
Visit my website @ http://katpadi.ph.

What audiobook did you last listen to?

Kat Padilla — Tue, 20 Mar 2018 12:14:58 +0000

I listened to The Dispatcher by John Scalzi and it was awesome. My thoughts are here.

What did you last listen to?

Hi, I'm katpadi

Kat Padilla — Wed, 18 Jan 2017 03:52:16 +0000

I have been coding for ~7 years.

You can find me on Twitter as @katpadi

I live in Manila at the moment.

I work for a software company that helps brands increase their customer loyalty and engagement.

I mostly program in Ruby currently. I used to write PHP. I am also a bit functional with Python.

Nice to meet you.

TIL: Redis Keyspace Notifications

Kat Padilla — Thu, 12 Jan 2017 11:00:00 +0000

For some reason, I was figuring out a way to be able to listen to Redis so I can fire callbacks when the Redis command or event is encountered.

I wanted to do something like:

*When a key gets removed from the sorted set, it will trigger a script that will notify a user. *

Browsing online, I came across the so-called Redis Keyspace Notifications. It basically acts as a Pub/Sub mechanism that enables clients to subscribe to a Redis channel and receive “published” events whenever a Redis command or data alteration is encountered. The feature is available since version 2.8, apparently.

Diving in…

Literally a person who is diving in. LOL

The said notifications are enabled using the notify-keyspace-events of redis.conf or via the CONFIG SET. It is disabled by default.

Setup will look something like:

# Set it up.
$redis.config(:set, 'notify-keyspace-events', 'zE')

redis-kesypace-event.rb view raw

zE… wut???

zE looks so random if you weren’t briefed that each character in that string has a meaning.

Below is the list of variable mapping for keyspace events:

For my particular case, I needed to use “zE” string because I want to catch the name of the key (and not the name of the event) when zrem, which is a sorted set command, is encountered.

Keyspace v.s. Keyevent

Let’s cut to the chase! Use keyspace if you want to receive the name of the event. Use keyevent if you want to receive the name of the key.

TILception (TIL within a TIL): If you want to enable all events, use “KEA”.

So, here’s an example of what I played around with:

# Subscriber
$redis.psubscribe(' __keyevent@*__ :zrem') do |on|
  on.pmessage do |pattern, channel, message|
    puts "==== AFTER zrem EVENT ====="
    # Available things:
    puts "pattern: #{pattern}"
    puts "channel: #{channel}"
    puts "message: #{message}"
  end
end

redis-kesypace-event.rb view raw

As I told earlier, I wanted to catch the “key” as a message and not the “event”. So, the message variable in the above code will output the key.

Now, in your redis-cli, if you try to prepare a sorted set by using zadd and then doing a zrem, you will see the output in your console.

Now it’s time for me to send that notification!

The post TIL: Redis Keyspace Notifications appeared first on KatPadi's Point.

Git: Squash Commits

Kat Padilla — Mon, 12 Dec 2016 10:52:14 +0000

With git it’s possible to squash previous commits into one before sharing them with others.

Step 1

For example you want to squash your last 3 commits:

git rebase -i HEAD~3

On the other hand, if you want to just simply squash all the unpushed commits:

git rebase -i origin/master

If you have many commits and want to start from a certain commit:

git rebase -i

Any of the command above will prompt open your editor with something like:

pick a92f09 Added new feature X
pick ca9f90a Some other stuff I did
pick d18a6h1 More stuff I did

This will show up in your editor:

pick a92f09 Added new feature X
squash ca9f90a Some other stuff I did
squash d18a6h1 More stuff I did

Note: If you don’t have an editor defined in your config, you will encounter Could not execute editor. Just do git config --global core.editor /usr/bin/vim for you to be able to proceed.

Step 2

Next, we can configure git on what to do with the commits. Let’s say, I want to keep commit a92f09. Git squash-ing the following two commits into the first one will leave us with one single commit with all the commits in it. To do that, change your file to this:

pick a92f09 Added new feature X
squash ca9f90a Some other stuff I did
squash d18a6h1 More stuff I did

Save and exit.

That’s it. Git squash is especially useful if you want to wrap up “all in a day’s work” or if you want to have a clean and concise git history.

TL;DR

Use git rebase -i origin/master and replace “pick” on the second and succeeding commits with “squash”.

The post Git: Squash Commits appeared first on KatPadi's Point.