DEV Community: Orlando Ascanio

Prompt Caching: A Friendly Guide to Saving Money and Time in AI Applications

Orlando Ascanio — Sat, 21 Mar 2026 04:12:30 +0000

If you've been working with AI applications lately, you might have heard about "prompt caching." Sounds technical, But stick with me. I'm going to explain it in a way that's easy to understand, even if you're just starting out.

What's This Prompt Caching Thing?

Let's start with a simple analogy. Imagine you're in a classroom learning math. When you learn how to multiply 7 x 8, the teacher doesn't make you relearn it every single time you need to do that calculation. You remember it, and you use it whenever needed. That's kind of like caching!

In AI terms:

A prompt is what you send to the AI (like a question or instruction)
Caching means saving something so you don't have to create it again

So prompt caching is simply: when you send the SAME question or instruction to the AI over and over, instead of processing it fresh every time, the AI remembers it and uses the cached version. Much faster and cheaper!

Why Should You Care?

Okay, let's talk about the real benefits:

1. Save Money 💰

Here's the thing: AI services charge you based on what they process. If you're sending the same system instruction over and over (like "You are a helpful assistant who explains things clearly"), you're paying for it each time. With caching, you pay once, and the AI remembers it. I've seen companies save HUGE amounts, like 70-90% on some costs!

2. Faster Responses ⚡

When the AI doesn't have to re-read and re-process the same prompt, replies come back faster. This matters a lot if you're building apps where users are waiting for responses. Nobody likes waiting!

3. More Consistent Results 🎯

When the AI uses a cached version of your prompt, it's more likely to give you consistent answers. This is important if you need reliable outputs for your application.

Which Apps Benefit Most?

Not every app needs prompt caching. Let me break down which ones really shine with this technique:

✅ Customer Support Chatbots

These chatbots often use the same base instructions (like "Be friendly, helpful, and stay professional"). Caching this setup means:

Faster responses for customers
Lower costs when you're handling hundreds of conversations
Same polite tone, every time

✅ Content Generation Tools

Think of tools that write blog posts, social media captions, or product descriptions. They usually start with instructions like "Write in a helpful tone, include examples, keep it under 200 words." Cache those, and you:

Get faster content
Save money as you generate more pieces
Maintain consistent style across all outputs

✅ Multi-step AI Workflows

If you're building an app where the AI does several steps (like researching, then writing, then editing), the setup prompts for each step can be cached. This makes the whole workflow:

Faster end-to-end
More cost-effective
Easier to debug

✅ Educational Apps & Tutors

AI tutors that explain concepts, check homework, or provide feedback all use similar base instructions. Caching means:

Quicker answers for students
Lower costs as you serve more learners
Consistent teaching style

❌ Where It Might NOT Help

Some applications don't benefit as much:

Apps where every prompt is completely unique (though these are rare)
One-off use cases where you only make a few requests
Situations where prompts need to change every single time

How to Use It (Without Getting Lost in the Tech)

Good news: you don't need to be a coding wizard to use prompt caching! Most AI platforms now make this easy:

Use the same prompt structure for similar tasks
Keep your system instructions consistent (don't change the basic setup every time)
Leverage built-in caching features: many platforms now have this as a toggle or automatic feature

The key is planning ahead. When designing your AI app, think: "What parts of my prompts stay the same?" Those are your caching opportunities!

A Quick Example

Let's say you're building an AI app that generates product descriptions for an e-commerce site:

Without caching:

Every product: "You are an e-commerce copywriter. Write engaging product descriptions. Keep it under 150 words. Include features and benefits." (pay for this each time!)
Plus the actual product details
Result: Expensive, slower

With caching:

Cache the instruction: "You are an e-commerce copywriter. Write engaging product descriptions. Keep it under 150 words. Include features and benefits." (pay once!)
Plus the actual product details for each unique item
Result: Cheaper, faster, same quality! 🚀

Getting Started

Ready to try prompt caching? Here's a simple checklist:

Review your current prompts: What repeats the most?
Identify patterns: Look for instructions that stay the same
Check your AI provider's docs: Most now support caching (or will soon)
Start small – Cache one or two prompts first and see the difference
Monitor your results: Track costs and speed improvements

Final Thoughts

Prompt caching isn't some super advanced technique reserved for AI gurus. It's just smart optimization, like turning off lights when you leave a room. You wouldn't leave them on, right? 🤷‍♀️

Whether you're building a simple chatbot or a complex AI application, if you're sending similar prompts repeatedly, prompt caching can save you money and make your app faster. And that's something everyone can benefit from!

What do you think? Are you using prompt caching in your projects? Have you noticed a difference? Drop a comment below, I'd love to hear about your experiences! 👇

I built a Pi extension that logs every coding session.

Orlando Ascanio — Thu, 19 Mar 2026 23:36:37 +0000

Every session with an AI coding assistant ends the same way for me. I close the terminal and lose the decisions.

Not the code, that's in git. The decisions. Why I went with approach A over B. What tradeoffs I accepted. What I left open for next time.

So I built a Pi extension that fixes this. Here's how it works, and how I solved the only hard part.

What Pi is

Pi is an agentic coding assistant, think Claude Code or Codex but open and extensible. It runs in your terminal, has access to your filesystem and bash, and supports TypeScript extensions that hook into its full lifecycle.

The naive approach (and why it fails)

The obvious solution: at session end, grab the conversation and summarize it.

Problem: session context is massive. Tool outputs alone can be thousands of tokens. Feeding that to an LLM is expensive, slow, and produces bloated summaries.

The actual solution: bounded extraction

Instead of feeding the whole session, extract only three things:

// User prompts — what was asked (capped at 300 chars each)
// Tool names — what was touched (bash, edit, read, grep...)
// Last assistant message — what was concluded (capped at 300 chars)

That's it. Tool output blobs are skipped entirely. The extract stays small regardless of session length.

function extractSessionData(ctx: ExtensionContext): string {
  const entries = ctx.sessionManager.getBranch();
  const userPrompts: string[] = [];
  const toolsUsed = new Set<string>();
  let lastAssistantText = "";

  for (const entry of entries) {
    if (entry.type !== "message") continue;
    const msg = entry.message;

    if (msg.role === "user") {
      const text = Array.isArray(msg.content)
        ? msg.content
            .filter((b: any) => b.type === "text")
            .map((b: any) => b.text.trim())
            .join(" ")
        : msg.content?.trim() ?? "";
      if (text.length > 10) userPrompts.push(text.slice(0, 300));
    }

    if (msg.role === "assistant" && Array.isArray(msg.content)) {
      msg.content
        .filter((b: any) => b.type === "tool_use")
        .forEach((b: any) => toolsUsed.add(b.name));

      const text = msg.content
        .filter((b: any) => b.type === "text")
        .map((b: any) => b.text.trim())
        .join(" ");

      // Keep overwriting — we want only the last one
      if (text.length > 20) lastAssistantText = text.slice(0, 300);
    }
  }

  // ... build extract string
}

The LLM call

Separate Haiku-or-equivalent call. Never touches the main session context.

I'm using Mistral Small via OpenRouter — ~$0.00039 per call, 350 max output tokens.

const res = await fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.OPENROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "mistralai/mistral-small-3.2-24b",
    max_tokens: 350,
    messages: [
      { role: "system", content: TIGHT_STRUCTURED_PROMPT },
      { role: "user", content: extract },
    ],
  }),
});

The system prompt forces structure:

**Accomplished:** concrete bullets
**Touched:** files and tools
**Decisions:** architectural choices
**Open:** unresolved items

The output

After a Stride (macOS app) session:

## unnamed — 02:19 PM

**Accomplished:**
- Created 4 new homepage widget components
- Modified CurrentSessionView.swift to integrate bento-grid layout
- Restructured homepage from single-column to two-column bento-grid

**Touched:**
- CurrentSessionView.swift (restructured layout)
- TodaySummaryWidget.swift (new)
- HabitSummaryWidget.swift (new)
- bash, edit, grep (analysis tools)

**Decisions:**
- Adopted bento-grid layout for better information density
- Maintained ambient background branding for consistency

**Open:**
- MCP module compilation issues (pre-existing, unrelated)

Install

cp session-log.ts ~/.pi/agent/extensions/session-log.ts
# then /reload inside Pi

Commands:

/log — generate and save
/log show — open today's file in your default editor

Full source: shittycodingagent.ai

Weekly Reflection + Weekly Log polish in Stride

Orlando Ascanio — Thu, 05 Mar 2026 03:53:03 +0000

Shipped a polish/reliability pass tonight on two features that users touch often.

1 Weekly Reflection

Introduced a compact layout
Improved KPI hierarchy for faster scanning
Reworked no-data states for clarity
Reduced visual noise and improved readability

2) Weekly Log

Improved save-flow reliability with stricter DB write validation
Removed first-launch sample-data inserts (no demo data pollution)
Fixed day-boundary handling for date queries
Fixed deterministic ordering for category suggestions
Polished the entry modal and improved week-cell responsiveness

Not a flashy release, but the UX feels noticeably better, and the data path is much more trustworthy.

Weekly Reflection + Weekly Log polish in Stride

Orlando Ascanio — Thu, 05 Mar 2026 03:53:03 +0000

Shipped a polish/reliability pass tonight on two features that users touch often.

1 Weekly Reflection

Introduced a compact layout
Improved KPI hierarchy for faster scanning
Reworked no-data states for clarity
Reduced visual noise and improved readability

2) Weekly Log

Improved save-flow reliability with stricter DB write validation
Removed first-launch sample-data inserts (no demo data pollution)
Fixed day-boundary handling for date queries
Fixed deterministic ordering for category suggestions
Polished the entry modal and improved week-cell responsiveness

Not a flashy release, but the UX feels noticeably better, and the data path is much more trustworthy.

My 2026 MCP Stack

Orlando Ascanio — Fri, 27 Feb 2026 20:08:54 +0000

My 2026 MCP Stack

Here’s what I’m using in 2026 to build, ship, and iterate.

Core Engineering & Infrastructure

Google Cloud MCP – cloud services and infrastructure access
PostgreSQL MCP – direct database interaction
Stripe MCP – payments and billing workflows
Zapier MCP – automation between tools

Product & Monitoring

PostHog MCP – product analytics
Sentry MCP – error tracking
Chrome DevTools MCP – browser inspection and performance
Playwright MCP – automated testing

Knowledge, Research & Search

Notion MCP – structured knowledge access
Exa MCP – connects AI assistants to Exa’s web and code search (open-source)
Perplexity MCP – research augmentation
Context7 MCP – structured context retrieval

Workflow & Collaboration

Linear / Jira MCP – task and project management
Excalidraw MCP – diagrams and system sketches
Canva MCP – quick design workflows

What MCPs are you using right now? Anything I should test next?

This article gives a clear test to tell whether you’re shipping “distribution” (wrappers/clones) vs “capability” (new primitives). Includes actionable heuristics for AI builders (evals, reliability, orchestration).

Orlando Ascanio — Thu, 26 Feb 2026 13:59:05 +0000

Horizontal vs Vertical Progress: Why Scaling What Works Isn’t Enough

Orlando Ascanio ・ Feb 26

#ai #startup #product #software

Horizontal vs Vertical Progress: Why Scaling What Works Isn’t Enough

Orlando Ascanio — Thu, 26 Feb 2026 13:54:55 +0000

We often confuse scale with progress.

Scale is expanding what already works: more markets, more distribution, more copies.

Progress is increasing what's possible.

A useful lens is to separate progress into two modes:

Horizontal progress: copying a proven system and spreading it wider.
Vertical progress: creating something fundamentally new, new capabilities, new primitives, new leverage.

Horizontal progress distributes value. Vertical progress creates new categories of value.

The Ceiling of Replication

At a macro level, horizontal progress resembles what globalization does: taking systems that work in one place and deploying them elsewhere. That pattern has generated massive wealth.

But replication has a ceiling. Distribution scales benefits and costs. If the base system is inefficient, wasting energy, attention, capital, or operational complexity, scaling it multiplies the waste. You get more output, but also more strain.

Vertical progress is what raises that ceiling.

The Illusion of Automatic Progress

There's a quiet assumption many of us inherit: "the future gets better by default."

The 20th century makes that belief feel true. But its prosperity came from breakthroughs that changed society's input/output ratio:

electricity
computing
aviation
antibiotics and modern medicine
networks and the internet

These weren't better copies. They were new primitives. Replication helped distribute the gains, but the gains themselves originated with the invention.

What This Means for Builders (Especially in AI)

In software, horizontal progress often looks like:

another CRUD SaaS with slightly better UX
"AI wrappers" with thin differentiation
niche clones of existing startups
the same workflow repackaged for a new audience

These products can be useful and profitable. But they usually don't create new capabilities, only new distribution.

Vertical progress looks like work that changes what's feasible:

new architectures and abstractions that simplify whole classes of problems
infrastructure layers that make systems meaningfully more reliable, secure, or efficient
tools that make complex workflows predictable and debuggable
primitives that turn demos into dependable systems

In AI, vertical progress shows up less in prettier dashboards and more in hard engineering:

evaluation systems (measuring quality beyond vibes)
reliability engineering (handling failures, drift, uncertainty)
cost/performance orchestration (routing, caching, model choice)
memory and state that's inspectable and controllable
tooling that makes "intelligence" more programmable

A lot of today's AI products are horizontal progress at API speed. That's not an insult, it's a phase. The enduring leverage comes from the vertical layer: the primitives that make AI systems reliable at scale.

Constraints Are the Point

Replication assumes we can scale indefinitely. In practice, we hit constraints:

energy and compute
attention and cognitive bandwidth
capital efficiency
infrastructure and operational complexity

If you scale an inefficient system, you scale the inefficiency.

Technology, when it's truly vertical, does the opposite: it increases output per unit of input. It bends constraints instead of multiplying them.

A Practical Test: Horizontal or Vertical?

Ask:

If competitors copy your UI in a weekend, do you still have an advantage?
Are you building a new capability, or just new packaging?
Does your product get structurally better via feedback loops/evals, or is it static?
Are you reducing a fundamental constraint (time, cost, error rate), or just moving it around?
Can you explain differentiation without mentioning your niche?

If the answers point mainly to "distribution" and "packaging," you're playing a horizontal game. That's fine, just be honest about it.

The Builder's Responsibility

Horizontal progress distributes gains. Vertical progress creates them.

Both matter. But only one raises the ceiling.

What are you building that moves things forward, not just outward?

I built Elevare because I kept drowning in tasks, even though I knew most of them didn’t matter. So, I asked: what if an AI could coach me to focus on just one thing, the ONE Thing that matters most?

Orlando Ascanio — Mon, 22 Sep 2025 03:50:42 +0000

Orlando Ascanio

Sep 22 '25

Elevare: Building an AI Assistant That Helps You Focus on Your ONE Thing

#ai #productivity #typescript #learning

Comments

1 min read

Elevare: Building an AI Assistant That Helps You Focus on Your ONE Thing

Orlando Ascanio — Mon, 22 Sep 2025 03:49:53 +0000

We live in an age of endless to-dos, but most of them don’t matter. Productivity isn’t about doing more, it’s about doing what matters most.

That’s the philosophy behind Elevare, a project I’ve been building.

✨ What is Elevare AI?

Elevare is an AI-powered productivity coach (Gemini + LangChain) that helps you:

🎯 Task Suggestion Mode: When you have tasks, it picks the ONE with the biggest leverage.
🧭 Exploration Mode: When you feel lost, it asks clarifying questions and brainstorms possible ONE Things.
🧠 Psychological Layer: Uses reflection prompts, progressive narrowing, and identity-based framing to make the next action stick.

🔧 Under the Hood

TypeScript + Express backend.
Gemini API (via LangChain) for AI reasoning.
SQLite for persistence (feedback storage, session context).
Clean architecture: prompt service, heuristic service, feedback service.

🧩 Why This Matters

Most apps drown you in todos. Elevare helps you cut through noise, reduce decision fatigue, and build momentum, one domino at a time.

"Success is not about doing everything right, but about doing the right thing."

Demo: https://elevare-ai-assistant-demo.vercel.app

I’d love feedback, ideas, and collaborators. If you’ve ever felt overwhelmed by tasks and wished for a coach, not just a checklist, Elevare is for you.

I just had a technical interview, and it went terribly. I was guessing at answers, even to "tell me about yourself." Painful? Yes. But also one of the best wake-up calls I’ve had as a developer.

Orlando Ascanio — Thu, 18 Sep 2025 19:54:01 +0000

Orlando Ascanio

Sep 18 '25

First Technical Interview

#interview #learning #programming #career

Comments

1 min read

First Technical Interview

Orlando Ascanio — Thu, 18 Sep 2025 19:52:31 +0000

Yesterday I finished my first technical interview, and honestly, it didn't go as I expected.

They asked me deep questions about React, design principles, UX, and even threw in a LeetCode problem. I found myself guessing on answers instead of confidently explaining what I know, even when they asked the classic “Tell me about yourself.” That moment of uncertainty was humbling, but also eye-opening.

Here’s what I’m taking away from this experience:

1️⃣ Preparation is more than knowledge – knowing something is different from being able to communicate it clearly under pressure.

2️⃣ Anchored examples matter – I need concrete stories from my projects to illustrate principles like React patterns, UX decisions, and problem-solving.

3️⃣ Confidence beats guessing – showing structured thinking is often more important than getting a perfect answer.

Mistakes are uncomfortable, but they are also the best teachers. I’m grateful for this experience because it highlighted exactly what I need to improve and that clarity is invaluable.

To anyone else who’s had a tough interview recently: take a deep breath, reflect, and make a plan. Growth comes from facing challenges head-on. 💪

How do you actually use an MCP client with GitHub? I wrote a step-by-step guide for devs, clear, simple!

Orlando Ascanio — Fri, 05 Sep 2025 05:15:55 +0000

Orlando Ascanio

Sep 5 '25

MCP: The USB-C for AI Development (Why You Should Care)

#todayilearned #mcp #ai #learning

Comments

3 min read