DEV Community: Syed Ibrahim

Why Not Every AI Application Needs Vector Embeddings

Syed Ibrahim — Wed, 15 Jul 2026 13:43:40 +0000

How building an AI chapter generator taught me to stop reaching for a vector database first.

When I started building my AI chapter generator, I spent hours researching vector databases. I was convinced I needed one.

I didn't.

The idea

The project was simple on paper. A user uploads a long lecture video, the app transcribes it, figures out where the instructor moves from one topic to the next, and spits out timestamped chapters.

Before I'd written a single line of the actual feature, my brain had already decided what the architecture should look like. Long text plus AI app equals: chunk it, embed it, store it, retrieve it. That's just the shape everyone's trained to see these days.

So that's the road I went down. I spent an evening comparing Pinecone, Chroma and FAISS. I read through chunking strategies, fixed size chunks, overlapping windows, semantic chunking, all of it. I had a RAG pipeline half sketched out in my notes app.

Then I stopped and asked myself something a lot more basic:

What problem am I actually trying to solve?

I didn't have a good answer. So I backed up and started over.

What embeddings actually do

If you're newer to this stuff, here's the plain version.

An embedding turns a piece of text into a list of numbers that captures what it means. Text with similar meaning ends up with similar numbers. That's really the whole trick behind it. It turns "how similar are these two ideas" into "how close are these two points," which is something a computer can check almost instantly, even across millions of documents.

That's a genuinely useful trick, but notice what it's actually for. It's for finding things. If you've got 10,000 support articles and someone asks a question, you can't just feed all 10,000 into the model every time. It wouldn't fit, and even if it somehow did, it'd be painfully slow. Embeddings let you narrow that pile down to the handful of articles that are actually relevant, and only those get handed to the model.

That narrowing down is retrieval. And retrieval turns out to be a different job than reasoning.

Retrieval vs reasoning

This is the part that took me a minute to actually get, and I think it's the bit most people gloss over.

Embeddings solve retrieval. They're good at finding a needle in a haystack you can't fully read.

LLMs handle reasoning. They think through whatever's in front of them.

RAG is called Retrieval Augmented Generation because retrieval is just a preprocessing step that decides what the model gets to look at. It doesn't make the model reason any better once it's looking at the right thing. It just makes sure it's looking at the right thing in the first place.

Which leads to an obvious question. What if the model already has the right thing?

That was basically my situation. The whole transcript was already right there. Nothing was buried somewhere I needed to dig it out of. There was nothing to find. Just something to work through, in order.

But don't long transcripts still need chunking?

Yes, and this tripped me up for a bit, so it's worth spelling out.

A one hour lecture transcript is way too long to comfortably send in one model call. So you still have to split it up somehow. That part's unavoidable.

But there are two different reasons you'd chunk something, and I'd been mushing them together in my head without realizing it.

Chunking for retrieval splits text so you can search it. Each chunk gets embedded and scored against a query, and only the best matches get used.

Chunking for context just splits text because of size limits. Every chunk still gets used, in order, nothing gets filtered out.

I needed the second kind. Not "which piece is most relevant," but "here's the whole thing in pieces, go through all of it, in order." That doesn't need embeddings. It just needs a loop.

The moment it clicked

Once I'd separated retrieval from reasoning in my head, and chunking to search from chunking to fit, things got a lot clearer fast.

I asked myself how a person would do this task. Not an AI, an actual human sitting there watching the lecture.

They wouldn't be scanning through looking for the paragraph most "similar" to some query, because there's no query. Nobody's asking a question. They'd just watch it from start to finish and notice, naturally, when the instructor wrapped up one idea and moved on to the next.

That's when it actually clicked. Chapter generation is about sequence, not similarity.

Embeddings answer "where in this pile does something like X show up." My problem was "where does this one, ordered thing change." Those aren't the same problem wearing different clothes. They're genuinely different problems, and no clever chunking-for-retrieval trick was going to fix that, because retrieval was never actually the job.

What I thought I needed

Transcript
   │
   ▼
Chunk for retrieval
   │
   ▼
Embed each chunk ──► Vector database
   │                         │
   │      (similarity search)│
   ▼                         ▼
        Query-relevant chunks
                │
                ▼
              LLM
                │
                ▼
            Chapters

What I actually built

Transcript
   │
   ▼
Chunk sequentially (for size, not search)
   │
   ▼
LLM reads each chunk in order,
carrying forward a rolling summary
   │
   ▼
Flag topic transitions
   │
   ▼
Merge + align to natural pauses
   │
   ▼
Chapters

Same input, same output, half the pipeline.

What I actually built

Here's the workflow, roughly:

Transcribe the video.
Split the transcript into sequential chunks of a few minutes each, keeping order and timestamps intact.
Feed each chunk to the model along with a short rolling summary of everything before it, and ask it to flag where the topic shifts.
Merge the transitions that land close together, and snap each boundary to the nearest natural pause so a chapter doesn't start mid sentence.

Step 3 is really the whole trick. Each chunk carries context forward from the ones before it. The model isn't staring at six disconnected fragments trying to guess how they relate, it's walking through the lecture the way a person would, carrying the thread as it goes. That continuity is what actually lets it notice "okay, this is where the topic changed." It's also exactly what a similarity search would throw away, since similarity search treats every chunk as its own isolated thing to be scored.

No vector database, no embedding calls, no retrieval step.

The real reason I dropped embeddings

I could frame this as a story about cost or speed. Fewer moving parts, lower latency, no vector DB bill. Those were real benefits and I did get them. But that's not actually why I made the change, and leading with that would kind of miss the point.

The real reason is simpler. Embeddings would have solved a problem I didn't have. My bottleneck was never "how do I find the relevant part of this transcript," because the whole transcript was the relevant part. Bolting retrieval on top wouldn't have made the output better, it would've just added machinery answering a question nobody was asking, while the actual question, where does the topic change, stayed exactly as unsolved as before.

Cost and latency got better as a side effect of cutting something unnecessary. They weren't the reason it was unnecessary.

Embeddings are still great, when you actually need retrieval

None of this is an argument against embeddings. I still use them all the time, just for a different kind of problem.

Say you're building a support bot on top of 10,000 help articles. Someone asks a question and there's no way to hand the model all 10,000 articles at once, it needs to find the handful that are actually relevant first. That's exactly the retrieval problem embeddings were built for, and it's a great use of a vector database there.

The difference between that and my chapter generator isn't the amount of text involved, both had plenty. The difference is whether the model needs to search across a bunch of separate things, or read through one thing it already has in full.

Needs retrieval: company docs, big knowledge bases, research paper libraries, product manuals, support article collections, long chat history archives.

Doesn't need retrieval: summarizing a single document, translating text, classifying content, pulling structured data out of one file, reviewing a single codebase, generating chapters from one transcript.

If the model already has everything it needs sitting in front of it, embeddings don't help it reason better. There's nothing to find, so there's nothing for retrieval to speed up.

A question worth stealing

Before I add a vector database to anything now, I ask myself one thing, and it's worth stealing if you don't already ask it:

Does my AI need to find information, or does it already have it?

If it already has everything, embeddings won't make the answer any sharper. They'll just add cost, latency, and one more system that can quietly break on you.

Final thoughts

I almost built a whole retrieval pipeline for a problem that had nothing to retrieve, just because that's the default move everyone reaches for. The real lesson here wasn't about embeddings specifically. It was about catching myself solving the problem the tutorials describe instead of the one actually sitting in front of me.

The best architecture isn't the one with the most pieces. It's the one that solves the actual problem with the least stuff bolted on.

See it in action

The ideas in this article weren't just theoretical—they came from building a real application.

If you'd like to try it yourself, here's the live demo:
https://trywaypoint.streamlit.app/

OpenSpec: The Missing Layer Between You and Your AI Coding Assistant

Syed Ibrahim — Mon, 15 Jun 2026 13:14:35 +0000

If you have spent any time doing AI-assisted development or vibe coding, you already know the problem.

You open a new chat.
Start building something.
The AI just runs with it.

No alignment, no scope, no shared understanding of what you are actually trying to do.

That works fine for small throwaway tasks. But when you are working on a real codebase, things go sideways fast.

The AI makes assumptions. Scope creeps. You end up with code that technically works but does not match what you had in mind.

OpenSpec fixes this by introducing one simple idea

agree on what you are building before any code gets written.

It gives you and your AI coding assistant a lightweight structure to follow for every change you make. Before implementation starts, you define the intent, the scope, and the approach. The AI reads that and works within those boundaries instead of guessing.

The result is more predictable output, less back and forth, and something that matters a lot in fast-paced development ~ the reasoning behind your decisions is actually preserved instead of disappearing into chat history.

What is OpenSpec?

OpenSpec is a spec-driven development framework. It sits between you and your AI coding assistant as a lightweight planning layer.

The idea is simple. Before your AI writes a single line of code, you and the AI agree on a spec. What you are building, why you are building it, how you are going to build it, and what is out of scope. Everything is written down and locked in before implementation starts.

It works with the tools you already use ~ Claude Code, Cursor, Codex, Copilot, Antigravity and more. No API keys, no complex setup. Your specs live right inside your repository as markdown files.

Every change you make in OpenSpec goes through four artifacts:

Proposal ~ the why. What you want to build and why it matters
Specs ~ the what. The requirements and expected behavior
Design ~ the how. The technical approach and decisions
Tasks ~ the steps. Broken down implementation plan

Installation and Setup

Prerequisites

Node.js 20.19.0 or higher

Install

npm install -g @fission-ai/openspec@latest

Initialize

cd <your-project>
openspec init

Once initialized, it will ask you to select which AI tools you want to set up. OpenSpec supports 28 tools including Claude Code, Cursor, Codex, Copilot and more. Use Space to toggle and Enter to confirm.

After selecting your tools, OpenSpec sets everything up automatically ~ agent skills, slash commands, and config files for each tool you selected.

Once setup is complete, restart your IDE for the slash commands to take effect. Then you are ready to start your first change with /opsx:propose <what-you-want-to-build>

The Four Artifacts

Every change in OpenSpec progresses through four artifacts.

Each artifact builds on the previous one, ensuring implementation starts only after the change is fully understood.

1. Proposal

The Why

The proposal captures the intent behind the change.

Questions answered here:

Why are we doing this?
What problem are we solving?
What value does this provide?

This is where intent gets locked in.

2. Specs

The What

Specs define the requirements and expected behavior.

Questions answered here:

What should happen?
What should the system do?
How should it behave?

These become the source of truth during implementation.

3. Design

The How

Design documents describe the technical approach.

Questions answered here:

Which architecture should we use?
What tradeoffs are involved?
What technical decisions need to be made?

This phase often prevents expensive mistakes before they reach production.

4. Tasks

The Steps

Tasks break the implementation into manageable pieces.

Questions answered here:

What needs to be done?
In what order?
How do we know we're finished?

This becomes the execution plan for both humans and AI agents.

Your First Change

Let's walk through a real example. We are going to add a new User Profile API endpoint using OpenSpec from start to finish.

Step 1: Start the Change

Run the propose command in your AI coding assistant:

/opsx:propose "add-user-profile-api-endpoint"

Your AI will generate all four artifact files automatically:

Created openspec/changes/add-user-profile-api-endpoint/
✓ proposal.md — why we're doing this, what's changing
✓ specs/       — requirements and scenarios
✓ design.md    — technical approach
✓ tasks.md     — implementation checklist
Ready for implementation!

Here is what each artifact looks like:

proposal.md

# Proposal: Add User Profile API Endpoint

## Intent
Users need a way to retrieve and update their profile information
through a dedicated API endpoint.

## Scope
- GET /api/users/:id endpoint
- PUT /api/users/:id endpoint
- Input validation and error handling

## Approach
Use existing REST conventions with JWT authentication middleware.

specs/api/spec.md

# Delta for API

## ADDED Requirements

### Requirement: User Profile Retrieval
The system MUST expose a GET endpoint to retrieve user profile data.

#### Scenario: Fetch profile
- GIVEN an authenticated user
- WHEN GET /api/users/:id is called
- THEN the user profile is returned

design.md

# Design: Add User Profile API Endpoint

## Approach
- Use Express router for route handling
- JWT middleware for authentication
- Joi for input validation
- Return 404 if user not found

tasks.md

# Tasks

## 1. Route Setup
- [ ] 1.1 Create user profile route file
- [ ] 1.2 Register route in main router

## 2. Handlers
- [ ] 2.1 Implement GET /api/users/:id handler
- [ ] 2.2 Implement PUT /api/users/:id handler

## 3. Validation and Error Handling
- [ ] 3.1 Add input validation
- [ ] 3.2 Add error handling middleware

Step 2: Implement

Once you are happy with all the artifacts, run:

/opsx:apply

Your AI now works through the tasks one by one following the spec exactly:

Working through tasks...
✓ 1.1 Created user profile route file
✓ 1.2 Registered route in main router
✓ 2.1 Implemented GET /api/users/:id handler
✓ 2.2 Implemented PUT /api/users/:id handler
✓ 3.1 Added input validation
✓ 3.2 Added error handling middleware
All tasks complete!

No guessing, no assumptions, no scope creep. Your AI follows the spec and nothing else.

Step 3: Archive

Once everything is verified, run:

/opsx:archive

This closes the change completely. It merges your delta specs into the main specs folder and moves the change to the archive permanently.

Archiving add-user-profile-api-endpoint...
✓ Merged specs into openspec/specs/api/spec.md
✓ Moved to openspec/changes/archive/2025-01-24-add-user-profile-api-endpoint/
Done! Ready for the next feature.

The reasoning behind every decision is now preserved in your repository forever. Next time anyone touches this code ~ including an AI agent ~ the why is already there.

Tips from Real Usage

After using OpenSpec for a few months in fast paced AI driven development, here are a few things I have learned.

Always review the proposal before moving on
The proposal is where intent gets locked in. If something is off there, it carries through to every other artifact. Spend an extra minute here and save yourself a lot of back and forth later.

Be specific with your scope
The more specific you are about what is out of scope, the better your AI performs. Writing "out of scope: custom error pages" is more useful than leaving it vague. AI agents take boundaries seriously when they are written down.

Do not skip the design artifact
It is tempting to jump straight to implementation. But the design phase is where you catch bad technical decisions before they become bad code. Many times the design phase alone has saved me from going down the wrong path.

Use it on existing codebases
OpenSpec is brownfield first. You do not need a fresh project to get value from it. Drop it into an existing codebase and start using it for your next change. The value shows up immediately.

Your specs are living documentation
Every time you archive a change, your specs folder gets more complete. Over time it becomes a genuine source of truth for how your system works and why decisions were made. That is valuable for your whole team, not just for AI agents.

Conclusion

AI coding assistants are powerful. But power without direction is just noise.

OpenSpec is not about slowing down. It is about making sure that when you move fast, you are moving in the right direction. In vibe coding and fast paced AI driven development, the biggest risk is not writing bad code. It is building the wrong thing confidently.

After using OpenSpec for a few months, the biggest shift I noticed was not in the code quality. It was in how much context was being preserved. The why behind decisions no longer lives in chat history or someone's head. It lives in the repository, right next to the code it describes.

If you are doing any kind of AI assisted development and you are not using a spec driven approach, give OpenSpec a try. It takes five minutes to set up and the value shows up on your very first change.

Start with:

/opsx:propose "your-first-change"

And see what happens.

Google Just Made AI Memory Official at Cloud NEXT '26 - And It Made Me Uncomfortable :(

Syed Ibrahim — Sun, 26 Apr 2026 11:39:11 +0000

This is a submission for the Google Cloud NEXT Writing Challenge

I'll be honest. I didn't sit through the full Google Cloud NEXT '26 keynotes. I caught the highlights in between work. That's my level of "following the news."

But one announcement stopped me: Agent Memory Bank, now generally available inside the Gemini Enterprise Agent Platform.

Here's what it does in plain English: your AI agent can now remember things across conversations. Not just inside one session, but across sessions. It dynamically builds memory from past interactions, stores it, and pulls it back when relevant. Using Memory Profiles, agents can recall high-accuracy details with low latency so they don't lose context. (Google Cloud)

Sounds great on paper. But it made me think about something that happened to me recently.

I Already Experienced AI Memory. Without Asking for It.

A few weeks ago I was trying to fix a Decoder init failure on specific Snapdragon/MediaTek chipset phones. It was showing up in Sentry on our Android player SDK, ExoPlayer related. So I was using an AI tool to work through it, asking questions, going back and forth.

At some point I closed that chat and started a completely new one. Different day, different question. But somewhere in that new conversation the AI referenced something specific from the ExoPlayer session. Something I hadn't brought up at all.

So I asked: "are you reading my previous chats?"

It said no.

I gave it the exact context it had referenced. Still denied it.

That's the part that didn't sit right with me. Not that it remembered. That it remembered and then said it didn't. If a system can recall something it claims it cannot access, and then confidently insists otherwise, that's not just a bug. That's a trust problem. Because now I'm sitting there wondering what else it knows that it's not telling me. And the easy fallback is always "AI can make mistakes" which okay, sure, but that line is doing a lot of heavy lifting. It quietly removes accountability. The system moves on. I'm left holding the confusion.

Now Google Is Making Memory an Official Feature

Agent Engine Sessions and Memory Bank, which give agents persistent context across interactions, are now generally available. (The Next Web)

In some ways that's actually more honest. At least now it's documented, intentional, and in an API you can actually inspect.

The idea makes sense for what I build. I work on Django-based AI SaaS. Right now every session resets. Users repeat themselves. Agents forget context. Google showed a demo where a customer moves from text chat to a phone call and the agent seamlessly remembers exactly where they left off. (Google Cloud) That kind of continuity is genuinely useful.

Think of it like a user_preferences table in Django, except Google manages the entire memory layer for you.

But My Trust Questions Remain

If unofficial memory already existed and got denied, what happens with the official version?

What exactly gets stored, and for how long?
Can my users actually delete their memory and trust it's gone?
How do I explain this to users in India who are already cautious about data privacy?
And most practically, where is the clean pip install Django walkthrough?

The codelab demo showed Memory Bank costing less than $5 (Google Codelabs), which is a good sign. But I need to see full production pricing before I build anything serious on top of it.

My Prediction

Persistent agent memory is the right direction. Google's pitch is a unified stack: chips designed for models, models grounded in your data, agents built with those models, all secured by the infrastructure. (Google Cloud) If that holds up, it could genuinely be the default for AI SaaS in 12 months.

But the trust layer has to come with the feature. Users need to know what's remembered, what's stored, and what "delete" actually means.

Google has a real chance to set that standard properly with Agent Memory Bank, especially because the unofficial version already exists and nobody is really talking about it clearly.

Make it transparent, Google. The feature is good. The trust needs to catch up.