DEV Community: Ndimofor Aretas

Building an Air-Gapped NotebookLM Alternative via Gemma 4 E4B, FAISS, and Strands Agents

Ndimofor Aretas — Sun, 24 May 2026 13:32:53 +0000

_This is a submission for the Gemma 4 Challenge: Build with Gemma 4_

What I Built

The Privacy Imperative: Why I Built CogniVault

As an IT trainer based in Germany helping career-changers pivot into the technology sector, a significant portion of my daily work is subject to strict non-disclosure agreements (NDAs), regional privacy mandates such as the EU General Data Protection Regulation (GDPR/DSGVO), and evolving legal frameworks like the EU AI Act. Whether handling proprietary corporate training material, confidential client curricula, or sensitive student records, data security is an absolute prerequisite.

When analyzing mainstream cloud-based AI tools, corporate compliance audits present an unyielding barrier: cloud dependencies represent an unacceptable data leak surface. For a legal compliance officer, "zero data retention" policies are insufficient; the core problem is that data leaves the hardware boundary at all. Furthermore, mandatory cloud registrations, per-token API overhead costs, and restrictive usage quotas severely limit the continuous, exploratory workflow required for deep academic and professional study.

Gemma CogniVault is my solution to this challenge. It is a 100% local, privacy-first AI study companion engineered to turn confidential documents into structured learning resources entirely on-device. By running localized inference and embedding models on existing workstation hardware, CogniVault delivers intelligent chat interfaces, multi-lesson workshops, visual mindmaps, customized quizzes, and interactive flashcard decks without a single byte of data ever escaping the machine.

CogniVault is built explicitly for professionals and organizations operating in highly regulated spaces:

Educators and Corporate Trainers who require an intelligent didactic assistant that honors ironclad institutional confidentiality.
Academic Researchers processing sensitive grant proposals, proprietary datasets, or pre-publication manuscripts.
Healthcare, Finance, and Legal Practitioners bound by regulatory frameworks (such as HIPAA, FINRA, or GDPR) who need advanced text and document analytics on isolated hardware.

System Architecture & Capabilities

CogniVault bridges the gap between secure, local data storage and advanced generative AI capabilities through the following core architectures:

Local Hybrid Retrieval-Augmented Generation (RAG): The platform ingests eight foundational document types: PDF (with native OCR fallback via Tesseract), DOCX, PPTX, XLSX, Markdown, CSV, TXT, and HTML. Files are parsed using structure-aware chunkers (e.g., Markdown retains header breadcrumbs; CSV chunks repeat the initial header row), translated into dense 768-dimensional vectors via embeddinggemma, and indexed using an in-memory FAISS IndexFlatIP coupled with a sparse BM25 retrieval layer. Sub-millisecond top-7 document matching is achieved through Reciprocal Rank Fusion (RRF).
Two-Phase Streaming with Auditable Chain-of-Thought: To build trust in high-stakes compliance environments, chat responses execute in two discrete phases. Phase 1 queries Gemma 4 with thinking: True enabled, streaming the model's raw reasoning steps directly into a collapsible frontend panel. Phase 2 utilizes a localized Strands Agent loop to evaluate relevant context chunks, generating a factual final answer pinned with interactive inline source citations.
Autonomous Strands Agent Loop: Gemma 4 E4B is integrated into the Strands Agents SDK and acts as a central decision engine equipped with 6 local tools: search_knowledge_base, list_documents, analyze_document, compare_documents, calculator, and current_time. The model evaluates incoming requests, determines optimal execution order, and solves complex analytical queries entirely on-device without hardcoded routing rules.
Grammar-Constrained Study Generation: The Study Hub generates interactive materials (quizzes, multi-lesson workshops, 3D flashcards, and radial mindmaps) using Ollama's grammar-constrained format="json" parameter. Token generation is strictly restricted at the runtime level to match target schema configurations, ensuring deterministic, valid JSON outputs that bypass typical parsing errors.
Crash-Safe Ingestion & Content Hashing: Ingestion operations run as durable, checkpointed workflows powered by a DBOS backend. If a device power cycle occurs mid-ingestion, processing resumes from the last successfully indexed batch. Additionally, SHA-256 content hashing automatically tracks file variations on re-upload, soft-deleting obsolete chunks while seamlessly re-indexing modified text.
Localized Analytics & Progress Tracking: Learning sessions and metrics are monitored completely offline using a standalone SQLite layer (progress.db). The dashboard tracks total study durations, active streaks, and automatically grants 25 distinct achievement badges across study milestones, presenting data visually via a GitHub-style 90-day activity heatmap.
Secure Document Exports: Generated quizzes, mindmaps, and workshop materials can be natively exported as Markdown, high-resolution PNGs, or printable PDFs. The system leverages the browser's native File System Access API to trigger authentic "Save As" OS dialog boxes entirely inside the client environment.

Demo

Watch the full architectural walkthrough and deep-dive on YouTube: [https://youtu.be/rw5Rse_TP2o?si=4rIh8staIPwTiuyc]

Code

Full source repository: [https://github.com/ndimoforaretas/local-gemma-rag]

Key modules and implementations to explore:

backend/services/rag_agent.py: Implements the two-phase asynchronous stream (raw reasoning extraction followed by Strands agent orchestration) alongside multi-session history encapsulation.
backend/services/vector_db.py: Coordinates the dense FAISS and sparse BM25 retrieval algorithms, executing Reciprocal Rank Fusion (RRF) and post-fusion scope metadata filtering.
backend/services/ingest.py: Houses the DBOS-checkpointed extraction routines for all 8 supported document formats and coordinates SHA-256 identity checks.
backend/services/quiz_generator.py (alongside workshop_, flashcard_, and mindmap_generator.py): Manages the structural formatting parameters, zero-comma JSON repair processes, and model retry fallbacks.
frontend/src/components/study/: Contains the React orchestrators, SVG canvas rendering engines, and custom animation definitions for the client-side user experience.

How I Used Gemma 4

Primary Inference Model: gemma4:e4b (4.5B effective parameters / 8B with embedded parameters, 128K context window, served locally via Ollama).

Vector Embedding Model: embeddinggemma (768-dimensional dense vector embeddings).

Model Selection Rationale

Choosing the right runtime model required balancing raw analytical reasoning with the hardware profiles of standard developer and trainee laptops. Testing determined that the E4B variant offered the ideal intersection of performance and local feasibility.

Architectural Feature / Benchmark Metric	Gemma 4 E2B	Gemma 4 E4B (Chosen)	Gemma 4 31B Dense
Effective Parameter Weight	2.3B (5.1B embedded)	4.5B (8B embedded)	31B Dense
Native Context Length	128K	128K	256K
Quantized VRAM/RAM Target (GGUF)	~2.5 GB	~5.1 GB	~20+ GB
Standard 16 GB Workstation Footprint	Low resource utilization	Optimal balance; ample memory headroom	Exceeds system limits; heavy thrashing
MMLU Pro (Reasoning Capability)	60.0%	69.4%	85.2%
GPQA Diamond (Advanced Science)	43.4%	58.6%	84.3%
MMMU Pro (Visual Analytics)	44.2%	52.6%	76.9%
MMMLU (Multilingual Evaluation)	67.4%	76.6%	88.4%
OmniDocBench 1.5 (Lower = Superior)	0.290	0.181	0.131

The choice of Gemma 4 E4B was driven by three architectural necessities:

Host Memory Headroom: At a quantized GGUF profile requiring roughly 5 GB of system memory, E4B loads comfortably alongside standard system processes, the FastAPI runtime, localized database containers, and browser development instances on a standard 16 GB laptop.
Context Density and Retention: The 128K token boundary easily accommodates extensive system prompts, specialized JSON schemas, recursive tool definitions, multi-session conversation text, and 15 highly detailed RAG text chunks without requiring aggressive context truncation techniques.
Multi-Lingual and Visual Consistency: Scoring 76.6% on the MMMLU benchmark means that E4B parses technical materials in German or English with identical analytical fidelity. Its strong visual reasoning score (0.181 on OmniDocBench) provides stable text extraction from complex whiteboard diagrams and smartphone photos uploaded directly into the chat interface.

End-to-End Implementation Flow

[Confidential Document] ──► Ingestion Pipeline (DBOS & SHA-256) ──► Chunking Strategy
                                                                         │
    ┌────────────────────────────────────────────────────────────────────┘
    ▼
Ollama Embeddings (embeddinggemma) ──► Local Hybrid Database (FAISS + BM25)
                                                                         │
    ┌────────────────────────────────────────────────────────────────────┘
    ▼
User Query ──► Phase 1: Local Asynchronous Streaming (gemma4:e4b - thinking=true)
    │
    └────────► Phase 2: Strands Agent Loop Evaluation ──► Deterministic GenUI Layout

Embedding Matrix Generation: Context blocks are vectorized via ollama.embed using embeddinggemma in automated batches of 5, speeding up local processing pipelines by avoiding single-item API round trips.
Phase 1 Reasoning Capture: Prompts clear a specialized context routing validation layer before launching an asynchronous stream using ollama.AsyncClient.chat with options={"thinking": True}. Raw internal analysis tokens are written directly to a customized UI panel.
Phase 2 Strands Execution: When execution transitions to the tool matrix, a dedicated strands.Agent initializes with an explicit, system-level operational block, suppressing further <think> output tags to maintain clean Markdown delivery in the primary workspace UI.
Grammar-Restricted Output Generation: Complex layouts utilize localized structural boundaries through Ollama's native grammar engine. System configurations employ lower temperature parameters (0.3 to 0.4) during data layout requests to guarantee strict adherence to data schemas, while using a standard 0.5 setting for rich long-form textual formatting.

Engineering Insights & Takeaways

Architecturally, Gemma 4’s format="json" capability entirely changes how local multi-tasking modules behave. Older edge-compute models frequently produced erratic token fragments or broke syntax constraints under long generation cycles, forcing developers to maintain fragile regex filters or expansive string recovery logic. By applying grammar-constrained validation natively at the token-generation level, CogniVault transforms local structured output from a brittle experimental feature into a guarantee.

Honest Design Compromises

Absolute Edge-Execution Boundary: CogniVault purposefully eliminates cloud network fallbacks. The absolute requirement of a ~10 GB local model initialization is the unavoidable trade-off for a system designed to guarantee complete data privacy.
Interim Audio Ingestion Strategy: While Gemma 4 E4B possesses an integrated audio encoder, local voice translation is handled by a separate Whisper container in the current version. Migrating this directly into the native Gemma pipeline remains a priority on the development roadmap.

What This Unlocks For The User

Until now, professionals working in highly regulated fields were effectively locked out of the generative AI landscape due to strict compliance standards and data security risks. The massive gap between cloud-based model performance and what could be safely deployed in an air-gapped corporate environment created a major barrier to adoption.

Gemma 4 successfully eliminates that barrier.

Corporate Training Instructors can systematically convert highly proprietary curricula into interactive tests without triggering network security audits.
Students and Trainees can safely query proprietary or confidential material without violating institutional privacy policies.
Academic and Scientific Researchers can analyze draft manuscripts or sensitive research data completely free from data-leak concerns.

By running advanced reasoning, multi-tool agent logic, native vision ingestion, and grammar-precise structural parsing on local hardware, CogniVault provides a secure, high-performance solution for the privacy-conscious professional.

_CogniVault is engineered to operate completely independent of external infrastructure, leveraging FastAPI, FAISS, DBOS, SQLite, and the Strands Agents SDK to deliver secure, local AI applications.

Connect with me on LinkedInto track ongoing development updates._

[Boost]

Ndimofor Aretas — Sun, 24 May 2026 05:13:01 +0000

Revive old projects using GitHub Copilot

Jess Lee for The DEV Team

May 21

Join the GitHub Finish-Up-A-Thon Challenge: $3,000 Prize Pool!

#githubchallenge #devchallenge #githubcopilot

189

3 min read

[Boost]

Ndimofor Aretas — Tue, 19 May 2026 16:34:40 +0000

Announcing the Gemma 4 Challenge

Jess Lee for The DEV Team

May 6

Join the Gemma 4 Challenge: $3,000 prize pool for TEN winners!

#devchallenge #gemmachallenge #gemma

485

5 min read

How to START and STAY MOTIVATED on your Coding journey.

Ndimofor Aretas — Thu, 07 Jul 2022 06:02:55 +0000

I recently noticed a rising demotivation trend among, especially code newbies on social media (Tech Twitter — under #100DaysOfCode) to be precise, who are at the quitting stage and feel that coding is not really “their thing”.

It often starts with the questions like;

— “How do you find time to code every day?”

Or maybe even, as the title suggests;

— “How do you stay motivated as you learned to code?”

Maybe you too or someone you know is having that thought or has told you or maybe asked similar questions.

Well, dear reader, worry no more!
I am a seasoned learner (Autodidact), Teacher, trainer, and also a coding enthusiast and I wish to share with you some of my top secrets on How you can (and should) START and STAY on your coding Journey; pushing through each day with consistent energy, enthusiasm, and gusto, as I am.

Rest assured that coding is A SKILL that can be acquired (learned) and or sharpened (honed) by ANYBODY with a good approach.

STARTING

The first 5 steps are all about choosing the right destination and preparing your backpack for it. I cannot emphasise enough how important it is to ensure that you know your destination and pack in the most important items you need.

“Before you begin scrambling up the ladder of success, make sure that it is leaning against the right building.”
— Stephen Covey

1) Define a PRECISE REASON and WRITE IT DOWN!

Start with a GOOD REASON! (The WHY):

WHY do you wish to write code?
Why is it vital that you learn that skill?
Be as personal and selfish as possible! Is it for the money? The ability to work remotely? Or maybe The need to belong to a unique group of extraordinary people that can “manipulate” that thing called the computer?

The more personal the reason is, the higher your chances are of staying on the journey. Whenever you feel discouraged, you can always remind yourself why you started in the first place.

“Definiteness of purpose is the starting point of all achievement”.
— W. Clement Stone

2) Define Your NOUNS and VERBS

How do you wish to be CALLED when you are done? Just google out “code-related job titles”.

What do you want to be able TO DO with your coding skills? Do you wish to design, develop, program, manage databases, etc? Again, google some more!

The more precise and clear you are about what you want, the easier it will be for you to complete the next step,
which is…

3) Find a MENTOR

I define a Mentor as someone who:

is already where you aspire to be,
has the job title (Noun) you wish to have
and is doing what you would like to do (the verb)

The best way to know the road ahead is to ask someone who is returning from the where you are going.
— Anonymous

When you pick out a mentor, ask them if they can mentor you. Your mentor is going to play a key role in guiding you along your journey. Share and revise steps 1, 2, and 4 with them. Be honest, open-minded, and ready to adapt!

I suggest that you go to your mentor with multiple choice questions and not essay questions.

Multiple choice questions present choices like; should I do ABC or XYZ?
Essay questions are stuff that you can easily google-out, like; how do I move from A to Z?

Take down notes to avoid repeating questions and making your mentor feel like you are wasting their time!

Be Mentor-able, but at the same time don’t expect to be spoon-fed!

By not being spoon-fed, I mean you should not always wait for everything to be given to you or explained to you. Of course, hints are allowed, like the ones I am giving to you in this article when I ask you to google-out.

4) Assess Yourself

This part of the journey is important for the moments when you need motivation and will also help save you from eventualities like Burnout.

It is all about psychology and doing some introspection (thinking about you!). It is unique for every individual. Nobody knows you better than YOU.

The following questions should help you have an idea;

Who are you? — a little personal SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats) is not such a bad idea.
How do you learn best?
When are you in your “A-game”? etc

Some people already have answers to such questions.

From my experience, the answers are made especially clear when you face challenging situations and the way you go about solving them.

5) Assess Your Needs;

a) What do you NEED TO KNOW, to be able to do what you want?

There are tons of programming languages to pick from, such as:

JavaScript, Python, Java, PHP, C#
Or frameworks — (Front-end: Express, Django, Rails, Laravel, Spring, etc…Back-end: Angular, React, Vue Ember, backbone)

b) What resources do you need TO USE?

Equipment (hardware and software)
Courses?
Offline (Books), online, or both?

Your mentor can help you make a choice.

Don’t wait to have EVERYTHING before you start, but there are some key elements you need.

Have you ever gone on a long trip but forgotten to pack your Notebook or phone charger? Or maybe a toothbrush? With phones and toothbrushes, it is easy to get around it. Notebooks? Almost impossible!

STAYING ON THE JOURNEY

Congratulations on reading this far!

I am certain that the first part of this article helped you make some necessary adjustments both physical and mental.

This next part is going to set you apart from the majority of learners out there! My intention here is to give you a few hints that will help you stay motivated and consistent while at the same time having lots of fun all the way!

6) Have an appropriate setup;

Do your best to prepare the area where you will be spending your learning time. This is also a matter of individual taste, preferences, and budget. I will not talk about the software setup here!

In my humble opinion, a coding setup should be Ergonomic enough. I am thinking about the long-term health consequences on your body especially;

Your eyes
Your neck and back (in fact your whole vertebral column)
Your major joints (knees and elbows)
Your butt also deserves some attention!

I am sure you get the whole idea.

Other aspects include but are not limited to:

Ventilation
Lighting
Temperature
Noise
Distractions
Aroma?

How you set up will determine if you will like to continue learning or not! Believe me!

Tech Twitter can help you with set up inspiration! So….

7) Join Twitter! (#100DaysOfCode)

This article is already getting too long and I do not wish to waste any more of your precious time yammering about how motivating it is to realize that you are not alone on this journey!

Ask your mentor to help you set up a suitable Twitter account.

Start simple, just post regularly about what you learned that day. Let it be interesting and straight to the point! Or you can comment on or retweet interesting posts.

Find and connect (follow) with people with experience in the same field as you are.

Help others when you can.

I will leave it at that… you will soon figure it all out!

Although I must mention a few aspects concerning communication!

8) Mind your communication

Take out some time to reflect on the information you send out (post) and how you receive and interpret information.

If you feel you have communication challenges, then one of your priorities should be improving your communication skills because you will be doing a lot of communication during your journey in form of questions and or answers through emails and social media posts.

Like Austin Kleon says in one of his bestsellers — KEEP GOING, resolve not to argue with strangers online!

I totally recommend his books! I am NOT marketing here! He doesn’t even know me!

9) Do not compare yourself with others!

Comparing with others is the fastest way to stop your journey!

Comparison will not only rob you of all your positive energy and creativity, it will leave negative feelings of envy and bitterness in their place! You end up feeling worse than when you started!
If you have never read or heard about THE DESIDERATA, then google it out AFTER you are done with this article!

If you must compare, grab a mirror and do it with the person you see in the mirror!

Each second you spend learning makes you better than you were before you started the learning session.

You can find that out when you evaluate your progress based on the objectives set at the beginning of the process.

Be optimistic and patient with yourself because sometimes it takes some time to notice huge improvements.

Some areas will be easier to learn and master than others.

Take note of your learning. It will give you something to post about on social media but most importantly it will boost you up. Even if it is just a single syntax or a new HTML tag! Be grateful for it…

10) Be appreciative — and show it!

Just say a thank you!

To your mentor
To yourself
To anyone who deserves it!

It is as simple as that!

Let them know what they have done!

Start now! Apply the knowledge and never forget this;

“You don’t have to be great to start, but you have to start to be great” — Zig Ziglar

THANK YOU for reading this far! 🙏🏾

If it was worth it, please;

clap, comment, and share with someone you feel might need it!
Let’s connect especially on Twitter!

If I made some mistakes please let me know (just me!) — I am only human too😌

I got some of my quotes from the Brainy Quotes website.