DEV Community: Rijul Rajesh

Build Your Own Personal AI Agent That Works for You With Hermes

Rijul Rajesh — Sun, 26 Jul 2026 19:32:48 +0000

Hello, I'm Rijul. I'm building git-lrc, a micro AI code reviewer that runs on every commit. It's free and source-available on GitHub. Star git-lrc to help more developers discover the project. Do give it a try and share your feedback

AI assistants can remember things about you now.

But what if you wanted to run your own personal AI agent that acts as your assistant across different parts of your life?

One that can:

Access your environment
Use tools
Remember the things you want it to remember
Continue working beyond a single chat window

For that, I want to introduce you to Hermes Agent.

What Is Hermes Agent?

Hermes Agent is an open-source personal AI agent developed by Nous Research.

At its core, it is a program that connects a language model to an environment where it can actually perform tasks.

You give Hermes a task.

Hermes sends that task to an AI model, receives the model's decisions, and then gives the model access to tools that can perform actions in the real world.

This makes it more than just a chatbot.

It is also different from traditional coding agents.

Coding agents are primarily designed to work on software projects.

Hermes is designed to act more like a personal AI assistant.

You can use it for tasks such as:

Research this topic.

Check my server.

Fix this code.

Search my previous conversations.

Run this workflow every morning.

Message me on Telegram when it's done.

The idea is to have an AI agent that can work across different areas instead of being limited to a single coding environment.

Now let's see how quickly we can set it up.

Setting Up Hermes Agent

You can find the repository here:

https://github.com/nousresearch/hermes-agent

The installation instructions are available in the README.

For this article, I'll demonstrate the Linux installation.

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

Once the installation is complete the setup wizard is launched, where you can configure the agent.

Once the setup is complete, we can start Hermes:

hermes

Hermes Agent in Action

Now let's try a simple example.

I'll give Hermes a workflow that I regularly follow when publishing technical articles.

I regularly publish technical articles.

When I finish an article, my usual workflow is:

1. Check the article structure
2. Create a thumbnail
3. Create a short LinkedIn post

Remember this workflow.

At this point, Hermes has been told about my workflow.

Now I'll close Hermes.

Later, I start it again and ask:

I just finished an article.

What should I do next?

Hermes is able to retrieve the workflow I previously told it about and suggest the next steps.

It even offers to help me with those steps.

This is the interesting part.

The memory belongs to an agent that I can run myself.

It is not limited to a single chat session.

I can run the agent, give it information, close it, and return later.

The agent can then use the information I previously asked it to remember.

That starts to feel much more like a personal assistant than a traditional chatbot.

More Than Just Memory

Memory is only one part of Hermes Agent.

It also supports various connectors and tools that allow the agent to interact with different systems.

This means you can potentially build workflows around things like:

Your local environment
Servers
Messaging platforms
Research
Scheduled tasks
Other external services

I'll cover some of those connectors in a separate article.

For now, the main idea is simple:

You can run your own personal AI agent, give it access to tools, teach it your workflows, and allow it to remember information across sessions.

That's it for this introduction to Hermes Agent.

Thanks for reading. I'd love to hear your thoughts in the comments.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Your AI Agent Has a Backpack. It's Called Retrieval Memory.

Rijul Rajesh — Sat, 25 Jul 2026 20:40:54 +0000

Imagine your AI agent has a backpack.

It stores your preferences, project details, past experiences, and procedures.

But it doesn't carry everything in that backpack into every conversation. When something becomes relevant, it reaches in and retrieves what it needs.

This is the basic idea behind retrieval memory.

The Context Problem

When you interact with an AI agent, it has a limited context window.

As the conversation continues, more information gets added to that context.

Eventually, the context can become full.

At that point, the system may need to:

Start a new session
Compress the existing context
Remove information that is no longer relevant

But there is another way to prevent the context from becoming unnecessarily large.

Instead of keeping every piece of information inside the active context, the system can store information externally and retrieve it only when it is needed.

This is the basic idea behind retrieval memory.

What Is Retrieval Memory?

Retrieval memory is a way for an AI system to remember information by storing it externally and retrieving the relevant parts when needed.

The basic flow looks like this:

User asks something
       ↓
Search stored memory
       ↓
Retrieve relevant information
       ↓
Add it to the AI's context
       ↓
Generate a response

The important idea is that the information does not need to remain inside the active context all the time.

It can be stored somewhere else and brought back when it becomes relevant.

A Simple Example

Suppose I tell an AI assistant:

My favourite programming language is Rust.

The system might store this as:

User preference:
Favourite programming language = Rust

Later, I ask:

Suggest a programming language for my next project.

The system can then:

Search its stored memories.
Retrieve the fact that I prefer Rust.
Add that information to the current context.
Use it when generating the response.

The AI might then suggest Rust, or at least consider it as one of the options.

The important point is that the preference did not need to remain in the active context between the two conversations.

It was stored externally and retrieved when relevant.

That is retrieval memory.

The memory is not directly inside the model. It is retrieved into the context when relevant.

How Is This Different From RAG?

Retrieval memory is similar to RAG, or Retrieval-Augmented Generation.

The basic process is similar:

Query
  ↓
Retrieve relevant information
  ↓
Add it to the context
  ↓
Generate a response

The main difference is usually the source of the information.

With traditional RAG, the system might retrieve information from:

Documentation
PDFs
Websites
Knowledge bases
Company documents

With retrieval memory, the system usually retrieves information specific to:

A particular user
A particular agent
A previous interaction
A project
A specific workflow

So the source is often personal or agent-specific memory, rather than a general document collection.

What Kind of Information Can Be Retrieved?

Retrieval memory can contain many different types of information.

User Preferences

The user prefers the Fetch API over Axios.

Facts

The user is working on Project X.

Past Experiences

This approach failed previously because the API rate limit was exceeded.

Procedures

When deploying this project, run these commands in this order.

Conversation Summaries

The previous discussion concluded that the system should use PostgreSQL instead of SQLite.

The system can then retrieve the relevant information when it becomes useful.

The Big Idea

Retrieval memory allows an AI agent to separate storage from active context.

Instead of keeping every piece of information in the context window all the time:

Store information externally
            ↓
User asks a question
            ↓
Search stored memory
            ↓
Retrieve what is relevant
            ↓
Add it to the current context

This helps the agent access information from previous interactions without carrying the entire history into every new conversation.

So when you see terms like semantic memory, episodic memory, or retrieval memory, they are describing different ways an AI system can store and use information.

Understanding these concepts makes it easier to understand how modern AI agents build memory systems that go beyond simply keeping a giant conversation history.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Context Compression: Making AI Agents Forget Without Losing the Plot

Rijul Rajesh — Fri, 24 Jul 2026 20:09:52 +0000

Suppose you are working with an AI agent to fix an API.

You give it a simple instruction:

Fix the API 500 error.

The agent might go through a workflow like this:

Agent:
→ Reads the logs
→ Searches the codebase
→ Checks the database
→ Checks recent commits
→ Runs tests
→ Tries a fix

After 30 tool calls, the agent's context can become huge.

But most of that information may no longer be useful.

The agent does not need to keep every detail of the investigation forever.

It does not need:

Every line from every log
Every failed search
Repeated information
Old tool outputs
Intermediate steps that are no longer relevant

What it really needs is the current state of the investigation.

The Context Problem

Imagine the agent has accumulated 40,000 tokens of context during the investigation.

That context takes up valuable space.

As the agent continues working, the context window gradually fills up.

Eventually, the agent may have less room for new information, which can lead to:

Higher costs
Slower processing
Less room for future tool calls
Important information being pushed out of context

So instead of carrying the entire history forward, we can compress it.

The original 40,000 tokens might become:

Goal:
Fix the API 500 error.

Found:
The error started after deployment v1.4.
The database is healthy.
The payment service is missing PAYMENT_API_KEY.

Tried:
Restarting the service. No effect.

Next:
Fix the environment configuration and retest.

This is context compression.

What Is Context Compression?

Context compression is not simply about making the context shorter.

The goal is to:

Remove the information that is no longer useful while preserving what the agent needs to continue working.

The agent does not need to remember every step it took.

It needs to remember the important conclusions from those steps.

Three Basic Context Compression Techniques

1. Pruning

The simplest technique is pruning.

You remove information that is no longer useful.

For example:

Agent:
→ Searched for config.yaml
→ Found nothing

Agent:
→ Searched for settings.yaml
→ Found nothing

Agent:
→ Searched environment variables
→ Found PAYMENT_API_KEY is missing

Once the agent has found the actual cause, the failed searches may no longer be useful.

They can be removed from the active context.

The important information is:

PAYMENT_API_KEY is missing.

Pruning is essentially:

Remove what the agent no longer needs.

2. Distillation

Instead of keeping the entire conversation, we can convert it into a structured summary.

For example:

Goal:
Fix the API 500 error.

Facts:
- The database is healthy.
- The error started after deployment v1.4.
- PAYMENT_API_KEY is missing.

Decisions:
- Do not modify the database.
- Fix the environment configuration.

Completed:
- Checked the logs.
- Verified database connectivity.
- Inspected environment variables.

Next Action:
- Add PAYMENT_API_KEY.
- Restart the service.
- Retest the API.

The original investigation may have taken thousands of tokens.

But the distilled state contains the information the agent needs to continue.

A useful structure might be:

Goal
Facts
Decisions
Completed Work
Next Action

Distillation is essentially:

Turn a long history into a structured state.

3. Generalisation

Sometimes an investigation contains knowledge that can be reused in future situations.

For example:

Specific experience:

Missing API key caused the payment API to fail.
              ↓
Reusable knowledge:

Check environment variables when an API integration fails.

The agent is no longer just remembering what happened in one specific incident.

It is extracting a general rule from that experience.

This can be useful for:

Future investigations
Runbooks
Agent skills
Long-term memory

Generalisation is essentially:

Turn a specific experience into reusable knowledge.

The Big Idea

An AI agent does not need to carry its entire history forever.

It needs to preserve the parts of that history that are still useful.

A long investigation might look like this:

40,000 tokens of raw history
              ↓
       Context compression
              ↓
      Goal + Facts + Decisions
              ↓
          Next Action

The agent can then continue working with a much smaller context while retaining the information that actually matters.

Context compression is not about forgetting everything.

It is about forgetting the right things.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

AI Runbooks Explained: How to Give AI Agents Procedures to Follow

Rijul Rajesh — Tue, 21 Jul 2026 19:59:48 +0000

If you have read about AI agents, you may have come across the term AI runbook.

You may also have seen people confidently talk about agents handling routine tasks, running commands, investigating issues, and performing actions much like a human engineer would.

But there is an important distinction here:

Just because an agent can use tools does not mean it knows what to do or what procedure to follow.

Think about the difference between an experienced engineer and a new employee joining a team.

An experienced engineer may already know the team's standard procedures.

A new employee might have the technical ability to do the work, but still needs to learn:

What to check first
Which tools to use
What actions to avoid
How the team normally handles a particular situation

AI agents are similar.

Suppose you tell an agent:

The API is broken. Fix it.

The agent might start investigating broadly.

It could check different files, inspect various configurations, run multiple commands, and try several approaches before figuring out what is actually wrong.

This can lead to:

Wasted time
Wasted tokens
Unnecessary tool calls
More iterations than necessary

But what if we already had a standard procedure for handling this particular problem?

The agent could follow that procedure and focus its investigation.

That is where AI runbooks come in.

What Is an AI Runbook?

A runbook is simply a documented procedure for handling a specific situation.

For example, suppose an API starts returning HTTP 500 errors.

A runbook might say:

The API is returning 500 errors.

1. Check if the API is running.
2. Inspect the logs.
3. Check the database configuration.
4. Verify that the database is accessible.
5. Make the smallest necessary fix.
6. Verify that the API is working again.

This gives the agent a procedure to follow.

The agent still has to perform the investigation and use its tools.

The runbook simply provides a structure for that investigation.

A Small AI Runbook Demo

To demonstrate this, I created a small task management API using FastAPI and SQLite.

You can also try the demo yourself through my repository

The project contains a runbook for handling API failures:

runbooks/
└── api-500-error.md

Now let's look at the scenario.

First, I start the server.

The API is working correctly.

Next, I intentionally break the application by changing the database path from:

./data/app.db

to:

./data1/app.db

Now, when I call the health endpoint, the API returns an error.

At this point, I ask my AI agent to investigate the issue.

The API is returning HTTP 500 errors.

Please check whether there is a relevant runbook in the runbooks/ directory and follow it.

Diagnose the root cause, make the smallest safe fix, and verify that the API is working again.

Instead of immediately changing application code, the agent can follow the procedure defined in the runbook:

Find the relevant runbook.
Check the API.
Inspect the logs.
Check the database configuration.
Identify the incorrect database path.
Fix the configuration.
Verify that the API is healthy again.

The agent still performs the investigation.

The runbook simply gives that investigation a structure.

In this case, the agent finds the root cause and applies the fix.

Prompt vs Runbook

Compare these two instructions.

A simple prompt might say:

Fix the API.

A runbook provides something more specific:

When the API returns a 500 error, check these things in this order.

Avoid making unnecessary changes.

After applying a fix, verify that the API is working again.

The difference is important.

With a simple prompt, the agent has to figure out the entire process by itself.

With a runbook, the agent gets procedural knowledge.

It knows:

What to check
In what order to check it
What actions to take
What actions to avoid
How to verify the result

This makes the agent's behavior more predictable.

The Big Idea

AI agents already have the ability to use tools.

They can:

Read files
Run commands
Inspect logs
Modify code
Call APIs

But tools only provide capabilities.

A runbook provides a procedure.

That is the key idea behind AI runbooks.

You are not just telling the agent:

Here are some tools. Figure it out.

You are giving it:

Here is how this team normally handles this situation. Follow these steps, use your judgment where necessary, and verify the result when you are done.

That is how you give an AI agent procedural knowledge.

And in many cases, that can lead to a much more predictable outcome than simply giving an agent a vague instruction and hoping it figures out the best approach.

That's it for this introduction to AI runbooks.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Semantic Memory: How AI Builds Knowledge About You

Rijul Rajesh — Mon, 20 Jul 2026 20:40:07 +0000

When building personal AI systems, one of the most important types of memory to understand is semantic memory.

In simple terms, semantic memory is a structured store of knowledge such as:

Facts
Concepts
Preferences
Relationships

It is the part of an AI memory system that helps answer:

"What do I know about this person, project, company, or concept?"

Episodic Memory vs Semantic Memory

Let's say the following conversation happens:

User: I usually use React and TypeScript for frontend projects.

AI: Got it.

This single conversation can be stored in two very different ways.

Episodic Memory

Episodic memory stores the event itself.

On July 21, 2026, the user said:

"I usually use React and TypeScript for frontend projects."

The system remembers:

What happened, when it happened, and what was said.

This is similar to remembering an experience.

Semantic Memory

Semantic memory extracts the knowledge from that conversation.

User
├── frontend_framework → React
└── primary_language → TypeScript

The system no longer needs to remember the original conversation to understand the underlying information.

It has converted the conversation into structured knowledge.

This is the key difference:

Episodic memory remembers the event.

Semantic memory remembers what was learned from the event.

Semantic Memory as a Knowledge Layer

You can think of semantic memory as a knowledge layer for an AI system.

For example, an AI assistant might maintain a structured representation of a user like this:

User
├── prefers → React
├── uses → TypeScript
├── prefers → Linux
├── works_on → E-commerce platform
├── interested_in → AI agents
└── prefers → concise technical explanations

This is semantic memory.

The information is no longer stored as a conversation.

Instead, it is stored as knowledge about entities and the relationships between them.

What Are Entities?

An entity is something that the AI system knows about.

Examples include:

User
Customer
Company
Product
Project
Document
Location
Organization

For example:

User ──works_on──> E-commerce platform

Here:

User is an entity
E-commerce platform is an entity
works_on is the relationship between them

This structure allows an AI system to reason about information instead of simply searching through old conversations.

Why Does This Matter?

Imagine an AI assistant that has stored hundreds of conversations with you.

If it only has episodic memory, it may need to search through those conversations every time it wants to answer a question about your preferences or projects.

With semantic memory, it can directly access structured knowledge such as:

User
├── uses → React
├── uses → TypeScript
├── works_on → E-commerce platform
└── interested_in → AI agents

The assistant does not need to reread every conversation to discover these facts.

The knowledge has already been extracted and organized.

That is what makes semantic memory such an important part of personal AI systems.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Are You Missing Out on Agent Skills? Here's How They Work

Rijul Rajesh — Fri, 17 Jul 2026 20:32:30 +0000

Do you keep hearing terms like “agent skills” being thrown around without really understanding what they mean?

And is the FOMO starting to get to you?

Let’s understand what agent skills are in this article.

What is a skill?

A skill is a way to give an AI agent procedural knowledge.

At its core, a skill is simply a SKILL.md file.

But what exactly does this file contain?

The YAML Frontmatter

At the top of the file, we have the YAML frontmatter.

At a minimum, it contains two things:

Name

This identifies the skill.

Description

This tells the agent:

What the skill does
When it should be used

The description essentially acts as the trigger condition for the skill.

Below the frontmatter, we have the body of the file.

The Body

This contains the actual instructions.

It explains what the agent needs to know to perform the task.

For example:

What steps should be followed
In what order should they be performed
What decisions should be made along the way
What rules or constraints should be followed

Skills can also contain optional directories.

scripts/

This can contain executable code that the agent can run when needed.

references/

This directory can contain additional documents that the agent can refer to.

assets/

This can contain static resources that may be needed while performing the task.

What Happens When an Agent Has Many Skills?

An agent can have a large number of skills.

But this raises an obvious question:

If an agent has hundreds of skills, does loading all of them consume a huge amount of context?

This is where a concept called progressive disclosure comes in.

The information is loaded in three stages.

1. Metadata Only

When the agent starts, it loads only the name and description of each skill.

This allows the agent to understand what skills are available without loading all of their instructions into the context.

2. Full Instructions

When the description of a skill matches the task, the agent loads the full instructions from that skill into its context.

3. Optional Resources

Scripts, references, and assets are loaded or used only when they are needed.

This prevents the agent from loading everything at once.

How Are Skills Different From Other AI Concepts?

There are a few concepts that may sound similar.

MCP: Tool Access

MCP gives agents the ability to interact with external tools and APIs.

It is mainly about what the agent can reach.

RAG: Factual Knowledge

RAG retrieves relevant information from a knowledge base.

It is mainly about giving the agent reference material and factual knowledge.

Fine-Tuning: Model Weights

Fine-tuning changes the model's weights.

The knowledge or behavior is effectively baked into the model itself.

Skills: Procedural Knowledge

Skills are different.

They provide the agent with procedural knowledge.

They tell the agent:

How to perform a task
What steps to follow
In what order to perform them
What decisions to make along the way

A skill can also use other technologies, such as MCP.

For example, a skill could provide the instructions and judgment for when to call a particular MCP tool.

An Open Standard

This format is based on the open standard from Agent Skills.

It has also been adopted by AI platforms such as Claude and Codex.

This makes skills a useful way to package and share procedural knowledge across different AI tools.

Be Careful With Skills

There is one important thing to keep in mind.

Skills can execute scripts.

That means installing a skill is not always as simple as downloading a text file.

A malicious skill could potentially introduce risks such as:

Prompt injection
Tool poisoning
Malicious scripts
Malware

So you should treat skill installation similarly to installing any other software.

Only install skills from sources you trust, and review what they contain before giving them access to your AI tools.

So that's it about this article, see you on the next one!

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Not Every LLM Is Built the Same: GPT vs Mixture of Experts (MoE)

Rijul Rajesh — Wed, 15 Jul 2026 09:19:08 +0000

Are you aware that there are different types of AI model architectures?

Two of the most common are Dense (GPT-style) models and Mixture of Experts (MoE) models.

I find myself using both.

For general day-to-day tasks, I usually prefer dense GPT-style models.

But when I'm researching a topic or working through a complex plan, I often reach for a Mixture of Experts model.

Why?

Let's look at how they differ.

Dense Models (Traditional GPT-Style)

A traditional GPT model is called a dense model.

To help you visualise better, I will use an example.

Imagine a company with 100 employees.

Every time a customer asks a question like:

"How do I bake a cake?"

All 100 employees gather together to answer it.

Even if only 10 of them actually know anything about baking, everyone still participates.

This is similar to how a dense model works.

Every layer of the model is used for every prompt.

All of the model's parameters are active during inference.

For example, if a dense model has 100 billion parameters, all 100 billion parameters are involved every time you ask it a question.

Mixture of Experts (MoE)

Now imagine the company works differently.

Instead of making everyone attend every meeting, there are specialists:

Programming expert
Law expert
Biology expert
Writing expert
Translation expert

When someone asks:

"Write Python code."

The manager says:

"Only the programming expert and the writing expert are needed."

The other experts stay idle.

This is exactly how a Mixture of Experts (MoE) model works.

A small component called the router decides which experts should handle your prompt.

So if a model contains 64 experts, perhaps only 2 or 4 experts are activated for a given request.

The rest remain inactive.

Examples of Dense and MoE Models

Dense (Traditional GPT-Style)

These models use all of their parameters for every request.

OpenAI GPT-3
OpenAI GPT-3.5
Meta Llama 1
Meta Llama 2
Meta Llama 3, 3.1, and 3.2
Google Gemma (all versions)
Qwen 2, 2.5, and the dense variants of Qwen 3
Mistral 7B
Microsoft Phi-2, Phi-3, and Phi-4

Mixture of Experts (MoE)

These models activate only a subset of experts for each request.

Mixtral 8x7B
Mixtral 8x22B
DeepSeek V2
DeepSeek V3
DeepSeek R1
Gemini 1.5
Gemini 2.0
Gemini 2.5
Grok 2
Grok 3
GPT-4 (widely reported as MoE)

Conclusion

Neither architecture is universally better.

Dense models are simpler and use their entire network for every prompt.

Mixture of Experts models improve efficiency by activating only the experts that are most relevant to the task.

As AI models continue to grow larger, MoE has become an increasingly popular approach for scaling performance without proportionally increasing inference costs.

Understanding the difference helps explain why modern LLMs can vary so much in speed, memory usage, and overall efficiency, even when they appear similar on the surface.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Finding the Right Local LLM Doesn't Have to Take Hours

Rijul Rajesh — Mon, 13 Jul 2026 20:18:49 +0000

Choosing a local LLM often comes down to trial and error. If you're not sure which models your hardware can comfortably run, here's a tool worth checking out.

Meet LLMFit

Repository:
https://github.com/AlexsJones/llmfit

llmfit analyzes your hardware and recommends the models that best fit your machine.

Instead of manually comparing VRAM requirements, quantizations, and model sizes, it tells you what is likely to run well on your system.

The installation instructions are available in the project's README.

For my machine, I installed it using:

curl -fsSL https://llmfit.axjns.dev/install.sh | sh

Once installed, simply run:

llmfit

This is the output I got:

Understanding the Output

At first glance, the table can look overwhelming.

Let's go through each column one by one.

Model

The name of the language model.

Examples include Llama 3.1 8B, Qwen2.5 14B, and many others.

Provider

The organization or repository that publishes the model.

Examples include Meta, bartowski, Unsloth, TheBloke, and other Hugging Face publishers.

Params

The number of parameters in the model.

Generally, larger models are more capable, but they also require more memory and compute.

Score

An overall recommendation score between 0 and 100.

It combines multiple factors such as:

Model quality
Generation speed
Hardware compatibility
Context length

Higher scores are generally better.

Tok/s

The estimated number of tokens the model can generate every second.

Higher values mean faster responses.

Quant

The quantization format used by the model.

Examples include:

Q8_0
Q6_K
Q4_K_M
Q3_K

Lower quantization levels usually reduce memory usage and improve speed, while slightly reducing quality.

Disk

The amount of disk space required by the selected model.

GPU

Shows how the model will actually run on your machine.

Examples include:

GPU
CPU + GPU
CPU

Mem %

The percentage of your available RAM or VRAM that the model is expected to use.

A value between 50% and 80% is usually a comfortable range.

Ctx

The maximum context window supported by the model.

A larger context window allows the model to process longer conversations and larger documents.

Date

The release date or last update date of the model.

This helps you judge how recent the model is.

Fit

Indicates how well the model matches your hardware.

Perfect — Excellent fit.
Good — Runs comfortably.
Marginal — Usable, but close to the hardware limit.
Too Tight — Not recommended for your machine.

Use Case

The primary scenario the model is optimized for.

Examples include:

General
Chat
Multimodal

That's all there is to it.

If you're planning to experiment with local LLMs, llmfit is a handy tool for figuring out which models are most suitable for your hardware before you spend time downloading them.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Typing Too Much While Using AI? This Tool Will Save You Hours

Rijul Rajesh — Mon, 06 Jul 2026 20:04:44 +0000

Are you tired of typing the same prompts over and over again?

I definitely was.

When working with AI tools, I noticed I kept writing the same instructions repeatedly. Sometimes I'd even dig through old chats just to copy and paste a prompt I had already written dozens of times.

It isn't difficult, but it breaks your flow.

Thankfully, there's a fantastic tool that completely solves this problem: Espanso.

You can create your own shortcuts, and Espanso instantly expands them into full text anywhere on your computer.

For example, here's the built-in :date shortcut.

As soon as I type :date, it automatically replaces it with today's date.

Pretty cool, right?

Let's install it and create our own shortcuts.

Installing Espanso

Head over to espanso.org.

You'll find the installation button right on the homepage.

Installation depends on your operating system:

macOS: Install the bundle or use Homebrew.
Windows: Download and run the installer.
Linux: Choose the installation method for your distribution.

Linux users

Before installing, check whether you're running X11 or Wayland.

echo $XDG_SESSION_TYPE

My machine is running X11, so I'll install the AppImage like so.

# Create the $HOME/opt destination folder
mkdir -p ~/opt

# Download the AppImage
wget -O ~/opt/Espanso.AppImage 'https://github.com/espanso/espanso/releases/latest/download/Espanso-X11.AppImage'

# Make it executable
chmod u+x ~/opt/Espanso.AppImage

# Create the "espanso" command alias
sudo ~/opt/Espanso.AppImage env-path register

Once the installation finishes, verify that everything is working.

espanso --version

Now start Espanso.

# Required only once
espanso service register

# Start Espanso
espanso start

That's it. Espanso is now running in the background.

If you'd like more installation details, the official documentation is available here:

https://espanso.org/docs/get-started/

Witness the Magic

Let's try one of the built-in shortcuts.

I'll type:

:espanso

As soon as I finish typing, Espanso replaces it with "Hi there!"

No copy-pasting.

No scripts.

Just instant text expansion.

Now let's create our own shortcut.

Creating Your Own Shortcuts

Espanso stores its configuration in a simple YAML file.

First, locate your configuration directory.

espanso path

Mine returns:

Config: /home/rtp/.config/espanso
Packages: /home/rtp/.config/espanso/match/packages
Runtime: /home/rtp/.cache/espanso

Open the Config directory and navigate to the match folder.

I will head to /home/rtp/.config/espanso

Inside, you'll find a file named base.yml.

Open it.

You'll notice some existing shortcuts already defined.

For example:

:date inserts the current date.
:espanso expands to Hi there!

Creating your own shortcut is just as easy.

A Real-World Example

When I'm prompting AI, I frequently include this instruction:

Make only minimal, non-destructive changes to the code.

It helps prevent the AI from making unnecessary modifications that could accidentally break my software.

The problem is that I end up typing this sentence countless times every week.

Instead, I created a shortcut.

- trigger: ":mdc"
  replace: "Make only minimal, non-destructive changes to the code."

After adding it to base.yml, my configuration looks like this:

Now, whenever I type :mdc, Espanso instantly expands it into the full sentence.

It feels like magic, and after a few days of using it, you'll wonder how you ever worked without it.

Wrapping up

Espanso is one of those small tools that quietly saves you hundreds of keystrokes every day.

Whether you're writing prompts, code snippets, email templates, commit messages, or anything repetitive, custom text expansions can make your workflow noticeably faster.

Huge kudos to Federico Terzi for creating such an awesome tool.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Building Word Embeddings with PyTorch and Lightning AI Part 2: Creating Labels for Next-Word Prediction

Rijul Rajesh — Sat, 04 Jul 2026 21:28:40 +0000

In the previous article, we created the training inputs for our word embedding model using one-hot encoding. In this article, we will define the labels and prepare the data for training

Let's continue by setting up the labels.

Our goal is to predict the next token given the current token.

Consider the sentence:

The Incredibles is great

If the current token is "The Incredibles", the next token is "is".

So, the correct label is the one-hot encoding for "is".

Next, if the current token is "is", the model should predict the one-hot encoding for "great".

After "great", the sentence ends. To continue training with the next sentence, we set the next token to the one-hot encoding for "Despicable Me".

Now let's convert these labels into a PyTorch tensor.

labels = torch.tensor([
    [0., 1., 0., 0.],
    [0., 0., 1., 0.],
    [0., 0., 0., 1.],
    [0., 1., 0., 0.]
])

At this point, we have finished encoding our training data.

Next, we combine the inputs and labels into a TensorDataset, and then use that dataset to create a DataLoader.

dataset = TensorDataset(inputs, labels)
dataloader = DataLoader(dataset)

Creating the Neural Network

Now let's start implementing the word embedding model.

class WordEmbedding(L.LightningModule):
    def __init__(self):
        # Initialize the weight tensors for the embedding network and loss function

    def forward(self):
        # Make a forward pass through the embedding network

    def configure_optimizers(self):
        # Configure the Adam optimizer

    def training_step(self, batch, batch_idx):
        # Calculate the loss (Cross Entropy Loss)

In the next article, we will begin implementing each of these methods one by one.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Building Word Embeddings with PyTorch and Lightning AI Part 1: Setting Up the Word Embedding Model

Rijul Rajesh — Fri, 03 Jul 2026 18:56:36 +0000

In this article, we will explore how to implement word embeddings using PyTorch and Lightning AI.

The implementation is based on the same example that we used in my Word2Vec article series.

First, let's import the required modules.

import torch
import torch.nn as nn

from torch.optim import Adam
from torch.distributions.uniform import Uniform
from torch.utils.data import TensorDataset, DataLoader

import lightning as L

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

These imports are used for the following purposes:

torch – Create tensors and use helper functions.
torch.nn – Create neural network layers and weights.
Adam – Optimize the neural network using backpropagation.
Uniform – Initialize the network weights.
TensorDataset and DataLoader – Prepare and load the training data.
pandas, matplotlib, and seaborn – Analyze and visualize the results.

Creating the Training Inputs

Now let's create the word embeddings for the following two sentences:

The Incredibles is great!
Despicable Me is great!

The first step is to create a token for every unique word in the training data and connect those tokens to the word embedding network.

Suppose we want to pass "The Incredibles" through the network.

In that case, we set its corresponding input to 1 and all the other inputs to 0.

This representation, where one input is 1 and all the others are 0, is called one-hot encoding.

In PyTorch, we can represent "The Incredibles" using a four-element vector with 1 in the first position and 0 everywhere else.

Similarly, we create one-hot vectors for the remaining words.

[
    [1., 0., 0., 0.],
    [0., 1., 0., 0.],
    [0., 0., 1., 0.],
    [0., 0., 0., 1.]
]

Finally, we convert these vectors into a PyTorch tensor.

inputs = torch.tensor([
    [1., 0., 0., 0.],
    [0., 1., 0., 0.],
    [0., 0., 1., 0.],
    [0., 0., 0., 1.]
])

Now that we have created the input data, the next step is to define the labels. We will explore that in the next article.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github

Building LSTMs with PyTorch and Lightning AI Part 9: Completing the Simplified LSTM

Rijul Rajesh — Thu, 02 Jul 2026 21:04:46 +0000

In the previous article, we just saw how we can start using a more simplified version of LSTM via pytorch via nn.LSTM()

In this article, we will continue building the simplified LSTM and test how it performs.

Implementing the `forward()` Method

Let's start by implementing the forward() method.

def forward(self, input):
    input_trans = input.view(len(input), 1)
    lstm_out, temp = self.lstm(input_trans)
    prediction = lstm_out[-1]
    return prediction

In input_trans, we reshape the input so that there is one row for each data point, regardless of how many data points we have.

Next, we specify that the input should have one column, since each data point contains only a single feature.

This reshaped input is then passed to the LSTM.

The output is stored in lstm_out.

lstm_out contains the short-term memory values produced by each LSTM unit as the sequence is processed.

In our example, the sequence contains four input values, so the LSTM is unrolled four times and lstm_out contains four outputs.

Next, we extract the prediction from the final LSTM unit by selecting the last element in the sequence using the index -1.

Finally, we return this prediction.

Configuring the Optimizer

Next, let's implement configure_optimizers().

It is almost identical to the previous implementation, except that we increase the learning rate from its default value of 0.001 to 0.1.

This allows us to observe how the Adam optimizer converges to the optimal weights and biases.

def configure_optimizers(self):
    return Adam(self.parameters(), lr=0.1)

Implementing `training_step()`

Finally, we implement training_step().

This method is exactly the same as before. It calculates the loss and logs the training progress.

def training_step(self, batch, batch_idx):
    input_i, label_i = batch
    output_i = self.forward(input_i[0])
    loss = (output_i - label_i)**2

    self.log("train_loss", loss)

    if label_i == 0:
        self.log("out_0", output_i)
    else:
        self.log("out_1", output_i)

    return loss

At this point, our model contains everything it needs:

__init__()
forward()
configure_optimizers()
training_step()

Testing the Model

Let's run the model before training and check its predictions.

model = LightningLSTM()

print("\nComparing observed and predicted values")

print(
    "Company A: Observed = 0, Predicted =",
    model(torch.tensor([0., 0.5, 0.25, 1.])).detach()
)

print(
    "Company B: Observed = 1, Predicted =",
    model(torch.tensor([1., 0.5, 0.25, 1.])).detach()
)

This produces:

Comparing observed and predicted values

Company A: Observed = 0, Predicted = tensor([0.0131])
Company B: Observed = 1, Predicted = tensor([0.0102])

Training the Model

Now let's create a Lightning trainer.

trainer = L.Trainer(max_epochs=300, log_every_n_steps=2)

We train for 300 epochs and set log_every_n_steps to 2.

By default, Lightning logs every 50 steps, which is too infrequent for a small training run like this.

Next, we simply call fit().

trainer.fit(model, train_dataloaders=dataloader)

Checking the Results

Once training is complete, we check the predictions again.

Comparing observed and predicted values

Company A: Observed = 0, Predicted = tensor([0.0001])
Company B: Observed = 1, Predicted = tensor([0.9857])

The prediction for Company A is now very close to 0, and the prediction for Company B is very close to 1.

Analyzing the Training

Let's open TensorBoard once again.

Notice that the graphs have flattened out as the model converged toward the desired predictions.

With that, we have completed our exploration of LSTMs.

We built an LSTM from scratch, then implemented a much simpler version using PyTorch's built-in functionality. Along the way, we also learned how to analyze the training process using TensorBoard and how to use those insights to make training decisions.

In the next series of articles, we will explore how to implement word embeddings using PyTorch and Lightning AI.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ star on Github