DEV Community: Akash Santra

Fixing AI Observability: How I Added GenAI Semantic Support for RAG Embedding Spans in Mastra

Akash Santra — Wed, 17 Jun 2026 12:51:08 +0000

OpenTelemetry has become the standard for observing modern systems.

But when you start building AI applications, traditional traces aren't enough.

You don't just want to know that a request happened.

You want to know:

Which model generated the output?
Which provider was used?
How many tokens were consumed?
What embedding model processed the documents?
How much did the operation cost?

These questions become even more important when building Retrieval-Augmented Generation (RAG) systems.

Recently while contributing to Mastra, I discovered an observability gap involving RAG embedding operations.

This led me to open a pull request that introduced proper OpenTelemetry GenAI semantic mappings for RAG_EMBEDDING spans.

The Problem

Mastra already exported rich metadata for several AI operations.

However, RAG embedding spans were missing standardized GenAI semantic attributes.

As a result, observability tools could see that an embedding operation occurred, but they couldn't easily understand:

Model information
Provider information
Token usage
Embedding-specific metadata

Without standardized semantic conventions, dashboards and tracing systems lose valuable context.

This becomes a bigger issue in production environments where teams need visibility into AI workloads.

Understanding RAG Embedding Spans

A typical RAG pipeline looks like this:

Documents
    ↓
Chunking
    ↓
Embedding Model
    ↓
Vector Database
    ↓
Similarity Search
    ↓
LLM Generation

The embedding stage is critical.

Every document chunk gets transformed into a vector representation.

If observability data from this stage is incomplete, debugging performance issues becomes significantly harder.

Why OpenTelemetry Semantic Conventions Matter

OpenTelemetry doesn't just define traces.

It also defines semantic conventions.

These conventions create a common language for telemetry data.

Instead of every framework inventing custom field names, everyone follows the same standard.

For GenAI workloads this means tools can automatically understand attributes such as:

gen_ai.system
gen_ai.request.model
gen_ai.response.model
gen_ai.usage.input_tokens
gen_ai.usage.output_tokens

Standardization enables better interoperability across platforms and observability vendors.

The Fix

The goal was straightforward:

Map RAG embedding telemetry data to OpenTelemetry's GenAI semantic conventions.

The implementation included:

Exporting embedding model metadata
Exporting provider information
Mapping token usage metrics
Aligning span attributes with OpenTelemetry standards
Preserving compatibility with existing tracing infrastructure

This allows downstream observability systems to understand embedding operations without requiring custom integrations.

Why This Matters for AI Engineers

As AI applications become more complex, observability becomes a first-class requirement.

Production AI systems need answers to questions like:

Which embedding model is causing latency spikes?
Which provider generates the highest cost?
How many tokens are consumed during indexing?
Which retrieval operations are failing?

Without standardized telemetry, these questions become difficult to answer.

With proper semantic conventions, observability tools can surface these insights automatically.

Lessons From Open Source

One thing I enjoy about open source is that small improvements often have larger impacts than expected.

This wasn't a flashy feature.

Users won't notice it immediately.

But maintainers, platform engineers, and teams operating AI workloads will benefit from more accurate telemetry and better visibility into their systems.

These kinds of contributions taught me an important lesson:

Not every valuable contribution adds new functionality.

Sometimes the most impactful improvements make existing systems easier to understand, monitor, and operate.

Final Thoughts

AI infrastructure is evolving rapidly.

Frameworks, observability platforms, and standards are all maturing at the same time.

Contributing to these ecosystems provides a unique opportunity to learn how modern AI systems work under the hood.

For me, this contribution was another reminder that reading unfamiliar codebases often leads to discovering interesting problems.

And occasionally, solving one of those problems helps improve the developer experience for everyone else.

If you're contributing to AI infrastructure projects, don't overlook observability.

The best AI systems aren't just intelligent.

They're observable too.

GitHub: https://github.com/Akash504-ai
Open Source Contributor | Backend Engineering | AI Systems | OSS

How I Finally Finished My Unfinished Chat App with GitHub Copilot

Akash Santra — Fri, 05 Jun 2026 06:46:26 +0000

This is a submission for the *GitHub Finish-Up-A-Thon Challenge*.

What I Built

PASO is a real-time chat platform inspired by apps like WhatsApp, Discord, and Slack.

It supports:

Real-time messaging
Group chats
Voice and video calling
AI-powered moderation
OTP-based password recovery
Admin dashboard
Redis-powered Socket.IO scaling
Machine Learning integration using FastAPI

I originally started this project as a way to learn full-stack development and distributed systems. Like many side projects, it reached a point where the core features worked, but there were still many unfinished parts, missing documentation, incomplete testing, and several production-level improvements that I wanted to add.

The GitHub Finish-Up-A-Thon gave me the perfect reason to come back and finally finish it properly.

Demo

Application Preview

The Comeback Story

One thing I like about this graph is that it tells the story better than I can.

I started PASO earlier as a learning project and made steady progress for a while. Like many side projects, it eventually slowed down and sat unfinished.

The GitHub Finish-Up-A-Thon gave me a reason to come back to it.

The spike in contributions during May reflects the work that went into finally completing the missing pieces: Redis scaling, OTP password recovery, testing improvements, documentation, roadmap planning, and overall project polish.

What started as "I'll come back to this someday" turned into a project I was finally proud to share publicly.

Before

Basic chat functionality
Limited documentation
Minimal test coverage
Single-instance Socket.IO setup
Missing password recovery flow
Several unfinished project docs
No clear future roadmap

What I Added

Redis Horizontal Scaling

One of the biggest upgrades was adding Redis support for Socket.IO.

This allowed the application to move beyond a single server setup and support communication across multiple instances using Redis Pub/Sub.

Working on this taught me a lot about distributed systems and real-time communication.

OTP Password Recovery

I implemented a complete OTP-based password recovery system.

Users can now:

Request password reset codes
Verify OTPs
Securely update passwords

This made the authentication flow much more realistic and production-ready.

Testing Improvements

I spent a lot of time improving tests across the project.

I added and expanded tests for:

Authentication
Messaging
Groups
Admin functionality
Middleware

This gave me much more confidence when making changes.

Documentation Overhaul

A huge part of this challenge was finishing the documentation.

I added and improved:

Architecture documentation
Deployment guides
Scaling documentation
Testing guides
Security documentation
Roadmap planning

The goal was to make the project easier for future contributors to understand and use.

Roadmap & Future Planning

I also completely updated the roadmap to better reflect where the project is today and where it can go in the future.

Future plans include:

End-to-End Encryption
Mobile Applications
Enterprise Features
Multi-region Scaling
Plugin Ecosystem

Community Response

One thing that surprised me was seeing people actually use and interact with the project.

After sharing PASO publicly and documenting the development journey, the project received:

5,000+ LinkedIn post views
8 GitHub stars
7 GitHub forks

Posts:

For a student-built open source project, that was incredibly motivating and reminded me why finishing projects matters.

My Experience with GitHub Copilot

GitHub Copilot played a huge role throughout this project. I didn't use it to blindly generate code.

Instead, I used it as a development partner.

Where Copilot Helped Most

Boilerplate Code

Copilot helped generate:

Controllers
Routes
Middleware
Database models
API handlers

This saved a lot of repetitive work.

Testing

One of the biggest benefits was test generation.

Copilot helped me create test cases much faster, which encouraged me to write more tests than I normally would.

Debugging

While implementing Redis scaling and authentication features, Copilot was extremely useful for:

Explaining errors
Suggesting fixes
Identifying edge cases
Reviewing implementation ideas

Documentation

A large portion of the documentation improvements were accelerated with Copilot.

It helped me structure technical documentation, explain architecture decisions, and improve overall clarity.

What I Learned

The biggest lesson from this project is that finishing a project requires a different mindset than starting one.

Building features is fun.

Writing tests, improving documentation, fixing edge cases, and polishing the developer experience is where real software engineering happens.

GitHub Copilot helped me spend less time on repetitive tasks and more time learning system design, architecture, and problem solving.

Final Thoughts

PASO started as a learning project.

This challenge pushed me to turn it into something much closer to a real production-ready application.

More importantly, it reminded me that unfinished projects still have value.

Sometimes the hardest part isn't building a project.

It's coming back and finishing it.

For me, that's what this challenge was really about.

Thanks GitHub and DEV for creating a challenge that encouraged builders to finish what they started. ❤️

If you'd like to check out the project, feedback and contributions are always welcome.

⭐ Project Repository: https://github.com/CodePlaygroundHub/paso-chat-app
GitHub: https://github.com/Akash504-ai
LinkedIn: https://www.linkedin.com/in/akash-santra-5823b42a6/
X (Twitter): https://x.com/akashsantra999

# Why Hermes Agent Feels Different From Most AI Agent Frameworks

Akash Santra — Sat, 30 May 2026 02:25:50 +0000

This is a submission for the Hermes Agent Challenge

Over the last year, I've been obsessed with building AI-powered systems. From experimenting with LLM applications to contributing to open source and building real-world products, I've spent countless hours trying to understand what separates a simple chatbot from a genuinely useful AI system.

While exploring different agent frameworks, I came across Hermes Agent. At first glance, it looked like another entry in the rapidly growing "AI agents" space. But after digging deeper, I realized it was aiming at a much more interesting problem: creating agents that can learn, remember, and become more useful over time.

That's why Hermes Agent immediately caught my attention.

The Problem With Most AI Agents

A lot of AI projects today stop at the "chat" stage.

They can answer questions, generate content, or call a few tools, but they often struggle when a task requires persistence, planning, memory, and adaptation over time.

Real-world work rarely happens in a single prompt.

Whether you're researching a topic, debugging a codebase, managing a project, or automating repetitive work, the system needs context and continuity.

This is where agentic systems become interesting.

What Makes Hermes Agent Different

What stood out to me about Hermes Agent is its focus on long-term usefulness rather than one-off interactions.

The idea of an agent that can:

Learn from previous interactions
Store useful knowledge
Search its own memory
Improve skills through experience
Work with different models and providers

makes it feel much closer to what developers actually want from AI assistants.

Instead of creating another chatbot, the goal becomes building a system that can evolve alongside its user.

Of course, no framework is a silver bullet.

The effectiveness of any AI agent still depends on model quality, tool integrations, prompting strategies, and the problems it's being asked to solve. Hermes Agent doesn't magically solve those challenges, but it does provide a strong foundation for developers who want to build more capable systems.

What I find particularly interesting is that it treats memory and learning as first-class concepts rather than optional add-ons. As AI systems become more integrated into our daily workflows, those capabilities will likely become increasingly important.

How I Would Use Hermes Agent

One project idea I've been exploring is a developer productivity agent.

Imagine an assistant that can:

Analyze GitHub issues
Generate implementation plans
Search project documentation
Summarize pull requests
Maintain context across sessions
Learn common workflows over time

Rather than repeating instructions every day, the agent gradually becomes more useful because it remembers how you work.

For engineers working on large projects, that kind of persistent context can save a surprising amount of time.

Why Open Source Matters

One aspect I particularly appreciate is that Hermes Agent is open source.

The AI ecosystem is increasingly dominated by closed platforms and proprietary systems. While those tools can be powerful, developers often have limited visibility into how they work and limited control over where they run.

With Hermes Agent, developers can inspect the code, modify behavior, self-host deployments, and experiment freely.

That flexibility is incredibly valuable for learning and innovation.

Personal Note

Alongside this exploration, I've been building a Hermes Agent-powered project that I'm genuinely excited about. It's still under active development, and I expect it will take another 3–5 weeks before it's ready to be shared publicly.

Rather than rushing out an unfinished version, I'm focusing on polishing the architecture, improving reliability, and making sure the user experience is something I'm proud of. Once it's ready, I'll publish a detailed technical breakdown covering the design decisions, implementation challenges, and lessons learned while building with Hermes Agent.

If you'd like to follow my work or future updates, you can find me on GitHub:

GitHub: https://github.com/Akash504-ai

Final Thoughts

The most exciting part of modern AI isn't generating text.

It's creating systems that can remember, reason, plan, adapt, and collaborate with humans over time.

Hermes Agent represents an interesting step toward that future. Whether you're building developer tools, research assistants, productivity systems, or entirely new categories of software, the ability to combine memory, learning, and tool use opens up possibilities that extend far beyond traditional chat interfaces.

I'm excited to continue experimenting with Hermes Agent, finish the project I'm currently building, and share the results with the community in the coming weeks.

Thanks for reading.

If you're building with Hermes Agent too, I'd love to hear what you're working on.

My CI/CD Architecture

Akash Santra — Sun, 24 May 2026 01:41:46 +0000

Why I Decided to Add CI/CD

As my AI-powered realtime communication platform started growing, manual deployments and inconsistent validation became difficult to manage. I wanted a more production-oriented workflow with automated checks, deployment pipelines, and scalable infrastructure practices.

Challenges Before Automation

Before introducing CI/CD:

Manual deployment workflows were error-prone
Frontend/backend validation was inconsistent
Merge stability became harder to maintain
Infrastructure scaling introduced additional complexity

CI/CD Workflow Architecture

The workflow is divided into two major phases:

CI Phase

Pull request validation
Linting and formatting
Build checks
Security and dependency scanning
Automated validation

CD Phase

Artifact generation
Docker image publishing
Staging deployment
Production deployment workflow

What I Learned

Building this pipeline helped me better understand:

Deployment automation
Fail-fast engineering workflows
Continuous integration principles
Infrastructure reliability
DevOps-oriented system design

What’s Next

I’m currently working on:

Redis-based scaling improvements
Docker Compose setup
Integration testing
Load balancing experiments
Architecture refinements

I Built an AI-Powered Real-Time Chat App with React, Node.js and Machine Learning

Akash Santra — Fri, 27 Mar 2026 04:22:52 +0000

I built a full-stack real-time chat application inspired by modern messaging platforms, but enhanced with AI automation and Machine Learning moderation.

GitHub Repo: https://github.com/Akash504-ai/Chat-app

Live Demo: https://chat-app-sooty-mu.vercel.app

What makes this project different?

This is not just a basic chat app. It is designed like a production-ready communication platform with real-world features.

Core Features

Real-time one-to-one and group chat using Socket.io
Message reactions, replies, pin and delete
Seen status with tick indicators
Global message search

AI and Machine Learning Features

AI chatbot integration using Groq API
Toxic message detection using ML models
Spam detection system
Admin moderation pipeline

Communication Features

Voice and video calling using ZegoCloud

Security and Authentication

Secure login and signup
Three-level security questions
Password recovery with identity verification

UI and Customization

Built with Tailwind CSS and DaisyUI
Dynamic themes
Custom chat wallpapers

Admin Panel

User management system
Report handling and moderation
Analytics dashboard
CSV export support

Tech Stack

Frontend

React.js
Tailwind CSS
DaisyUI

Backend

Node.js
Express.js
Socket.io

Machine Learning Service

FastAPI
Scikit-learn models

Database

MongoDB

Architecture

Frontend handles UI and client logic
Backend manages APIs and real-time communication
ML service handles moderation and predictions

This separation makes the system scalable and closer to real-world applications.

What I Learned

Building real-time systems with Socket.io
Integrating Machine Learning into production apps
Designing scalable architecture
Handling complex chat state and features

Future Improvements

Improving ML models
Adding push notifications
Better mobile responsiveness
UI and UX improvements
Use webRTC instead of ZegoCloud
Any feature which can improve this

Contributing

Contributions are welcome
Feel free to fork the repository and submit a pull request.

DEV Community: Akash Santra

Fixing AI Observability: How I Added GenAI Semantic Support for RAG Embedding Spans in Mastra

The Problem

Understanding RAG Embedding Spans

Why OpenTelemetry Semantic Conventions Matter

The Fix

Why This Matters for AI Engineers

Lessons From Open Source

Final Thoughts

How I Finally Finished My Unfinished Chat App with GitHub Copilot

What I Built

Demo

Live Demo

GitHub Repository

Screenshots

Application Preview

The Comeback Story

Before

What I Added

Redis Horizontal Scaling

OTP Password Recovery

Testing Improvements

Documentation Overhaul

Roadmap & Future Planning

Community Response

My Experience with GitHub Copilot

Where Copilot Helped Most

Boilerplate Code

Testing

Debugging

Documentation

What I Learned

Final Thoughts

# Why Hermes Agent Feels Different From Most AI Agent Frameworks

The Problem With Most AI Agents

What Makes Hermes Agent Different

How I Would Use Hermes Agent

Why Open Source Matters

Personal Note

Final Thoughts

My CI/CD Architecture

Why I Decided to Add CI/CD

Challenges Before Automation

CI/CD Workflow Architecture

CI Phase

CD Phase

What I Learned

What’s Next

I Built an AI-Powered Real-Time Chat App with React, Node.js and Machine Learning