DEV Community: Saji John Miranda

Why GenAI Observability Breaks in Production

Saji John Miranda — Mon, 15 Dec 2025 07:44:35 +0000

GenAI systems usually look fine in development and staging.

Latency is predictable.
Token usage looks reasonable.
Costs seem under control.

Then the system moves to production — and something changes.

Costs creep up quietly.
Latency becomes inconsistent.
Retries and fallbacks increase.

But when teams look at their dashboards, nothing obvious is “broken”.

The problem: infrastructure metrics don’t explain AI behavior
Traditional observability answers questions like:

Is the service up?
Are requests failing?
Is CPU or memory saturated?

Those signals are useful — but they don’t explain how AI behavior changes over time.

In GenAI systems, cost and reliability are driven by things infra tools don’t model well:

token expansion across prompts
retries and partial failures
fallback model usage
routing changes
temperature and sampling effects
subtle execution drift without code changes

From the outside, the system looks healthy.
From the inside, behavior is changing.

That’s the gap most teams hit after production rollout.

The GenAI production blind spot
This is what teams usually miss:

GenAI systems don’t fail loudly — they drift quietly.

Costs don’t spike in a single incident.
Latency doesn’t collapse across the board.

Instead:

average cost per request slowly rises
tail latency worsens
retries become more frequent
fallback paths get exercised more often

And because prompts and responses are sensitive, many teams avoid collecting anything beyond coarse metrics.

So the very signals that explain why things are changing never get captured.

Why this is hard to spot early
Two reasons:

Behavioral signals aren’t first-class metrics
Tokens, retries, routing decisions, and execution paths aren’t treated like CPU or memory.
Prompt-level data is sensitive
Storing raw prompts or outputs creates privacy, compliance, and security concerns.

As a result, teams either:

collect too little and fly blind, or
collect too much and create risk.

A short visual explanation
I put together a short video that explains this production blind spot visually — why teams lose visibility once GenAI systems go live, and what kind of signals actually matter.

🎥 The GenAI Production Blind Spot
https://youtu.be/8O61U5EQpS0

It’s not a demo or a tutorial — just a clear explanation of the gap that appears in production.

Local LLM Observability for Developers — Introducing DoCoreAI (Free)

Saji John Miranda — Thu, 11 Dec 2025 13:31:00 +0000

Most LLM debugging tools only work in the cloud.

That means your prompts, responses, latencies, and costs get routed through external systems — which makes it hard to see what your model is really doing inside your own environment.

So we built something different.

🔥 Meet DoCoreAI — Local Observability for GenAI Apps

DoCoreAI runs locally, right beside your Python application, and gives developers instant visibility into:

🔹 Token usage
🔹 Latency & bottlenecks
🔹 Cost per prompt
🔹 Temperature behavior
🔹 Model drift
🔹 Response variations

All without sending any data to a cloud server.

Your prompts stay on your machine.

You get production-grade observability without cloud dashboards, lock-in, or complex setup.

🚀 Why Local Observability Matters

Cloud-based LLM monitoring tools (Langfuse, PromptLayer, etc.) are great, but they don’t always show:

✔ Real latency inside your environment

✔ Real behavior drift with your data

✔ Real cost impact inside your infra

✔ How temperature affects output in your pipeline

Local-first debugging gives you the truth.

If you're building:

AI agents
RAG applications
customer support tools
automation workflows
prompt pipelines
multi-model switching systems

…you need visibility before deployment.

⚡ Install & Run (It’s Just 3 Commands)

pip install docoreai
docoreai start
docoreai stop

This starts a local collector that observes your LLM calls (OpenAI, Anthropic, Gemini, etc.) and displays charts in your browser.

Free for up to 10 prompts.
Login to see full dashboards.

📊 What You’ll See

When DoCoreAI is running alongside your app, you’ll get:

🧠 Token usage breakdown

Which prompts are consuming the most?

⏱ Latency visualization

Where are you losing time?

📉 Operational Cost

Which prompts are bleeding your token budget?

🔥 Temperature behavior graph

How does temperature affect accuracy or creativity?

🌡 Model drift indicators

Detect inconsistencies early.

Most developers are surprised by how much inefficiency becomes obvious once they visualize real-world usage.

🔒 Privacy First: Nothing Leaves Your Machine

This is one of the biggest differences between DoCoreAI and cloud-based monitoring:

✔ All your prompts stay local
✔ All metrics are generated locally
✔ No prompt data is transmitted
✔ No vendor lock-in

If you’re working with sensitive data, this matters.

🎁 Free Tier for Individual Developers

You get:

10 prompts fully visualized
local collector
latency + token + cost + drift charts
temperature evaluator
developer playground behavior (via your own app)

Perfect for:

debugging
optimizing
comparing models
understanding behavior differences

🧑‍💻 Try It Yourself

pip install docoreai
docoreai start

💬 Feedback Wanted

We built DoCoreAI because we wanted an easier way to debug GenAI applications without sending prompts to external systems.

If you try it, I’d love to hear:

What metrics matter most?
What should we add next?
Would you use a local developer playground?
Should we open-source parts of the collector?
Comment below — I’ll respond to everyone.

⭐ Final Thoughts

LLMs are powerful, but debugging them is still painful.
Local observability gives developers the visibility they need to build faster, cheaper, and more reliable AI systems.

If you're tired of guessing what your model is doing, give DoCoreAI a try.

Happy prompting! 🚀

🔗 Register → https://docoreai.com/register

🔐 Generate Token → https://docoreai.com/generate-token

📘 Docs → https://docoreai.com/docs

🚀 DoCoreAI Is Now MIT Licensed — Join the Mission to Optimize AI Prompts Dynamically

Saji John Miranda — Mon, 14 Apr 2025 16:00:35 +0000

Hey devs 👋

After 30 incredible days of growth — 8,000+ downloads, 31 GitHub stars, and feedback from around the globe — I'm thrilled to announce something important for the community:

DoCoreAI is now MIT Licensed 🎉
(Goodbye CC-BY-NC-4.0, Hello open contribution freedom!)

🔍 What is DoCoreAI?

DoCoreAI (Dynamic Optimization & Contextual Response Engine) is a lightweight Python package that tackles a subtle but critical challenge in LLM usage:

🌡️ Prompt temperature tuning.

You’ve probably seen this before:

response = openai.ChatCompletion.create(
  prompt="Explain relativity to a child",
  temperature=0.7
)

That temperature value? It decides the creativity of the AI response — but most of us are just guessing.

DoCoreAI changes that.
It calculates the optimal temperature based on your prompt’s intent — analyzing parameters like:

🤔 Reasoning
🎨 Creativity
🎯 Precision

All without fine-tuning.
All from the prompt itself.

🧪 Example

from DoCoreAI import intelligence_profiler

params = intelligence_profiler("Write a fun poem about gravity", return_params=True)

# Output: {'reasoning': 3.2, 'creativity': 4.7, 'precision': 2.1, 'temperature': 0.84}

Now that’s smarter prompt engineering!

💡 Why the License Change?

Moving to the MIT License was a deliberate decision.

It means:

✅ No commercial restrictions
🤝 Easier collaboration and forking
🌍 Aligning with the open-source spirit

Whether you're building internal tools, LLM wrappers, or AI agents — you're welcome to use, remix, and build on top of DoCoreAI.

🌟 Join the Dev Mission

📦 Install:

pip install docoreai

💻 GitHub:
👉 https://github.com/SajiJohnMiranda/DoCoreAI

🔧 Explore, star ⭐️ the repo, raise issues, or contribute your ideas. Whether it’s performance tweaks, new metrics, or creative plugins — let’s push the limits of contextual optimization together.

Let’s stop guessing AI parameters —
and start letting them intelligently adapt.

🎉 8,215+ downloads in just 30 days!

Saji John Miranda — Sat, 12 Apr 2025 10:58:32 +0000

What started as a wild idea — AI that understands how creative or precise it needs to be — is now helping devs dynamically balance creativity + control.

🔥 Meet the brain behind it: DoCoreAI

💻 GitHub: https://github.com/SajiJohnMiranda/DoCoreAI

If you're tired of tweaking temperatures manually... this one's for you.

DoCoreAI

Introducing DoCoreAI – Dynamic Optimization & Contextual Response Engine

Saji John Miranda — Tue, 08 Apr 2025 04:20:19 +0000

AI-driven businesses and IT professionals, are you looking for faster, smarter, and more accurate AI responses? DoCoreAI is designed to optimize AI-generated outputs dynamically, eliminating the need for constant fine-tuning while enhancing accuracy, efficiency, and adaptability.

🔥 What’s inside the video?
✅ A quick intro to DoCoreAI
✅ A live demo showcasing its power
✅ The key benefits for businesses and AI professionals

🔹 Why should you care?
📌 AI Optimization at Scale – No more manual prompt tweaking
📌 Cost-Effective – Reduce fine-tuning expenses
📌 High Precision & Adaptability – Get intelligent responses dynamically

🔍 Who should watch this?
📊 Business leaders exploring AI-driven decision-making
💡 AI professionals & developers optimizing LLM workflows
🚀 Tech entrepreneurs looking for innovation in AI-powered customer support and automation

📢 Watch the full video and see how DoCoreAI transforms AI-driven interactions!

Github: https://github.com/SajiJohnMiranda/DoCoreAI

Pypi Installer: https://pypi.org/project/docoreai

Highlights:

👥 Tag your network of AI professionals, business leaders, and innovators who might benefit from DoCoreAI! Let’s build the future of AI optimization together

25 Days of DoCoreAI – A Reflection for Builders & Curious Developers

Saji John Miranda — Mon, 07 Apr 2025 11:25:20 +0000

Just 25 days ago, I launched DoCoreAI — a lightweight Python package designed to eliminate the guesswork of AI temperature tuning through something I call dynamic temperature profiling.

It began with a single idea:
“What if temperature could be optimized automatically, based on the intent of the user?”

Since then, this small utility has evolved into a surprisingly powerful helper for anyone working with prompts, especially developers and researchers trying to balance creativity, accuracy, and control.

🚀 What’s Happened So Far?

✅ 6,118+ Downloads on PyPI

✅ Over 20,000+ views from Reddit

✅ 950+ impressions on LinkedIn

✅ Launched on Product Hunt (with great early feedback)

✅ Positive chatter from developers across Reddit, Product Hunt, LinkedIn, and Twitter

✅ Started receiving profile views and DMs from curious developers

Here’s the image I often use to explain it visually:

💡 What Makes DoCoreAI Unique?

Traditional prompt engineering forces developers to manually guess a temperature value like 0.7, 0.3, or 1.0 — depending on the need for creativity, precision, or something in between.

DoCoreAI flips that process.

It analyzes your prompt context and applies dynamic tuning based on a profiling system that balances:

🎯 Precision
🧠 Reasoning
💭 Creativity
🔥 Temperature

All while letting you stay focused on the core logic of your app or experiment.

No fine-tuning.
No heavy compute.
Just plug in, and it works.

🧪 Who's It For?

If you're working on:

LLM-based apps (chatbots, agents, tools)
GenAI research and prompt optimization
OpenAI / Groq / Claude / Gemini integrations
RAG systems or structured AI pipelines

...and you've ever wished your LLM "just knew" how creative or precise it should be — this might be for you.

❤️ Community Reactions

Some of my favorite feedback so far:

“This is such a clever solution to a common problem with AI prompts!”
– Product Hunt

“Damn! That’s actually a genius concept. Why don’t they do this already?”
– Reddit IndieHackers

“Your temperature setting should be applied like a heatwave map…”
– PromptEngineering subreddit

From my LinkedIn

🛠️ What's Next?

Here’s what I’m planning for the second wave:

A GitHub star campaign
A comparison benchmark using HumanEval
Possibly converting it into a small SaaS tool or micro-API
Exploring early-stage partnerships or even media coverage (if the interest continues)

🤝 Get Involved

Star it on GitHub: DoCoreAI on GitHub

Try it on PyPI:

pip install docoreai

Share feedback or ideas: Drop a comment or ping me on Twitter / LinkedIn

Looking forward to hearing your thoughts!

Thanks for reading — and for everyone who's tried, tested, or even just asked a question about DoCoreAI:
You're shaping its evolution. 🙌

🛑 The End of AI Trial & Error? DoCoreAI Has Arrived!

Saji John Miranda — Thu, 20 Mar 2025 06:57:39 +0000

If you've ever worked on an AI project using large language models (LLMs), you've likely wrestled with a tricky setting: temperature. Set it too high, and your model starts generating unpredictable, overly creative responses. Set it too low, and your outputs become rigid and uninspiring.

Tuning the right temperature manually for every use case is a frustrating process of trial and error—until now.

🚀 Introducing DoCoreAI: A solution that dynamically adjusts temperature for you, ensuring each response is optimized for clarity, precision, and engagement.

The Developer’s Dilemma: Finding the Right Temperature

Temperature in LLMs controls the randomness of responses:

Higher temperature (0.7 - 1.0) → More creative & engaging, good for storytelling, brainstorming, or open-ended responses.
Lower temperature (0.2 - 0.4) → More precise & factual, good for technical, legal, or research-oriented answers.
Balanced temperature (0.4 - 0.7) → A mix of clarity and creativity, ideal for explanations and teaching.

The Problem? Manual Fine-Tuning Is a Pain.

Developers often:

Manually test multiple temperatures for every scenario.
Waste time tweaking values instead of focusing on AI logic.
End up with inconsistent results that don’t match user needs.

This slows down development and makes AI adoption more challenging.

How DoCoreAI Solves This Challenge

Instead of guessing the right temperature, DoCoreAI does the work for you using Intelligence Profiling. It dynamically determines the best temperature for your query based on context, user role, and intent.

Here’s how simple it is:

✅ No need to specify a temperature—DoCoreAI analyzes the request and picks the optimal value automatically.

Real-World Impact: Why This Matters for Developers

With DoCoreAI, you get:

🔹 Less trial and error → No more wasting time on tuning temperature settings.

🔹 More consistency → Reliable, high-quality responses across different use cases.

🔹 Better user experience → The right balance of clarity, engagement, and precision in AI interactions.

Comparison: Manual vs. DoCoreAI

Query	Manual (T=0.2)	Manual (T=0.8)	DoCoreAI
"Tell me a bedtime story"	Too dry	Too random	Balanced & engaging
"Summarize a legal contract"	Too rigid	Too vague	Precise & clear

With DoCoreAI, you get the best output without tweaking a single setting.

Final Thoughts: Smarter AI, Less Hassle

If you're tired of manually adjusting temperature settings in your AI projects, DoCoreAI is the game-changer you need!

✅ One function call → Optimized responses, every time.

✅ No manual tuning → Focus on building, not fine-tuning.

🚀 Try it now:

Github DoCoreAI

Start coding smarter, not harder. Let DoCoreAI handle the optimization, so you can focus on innovation.

What do you think? Drop your thoughts in the comments below! 👇

🚀 DoCoreAI: Supercharge Your AI Prompts with Dynamic Optimization!

Saji John Miranda — Thu, 13 Mar 2025 00:37:52 +0000

1. The Prompt Struggle is Real 😅

We've all been there—spending countless hours tweaking AI prompts, trying to get the perfect response from language models. Static prompts often lead to inconsistent results, leaving us frustrated and yearning for a smarter solution. ⚡Enter DoCoreAI

2. Introducing DoCoreAI: The Game-Changer 🎯

DoCoreAI is a next-gen open-source AI profiler that optimizes reasoning, creativity, precision, and temperature in a single step—cutting token usage by 15-30% and lowering LLM API costs.
It dynamically adjusts intelligence parameters, tailoring prompts for various AI roles, and ensures more accurate and efficient responses. This means less time spent on manual prompt engineering and more on building impactful applications.

Why Game-Changer?: ⚡ AI Meets Cognitive Intelligence

What if AI could think smarter, not harder? DoCoreAI introduces dynamic intelligence profiling, optimizing LLM interactions with human-like reasoning, creativity, and precision—a game-changer for developers building next-gen AI solutions.

DoCoreAI isn’t just evolving with AI innovation—it’s pioneering a revolution.

3. Under the Hood: How It Works ⚙️

Every DoCoreAI prompt is context-aware, with a role assigned to each query, ensuring accurate intent recognition.
The core cognitive skills of human intelligence— Reasoning, Creativity, and Precision —are dynamically analyzed.
DoCoreAI predicts the optimal levels of these skills based on the context of the prompt.
The carefully crafted system message instructs the AI to predict and assign these values dynamically based on context.
The temperature (T) is then predicted and set based on these values:
- - C = Creativity
- - P = Precision
- - R = Reasoning
The LLM then generates responses using these intelligence parameters, optimizing for accuracy, coherence, and efficiency—all in a single step.

🔹 What does this mean?

Instead of manual tuning, DoCoreAI self-optimizes prompts, making LLMs more intelligent, cost-efficient, and effective. 🚀

At its core, DoCoreAI utilizes intelligence profiling to fine-tune key parameters:

Reasoning: Enhances the AI's logical processing capabilities.
Creativity: Adjusts the model's ability to generate innovative responses.
Precision: Controls the specificity and accuracy of outputs.
Temperature: Modulates the randomness of the AI's responses.

By optimizing these parameters dynamically, DoCoreAI ensures that AI responses are contextually appropriate and role-specific.
👉Read more...

Show Me the Code! 🚀
Let's dive into an example to understand basic DoCoreAI:

from docore_ai import intelligence_profiler

# Initialize DoCoreAI with your API key
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Define your prompt
prompt = "Explain the theory of relativity in simple terms."

# Optimize the prompt for a 'Teacher' role
optimized_prompt = intelligence_profiler(prompt, role='Physics Teacher')

# Get the AI's response
response = clean_response(optimized_prompt)

print(response)

In this snippet:

We initialize DoCoreAI with an OpenAI API key.
Define a prompt asking for an explanation of the theory of relativity.
Optimize the prompt for a 'Teacher' role, which adjusts the intelligence parameters accordingly.
Retrieve and print the AI's response.

This approach ensures that the AI provides a clear and concise explanation suitable for teaching.

4. Why Should You Care? 🤔

Implementing DoCoreAI offers several benefits:

Sharper AI responses: Role-aware optimizations lead to more relevant outputs.
Time-saving for Developers: Reduces the need for manual prompt adjustments.
Cost-efficient: Optimized prompts can cut token usage by 15-30%, lowering API costs.

5. Get Started in 30 Seconds! ⏳

Ready to enhance your AI prompts? Here's how to get started:

Installation:

pip install docoreai

Explore the 👉GitHub Repository: Check out the DoCoreAI GitHub repo for more details and contribute to the project.
Join the Community: Share your experiences, provide feedback, and collaborate with other developers to improve DoCoreAI.

📺 DoCoreAI: The End of AI Trial & Error Begins Now!

6. The Future of AI Prompting 🚀

Imagine an AI that always knows exactly how to respond, adapting dynamically to various contexts and roles. With tools like DoCoreAI, we're moving closer to that reality, making AI interactions more intuitive and efficient.

Time for Action 🎬:

Fork the repo & experiment 🛠️
Share feedback—what use cases would YOU optimize? 🤔
Drop a comment with your thoughts & improvements!
Let's revolutionize AI prompting together!

To Summarize DoCoreAI:

✔ Analyses the Complexity of the query/user-request
✔ Adds the Intelligence required via intelligence parameters
✔ Responds on Role-based expertise (e.g., technical support vs. creative storytelling)
✔ Automatically decides the response content structure
✔ Predicts the optimal temperature required for your prompt
✔ Efficient Response based on the predefined intelligence parameters
✔ Saves tokens providing monetary benefits

Besides, eliminates the pain of manually tweaking temperature and other parameters

🔬 Experimenting with Intelligence Parameters

We tested DoCoreAI by comparing responses for the same question with and without intelligence profiling.

💡 Eg Query: "How to connect an Apple computer to my network?"

🔍 Conclusion: DoCoreAI enhances responses by tailoring reasoning, creativity, and precision dynamically.

🛠️ Future of DoCoreAI

The journey for DoCoreAI is just beginning...
We envision DoCoreAI as an essential AI tool for redefining prompt engineering, customer support automation, and personalized AI interactions.

Next Steps:

🔹 Work in progress: > Evals & Benchmarks
🔹 Open-source release
🔹 Blog series explaining key use cases
🔹 Optimizations for additional AI providers

🚀 Try DoCoreAI today and let AI think smarter, not just generate text.

👉 GitHub Repo

📌 Detailed Content:
Want to dive deeper into the concept of Intelligent Prompt Optimization? Check out my detailed write-up on Medium:
Intelligent Prompt Optimization

Pypi: DoCoreAI
Care to ⭐Star the repo:

What do you think about DoCoreAI? Let’s discuss in the comments! 💬🔥