DEV Community: Sultan Ali Khan

LLMs as Analytics Translators: Can Generative AI Replace the Business Dashboard?

Sultan Ali Khan — Sun, 06 Sep 2026 06:27:29 +0000

Subtitle: Building a full-stack analytics dashboard that ingests Google/Meta/TikTok ad data and translates ROAS/CTR into plain-English business advice.

The hardest part of data analytics isn't collecting the numbers—it's interpreting them. Marketing managers stare at spreadsheets full of ROAS, CPC, and CTR, but they often ask, "So, is this good or bad?"

I built Adlytix AI to answer that question. It is a Streamlit-based marketing analytics dashboard that ingests CSV/Excel data from Google Ads, Meta, and TikTok, and automatically generates human-readable business insights using the Anthropic Claude API.

Live deployment: https://adlytix-ai.streamlit.app/

The Architecture
Data Ingestion: Users upload CSV/Excel files. The backend (Python/Pandas) automatically detects column structures and cleans the data.

Metric Calculation: The engine computes ROI, ROAS, CPC, and CTR dynamically.

The Generative Layer (The Magic): The structured metrics are passed to a pre-engineered prompt that asks Claude to:

Identify the Top 3 Best Performing campaigns and explain why.

Identify the Bottom 3 Worst Performing campaigns and suggest actionable fixes.

Summarize the overall health of the marketing portfolio.

Visualization: Plotly renders interactive charts showing trends.

The Prompt Engineering Strategy
Getting Claude to generate accurate insights was the hardest part. If you just dump a Pandas DataFrame into the prompt, the LLM gets confused.

My successful prompt structure:

System Role: "You are a Senior Marketing Analytics Consultant. You speak plainly. Do not use jargon unless absolutely necessary."

Data Injection: Insert the summarized table (mean/median performance) rather than raw rows to save tokens.

Constraint: "If the data suggests a campaign is underperforming, provide 3 specific reasons why it might be happening (e.g., low audience targeting, high competition, poor creative)."

Output Format: Enforce a strict structure: ### Key Takeaways -> ### Campaign Deep Dive -> ### Actionable Recommendations.

The Impact (User Feedback)
I deployed this to a small group of 5 early-stage startup founders. The feedback was phenomenal:

65% reduction in time spent interpreting dashboard metrics. They said they used to spend 2 hours analyzing the spreadsheet; now they spend 5 minutes reading the AI summary and 30 minutes acting on the recommendations.

The "Why" factor: They loved that the AI didn't just flag the losing campaign but hypothesized why it was losing (e.g., "Campaign X has a high CPC relative to the average. This suggests increased competition for these keywords.").

Research Caveats and Lessons
Hallucination in Numbers: Claude occasionally misreads a number if the CSV has formatting issues (e.g., "$1,000" vs "1000"). I mitigated this by standardizing all numbers to floats before passing them to the prompt.

Generic Advice: If you don't give the LLM enough context about the industry, it gives generic advice. I added an optional input field where the user can specify their industry (e.g., "SaaS B2B") which significantly improved the relevance of the generated insights.

Cost: For small dashboards (500 rows), the cost per insight generation is negligible (<$0.01). For massive datasets, I use aggregation first to avoid high token costs.

Conclusion
Can LLMs replace the dashboard? No. Dashboards are for monitoring; LLMs are for translating. Adlytix AI proves that LLMs are incredibly effective at bridging the gap between complex data science and practical business decision-making.

This project is open-source on my GitHub, so feel free to fork it and add your own connectors!

Author: Sultan Ali Khan | AI/ML Engineer |

sultanalikhan7543 / adlytix-ai

AI Marketing Analytics Dashboard

📊 Adlytix AI — AI Marketing Analytics Dashboard

Stop guessing. Start knowing.

Upload your ad performance data (Google Ads, Meta, TikTok) and get instant metrics + AI-powered insights in seconds.

✨ Features

📁 Upload CSV or Excel ad exports from any platform
📐 Auto-calculates ROI, ROAS, CPC, CTR
📊 Interactive charts (ROAS, Spend vs Revenue, CTR, CPC)
🤖 AI-generated plain-English insights powered by Claude
🏆 Best and worst campaign detection
💡 Zero setup required

🚀 Live Demo

👉 adlytix-ai.streamlit.app

🧰 Built With

Python 3
Streamlit
Pandas
Plotly
Anthropic Claude API

📄 How to Run Locally

git clone https://github.com/YOUR_USERNAME/adlytix-ai.git
cd adlytix-ai
pip install -r requirements.txt
streamlit run app.py

Add your ANTHROPIC_API_KEY to a .env file.

📊 Sample File Format

Campaign	Spend ($)	Clicks	Impressions	Revenue ($)
Summer Sale	500	1200	45000	2100

📬 Contact

Built by Sultan Ali Khan · sultanalikhan0344@gmail.com

View on GitHub

When Synthetic Data Isn't Enough: A Real Talk About Validating Financial ML Models

Sultan Ali Khan — Fri, 04 Sep 2026 13:58:31 +0000

Subtitle: Building a real-time order-flow imbalance predictor with LightGBM, and why specifying your model's limitations is just as important as reporting its accuracy.

In the ML community, we love to celebrate the shiny metrics (AUC, F1, Sharpe Ratio). But in the real world, validation is brutally hard.

I recently completed a project called Flux—a live signal engine designed to predict short-horizon order-flow imbalance using live cryptocurrency microstructure data. I built it with a live Binance WebSocket feed, an SQLite-backed signal logger, and a real-time browser dashboard.

But here is the most critical part of this project: I explicitly documented the limitations of my synthetic training data and outlined exactly what real data is required for genuine predictive validation.

This is the story of why that matters.

The Project Overview
Goal: Predict short-term order-flow imbalance (buy/sell pressure) from live market microstructure data.

Model: Gradient-Boosted Trees (LightGBM).

Infrastructure: Python backend consuming Binance WebSocket; SQLite for logging predictions; Plotly Dash for monitoring.

The Synthetic Data Trap
When I started, I didn't have access to a rich, annotated historical order-book dataset. To get the system working, I generated synthetic training data based on my assumptions of market microstructure—things like mean reversion, spread dynamics, and typical tick volatility.

The model trained beautifully. It hit 92% accuracy on my validation split.

I was thrilled for about 10 minutes.

Then reality hit me: Models trained on synthetic assumptions often fail catastrophically when exposed to the chaotic, irrational nature of human trading and market manipulation.

How I Handled the Limitation (The Research Approach)
Instead of ignoring this massive red flag, I treated the limitation analysis as the core contribution of the project. Here is what I did:

Explicitly Called It Out: I wrote a dedicated section in my project README titled "Validation Limitations." I did not hide it in a footnote.

Specified the Exact Real Data Required: I detailed the specific order-book history needed for genuine validation:

Level 2 Data: Full depth of the order book, not just top-of-book.

Timeframe: At least 6 months of tick-by-tick data covering both bull and bear regimes.

Latency Metadata: Timestamp precision down to milliseconds to match the model's prediction window.

Deployed a "Shadow Mode" Monitor: The engine currently runs in "shadow mode" in production. It makes predictions and logs them to SQLite, but it does not execute trades. This allows me to collect real-world performance data to compare against the synthetic baseline.

The Real-Time Dashboard
To observe the model's behavior, I built a lightweight browser dashboard using Flask and Plotly. It tracks:

Real-time prediction scores.

Feature importance drift (to see if market conditions are changing).

Key Lessons for Aspiring ML Engineers
Synthetic Data is a Prototyping Tool, Not a Production Solution: Use it to build the pipeline, but don't trust the accuracy scores.

Do not "Hack" Validation: It is tempting to choose a validation set that fits your narrative. Don't. Clearly state what your model can't do.

Document Your Blind Spots: Professors and senior engineers look for this. A researcher who knows why their model might fail is infinitely more valuable than one who claims their model is perfect.

Future Work
The immediate next step is to acquire real Level 2 order-book data and retrain the model. I am currently sourcing historical data from exchange APIs to replace the synthetic baseline.

If you are applying for research positions in ML/Finance, always highlight what your model can't do. It shows you think like a scientist, not just a coder.

Author: Sultan Ali Khan | Independent AI Researcher |

sultanalikhan7543 (Sultan Ali Khan) · GitHub

I am a recent Software Engineering graduate with a robust background in development in Python, Machine Learning and SQL. - sultanalikhan7543

github.com

Building Conversational LLM Applications: Why Context Gathering Beats Single-Shot Prompting

Sultan Ali Khan — Fri, 04 Sep 2026 12:16:43 +0000

Subtitle: A deep dive into building a 9-tool multi-domain generation suite with Claude, and why asking clarifying questions is the secret to reducing hallucinations.

As an independent AI researcher, I recently built Meridian—a conversational AI suite that powers 9 distinct tools (ranging from an SOP writer to a job search advisor) from a single core engine.

The standard approach to building these tools is single-shot prompting: The user asks, the LLM answers. It is fast, simple, and often wrong.

In this post, I’ll walk you through why I deliberately abandoned zero-shot generation for a conversational context-gathering architecture, and how this modular approach drastically improved output relevance.

The Problem with "One and Done"
When I first started building Meridian, I fell into the classic trap. I built a prompt that said: "You are a CV builder. Here is the user's raw data. Generate a perfect CV."

The results were generic, filled with hallucinated achievements, and required massive manual editing. Why? Because the LLM lacked context. It didn't know the user's specific career goals, the target industry, or their unique strengths.

The Research Question: If we force the LLM to ask clarifying questions before generating, does output quality measurably improve?

The Solution: A Conversational Gatekeeper
I designed a two-step architecture:

The Contextual Scraper (Chat Mode): The user selects a tool (e.g., "CV Builder"). Instead of generating immediately, the engine switches to "chat mode." It asks 3-4 clarifying questions: "What specific role are you targeting?", "Which metric from your last job best demonstrates impact?", "What are your 3 core technical strengths?".

The Generational Engine (Prompt Mode): Once the user answers, the engine takes the original system prompt, injects the accumulated context, and generates the final output.

We built the frontend in HTML/JS and connected it directly to the Anthropic Claude API.

Key Architecture Design: The 9-Tool Modular Suite
Instead of building 9 separate applications, I built one core engine and a library of Domain-Specific System Prompts.

The Core: Handles API calls, context memory, and response parsing.

The Prompts: Each tool (SOP Writer, Thesis Writer, University Finder, etc.) has its own dedicated system prompt instructing the LLM on the exact formatting, tone, and structure required for that domain.

This modular approach allows me to update or improve a single tool without breaking the entire ecosystem.

The Results (Qualitative Evaluation)
While formal statistical testing is ongoing, early user testing (n=12) revealed a striking insight:

2–3 rounds of clarifying questions increased output relevance scores (based on user feedback) by approximately 40%.

Users reported that the "Chat then Generate" flow felt more like collaborating with a human assistant rather than querying a search engine.

Hallucinations regarding specific facts (like dates, numbers, or names) dropped noticeably when the system had exact user-provided context.

Major Lessons Learned
System Prompts are the Glue: You cannot just rely on the base model's intelligence. The system prompt must explicitly instruct the model on how to ask questions (e.g., "Ask one question at a time, do not overwhelm the user").

State Management is Hard: When you switch from context-gathering to generation, you must aggressively compress the chat history to avoid exceeding the context window. I used summarization checkpoints to handle this.

Security: Since I call the Claude API directly from the frontend, I had to implement a serverless proxy for API-key protection. Never expose your API key to the client.

Conclusion
Meridian proved that conversational context-gathering significantly reduces the friction between user intent and LLM output. The 9-tool modular architecture is now my go-to template for building multi-purpose generative AI applications.

Next Steps: I am currently preparing a formal paper for an ACL Workshop based on these findings. If you are working on similar LLM architectures, I'd love to connect.

Author: Sultan Ali Khan | Independent AI Researcher | https://github.com/sultanalikhan7543/Meridian | https://www.linkedin.com/in/sultanalikhan149/