Just 5 days ago, a persistent bottleneck in custom software delivery and solutions engineering landed on my desk: the Business Discovery Phase. It is notoriously manual, fragmented, and time-consuming. Teams regularly spend days or weeks sifting through unstructured meeting recordings, chaotic chat exports, PDF transcripts, and legacy UI screenshots just to draft a basic requirements document and propose an initial Proof of Concept (POC).
I wanted to tackle this challenge directly: Could a local, multi-agent AI system compress this entire discovery-to-prototype lifecycle into just a few minutes?
I gave myself a strict sprint to build a working prototype from scratch. Here is what I engineered, the technical hurdles I hit along the way, and the architecture that emerged from the experiment.
The Naive Prototype
I started by designing a linear 5-node LangGraph pipeline running on local Ollama models, wrapped inside a Streamlit interface:
- Ingest raw multimodal inputs (PDF notes, chat logs, screenshots).
- Synthesize operational pain points and missing requirements.
- Propose three architectural solutions.
- Generate a single-file Streamlit web application.
- Validate & Compile the code in a sandbox.
While the flow was clean conceptually, testing it against real-world inputs broke it immediately.
I was initially executing the generated Streamlit script using Python's exec() directly inside the parent dashboard's execution context. This triggered immediate React DOM tree collisions and duplicate widget ID errors (DuplicateKeyError), which crashed the parent app’s session state.
True Process Isolation
When an application's primary function is generating and hosting another interactive web application, in-process execution is a major failure point.
My first key breakthrough was completely isolating the generated application into a background subprocess on a dedicated port (8502).
Here is how I managed the lifecycle controls in main.py:
def launch_poc_subprocess(app_file: str, port: int = 8502):
"""Safely terminates old instances and spawns a new isolated background Streamlit process on port 8502."""
stop_poc_subprocess()
cmd = [
sys.executable, "-m", "streamlit", "run", app_file,
f"--server.port={port}",
"--server.headless=true",
"--browser.gatherUsageStats=false"
]
new_proc = subprocess.Popen(
cmd,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
)
st.session_state["poc_subprocess"] = new_proc
# Poll until child server responds with HTTP 200 OK
for _ in range(15):
time.sleep(0.3)
try:
with urllib.request.urlopen(f"http://localhost:{port}", timeout=1) as res:
if res.status == 200:
break
except Exception:
pass
With the child process running independently, I embedded the live application into the parent UI using a clean st.iframe.
Ghost Widgets and Database Nightmares
As I pushed further, two deeper runtime issues emerged.
First, to verify that generated code would not crash prior to rendering, I needed a headless simulation harness. I wrote a MockStreamlit class and passed it into exec(). However, when the generated code executed import streamlit as st, Python's module loader resolved streamlit directly from sys.modules, overwrote the local mock, and rendered interactive "ghost widgets" across the parent UI during step execution!
Note: What is a "Ghost Widget"?
In this context, a ghost widget is an unintended, live UI component that accidentally renders across the main parent application's interface during background testing. This happens when the code execution bypasses local mocks and leaks into the global Python state, causing the background test to physically draw widgets on your screen instead of just simulating them in memory.
To fix this CPython import leak, I stripped out all Streamlit imports with regex before passing the string to exec():
class MockSessionState(dict):
def __getattr__(self, key):
return self.get(key)
def __setattr__(self, key, value):
self[key] = value
class MockStreamlit:
def __init__(self):
self.session_state = MockSessionState()
self.sidebar = self
def __getattr__(self, name):
# Gracefully handle arbitrary Streamlit calls
return lambda *a, **kw: None
# Strip imports prior to headless execution
stripped_code = re.sub(
r'^(?:import\s+streamlit.*|from\s+streamlit\s+import.*)$',
'# [stripped by testing agent]',
poc_code,
flags=re.MULTILINE
)
mock_globals = {"st": MockStreamlit(), "__name__": "__main__"}
exec(stripped_code, mock_globals)
The second issue was database lifecycles. The LLM repeatedly generated SQLite databases with top-level conn.close() calls, causing locked connections or wiping data on every Streamlit widget rerun. I established a strict Zero-Database Architecture, enforcing via prompt constraints that all transient data must live strictly in native st.session_state dictionaries and lists.
The Automated Self-Correction Loop
Once the generation was functioning, I noticed the model still occasionally hallucinated unimported dependencies like pd.DataFrame or timedelta.
Rather than relying on one-shot generation, I introduced a dedicated Testing & QA Agent (Node 6) into the LangGraph orchestration topology.
I implemented an AST (Abstract Syntax Tree) scanner to lint the generated code structure, flag forbidden database drivers, and detect missing imports. If the generated code fails AST verification, headless mock execution, or semantic outline auditing, the testing agent compiles a structured diagnostic report and routes the graph state back to the generator node for automatic self-healing.
# Flag any forbidden third-party DB / pandas imports
banned_db_modules = {"sqlite3", "sqlalchemy", "pymongo", "psycopg2", "mysql", "cx_Oracle", "pyodbc", "pandas"}
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name.split(".")[0] in banned_db_modules:
errors.append(f"ForbiddenImport: `import {alias.name}` - store all data in st.session_state.")
elif isinstance(node, ast.ImportFrom):
if node.module and node.module.split(".")[0] in banned_db_modules:
errors.append(f"ForbiddenImport: `from {node.module}` - database libraries are forbidden.")
The Final Architecture & End-to-End Validation
The system ultimately evolved into a self-correcting 6-node multi-agent engine powered entirely by local open-weight models (qwen2.5:14b-instruct, qwen2.5-coder:14b-instruct, minicpm-v:latest) orchestrated via LangGraph and Qdrant.
To evaluate how well it generalized, I ran a benchmark session using an unstructured Fleet Dispatch & Maintenance Log scenario containing WhatsApp chats, scanned driver paper timesheets, and a meeting transcript PDF.
The Output:
- Process Understanding: Identified that manual paper records were creating high vehicle downtime and delayed maintenance schedules.
- Solution Proposal: Formulated a winning design for a Central Dispatcher Command Center & Automated Maintenance Scheduler.
- Code Generation & Verification: Built a 180-line Streamlit application that passed static AST analysis, headless mock execution, and semantic QA checks on its first self-correction pass.
- Deployment: Hosted the live application cleanly on port 8502 inside the studio's isolated iframe.
Key Takeaways
- Subprocess Isolation is Essential for Meta-Apps: When building tools that generate and run other web applications, in-process execution will inevitably pollute global runtime state. Decoupling into isolated child processes and iframe embedding is the most resilient pattern.
- Constraint-Driven Generation Reduces Failure Modes: Constraining code generation to an in-memory, zero-database state model (st.session_state) vastly improved reliability compared to allowing arbitrary third-party library imports.
- Multi-Tier Testing is Non-Negotiable: Combining static AST checks, headless runtime mocking, and semantic LLM auditing creates a dependable self-healing loop for AI-generated code.
If you want to explore the implementation or run the pipeline locally, the repository is available here: https://github.com/shantanavKapse/poc_agent
Top comments (0)