DEV Community: Neel B

I Built a Finance Assistant Where Six AI Agents Check Each Other's Work

Neel B — Thu, 22 Oct 2026 19:14:05 +0000

I typed "can you check my portfolio" into my own app. About 13 seconds later it told me the holdings were worth ₹15,463.77, up ₹2,235.22 (16.9%) overall and down ₹269.55 for the day, then listed the three biggest positions with their weights. Every number came from the actual holdings data, and the answer said where it came from.

That one question went through four steps. A planner decided it was a portfolio question, a Portfolio agent read the holdings, a reviewer checked the findings, and a final step wrote the answer. Bigger questions fan out to several agents at once.

The app is called Finance Agents. It's an educational assistant for Indian and world markets, built with LangGraph.js. A supervisor plans each question, sends it to six specialist agents that run in parallel, checks what they found, and writes one answer. The agents can search your own PDFs (with RAG that runs entirely on your machine), read live market data, and see your Zerodha Kite holdings. It's React and TypeScript in one folder, and it's open source.

One thing up front: it's for learning about money, not for telling you what to buy. Nothing it says is financial advice.

Repo: github.com/neel-turbo/multi-agent-langraph-rag-finance-app

What it looks like

There are two screens. The chat screen has your past conversations on the left, the chat in the middle and market movers on the right, with a live ticker running across the top. On a tablet it drops to two columns, and on a phone the conversation list folds into a toggle.

The market panel switches between India (Nifty, Sensex) and world indices, and shows the top gainers and losers with little sparklines. It uses Yahoo by default, so you don't need an API key for any of it.

The second screen is the portfolio. Holdings come from Zerodha Kite, and you get summary tiles (current value, invested, total P&L, today's change), allocation by value, P&L per holding, a price-history chart with your average buy price, and the full holdings table. Click a bar or a row and the chart switches to that stock.

By default it reads Zerodha's public demo account (17 holdings), so you can try everything without connecting your own. Every chat is also saved as a LangGraph thread, so you can close the tab, come back later and pick up an old conversation where you left it.

How the agents work together

The whole thing is one LangGraph graph called supervisor. Here's what happens to every question:

plan ──Send──▶ agents (parallel) ──▶ review ──┬──Send──▶ agents ──▶ review ...
  │                                           └──────▶ synthesize ──▶ END
  └── small talk ──▶ END

And here's the same graph in LangSmith Studio, with all six specialists between plan and review:

1. Plan. If you just type "hi" or "thanks", a regex answers instantly and no model is called at all. Anything else goes to the planner, which splits the question into tasks and gives each one to the specialist whose tools can answer it. A task can depend on another one. A question like "can I reach my goal with what I hold now?" needs the Portfolio agent to finish before Goal Planning can start.

2. Agents. Each specialist is a ReAct agent: it thinks, calls a tool, looks at the result and repeats until it has an answer with sources. Agents that don't depend on each other run in parallel. LangGraph's Send makes that fan-out short:

function fanOut(state: SupervisorStateType): Send[] {
  const context = formatOutputs(state.agentOutputs);
  return state.dispatch.map(
    (task) =>
      new Send(task.agent, {
        task,
        context,
        userProfile: state.userProfile,
        round: state.round,
      } satisfies AgentTaskInput)
  );
}

Each agent gets a small private slice (its task, what the other agents found so far, the user's profile and the round number) instead of the whole graph state. That keeps every agent's prompt short and focused.

3. Review. This is the part I'm happiest with. It starts with plain code: any task whose dependencies are now done gets released, with no model call. If one agent answered a simple question in the first round, review gets out of the way. Otherwise a reviewer model reads all the findings together and looks for contradictions, gaps and numbers without a source. It can send follow-up tasks back to the agents, for up to 3 rounds.

4. Synthesize. Everything gets merged into one answer, along with the reviewer's notes, and streamed to the browser. Only this step streams. The planning and agent calls are tagged so their tokens stay off the screen, and the chat shows progress lines like "Planned: Finance Q&A" instead.

These are the six specialists and the tools each one gets:

Agent	What it handles	Tools
Finance Q&A	Concepts and definitions, explained for a learner	`rag_search`
Portfolio	Your holdings: allocation, concentration, risk, P&L	Kite holdings (in its prompt), `stock_quote`, `rag_search`, `sip_future_value`, `cagr`
Market	Indices, gainers and losers, a stock's price, why the market moved	`market_movers`, `stock_quote`, `market_search`
Goal Planning	Target amount, time horizon, monthly saving needed	`sip_future_value`, `cagr`, `rag_search`
News	Recent financial news and why it matters	`news_search`, `market_search`
Tax	Tax rules, slabs, deductions, account types	`rag_search`

The Portfolio agent is the odd one out. It doesn't have to call a tool to see your holdings: they're put straight into its task as compact JSON, so it starts with the data in front of it. The planner sends any "my portfolio", "my stocks" or "my P&L" question its way.

RAG that never leaves your machine

The Finance Q&A, Portfolio, Goal Planning and Tax agents can search your own PDFs: a tax guide, a fund factsheet, a textbook chapter. I wanted that part to be fully local, so there's no parsing API, no embedding API, and no document goes anywhere.

PDF ──LiteParse──▶ Markdown, page by page ──chunker──▶ chunks + metadata
    ──bge-small (local)──▶ vectors ──▶ ChromaDB "finance_docs"

You add documents with one command, tagging them so agents can narrow their search later:

npm run ingest -- pdfs/tax_guide_2025.pdf --domain tax --jurisdiction IN --tax-year 2025

Parsing. LiteParse is LlamaIndex's local PDF parser. It's written in Rust, runs inside Node and turns the PDF layout into Markdown, so headings stay headings and tables stay tables. Scanned PDFs can go through OCR with --ocr.

Chunking. Every heading starts a new chunk and gets copied to the top of it, so a chunk still makes sense when an agent reads it on its own. Tables always get their own chunk and are never split, so a tax slab table stays whole. Text fills a chunk up to 1,800 characters (about 450 tokens, inside bge-small's 512-token limit). Anything under 120 characters gets dropped, which clears out stray page numbers and footers. Chunks never cross a page boundary, so every chunk has exactly one page number to cite.

Embedding. bge-small-en-v1.5 runs in Node through Transformers.js. No Python. The model is about 130 MB, downloads once, and works offline after that. Each chunk becomes a 384-number vector. Search questions get the query prefix bge recommends; stored passages don't.

Searching. When an agent calls rag_search, it can filter by domain, country and tax year, or ask for tables only (the Tax agent does this for rates and slabs). The tool takes the 6 closest chunks and keeps only the ones with a similarity of at least 0.68. Each passage comes back with its citation, like [tax_guide_2025.pdf, page 42 — Capital gains on equity], and agents are told to cite every fact that way.

If nothing clears the threshold, or ChromaDB isn't running, the tool searches the web with Tavily instead and says so in its answer. The agent keeps going instead of failing, and you can see from the citations whether an answer came from your documents or the web.

Things that bit me

The parser threw away page numbers. LiteParse gives you a whole document as one Markdown string, with nothing marking where a page ends. That's useless when every answer has to cite [document, page]. So the ingest script counts the pages first, then parses them one at a time and drops a separator line before each page's Markdown. The chunker reads those separators to know which page it's on. It's an extra pass per PDF, but now every chunk carries exactly one page number.

Picking the similarity cut-off. Too low and the agents quote irrelevant passages with total confidence. Too high and they never use your documents. I measured it on my own index: on-topic tax questions scored between 0.71 and 0.88 with bge-small, off-topic ones between 0.52 and 0.63. The default of 0.68 sits in that gap, and you can change it with RAG_MIN_SIMILARITY if your documents behave differently.

Small models choked on JSON objects. The Portfolio agent gets your holdings in its prompt. Sending one object per holding repeated every key 17 times, and small local models with a 4K context started rambling. Switching to one list of column names plus plain rows fixed it:

// Columns + rows instead of one object per holding: under half the tokens, and small local
// models (4K context) read it reliably where the object form made them ramble.
const rows = holdings.map((h) => HOLDING_COLUMNS.map((column) => h[column]));
const json = JSON.stringify({ totals, columns: HOLDING_COLUMNS, rows });

One failing agent shouldn't sink the whole answer. Every agent retries on a fallback provider (local Ollama by default) if the main one hits a quota or an outage. If the fallback fails too, that agent returns a short [portfolio failed: …] note instead of throwing, and the other agents carry on. The reviewer and the final step both see that note, so the answer can say what's missing instead of the whole turn erroring out.

Every provider wants different settings. Current Claude models and OpenAI's GPT-5 family reject a custom temperature, so the app doesn't send one to them. Gemini and Ollama get temperature: 0 for consistent answers. Claude's structured output also had to use its native JSON-schema mode, because a forced tool call isn't allowed while thinking is on. All of that lives in one file, models.ts, so the agents never have to care which model they're running on.

API keys can't leak into the browser by accident. The browser's TypeScript project excludes src/server entirely. If a React component ever imports server code, the build fails with a compile error instead of quietly shipping a key to the client.

Try it yourself

You'll need:

Node.js 20 or newer
An LLM API key: Gemini by default, or Claude or OpenAI
A Tavily API key for web, news and market search
Ollama, if you want a local fallback model (optional)

Market data from Yahoo and the Kite demo portfolio need no keys at all. ChromaDB comes with the project's npm packages, so you don't need Python or Docker for it.

# 1. Install
npm install

# 2. Configure: set TAVILY_API_KEY and your LLM key (GOOGLE_API_KEY by default)
cp .env.example .env

# 3. Start ChromaDB in its own terminal and leave it running
npx chroma run --path ./local-db-data

# 4. Add your PDFs (once per document)
mkdir -p pdfs
npm run ingest -- pdfs --domain general

# 5. Optional: a local fallback model (it must support tool calling)
ollama pull qwen3.5:4b

# 6. Start the web app, the agents and the market API
npm run dev

Then open http://localhost:5173. One gotcha: npm run dev does not start ChromaDB, so keep step 3 running in a second terminal. If Chroma is down the app still works, but every answer will cite the web instead of your PDFs.

You can switch every agent to a different provider from .env, without touching code. You can also mix them per role, for example a strong model for planning and a cheaper one for the many agent calls:

PLANNER_PROVIDER=anthropic   # planning, review and the final answer
PLANNER_MODEL=claude-opus-5
AGENT_PROVIDER=openai        # the specialists' tool loops
AGENT_MODEL=gpt-5-mini

That's worth doing, because one question makes several model calls: a plan, one or more agents, a review and the answer. The server prints which model each role uses when it starts, and a missing key stops it straight away with a clear message instead of failing on your first question.

The LangGraph dev server also connects to LangSmith Studio, so you can watch every run step by step. In the trace for my portfolio question, review took 0.01 seconds: one agent had answered a simple question, so the fast path skipped the reviewer model completely.

What's still rough

I'd rather you hear it from me:

There's no login yet. The user id is hardcoded, so every conversation belongs to one user.
Top gainers and losers are ranked within the app's own watchlists, not the whole market.
Yahoo market data comes from an unofficial API. If it breaks, you can switch to Twelve Data or Alpha Vantage with a key.
The Kite demo account's prices are a fixed snapshot from 2023, while the price chart uses today's Yahoo prices, so the two won't match. A real Kite account fixes that, but its access token expires every day.
The LangGraph dev server is for local use. For production you'd deploy the graph with a persistent checkpointer.

That's it

What I like about this setup is that every answer is checkable. You can see which agents ran, what each one found, and which page of which document a number came from. When the documents don't have the answer, it tells you it went to the web.

The code is on GitHub: neel-turbo/multi-agent-langraph-rag-finance-app. If you try it with your own PDFs, I'd like to hear which questions it handles well and where it stumbles. Issues and PRs are welcome.

And if this was useful, a star on the repo helps other people find it.

I Built an App That Turns Phone Photos into 3D Models, and Nothing Leaves My Wi-Fi

Neel B — Thu, 22 Oct 2026 18:13:29 +0000

That mouse on the rug isn't really there. The real one is sitting on my desk. What you're looking at is a 3D copy of it, dropped into my room in AR, and it was built from photos I took on my phone.

The part I care about most is where it was built. Not on someone's GPU farm. On my MacBook, over my home Wi-Fi. The photos never left the house, and there was no sign-up, no upload and no credits to buy.

The app is called ThreeD. It's a React Native app for iPhone and Android, plus a small Node server that runs on your Mac and drives three engines: Apple's Object Capture, Tencent's Hunyuan3D (through ComfyUI) and a small vision model in Ollama. It's open source. In this post I'll show you what it does, how it's put together, and the things that bit me while building it.

Repo: github.com/neel-turbo/app-image-to-3d-local-ai-rn

Three ways to make a model

Open the app and the first thing it asks is how you want to capture. There are three answers, and each one trades accuracy for effort.

Photo set is the one I recommend. You walk around the object three times (eye level, from above, from low down) and take 12 photos on each ring, roughly 30° apart. That's 36 photos, although the Build button unlocks once you have 20 good ones. The Mac runs Apple's Object Capture on them and you get the object's real shape with its real colours. Small things like a logo or a button survive.

4 photos (AI) is the middle ground. Front, back and both sides go to Hunyuan3D-2's multi-view model, so the AI isn't guessing what the back looks like. It saw it.

1 photo (AI) is the quick try. One clear photo, and Hunyuan3D-2 mini imagines the rest. Front and sides look good. The back is a guess, and sometimes a funny one: details from the front, like a face, get mirrored onto the back of the head.

If you just want to play, the single photo is fine. If you want something you'll keep, take the extra two minutes and walk around it.

Most of the work is stopping bad photos

The hard part of photogrammetry isn't the maths. It's getting a human (me included) to take decent photos. A blurry shot, two photos from the same spot, the object half out of frame: any of those and Object Capture either hands you a broken mesh or gives up with "not enough features". So most of the phone-side code exists to catch bad photos before they ever reach the Mac.

First, the camera tells you where to stand. A guide grid sits over the preview with a top-down dial of the 12 spots around the object, a 3 × 12 grid of every photo you still need (Above, Eye level, Low), and one plain line like "Step about 30° to your right". Once you get the hang of it you can turn it off.

Auto-capture is my favourite trick in the app. When you walk around an object, your phone turns by the same angle you walked. So the app reads the motion sensors (CoreMotion on iPhone, the rotation vector on Android) and waits until you've moved about 30° to a new spot, at the right height for the ring. Hold still and it takes the shot. No two photos from the same place, and no tapping.

Every photo is then checked in C++: sharpness (variance of the Laplacian), exposure, contrast, and overlap with the previous shot using a 64-bit dHash. It's one pure C++ Turbo Module, so the same code runs on iOS and Android, and I can unit-test it on my laptop. JavaScript only decides what to do with the numbers ("too blurry, try again"). A bad shot gets rejected on the spot, with a tip.

The phone also cuts the object out of the background, with Apple Vision on iPhone and ML Kit Subject Segmentation on Android. For the AI modes that cut-out is centred on white with a 15% border, because that's the kind of input Hunyuan3D was trained on.

Finally there's an "Ask local AI" button. A 4B vision model (qwen3.5:4b, running in Ollama on your Mac) names the object, rates the photo Good, OK or Poor, and gives you tips. I didn't want a small model overruling a Laplacian filter about whether a photo is sharp, so the phone sends its own measurements along with the image and the gateway drops any tip that contradicts them. It runs at temperature 0 with a fixed seed too, so the same photo always gets the same advice.

Then you get to play with it

When a build finishes, it shows up in the Gallery tab. Every model your Mac has built is there.

Tap one and you get a proper 3D viewer. Drag in any direction (up and down too, so you can check the underside), pinch to zoom, or jump straight to Front, Back, Top and the rest. The viewer is Google's <model-viewer>, but it's served by the Mac rather than a CDN, so it works with no internet at all. It also uses soft studio lighting, so the colours look like the real object.

View in AR is the fun bit. On iPhone it opens AR Quick Look with a .usdz file. On Android it hands a .glb to Google's Scene Viewer. Either way you can put the model on your floor and walk around it like it's actually there.

Sharing goes through the normal share sheet. You can send the 3D file, the AR file (iPhone), a picture of the current view, or a view-only link. The link works in any browser on the same Wi-Fi for 24 hours, needs no app and no pairing, and only ever exposes a random link id, never your device token.

How it's put together

There are two halves. The phone does the capturing and all the per-photo maths. The Mac does the heavy 3D work. They talk over plain HTTP and a WebSocket on your Wi-Fi, and the phone finds the Mac with Bonjour (mDNS).

 📱 Phone app                          💻 Your Mac (same Wi-Fi)
 ─────────────                         ────────────────────────
 1. Take or choose ── upload ───────▶  2. ThreeD gateway (Fastify)
    photos (checked                        ├─ Apple Object Capture  (photo set → 3D)
    and cut out                            ├─ ComfyUI + Hunyuan3D   (1 or 4 photos → 3D)
    on the phone)                          └─ Ollama                (capture tips)
                                                    │
 4. View, AR, share ◀──── 3D model ─────  3. Builds and cleans the model

The phone app is a tiny super app

If you write React Native, this is probably the interesting bit. The app is a shell plus two mini apps (Gallery and Profile) that are downloaded over the air. That means I can ship a Gallery fix without waiting on an App Store review.

Metro can't load separately built bundles at runtime, so the project uses Re.Pack (Rspack) with Module Federation 2 instead. The shell doesn't hard-code any remotes. On launch it reads a manifest from the Mac and registers each mini app with registerRemotes(), so the version can change without a new binary.

A few rules made this hold together:

Every native module lives in the shell. The stores don't let you download native code, so mini apps ship JavaScript only. VisionCamera, Firebase, the WebView, my C++ module: all of it is in the shell and shared as Module Federation singletons. There's a Jest test that fails if a native dependency isn't shared or its version drifts.
One Redux store for everything. The Gallery mini app injects its own slice when its bundle runs, so its state is code-split along with its code.
Every bundle is signed. Re.Pack signs each chunk with RSA and the shell refuses anything unsigned or modified. Rollouts are staged (say, 20% of phones first), and if a version crashes while rendering, that phone blacklists it and falls back to the previous one. The shell itself never goes down.

The Mac runs a small gateway

The phone never talks to Ollama or ComfyUI directly. Neither has any authentication, and on an 8 GB Mac they can't both be loaded at once anyway. So there's a Fastify server in the middle that does four jobs:

Pairs phones with a 6-digit code (the device token is only stored as a SHA-256 hash).
Takes the uploads, re-hashes them and throws away duplicates or anything that isn't really a JPEG or PNG.
Runs one heavy job at a time and evicts the other models from memory first.
Cleans up the mesh afterwards and colours it from your photos.

Don't underestimate the clean-up step. Raw AI meshes come out with floating bits, small holes and far too many triangles. The gateway welds the mesh, removes the floaters, fills holes, simplifies it with meshoptimizer to around 80,000 triangles and smooths the normals. Then it projects colour from your photo (or all four photos) with a depth test, so colour never "shoots through" to the wrong side.

Things that bit me

8 GB of memory is a hard wall. My test machine is an M1 MacBook Air with 8 GB. Object Capture, a Hunyuan3D diffusion model and a vision LLM simply don't fit in unified memory together. So before every 3D job the gateway unloads the Ollama model, and after the job it asks ComfyUI to free its memory. Unloading Ollama turned out to be one tiny request:

/** keep_alive: 0 evicts the model from unified memory immediately. */
async unload(): Promise<void> {
  await this.fetchImpl(`${this.cfg.url}/api/generate`, {
    method: 'POST',
    headers: {'content-type': 'application/json'},
    body: JSON.stringify({model: this.cfg.model, keep_alive: 0}),
    signal: AbortSignal.timeout(5000),
  }).catch(() => undefined);
}

While a build owns the memory, "Ask local AI" gets an HTTP 409 ("Your Mac is building a model") instead of a crash. Not glamorous, but it's the difference between a slow app and a Mac that locks up.

Shiny, thin and dark things don't scan. Photogrammetry needs texture to match points between photos. A laptop or a phone screen comes out broken no matter how careful you are. Matte, solid objects on a textured table work best. When Object Capture fails with "not enough features", the gateway retries once at high sensitivity before giving up.

Android AR went black, and it wasn't my bug. At one point, tapping View in AR on Android just showed a black screen and bounced back to the app. Google's Scene Viewer (it lives inside the Google app) was crashing, and it crashed with Google's own sample models too. Updating or clearing the cache of the Google app fixes it. Separately, Scene Viewer was dropping big textures, so the gateway now makes a lighter Android AR copy with textures capped at 2048 px, and colours always show.

AI models came out grey. Hunyuan3D-2 mini gives you shape only, no colour. The gateway now paints the mesh from your photo. For models built before that fix there's a repair script, pnpm --filter @threed/gateway fix-ar-textures, which colours the old grey ones.

Model files lie about what they are. People rename checkpoints, and ComfyUI fails with a cryptic Value not in list: ckpt_name. Now the gateway opens the .safetensors header and works out what's actually inside: Hunyuan3D mini, full v2, 2.1, a multi-view model, or a plain 2D model like SD 1.5 that can't make meshes at all. Then it picks the right settings by itself.

OTA is not a loophole. Apple and Google allow interpreted-code updates that don't change what the app does, so I use OTA for fixes and tweaks. When sharing needed new native modules (react-native-share and a file-system module), that meant a new shell release, 1.1. The Gallery 1.1 update is still safe to push to 1.0 shells: they just fall back to React Native's built-in Share for links.

What to expect on an 8 GB M1

These are measured on the base M1 MacBook Air, so pretty much any Apple Silicon Mac should do at least this well.

Mode	Build time	What you get	Tip
Photo set (Object Capture)	1–2 min for 24–37 photos	The best: true shape, real colours, small details like logos and buttons	The underside stays open unless you photograph it too
4 photos (Hunyuan3D multi-view)	About 35 min	Good all round; back and sides follow your real photos	Needs about 7 GB, so close every other app first
1 photo (Hunyuan3D mini)	About 6 min	Good from the front and sides; the back is a guess	Use a clear three-quarter view on a plain background

The 35 minutes for multi-view is the memory talking, not the GPU. On a 16 GB Mac it's much faster. More memory also buys more detail automatically: Macs with 16 GB or more get full-detail photo sets and 4K textures, without you changing a setting.

Try it yourself

You'll need:

An Apple Silicon Mac on macOS 14 or newer (an 8 GB M1 is enough)
An iPhone or Android phone on the same Wi-Fi
Node.js 22.13+ and pnpm 10+
Xcode 16+ for iOS, Android Studio for Android
Ollama for the capture tips, and ComfyUI only if you want the two AI modes

Then:

# 1. Install
pnpm install

# 2. Get the model for capture tips (keep the Ollama app running)
ollama pull qwen3.5:4b

# 3. Check that everything is set up (it tells you how to fix what's missing)
pnpm check-setup

# 4. Start the Mac gateway and the app's dev servers
pnpm start --real

# 5. In a second terminal, run the app
pnpm pods && pnpm ios      # iPhone simulator
pnpm android               # Android over USB

On a real iPhone, start the dev servers with pnpm start --real --device so the phone can reach them over Wi-Fi, and run from Xcode. Then on the phone go to Profile → Your Mac → Find Macs on this Wi-Fi and type the 6-digit code the gateway prints in its terminal.

One thing I'm glad I added: if you don't have any AI installed yet, plain pnpm start (without --real) runs a fake gateway that still produces real 3D files. You can click through the whole app before downloading a single model. And pnpm check-setup checks Node, Ollama, ComfyUI, Object Capture, the gateway and your Wi-Fi address, and tells you the exact fix for anything that's off.

The repo also comes with about 400 Jest tests, 39 C++ tests and Detox end-to-end tests, plus setup, architecture, technology and security guides in the docs/ folder.

That's it

I wanted to know how far you can get with the hardware that's already on your desk. Pretty far, it turns out. A phone that's fussy about its photos and a base-model Mac that's careful with its memory can turn a real object into something you can spin around, drop on your floor and send to a friend. None of it touches the cloud.

The code is all on GitHub: neel-turbo/app-image-to-3d-local-ai-rn. If you try it, I'd love to see what you scan. If something breaks, open an issue and tell me what you were pointing the camera at.

And if this was useful, a star on the repo helps other people find it.