We used to run chat and itinerary planning as two services. Two session stores, two deploy paths, and the same tool list for everyone. Membership did almost nothing at the chat layer.
This post is how we folded planning into the chat process, kept the old stream endpoint stable, and only opened booking / itinerary tools for members.
Why merge
Chat answered questions. Planning lived elsewhere: another process, another port, another session id. The mini program had to know which backend to call. When something broke, you debugged two stacks.
Worse: free and paid users saw the same tools. "Book a table" and "build a day plan" were either on for everyone or nowhere near the model.
Goals:
- One SSE stream for conversation
- Tools trimmed by a
profile(standard|member) - Planning as a library in the same process, not a second microservice
- Old clients keep working on v1
Keep v1, put new behavior on v2
Do not change the live stream contract for clients already in production.
| Endpoint | Who uses it | Behavior |
|---|---|---|
POST /api/chat/stream |
Old clients | Same as before |
POST /api/chat-v2/stream |
New clients | Accepts profile; members get extra tools |
POST /api/plan |
Itinerary screen | Sync plan; same run_plan as the chat tool |
Auth stays in the BFF in front of the mini program. Chat only trusts profile. Optionally require an internal header (X-Internal-Token) on v2 and /api/plan so random traffic cannot pick member itself.
Tools by tier
Standard: wiki / FAQ lookup, transit, souvenirs, restaurant suggestions, small talk.
Member also gets: table booking, generate / refine itinerary.
A question like "is two days enough?" stays on wiki-style retrieval. It is not a planning job.
In the mini program, something like mode=pro plus a valid membership becomes profile=member in the BFF before the stream call.
Bringing planning into the same process
Move the planning package next to the chat app. Put model artifacts (pickles, embeddings, local embedding weights) under a data directory the process can read.
Refine only when the same session_id already has a stored plan. Otherwise treat the call as a fresh generate.
Chat still keys sessions with session_id. The itinerary page still uses hash_id. Those two ids do not map to each other yet. Say that out loud in the PR; people will assume they do.
Mini program changes
- Send
modewith each message; switching to Pro checks membership first - BFF proxies v2 and attaches
profile - Itinerary page calls
http://127.0.0.1:{chat_port}/api/planinstead of the old planning port
Checks we ran
- Standard profile: cannot book, cannot build an itinerary
- Member: can book; can generate; refine needs a prior generate on the same session
- Known gaps logged as examples: empty session asking to "change" a plan, wanting a brand-new plan on the same session, talk ids that still differ between chat and the itinerary page
Deploy notes
- Upload scripts skip heavy plan assets; copy them on the server from the old tree
- Order on the box: ship code → rsync artifacts →
poetry installinto the project venv →pm2 reload - Leave the old planning process up until you are sure nothing still hits it
- v1 chat should not move during this cutover
Watch the conda vs poetry mix. We pin installs to the project .venv so reload does not pick a random global env.
Still open
- "Change my plan" only looks for a stored plan; it does not parse whether the user meant edit vs new
- Chat session and itinerary-page session stay disconnected
- Our refine tests only assert the tool ran, not that attractions actually swapped
Diagrams worth drawing
- Before: mini program → BFF → chat or planning (two paths)
- After: conversation on v2 +
profile; itinerary page on same-process/api/plan - Member vs standard tool tables
What stuck
Merge the capability into one process, freeze the old stream URL, and put membership on the tool list rather than a second chat service. Planning data can still be heavy; ship the code first and copy artifacts where they already live.
Top comments (0)