I'm developing a system and could use some expert insights on its architecture and implementation.
Here's a brief overview of the current flow:
- User fills out an intake questionnaire with X questions
- System references datasets/examples from existing system folders
- Multiple parallel flows run with Claude API to generate texts based on guidelines and relevant intake responses
Now, I'm grappling with some key issues:
- Overall Architecture
- Should I use Agentic AI frameworks? If so, which ones?
- Or is there a more efficient approach for a system like this?
- Data Storage and Management
- How to store intake responses for efficient AI flow access?
- Is RAG (Retrieval-Augmented Generation) advisable for storage and retrieval? If not, what are the alternatives?
- Existing Dataset Integration
- What's the most efficient way to incorporate examples from existing folders into the flow?
- Better to train a small model on the dataset or integrate it directly?
- Parallel Process Optimization
- How to efficiently manage and synchronize the parallel flows?
Important notes:
- Output (generated texts) and number of intake questions remain constant
- It's a complex flow split into several sub-flows, not a single long prompt
Top comments (0)