For serving LLMs with respect to cost, privacy, and compliance controls, I've been playing with ideas for diverting relevant prompts to specialized or private model(s) and handling the rest with a cost-controlled frontier model.
Read the blog post or skip directly to the demo recipe in GitHub.
Top comments (0)