AI API cost is expensive.
The problem really hits when you deal with long context tasks. I'm bootstrapping, so cost is a real concern. That's why I've tried different ways to deal with this specific problem.
Of all the strategies I tried, there's one that I keep using in every app I build: Task Decomposition.
I hope this post will save you some money to make your AI SaaS idea viable.
What is Task Decomposition?
The answer: Breaking down a complex task into simpler steps.
This strategy lowers the level of intelligence required for the tasks. Thus, it allows you to replace big, costly models with smaller, cheaper ones. Like, you need a senior to architect the whole app, but you only need an intern to write a component.
Let's take a look at an example:
Example: The Landing Page Generator
Let's say you're building an AI landing page copywriter. Instead of expecting a long response from the model, use the PASTOR framework to break the task down:
Problem: Define the user's pain point.
Amplify: Highlight the consequences of that pain.
Solution: Introduce your... well, solution.
Transformation: Pain a picture of a great future.
Offer: The stuff you sell plus bonuses.
Response: Call to action.
Now, you can have a function that loops through different system prompts for each part. You can even break this down further if you like.
Why It Works
There are 4 huge advantages with task decomposition:
Clear and focused prompt. Instead of trying to teach the AI everything about marketing in one prompt, you just need to teach it how to write one section. The AI will make less mistakes because it's less confused.
More room for examples. With a focused prompt, you can give the AI more nuanced examples without confusing it. When your AI can see a clear pattern, it's more reliable.
Easier evaluation. It's less confusing to judge the quality of a single section than an entire page. This makes iteration faster and cheaper.
More direct and less costly validation. This marketing examples don't quite show it. But when you have tool APIs with complex types, you don't want to retry an API call just because the AI gets one type wrong.
If you noticed, I used the word "confuse" several times in the first 3 points.
Prompt engineering IS communication. Therefore, clarity is key. Often times, the AI doesn't meet your expectations because it doesn't understand what you want.
Everyone tells you to be "clear and concise" with your prompts. Now you know how to do that with task decomposition.
What task decomposition unlocks:
The ability to steer the AI's thought process to your needs.
With task decomposition, not only can you improve output quality, but you can also ensure that it stays high quality. Because you take more control in the generation process. Your prompts, along with the examples, will handle most of the thought process for the AI.
The result is a better and more reliable system.
Conclusion
If you think about it, this is not that different from how humans work.
We are not good at dealing with big problems at once. We get overwhelmed and avoid them altogether. AI is built to mimic human's intelligence so they are not that different.
A lot of what we learned about humans can be adapted to AI.
Anyways, thanks for reading. I hope you found something useful.
Let me know what you want to learn more about AI app development. I will write an article about it (if it's something I know).
Until next time, happy building ;))
-Ryan
Top comments (0)