5 tools we wish were on the Awesome AI Tools list

Mounir Mouawad — Fri, 15 Aug 2025 12:15:07 +0000

We’re big fans of the Awesome AI tools list and we all use it to discover new AI tools over at Portia AI. My latest and favourite find is Merlin: A Chrome extension that allows me to ask “how to” questions on any app rather than flipping over to ChatGPT or Claude to ask.

Here are five tools we use a lot and wish were on the Awesome AI Tools list:

Textual – We love spicing up our terminal interface for using the Portia SDK and even non-technical customers love it when I run demos from the terminal now. We all have our favourite terminal flavour of it – I made mine with Atari retro vibes 🕹️holler if you’re using Portia and want the code for it!
Mistral OCR – We think it’s the best balance of cost, speed and performance for OCR on the market right now. We also admittedly have soft spot for our neighbours across the English Channel over in La France 🥐.
Visily – Figma for non-designers, it’s my go-to when brainstorming early UX mocks with front-end engineers and UX designers. I especially love the ability to turn any screenshot into a wireframe because I can bring inspirations to life with some tweaks super quickly.
Podcastfy – I can’t say for sure why they skipped the “i” in their name but we love that they are an open source and equally powerful alternative to NotebookLM. One of our engineers built a bite-sized AI news podcast that I listen to during my commute daily. You can recreate it here using Portia SDK or get the daily podcast on our Discord server’s #ai-news channel.
OpenRouter - We love OpenRouter because it allows you to easily try out new models and load balance between models. We actually got an open source contribution for this one recently, so we should be supporting it ❤️

Code vs LLM in a simple planning poker agent example

Mounir Mouawad — Wed, 09 Jul 2025 17:00:33 +0000

If you're building AI agents, chances are you often had to consider how much logic you want to handle through the LLM versus through traditional code. I wanted to share my experience with it this morning as a conversation starter and get your thoughts!

What I wanted the agent to do

I normally spend a ton of time gathering feedback from our users. In a previous life I would put those insights into tickets in Linear and spend a ton of mental cycles trying to size the return on effort to inform our prioritisation. In this bold new world of AI, I figured I would instead write up a planning poker agent to help me do t-shirt sizing of some of those tickets in Linear. Built on the Portia SDK, the agent would:

Fetch relevant linear tickets using the remote MCP server for Linear, which is one of 1000s of tools we have with built-in auth.
Simulate sizing estimates from multiple developer personas and get to a consensus for each ticket's effort sizing. Here I wanted to create a ticket estimator tool using a subclass of our LLM tool that would return estimates as structured outputs. The tool would take a context.md file where I keep a summary of the architecture and core abstractions that make up the Portia SDK so it can help the LLM with effort sizing.

As it turns out, I had asked one of our devs (we'll call him Ethan) to do this and forgotten! So we both wrote this thing up at the same time except...I relied quite heavily on the LLM to handle the task while he relied way more heavily on code. Let's unpack how our approches compared.

How each of us built it

Full code in our agent examples repo here.

🧠 LLM-heavy: I relied on a robust prompt and the Portia planning agent to figure out the entire set of steps that need to be taken, that is fetch and filter tickets from Linear, then get estimates for ticket sizes from each developer persona and average them out. Essentially I relied on the LLM itself to 1) index and aggregate the sizing estimates by Linear ticket id and persona, 2) figure out how many tool call iterations (a.k.a. "unrolling") to make to handle all ticket id and persona combinations. Here's the code snippet where the magic happens:

# Get tickets from Linear and estimate the size of the tickets
project = "Async Portia"
query = f"""Get the tickets i'm working on from Linear with a limit of 3 on the tool call. Then filter specifically for those regarding the {project} project.
    For each combination of the tickets above and the following personas, estimate the size of the ticket.
    {personas}

    Return the estimates in a list of PlanningPokerEstimate objects, with estimate sizes averaged across the personas for each ticket.
    """
estimates = portia.run(
    query=query,
    structured_output_schema=PlanningPokerEstimateList,
).outputs.final_output.value.estimates

🧑🏻‍💻 Code-heavy: Ethan on the other hand figured that we don't really need to rely on the LLM, neither for planning nor for indexing / aggregating / iterating on estimates. Instead he used Portia's declarative PlanBuilder interface to enumerate the steps and tool calls needed. He fetched the tickets using a first Portia plan run into LinearTicket objects using structured outputs. To generate sizing estimates, he then iterated with conventional code over each developer persona and over each ticket element in the list returned from the previous plan run. Each iteration called the ticket estimator tool in a single step Portia plan run. Here's a code snippet containing both the ticket fetching plan run and the ticket sizing iterations:

# Fetch Linear tickets
project = "Async SDK"
query = f"Get the tickets i'm working on from linear regarding the {project} project"
plan = PlanBuilder(
    query, structured_output_schema=LinearTicketList
).step(
    query + " and only call the tool with a limit of 3", tool_id="portia:mcp:mcp.linear.app:list_my_issues"
).step(
    f"Filter the tickets to only include specifically the ones related to {project}", tool_id="llm_tool"
).build()
plan_run = portia.run_plan(plan)
tickets = plan_run.outputs.final_output.value.tickets

# Iterate over tickets and persona to generate estimates
for ticket in tickets:
    estimates = []
    estimate_plan = PlanBuilder(
        "estimate the size of the ticket", structured_output_schema=PlanningPokerEstimate
    ).step(f"Estimate the size of the ticket: {ticket.title}\n\n{ticket.description}", tool_id="ticket_estimator_tool").build()
    for persona in personas:
        context = f"""
        {persona}
        {tool_context}
        """
        estimate_tool.tool_context = context
        portia.tool_registry.with_tool(estimate_tool, overwrite=True)
        estimate = portia.run_plan(estimate_plan)
        if estimate.state == PlanRunState.COMPLETE:
            estimates.append(estimate.outputs.final_output.value)

What we learned

Let's compare both approaches side by side and draw some conclusions. I hooked up Langsmith to Portia for observability so I could obtain the metrics shown below.

	LLM-heavy	Code-heavy
Effort	Lowest	Highest
Total tokens	70k	30k
Cost	$0.12	$0.06
Latency [P99]	28.95s	9.70s

So what conclusions can we draw from this exercise?
💡 Reliability: You can trust your Portia agents to figure out the right sequence of steps and to unroll (iterate on) the tool calls correctly so that definitely simplifies development, kinda like a form of vibe coding...but much like vibe coding it does take a bit of 'LLM-whispering' (a.k.a. prompt engineering) and using the right underlying model. For plan runs with heavy iteration expectations in particular, you will need robust eval sets in place to keep tabs on reliability lest you aim for a Mona Lisa and end up with a Picasso.
👣 Traceability: Relying on the LLM to handle planning and execution to the extent I did does make tracing particularly easy. One single PlanRunState instance in the Portia dashboard showed me the entirety of the work done by the underlying subagents. This also makes revisiting the output of the plan run easier of course. Ethan on the other hand ended up with numerous plan runs, which makes auditing and / or debugging harder.
💸 Cost: As you'd expect the LLM-heavy method is slower and costlier. Ultimately we're still processing the same amount of context presumably (same number of tickets and estimations) but the overhead of passing along a growing context window across all execution agents during the plan run means that the LLM-heavy method is inevitably slower and costlier. You're also opening yourself up to the stochasticity of LLMs when code could do the trick.

A parting thought

One aspect I don't consider in the comparison above is autonomy. Because the task is neatly scoped in this example (planning poker agent = fetch and filter tickets + estimate per persona + summarise consensus) you can make the argument that at production scale one should restrict LLM usage only to the tasks that traditional code can't handle as easily (e.g. natural language processing). BUT where inputs from the environment change or the scope of the task is fluid, the LLM-heavy approach truly thrives. I'll try and tease that more obviously in a subsequent post.
👉🏼 If you're interested please shout in the comments down below!

About Portia

Portia AI is an open-source framework for building predictable, stateful, authenticated agentic workflows.

We allow developers to have as much or as little oversight as they’d like over their multi-agent deployments and we are obsessively focused on production readiness.

We invite you to play around with our SDK, break things, and tell us how you're getting on in Discord