DEV Community

Pavel Koryagin
Pavel Koryagin

Posted on

1

CrewAI’s output_pydantic is almost perfect. Here’s the missing part

output_pydantic is awesome and guarantees the output format to you. But its implementation is missing an important obvious thing.

ℹ️ This post is part of the “Crew AI Caveats” series, which I create to fill in the gaps left by official courses and to help you master CrewAI faster and easier.

If you use monitoring, you could make interesting discoveries, and one of mine is this.

When you use output_pydantic, the LLM does not see your field descriptions.

Code:

class FillContentPlanGapsOutput(BaseModel):
    date: str
    time: str
    channel: str = Field(..., description="Channel ID. Lowercase name, e.g. twitter, youtube, etc.")
    typeOfContent: str
Enter fullscreen mode Exit fullscreen mode

LLM prompt from monitoring:

Screen showing that the description part is lost

Here’s my topic where I approach the solutions.

And here’s a very early draft of a solution:

@task
def fill_content_plan_gaps(self) -> Task:
    field_info = "\nOutput fields:\n"
    for field_name, field_instance in FillContentPlanGapsOutput.model_fields.items():
        field_info += "- " + field_name + ((": " + field_instance.description) if field_instance.description is not None else "") + "\n"

    return Task(
        config=self.tasks_config['fill_content_plan_gaps'],
        expected_output=self.tasks_config['fill_content_plan_gaps']['expected_output'] + field_info,
        output_pydantic=FillContentPlanGapsOutput,
    )
Enter fullscreen mode Exit fullscreen mode

Logs:

This is the expect criteria for your final answer: List of content requests with channel, content type, and time slot.

Output fields:
- date
- time
- channel: Channel ID. Lowercase name, e.g. twitter, youtube, etc.
- typeOfContent

you MUST return the actual complete content as the final answer, not a summary.
Ensure your final answer contains only the content in the following format: {
  "date": str,
  "time": str,
  "channel": str,
  "typeOfContent": str
}
Enter fullscreen mode Exit fullscreen mode

When you read it, the problem might be already fixed in the framework. Otherwise, make a generic implementation of the prompt customizer that I haven’t made yet.

Stay tuned

In the next post: No revolutionary observations, just a heads-up about a discrepancy in official boilerplate versions.

Billboard image

Monitoring as code

With Checkly, you can use Playwright tests and Javascript to monitor end-to-end scenarios in your NextJS, Astro, Remix, or other application.

Get started now!

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay