DEV Community

Cover image for Optimas + SuperOptiX: Global‑Reward Optimization for DSPy, CrewAI, AutoGen, and OpenAI Agents SDK
Shashi Jagtap
Shashi Jagtap

Posted on

Optimas + SuperOptiX: Global‑Reward Optimization for DSPy, CrewAI, AutoGen, and OpenAI Agents SDK

Optimization has been central to SuperOptiX from day one—whether it's prompts, weights, parameters, or compute. It began with DSPy-style programmatic prompt engineering and teleprompting as it was the only framework doing prompt optimization. It was surprising that other frameworks couldn't figure out ways to optimize prompts like DSPy, but now we have a solution. Today, we're bringing Optimas into the SuperOptiX ecosystem so you can apply globally aligned local rewards across multiple frameworks: OpenAI Agent SDK, CrewAI, AutoGen, and DSPy. You can check out the Optimas and SuperOptiX integration here.

Optimizing a single prompt isn't enough for modern "compound" AI systems. Real systems chain LLMs, tools, and traditional ML into multi‑step workflows, and the right unit of optimization is the whole pipeline. Optimas introduces globally aligned local rewards that make per‑component improvements reliably lift end‑to‑end performance. SuperOptiX now brings Optimas to your existing agent stacks—OpenAI Agent SDK, CrewAI, AutoGen, and DSPy—behind one practical CLI, so you can go from baseline to optimized without changing frameworks.

What Optimas is (and why it matters)

Optimas is a unified optimization framework for compound AI systems. It:

  • Learns an LRF per component that remains globally aligned, so local updates are safe and beneficial to the whole system. This enables efficient optimization without always running the entire pipeline for every candidate. See the method and guarantees: arXiv paper.
  • Supports heterogeneous configuration types:
    • Prompts and textual instructions via metric‑guided search
    • Hyperparameters and discrete choices like top‑k, tool/model selection, routing
    • Model parameters where supported (e.g., RL with PPO)
  • Works across frameworks (OpenAI Agent SDK, CrewAI, AutoGen, DSPy) through target adapters.
  • Compound‑system optimization: works across multiple components and tools, not just single prompts.
  • Global alignment via local rewards: optimize each component while ensuring the whole system converges.
  • Multiple optimizers:
    • OPRO: single‑iteration prompting optimization
    • MIPRO: multi‑iteration prompting
    • COPRO: cooperative optimization

What this unlocks

  • Optimize prompts, hyperparameters, model parameters, and model routers across compound AI systems.
  • Run OPRO, MIPRO, and COPRO optimization loops using a single CLI workflow.
  • Keep your preferred agent stack (DSPy, CrewAI, AutoGen, OpenAI SDK) and get consistent optimization behavior.

Why this is impactful

  • Optimas learns a local reward function (LRF) for each component that stays aligned with a global objective. Independently maximizing a component's local reward still increases overall system quality. This is data‑efficient and avoids excessive full system runs.
  • It supports heterogeneous updates across prompts, hyperparameters, model selection/routing, and (where applicable) model parameters via RL, achieving consistent gains across diverse pipelines.
  • The Optimas paper reports an average relative improvement of 11.92% across five complex compound systems, with theoretical guarantees and strong empirical results. See:

Where Optimas fits in SuperOptiX

SuperOptiX as the name suggests, is built around optimization; Optimas plugs directly into its lifecycle:

  1. Compile your agent into a runnable pipeline for a specific target.
  2. Evaluate to get a baseline.
  3. Optimize with Optimas (OPRO/MIPRO/COPRO) using the same CLI across targets.
  4. Run the optimized agent.

This extends optimization beyond prompts to hyperparameters, model selection, routing, and parameters where supported.

  • Focus‑aligned: SuperOptiX is built around optimization; Optimas operationalizes optimization across agents and tools.
  • Beyond prompts: SuperOptiX + Optimas can optimize prompts, hyperparameters, model parameters, and even model routers, aligning with complex production workflows.
  • One CLI to rule them all: Use a consistent sequence—compile, evaluate, optimize, run—across all targets.

Optimas vs. DSPy (complementary)

  • DSPy is a framework for composing LLM pipelines and programmatic teleprompting.
  • Optimas is an optimization engine that runs globally aligned local updates across multi‑component systems, regardless of the underlying framework (including DSPy).
  • In practice: build in your preferred stack; use Optimas to optimize it end‑to‑end. If your system is in DSPy, try --optimizer mipro for deeper prompt refinement; OPRO and COPRO are also available.
  • In short: DSPy builds and teleprompts pipelines; Optimas runs global optimization across pipelines and frameworks. They complement each other.

How SuperOptiX uses Optimas

SuperOptiX ships templates and playbooks for each target:

  • Targets: optimas-openai, optimas-crewai, optimas-autogen, optimas-dspy
  • Agents: optimas_openai, optimas_crewai, optimas_autogen, optimas_dspy
  • Optimizers: --optimizer opro|mipro|copro
  • Engine: --engine optimas to engage Optimas in the pipeline

Use a consistent CLI surface:

  • Compile with --target
  • Evaluate with --engine optimas
  • Optimize with --engine optimas and --optimizer
  • Run with the chosen --target and --engine when applicable

Installation 

# Core Optimas integration
pip install "superoptix[optimas]"

# Target-specific extras (choose as needed)
pip install "superoptix[optimas,optimas-openai]"
pip install "superoptix[optimas,optimas-crewai]"
pip install "superoptix[optimas,optimas-autogen]"
pip install "superoptix[optimas,optimas-dspy]"

# CrewAI note: resolve dependency pin by installing CrewAI without deps, then json-repair
pip install crewai==0.157.0 --no-deps
pip install "json-repair>=0.30.0"
Enter fullscreen mode Exit fullscreen mode

CrewAI has a hard dependency on json-repair 0.26.0 while DSPy 3.0.0 needs it 0.30.0.

Quick start across all targets

Project setup and demo agents:

# Initialize a new project
super init test_optimas
cd test_optimas

# Pull demo playbooks
super agent pull optimas_openai      # OpenAI SDK (recommended)
super agent pull optimas_crewai      # CrewAI
super agent pull optimas_autogen     # AutoGen
super agent pull optimas_dspy        # DSPy
Enter fullscreen mode Exit fullscreen mode

The unified lifecycle (works the same for all targets)

The sequence is consistent: compile → evaluate → optimize → run. Below are target‑specific recipes with practical knobs.

OpenAI Agent SDK (recommended for production)

This is the fastest, most stable path to results with minimal dependency friction.

# Compile → Evaluate
super agent compile optimas_openai --target optimas-openai
super agent evaluate optimas_openai --engine optimas --target optimas-openai

# Optimize (OPRO shown; adjust search breadth, temperature, and timeout inline)
SUPEROPTIX_OPRO_NUM_CANDIDATES=3 \
SUPEROPTIX_OPRO_MAX_WORKERS=3 \
SUPEROPTIX_OPRO_TEMPERATURE=0.8 \
SUPEROPTIX_OPRO_COMPILE_TIMEOUT=120 \
super agent optimize optimas_openai --engine optimas --target optimas-openai --optimizer opro

# Run
super agent run optimas_openai --engine optimas --target optimas-openai \
  --goal "Write a Python function to add two numbers"
Enter fullscreen mode Exit fullscreen mode

Why these knobs matter: candidates broaden the search; temperature encourages variation; compile timeout helps for larger models; workers control concurrency for faster iterations.

CrewAI (great for role‑based multi‑agent workflows)

If you orchestrate crews of agents, Optimas can optimize prompts and task hyperparameters in the same loop.

# Compile → Evaluate
super agent compile optimas_crewai --target optimas-crewai
super agent evaluate optimas_crewai --engine optimas --target optimas-crewai

# Optimize (tune LiteLLM behavior; keep workers modest)
LITELLM_TIMEOUT=60 \
LITELLM_MAX_RETRIES=3 \
SUPEROPTIX_OPRO_MAX_WORKERS=3 \
super agent optimize optimas_crewai --engine optimas --target optimas-crewai --optimizer opro

# Run
super agent run optimas_crewai --engine optimas --target optimas-crewai \
  --goal "Write a Python function to calculate factorial"
Enter fullscreen mode Exit fullscreen mode

Tip: retries and timeouts harden long‑running optimization loops against transient provider hiccups. For model client behavior, see LiteLLM.

AutoGen (strong for conversational/multi‑agent; optimization can be slower)

AutoGen excels at complex, multi‑turn agent interactions; give the optimizer more headroom.

# Compile → Evaluate
super agent compile optimas_autogen --target optimas-autogen
super agent evaluate optimas_autogen --engine optimas --target optimas-autogen

# Optimize (increase compile timeout for heavier pipelines)
LITELLM_TIMEOUT=60 \
LITELLM_MAX_RETRIES=3 \
SUPEROPTIX_OPRO_MAX_WORKERS=3 \
SUPEROPTIX_OPRO_COMPILE_TIMEOUT=180 \
super agent optimize optimas_autogen --engine optimas --target optimas-autogen --optimizer opro

# Run
super agent run optimas_autogen --engine optimas --target optimas-autogen \
  --goal "Write a Python function to reverse a string"
Enter fullscreen mode Exit fullscreen mode

Why timeouts help: larger or tool‑heavy pipelines can exceed quick compile windows; a higher timeout reduces spurious failures during candidate generation.

DSPy (fully supported; tune concurrency if needed)

DSPy is a natural fit if your system is authored in DSPy. Start with OPRO for reliability; try MIPRO for deeper prompt improvements.

# Compile → Evaluate
super agent compile optimas_dspy --target optimas-dspy
super agent evaluate optimas_dspy --engine optimas --target optimas-dspy

# Optimize (start with OPRO; adjust temperature/workers)
SUPEROPTIX_OPRO_MAX_WORKERS=3 \
SUPEROPTIX_OPRO_TEMPERATURE=0.8 \
super agent optimize optimas_dspy --engine optimas --target optimas-dspy --optimizer opro

# Run
super agent run optimas_dspy --engine optimas --target optimas-dspy \
  --goal "Write a Python function to calculate fibonacci numbers"
Enter fullscreen mode Exit fullscreen mode

If you see any concurrency/threading issues in your model client stack, set SUPEROPTIX_OPRO_MAX_WORKERS=1 during optimization to serialize candidate evaluations.

Choosing an optimizer (and when to use which)

  • OPRO (Optimization by Prompting): default choice; single‑iteration, dependable progress. super agent optimize <agent> --engine optimas --target <target> --optimizer opro
  • MIPRO (Multi‑Iteration Prompting): deeper prompt refinement across rounds; especially good for DSPy prompt programs. super agent optimize <agent> --engine optimas --target <target> --optimizer mipro
  • COPRO (Cooperative Prompting): coordinate improvements when several components benefit from joint search. super agent optimize <agent> --engine optimas --target <target> --optimizer copro

Practical tuning (inline, reproducible)

Keep configuration inline to make runs easy to share and reproduce:

# OPRO baseline
SUPEROPTIX_OPRO_NUM_CANDIDATES=3 \
SUPEROPTIX_OPRO_MAX_WORKERS=3 \
SUPEROPTIX_OPRO_TEMPERATURE=0.7 \
SUPEROPTIX_OPRO_COMPILE_TIMEOUT=60 \
super agent optimize <agent> --engine optimas --target <target> --optimizer opro
Enter fullscreen mode Exit fullscreen mode 
# Quality pass (longer search)
SUPEROPTIX_OPRO_TEMPERATURE=0.9 \
SUPEROPTIX_OPRO_COMPILE_TIMEOUT=300 \
super agent optimize <agent> --engine optimas --target <target> --optimizer opro
Enter fullscreen mode Exit fullscreen mode 
# Harden model client behavior (affects DSPy and CrewAI)
LITELLM_TIMEOUT=60 \
LITELLM_MAX_RETRIES=3 \
LITELLM_CACHE_ENABLED=false \
LITELLM_LOG_LEVEL=ERROR \
super agent optimize <agent> --engine optimas --target <target>

# If you see any concurrency/threading issues
SUPEROPTIX_OPRO_MAX_WORKERS=1 \
super agent optimize <agent> --engine optimas --target <target>
Enter fullscreen mode Exit fullscreen mode

When to pick which target

  • OpenAI Agent SDK: most stable and fast—use this for production optimization and quick benchmarks.
  • CrewAI: best for role‑based multi‑agent workflows; follow the install note for json-repair.
  • AutoGen: designed for multi‑turn and conversational agents; increase compile timeouts during optimization.
  • DSPy: ideal if your pipeline is authored in DSPy; start with OPRO, then explore MIPRO for deeper improvements.

Why this approach scales

  • Global‑local alignment ensures local progress translates into higher system performance, avoiding the "optimize one, regress the rest" trap common in compound systems. See theory and results: arXiv.
  • Data efficiency comes from optimizing locally with learned LRFs instead of always running the full system per candidate. The paper shows consistent gains with fewer full passes.
  • Heterogeneous optimization unifies prompt search, discrete hyperparameter/model selection, routing, and model parameter updates (e.g., PPO) into one loop.

What's next

  • Add a GEPA adapter so richer program synthesis traces and execution feedback can be optimized end‑to‑end inside SuperOptiX.
  • Continue collaborating with the Optimas team to upstream adapters and keep parity across targets and optimizers.
  • Expand optimization scope from prompts/hyperparameters into richer parameter and routing policies, guided by the LRF methodology.

Demo

Watch the video

Conclusion

SuperOptiX is not limited to prompt optimization; Optimas unlocked power to go beyond prompt optimization and support existing frameworks. This is an early stage for Optimas and we will evolve together as framework support gets stronger.

Top comments (0)