Workflow vs agent

Workflow vs Agent: Deterministic Orchestration & Dynamic Tool Selection

---

Quick Diagnosis

Situation	Recommended approach	Why it fits
All steps known ahead of time (e.g., fetch data → prompt → parse → store)	Workflow	Immutable execution, easy testing, predictable token usage
Path depends on content or user intent (e.g., “answer any compliance question”)	Agent	Chooses tools at runtime, handles unknown sub‑tasks
Regulatory audit required (immutable logs, version‑controlled prompts)	Workflow	Deterministic runs produce repeatable audit trails
Live numeric value needed (credit score, market rate)	Tool call from workflow (authoritative API)	Guarantees freshness; does not rely on stale retrieval
Batch processing with tight cost budget	Workflow	One LLM call per step, predictable token usage
Real‑time, ad‑hoc question answering	Agent	Dynamic reasoning, on‑the‑fly tool selection
Mix of known skeleton + occasional live‑value request	Hybrid (workflow + runtime API/agent)	Keeps core deterministic, adds flexibility only where required

---

Decision Flow

flowchart TD
    A["Is the full execution path known ahead of time?"]
    B["Can every required tool be enumerated?"]
    C["Use a deterministic workflow"]
    D["Is the request likely to need unknown sub‑tasks?"]
    E["Add an agent layer for dynamic tool selection"]
    F["Can live values be fetched via an API?"]
    G["Expose API call from the workflow"]
    H["Leave decision to the agent at runtime"]

    A -->|Yes| B
    B -->|Yes| C
    B -->|No| D
    D -->|Yes| E
    D -->|No| F
    F -->|Yes| G
    F -->|No| H

---

Architectural Responsibilities

Layer	Core duty	Typical implementation
Deterministic Orchestrator	Define the fixed sequence, validate inputs, enforce business rules, emit immutable logs	Azure Logic Apps, Temporal, Airflow, hand‑rolled Python DAG
Agent Core	Detect intent, decide which tool or retrieval source to invoke, manage ReAct‑style loops, handle fall‑backs	LLM prompting with function‑calling schema, structured reasoning templates
Tool / Retrieval Layer	Provide static knowledge (document store) or fresh data (authoritative API)	Azure Cognitive Search, public REST endpoints, token‑protected credit‑bureau API
Governance & Auditing	Record every step, version‑control prompts, certify compliance checks	Write‑once log storage, signed manifests, CI linting of workflow definitions
Cost & Latency Monitor	Track token usage, external‑API count, per‑step latency	OpenTelemetry exporters, custom metric dashboards

---

Python Decision Helper (Teaching‑Only)

# decision_helper.py – pure stdlib, no external packages
import json
from pathlib import Path
from typing import Literal, TypedDict

class DecisionInput(TypedDict):
    """Minimal data required to pick workflow vs. agent."""
    steps_known: bool          # True if the entire sequence can be enumerated
    tools_enumerated: bool     # True if every tool can be listed up‑front
    live_value_needed: bool    # True if an authoritative API must be called
    dynamic_subtasks: bool     # True if request may spawn unknown sub‑tasks

def choose_strategy(data: DecisionInput) -> Literal["workflow", "agent", "hybrid"]:
    """
    Return the recommended execution style.

    * `workflow` – fully deterministic pipeline
    * `agent`    – pure agentic control
    * `hybrid`   – deterministic skeleton with optional runtime API or agent
    """
    # Basic validation – fail fast if the contract is broken
    assert isinstance(data["steps_known"], bool), "steps_known must be bool"
    assert isinstance(data["tools_enumerated"], bool), "tools_enumerated must be bool"
    assert isinstance(data["live_value_needed"], bool), "live_value_needed must be bool"
    assert isinstance(data["dynamic_subtasks"], bool), "dynamic_subtasks must be bool"

    if data["steps_known"] and data["tools_enumerated"]:
        # All actions are known → deterministic flow
        return "workflow"
    if data["dynamic_subtasks"]:
        # Content can generate new branches → need agentic reasoning
        return "agent"
    # Mixed case – known skeleton but may need live data or optional tool selection
    return "hybrid"

# -------------------------------------------------
# Example usage (for teaching only – not part of production)
if __name__ == "__main__":
    example = {
        "steps_known": True,
        "tools_enumerated": True,
        "live_value_needed": False,
        "dynamic_subtasks": False,
    }
    print(json.dumps({"strategy": choose_strategy(example)}))

Teaching‑Only Note – Production systems should encode this decision in a CI‑validated configuration file rather than an ad‑hoc script.

---

Architectural Decision Records (ADRs)

ADR 1 – Introducing an Agent for Policy Queries

Context

A fintech onboarding pipeline executes three fixed steps (identity → credit → welcome email). Occasionally, a regulator submits a policy_query that does not fit the fixed path.

Decision

Add an optional agent that is triggered only when the incoming payload contains a policy_query field. The agent will invoke a policy‑search tool, retrieve the relevant text, and attach it to the final output.

Consequences

• The core majority of requests remain in a deterministic workflow, preserving auditability.
• New logs capture the agent’s chosen tool, LLM prompt, and tool response, enabling traceability.
• Latency impact is limited to the subset of requests that require the agent.

Acceptance Criteria

1. Agent‑enabled paths generate a log entry that records the selected tool, the exact prompt sent to the LLM, and the tool’s raw response.
2. All agent‑generated outputs are stored with a cryptographic hash for integrity verification.
3. CI pipeline includes unit tests for both “with query” and “without query” scenarios, and validates that the workflow remains deterministic when the agent is not invoked.

---

ADR 2 – Handling Live Credit‑Limit Values

Context

The onboarding flow must embed the applicant’s current credit limit, which changes frequently. A stale document store could return an outdated figure.

Decision

Expose a tool in the workflow that calls the credit‑bureau’s authoritative REST endpoint. The workflow pauses, fetches the live limit, and continues deterministically. No caching of the numeric value is permitted.

Consequences

• Guarantees freshness of a critical compliance datum.
• Introduces a single external API call per onboarding, which is observable in cost‑monitoring dashboards.
• Enables deterministic testing by mocking the API response within a test harness.

Acceptance Criteria

1. Integration tests with a mock endpoint produce the same structural response shape as the real service.
2. Production logs record request‑ID, endpoint URL, response timestamp, and the returned numeric value.
3. Any failure to reach the API aborts the workflow with a defined error code and escalates to a human reviewer.

---

Evaluation & Maintenance Checklist

• Functional correctness

  • ✅ Deterministic steps produce identical output for identical inputs (regression suite).
  • ✅ Agent‑enabled paths have mocked‑LLM tests covering tool selection logic.

• Governance

  • ✅ Prompt templates are version‑controlled, signed, and reviewed.
  • ✅ Business rules (e.g., “identity verification before credit check”) live in code, not in LLM prompts.

• Cost visibility

  • ✅ Token usage per step logged to a cost‑center dashboard.
  • ✅ External‑API call count tracked; alerts trigger on abnormal spikes.

• Latency observability

  • ✅ End‑to‑end latency recorded; thresholds defined per SLA.
  • ✅ Separate latency metric for agent loops to bound their impact.

• Knowledge freshness

  • ✅ Retrieval index (Azure Cognitive Search) refreshed on a schedule aligned with policy updates.
  • ✅ Live‑value endpoints re‑validated after any vendor version change.

• Model updates

  • ✅ Prompting is the first line of defence; LoRA fine‑tuning is applied only after a documented gap analysis shows prompting insufficient.
  • ✅ When LoRA is applied, a full regression suite on pre‑existing deterministic tests is run; behavior preservation is not assumed.

• Security & Auditing

  • ✅ Secrets stored in a vault; never hard‑coded.
  • ✅ Execution logs are immutable, encrypted, and retained per compliance window.

---

References

• Microsoft Retrieval‑Augmented Generation concepts – https://learn.microsoft.com/en-us/azure/foundry/concepts/retrieval-augmented-generation
• “LoRA: Low‑Rank Adaptation of Large Language Models” – https://arxiv.org/abs/2106.09685