DEV Community

Sridhar S
Sridhar S

Posted on

My AI Agent Was Escalating Every Contract. One Decision Layer Fixed It 📑🤖📑🤖

#hermesagentchallenge #python #agents #ai

Hermes Agent Challenge Submission: Build With Hermes Agent

This is a submission for the Hermes Agent Challenge: Build With Hermes Agent

My Hermes Agent Couldn’t Decide Which Contracts Needed Legal Review. One Planning Layer Fixed It. 📑🤖

What I Built

While experimenting with enterprise AI agents, I noticed a common problem:

Contract reviews are painfully manual.

Vendor agreements, NDAs, MSAs, and SOWs often require legal teams to manually inspect:

  • missing clauses
  • unclear liabilities
  • compliance gaps
  • termination conditions
  • SLA definitions

I wanted to see:

Can an AI agent intelligently decide what to review and when to escalate?

So I built an Enterprise Contract Intelligence Agent powered by Hermes Agent.

Instead of simply extracting text from contracts, the agent plans tasks, invokes tools, reasons through risks, and decides whether a contract actually requires legal review.

The interesting part?

My first version failed badly.

Hermes Agent was escalating almost every contract.

NDAs.

Vendor agreements.

Even low-risk contracts.

Technically the system worked.

Practically?

Completely unusable.

The issue turned out to be simple:

The agent lacked a confidence-based decision layer.

If a single clause looked risky, Hermes escalated immediately.

That created too many false positives.

So I redesigned the workflow.

Now Hermes Agent:

  1. Reads the uploaded contract
  2. Detects contract type
  3. Extracts clauses
  4. Identifies risk signals
  5. Calculates confidence score
  6. Determines escalation need
  7. Generates executive summary

The result:

Hermes now behaves much more like a real enterprise analyst instead of a rule-based script.

Example output:

Contract Type:
Vendor Agreement

Risk Score:
7.2/10

Issues Found:
❌ Missing termination clause
❌ SLA definition unclear
⚠ Liability section weak

Confidence:
89%

Recommendation:
Escalate to Legal Review
Enter fullscreen mode Exit fullscreen mode

For low-risk contracts:

Contract Type:
NDA

Risk Score:
2.1/10

Issues Found:
✅ Confidentiality present
✅ Termination clause present

Confidence:
94%

Recommendation:
Approved
Enter fullscreen mode Exit fullscreen mode

Demo

Workflow 

Contract PDF
        ↓
Hermes Master Agent
        ↓
Task Planning
        ↓
Clause Extraction
        ↓
Risk Detection
        ↓
Confidence Scoring
        ↓
Compliance Check
        ↓
Final Recommendation
Enter fullscreen mode Exit fullscreen mode

Example Agent Plan 

1. Read uploaded contract
2. Identify contract type
3. Extract important clauses
4. Detect missing sections
5. Evaluate business risk
6. Calculate confidence
7. Decide escalation
Enter fullscreen mode Exit fullscreen mode

(Adding screenshots/video walkthrough soon 🚀)

Code

Repository:

https://github.com/yourusername/hermes-contract-intelligence-agent
Enter fullscreen mode Exit fullscreen mode

Example decision logic:

class ContractDecisionAgent:

    def should_escalate(
        self,
        risk_score,
        confidence
    ):

        if (
            risk_score > 0.7
            and confidence > 0.8
        ):

            return (
                "legal_review"
            )

        return (
            "approved"
        )
Enter fullscreen mode Exit fullscreen mode

My Tech Stack

  • Hermes Agent
  • Python
  • Azure Document Intelligence
  • PDFPlumber
  • PyPDF
  • FastAPI / Streamlit
  • LangChain
  • OpenAI / Azure OpenAI

How I Used Hermes Agent

Hermes Agent sits at the center of the system.

Instead of hardcoding a workflow, I used Hermes for:

1. Planning

Hermes breaks the task into smaller reasoning steps.

Example:

Read contract
↓
Determine type
↓
Extract clauses
↓
Evaluate risk
↓
Decide escalation
Enter fullscreen mode Exit fullscreen mode

2. Tool Use

Hermes invokes multiple tools dynamically:

parse_pdf()

extract_clauses()

risk_detector()

compliance_checker()

summary_generator()
Enter fullscreen mode Exit fullscreen mode

Different contract types require different reasoning paths, and Hermes dynamically chooses what to do next.

3. Multi-Step Reasoning

The agent doesn't just summarize documents.

It reasons through:

  • missing legal clauses
  • business risk
  • confidence levels
  • escalation decisions

This felt like a much more realistic enterprise use case for AI agents.

One big lesson from building this:

Agentic systems become useful only when they can decide what to do next, not just generate text.

That’s where Hermes Agent really stood out for me.

Thanks for reading 🚀

hermesagentchallenge #devchallenge #agents #python

Top comments (1)

Collapse
 
xulingfeng profile image
xulingfeng •

Nice to see another Hermes user in the wild! We ran into a similar decision-fork problem with multi-agent memory writes. What worked for us was adding a lightweight planning step before the agent picks a tool — basically a 'stop and think' phase that doesn't burn a full turn. Your legal review use case is a great fit for that pattern.