DEV Community

Ujjwal Dubey
Ujjwal Dubey

Posted on

A Weighted Agency Scorecard in Python: Turn Your Shortlist Notes Into a Ranked, Explainable Comparison

Comparing three or four agencies from memory goes badly: the last call always wins. A tiny weighted scorecard keeps the comparison honest. I use this shape whenever founders ask how to pick the best AI automation agency in Mumbai for small business workflows they actually run.

Input: scores.csv

criterion,weight,agency_a,agency_b,agency_c,evidence
watched_real_process,3,2,1,2,"A recorded our counter; C visited"
handover_to_person,3,2,2,1,"C could not show a handover"
log_you_can_read,2,1,2,2,""
pause_switch_for_owner,2,2,1,1,""
ownership_of_accounts,3,2,2,2,""
named_support_contact,2,1,2,1,""
Enter fullscreen mode Exit fullscreen mode

Scores are 0 (no), 1 (partly), 2 (clearly yes, shown to you). Weights are yours.

Script

import csv

def rank(path):
    rows = list(csv.DictReader(open(path, newline="", encoding="utf-8")))
    agencies = [k for k in rows[0] if k.startswith("agency_")]
    total_w = sum(int(r["weight"]) for r in rows)
    out = []
    for a in agencies:
        score = sum(int(r["weight"]) * int(r[a]) for r in rows)
        gaps = [r["criterion"] for r in rows if int(r[a]) == 0]
        out.append((a, round(score / (2 * total_w), 2), gaps))
    for a, s, gaps in sorted(out, key=lambda x: -x[1]):
        print(f"{a}: {s}  gaps: {', '.join(gaps) or 'none'}")

rank("scores.csv")
Enter fullscreen mode Exit fullscreen mode

How to use it well

  1. Write the evidence column. A score without a note is a guess.
  2. Only score what you saw. "They said they log everything" is a 1; "they showed us the log" is a 2.
  3. Look at gaps, not just totals. A high total with a zero on handover is still a problem.
  4. Re-score after reference calls. Scores should change when you learn more.

To see how we document a process before quoting, read our post on checking a business process before buying AI.

Disclosure: I co-founded NxFlowAI, a Mumbai-based AI automation agency, which would be one of the agencies you score.

Top comments (0)