DEV Community: Syntora

Automating Financial Reconciliation with Python and Plaid

Syntora — Sat, 14 Feb 2026 01:42:47 +0000

If you run a small business, you know the drill. Every month you sit down with your bank statements, open QuickBooks, and start the tedious process of matching transactions. Did that $47.99 charge from "SQ *COFFEE HOUSE LLC" match the office supplies entry or the client meeting expense? Multiply that ambiguity by hundreds of transactions and you have a full afternoon gone.

I built a pipeline that handles this automatically. It pulls transactions from Plaid, categorizes them with AI, reconciles them against expected journal entries, and flags anything it is not confident about. The whole thing runs on Python with Supabase as the data store.

This article walks through the architecture and the code. I am Parker Gawne, founder of Syntora, and this came out of a real project we built for accounting automation.

Architecture Overview

The pipeline has four stages:

Pull - Plaid API syncs bank transactions into Supabase
Categorize - Each transaction gets classified. Ambiguous ones go through Claude for AI categorization.
Reconcile - Categorized transactions get matched against expected entries from QuickBooks
Review - High-confidence matches auto-approve. Low-confidence matches get flagged for human review.

Every step writes to Supabase so there is a full audit trail. Nothing gets deleted. Nothing gets overwritten. Financial data demands that level of care.

Setting Up the Plaid Client

Plaid gives you access to bank transaction data through a clean API. You connect a bank account once through their Link flow, then pull transactions on a schedule.

import plaid
from plaid.api import plaid_api
from plaid.model.transactions_sync_request import TransactionsSyncRequest
import os

def get_plaid_client():
    configuration = plaid.Configuration(
        host=plaid.Environment.Production,
        api_key={
            "clientId": os.environ["PLAID_CLIENT_ID"],
            "secret": os.environ["PLAID_SECRET"],
        },
    )
    api_client = plaid.ApiClient(configuration)
    return plaid_api.PlaidApi(api_client)

I use the Transactions Sync endpoint rather than the older Transactions Get. Sync gives you a cursor-based approach so you only pull new transactions each run instead of re-fetching everything.

Syncing Transactions

from supabase import create_client
from datetime import datetime

supabase = create_client(
    os.environ["SUPABASE_URL"],
    os.environ["SUPABASE_SERVICE_KEY"],
)

def sync_transactions(access_token: str, account_id: str):
    client = get_plaid_client()

    # Get the last cursor from Supabase
    result = supabase.table("plaid_sync_cursors").select("cursor").eq(
        "account_id", account_id
    ).single().execute()

    cursor = result.data["cursor"] if result.data else ""

    has_more = True
    all_added = []

    while has_more:
        request = TransactionsSyncRequest(
            access_token=access_token,
            cursor=cursor,
        )
        response = client.transactions_sync(request)
        all_added.extend(response.added)
        has_more = response.has_more
        cursor = response.next_cursor

    # Store transactions in Supabase
    for txn in all_added:
        supabase.table("transactions").upsert({
            "plaid_transaction_id": txn.transaction_id,
            "account_id": account_id,
            "amount": float(txn.amount),
            "date": txn.date.isoformat(),
            "merchant_name": txn.merchant_name,
            "name": txn.name,
            "category": txn.personal_finance_category.primary if txn.personal_finance_category else None,
            "pending": txn.pending,
            "status": "pending_categorization",
            "synced_at": datetime.utcnow().isoformat(),
        }).execute()

    # Update cursor
    supabase.table("plaid_sync_cursors").upsert({
        "account_id": account_id,
        "cursor": cursor,
    }).execute()

    return len(all_added)

Plaid provides its own category field, but it is too broad for accounting purposes. "Food and Drink" does not tell you whether it was a client dinner or a personal lunch. That is where AI categorization comes in.

AI Categorization with Claude

Most transactions are straightforward. "ADOBE CREATIVE CLOUD" is obviously a software subscription. "AMZN MKTP US" is probably office supplies, but it could be anything. The ambiguous ones are where Claude earns its keep.

I define a set of accounting categories that map to my chart of accounts, then ask Claude to classify each transaction.

import anthropic

claude = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

ACCOUNTING_CATEGORIES = [
    "software_subscriptions",
    "office_supplies",
    "travel",
    "meals_entertainment",
    "professional_services",
    "advertising",
    "utilities",
    "insurance",
    "payroll",
    "contractor_payments",
    "client_reimbursable",
    "owner_distribution",
    "other",
]

def categorize_transaction(txn: dict) -> dict:
    prompt = f"""You are a bookkeeper categorizing bank transactions for a small tech consultancy.

Given this transaction, classify it into exactly one category and provide a confidence score from 0.0 to 1.0.

Transaction:
- Merchant: {txn["merchant_name"]}
- Description: {txn["name"]}
- Amount: ${txn["amount"]}
- Date: {txn["date"]}
- Plaid Category: {txn.get("category", "unknown")}

Valid categories: {", ".join(ACCOUNTING_CATEGORIES)}

Respond in JSON format:
{{"category": "category_name", "confidence": 0.85, "reasoning": "brief explanation"}}

Rules:
- If the merchant name is ambiguous (e.g., Amazon, Square), use the amount and Plaid category as context clues
- If you are not confident, say so with a low score rather than guessing
- "other" should only be used when nothing else fits"""

    response = claude.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=256,
        messages=[{"role": "user", "content": prompt}],
    )

    import json
    result = json.loads(response.content[0].text)
    return result

The confidence score is critical. I set two thresholds:

Above 0.85: Auto-approve. The categorization gets applied directly.
Between 0.5 and 0.85: Needs review. Stored with a "needs_review" status.
Below 0.5: Flagged as uncertain. These go into a manual review queue.

def process_pending_transactions():
    result = supabase.table("transactions").select("*").eq(
        "status", "pending_categorization"
    ).execute()

    for txn in result.data:
        classification = categorize_transaction(txn)

        confidence = classification["confidence"]
        if confidence >= 0.85:
            status = "categorized"
        elif confidence >= 0.5:
            status = "needs_review"
        else:
            status = "uncertain"

        supabase.table("transactions").update({
            "ai_category": classification["category"],
            "ai_confidence": confidence,
            "ai_reasoning": classification["reasoning"],
            "status": status,
            "categorized_at": datetime.utcnow().isoformat(),
        }).eq("plaid_transaction_id", txn["plaid_transaction_id"]).execute()

In practice, about 70% of transactions land above the 0.85 threshold. Another 20% fall into the review bucket. Only about 10% are truly ambiguous. That is a significant time savings compared to manually categorizing everything.

Reconciliation Against Expected Entries

Once transactions are categorized, they need to be matched against the entries you expect to see. Recurring charges like software subscriptions, payroll, and contractor payments should show up on predictable schedules.

I keep a table of expected entries in Supabase that gets populated from QuickBooks recurring transactions.

def reconcile_transactions():
    # Get categorized transactions that have not been reconciled
    txns = supabase.table("transactions").select("*").eq(
        "status", "categorized"
    ).execute()

    # Get expected entries for this period
    expected = supabase.table("expected_entries").select("*").eq(
        "period", datetime.utcnow().strftime("%Y-%m")
    ).execute()

    for txn in txns.data:
        match = find_best_match(txn, expected.data)

        if match:
            supabase.table("journal_entries").insert({
                "transaction_id": txn["plaid_transaction_id"],
                "expected_entry_id": match["id"],
                "amount": txn["amount"],
                "category": txn["ai_category"],
                "match_confidence": match["score"],
                "status": "matched",
                "created_at": datetime.utcnow().isoformat(),
            }).execute()

            supabase.table("transactions").update({
                "status": "reconciled",
            }).eq("plaid_transaction_id", txn["plaid_transaction_id"]).execute()
        else:
            # No match found, flag as new/unexpected
            supabase.table("journal_entries").insert({
                "transaction_id": txn["plaid_transaction_id"],
                "expected_entry_id": None,
                "amount": txn["amount"],
                "category": txn["ai_category"],
                "match_confidence": 0.0,
                "status": "unmatched",
                "created_at": datetime.utcnow().isoformat(),
            }).execute()

def find_best_match(txn: dict, expected_entries: list) -> dict | None:
    best = None
    best_score = 0.0

    for entry in expected_entries:
        score = 0.0

        # Category match
        if txn["ai_category"] == entry["category"]:
            score += 0.4

        # Amount within 5% tolerance
        if entry["expected_amount"]:
            diff = abs(txn["amount"] - entry["expected_amount"])
            tolerance = entry["expected_amount"] * 0.05
            if diff <= tolerance:
                score += 0.4
            elif diff <= tolerance * 2:
                score += 0.2

        # Merchant name similarity
        if txn.get("merchant_name") and entry.get("merchant_pattern"):
            if entry["merchant_pattern"].lower() in txn["merchant_name"].lower():
                score += 0.2

        if score > best_score and score >= 0.6:
            best_score = score
            best = {"id": entry["id"], "score": score}

    return best

The matching is intentionally conservative. A 60% threshold for matches means both the category and the amount need to line up before it will auto-match. Anything that does not meet that bar goes into the unmatched queue.

The Supabase Schema

The data model is straightforward. Four tables handle the core workflow.

-- Transactions from Plaid
create table transactions (
    plaid_transaction_id text primary key,
    account_id text not null,
    amount numeric not null,
    date date not null,
    merchant_name text,
    name text,
    category text,
    pending boolean default false,
    status text default 'pending_categorization',
    ai_category text,
    ai_confidence numeric,
    ai_reasoning text,
    synced_at timestamptz,
    categorized_at timestamptz
);

-- Expected recurring entries from QuickBooks
create table expected_entries (
    id uuid primary key default gen_random_uuid(),
    period text not null,
    category text not null,
    expected_amount numeric,
    merchant_pattern text,
    description text
);

-- Reconciled journal entries
create table journal_entries (
    id uuid primary key default gen_random_uuid(),
    transaction_id text references transactions(plaid_transaction_id),
    expected_entry_id uuid references expected_entries(id),
    amount numeric not null,
    category text not null,
    match_confidence numeric,
    status text default 'pending',
    created_at timestamptz default now()
);

-- Plaid sync cursors for incremental sync
create table plaid_sync_cursors (
    account_id text primary key,
    cursor text not null
);

Every AI categorization stores the reasoning alongside the confidence score. When a human reviewer looks at a flagged transaction, they can see why the system was uncertain and make a faster decision.

Running It on a Schedule

The full pipeline runs as a scheduled Python script. In production I trigger it with a cron job, but you could just as easily use a Supabase Edge Function or a cloud scheduler.

def run_pipeline():
    accounts = supabase.table("plaid_accounts").select("*").execute()

    for account in accounts.data:
        count = sync_transactions(account["access_token"], account["account_id"])
        print(f"Synced {count} transactions for {account['account_id']}")

    process_pending_transactions()
    reconcile_transactions()

    # Summary
    stats = supabase.table("transactions").select("status", count="exact").execute()
    print(f"Pipeline complete. Transaction summary: {stats.count}")

if __name__ == "__main__":
    run_pipeline()

What I Learned Building This

Financial data is unforgiving. Unlike a content management system where a miscategorization is annoying but harmless, a wrong journal entry creates real accounting problems. That is why every decision the system makes is logged with a confidence score and reasoning. The audit trail is not optional.

AI categorization is good enough to save time, not good enough to trust blindly. The 0.85 confidence threshold was calibrated over a few months of running the pipeline and checking results. Lowering it to 0.7 would auto-approve more transactions but would also introduce errors that take longer to fix than the time saved.

Plaid's transaction data varies wildly by bank. Some banks send clean merchant names. Others send cryptic strings like "POS DEBIT 02/03 SQ *STORE." Building the AI categorizer was partly motivated by this inconsistency. Rule-based matching breaks down fast when merchant names are unpredictable.

Start with a narrow scope. This pipeline handles a single entity with a handful of bank accounts. Scaling to multiple entities or multi-currency would add significant complexity. Build for what you need today.

If you are dealing with similar automation challenges, whether it is financial reconciliation or another data pipeline that needs AI classification, this is exactly the kind of work we do at Syntora.

The Honest Caveats

Never auto-approve financial transactions without a confidence threshold. Never delete transaction records. Never skip the audit trail. These are not suggestions. If your accountant cannot trace every automated decision back to its source data, you have a compliance problem.

This pipeline does not replace an accountant. It replaces the manual data entry and categorization that takes up most of their time. The accountant still reviews flagged items, validates the journal entries, and signs off on the books. The pipeline just makes sure they are spending their time on judgment calls rather than data entry.

Built by Syntora - automation and AI consulting for teams that ship.

Real Numbers: What AI Automation Actually Costs for Small Business

Syntora — Sat, 14 Feb 2026 01:35:43 +0000

Every small business owner I talk to asks the same question: "What does this actually cost?" And they ask it because the automation industry has done a terrible job of answering it.

Most vendors hide behind "custom quotes" and "it depends" because they do not want you comparing prices. I get it. Every project is different. But there are patterns, and if you know the patterns, you can budget with confidence before you talk to anyone.

This is the cost breakdown I wish someone had given me when I started building automation systems. Every number here comes from real projects, real invoices, and real client outcomes.

How Much Does Business Automation Software Cost for Small Companies?

The honest answer is somewhere between $50 per month and $40,000 upfront, depending on which tier fits your situation. That is a wide range, so let me break it into three clear categories.

Tier 1: DIY with No-Code Tools

Monthly tool costs: $50 to $500/mo
Development costs: $0
Maintenance time: 5 to 15 hours/month (yours)

This is the Zapier and Make tier. You sign up, connect your apps, build workflows in a visual editor, and handle everything yourself.

Here is what the real monthly spend looks like:

Tool	Cost	What It Does
Zapier (Starter)	$19.99/mo	750 tasks/month
Zapier (Professional)	$49/mo	2,000 tasks/month
Make (Core)	$10.59/mo	10,000 ops/month
Make (Pro)	$18.82/mo	10,000 ops + priority
Google Workspace	$7.20/mo	Email, Sheets, Drive
Airtable (Team)	$20/mo	Database layer

A typical small business running 5 to 10 automations on Zapier or Make lands around $100 to $300 per month in tool costs. The hidden cost is your time. You are the developer, the tester, the debugger, and the maintenance crew. When a Zap breaks at 2 AM because an API changed its response format, you are the one fixing it.

This tier works when:

Your automations are simple (move data from A to B)
Volume is low (under 5,000 operations per month)
You have time to maintain things yourself
You are testing whether automation works for your use case at all

This tier breaks when:

You need conditional logic or error handling
Volume spikes and per-task pricing becomes expensive
You are spending more time maintaining automations than doing the work they replaced

Tier 2: Hybrid (Some Custom + Some Tools)

Monthly tool costs: $200 to $800/mo
Development costs: $5,000 to $15,000 (one-time)
Maintenance time: 1 to 3 hours/month

This is where you bring in a developer or consultancy to build the complex parts while keeping simple automations on no-code tools. Most of the small businesses I work with start here.

A typical hybrid setup looks like this:

Component	Cost	Notes
n8n (self-hosted)	$5 to $20/mo	Hosting on a VPS
Supabase (database)	$0 to $25/mo	Free tier handles most cases
Custom scripts	$5,000 to $15,000	One-time development
Simple Zaps (kept)	$20 to $50/mo	For truly simple triggers
Monitoring tools	$0 to $20/mo	Uptime and error alerts

The development cost covers building the pieces that no-code tools cannot handle well: custom API integrations, data transformation pipelines, multi-step workflows with real error handling, and reporting dashboards.

This tier works when:

You have 3 to 5 processes that are too complex for Zapier
You want someone else to build and document the system
You need reliability but not full custom infrastructure
Your budget is in the $10,000 to $20,000 range for the first year

This tier breaks when:

You are stitching together so many tools that the integration layer becomes its own maintenance burden
Data privacy matters and you cannot have information flowing through third-party servers

Tier 3: Full Custom Python Infrastructure

Monthly hosting costs: $5 to $50/mo
Development costs: $10,000 to $40,000 (one-time)
Maintenance time: Under 1 hour/month

This is what we build at Syntora for clients who have outgrown the tool-stitching phase. The monthly operating cost drops dramatically because you are paying for compute, not per-task pricing.

Here are the actual infrastructure costs from recent client projects:

Component	Monthly Cost
DigitalOcean Droplet (basic)	$5 to $12/mo
Supabase (free tier)	$0/mo
Supabase (Pro, if needed)	$25/mo
Claude API (Haiku, light use)	$3 to $8/mo
Claude API (Sonnet, moderate use)	$10 to $15/mo
Domain and DNS	$1 to $2/mo
Email sending (Resend)	$0 to $20/mo

A fully custom system running on a $12/month DigitalOcean droplet with Supabase's free tier and moderate Claude API usage costs about $25 to $50 per month to operate. Compare that to $300 to $800 per month in tool subscriptions for the same functionality.

The development cost is higher upfront. A typical engagement runs $15,000 to $30,000 depending on complexity. But you own the code, you control the infrastructure, and your monthly costs stay flat regardless of volume.

This tier works when:

You have proven the automation concept and need it to scale
Per-task pricing is eating your budget
Data privacy or compliance requires self-hosted infrastructure
You want the lowest possible long-term cost

The Math: When Custom Pays for Itself

Here is a real example from a client engagement last year. They were spending 15 hours per week on data entry, report generation, and client communication across three tools.

The numbers:

Factor	Value
Hours saved per week	15
Value of that time (at $50/hr)	$750/week
Monthly value of time saved	$3,250/month
Custom build cost	$20,000
Monthly operating cost	$35/month
Previous tool costs replaced	$420/month
Net monthly savings	$3,635/month
Payback period	5.5 months

Even if you cut the time savings in half to be conservative, a $20,000 build that saves 7.5 hours per week still pays for itself in about 11 months. After that, you are saving $1,500+ per month indefinitely.

This is why I push clients toward custom when the volume justifies it. The upfront cost feels significant, but the compounding savings make it the cheapest option over any time horizon longer than a year.

Is AI Automation Worth It for Small Businesses with Limited Budgets?

Sometimes the answer is no. I have turned away projects where automation was not the right move, and I would rather be honest about that than sell work that does not deliver value.

Do not invest in automation if:

Your process is not stable yet. If you are still changing how you do things every month, automating a moving target wastes money. Stabilize the process first, then automate.
Volume is too low. If a task happens 3 times a week and takes 5 minutes, you are saving 15 minutes per week. That does not justify a $10,000 build. It barely justifies a $50/month Zapier plan.
You cannot define the rules. Automation needs clear logic: if this, then that. If your process relies heavily on judgment calls that change based on context, AI can help but the system design is more complex and more expensive.
You are automating a bad process. Automating a broken workflow just makes it break faster. Fix the process first.

The threshold I use: if the automation saves at least 5 hours per week and the process has been stable for at least 3 months, it is worth exploring. Below that, stick with Tier 1 tools or keep doing it manually.

Actual API Costs: What AI Models Cost to Run

One of the most common questions I get is about AI API pricing. Here is what the Claude API actually costs in production, based on real usage across client projects:

Model	Cost per 1,000 Calls	Best For
Claude Haiku	$0.25 to $1.25	Classification, extraction, simple Q&A
Claude Sonnet	$3 to $9	Complex analysis, content generation
Claude Opus	$15 to $75	Deep reasoning, multi-step analysis

Most small business automation uses Haiku or Sonnet. A client processing 200 documents per day with Haiku-level classification spends about $3 to $5 per month on API costs. That is not a typo. The per-call cost is fractions of a cent.

The expensive part is never the API. It is the development time to build the system that calls the API correctly, handles errors, and presents results in a useful format.

How Much Should a Small Business Spend on AI Automation Tools?

My rule of thumb: your first-year automation budget should be 2 to 5 percent of the annual cost of the labor it replaces. If you are spending $80,000 per year on tasks that can be automated, a $2,000 to $4,000 investment in the first year is reasonable for Tier 1. A $15,000 to $25,000 investment is reasonable for Tier 3 if the savings justify it.

Do not start with Tier 3. Start with Tier 1 to validate the concept, graduate to Tier 2 when the limitations bite, and invest in Tier 3 when you have proven the value and need it to scale.

If you are not sure which tier fits, we offer a free discovery call where I will walk through your specific situation and give you a straight answer. No pitch, no pressure, just numbers.

The Cost Summary

	DIY (Tier 1)	Hybrid (Tier 2)	Custom (Tier 3)
Monthly tools	$50 to $500	$200 to $800	$5 to $50
Upfront build	$0	$5K to $15K	$10K to $40K
Your maintenance time	5 to 15 hrs/mo	1 to 3 hrs/mo	Under 1 hr/mo
Best for	Testing, low volume	Proven concept, moderate volume	Scale, privacy, long-term savings
1-year total cost	$600 to $6,000	$7,400 to $24,600	$10,060 to $40,600
3-year total cost	$1,800 to $18,000	$12,200 to $43,800	$10,180 to $41,800

Notice that Tier 3 barely increases between year 1 and year 3. That is the power of owning your infrastructure. You pay once to build it, and then operating costs stay flat.

Where to Start

If you are reading this and wondering which tier applies to you, start with two questions:

How many hours per week do you spend on the tasks you want to automate? If it is under 5 hours, start with Tier 1 or wait. If it is over 10 hours, Tier 2 or 3 will pay for itself.
Has the process been the same way for at least 3 months? If yes, it is stable enough to automate. If no, stabilize first.

From there, the decision is mostly about budget and timeline. Tier 1 is fast and cheap. Tier 3 is slower to build but cheaper to run for years.

We build custom automation infrastructure for small and mid-size businesses. If you want to talk through the numbers for your specific case, reach out here. I will tell you which tier makes sense and whether it is worth doing at all.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io

Building AI Agents That Actually Work in Business Workflows

Syntora — Sat, 14 Feb 2026 01:30:35 +0000

Everyone is talking about AI agents. The pitch is compelling: autonomous software that reasons, plans, and executes multi-step tasks without human intervention. The reality is more nuanced. Most businesses do not need a fully autonomous agent. They need targeted automation with intelligence at specific decision points.

I run Syntora, a consultancy that builds custom AI automation for small and mid-size businesses. Over the past year, I have built and deployed production agents using the Claude API from Anthropic. Some worked exactly as planned. Others taught me expensive lessons about where agents break down.

This is what I have learned about which patterns actually work and which ones will burn your budget before delivering value.

Can an Agentic AI Handle Complex Decision-Making in Business Workflows?

Yes, but only within well-defined boundaries. The word "complex" is doing a lot of heavy lifting in that question. An agent can handle complex classification, complex extraction, and complex generation. It cannot reliably handle complex judgment calls that require institutional knowledge, regulatory awareness, or relationship context.

The distinction matters. A document classification agent that routes incoming contracts to the right department based on content analysis? That works. An agent that decides whether to accept a contract's terms? That fails in ways that cost real money.

Here are the three patterns I keep coming back to because they deliver consistent results in production.

Three Patterns That Work

1. Extraction Agents: Unstructured to Structured

The most reliable agent pattern takes messy, unstructured input and converts it into clean, structured output. Think invoices, contracts, emails, PDFs, and support tickets. Humans send information in wildly inconsistent formats. An extraction agent normalizes all of it.

This pattern works because the success criteria is clear. Either the extracted data matches the source document or it does not. There is no ambiguity, no judgment call, no creative interpretation needed.

Here is a simplified extraction agent using the Anthropic SDK:

import anthropic
import json

client = anthropic.Anthropic()

def extract_invoice_data(raw_text: str) -> dict:
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": f"""Extract the following fields from this invoice text.
Return valid JSON only. No explanation.

Fields: vendor_name, invoice_number, date, line_items (array of description, quantity, unit_price, total), subtotal, tax, grand_total

Invoice text:
{raw_text}"""
            }
        ]
    )
    return json.loads(message.content[0].text)

Sonnet handles this perfectly. I have never needed to use Opus for extraction tasks. The cost difference matters at scale: processing 10,000 invoices per month on Opus would cost roughly 5x more than Sonnet with no measurable improvement in extraction accuracy.

2. Classification Agents: Routing and Triage

Classification agents read incoming content and decide where it should go. Support ticket triage, lead qualification, document routing, email categorization. The agent acts as an intelligent switchboard.

This pattern works because the output space is constrained. The agent picks from a finite set of categories. Even when the input is ambiguous, the worst case is a misroute that a human catches and corrects, not a catastrophic failure.

I will walk through a full example of this pattern below.

3. Generation Agents: Content and Proposals

Generation agents produce first drafts of structured content. Proposal templates, report summaries, email responses, status updates. The key word is "first draft." These agents do not ship content directly to customers. They produce output that a human reviews, edits, and approves.

This pattern works because it eliminates the blank page problem. A business development team that spends 45 minutes writing each proposal from scratch can instead spend 10 minutes editing an agent-generated draft. The agent handles structure and boilerplate. The human adds nuance and relationship context.

If you are evaluating where AI agents could fit in your workflow, we can help you identify the right use cases.

Three Patterns That Fail

1. Fully Autonomous Customer-Facing Agents

What worries you most about AI agents interacting directly with customers? It should be the failure modes. When an extraction agent makes a mistake, a human catches it during review. When a customer-facing agent makes a mistake, the customer experiences it in real time.

I have seen businesses deploy chatbots that can issue refunds, modify accounts, and make promises. The bot works correctly 95% of the time. The other 5% generates support escalations, social media complaints, and lost customers. A 95% accuracy rate sounds good until you realize that means 1 in 20 customers has a bad experience with no human to intervene.

The fix is not to avoid customer-facing AI entirely. It is to constrain the agent's authority. Let it answer questions, surface relevant information, and draft responses. Do not let it take irreversible actions without human approval.

2. Agents With Unbounded Tool Access

Giving an agent access to your database, your email system, your CRM, and your file storage feels powerful. It is also dangerous. An agent with broad tool access can take actions you did not anticipate in contexts you did not test.

Every tool an agent can access is a surface area for failure. I scope tool access to the minimum required for the task. A classification agent needs read access to incoming documents and write access to a routing queue. It does not need access to your CRM, your email, or your payment system.

3. Agents Without Human-in-the-Loop for High-Stakes Decisions

Any decision that involves money, legal exposure, or customer relationships needs a human checkpoint. Full stop. I build agents that recommend, draft, and classify. I do not build agents that approve, commit, or send without review.

The human-in-the-loop is not a weakness of the system. It is a feature. The agent handles the 80% of the work that is mechanical and repetitive. The human handles the 20% that requires judgment, context, and accountability.

Practical Example: Document Classification Agent

Here is a production-style document classification agent that routes incoming PDFs. This is the pattern I deploy most frequently because it solves a real problem at every business that processes inbound documents.

import anthropic
import json
from dataclasses import dataclass

client = anthropic.Anthropic()

CATEGORIES = {
    "contract": {
        "route_to": "legal",
        "priority": "high",
        "sla_hours": 24
    },
    "invoice": {
        "route_to": "accounting",
        "priority": "medium",
        "sla_hours": 48
    },
    "proposal": {
        "route_to": "business_dev",
        "priority": "medium",
        "sla_hours": 48
    },
    "support_request": {
        "route_to": "customer_success",
        "priority": "high",
        "sla_hours": 4
    },
    "general_correspondence": {
        "route_to": "admin",
        "priority": "low",
        "sla_hours": 72
    }
}

@dataclass
class ClassificationResult:
    category: str
    confidence: float
    route_to: str
    priority: str
    sla_hours: int
    summary: str

def classify_document(document_text: str) -> ClassificationResult:
    category_list = ", ".join(CATEGORIES.keys())

    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=512,
        messages=[
            {
                "role": "user",
                "content": f"""Classify this document into exactly one category.

Categories: {category_list}

Return valid JSON with these fields:
- category: one of the categories above
- confidence: float between 0 and 1
- summary: one sentence describing the document

Document text:
{document_text}"""
            }
        ]
    )

    result = json.loads(message.content[0].text)
    routing = CATEGORIES[result["category"]]

    return ClassificationResult(
        category=result["category"],
        confidence=result["confidence"],
        route_to=routing["route_to"],
        priority=routing["priority"],
        sla_hours=routing["sla_hours"],
        summary=result["summary"]
    )

def process_incoming_document(document_text: str) -> dict:
    result = classify_document(document_text)

    if result.confidence < 0.7:
        return {
            "action": "manual_review",
            "reason": "Low confidence classification",
            "result": result
        }

    return {
        "action": "auto_route",
        "destination": result.route_to,
        "priority": result.priority,
        "sla_hours": result.sla_hours,
        "result": result
    }

Notice the confidence threshold. When the agent is not sure, it flags the document for manual review instead of guessing. That one check prevents the majority of misrouting errors.

Cost Control: How to Not Burn Money

AI agent costs come from three places: API calls, token usage, and iteration loops. Here is how I manage each.

Model selection. Sonnet handles 90% of agent tasks. I only use Opus for tasks that require deep reasoning across long documents or nuanced multi-step analysis. For a typical classification or extraction agent, Sonnet is faster, cheaper, and equally accurate.

Token budgets. Every agent gets a max_tokens ceiling. If your extraction agent is returning 4,000 tokens when the structured output should be 500, something is wrong. Set the ceiling close to your expected output size. This prevents runaway costs from malformed prompts or unexpected inputs.

Batch processing. When you are processing hundreds of documents, use the Anthropic batch API instead of individual requests. The cost savings and rate limit benefits add up fast.

Here is a rough cost comparison for processing 10,000 documents per month:

Approach	Estimated Monthly Cost
Opus, individual requests	$150-300
Sonnet, individual requests	$30-60
Sonnet, batch API	$15-30

The difference between the cheapest and most expensive approach is 10x. For most classification and extraction tasks, the cheapest option delivers identical results.

Scoping an Agent Project

When a client asks me to build an AI agent, I start with three questions:

What is the input? If the input is unstructured text, images, or documents, an agent can help. If the input is already structured data, you probably need a regular script, not an agent.
What is the output? If the output is a classification, a structured data object, or a first draft, an agent fits. If the output is a final decision with no human review, proceed with extreme caution.
What happens when it is wrong? If a mistake means a document goes to the wrong queue and someone re-routes it, the risk is low. If a mistake means a customer gets the wrong information or money moves to the wrong account, you need human-in-the-loop at minimum and possibly a different approach entirely.

These three questions filter out 80% of bad agent ideas before any code gets written.

The Bottom Line

AI agents work in business when they are scoped tightly, given constrained tool access, and paired with human oversight for high-stakes decisions. The three patterns that consistently deliver value are extraction, classification, and generation. The pattern that consistently fails is full autonomy without guardrails.

Build agents that make your team faster, not agents that replace your team's judgment.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io

How I Handle Mismatched Data Across Shopify, QuickBooks, and 3 Other Systems

Syntora — Sat, 14 Feb 2026 01:29:05 +0000

If you run a business that touches more than two platforms, you already know the problem. Shopify calls it an "order." QuickBooks calls it an "invoice." Your CRM calls it a "deal." They all represent the same transaction, but none of them agree on the ID, the field names, or even the dollar amount after fees and tax adjustments.

I am Parker Gawne, founder of Syntora, and this is one of the most common problems I solve for clients. Not the kind of problem where you install a Zapier integration and walk away. The kind where you have thousands of records drifting out of sync across five systems and no one knows which version of the truth is correct.

Here is how I build reconciliation pipelines that actually work.

The Real Problem: No System Agrees on Anything

Consider a typical e-commerce operation. You might have:

Shopify for orders and products
QuickBooks Online for invoices and accounting
a CRM (HubSpot, Salesforce, etc.) for customer records
Plaid for bank transaction matching
a shipping platform (ShipStation, EasyPost) for fulfillment

Each system has its own internal ID for a customer, its own format for addresses, its own rules for how refunds get recorded. When a customer places an order, that single event creates records in at least three of these systems. Within a week, those records have diverged.

The symptoms are familiar: duplicate customer entries, invoices that do not match orders, revenue numbers that differ between your accounting software and your storefront, bank transactions that cannot be traced back to a specific order.

Manual reconciliation does not scale. I have watched operations teams burn entire days cross-referencing spreadsheets. The fix is a dedicated reconciliation service.

The Architecture: A Canonical Data Model

The core idea is simple. Instead of trying to sync System A directly to System B, you define a single internal data model that every system maps into. I call this the canonical model.

from dataclasses import dataclass, field
from datetime import datetime
from decimal import Decimal
from typing import Optional


@dataclass
class CanonicalCustomer:
    canonical_id: str
    email: str
    first_name: str
    last_name: str
    phone: Optional[str] = None
    source_ids: dict = field(default_factory=dict)
    created_at: Optional[datetime] = None


@dataclass
class CanonicalOrder:
    canonical_id: str
    customer_canonical_id: str
    total: Decimal
    currency: str
    status: str
    line_items: list = field(default_factory=list)
    source_ids: dict = field(default_factory=dict)
    created_at: Optional[datetime] = None

The source_ids dictionary is the key piece. It maps each external system to its own ID for that record. A single customer might look like this:

customer = CanonicalCustomer(
    canonical_id="cust_abc123",
    email="jane@example.com",
    first_name="Jane",
    last_name="Doe",
    phone="+15551234567",
    source_ids={
        "shopify": "7891011",
        "quickbooks": "CUS-442",
        "hubspot": "contact_98765",
    },
)

This gives you a single place to look up any customer regardless of which system you are working with.

Normalizer Functions: Making Every System Speak the Same Language

Each platform returns data in a different shape. Shopify gives you nested JSON with snake_case fields. QuickBooks uses PascalCase with deeply nested reference objects. Your CRM has its own conventions entirely.

I write a normalizer for each system. Each one takes the raw API response and returns a canonical object.

from decimal import Decimal


def normalize_shopify_order(raw: dict) -> CanonicalOrder:
    return CanonicalOrder(
        canonical_id="",  # assigned during reconciliation
        customer_canonical_id="",  # resolved via customer matching
        total=Decimal(str(raw["total_price"])),
        currency=raw["currency"].upper(),
        status=map_shopify_status(raw["financial_status"]),
        line_items=[
            {
                "sku": item["sku"],
                "quantity": item["quantity"],
                "unit_price": Decimal(str(item["price"])),
            }
            for item in raw["line_items"]
        ],
        source_ids={"shopify": str(raw["id"])},
        created_at=parse_iso(raw["created_at"]),
    )


def normalize_quickbooks_invoice(raw: dict) -> CanonicalOrder:
    return CanonicalOrder(
        canonical_id="",
        customer_canonical_id="",
        total=Decimal(str(raw["TotalAmt"])),
        currency=raw.get("CurrencyRef", {}).get("value", "USD").upper(),
        status=map_qb_status(raw.get("Balance", 0), raw["TotalAmt"]),
        line_items=[
            {
                "sku": line.get("SalesItemLineDetail", {})
                .get("ItemRef", {})
                .get("value", ""),
                "quantity": int(
                    line.get("SalesItemLineDetail", {}).get("Qty", 0)
                ),
                "unit_price": Decimal(
                    str(
                        line.get("SalesItemLineDetail", {}).get(
                            "UnitPrice", 0
                        )
                    )
                ),
            }
            for line in raw.get("Line", [])
            if line.get("DetailType") == "SalesItemLineDetail"
        ],
        source_ids={"quickbooks": str(raw["Id"])},
        created_at=parse_iso(raw.get("TxnDate")),
    )

The status mapping functions handle the vocabulary differences between systems. Shopify uses "paid," "partially_refunded," "refunded." QuickBooks does not have a status field at all; you infer payment status by comparing Balance to TotalAmt.

def map_shopify_status(financial_status: str) -> str:
    mapping = {
        "paid": "completed",
        "partially_refunded": "partial_refund",
        "refunded": "refunded",
        "pending": "pending",
        "authorized": "pending",
        "voided": "cancelled",
    }
    return mapping.get(financial_status, "unknown")


def map_qb_status(balance: float, total: float) -> str:
    if balance == 0:
        return "completed"
    if balance < total:
        return "partial_payment"
    return "pending"

This is where I encourage clients to invest time upfront. Getting the normalizers right means every downstream comparison is clean. If you want to see how we approach this for other integration types, check out our data automation services.

Deduplication: Matching Records Across Systems

Once everything is normalized, you need to figure out which records refer to the same real-world entity. For customers, email is usually the strongest match key. But email alone is not enough because people use different addresses on different platforms.

I use a scoring approach:

from difflib import SequenceMatcher


def calculate_match_score(a: CanonicalCustomer, b: CanonicalCustomer) -> float:
    score = 0.0

    if a.email and b.email and a.email.lower() == b.email.lower():
        score += 0.5

    if a.phone and b.phone:
        a_digits = "".join(filter(str.isdigit, a.phone))[-10:]
        b_digits = "".join(filter(str.isdigit, b.phone))[-10:]
        if a_digits == b_digits:
            score += 0.3

    name_a = f"{a.first_name} {a.last_name}".lower().strip()
    name_b = f"{b.first_name} {b.last_name}".lower().strip()
    name_similarity = SequenceMatcher(None, name_a, name_b).ratio()
    score += name_similarity * 0.2

    return score


MATCH_THRESHOLD = 0.7


def find_duplicates(
    customers: list[CanonicalCustomer],
) -> list[tuple[CanonicalCustomer, CanonicalCustomer, float]]:
    duplicates = []
    for i, a in enumerate(customers):
        for b in customers[i + 1 :]:
            score = calculate_match_score(a, b)
            if score >= MATCH_THRESHOLD:
                duplicates.append((a, b, score))
    return duplicates

For orders, matching is usually easier. Most businesses have an order number that appears in both Shopify and QuickBooks (often as a custom field or memo line on the invoice). When that is not available, I match on date plus total amount plus customer, which catches the vast majority of cases.

The Reconciliation Flow

Here is the full pipeline, run on a schedule (usually nightly for most clients, hourly for high-volume operations):

from dataclasses import dataclass


@dataclass
class Discrepancy:
    canonical_id: str
    field: str
    source_a: str
    value_a: str
    source_b: str
    value_b: str
    severity: str


def reconcile_orders(
    shopify_orders: list[CanonicalOrder],
    qb_invoices: list[CanonicalOrder],
) -> list[Discrepancy]:
    discrepancies = []

    qb_lookup = {}
    for inv in qb_invoices:
        qb_lookup[inv.source_ids.get("quickbooks", "")] = inv

    for order in shopify_orders:
        matched_invoice = find_matching_invoice(order, qb_invoices)

        if matched_invoice is None:
            discrepancies.append(
                Discrepancy(
                    canonical_id=order.canonical_id,
                    field="existence",
                    source_a="shopify",
                    value_a=order.source_ids.get("shopify", ""),
                    source_b="quickbooks",
                    value_b="MISSING",
                    severity="high",
                )
            )
            continue

        if order.total != matched_invoice.total:
            discrepancies.append(
                Discrepancy(
                    canonical_id=order.canonical_id,
                    field="total",
                    source_a="shopify",
                    value_a=str(order.total),
                    source_b="quickbooks",
                    value_b=str(matched_invoice.total),
                    severity="high",
                )
            )

        if order.status != matched_invoice.status:
            discrepancies.append(
                Discrepancy(
                    canonical_id=order.canonical_id,
                    field="status",
                    source_a="shopify",
                    value_a=order.status,
                    source_b="quickbooks",
                    value_b=matched_invoice.status,
                    severity="medium",
                )
            )

    return discrepancies

The pipeline follows these steps:

Pull fresh data from each API (Shopify orders, QuickBooks invoices, CRM contacts)
Normalize everything into canonical objects using the per-system normalizers
Match records across systems using IDs, emails, or scoring
Compare matched records field by field
Flag discrepancies with severity levels
Push corrections back to the appropriate system or queue them for review

Not every discrepancy should be auto-corrected. I categorize them by severity. A missing invoice in QuickBooks is high severity and might be auto-created. A minor name spelling difference is low severity and goes into a review queue. Total mismatches always get flagged for human review because they could indicate a real accounting issue.

Bank Reconciliation with Plaid

One area where this approach pays off immediately is bank reconciliation. We have built pipelines for clients that pull transaction data from Plaid and match it against expected deposits from Shopify payouts and recorded revenue in QuickBooks.

The logic is similar. Normalize Plaid transactions into a canonical format, then match them against expected payouts by date range and amount. The tricky part is that payment processors batch transactions, so a single bank deposit might correspond to dozens of individual orders minus processing fees. You have to account for that grouping and the fee deductions when comparing amounts.

This is where Decimal arithmetic matters. Floating-point rounding errors will create phantom discrepancies across thousands of records. Always use Decimal for money.

Conflict Resolution: Deciding Which System Wins

When two systems disagree, you need a policy for which one is the source of truth. This varies by field:

Price and totals: the payment processor or accounting system wins (QuickBooks for invoices, Shopify for order amounts before fees)
Customer contact info: the most recently updated record wins
Order status: the system closest to the actual event wins (shipping platform for fulfillment status, payment processor for payment status)
Product data: the storefront wins (Shopify for SKUs and descriptions)

I encode these rules explicitly:

AUTHORITY_MAP = {
    "total": "quickbooks",
    "status": "shopify",
    "email": "most_recent",
    "shipping_status": "shipstation",
    "sku": "shopify",
    "description": "shopify",
}


def resolve_conflict(
    field: str,
    value_a: str,
    source_a: str,
    value_b: str,
    source_b: str,
    updated_a: datetime,
    updated_b: datetime,
) -> tuple[str, str]:
    authority = AUTHORITY_MAP.get(field)

    if authority == "most_recent":
        if updated_a >= updated_b:
            return value_a, source_a
        return value_b, source_b

    if authority == source_a:
        return value_a, source_a
    if authority == source_b:
        return value_b, source_b

    return value_a, source_a

Having this codified means the pipeline can auto-resolve most conflicts without human intervention. The ones it cannot resolve get queued with full context so the operations team can make a decision in seconds instead of minutes.

What This Looks Like in Production

For a typical client running Shopify plus QuickBooks plus a CRM, the reconciliation service runs as a scheduled Python job. It processes the last 48 hours of data on each run (overlapping windows catch late-arriving records), writes discrepancies to a database table, and sends a daily summary to Slack or email.

The first run usually surfaces hundreds of issues that have been accumulating for months. After the initial cleanup, the daily count drops to single digits. Most of those are legitimate timing differences (an order placed at 11:58 PM that does not hit QuickBooks until the next day) that resolve themselves on the next run.

The result is that the client's operations team stops spending hours on manual reconciliation and starts trusting their numbers again. When the CEO asks "what was revenue last month," the answer is the same regardless of which system you pull it from.

Wrapping Up

Data mismatch across business systems is not a tooling problem. It is an architecture problem. No off-the-shelf connector will solve it because the real work is in defining what "correct" means for your specific business and encoding those rules into a pipeline that runs reliably.

The pattern is always the same: canonical model, normalizers per system, matching logic, field-by-field comparison, and explicit conflict resolution. The specifics change for every client, but the structure does not.

If your team is burning time on manual reconciliation or you have stopped trusting the numbers coming out of your systems, this is exactly the kind of problem we solve at Syntora.

Built by Syntora -- automation infrastructure for growing businesses.

When to Move From Zapier and Make to Custom Python Automation

Syntora — Sat, 14 Feb 2026 01:18:55 +0000

Zapier and Make are genuinely good products. I have recommended them to clients, used them myself, and watched them save small teams hundreds of hours. If you are running a handful of automations that move data between two or three apps, those tools are the right choice. You should not be writing Python to send a Slack message when a form is submitted.

But there is a ceiling. I have watched dozens of businesses hit it, and the pattern is always the same: what started as five clean Zaps turns into forty tangled workflows with retry logic duct-taped onto error handlers, and a monthly bill that rivals a junior developer's salary.

This article is about recognizing that ceiling before you slam into it, and understanding what sits on the other side.

Five Signals You Have Outgrown No-Code Automation

1. You Are Hitting Rate Limits and Task Caps

Zapier's pricing is task-based. Every time a Zap fires, that counts. When you are processing hundreds or thousands of events per day, you burn through task allotments fast. Make uses operations the same way.

The real problem is not the bill (though we will get to that). The problem is that rate limits introduce silent failures. Your CRM sync missed 200 contacts because you hit your daily cap at 2 PM. Nobody noticed until Friday.

Custom Python does not have artificial task limits. A script running on a VPS processes as many events as the hardware allows. You pay for compute, not per execution.

2. Your Monthly Bill Exceeds $500

Here is a rough cost comparison that I walk through with clients regularly:

Scenario	Zapier/Make Cost	Custom Python Cost
5,000 tasks/month, basic workflows	$70-100/mo	Overkill, stay on Zapier
20,000 tasks/month, 30+ Zaps	$300-500/mo	$5-20/mo VPS
100,000 tasks/month, complex logic	$1,000-2,000/mo	$20-40/mo VPS
500,000+ tasks/month, data pipelines	$2,000+/mo	$40-80/mo VPS

The custom Python column assumes a basic VPS (DigitalOcean, Hetzner, or Railway) running your scripts. The infrastructure cost is almost always under $50/month, even at high volume. The real cost is the development time to build it, but that is a one-time investment that pays for itself within a few months at the higher tiers.

3. You Need Conditional Logic That No-Code Cannot Express

Zapier paths and Make routers handle simple branching. But when you need to evaluate data against business rules that involve lookups, calculations, or stateful decisions, you start fighting the visual builder instead of working with it.

I had a client who built a 47-step Zap with 12 paths to handle lead routing. It took 30 seconds to execute and failed silently when any upstream API was slow. The Python replacement was 80 lines and ran in under a second.

4. Data Transformation Gets Complex

No-code tools handle simple mapping well. Field A goes to Field B. But the moment you need to reshape nested JSON, aggregate records, deduplicate based on fuzzy matching, or transform data formats, you are writing JavaScript in Zapier's code steps anyway.

If you are already writing code inside your no-code tool, you have outgrown the no-code tool.

5. Error Handling Requires More Than a Retry

Zapier's error handling is: retry, then notify you. Make is slightly better with error routes. But neither gives you what production systems actually need: dead letter queues, partial failure recovery, idempotency guarantees, or structured logging that lets you debug an issue three weeks after it happened.

When a failure at step 7 of 12 means you need to roll back steps 3 through 6, no visual builder is going to save you.

What Custom Python Automation Actually Looks Like

Let me show you concrete examples of what replaces common Zapier patterns. These are simplified versions of real code I have built for clients at Syntora.

Replacing a Webhook-Triggered Zap

A common Zapier pattern: receive a webhook from Stripe, then create a record in your CRM and send a Slack notification.

In Python with FastAPI:

from fastapi import FastAPI, Request
import httpx

app = FastAPI()

@app.post("/webhooks/stripe")
async def handle_stripe_webhook(request: Request):
    payload = await request.json()
    event_type = payload.get("type", "")

    if event_type == "checkout.session.completed":
        session = payload["data"]["object"]
        customer_email = session["customer_details"]["email"]
        amount = session["amount_total"] / 100

        # Create CRM contact
        async with httpx.AsyncClient() as client:
            await client.post(
                "https://api.your-crm.com/contacts",
                json={
                    "email": customer_email,
                    "deal_value": amount,
                    "source": "stripe_checkout"
                },
                headers={"Authorization": "Bearer YOUR_CRM_KEY"}
            )

            # Notify the team
            await client.post(
                "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK",
                json={
                    "text": f"New sale: {customer_email} for ${amount:.2f}"
                }
            )

    return {"status": "ok"}

This handles the same flow as a 3-step Zap, but now you can add any logic you want: validation, deduplication, conditional routing, database lookups. No task limits, no per-execution billing.

Replacing a Scheduled Data Sync

Many businesses use Zapier's scheduled triggers to sync data between systems every 15 minutes. Here is the Python equivalent using a simple scheduler:

import schedule
import time
import httpx
import logging

logging.basicConfig(
    filename="sync.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

def sync_contacts():
    try:
        with httpx.Client() as client:
            # Pull new contacts from source
            response = client.get(
                "https://api.source-app.com/contacts",
                params={"updated_since": get_last_sync_time()},
                headers={"Authorization": "Bearer SOURCE_KEY"}
            )
            contacts = response.json()["data"]

            if not contacts:
                logging.info("No new contacts to sync")
                return

            # Transform and push to destination
            transformed = [
                {
                    "full_name": f"{c['first_name']} {c['last_name']}",
                    "email": c["email"].lower().strip(),
                    "company": c.get("organization", "Unknown"),
                    "value": calculate_lead_score(c)
                }
                for c in contacts
            ]

            result = client.post(
                "https://api.dest-app.com/contacts/batch",
                json={"contacts": transformed},
                headers={"Authorization": "Bearer DEST_KEY"}
            )

            logging.info(f"Synced {len(transformed)} contacts: {result.status_code}")
            save_last_sync_time()

    except Exception as e:
        logging.error(f"Sync failed: {e}")
        send_alert(f"Contact sync failed: {e}")

schedule.every(15).minutes.do(sync_contacts)

while True:
    schedule.run_pending()
    time.sleep(1)

Notice what you get for free here: structured logging, error handling with alerts, data transformation that would be painful in a visual builder, and batch operations that reduce API calls.

Replacing Complex Data Transformation

This is where Python really pulls ahead. Here is a real pattern: pulling order data from one system, enriching it with inventory data from another, and pushing a summary report.

from collections import defaultdict

def build_daily_report(orders, inventory):
    # Group orders by product
    product_totals = defaultdict(lambda: {"units": 0, "revenue": 0.0})

    for order in orders:
        for item in order["line_items"]:
            sku = item["sku"]
            product_totals[sku]["units"] += item["quantity"]
            product_totals[sku]["revenue"] += item["quantity"] * item["unit_price"]

    # Enrich with inventory data
    report_rows = []
    for sku, totals in product_totals.items():
        stock = inventory.get(sku, {})
        report_rows.append({
            "sku": sku,
            "units_sold": totals["units"],
            "revenue": round(totals["revenue"], 2),
            "current_stock": stock.get("quantity", 0),
            "reorder_needed": stock.get("quantity", 0) < totals["units"] * 3,
            "days_of_stock": (
                round(stock.get("quantity", 0) / totals["units"], 1)
                if totals["units"] > 0 else None
            )
        })

    # Sort by revenue descending
    report_rows.sort(key=lambda r: r["revenue"], reverse=True)
    return report_rows

Try building that in Zapier. You would need multiple Zaps, a database in between, and Zapier's code steps straining under the weight of business logic they were never designed to carry.

The Migration Does Not Have to Be All-or-Nothing

One misconception I run into constantly: people think moving to custom Python means ripping out every Zap on day one. That is not how we approach it.

The practical path looks like this:

Phase 1: Identify your most expensive or fragile automations. These are the ones failing regularly, burning through tasks, or requiring code steps that are hard to maintain. Migrate those first.

Phase 2: Build a small Python service that handles the heavy lifting. This might be a single FastAPI app on a VPS that processes webhooks and runs scheduled jobs. Keep your simple Zaps running for the stuff that genuinely works fine.

Phase 3: Gradually consolidate. As you add capabilities to your Python service, you can retire Zaps one by one. No rush, no big-bang migration.

This phased approach means you are never without automation during the transition, and you can validate the custom solution against real workloads before going all-in.

What You Need to Run Custom Python in Production

The infrastructure is simpler than most people expect:

A VPS ($5-20/month on DigitalOcean, Hetzner, or Railway)
Supervisor or systemd to keep your processes running
A logging solution (even just structured log files to start)
A simple deployment process (git pull and restart, or a basic CI/CD pipeline)
Monitoring (a health check endpoint and uptime monitoring like UptimeRobot)

You do not need Kubernetes. You do not need microservices. You do not need a team of DevOps engineers. A single Python process on a $10/month VPS can handle more throughput than most businesses will ever need from their automations.

When to Stay on Zapier

To be clear about when no-code is still the right answer:

You have fewer than 20 automations and they are straightforward
Your total monthly cost is under $200 and predictable
You do not have anyone on the team who can maintain Python
Your automations do not require complex error recovery
Speed of setup matters more than long-term cost

There is no shame in using the right tool for the job. Zapier and Make are the right tools for a lot of jobs. They just stop being the right tool when your automation needs cross a complexity and volume threshold.

The Bottom Line

No-code automation tools solve a real problem: they let non-technical teams automate workflows without hiring developers. That value is real and I am not dismissing it.

But when you are spending $1,000+ per month on Zapier, debugging 40-step Zaps at midnight, and losing data to silent failures, you have crossed the line where custom infrastructure costs less and does more. A Python service on a cheap VPS, built once and maintained incrementally, will outperform a tangle of no-code workflows in reliability, cost, and capability.

The question is not whether to make the move. The question is whether you have already passed the point where you should have.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io

Automating Client Onboarding with Supabase Edge Functions and Webhooks

Syntora — Sat, 14 Feb 2026 00:56:55 +0000

Every time I signed a new client, I did the same 15-20 minutes of busywork. Create a record. Set up their folder structure. Provision portal access. Send a welcome email. Log everything. Five steps, every single time, all manual.

I run Syntora, a small automation consultancy. The irony of manually onboarding clients at an automation company was not lost on me. So I built a system where signing a proposal triggers all five steps automatically, with zero clicks after the signature.

This article walks through the full build: the database schema, the Edge Function that orchestrates everything, the webhook trigger, error handling for partial failures, and RLS policies for client isolation. If you run any kind of agency or SaaS and onboard clients regularly, this pattern will save you serious time.

The Architecture

The flow is simple:

Client signs a proposal (hosted on an external platform).
The signing platform fires a webhook to a Supabase Edge Function.
The Edge Function orchestrates five steps in sequence.
Every step is logged to an activity log table.
If any step fails, the function logs the failure and continues with what it can.

Everything lives in Supabase: Postgres for data, Auth for portal access, Edge Functions for orchestration.

Step 1: The Database Schema

We need four tables: clients, projects, portal_users, and activity_log. Here is the full migration.

-- clients: the core record created on signature
create table public.clients (
  id uuid primary key default gen_random_uuid(),
  name text not null,
  email text not null unique,
  company text,
  status text not null default 'active'
    check (status in ('active', 'paused', 'churned')),
  proposal_signed_at timestamptz,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now()
);

-- projects: folder/workspace structure per client
create table public.projects (
  id uuid primary key default gen_random_uuid(),
  client_id uuid not null references public.clients(id) on delete cascade,
  name text not null,
  slug text not null,
  status text not null default 'setup'
    check (status in ('setup', 'active', 'completed', 'archived')),
  config jsonb not null default '{}'::jsonb,
  created_at timestamptz not null default now(),
  updated_at timestamptz not null default now(),
  unique(client_id, slug)
);

-- portal_users: links Supabase Auth users to clients
create table public.portal_users (
  id uuid primary key default gen_random_uuid(),
  auth_user_id uuid not null references auth.users(id) on delete cascade,
  client_id uuid not null references public.clients(id) on delete cascade,
  role text not null default 'viewer'
    check (role in ('owner', 'editor', 'viewer')),
  invited_at timestamptz not null default now(),
  accepted_at timestamptz,
  unique(auth_user_id, client_id)
);

-- activity_log: append-only audit trail
create table public.activity_log (
  id uuid primary key default gen_random_uuid(),
  client_id uuid references public.clients(id) on delete set null,
  action text not null,
  details jsonb not null default '{}'::jsonb,
  status text not null default 'success'
    check (status in ('success', 'failure', 'skipped')),
  error_message text,
  created_at timestamptz not null default now()
);

-- indexes for common queries
create index idx_projects_client_id on public.projects(client_id);
create index idx_portal_users_client_id on public.portal_users(client_id);
create index idx_portal_users_auth_id on public.portal_users(auth_user_id);
create index idx_activity_log_client_id on public.activity_log(client_id);
create index idx_activity_log_created_at on public.activity_log(created_at desc);

The activity_log table is the most important piece for debugging. Every step in the onboarding flow writes a row here, whether it succeeds or fails. The details jsonb column holds whatever context is relevant: the email that was sent, the project config that was created, the error stack if something broke.

Step 2: RLS Policies for Client Isolation

Every client should only see their own data. This is non-negotiable in a multi-tenant system. Here are the RLS policies.

-- enable RLS on all tables
alter table public.clients enable row level security;
alter table public.projects enable row level security;
alter table public.portal_users enable row level security;
alter table public.activity_log enable row level security;

-- portal users can read their own client record
create policy "clients_select_own" on public.clients
  for select using (
    id in (
      select client_id from public.portal_users
      where auth_user_id = auth.uid()
    )
  );

-- portal users can read projects for their client
create policy "projects_select_own" on public.projects
  for select using (
    client_id in (
      select client_id from public.portal_users
      where auth_user_id = auth.uid()
    )
  );

-- portal users can only see their own portal_users row
create policy "portal_users_select_own" on public.portal_users
  for select using (auth_user_id = auth.uid());

-- activity log: clients see logs for their own client_id
create policy "activity_log_select_own" on public.activity_log
  for select using (
    client_id in (
      select client_id from public.portal_users
      where auth_user_id = auth.uid()
    )
  );

Notice that every policy routes through the portal_users table. That table is the single join between a Supabase Auth user and the client they belong to. This keeps the authorization model clean: one lookup, consistent across all tables.

The Edge Function itself runs with the service_role key, so it bypasses RLS when inserting records during onboarding. RLS only applies to client-facing queries through the Supabase client library.

Step 3: The Edge Function

This is the core of the system. A single Edge Function receives the webhook, validates the payload, and runs all five onboarding steps in sequence.

import "jsr:@supabase/functions-js/edge-runtime.d.ts";
import { createClient } from "jsr:@supabase/supabase-js@2";

interface WebhookPayload {
  event: string;
  client_name: string;
  client_email: string;
  company: string;
  signed_at: string;
}

interface OnboardingContext {
  supabase: ReturnType<typeof createClient>;
  clientId: string | null;
  payload: WebhookPayload;
}

async function logActivity(
  ctx: OnboardingContext,
  action: string,
  status: "success" | "failure" | "skipped",
  details: Record<string, unknown> = {},
  errorMessage?: string
) {
  await ctx.supabase.from("activity_log").insert({
    client_id: ctx.clientId,
    action,
    status,
    details,
    error_message: errorMessage ?? null,
  });
}

// Step 1: Create client record
async function createClientRecord(ctx: OnboardingContext) {
  const { data, error } = await ctx.supabase
    .from("clients")
    .insert({
      name: ctx.payload.client_name,
      email: ctx.payload.client_email,
      company: ctx.payload.company,
      proposal_signed_at: ctx.payload.signed_at,
    })
    .select("id")
    .single();

  if (error) throw new Error(`Client insert failed: ${error.message}`);

  ctx.clientId = data.id;
  await logActivity(ctx, "client_created", "success", {
    email: ctx.payload.client_email,
  });

  return data.id;
}

// Step 2: Create project structure
async function createProjectStructure(ctx: OnboardingContext) {
  const slug = ctx.payload.company
    .toLowerCase()
    .replace(/[^a-z0-9]+/g, "-")
    .replace(/^-|-$/g, "");

  const defaultProjects = [
    {
      client_id: ctx.clientId,
      name: "Primary Project",
      slug: `${slug}-primary`,
      config: { channels: ["email", "portal"], timezone: "America/Chicago" },
    },
  ];

  const { error } = await ctx.supabase
    .from("projects")
    .insert(defaultProjects);

  if (error) throw new Error(`Project setup failed: ${error.message}`);

  await logActivity(ctx, "project_created", "success", {
    projects: defaultProjects.map((p) => p.slug),
  });
}

// Step 3: Provision portal access via Supabase Auth
async function provisionPortalAccess(ctx: OnboardingContext) {
  const { data: authData, error: authError } =
    await ctx.supabase.auth.admin.inviteUserByEmail(
      ctx.payload.client_email,
      {
        data: {
          client_id: ctx.clientId,
          role: "owner",
          invited_by: "onboarding_automation",
        },
        redirectTo: `${Deno.env.get("PORTAL_URL")}/welcome`,
      }
    );

  if (authError)
    throw new Error(`Auth invite failed: ${authError.message}`);

  const { error: portalError } = await ctx.supabase
    .from("portal_users")
    .insert({
      auth_user_id: authData.user.id,
      client_id: ctx.clientId,
      role: "owner",
    });

  if (portalError)
    throw new Error(`Portal user insert failed: ${portalError.message}`);

  await logActivity(ctx, "portal_access_provisioned", "success", {
    auth_user_id: authData.user.id,
  });
}

// Step 4: Send welcome email
async function sendWelcomeEmail(ctx: OnboardingContext) {
  const response = await fetch("https://api.resend.com/emails", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${Deno.env.get("RESEND_API_KEY")}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      from: "Parker <parker@syntora.io>",
      to: ctx.payload.client_email,
      subject: `Welcome to Syntora, ${ctx.payload.client_name}`,
      html: `
        <p>Hi ${ctx.payload.client_name},</p>
        <p>Thanks for signing with Syntora. Your portal access is being set up now.
        You will receive a separate email with your login link shortly.</p>
        <p>In the meantime, here is what happens next:</p>
        <ul>
          <li>Your project workspace is ready</li>
          <li>Portal access is provisioned (check your email for the invite)</li>
          <li>I will reach out within 24 hours to schedule our kickoff</li>
        </ul>
        <p>Looking forward to working together.</p>
        <p>Parker Gawne<br/>Syntora</p>
      `,
    }),
  });

  if (!response.ok) {
    const body = await response.text();
    throw new Error(`Email send failed: ${body}`);
  }

  await logActivity(ctx, "welcome_email_sent", "success", {
    to: ctx.payload.client_email,
  });
}

// Step 5: Final log entry
async function logOnboardingComplete(ctx: OnboardingContext) {
  await logActivity(ctx, "onboarding_complete", "success", {
    client_id: ctx.clientId,
    email: ctx.payload.client_email,
    company: ctx.payload.company,
  });
}

Deno.serve(async (req: Request) => {
  // Validate webhook secret
  const webhookSecret = req.headers.get("x-webhook-secret");
  if (webhookSecret !== Deno.env.get("ONBOARDING_WEBHOOK_SECRET")) {
    return new Response(JSON.stringify({ error: "Unauthorized" }), {
      status: 401,
    });
  }

  const payload: WebhookPayload = await req.json();

  if (payload.event !== "proposal.signed") {
    return new Response(JSON.stringify({ status: "ignored" }), {
      status: 200,
    });
  }

  const supabase = createClient(
    Deno.env.get("SUPABASE_URL")!,
    Deno.env.get("SUPABASE_SERVICE_ROLE_KEY")!
  );

  const ctx: OnboardingContext = {
    supabase,
    clientId: null,
    payload,
  };

  const steps = [
    { name: "create_client", fn: createClientRecord, critical: true },
    { name: "create_project", fn: createProjectStructure, critical: false },
    { name: "provision_portal", fn: provisionPortalAccess, critical: false },
    { name: "send_welcome_email", fn: sendWelcomeEmail, critical: false },
    { name: "log_completion", fn: logOnboardingComplete, critical: false },
  ];

  const results: Record<string, string> = {};

  for (const step of steps) {
    try {
      await step.fn(ctx);
      results[step.name] = "success";
    } catch (err) {
      const message = err instanceof Error ? err.message : String(err);
      results[step.name] = "failure";

      await logActivity(ctx, step.name, "failure", {}, message);

      // If a critical step fails, abort the rest
      if (step.critical) {
        return new Response(
          JSON.stringify({
            status: "failed",
            failed_step: step.name,
            error: message,
            results,
          }),
          { status: 500 }
        );
      }
    }
  }

  return new Response(JSON.stringify({ status: "completed", results }), {
    status: 200,
    headers: { "Content-Type": "application/json" },
  });
});

Handling Partial Failures

The most important design decision in this function is the critical flag on each step. Step 1 (creating the client record) is critical. If it fails, nothing else can proceed because every subsequent step depends on the client_id. But steps 2 through 5 are independent enough to continue if one fails.

If step 3 (portal access) throws an error but steps 1 and 2 succeeded, the function logs the failure and keeps going. The welcome email still sends. The completion log still writes. I get a failure record in the activity log, and I can manually fix the portal access later.

This is better than the alternative: wrapping everything in a single transaction that rolls back entirely. In onboarding workflows, a partial success is almost always better than a full rollback. The client record exists, the project is set up, and I just need to re-run the failed step.

Step 4: The Webhook Trigger

The webhook configuration depends on your proposal platform. Most tools (PandaDoc, DocuSign, Proposify, HoneyBook) support webhook callbacks on signature events. The setup is the same across all of them:

Set the webhook URL to your Edge Function endpoint:
https://<project-ref>.supabase.co/functions/v1/onboard-client
Set a shared secret header (x-webhook-secret) that your function validates.
Map the payload fields to match the WebhookPayload interface.

If your proposal tool sends a different payload shape, add a normalization layer at the top of the Edge Function that maps their fields to yours. Keep the core orchestration code clean.

To deploy the function with the Supabase CLI:

supabase functions deploy onboard-client

Then set your secrets:

supabase secrets set ONBOARDING_WEBHOOK_SECRET=your-secret-here
supabase secrets set RESEND_API_KEY=re_your-key-here
supabase secrets set PORTAL_URL=https://portal.yourdomain.com

Step 5: Querying the Activity Log

Once this is running, the activity log becomes your single source of truth for onboarding status. Here are a few useful queries.

Check the full onboarding history for a client:

select action, status, details, error_message, created_at
from activity_log
where client_id = '<client-uuid>'
order by created_at asc;

Find all failed steps across all clients (useful for a daily check):

select
  c.name as client_name,
  al.action,
  al.error_message,
  al.created_at
from activity_log al
join clients c on c.id = al.client_id
where al.status = 'failure'
order by al.created_at desc;

Get a count of onboardings by status over the last 30 days:

select
  status,
  count(*) as total
from activity_log
where action = 'onboarding_complete'
  and created_at > now() - interval '30 days'
group by status;

What I Learned

Log everything, even successes. The activity log is not just for debugging failures. It is a full audit trail. When a client asks "when was my account set up?" you have the answer in one query.

Partial success beats full rollback. For workflow orchestration (as opposed to financial transactions), continuing through non-critical failures and fixing them later is almost always the right call. Your activity log tells you exactly what needs manual attention.

RLS through a single join table. Routing all authorization through portal_users keeps the policy definitions simple and consistent. Adding a new table to the system means writing one policy that joins to portal_users, and you are done.

Edge Functions are the right fit for webhooks. They cold-start fast, run close to your database, and share the same auth infrastructure. For orchestration tasks like this, where you are calling your own database and a couple external APIs, they are the simplest option.

The full onboarding now takes about 3 seconds from proposal signature to completion. No clicking, no copy-pasting, no forgetting steps. The activity log catches anything that goes wrong, and I fix it once instead of discovering it three days later when a client asks why they cannot log in.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io

Building a Prompt Engineering Feedback Loop: The System That Made My AI Prompts 3x More Effective

Syntora — Sat, 14 Feb 2026 00:50:04 +0000

Most developers treat prompt engineering like a one-time skill. You read a guide, learn a few tricks, then wing it from there. That is how I started too. It did not work.

I run an AI automation agency. I use the Claude API and Claude Code daily for production systems, everything from generating content at scale to building full-stack features. When your prompts power revenue-generating infrastructure, "good enough" prompts cost real money in wasted tokens, bad outputs, and manual rework.

So I built a feedback loop. After three months of running it, I have 9 reusable prompt templates, 6 saved examples I reference constantly, and a documented list of anti-patterns that would have kept burning me. Here is the system.

The Rating Schema

After every meaningful AI session, I spend 60 seconds recording a rating. The key is making this fast enough that you actually do it.

Rating scale (1-5):

Score	Meaning	Trigger
1	Unusable	Output required complete rewrite or was factually wrong
2	Poor	Correct direction but needed heavy editing (>50% changed)
3	Acceptable	Usable with moderate edits, nothing surprising
4	Good	Minor tweaks only, output matched intent well
5	Excellent	Copy-paste ready, better than what I would have written manually

What gets recorded per session:

## 2026-02-10 | Score: 4 | Task: API endpoint generation

**Prompt type:** Code generation (Express handler)
**Model:** claude-sonnet-4-20250514
**What worked:** Giving it the existing handler pattern as a reference.
  Output matched project conventions perfectly.
**What failed:** Did not include error handling until I asked.
**Key insight:** Always include "follow the error handling pattern from
  [reference]" in code gen prompts.
**Template updated:** Yes, added to code-gen-v3.md

This takes less than a minute. The discipline is not in the writing. It is in doing it every single time, especially when the session goes well. You learn as much from a 5 as you do from a 1.

The Review Cadence

Weekly (15 minutes): Scan the last 7 entries. Look for two things: techniques that keep producing 4s and 5s, and failure modes that keep producing 1s and 2s. If you see the same insight three times, it is a pattern. Write it down.

Monthly (30 minutes): Update your templates. Patterns that proved out across multiple weeks get added to your prompt templates. Repeated failures get added to the anti-patterns list. Delete anything that stopped being useful.

Anti-Patterns: Prompts That Consistently Fail

These are specific patterns I tracked across dozens of sessions. Each one seemed reasonable but produced reliably poor results.

1. The Vague Scope Dump

Bad: "Build me a user authentication system with all the features
     a modern app would need."

Score: 1-2, every time.

This fails because "all the features" is unbounded. The model guesses at scope and inevitably picks the wrong features or implements them at the wrong depth. You get a sprawling mess that matches nothing you actually needed.

2. The Over-Constrained Micromanager

Bad: "Write a Python function that takes exactly two arguments,
     the first being a string of length 1-255, validates it
     using regex pattern ^[a-zA-Z0-9_]+$, raises ValueError
     with message 'Invalid input: {input}' if validation fails,
     logs to stdout using print() not logging, returns a dict
     with keys 'status' and 'result'..."

This looks thorough but it produces brittle, literal-minded code. The model follows your spec so precisely that it misses obvious improvements. Worse, you spend more time writing the prompt than you would writing the code.

3. The Missing Context Assumption

Bad: "Add pagination to the list endpoint."

This assumes the model knows your pagination style, your ORM, your response format, your frontend expectations. It does not. You get generic offset/limit pagination when you needed cursor-based, or you get a completely different response envelope than your other endpoints use.

4. The "Be Creative" Trap

Bad: "Write me a really creative and unique solution for caching
     API responses."

"Creative" is not a technical requirement. This produces over-engineered novelty code, often using obscure patterns the model is less reliable at implementing. Your caching layer does not need to be creative. It needs to work.

Proven Patterns: What Consistently Scores 4-5

1. Reference-Based Generation

Proven: "Here is an existing handler that follows our project
conventions:

[paste 30-50 lines of a real handler]

Write a new handler for the /users/:id/preferences endpoint
that follows the same patterns for error handling, response
format, and input validation."

This works because you are giving the model a concrete target instead of an abstract description. It matches style, structure, and conventions without you having to enumerate every rule.

2. Constraint Sandwich (Context, Task, Boundaries)

Proven: "Context: Python FastAPI service, SQLAlchemy ORM,
Pydantic v2 for schemas. We use repository pattern for
data access.

Task: Write a new endpoint POST /api/v1/reports that accepts
a date range and report type, queries the database, and
returns aggregated results.

Boundaries: Do not add any new dependencies. Use existing
database models. Keep the endpoint under 40 lines. Return
errors as HTTPException with appropriate status codes."

Context sets the environment. Task defines the deliverable. Boundaries prevent scope creep. This three-part structure consistently produces focused, usable output.

3. Iterative Refinement With Explicit Feedback

Instead of one mega-prompt, break it into steps and give explicit feedback:

Step 1: "Write the Pydantic schemas for a report request
and response."

[Review output, then:]

Step 2: "Good. The request schema is right. For the response,
change 'data' to 'rows' and add a 'generated_at' timestamp
field. Now write the repository method that queries the
database using these schemas."

This consistently outperforms trying to get everything right in a single shot. Each step is small enough that the model gets it right, and your corrections compound.

4. Anti-Pattern Fencing

Proven: "Write a database migration to add a 'status' column
to the orders table.

Do NOT: create a new table, modify existing columns, add
indexes (we will do that separately), or include seed data."

Explicitly stating what you do not want is as valuable as stating what you do. This eliminates the most common failure mode: the model "helping" by doing more than you asked.

Prompt Template: v1 vs v3

Here is a real template from my system. The v1 version is what I started with. The v3 version is what three months of feedback produced.

v1 (Naive):

# Code Generation Prompt

Write [description of what I need] in [language].
Make it production-ready.

v3 (After Feedback Loop):

# Code Generation Prompt v3

## Reference
[Paste 1 existing file that follows project conventions]

## Context
- Language/framework: [e.g., Python 3.12, FastAPI]
- ORM/DB: [e.g., SQLAlchemy 2.0, PostgreSQL]
- Project patterns: [e.g., repository pattern, dependency injection]

## Task
[One clear deliverable, 1-3 sentences max]

## Boundaries
- Do not add new dependencies
- Do not modify existing files unless specified
- Match the error handling pattern from the reference
- [Any other project-specific constraints]

## Output format
- Single file, ready to save
- Include type hints
- Include docstring with one-line description

What changed between v1 and v3:

Added Reference section. This was the single biggest improvement. Giving the model a real example to match cut my edit time by half.
Split "make it production-ready" into explicit Boundaries. "Production-ready" means different things to every developer. Spelling out the constraints removed ambiguity.
Added Output format. Specifying "single file, ready to save" eliminated the model's tendency to split code across multiple files or add explanatory text I had to strip out.
Removed vague qualifiers. No more "make it clean" or "make it good." Every instruction is specific and verifiable.

File Structure for Storing the System

prompt-engineering/
  ratings/
    2026-01.md          # Monthly rating logs
    2026-02.md
  templates/
    code-gen-v3.md       # Code generation (current)
    code-review-v2.md    # Code review checklist
    api-design-v1.md     # API endpoint design
    content-gen-v4.md    # Blog/marketing content
    migration-v2.md      # Database migrations
    bug-fix-v3.md        # Debugging assistance
    refactor-v2.md       # Code refactoring
    test-gen-v2.md       # Test writing
    data-transform-v1.md # Data pipeline scripts
  examples/
    great-code-gen.md    # Scored 5, reference prompt+output
    great-refactor.md    # Scored 5, complex refactor
    great-migration.md   # Scored 5, zero-downtime migration
    failed-vague.md      # Scored 1, lesson in specificity
    failed-scope.md      # Scored 1, scope explosion
    failed-creative.md   # Scored 2, over-engineered result
  anti-patterns.md       # Documented failure modes
  patterns.md            # Documented success patterns
  CHANGELOG.md           # Template version history

Everything is plain Markdown. No special tooling. I keep it in a git repo so template changes are tracked, but a folder on your desktop works fine as a starting point. The format matters less than the habit.

Applying This to Production API Calls

The feedback loop becomes even more valuable when you are making programmatic API calls. In a chat session, you can course-correct in real time. In production, a bad prompt runs hundreds of times before you notice.

Here is how I apply the system to API calls in Python:

import anthropic
from datetime import datetime

client = anthropic.Anthropic()

# Template loaded from your templates/ directory
PROMPT_TEMPLATE = """
## Reference
{reference_example}

## Context
Service: Content generation pipeline
Output format: JSON with keys: title, body, meta_description
Word count target: {word_count}

## Task
{task_description}

## Boundaries
- Output valid JSON only, no markdown wrapping
- Do not include placeholder text
- meta_description must be under 160 characters
"""

def generate_content(task: str, reference: str, word_count: int) -> dict:
    prompt = PROMPT_TEMPLATE.format(
        reference_example=reference,
        task_description=task,
        word_count=word_count,
    )

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=2048,
        messages=[{"role": "user", "content": prompt}],
    )

    return response

Key differences for production API prompts:

Version your templates. Store them as files, not inline strings. When you update a template, you can diff it against the previous version and measure whether outputs improved.
Log inputs and outputs. Every API call should log the prompt version, the input variables, and a quality score (automated or manual). This is your ratings/ data at scale.
A/B test template changes. When you update from v2 to v3, run both versions on the same inputs for a week. Compare output quality before fully switching.
Set up automated quality checks. For structured outputs, validate the schema. For content, check word count, reading level, and keyword presence. These automated scores supplement your manual ratings.

The same feedback loop applies. You review production logs weekly, identify which prompts produce the most rework, and update those templates first. The difference is that a 10% improvement in a production prompt template saves hundreds of manual corrections per month.

Getting Started

You do not need to build all of this on day one. Start with the rating habit. After every AI session, spend 60 seconds recording a score and one key insight. Do that for two weeks. The patterns will be obvious, and you will naturally start building templates from what works.

The compound effect is real. Three months in, I spend less time prompting and get better results than I did when I started. Not because I memorized tricks, but because I built a system that learns from every session.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io
(https://syntora.io)

Designing a CLAUDE.md Context System: How I Give AI Full Project Context Without Re-explaining Anything

Syntora — Sat, 14 Feb 2026 00:48:46 +0000

Every time I open Claude Code on a new project, it knows who I am, which client I'm working on, what tech stack the project uses, who the key people are, and what my code standards look like. I never re-explain any of it.

This is not magic. It is a file system.

I run Syntora, a small consultancy with multiple active clients. Each client has its own codebase, its own Supabase project, its own conventions. Switching between them used to mean spending the first five minutes of every session re-teaching Claude what "OD" means or reminding it not to use global singletons. That friction compounds. Over a full day of context-switching, I was losing 30+ minutes to re-orientation alone.

So I built a hierarchical context system using CLAUDE.md files. It draws from two ideas that have nothing to do with AI tooling.

The Two Ideas Behind the System

Real estate LLC isolation. In commercial real estate, the standard structure is one LLC per property with a parent LLC governing all of them. Each property is legally isolated, but the parent entity sets universal rules. If one property has a liability issue, the others are protected. The parent LLC does not contain property-specific details. It contains the rules that apply everywhere.

Monorepos are about shared context. A backend architect I work with made this observation: "Monorepos aren't about code, they're about shared context." The value of a monorepo is not that the code lives together. It is that developers share a single source of truth for conventions, types, and standards. When you change a shared interface, every consumer sees it immediately.

These two ideas gave me the architecture: isolated client folders (like separate LLCs) governed by a shared context layer (like the parent LLC), with on-demand deep storage for details that do not need to be in memory at all times.

The Three Layers

Layer 1: Global Context (Always Loaded)

Location: ~/.claude/CLAUDE.md

Claude Code automatically reads this file every time it starts, regardless of which directory you are in. This is your hot cache. It should contain only what Claude needs on every single session.

Here is a sanitized version of mine:

# Your Name - Company

## Identity
Founder at CompanyName. Consultancy focus. Central Time.

## Active Clients
| Short | Client           | Supabase ID          |
|-------|------------------|----------------------|
| CL1   | Client One       | abc123def456         |
| CL2   | Client Two       | ghi789jkl012         |

-> Details: ~/.claude/memory/clients/

## Key People
| Who    | Context                           |
|--------|-----------------------------------|
| Alice  | CL1 Product Owner                 |
| Bob    | CL2 Backend Lead                  |

## Preferences
- Fast, direct, no fluff
- Infrastructure over instructions
- One thing per file, use dependency injection
- NO em-dashes, NO emojis

## Folder Map
Desktop/Company/
  Clients/        -> Client projects
  Internal/       -> Company operations

## Shell Shortcuts
| Command | Path                              |
|---------|-----------------------------------|
| cl1     | Clients/client-one/               |
| cl2     | Clients/client-two/               |

## Critical Rules
1. Never commit .env or credentials
2. Use Context7 MCP for external library docs
3. Always verify build + test after changes

Notice what is NOT here: detailed tech stacks, API endpoints, database schemas, deployment procedures. Those belong in Layer 2. The global file is a routing table. It tells Claude who I am, who the clients are, what my universal preferences look like, and where to find everything else.

The -> Details: ~/.claude/memory/clients/ pointers are important. Claude Code follows these references. When a conversation requires deeper client context, I can tell it to read the relevant memory file and it knows exactly where to look.

Keep this file under 100 lines. Every token in the global context is loaded on every session. If you put your entire project documentation here, you are burning context window on information that is irrelevant 80% of the time.

Layer 2: Project Context (Loaded Per Directory)

Location: <project-root>/CLAUDE.md

When you open Claude Code inside a project directory, it reads both the global ~/.claude/CLAUDE.md and the local CLAUDE.md in that directory. This is where project-specific context lives.

A typical project-level file looks like this:

# Client One - Frontend

## Stack
- Next.js 14, TypeScript, Tailwind
- Supabase (auth, database, storage)
- Express.js backend on port 3001

## Architecture
- Dependency injection throughout
- One component per file
- Interfaces for all external dependencies

## Commands
| Command          | What It Does            |
|------------------|------------------------|
| npm run dev      | Dev server on port 3000 |
| npm run validate | Lint + build + test     |

## Key Patterns
- All API calls go through src/lib/api-client.ts
- Auth context provided via AuthProvider wrapper
- Database types auto-generated, never hand-written

## Current Sprint
- Feature: conversational AI assistant
- Branch naming: feature/scout-{description}

This file can be longer than the global one because it only loads when you are actually working in that project. I keep mine between 50 and 150 lines depending on the project's complexity.

The critical thing here: client folders stay completely separate. When I am working in Client One's directory, Claude has zero context about Client Two's codebase, conventions, or data. This is security by architecture, not by policy. There is no prompt that says "don't mix up clients." The file system makes it structurally impossible.

Layer 3: Deep Storage (Loaded On Demand)

Location: ~/.claude/memory/

This is the reference library. It contains information that is useful but not needed on every session.

~/.claude/memory/
  glossary.md          <- Acronyms and shorthand
  clients/
    client-one.md      <- Full client profile
    client-two.md      <- Full client profile
  people/
    alice.md           <- Contact details, preferences
    bob.md             <- Communication style, role
  context/
    company.md         <- Company overview
    tech-standards.md  <- Universal code standards

My glossary file decodes every acronym I use. When I type "check the RLS policies on the OD Supabase project," Claude knows RLS means Row Level Security and OD means Open Decision because the global context points to this glossary.

The tech standards file contains universal rules like dependency injection patterns, file organization conventions, and stack details per project. It only gets loaded when Claude needs to make architectural decisions.

The key principle: reference, don't inline. The global context file points to deep storage. It does not duplicate it. This keeps the hot cache small while making the full knowledge base accessible.

How Context Scoping Prevents Bleed

The isolation model is what makes this work for multiple clients. Here is how the layers interact depending on where I am working:

Working in Client One's directory:

Loaded: Global CLAUDE.md + Client One's CLAUDE.md
Available on demand: Deep storage (glossary, tech standards)
Not loaded: Client Two's CLAUDE.md, Client Three's CLAUDE.md

Working in Internal/Syntora operations:

Loaded: Global CLAUDE.md + Internal CLAUDE.md
Available on demand: Deep storage
Not loaded: Any client-specific context

Opening Claude Code from home directory:

Loaded: Global CLAUDE.md only
Available on demand: Deep storage
Not loaded: Any project or client context

This means I can never accidentally reference Client Two's database schema while working on Client One. I can never accidentally apply Client One's code conventions to Client Two's codebase. The isolation is structural.

Real Workflow Example

Here is what a typical context switch looks like:

I finish work on Client One. Close the terminal.
I open a new terminal in Client Two's directory.
Claude Code starts. It reads the global context (knows who I am, my preferences, all client shorthand). It reads Client Two's CLAUDE.md (knows the stack, conventions, current sprint).
I type: "Add input validation to the submission endpoint."
Claude already knows Client Two uses FastAPI, that validation should use Pydantic models, that tests go in tests/, and that I want dependency injection. It writes the code correctly on the first try.

No preamble. No "here is my project structure." No copy-pasting README files into the chat. The context system handled all of it before I typed a single word.

Starter Template

If you want to build this yourself, here is the minimal structure:

1. Create the global context file:

mkdir -p ~/.claude
touch ~/.claude/CLAUDE.md

2. Populate it with your identity and preferences:

# Your Name

## Identity
Your role. Your timezone.

## Preferences
- List your coding style preferences
- List your communication preferences
- List your absolute rules (things AI should never do)

## Folder Map
Where your projects live.

## Critical Rules
1. Security rules (never commit secrets)
2. Architecture rules (DI, file organization)
3. Tool rules (which MCP servers to use)

3. Create deep storage for reference material:

mkdir -p ~/.claude/memory/context

Add a glossary.md for acronyms and a tech-standards.md for universal coding conventions.

4. Add a CLAUDE.md to each project:

# Project Name

## Stack
What technologies this project uses.

## Commands
How to run, build, test, deploy.

## Architecture
Key patterns and conventions specific to this project.

## Current Focus
What you are actively working on.

That is the entire system. Four files minimum, scaling to as many as you need.

Scaling to More Clients

When I onboard a new client at Syntora, the context system scales with three steps:

Add one row to the Active Clients table in the global CLAUDE.md.
Create the client's project directory with its own CLAUDE.md.
Optionally add a client profile to ~/.claude/memory/clients/.

The global file grows by one line. The deep storage grows by one file. The existing clients are completely unaffected. There is no monolithic config file that gets unwieldy at five clients. The system scales linearly.

What I Got Wrong at First

My first version put too much in the global file. It was 300+ lines with full tech stacks, all deployment procedures, and detailed client profiles. Claude Code loaded all of it on every session, wasting context window on information that was only relevant 10% of the time.

The fix was aggressive: I cut the global file to under 100 lines and moved everything else to deep storage or project-level files. The rule I follow now is simple. If the information is needed on every session regardless of project, it goes in the global file. If it is needed only when working on a specific project, it goes in that project's CLAUDE.md. If it is reference material that is occasionally useful, it goes in deep storage.

Why This Matters

The developers who get the most out of AI coding tools are not the ones writing better prompts. They are the ones building better context infrastructure. A well-structured CLAUDE.md system means Claude starts every session with the equivalent of a senior team member's institutional knowledge. It knows the codebase, the conventions, the people, and the priorities.

That is not a productivity hack. It is infrastructure. And infrastructure compounds.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io
(https://syntora.io)

Building an AI Proposal Pipeline: From Call Transcripts to Branded Web Pages with Python and Supabase

Syntora — Sat, 14 Feb 2026 00:39:18 +0000

Every client engagement at my agency starts the same way: a 30-minute discovery call, followed by me spending 2-3 hours writing a proposal. Scoping phases, listing deliverables, pricing line items, formatting it into something presentable. It was the most repetitive part of running the business.

So I built a pipeline that turns a call recording into a branded, client-facing proposal page. The whole flow takes about 20 minutes now, and most of that is me reviewing the output and adjusting pricing.

This post walks through the full technical implementation: the Python extraction layer, the Claude API integration, the Supabase schema, and the Next.js rendering. Everything here is production code (simplified for clarity).

Pipeline Overview

The system has six stages:

Discovery call gets recorded and transcribed (I use Otter.ai, but any transcription service works).
A Python script extracts structured data from the raw transcript.
Claude generates a full proposal JSON with scope, phases, and deliverables.
I review the output in a simple terminal UI and adjust pricing.
One click publishes the proposal to Supabase.
The client receives a link to a branded proposal page rendered by Next.js.

The core principle: every step produces a well-defined data structure that the next step consumes. No magic, no ambiguity between stages.

Step 1: Transcript Ingestion

Nothing fancy here. I export the transcript as a .txt file and drop it into a watched directory. The script picks it up and reads it in.

from pathlib import Path

def load_transcript(file_path: str) -> str:
    path = Path(file_path)
    if not path.exists():
        raise FileNotFoundError(f"Transcript not found: {file_path}")

    text = path.read_text(encoding="utf-8")
    return text.strip()

If you are using an API-based transcription service (Deepgram, AssemblyAI), you can skip the file step and pipe the transcript string directly into the next stage.

Step 2: Extracting Key Info with Claude

This is where the pipeline gets interesting. I send the raw transcript to Claude with a structured extraction prompt. The goal is to pull out everything I need to scope a proposal: what the client wants, what their current setup looks like, technical constraints, timeline expectations, and budget signals.

import anthropic
import json

client = anthropic.Anthropic()

EXTRACTION_PROMPT = """You are an expert business analyst. Extract the following
structured information from this discovery call transcript.

Return valid JSON with these fields:
- company_name: string
- contact_name: string
- contact_email: string (if mentioned, else null)
- current_stack: list of strings (technologies they currently use)
- pain_points: list of strings (specific problems they described)
- desired_outcomes: list of strings (what success looks like to them)
- timeline: string (any timeline mentioned)
- budget_signals: string (any budget context)
- project_type: one of [automation, web_app, data_pipeline, integration, other]
- raw_requirements: list of strings (specific features or deliverables they asked for)
- red_flags: list of strings (scope creep risks, unrealistic expectations, etc.)

Only include information that was explicitly stated or strongly implied.
Do not fabricate details."""


def extract_call_data(transcript: str) -> dict:
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=2000,
        messages=[
            {
                "role": "user",
                "content": EXTRACTION_PROMPT + "\n\nTranscript:\n" + transcript
            }
        ]
    )

    response_text = message.content[0].text
    return json.loads(response_text.strip())

A few things worth noting:

I use claude-sonnet-4-20250514 for extraction. It is fast, cheap, and accurate for structured extraction tasks. You do not need Opus here.
The red_flags field has saved me from bad engagements more than once. Claude is surprisingly good at catching scope creep signals in natural conversation.

Step 3: Generating the Proposal JSON

With the extracted data in hand, a second Claude call generates the full proposal structure. I keep extraction and generation as separate calls because they are fundamentally different tasks, and separating them makes each prompt simpler and more reliable.

PROPOSAL_PROMPT = """You are a senior technical consultant writing a project proposal.
Given the following extracted call data, generate a complete proposal JSON.

Rules:
- Break the project into logical phases (2-4 phases typical)
- Each phase has deliverables with descriptions
- Include estimated hours per deliverable (be conservative)
- Do NOT include pricing. I will add that manually.
- Write deliverable descriptions in plain English, not jargon

Return valid JSON with: title, summary, phases (each with name,
description, deliverables), assumptions, out_of_scope, timeline_estimate."""


def generate_proposal(call_data: dict) -> dict:
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=4000,
        messages=[
            {
                "role": "user",
                "content": PROPOSAL_PROMPT + "\n\nCall data:\n" + json.dumps(call_data, indent=2)
            }
        ]
    )

    response_text = message.content[0].text
    proposal = json.loads(response_text.strip())

    # Inject metadata from the extraction step
    proposal["company_name"] = call_data["company_name"]
    proposal["contact_name"] = call_data["contact_name"]
    proposal["project_type"] = call_data["project_type"]

    return proposal

Here is what a real proposal JSON looks like after generation (abbreviated):

{
  "title": "CRM Integration and Automated Reporting Pipeline",
  "company_name": "Acme Corp",
  "contact_name": "Jane Smith",
  "project_type": "integration",
  "summary": "Build an automated pipeline connecting HubSpot CRM to the internal reporting dashboard, eliminating manual CSV exports and reducing reporting lag from 3 days to real-time.",
  "phases": [
    {
      "name": "Discovery and Architecture",
      "description": "Audit the current HubSpot configuration and design the integration architecture.",
      "deliverables": [
        {
          "name": "Data Flow Diagram",
          "description": "Visual map of all data moving between HubSpot and downstream dashboards.",
          "estimated_hours": 4
        },
        {
          "name": "Technical Specification",
          "description": "Detailed spec covering API endpoints, data transformations, and error handling.",
          "estimated_hours": 6
        }
      ]
    },
    {
      "name": "Core Integration Build",
      "description": "Implement the data pipeline with scheduling, error handling, and monitoring.",
      "deliverables": [
        {
          "name": "HubSpot API Connector",
          "description": "Python service that pulls contact, deal, and activity data on a configurable schedule.",
          "estimated_hours": 12
        }
      ]
    }
  ],
  "assumptions": [
    "Acme provides API credentials and a staging HubSpot environment",
    "Data volume is under 100k records per sync"
  ],
  "out_of_scope": [
    "Modifications to the existing reporting dashboard UI",
    "Historical data backfill beyond 12 months"
  ],
  "timeline_estimate": "4-6 weeks"
}

At this point I open the JSON in my terminal, review the phases, and manually add pricing. This is the one step I refuse to automate. Pricing requires judgment that depends on client relationship, strategic value, and context that no LLM has.

Step 4: The Supabase Schema

The database schema is straightforward. A proposals table holds the top-level metadata, and a proposal_phases table holds the phase/deliverable breakdown. I use Supabase's built-in jsonb support for deliverables since they are always read as a unit with their parent phase.

create table public.proposals (
  id uuid default gen_random_uuid() primary key,
  slug text unique not null,
  company_name text not null,
  contact_name text not null,
  project_type text not null,
  title text not null,
  summary text not null,
  assumptions jsonb default '[]'::jsonb,
  out_of_scope jsonb default '[]'::jsonb,
  timeline_estimate text,
  total_price numeric(10, 2),
  status text default 'draft'
    check (status in ('draft', 'published', 'accepted', 'expired')),
  published_at timestamptz,
  expires_at timestamptz,
  created_at timestamptz default now(),
  updated_at timestamptz default now()
);

create table public.proposal_phases (
  id uuid default gen_random_uuid() primary key,
  proposal_id uuid references public.proposals(id) on delete cascade,
  phase_order integer not null,
  name text not null,
  description text not null,
  deliverables jsonb not null default '[]'::jsonb,
  phase_price numeric(10, 2),
  created_at timestamptz default now()
);

create index idx_proposals_slug on public.proposals(slug);

alter table public.proposals enable row level security;

create policy "Published proposals are publicly readable"
  on public.proposals for select
  using (status = 'published');

create policy "Authenticated users can manage proposals"
  on public.proposals for all
  using (auth.role() = 'authenticated');

Two design decisions worth explaining:

Slug-based lookups. The client-facing URL uses a slug (e.g., /proposals/acme-crm-integration) rather than a UUID. Easier to share, easier to remember.
RLS for access control. Published proposals are world-readable. Drafts require authentication. This means I do not need a separate API layer for access control.

Step 5: Publishing to the Database

Once I have reviewed and priced the proposal, a single function pushes it to Supabase.

from supabase import create_client
import os
import re

supabase = create_client(
    os.environ["SUPABASE_URL"],
    os.environ["SUPABASE_SERVICE_ROLE_KEY"]
)


def slugify(text: str) -> str:
    text = text.lower().strip()
    text = re.sub(r"[^\w\s-]", "", text)
    return re.sub(r"[-\s]+", "-", text)


def publish_proposal(proposal: dict, prices: dict) -> str:
    slug = slugify(proposal["company_name"] + "-" + proposal["title"])

    result = supabase.table("proposals").insert({
        "slug": slug,
        "company_name": proposal["company_name"],
        "contact_name": proposal["contact_name"],
        "project_type": proposal["project_type"],
        "title": proposal["title"],
        "summary": proposal["summary"],
        "assumptions": proposal["assumptions"],
        "out_of_scope": proposal["out_of_scope"],
        "timeline_estimate": proposal["timeline_estimate"],
        "total_price": prices["total"],
        "status": "published",
        "published_at": "now()",
    }).execute()

    proposal_id = result.data[0]["id"]

    for i, phase in enumerate(proposal["phases"]):
        supabase.table("proposal_phases").insert({
            "proposal_id": proposal_id,
            "phase_order": i + 1,
            "name": phase["name"],
            "description": phase["description"],
            "deliverables": phase["deliverables"],
            "phase_price": prices["phases"][i],
        }).execute()

    return slug

Step 6: Rendering with Next.js

The frontend is a Next.js app with a dynamic route at /proposals/[slug]. It fetches the proposal data server-side and renders a clean, branded page.

The server component calls Supabase, fetches the proposal by slug, joins the phases table, sorts phases by order, and renders everything in a responsive layout. Since the proposal row has RLS allowing public reads for published proposals, the server component can use the anon key. No auth required on the client side for viewing.

The page has three sections:

Header with the proposal title, company name, and summary.
Phases listed sequentially, each with its deliverables in a bordered card layout and the phase price aligned right.
Footer with assumptions and out-of-scope items in a two-column grid, and the total price centered at the bottom.

The rendering code is about 80 lines of standard Next.js with TypeScript interfaces for type safety. Nothing exotic. The data structure from Supabase maps directly to the component hierarchy, which is the whole point of designing the schema around the rendering needs.

Results and What I Would Change

The pipeline has processed around 40 proposals since I built it. Actual numbers:

Time per proposal: dropped from 2-3 hours to roughly 20 minutes. Most of that 20 minutes is me reviewing scope and setting prices.
Consistency: every proposal follows the same structure. Clients get a predictable, professional format every time.
Accuracy: Claude's extraction catches details I used to miss when writing proposals manually, especially around assumptions and scope boundaries.

If I were rebuilding this from scratch, I would change two things. First, I would add a webhook so Otter.ai pushes transcripts directly into the pipeline instead of me dropping files manually. Second, I would version proposals in the database so clients can see revision history when scope changes during negotiation.

The broader point: proposals are one of dozens of business processes that follow the pattern of "take unstructured input, produce structured output, render it somewhere." Once you have the extraction-to-rendering pipeline working for one use case, adapting it to others (SOWs, invoices, project briefs) is mostly just changing the prompts and the schema.

I'm Parker Gawne, founder of Syntora. We build custom Python infrastructure for small and mid-size businesses. syntora.io