DEV Community: 左长

Before You Pay for an AI Tool, Check These Five Things

左长 — Thu, 01 Oct 2026 10:44:40 +0000

Most AI tool roundups answer one question: what does this tool do?

That is the easy question. The expensive one is what happens after you pay - and almost nobody writes that down.

Here are the five checks I run before subscribing to anything. Ten minutes, and they have saved me more than any discount code.

1. The price at your usage level, not the entry tier

Pricing pages open with the cheapest tier and a small asterisk. The number that matters is the one you will actually be billed.

Work out your real usage first - seats, tokens, minutes of transcription, renders - then find that row of the table. A plan that looks like $12/month is regularly $60 once you cross the included quota. If a vendor will not publish an overage rate, treat that as the answer.

2. What breaks when the free tier ends

Many tools are generous until they are not. Before paying, check:

Is the free tier a trial that expires, or a permanent free plan that is throttled?
Does the throttle hit a limit you will cross in a normal week?
Can you keep the work you made on the free tier?

3. Whether you can get your data out

The single most useful line on any listing is the export format.

CSV or JSON export: good.
API access to your own data: better.
Contact support to request an export: that is a hostage situation with extra steps.

If the answer is not in the docs, that tells you something too.

4. What you lose when you stop paying

Downgrades are rarely clean. Some tools keep your data read-only. Some delete it after 30 days. Some keep billing for seats you thought you removed.

Search the docs for downgrade, cancel and data retention before you buy - not after.

5. Whether the listing is current

This market replaces its own top ten roughly every quarter. A review from eighteen months ago is describing a different product. Check the last-updated date on anything you read, including this post.

None of this is exotic. It is the boring half of the decision, and it is the half most directories skip, because feature lists are easier to write than pricing tables.

I maintain haiai123, an independent directory of 1,487 AI tools across 17 categories. Every listing records which step of a workflow the tool replaces, what it costs per month at typical usage, and how you get your data out. Alongside the directory the site publishes model capability, API usage and AI app popularity leaderboards, each labelled with its data source and refresh date.

If you run a sixth check before you subscribe, I would like to hear it.

Wiring an LLM API Into a Product: 8 Things That Decide Your Bill

左长 — Thu, 01 Oct 2026 06:19:45 +0000

Wiring an LLM API into a product looks like a two-day job. The first version usually is. The part that takes longer is everything around the call itself, and that is where the money leaks.

This is the checklist I would hand someone who is about to put a model behind a real feature.

1. Count output tokens, not just input

Teams budget with the input price, then get an invoice that is several times higher. Output tokens usually cost multiples of input tokens, and chatty assistants produce far more output than input.

Two habits fix this: cap max_tokens per call, and log input and output counts separately from day one. A single combined number hides the line you actually need to watch.

2. Make retries explicit and cheap

A malformed response still costs tokens. If your client retries three times by default, a broken prompt is billed four times.

Set an explicit retry count, not the library default.
Only retry on transport errors and rate limits, not on every exception.
Validate the shape of the response before you use it, so you retry less.

3. Turn on prompt caching if your prompt has a stable prefix

If every request carries the same system prompt, tool definitions and documentation, you may be paying for it repeatedly. Providers that support prefix caching charge less for the repeated part. This is usually the single largest line item you can cut without touching quality.

4. Decide what happens when the model is slow before it is slow

Streaming helps perceived speed but makes errors harder to handle: you have already sent half an answer when the connection drops. Pick a rule now - either you stream and buffer for the client, or you do not stream and hold a timeout budget.

Write the fallback down: smaller model, cached answer, or an honest "try again". "Nothing" is also a choice, just an expensive one.

5. Version your prompt like code

Prompts change behaviour in ways that are invisible in a diff. Keep them in files, review them, and record which version produced which request. When quality drops, the first question is always "what changed", and you want an answer that is not a guess.

6. Build a tiny eval set before you tune anything

Twenty real examples with expected properties beat a hundred vibes. You do not need a framework. A JSON file and a script that prints a pass rate is enough to stop you from shipping a regression because it read better in a demo.

7. Track cost per feature, not cost per account

An account that uses chat heavily and one that runs a nightly batch job are not comparable. Attribute spend to the feature that caused it. That is the number that tells you whether a feature is worth keeping.

8. Have an exit

Models change price, get deprecated, or get worse. Keep the call behind one interface, and keep the prompt portable. If swapping providers is a two-week project, that is a business risk, not an engineering preference.

The short version

Cap your output, retry on purpose, cache the prefix, budget the timeout, version the prompt, measure per feature, and keep the door open.

I keep pricing and capability notes for the models I use at haiai123 - 1,487 tools across 17 categories, with the data source and refresh date printed on every table.

How I Screen an AI Coding Agent Before Letting It Near My Repo

左长 — Thu, 01 Oct 2026 02:34:30 +0000

Every week another "autonomous AI coding agent" launches, and every week someone I know lets it loose on a production repository, then spends the evening reviewing a 900-line diff that touches files nobody asked it to touch.

The fix isn't a better benchmark table. It's a cheap trial.

The 20-minute trial

Pick one function you know well - ideally slightly messy, with a couple of edge cases - and give the agent exactly one instruction:

"Add input validation to parseConfig and a test that covers the empty-string and null cases. Don't change anything else."

Then watch four things:

Did it ask anything? A good agent asks when the requirement is ambiguous. A bad one invents a spec.
Did the diff stay in scope? One function and one test file means one function and one test file.
Did it actually run the test? Not "the test should pass" - running it and showing output.
What did it do when the test failed? Stopping and reporting beats thrashing for six minutes.

Six things worth checking before you commit to a tool

1. Plan before edit. You want to see the intended change list before anything is written to disk.

2. Permission scoping. Can it run shell commands? Which ones? Can it read .env files, SSH keys, or your cloud credentials? A sandbox or an approval prompt isn't friction - it's the whole safety model.

3. Blast radius. How many files does a typical task touch? An agent that "helpfully" reformats your project while fixing a bug is worse than useless in a team.

4. Failure behaviour. Retry loops that quietly change the goal are the single most expensive failure mode. You want loud, early, specific failure.

5. Context strategy. Does it index the repository, or only see the files you mention? This decides whether it will follow your existing patterns or happily introduce a second HTTP client.

6. Cost per completed task. Token pricing tells you almost nothing. Divide your monthly spend by merged pull requests.

What I stopped caring about

Which model is inside. The wrapper, the context management, and the permission model matter more than the logo.
Benchmark scores. Coding benchmarks mostly measure single-shot answers to self-contained puzzles. Your repository is neither.
The word "agentic". It's a marketing term until you've seen it handle a failing test.

Where I keep the shortlist

Anything that passes the trial goes on a short list with a note about what it's good at: refactors, tests, glue code, migrations. I keep the longer teaching version of this checklist - the one with the exact prompts - at haiai123's beginner guide to choosing an AI coding agent, and I check the coding leaderboard before any of it to see how the underlying models are moving.

The rule that actually keeps quality up

Treat the agent as a very fast junior developer who has never read your codebase conventions and forgets everything overnight. You wouldn't merge a junior's branch without reading it, and you wouldn't let one run rm -rf unsupervised. The agent changes the speed of writing code - not the standard for shipping it.