How we built a machine-readable catalog of free LLM APIs
Why "free" is not one data type
When we started evaluating "free" LLM APIs for developers in Iran, we quickly realized that the word "free" means different things to different providers. Some offer a permanent free tier that renews monthly. Others provide a limited-time trial. Many give one-time credits that expire after a certain amount of usage. And some label their service as "free" but require a credit card just to sign up.
Treating all of these as equivalent creates misleading comparisons. A provider offering 100 requests per day indefinitely is fundamentally different from one offering a $5 credit that runs out after a week. We needed a data model that captures these differences explicitly.
Designing the provider schema
Each provider in our catalog is represented by a structured JSON record with clearly separated fields:
- freeTier type — permanent, trial, credit, or free model
- quotas — RPM, RPD, TPM as documented by the provider
- payment required — whether a credit card is needed for signup
- OpenAI compatibility — protocol/endpoint compatibility and base URL
- regional access — separate fields for direct access, VPN access, and official policy
- evidence — dated source URLs and test results
This structure makes it possible to query, filter, and compare providers programmatically rather than relying on prose descriptions.
Regional-access evidence without overclaiming
Regional availability is one of the most important yet most misrepresented fields in free API lists. We track three independent evidence categories:
- Official documentation — what the provider states in their ToS or documentation about regional restrictions
- Direct observation — a sanitized, dated test result from an Iranian IP
- VPN observation — a dated test result from an authorized VPN connection
These categories are never merged. A VPN result is not treated as direct Iran access. "Unknown" means insufficient evidence exists, not that the service works. This discipline prevents misleading claims that could affect developers' decisions.
Preventing stale data
Free-tier policies change frequently. A provider that is free today may reduce quotas tomorrow or introduce payment requirements next month. We address this with:
- Dated evidence — every record includes a last-checked date
- Validation scripts — CI runs schema checks, freshness warnings, and contract tests
- Stale-data warnings — providers without recent checks are flagged in the output
The validation pipeline includes 24+ test suites covering provider schemas, privacy, launch links, benchmarks, and deployment contracts.
Generating pages and machine-readable outputs
The same validated dataset powers multiple outputs:
- Provider pages — dedicated HTML pages for each of 22 providers with full details
- Practical guides — 6 educational guides for developers
- JSON catalog — machine-readable catalog.json for programmatic access
- data.json — normalized projection for external consumers
- Sitemap and llms.txt — for search engine and LLM discovery
- Iran mirror — an alternative deployment for users in Iran
All outputs are generated deterministically from the source data using Node.js scripts. No hand-editing of generated sections.
CI contracts and privacy checks
Every change runs through validation gates:
- Provider JSON schema validation
- Publication privacy checks (no credentials, tokens, or internal URLs)
- Launch link contract (28 UTM links across 14 sources)
- Launch log contract (17 publication rows)
- Benchmark and advisor scoring validation
- Deployment and rollback integration tests
- VPN verification configuration validation
These contracts catch data inconsistencies, privacy leaks, and stale output before any change reaches production.
Contribution workflow
The project is open source and contributions follow a structured workflow:
- Research provider documentation and run verification tests
- Edit the provider JSON file with dated evidence
- Run generators and tests locally
- Open a draft pull request with source URLs, evidence class, and limitations
We welcome new providers, updated evidence, and corrections to stale data. The key requirement is that every factual claim must have a dated, verifiable source — not a guess or third-hand report.
The complete catalog, provider pages, and guides are available at:
https://llm.persiantoolbox.ir/?utm_source=devto&utm_medium=article&utm_campaign=international_launch
Source code and contribution guide:
https://github.com/alirezasafaei-dev/awesome-free-llm-apis-ir
Top comments (0)