DEV Community

Cover image for Two databases, one question: how I built Eligible
Harish Kotra (he/him)
Harish Kotra (he/him)

Posted on AI-assisted

Two databases, one question: how I built Eligible

#ai #productivity #appwrite #dailybuild2026

A government benefits finder that answers one sentence, and shows its work.

Type this:

I am a 34 year old tailor in Hyderabad, I make about 2.4 lakh a year, two kids in school

You get back 488 Indian government schemes you actually qualify for. Each card names the rule that let it through. Underneath, 72 schemes that came close, each one naming the exact number that stopped it: "Your ₹2.4 lakh is ₹10,000 over the ₹2.4 lakh ceiling."

That is Eligible. It reads 2,066 real schemes from the MyScheme corpus: 527 central, 1,539 state, 80 issuing ministries and departments, 13.4 million characters of eligibility prose written by clerks, not copywriters.

This post is about the design decision that made it work, the four bugs that nearly broke it, and the interface rebuild that came after the first version looked too much like every other tool I have shipped.

The wrong first design

The obvious build is one table, one search, one list.

schemes: name, summary, state, income_ceiling, min_age, max_age
Enter fullscreen mode Exit fullscreen mode

Ask for everything, filter by income and age, done in an afternoon.

It fails immediately, and the failure is interesting. A tailor in Hyderabad is not searching for the word "tailor". He needs the scheme written for "artisans and weavers engaged in the handloom sector". There is no keyword overlap at all. Full text search returns nothing, and the person who most needed the scheme goes away thinking it does not exist.

So the first half of the problem is meaning, and meaning wants embeddings.

But the second half is arithmetic, and embeddings are terrible at arithmetic. A cosine similarity score of 0.83 says nothing about whether ₹2,40,000 sits under a ₹2,50,000 ceiling. Filtering 80 float scores in application memory means the database did nothing and the process did everything, which is exactly what the "how deliberately did you use the database" question is asking about.

Eligibility is two questions. So it gets two databases.

flowchart TD
    B["Browser<br/>React 18 + Vite, one screen"] -->|"POST /api/search"| A["Express API<br/>64 kB body cap"]
    A --> P["parseProfile()<br/>optional LLM, output validated"]
    P --> V["VectorsDB<br/>slug + embedText<br/>cosine similarity, top 80"]
    P --> T["TablesDB<br/>23 typed columns, 5 indexes<br/>range scans on age + income"]
    V -->|"slug"| U{{"union, deduped by slug"}}
    T -->|"slug"| U
    U --> R["rules.ts<br/>pure, no I/O<br/>reject only on a stated field"]
    R --> M["matched 488"]
    R --> N["near misses 72<br/>each carrying the number that killed it"]
    R --> L["long tail 464"]
    R --> C["receipt: candidates, kills, ms, which layer ran"]

VectorsDB holds one document per scheme: the slug and the embedded eligibility paragraph. Nothing else. TablesDB holds the arithmetic: jurisdiction, level, minimum and maximum age, income ceiling, benefit amount, document count, women-only and disability-only flags. A slug joins them.

The rule I keep coming back to: if both databases stored the income ceiling, there would be two sources of truth for the same fact, and the day they disagreed the app would start lying. The vector store knows meaning. The table knows facts.

Reading 2,066 schemes without inventing thresholds

server/extract.ts turns prose into typed facts. The hard part is not parsing. It is deciding what to do when the document says nothing.

A wrong threshold is worse than a missing one. If a scheme states no income ceiling and the parser writes 0, that scheme silently excludes every human on earth. If it writes Infinity, nobody notices a broken parse until a ministry changes the rules. So the parser returns whether a rule was stated, alongside the value:

export function parseIncomeCeiling(text: string): { ceiling: number; stated: boolean; clause: string } {
  const parts = sentences(text);
  let best: { ceiling: number; clause: string } | null = null;
  for (const sentence of parts) {
    const low = sentence.toLowerCase();
    if (!low.includes('income')) continue;
    const re = /(?:₹|rs\.?|inr)?\s*(\d[\d,]*(?:\.\d+)?)\s*(crore|cr\b|lakh|lac|lacs|lakhs|thousand|k\b)?/gi;
    let m: RegExpExecArray | null;
    while ((m = re.exec(sentence))) {
      const value = parseRupees(m[0]);
      if (!value) continue;
      if (value < 5000) continue;
      if (value > 50_000_000) continue;
      if (!best || value > best.ceiling) best = { ceiling: value, clause: sentence.trim() };
    }
  }
  if (!best) return { ceiling: NO_INCOME_CEILING, stated: false, clause: '' };
  return { ceiling: best.ceiling, stated: true, clause: best.clause.slice(0, 260) };
}
Enter fullscreen mode Exit fullscreen mode

Three details matter more than they look:

  • The sentence filter. Only lines containing "income" are considered. Spend ceilings, project costs and loan amounts all say "lakh" and are not income limits.
  • The best rule. When a paragraph states several amounts, the parser takes the highest under "income". A scheme that says "up to ₹2 lakh, ₹2.5 lakh for SC/ST" admits everyone at 2.4 lakh if you take the wrong one. Taking the maximum is the choice that broadens access, which is the safer error for a benefits tool.
  • The clause. Every ceiling keeps the sentence it came from, so the UI can show the government's own words instead of a number with no provenance.

The same shape applies to age bands and to the two flags that cannot be guessed: womenOnly and disabilityOnly. Those are computed with left anchored regexes, because the first version matched male inside female and turned a women's scheme into a men's scheme.

The engine, and the comment that is a bug report

server/rules.ts is a pure function. No database, no network, no model. That is deliberate: the decision is the part that has to be auditable, so it is the part with no dependencies.

// Central schemes apply everywhere, however the row spells it: a table that
// stores `level = central` must not be judged by the string in jurisdiction.
const everywhere =
  scheme.level === 'central' || scheme.jurisdiction === 'All India' || scheme.jurisdiction === 'Central' || !scheme.jurisdiction;

if (!everywhere && profile.state) {
  if (scheme.jurisdiction === profile.state) {
    reasons.push(`${scheme.jurisdiction} scheme`);
    score += 0.2;
  } else {
    blockers.push({
      field: 'state',
      detail: `Only for people in ${scheme.jurisdiction}`,
      gap: `${profile.state} is not ${scheme.jurisdiction}`,
    });
  }
}
Enter fullscreen mode Exit fullscreen mode

That comment is me explaining a bug I shipped. The first version compared jurisdiction strings. It worked for the local fixture, because ingest.ts writes All India for central schemes. Then the rows landed in Appwrite and a few came back with jurisdiction: Central and level: central. Every one of those schemes was rejected for a Telangana user, correctly spelled and completely wrong. level decides now. The string is decoration.

The second rule is the one that keeps the tool honest:

A scheme is only rejected on a field it actually states.

Every blocker carries a gap string the card prints verbatim. That is why a near miss reads "Your income, ₹2,40,000, is above the ₹1,00,000 ceiling" instead of "not eligible". The number is the product. People do not want a verdict, they want to know how far away they are.

The join, and what the vector store is not allowed to hold 

export async function semanticCandidates(text: string, limit = 80): Promise<Hit[]> {
  const { vectors } = services();
  const [vector] = await embed([text]);
  if (!vector) return [];
  const list = await vectors.listDocuments({
    databaseId: VDB,
    collectionId: V_COLLECTION,
    queries: [Query.vectorCosine('embedding', vector), Query.limit(limit)],
  });
  return (list.documents as unknown as Record<string, unknown>[]).map((doc) => ({
    slug: String(doc.slug ?? doc.$id),
    similarity: readSimilarity(doc),
  }));
}
Enter fullscreen mode Exit fullscreen mode

Eighty candidates by cosine similarity. Then the table layer runs its own predicate query, and the union of the two goes to the rules engine. Neither layer decides anything, which is the point: recall and truth are separate jobs, and the engine is the only thing allowed to say yes.

Illustration of Eligible

A request looks like this end to end:

semanticCandidates 80   tableCandidates 500   union 560
killedByRules 72        matched 488           total 95 ms
Enter fullscreen mode Exit fullscreen mode

Those are real numbers from the demo sentence on the real corpus, printed in the UI on every search. I would rather ship a receipt than a spinner.

Four bugs worth writing down

1. Invalid query: Syntax error. The setup script passed queries: ['limit(1)'] and this SDK version wants Query.limit(1). The app code had it right; only the scripts were wrong. Half an hour lost to a raw string.

2. Cannot read properties of undefined (reading 'length'). ingest.ts writes a plain array of 2,066 schemes. The setup script expected { rows, embedText, meta }. Every schema statement succeeded, then the first row call read .length off undefined. The fix reads either shape, because a fixture format that changes should not look like a database outage.

3. Invalid document structure: Unknown attribute: "issuer". This one cost the whole load. Tables and indexes were created, then every batch failed and zero of 2,066 rows were saved. Appwrite validates a batch as one payload: one undeclared key rejects all hundred rows in it, and the error names whichever key it hit first, which is not the same as the list of things wrong.

The fixture carries 28 fields, because the engine wants prose and clauses the table has no use for (issuer, embedText, eligibilityExcerpt, ageClause, incomeClause, genders). The table declares 23. The fix is projection, not schema growth:

const payload = chunk.map((r) => ({ ...toRow(r as never), $id: String(r.slug) }));
Enter fullscreen mode Exit fullscreen mode

toRow() is now the only thing allowed to decide what a row looks like on the wire, and it drops every key the table does not declare. A second, quieter bug was hiding in the same function: it emitted documentsCount while the schema and the API read docsCount, so even a successful load would have shown a zero document count on every card. Neither bug tripped tsc, because both sides were typed as Record<string, unknown>. Schema drift is not a type error, it is a test you did not write. So:

columns 23  rows 2066  violations 0
payload keys 23/23  missing []  extra []  max row 2741 bytes
Enter fullscreen mode Exit fullscreen mode

4. A blank screen with a valid key. A project can be configured and empty. No tables, no rows, no vectors. The honest version of that state is a fallback that says so on the receipt:

if (rows.length === 0 && hits.length === 0) {
  const result = searchLocal({ text, profile: merged, parsedBy });
  result.receipts.vectorLayer = 'local fallback: Appwrite returned no rows (run npm run verify:appwrite)';
  res.json(result);
  return;
}
Enter fullscreen mode Exit fullscreen mode

A wrong key should degrade with a label, not a white page.

The interface is the argument

The first build looked like the last thing I shipped: rounded cards on warm paper with a green-and-saffron accent. Side by side with my earlier entry it read as the same product with different words, and it was rejected for exactly that reason.

The rebuild starts from what the product actually is. Eligibility is not a marketplace listing, it is a document you are entitled to. So the page is a white sheet on a cool grey desk, divided by hairline rules instead of card borders, with a single red seal that appears only where a scheme was refused. Red is not decoration here. Red means no.

Three decisions did most of the work:

One seal, one meaning. --seal: #a32a1c is used for refusals, the "number that killed it" figure, and nothing else. The moment red also highlights a button, the refusal loses its weight. On the dark verdict band the same intent needs a lighter mix, #e2644f, because #a32a1c on #14181b measures 4.39:1 and fails AA. Two reds, two jobs.

A bar drawn to scale. The answer opens with four bands: everything read, everything shortlisted by meaning, everything that passed, everything refused. They are sized in proportion, not as equal pills, so the refusal strip is visibly thin. 23.6% passed and 3.5% was refused, and you can see that before you read a word.

Type set large on purpose. Newsreader for statements and amounts, Public Sans for controls and rows, both self-hosted variable woff2 files preloaded from public/fonts. 18px root, 52px submit button, 44px minimum on everything else. This is read on a phone at 9pm by someone who may be 60.

The part I did not expect to matter was the placeholder. It sat at #8b959c on white, which is 3.05:1, technically fine for placeholder text and genuinely hard to read. It is #6b757c now, 4.71:1. The only reason I know that is that the audit prints numbers instead of opinions:

const need = size >= 24 || (size >= 18.66 && bold) ? 3 : 4.5;
const r = ratio(rgb(cs.color), bgOf(el));
if (r < need) low.push({ text, cls, ratio: r, need, size });
Enter fullscreen mode Exit fullscreen mode

npm run audit drives Chrome at 1440, 1024 and 390 pixels, walks every label on the result screen, computes contrast against the actual painted background, measures every control, checks for horizontal overflow, and exits non-zero on any regression:

1440px  overflow 0  sub-AA 0  targets<44 0  errors 0  bar 23.6/3.5%  grid 148px 644px 128px
1024px  overflow 0  sub-AA 0  targets<44 0  errors 0  bar 23.6/3.5%  grid 148px 540px 128px
390px   overflow 0  sub-AA 0  targets<44 0  errors 0  bar 23.6/3.5%  grid 318px
Enter fullscreen mode Exit fullscreen mode

That is the whole design system, defended by a script instead of a screenshot.

What it costs, honestly

TablesDB on shared compute is free. VectorsDB needs dedicated compute, about $10/month, because an HNSW index keeps its graph in memory. The setup script creates the table layer first and the vector layer last, and if the project cannot host the second one it says so and leaves you a working app with lexical recall. The receipt then reads semantic: lexical. It never claims a vector search it did not run.

That is the whole rule of the project. If the tool cannot do the thing, it says what it did instead.

What I would build next

  • Deadlines. Most schemes open in windows. A closesOn column turns a search result into a calendar.
  • Follow up questions. "Two kids in school" should re-rank the list, not clear it.
  • Document checklists. The data knows how many documents a scheme needs. It does not yet know which ones.
  • Saved profiles. The profiles and searches tables are provisioned and barely used. A returning user should see what changed since last time.
  • Telugu and Hindi. The interface is the easy half; 2,066 cleaned summaries is the real work.

Two databases, one question, and a receipt on every answer. That is the whole project.

How Eligible Looks

Code & more: https://www.dailybuild.xyz/project/303-eligible

Top comments (0)