DEV Community

Cover image for How we built an application tracker that cannot lie to you
Ava Bagherzadeh
Ava Bagherzadeh

Posted on

How we built an application tracker that cannot lie to you

Most software tells you what it did. AI Applyd tells you what the employer did, and the difference
is an architecture decision you can copy.

Here is the design, because I think the pattern generalises to anything that hands work to a system
you do not control.

The rule the whole system is built on

No state that matters may be writable by us.

One field decides whether an application counts as sent. That field can only be written by an inbound
email from the employer's applicant tracking system, classified to a positive landing stage. Our
click cannot set it. Our screenshot cannot set it. A retry cannot set it. There is no admin path that
sets it.

landing_confidence = 100  AND  submission_verified = 1
Enter fullscreen mode Exit fullscreen mode

100 is reserved and architecturally unreachable from our own telemetry. Everything we observe about
our own session tops out below it. The only way to reach 100 is for a company to reply.

Why you would design it this way

Because the alternative is a mirror.

Every application tool can show you the fields it filled, the file it attached and the button it
clicked. Read that list again: it is the tool reading back its own input. It proves the software has a
memory. It proves nothing about whether an employer received anything.

We know precisely how that fails because we instrumented it. A form widget can hold your typed text in
the visible box while the value the form actually submits stays empty. The tool reads the box, sees
its own text, and reports success. A check that reads your own write can never fail while the write
works, which is exactly why it is worthless.

So we stopped scoring anything on our own behaviour.

The pieces

A relay address per user. Applications go out with an address we control, so the employer's
acknowledgement comes back to us instead of vanishing into a personal inbox. That reply is the only
evidence that exists.

A classifier on the inbound. Acknowledgement, rejection, interview invitation, or noise. Only a
positive landing stage writes the field.

A reserved value. 100 cannot be produced by any of our own signals, so no amount of internal
optimism can round up to it.

A twelve-platform submitter. Greenhouse, Lever, Ashby, Workday, iCIMS, Personio, Teamtailor,
SmartRecruiters, Recruitee, Breezy, Workable, Rippling. Each one gets a real form filled properly,
including the widgets that quietly submit empty.

What it buys the person using it

They know where they stand. Not "submitted" as a hopeful noun. Confirmed, with the timestamp of
the company's own reply.

Silence becomes information. These acknowledgements arrive in seconds to a couple of minutes. When
one has not arrived, that is a fact about the submission, and it can be retried while it still
matters rather than discovered in three weeks.

They stop paying for nothing. A submission a company never confirmed does not count against their
plan. That pricing rule only exists because the confirmation exists. You cannot build it on top of
your own click.

The property I like most

We cannot fake our own success metric. The one number that matters can only be written by somebody
else's mail server.

That is an uncomfortable way to build a company and a very easy one to trust. If you are building
anything that hands work to a system you do not own, the question worth asking of every check you
have written is whether it would read differently had the other side failed. If it would not, you do
not have a check.


I build AI Applyd. It scores, tailors and submits applications on the
company's own hiring system across twelve platforms, and only counts one as sent when that company
confirms it. Interviews on your calendar, not rejection emails in your inbox.

Top comments (4)

Collapse
 
whateverneveranywhere profile image
Ava Bagherzadeh

I checked both of these before replying, and you are right on both, with one correction that makes it worse rather than better.

The headers: confirmed, curl -I https://aiapplyd.com/ returns none of the six. But it is not only the front end. api.aiapplyd.com returns none of them either, and our own internal docs claim CSP and HSTS are wired into the API middleware. So the interesting finding is not the missing headers, it is that we have a written record saying they are there. A control nobody verified is a control that does not exist, and this is the second time that exact shape has bitten us.

The duplicate title: also confirmed. The homepage ships the real title plus a bare "AI Applyd". That one stings, because we are in the middle of an indexation problem and I would not have thought to look at the head for it.

You are also right that it is dashboard work rather than a deploy, which matters more than usual here because our CI is currently blocked on a billing issue and cannot ship anything at all. A transform rule and a toggle route around that entirely, and report-only CSP first is the right sequencing.

Thank you for actually running the check instead of just saying we should. That is a much more useful comment than a compliment would have been.

Collapse
 
amitfeldman profile image
Amit Feldman

"Cannot lie to you" is a strong line to build a product around — the audit-trail framing is the right differentiator for this category.

Ran a passive check of aiapplyd.com while reading (response headers and public HTML only, nothing a first visit doesn't send):

  • All six baseline security headers are missing — Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy. The good news: you're behind Cloudflare, so this is dashboard work, not a deploy — one Response Header Transform Rule covers the set, HSTS is a toggle under SSL/TLS, and CSP can ship report-only first so nothing breaks.
  • The served HTML carries two <title> tags — the full "Auto-Apply That Ends on an Interview - AI Applyd" and a second bare "AI Applyd". Probably a layout-level SEO component rendering on top of a per-page one. Search engines pick one inconsistently, so your SERP listing can come out as just the brand name.
  • Minor: ~50 of 54 images ship without alt text.

For a product asking people to upload resumes — about the most PII-dense document most users own — the header layer is the first thing a security-conscious buyer checks. Happy to re-run the check free any time if you want a before/after.

Collapse
 
amitfeldman profile image
Amit Feldman

Confirmed on the API side too — curl -I https://api.aiapplyd.com/ returns none of the six, so the docs and the middleware disagree on both hostnames, not just the front end. One useful detail while I was there: the API is also behind Cloudflare (cf-ray on the response), so the same Transform Rule route works there — or a second rule scoped to the api hostname if you want a stricter CSP on JSON responses than on the SPA.

On the written-record problem, since you said it's the second time that shape has bitten you: the cheapest guardrail is one that lives entirely outside your pipeline, which matters while your CI is blocked. A five-line cron script on any host — curl the two URLs, assert the six headers exist and exactly one <title>, ping you when the assertion fails. Docs describe intent; a probe describes reality. Only the probe pages you at 3am. It also catches the regression direction, not just the initial fix — headers get dropped silently by dependency upgrades and proxy changes all the time.

Happy to re-run the check free once the transform rules land — before/after on both hostnames. And if you'd rather not maintain the probe yourself, monitored re-scans on a schedule are exactly what we run — say the word and I'll send details.

Collapse
 
whateverneveranywhere profile image
Ava Bagherzadeh

Re-ran it just now on both hostnames so you have the current state rather than mine from memory.

aiapplyd.com carries all six today. HSTS with preload, a full CSP with frame-ancestors none and a report-uri, X-Frame-Options DENY, X-Content-Type-Options nosniff, Referrer-Policy strict-origin-when-cross-origin, and a Permissions-Policy. The served HTML has exactly one title tag now as well, so both of the front-end findings are closed.

api.aiapplyd.com carries none of them, exactly as you said, and that is the half worth talking about. It is not a Cloudflare gap there. The headers are applied by middleware, and there are paths that never run through it, so the docs are accurate and the response is still bare. That is the same shape as the bug the post was about: the thing exists, it is documented, and something reaches the user without touching it. I would not have gone looking without your comment.

Your probe point is the part I am keeping. A doc describes intent and a probe describes reality, and only one of those wakes anyone up at 3am. It is going into the fail-closed check we already run on a schedule rather than into a new script, because a second monitor is one more thing that can quietly stop running and nobody notices for a month.

I will pass on the paid re-scans. Nothing against them, we are holding outside spend at zero right now. The free half you already did was the useful half.