The Hackers Who Tracked My Sleep Cycle

#security #programming #webdev

This is a short story about how I caught hackers timing their attacks around my daily routine.

A couple of weeks ago, in the middle of the night, I got alerts on my phone that various metrics were going haywire.

The only thing I noticed was a huge spike in sign-ups. This coincided with one of my articles being on the front page of Hacker News, so I didn't think much of it.

It had spiked for a few hours and flattened out just before I started troubleshooting, so I went back to sleep.

When I woke up the next morning, a few thousand more accounts had appeared.

I thought this was a bit suspicious, but ... even if someone was being naughty, they seemed to have given up.

Nothing else was out of the ordinary that day. I even decided to stay up later that night to see if the pattern would repeat. Nothing.

As a small precaution, I activated CAPTCHA to slow down the sign-ups and went back to sleep.

The next morning, you guessed it... same pattern.

This time, I decided to do a deep dive into the data.

What I found was that the hackers were...

creating thousands of accounts
adding a valid payment method to each account
running a single very expensive LLM call (2-3 USD)

This would let the first request go through, then trigger a charge to their payment method. The payment method gets rejected, but the request has already been processed.

The reason the first request goes through is that we deposit a little bit of money into the account when it's created. A nominal amount that's enough to play with the API. However, if a payment method is added, we allow people to go into overdraft, which is why their expensive LLM call goes through.

Anyway, using this method, they would get away with about a thousand dollars' worth of credits every night, which kept them interested in the service.

But what caught my attention wasn't the money – it was the timing. The attacks coincided with my sleep cycle.

At this point, I still thought of it as unlucky timing, but even then something just didn't sit right with me.

Coincidentally, that day I decided to take a break and disconnect from my computer early. And lo and behold, just 30 minutes after I shut down my computer, I got the first notification.

I logged in to check, and it stopped.

Went to play some games and ... 30 minutes later, I got the second notification.

That's when it clicked – the timing of the attacks wasn't random. They were checking my Discord status to see if I was online.

Sure enough, I confirmed this by setting myself as offline on Discord, and the attacks popped right back up.

Over the next few days, I used this insight to mess with the hackers and to use them as my personal pen testers.

I didn't want to remove free credits for everyone, so I began experimenting with different ways to deter future attackers.

I would make a change, then go 'offline'. I'd watch them troubleshoot their automations until they figured out a workaround. Then I'd go back 'online'. The attacks would resume, and the cycle would repeat.

I mostly forgot about the entire incident until they came back to try their luck again the following week. But despite a few nights of alerts about tripwires getting triggered, they never managed to get more than the few cents we deposited into new accounts – not enough of an incentive to keep trying.

In the end, it was the cat-and-mouse game that made the whole experience worth it. I got free pen testing; they got a few dollars.

Card testing vulnerability

I wasn't surprised about the overdraft feature being abused. This was something we were aware of and treated as a conscious trade-off between convenience and risk of abuse.

The bigger issue was that this made me realize that a malicious actor could abuse our system for card testing. That's a widespread problem and one that will get your Stripe account flagged. When researching this problem, I didn't find many effective solutions, so I wanted to dedicate part of this blog post to sharing what I learned.

Here's what I tried and how it held up:

Method	Effectiveness
Device fingerprinting	Ineffective. Fingerprints are great for detecting legitimate returning users (e.g. to bypass CAPTCHA), but because they are easy to fake, they are not effective at detecting malicious actors.
IP address blocking	Ineffective. Residential proxies are cheap and easy to get.
CAPTCHA	Mild deterrent. Ineffective. Many existing solutions to bypass CAPTCHA.
OTP	Mild deterrent. Ineffective. Many existing solutions to bypass OTP.
JA4	Somewhat effective. JA4 is a TLS fingerprinting method that identifies clients based on how they negotiate TLS connections. Of all data points that we collect, JA4 is the most stable identifier.
ALTCHA	Somewhat effective. ALTCHA is a proof-of-work challenge that requires the client to solve a computational puzzle before submitting a request. When combined with prior methods, can slow down the attacks enough to deter the attacker.
Rate limiting	Somewhat effective. Slows down the attacks, but may hurt legitimate users.