We coded 369 prop firm reviews for "did they pay". Here is what the number can and cannot support

A review saying "they refused my payout" is evidence that someone wrote it. It is not evidence that the refusal was unjustified, and it is certainly not a rate. Most of the work in coding reviews is refusing to pretend otherwise.

I run a comparison site for crypto prop firms, which is an affiliate business, so read the rest with that in mind. The coding below is published in full, including the firms that look bad in it.

What I did

For each of twenty crypto prop firm profiles on Trustpilot, I read the most recent twenty reviews in full and coded two things: whether the review was negative at all, and whether it described a problem with a payout — a refusal, a delay without explanation, a deduction, or an account closed after a profitable period.

369 reviews, read by hand on 5 November. 148 coded negative. 54 describing a payout problem, about 15% of the sample.

Coding is binary and it is about the text. I did not check whether any complaint was justified, I could not see the firm's side of any dispute, and I did not code "the platform froze on news" as a payout problem even though it might end in one.

The arithmetic that kills the obvious reading

Here is why "13 of 20 reviews at Firm A describe payout problems" is not a complaint rate.

Firm A had 365 reviews in the previous year. Firm B had eight. Twenty recent reviews from each is the same size of sample and 5% versus 250% of the population. A firm with 40,000 reviews that pays 99.5% of requests on time still produces complaints in absolute numbers that dwarf a small firm's entire customer base.

So the honest statement is: of the 20 most recent reviews I read, 13 described a payout problem. That is a fact about the sample. It becomes a fact about the firm only if you also know the denominator, which the sampling does not give you.

What the sample is genuinely good for is triage. If you are about to pay a firm, read its last twenty reviews and count how many are about money not arriving. You do not need a rate to learn something from that.

What showed up in the reading

Thirteen of AquaFunded's twenty recent reviews described a payout problem. DNA Funded and Instant Funding had seven each, E8Markets and Blue Guardian six. Breakout, HyroTrader, Fundedbit, City Traders Imperium and FundedNext had none in their samples.

Eight of the twenty profiles displayed no Trustpilot score at all. Several carried Trustpilot's own notice that the score was unavailable following a breach of its guidelines. That is a fact about the platform's view of those reviews, and it is not the same as a firm that refused to pay. It does mean a withheld score can be made to look like a good one in a screenshot, so check the profile rather than the quote.

The recurring complaint shapes were consistent enough to be worth naming: payouts refused after a strong month, payouts delayed past a stated window without explanation, and accounts closed after a profitable run with the fee not returned. Whether those are bad faith or bad risk management I cannot tell from the text, and a dataset should not pretend to.

Why publish the coding at all

Because the alternative is a star rating. A score out of five from someone who has never traded a funded account is an invented number with a decimal point, and it is the most common thing in this niche.

The coded sample is also a dataset: every review carries its date, its URL and a verbatim quote, so anyone can re-read the same twenty reviews and disagree with my coding. That is the point. A claim you can check is worth more than a confident one you cannot.

• Coded sample, CC BY 4.0: https://cryptopropfirms.com/complaints/
• The DOI, if you want to cite it: https://doi.org/10.5281/zenodo.24053513
• The price and terms dataset: https://cryptopropfirms.com/prices/

If you send me a firm whose reviews deserve a second pass, the sample gets re-read and the page changes. Corrections are the only part of this that compounds.