Victor

Posted on Oct 15

I Built an AI Financial Advisor. Now I Need You to Break It.

#testing #programming #finance #ai

Vicquant is open for beta testers: pick a frontend design, run four short tasks, and tell me exactly what feels wrong.

The problem with asking "What do you think?"

When I first put Vicquant in front of beta testers, I asked the question every builder asks:
"What do you think?" The answers I got back were polite and vague. "Looks good." "Nice app."
Not one of them told me where someone got stuck, what confused them, or what they expected to happen and didn't.
That is my mistake, not theirs. Open-ended questions produce open-ended politeness.

So this time I am doing it differently: I am giving you specific things to do, a short way to report what
happened, and a say in how the app looks. If you have ever wanted to shape a product while it is still early, this is that chance.

What is Vicquant?

Vicquant is an AI-assisted personal finance and market-analysis app. It started as a small local chatbot that helped people plan debt repayment and savings, and it has grown into something much bigger. One rule shaped it from day one: the AI never does the arithmetic. All the
money math (balances, payoffs, risk numbers) is computed by normal, testable code, and the language model only explains the results. That way a language model's confident guess cannot quietly become your budget.

What you can try in the beta:

• AI Advisor and Budget: ask questions about debt, savings and spending in plain language.
• RAG Vault: upload your own financial documents and get answers with citations back to the source, so you can check the answer instead of trusting it.
• Stocks and Portfolio: track holdings and see risk measures such as Sharpe, Sortino, VaR and CVaR, without needing a finance degree to read them.
• Market regime detection and alerts: a statistical model that tries to tell calm markets from turbulent ones.
• Proof ledger: trading signals are recorded in a tamper-evident, hash-chained log with fixed rules set in advance, so results cannot be quietly rewritten after the fact. No win rate is shown until at least 30 signals have been resolved, because a handful of lucky calls proves
nothing.

Honest status: what is rough right now

A beta invitation is only useful if it is honest, so here is where things stand:
• The AI currently runs through a hosted model service rather than fully on your own device.
Running it locally again is on the roadmap, but it is not the current setup.
• The mobile web experience is not good enough. A native Android and iOS app is planned, so please judge the layout with that in mind.
• Vicquant gives information and analysis. It is not licensed financial advice, and nothing in it is a guarantee of returns.
• Some features are still being tuned, and speed can vary. If something feels slow, I want to know exactly when it happened.
• The frontend UI and UX design also needs working on. This is my first time on working on something of this scale.
Please, I need your honest review and contribution, so i can improve on it

Please use sample numbers. During the beta, enter made-up balances and debts rather than real account details.You can test everything that matters without exposing anything
personal.

Help me choose the design

I have three frontend designs and I do not want to choose alone. The feedback I got earlier was that the interface needs to feel easier for someone who is not a finance expert, so this is where your eyes matter most. The UI design is okay for laptop size screen but for the mobile, i am still working on it.

When you vote, do not only tell me which one you like. Tell me which one you understood fastest. In a finance app, "pretty" matters less than "I knew what to tap."

Your 15-minute mission

Four tasks. Do them in order, and write down anywhere you hesitated, even for a second.

How to send feedback (it takes two minutes)
After the four tasks, answer these five questions in the feedback form:

• Which design did you pick, and which one did you understand fastest?
• Where did you get stuck or confused? Name the screen.
• What did you expect to happen that did not?
• Did any AI answer feel wrong, vague or untrustworthy? Paste it if you can.
• Would you use this again? What single change would make you say yes?

Bug reports are gold. A screenshot, your device and browser, and one sentence about what you were doing is all I need.

Join the beta
If you are a developer, a student, a finance enthusiast, or simply someone who gets confused by money apps, you are exactly who I need. Beginners are especially welcome, because they
find the problems experts forgive.

• Beta link: https://budget-planner-advisor-rho.vercel.app/
• Reply here or message me with questions.

Thank you for reading, and for being willing to break something so it can be built better. Tell me what is wrong with it, and I will make sure the next version is better because of you.

Top comments (16)

wolf Valencia • Oct 19

You asked people to break it, so here's a small one. I tried a made-up path on the beta and it still rendered as a normal success page instead of a clear not-found. Soft 404s like that make bad links look "fine" to users and let junk URLs stick in search. The site scored 94/100 (A) in a pass that also surfaced 8 other critical/warning issues. I'm a Staff SDET, so happy to dig into more of the four-task flow if that's useful.

Victor • Oct 19

Thank you for sharing this feedback, This is a high value catch that offers bad UX and harm SEO health.
The Strategy i thought of to resolve this issue for the frontend and backend is to implement a dedicated responsive component with clear HTTP 404 semantics and a direct CTA back to the dashboard while for the backend, I will ensure that any unknow api routes and static asset routes return an explicit HTTP status with a standardized JSON error payload...

Yes, that’s very useful. Please dig into the remaining four-task flow and let me know the eight other critical issues.

Thank you very much.

wolf Valencia • Oct 19

Glad it helped, Victor. Your plan sounds right. One thing to watch: on Vercel a not-found component can still come back as a 200 if the app shell catches the route, so after you ship it, hit a made-up path with curl -I and make sure the status line really says 404.

Small correction on my end: the soft 404 was the only critical one, the other 8 are warnings. The two I'd do next are adding an H1 heading to the home page, since search engines lean on it to understand what the page is about, and putting one clear "call to action" (button or text) above the fold so a new visitor knows the first step.

Victor • Oct 20

Thanks for the heads up about how Vercel catches route! Testing a random-path with curl -I is a smart safety check, and I'll add that to my deployment phase.

Thanks for clarifying the warnings. I am already thinking and going through the codebase to try and locate part of the critical issues. Adding the H1 to the homepage and optimizing the above-the-fold CTA make perfect sense. What are the other 6 warnings on the list? I'd love to get them on my radar so I can clean them up next. Is there anything you can say about the UI and UX of the web app ? Thanks Valencia

wolf Valencia • Oct 20

Nice, putting the curl -I check into your deploy step means the 404 can't quietly come back.

Of the six left, two are worth doing:

I couldn't find a privacy policy linked anywhere on the site. When people are typing in their income and spending, a short privacy page linked from the footer goes a long way for trust.
The llms.txt one is really part of the 404 issue. There's no llms.txt file yet, so that address hands back your normal app page, and the scan read it as a file with no title. If you want one, add a real llms.txt with a first line like # Vicquant. If not, don't worry about it.

The rest you can mostly skip: three are email DNS records (SPF, DMARC, MX) that don't apply while you're on a vercel.app address, and one is a phone-number check meant for local businesses.

Here's the page with your score and problem areas: qa-audit.com/teaser/759e205d-7b82-...

The full report has every issue with a fix, plus a re-scan to confirm once you've fixed things.

On UI and UX, the scan doesn't judge that, so I won't guess. The cheapest test I know: sit a friend who's never seen it in front of the four tasks, say nothing, and watch where they pause. That usually shows you more than any checklist.

Victor • Oct 20

Thanks. You have been helpful

wolf Valencia • Oct 20

Anytime, Victor. Good luck with the launch, and ping me once the 404 fix is live if you want a second look.

Victor • Oct 21

I have fix the issue. Can you help me check it out again. I really apprecoate

wolf Valencia • Oct 21

Checked it this morning. The H1 is in and the privacy and terms pages load, nice work.

The 404 and llms.txt issues aren't resolved yet. Both still come back as your home page, and they share one root cause that shapes how every crawler sees the site, so it's the one worth getting right. There's also a new one since the change: your canonical tag points at a domain that doesn't load yet, which can lead search engines to skip the page. I won't go too deep here, but the full report walks through the exact fix for each and includes a re-scan to confirm once it ships. Happy to help you leverage it.

Victor • Oct 22

Thanks for the help. Is there a way for me to get the full report

Arhan Canli • Oct 15

The "no win rate until 30 resolved signals" rule is a good instinct, but 30 is still a thin sample, so I'd show the interval next to the number. I computed Wilson 95% intervals for n=30: 18 wins (60%) gives 42%-75%, and 20 wins (67%) gives 49%-81%. The chance of 20 or more wins out of 30 from a coin-flip strategy is 4.9%, and 21 or more is 2.1%. So a tester can see a 67% "win rate" at the first moment it is displayed and it still can't rule out luck once you have tried more than a few rules. To pin the half-width at about 5 points around 55% you'd need roughly 380 resolved signals.

Two things that would make the proof ledger stronger: record the rule version in each hash-chained entry (so changing a threshold later is visible), and show the count of rules or parameter sets tried before this one went live, since that count is what inflates the best one.

Also worth saying on the Sharpe/Sortino side of the app: with a few months of daily returns those ratios have wide error bars as well, so a "no finance degree" display should probably carry the same caveat. I built canli-mcp (free, MIT, no key), npx -y canli-mcp, and github.com/arhancanli/canlicapital has the free overfitting checks if you want something to test the ledger against.

Victor • Oct 16

This is top-tier feedback, thank you for taking the time to run the exact numbers. You’re completely right across all three fronts:

For the Wilson 95% Intervals & Coin-Flip Luck:
You nailed it on the thin sample illusion. Showing a standalone "67% win rate" at $n=30$ without the $[49\%, 81\%]$ interval creates a false sense of certainty, especially when a fair coin gives 20+ wins nearly 5% of the time. Because of that, I’ve updated the headline display so that the win rate never appears without its Wilson 95% interval right beside it (e.g., 66.7% [48.8%–80.8%] (95% CI)), along with the coin-flip luck probability and an explicit sample size indicator showing that ~380 resolved signals are needed to pin the margin of error down to $\pm 5\%$.

For the Rule Versioning & Parameter Sets Tried in the Ledger:
Adding rule_version directly into the canonical payload and SHA-256 hash chain is brilliant, it prevents silent threshold creeping after the fact. More importantly, recording the count of rules sets tried ($M$) before going live tackles selection bias head-on. If you try 15 – 20 parameter variations, the multiple-testing false positive rate balloons even if a single trial looks nominally significant.

For the Sharpe/Sortino Error Bars & "No Finance Degree" Caveats:
Spot-on. A 3-month annualized Sharpe of 1.50 has an enormous standard error (~2.0 under Lo's asymptotic formulation), meaning the true 95% interval easily spans from negative territory to over 3.0+. I also added the standard error / confidence intervals alongside the ratios, plus a clear, plain-language caveat that short-term risk metrics are preliminary noise until at least 2–3 years (500+ trading days) of history accumulate.

I Really appreciate the pointer to canli-mcp. I will definitely test the ledger and overfitting guards against your framework