DEV Community: eventhorizon-ai

I'm 14 and Spent 276 Days Building an AI Platform. Here's What I Learned

eventhorizon-ai — Mon, 12 Oct 2026 18:55:47 +0000

276 days ago, I started building EventHorizon.

I wanted to create software that could solve real problems using machine learning. I started with more ambition than experience, and over time, I learned that building a model is only one part of building a useful product.

Since then, I've experimented with demand forecasting, cryptocurrency prediction, validation tools, and customer outreach.

Here's what I learned—including what didn't work.

1. Building the Models Was Only the Beginning

One of my main projects became EventHorizon-AI, a demand forecasting platform designed for small businesses.

The idea was straightforward: help businesses make better decisions about how much stock to prepare or order by predicting future demand.

I also experimented with short-term BTC/USDT price direction prediction.

These projects had different goals, but they taught me the same lesson: a model that produces predictions isn't automatically useful.

You need to understand what the numbers actually tell you, how the model was evaluated, and whether its predictions can create value in the real world.

2. Demand Forecasting: A Measurable Improvement

For demand forecasting, I experimented with the M5 Walmart dataset.

My model used features such as historical sales, lagged values, seasonality, prices, and promotions.

The validation results were:

Model	WAPE
Baseline	44.4%
EventHorizon model	31.03%

That's approximately a 30% reduction in WAPE relative to the baseline.

WAPE, or Weighted Absolute Percentage Error, measures total absolute prediction error relative to total actual demand.

This was an encouraging result. In my evaluation, the improvement appeared across all seven stores.

However, a benchmark result is not the same as a successful deployment in a real business.

The M5 dataset provides a useful environment for experimentation, but it doesn't prove that the same performance will transfer to a particular bakery, florist, or other small business. Real-world validation is a separate challenge.

3. Cryptocurrency Prediction: Better Accuracy Wasn't Enough

I also built a short-term BTC/USDT directional prediction system.

The experiment used a Transformer-based architecture combined with a BiLSTM and cross-attention. It was trained on approximately 1.5 million samples, with predictions across multiple time horizons.

In my evaluation, the model achieved around 60% directional accuracy, compared with a baseline of approximately 52%.

At first glance, that looked promising. But there was a problem:

Statistical performance isn't the same as economic profitability.

Once I considered transaction fees and execution-related costs, the apparent edge wasn't economically viable under the conditions I tested.

That changed how I thought about machine learning projects.

A model can outperform a baseline and still fail at the task that actually matters. The evaluation objective must reflect the intended use of the system.

For a trading model, directional accuracy alone isn't enough. For a demand forecasting model, lower prediction error alone doesn't guarantee better business outcomes.

The metric must connect to the real-world objective.

4. Validation Deserves Its Own Attention

As I worked on these projects, I became increasingly interested in how to determine whether an apparent improvement was actually meaningful.

I developed an honest-validation toolkit with utilities for:

Walk-forward validation splits.
Block bootstrap and gap bootstrap procedures.
Permutation testing.
WAPE and MASE evaluation metrics.

These tools helped me approach model evaluation more systematically.

I learned to pay attention to temporal structure, possible information leakage, baseline selection, and the difference between an observed improvement and evidence that a model will generalize.

A good metric alone isn't sufficient evidence. The evaluation process matters just as much as the result.

5. Building Something Useful Requires Talking to People

Technical experiments were only one side of the project.

I also started reaching out to small businesses, including bakeries, florists, and other potential customers. I wanted to understand how they made decisions about demand, inventory, and daily production.

I learned that the challenge isn't simply building a prediction model. Businesses already have their own methods, experience, constraints, and sources of information.

A technically interesting solution isn't automatically one that someone will adopt.

My outreach hasn't produced a paying customer or a real pilot so far. That's an important part of the story, too.

I still need to validate whether the platform can provide enough practical value for a business to use it. Better model metrics don't answer that question by themselves.

6. What I Would Do Differently

Looking back, I'd put even more emphasis on a few things from the beginning:

Define the baseline first. A model's performance means little without a meaningful comparison.
Design validation around the data. Time-dependent data requires particular care with splits, leakage, and evaluation methodology.
Separate technical progress from product progress. A better model doesn't necessarily mean a better product.
Document limitations honestly. Failed experiments and negative results can be as informative as successful ones.
Talk to potential users earlier. It's possible to spend months solving a technically interesting problem without establishing that the proposed solution fits how people actually work.

These lessons are changing how I approach my next experiments.

7. What Comes Next

I'm continuing to work on EventHorizon with a stronger focus on reliable evaluation, practical usefulness, and understanding where forecasting can genuinely help small businesses.

I want to build systems that do more than produce impressive-looking numbers. I want to understand why they work, where they fail, and under which conditions their results can be trusted.

There's still a lot to learn, both technically and commercially. And that's okay.

Final Thoughts

I'm 14, and I spent 276 days building and experimenting with an AI platform.

I haven't built a successful business yet. I haven't demonstrated that every model I've developed is useful in production. Some experiments showed promising results; others taught me why an apparently good result isn't enough.

But I now have a much better understanding of model evaluation, time-series forecasting, statistical validation, and the gap between a working experiment and a useful product.

For me, the biggest lesson has been this:

Building a model is an engineering task. Determining whether you can trust it—and whether it solves a real problem—is a much bigger challenge.

I'm continuing to learn, experiment, and document what I find.

If you're working on forecasting, machine learning validation, or independent software projects, I'd be interested to hear what you've learned from your own experiments.

Explore EventHorizon

If you'd like to explore the project or follow my work:

GitHub: https://github.com/EventHorizon-ia/EventHorizon

I'm continuing to experiment with demand forecasting, model validation, and practical applications of machine learning. Feedback and technical discussions are welcome

Lessons from building a Multi-Horizon Crypto Prediction System with Transformers

eventhorizon-ai — Sat, 11 Jul 2026 14:24:31 +0000

I started this project in January 2026 with a simple question: can a Transformer trained on technical indicators predict Bitcoin's direction better than a coin toss?

By June, the system had processed over 87,000 training generations, survived three major architectural bugs, and taught me more about machine learning than any course could.

This is not a "how to get rich with AI" post. This is the engineering story.

Architecture Overview

The system uses a hybrid Transformer + BiLSTM architecture:

Transformer: 6 blocks, 8 attention heads. Same architecture that powers GPT and BERT, adapted for financial time series. The self-attention mechanism captures short-term patterns across different technical indicators.
BiLSTM: 2-layer bidirectional LSTM. Handles long-range dependencies that the Transformer might miss — for example, regime changes that unfold over hours or days.
Cross-attention layer: Fuses the representations from both branches before feeding them to the prediction heads.
10 specialized heads: One per temporal horizon — from 5 seconds to 1 day. Each head has a different depth and regularization, reflecting the noise profile of its horizon.

The entire network is custom-built in PyTorch. No AutoML. No third-party trading libraries. Every tensor, every gradient, every loss function — implemented from scratch.

The Verification Pipeline

Most crypto prediction systems show backtest results. Backtests lie.

I built a verification pipeline that timestamps every prediction. When the target horizon elapses, the system compares the prediction against the real Binance price and logs the result. No cherry-picking. No deleted trades. The audit panel is public.

This commitment to transparency became the project's core value. But it also exposed a bug that almost made me quit.

Bug #1: Autocorrelation (The 96% Lie)

I was getting 96% accuracy on 5-minute predictions. I celebrated. Then I looked at the raw data:

[0, 0, 0, 0, 0, 0, 0, 0, ... 300 zeros in a row ...]
[1, 1, 1, 1, 1, 1, 1, 1, ... 200 ones in a row ...]

No model produces results like this. The problem was temporal dependence: I was recording a "trade" every 2 seconds, then verifying all ~900 snapshots against the same market event. The model wasn't predicting price direction — it was just reflecting whether the market went up or down in that window.
I fixed this with statistical subsampling:

_FATOR_SUBSAMPLE = {
    5: 1,       # 5s: record every tick
    60: 5,      # 1min: record every 5th tick
    300: 25,    # 5min: record every 25th tick
    1800: 150,  # 30min: record every 150th tick (5 min)
    3600: 300,  # 1h: record every 300th tick (10 min)
}

Accuracy dropped from 96% to ~52%. It hurt. But now every trade in the history represents an independent decision by the model.

Bug #2: Look-Ahead Bias

If you predict a 1-hour horizon, you should only receive the reward 1 hour later — not a second sooner. But my early implementation allowed future price information to leak into the training signal.
The fix was a Delayed Reward Buffer. Every prediction is stored with its timestamp. The reward is computed and applied only when the target horizon actually elapses. The model never sees the future during training.

Bug #3: Gradient Truncation

For weeks, my Transformer and LSTM weren't actually learning. The gradient from the loss function was stopping at the prediction heads — it never reached the backbone.
I had to recompute the full forward pass during gradient computation, ensuring the error signal propagated through the entire network. My computer fans immediately spun up louder. The backbone was finally learning.

Current State

After fixing these bugs and training on 365 days of Binance data with a T4 GPU:

54.9% average directional accuracy (5s–1h horizons)
59.8% on 1-hour predictions (best individual horizon)
All metrics are publicly auditable against real Binance prices These numbers won't make anyone rich overnight. But they're real. And in a market saturated with fake performance claims, real metrics are a differentiator. ## What I Learned Building this system taught me three things that no course could:
Trust metrics, not feelings. When accuracy is 96%, something is broken. When it's 53%, it might be working perfectly. The numbers tell the truth — if you let them.
Transparency compounds. The audit panel was a burden to maintain at first. Now it's my strongest signal of credibility. In a market full of scams, showing your losses is more convincing than hiding them.
Patience over hype. The hardest bug (autocorrelation) took three days to diagnose and one line to fix. There is no shortcut.