What Predicts a Hit? I Trained 3 ML Models to Find Out

Cara Jung — Mon, 06 Apr 2026 07:00:00 +0000

In many entertainment adaptation decisions, content selections are still instinct-driven. Maybe a producer was vibing with a story or overheard their Gen Alpha nephew mentioning a GOAT title. This subjective approach has often led to expensive missteps and wasted resources for studios when the feature or show turns into a flop.

As someone who has worked in the breeding ground of popular webcomics, I asked: what if there was a system that could measure “success potential” of IPs based on real user behavior? Using ML, I wanted to see if I could build a forecasting model that could rank unadapted titles by their predicted commercial success.

The Data

For my endeavor, I worked with three datasets:

Source material metadata of roughly 1,500 titles that included engagement metrics such as views, likes, subscribers, genre, release schedule, and creator usernames
Produced show metadata of 1,977 titles including ratings, watcher counts, genre, episode count, and cast
Historical webcomic adaptation records of 424 cross-referenced titles that went from source material to screen, with data pulled from both sides

Before any modeling, I ran exploratory data analysis on all three and found a few things:

Engagement metrics (likes, views, subscribers) were strongly correlated with each other and overall popularity
Genre and tags correlated with watcher counts in the produced show data
Creator frequency showed no statistically significant impact on adaptation success, which directly contradicted what studios commonly assume

Engineering the Target Variable

One hurdle I ran into was that I couldn't directly measure adaptation “success" from the source material side alone. So I engineered a composite Popularity Score by normalizing and combining views, likes, and subscribers into a single metric representing audience appeal, which became the target variable for prediction.

For the produced show data, I created a parallel score using rating and watcher count.

Since correlation analysis confirmed that source popularity and show popularity moved together in historical adaptations, I used source popularity as a proxy target.

Simple vs Complex Models

I implemented three models: Random Forest, XGBoost, and Ridge Regression. If you worked with ML models, there’s an expectation that the more complex models will win. However, this wasn’t the case. Ridge Regression became the unexpected underdog model that won:

I cross-validated all three models to reduce overfitting and validate stability.

Likes = Success

Using standardized coefficients for feature importance in the Ridge model, the ranking was as follows:

Likes (strongest predictor by a significant margin)
Views
Subscribers

The factors that studios often focus on such as creator reputation, genre, rating, and engagement rate showed weak or no statistical significance.

I validated this further using Mann-Whitney U tests comparing adapted titles against the general pool. Adapted titles showed significantly higher “likes” than non-adapted ones and the difference was meaningful.

Feature Importance for Ridge regression(standardized coefficients)

So why “likes”?

One interpretation is that likes are intentional. A view can be passive while a subscription can be habitual. But giving a “like” is an act of emotional investment and this behavior is exactly what translates from IP to screen.

The Output

The final model produced a ranked list of the top 10 unadapted webcomic titles by predicted success, along with contextual signals for each including genre appeal, subscriber trends, engagement consistency, and creator track record.

Qualitative review of the top 10 confirmed alignment with the engagement patterns seen in historically successful adaptations. Cliff's Delta calculations showed that the predicted top titles had significantly higher likes than past adaptations.

Limitations on the Model

Part of doing good data work is being honest about the limitations. There were a few things that fell short:

Small adaptation dataset. 424 entries is workable, but more data would reduce overfitting risk and better generalization.
Proxy target variable. Using source popularity instead of actual show performance is a justified simplification, but it means the model can't fully capture real-world production quality, casting, or distribution reach.
Categorical features dropped. Creator and genre have too many levels and their coefficients dominated the model without adding significance. Excluding them improved interpretability but at the cost of losing some nuance.

What I'd Do Next

If I extended this project, I'd rethink how signal is captured and focus on the following:

Use NLP for deeper context
- Synopsis embeddings or sentiment analysis on reader reviews could capture thematic richness that raw engagement metrics miss.
Take a hybrid ranking approach
- Combining regression with a learning-to-rank algorithm could improve recommendation quality at the top of the list, where small differences actually matter.
Longitudinal validation
- The real test is tracking what happens when predicted titles actually get produced. Building a feedback loop into the model would sharpen it over time.

Final Thoughts

The core insight here doesn’t only strictly apply to entertainment. It can apply to decisions that are being made by intuition or legacy practice. As the models showed, behavioral signals from real users outperform assumptions about what will succeed.

Likes beat creator prestige. Engagement beat genre conventions. The audience’s preferences, not the ones from industry decision makers, predicted outcomes more reliably.

Whether you're choosing which content to produce, which features to build, or which markets to enter, the same principle applies. The answers are within the data, but we often overlook the right signals.

DEV Community: Cara Jung

What Predicts a Hit? I Trained 3 ML Models to Find Out