Why a random train/test split flatters concrete-strength models (and what we did instead)

ProTechSC Academy — Sat, 14 Nov 2026 15:26:55 +0000

Predicting the compressive strength of concrete from its mix is one of the most popular machine-learning exercises in civil engineering. Most published models report an R² above 0.9. We rebuilt the problem from scratch and found that how you split the data matters as much as which model you pick.

The data

We used the public UCI Concrete Compressive Strength dataset (I-C. Yeh, 1998, CC BY 4.0). It contains 1,030 laboratory tests:

8 inputs: cement, GGBS (slag), fly ash, water, superplasticizer, coarse and fine aggregate (kg/m³), and age (1–365 days)
output: compressive strength, 2.3–82.6 MPa

A detail that is easy to miss is that the 1,030 rows are only 428 distinct mixes. The same mix was tested at several ages. We also found 25 exact duplicate rows, which we removed.

The trap: random splits leak information

With a random 80/20 split by row, the same mix tested at 7 days can land in training while its 28-day result lands in the test set. The model never has to generalise to a new mix. It only interpolates along an age curve it has already seen.

A real engineer asks a different question: "what will this new mix give?" So we compared three protocols:

Random 80/20 by row, as in most papers (optimistic)
Grouped 80/20 by mix and grouped 10-fold CV, where a whole mix is either in training or in test. We used this for model selection.
Age extrapolation: train on tests up to 28 days, predict 56–365 days

RMSE of 11 models under the three validation protocols

Physics first, then machine learning

Before any ML, we fitted a white-box baseline with only 7 parameters:

Abrams' law (strength falls exponentially with the water/binder ratio), with efficiency factors for slag and fly ash
the ACI 209 strength-gain curve for age

This baseline alone gives an RMSE of 7.35 MPa (R² 0.80). It is not accurate enough, but every term can be checked by a civil engineer.

We then trained 10 ML models: linear, quadratic, tree, SVM, Gaussian process, random forest, boosted trees, neural network and monotonic boosted trees. Finally, we trained a hybrid, in which the physics model gives the trend and a boosted tree learns only the correction:

strength = f_Abrams-ACI(mix, age) × exp(boosted_tree_residual(mix, age))

What we measured

Model	Grouped test RMSE	Age-extrapolation RMSE
Abrams-ACI physics only	7.35 MPa	—
Random forest	4.74 MPa	9.54 MPa
Boosted trees (LSBoost)	4.82 MPa	8.93 MPa
Gaussian process	5.40 MPa	5.60 MPa
Hybrid physics + boosted trees	4.62 MPa (R² 0.92)	6.03 MPa

The pure tree models look almost as good as the hybrid on new mixes. Ask them about a later age and they fail, because a tree cannot extrapolate beyond the ages it was trained on. The physics trend in the hybrid keeps it sensible.

Trained up to 28 days, tested at 56–365 days

We also report a 90% prediction interval (±7.7 MPa) using split-conformal prediction on grouped out-of-fold residuals. It achieved 89% coverage on unseen mixes. A single "R² = 0.92" hides how uncertain any one prediction is.

Using the model in reverse: cheaper, lower-CO₂ mixes

Once the model is trustworthy, it can design mixes. Differential evolution searched for the cheapest or lowest-CO₂ mix for each grade, M25 to M50. The search had to satisfy these constraints:

the IS 10262 / IS 456 / IS 16415 limits: target mean strength, maximum w/b, minimum binder, and the slag and fly-ash caps
the range of the training data
a physics-consistency check: the Abrams-ACI model must also agree within 7 MPa

That last constraint turned out to be essential. Without it, the optimiser found mixes where the ML model predicted 48.6 MPa but physics said 38.5 MPa. In other words, it was exploiting the model's errors instead of finding better concrete.

With the check in place, the model proposes 22–29% lower material cost and 50–59% lower embodied CO₂ than a conventional OPC design. The main reason is a higher use of GGBS and fly ash.

Conventional vs optimised mixes, M25–M50

Limitations

These are model proposals. They need trial batches and IS 516 cube tests before use.
The dataset is from one laboratory's 1990s materials and gives cylinder strengths, not Indian cube strengths. A local calibration is needed.
Part of the cost saving comes from IS 456 durability limits, which make conventional OPC mixes stronger than needed at low grades.

Takeaways for your own ML project

Check whether your rows are independent. If they are not, split by group (here, by mix).
Start with a physics baseline. It shows how much the ML actually adds, and it rescues extrapolation.
Report an interval, not just R².
When you optimise on a surrogate, guard against it being exploited.

The full project (report, MATLAB and Python versions, web application, video walkthrough) is described on our project page: Concrete strength prediction — ProTechSC Academy. A short video walkthrough: youtu.be/Lu_MFsTvl6o.

We guide final-year and PhD research projects in engineering and AI. WhatsApp +91 97907 09424.

DEV Community: ProTechSC Academy