A known-data check of red and white wine domain transfer
A model that looks useful on its own training source can still make worse per-record choices than a constant once it crosses to another source. I used the UCI Wine Quality files to make that gap visible with two small regressors, one trained on red wine records and one on white. The target is the sensory quality score from the source. This article records the saved run from November 4, 2026; publishing the article adds no new training.
The source files contain 1,599 red rows and 4,898 white rows, 6,497 in total. The UCI page summary of 4,898 instances describes white only. Each row has 11 physicochemical inputs and an ordered quality score from 0 to 10. The source names state that the score is the median of at least three expert evaluations. Individual chemistry units are not supplied in the files, so I keep the numeric scales as they appear. Treating that ordered score as an equally spaced number for MAE and MSE is a modeling assumption, not evidence that the score sits on an interval scale.
Before training, I assigned each record a role using only its 11 canonical feature values. I decimal-normalized the tokens, joined them, hashed the prefixed key with SHA-256, and used the integer modulo 10. Values 0 through 5 became a fit pool, 6 and 7 became a validation pool, and 8 and 9 became test. The same feature vector receives the same initial role in both colors, and quality and color are excluded from the key. That produces 5,318 unique groups among 6,497 rows and 1,179 duplicate-excess rows. Two exact input vectors appear in both colors. Exact vector identity is a duplicate control, not a producer or subject ID, and it does not prove that repeated rows are independent samples. Duplicate rows still enter the batch arithmetic, so they contribute repeated weight. Whether those repeated rows are ultimately independent measurements does not change that computational weighting.
The red source had 923 fit-pool rows, 335 validation-pool rows, and 341 test rows. The white source had 2,939 fit-pool candidates and 977 validation-pool candidates. I sampled 923 fit rows and 335 validation rows from white, leaving 2,016 fit candidates and 642 validation candidates unused. Tests were never downsampled, so the frozen test set is the full 341 red rows plus 982 white rows, 1,323 cases in total. Those same 1,323 cases are reused across three seeds in the paired comparison below.
Each source fit has 923 rows, 100 epochs, and batch size 64, giving 1,500 updates and 92,300 row visits per fit. The network is 11 inputs, 24 tanh units, and one linear output, 313 parameters. Seeds 17, 41, and 73 run for each source, giving six saved models. One initial parameter set per seed is shared between the red and white fits, three sets total; each set holds W1, b1, W2, and b2. Because each domain has its own train-only normalizer, a shared set does not necessarily define the same function on raw input scale. Validation and test rows never enter the normalizer statistics.
Checkpoint selection used each model's own 335-row validation set. I picked the earliest epoch with the strictly lowest MAE among all 100 checkpoints. Selected epochs were 31, 78, and 31 for red, and 52, 39, and 39 for white. Cross-domain evaluation never used the target color's validation set. An in-domain model's own source is also its target source, so that model did see the target-source validation set.
The aggregate MAE numbers are arithmetic means over three seeds, not ensembles and not confidence intervals.
| Test domain | Red-trained | White-trained | Red fit mean | White fit mean | Cases per seed |
|---|---|---|---|---|---|
| Red | 0.476144 | 0.746975 | 0.647566 | 0.636872 | 341 |
| White | 0.649987 | 0.586350 | 0.748883 | 0.674610 | 982 |
The source-matched model has the lower mean error on both targets. The negative result sits inside that summary: the white-trained network is worse on the red holdout than its own fit-mean constant, 0.746975 versus 0.636872. Per seed, the three white-to-red MAE values are 0.657548, 0.732750, and 0.850627, each above that constant. The red-trained network beats the red constant on red, and the white-trained network beats the white constant on white. This is a specific cross-domain failure, not a statement that every neural model loses to every constant. The numbers also do not say that the white-trained model mostly emits its white training center, or that it learned no joint relation. They record that the aggregate error is worse than ignoring the inputs and returning the white fit mean.
The per-case view shows how much an average hides. Across 3,969 same-seed, same-case paired comparisons, the in-domain model had the larger absolute error in 1,585 pairs. By target domain, 321 of 1,023 red-target pairs and 1,264 of 2,946 white-target pairs went against the in-domain model. A negative value in the paired difference means the in-domain model was worse. These counts reuse the same 1,323 cases over three seeds. They are descriptive repetition, not 3,969 independent cases, and I did not run a significance test. The target-domain split is reported, but the denominators differ, and the repeated cases mean I cannot infer from those totals that one target domain explains the failures.
The fixed default case is red:0002. I selected it before any current-model test prediction as the lowest red test source ordinal. At seed 17, the red-trained model predicts 5.365765 and the white-trained model predicts 4.872778 for a true score of 5. The cross-domain model has the smaller error. I keep this case as a negative, not as a selected winner.
Figure: the actual study.png at 2015 by 1271 pixels, with all 600 own-source validation MAE points, the four seed-mean MAE bars and analytic fit-mean diamonds, and the full paired difference distributions. It is a descriptive known-data contrast, not a training-loss plot, screenshot, or causal diagram.
Equal rows, equal updates, and paired initial sets are computation controls. They do not isolate a source shift. The red fit pool contains all 923 red fit candidates. The white fit pool samples 923 from 2,939 white fit candidates. Train-only normalizers differ between the two pools. Feature support, quality distribution, own validation selection, and duplicate weighting differ at the same time. The source label is not the only thing that changes. A difference between the two training-pool feature means only shows that the pools differ. It does not prove that the white-to-red failure comes from a wrong center, a missing chemical relation, or another single mechanism.
Before assigning a mechanism, I would need separate runs that actually change the sampled training records under stated controls. One candidate would match the two pools by quality counts and overlapping feature support, keeping each domain's own normalizer and validation split, then check whether the white-to-red network still loses to its fit mean. Another would reweight duplicate rows down and see whether the headline errors move. Neither is part of this frozen run. The current result is a known-data descriptive comparison with the failures retained.
The fixed repository includes PROTOCOL.md, evaluation-001/results.json, and evaluation-001/paired-comparisons.json. Downloading only the static app page does not give you the complete set of research files. From the project root, after obtaining the fixed checkout at commit 2431a336acc4cef56f12b22a90bc77baed8c6863, the following Python 3 standard-library command recounts the headline numbers and the worse-pair split. It opens the two JSON files read-only. It does not download anything, install anything, fit anything, or write anything.
python3 - <<'PY'
import json, collections
r = json.load(open("evaluation-001/results.json"))
p = json.load(open("evaluation-001/paired-comparisons.json"))
for d in r["descriptive_seed_metric_means_not_ensemble"]:
print("neural", d["target_domain"], d["trained_domain"], "mae", d["arithmetic_mean_seed_mae"])
for b in r["all_mean_baselines"]:
print("mean", b["trained_domain"], "->", b["target_domain"], "mae", b["mae"])
c = collections.Counter((x["target_domain"], x["in_domain_worse"]) for x in p)
print("worse_split", "red", c[("red", True)], "white", c[("white", True)], "total", r["in_domain_worse_pairs"])
print("reuse_check", r["paired_rows"], "==", 3, "*", r["test_cases"], ":", r["paired_rows"] == 3 * r["test_cases"])
PY
The official space and the static demo load six saved forward models and two analytic constants. The demo is static, with no external API or browser storage. Edits entered in the page live in browser memory, so a refresh discards them. A deliberate export writes all eleven inputs, the six model outputs, the two mean baselines, normalizers, source matches, and provenance to a local JSON file; the file stays where the reader saves it. I state this so no one trusts unsaved work to survive. Input range checks are marginal bounds, not joint chemical support. An exact feature-key match is numeric consistency, not measurement provenance. Outputs are unrounded and unclipped, with no calibrated uncertainty or probability.
The original red and white CSV files and the derived weights, figures, and results retain attribution to Cortez, Cerdeira, Almeida, Matos, and Reis, UCI 186, DOI 10.24432/C56S3T, under CC BY 4.0. My code and documents are Apache 2.0. That split is recorded in NOTICE and LICENSES.md. The 2009 paper is cited as metadata; I did not read its full text, did not reproduce its model or scores, and did not use its results as my own. No tasting, health, certification, or future-vintage claim follows from this experiment.

Top comments (1)
Showing that the white-trained model loses to a constant on red, with all three seeds above the 0.637 fit mean, says more than "transfer is worse" would. Keeping the 1,585 reversed pairs next to the averages helps too. I ran into something similar with retinal images: a model that transferred well to one external set did clearly worse on another, and the average hid that the failure sat in one grade.
On your first open question, comparing the spread of the white-trained predictions on red with the spread of red's true scores might separate "outputs its training center" from a learned relation fairly cheaply. Did you look at the prediction variance per seed?