Pocket Data Science IV: Tackling Kaggle MNIST on Android with Antigravity CLI

💻 GitHub Repository: All 10 self-contained experiment scripts, requirements, and reproduction instructions are published open-source on GitHub: myhlow/kaggle-digit-recognizer-mnist.

A hands-on walkthrough exploring computer vision baselines on Kaggle's Digit Recognizer (MNIST) benchmark: from empirical random guessing and prototype centroid templates to multinomial softmax regression and PCA-accelerated non-linear ensembles on an Android phone running Termux and Google Antigravity CLI.

---

In our previous explorations across the Pocket Data Science series (Titanic, Spaceship Titanic, and House Prices), we established disciplined, reproducible machine learning workflows on mobile hardware using Google Antigravity CLI (agy) inside a Termux Linux environment.

In this fourth installment, we step into computer vision using the classic Kaggle Digit Recognizer (MNIST) benchmark (42,000 training images, 28,000 test images, 784 grayscale pixels each).

Instead of jumping straight into heavy convolutional neural networks or spinning up cloud GPUs, we maintained our strict experimental discipline: understanding empirical priors, zero-learning prototype templates, linear decision frontiers, and memory-conscious non-linear ensembles running strictly on an ARM64 smartphone processor.

Across 10 distinct experiments over 2 days, our submissions climbed from naive random guessing (0.10139) all the way to 0.98792 (Rank #406 of 863 teams, Top 47.05%, beating 457 teams).

Here is the complete engineering breakdown of what each layer contributed and the mathematical mechanics behind the leaps.

---

1 · The Mobile ML Environment

All data loading, matrix transformations, cross-validation, and Kaggle submissions were executed natively on a consumer smartphone:

• Host Architecture: Android 14 running Termux with a Debian userspace via PRoot Distro on 64-bit ARM (aarch64).
• Agentic CLI: Google Antigravity CLI (agy) pair-programming in bash, managing background tasks, and verifying data pipelines.
• Environment: Python 3.14 with pandas, numpy, scipy, scikit-learn, and the official kaggle CLI.
• Constraints: Operating within smartphone RAM and thermal limits requires algorithmic efficiency—such as avoiding raw $\mathcal{O}(N^2)$ distance calculations across $42,000 \times 28,000 \times 784$ floating-point matrices.

---

2 · The 10-Tier Ladder (Day 1 & Day 2 Results)

Below is the chronological sequence of our 10 submissions evaluated against Kaggle's public test set (28,000 unseen images):

Exp	Model & Strategy	CV Accuracy	Kaggle Public Score	LB Rank	Percentile	Teams Beaten
Exp 01	Empirical Random Prior	N/A	0.10139 (10.14%)	#847	Bottom 1.9%	16
Exp 02	Majority Class Baseline (Digit 1)	11.35%	0.11403 (11.40%)	#846	Bottom 2.0%	17
Exp 03	Nearest Centroid ("Ghost Templates")	82.04%	0.81446 (81.45%)	#835	Bottom 3.2%	28
Exp 04	Multinomial Softmax Logistic Regression	92.56%	0.92089 (92.09%)	#806	Bottom 6.6%	57
Exp 05	Latent Ensemble: PCA(55) + k-NN + ExtraTrees	97.46%	0.97578 (97.58%)	#568	Top 65.9%	294
Exp 06	Tri-Blend: Deep MLP + k-NN + ExtraTrees	97.98%	0.98085 (98.09%)	#523	Top 60.7%	339
Exp 07	Orthogonal Subspace SVM: PCA(55) + RBF-SVM	98.60%	0.98453 (98.45%)	#487	Top 56.4%	376
Exp 08	Dual Meta-Ensemble: 90% RBF-SVM + 10% MLP	98.66%	0.98514 (98.51%)	#481	Top 55.7%	382
Exp 09	2× Translation-Augmented Subspace SVM (84k)	98.68%	0.98696 (98.70%)	#438	Top 50.8%	425
Exp 10	3× Dual-Axis Augmented Subspace SVM (126k)	98.78%	0.98792 (98.79%)	#406	Top 47.05%	457

(Note: Leaderboard statistics based on 863 active teams on Kaggle as of September 2026).

---

3 · Tier 1: Zero-Training Baselines & "Ghost Templates"

Before training parameterized models, baseline tests define the floor of predictability:

• Empirical Random Guessing (10.14%): Sampling labels according to the empirical training class distribution matched theoretical expectation for 10 balanced classes (~10%).
• Constant Majority Class (11.40%): Digit 1 is slightly more frequent than others (4,684 of 42,000 samples, or 11.15%). Predicting 1 for all 28,000 test images yielded 11.40% on Kaggle.
• Nearest Centroid Classifier (81.45%): By calculating the element-wise arithmetic mean of all 4,200 training images per digit class, we obtain 10 average "ghost templates". Classifying each test image to the template with highest cosine similarity requires zero parameter tuning or gradient updates, yet hits 81.45% accuracy, demonstrating the strong geometric separation already inherent in raw pixel space.

---

4 · Tier 2: Linear Frontier to Latent Subspace Ensembles

• Multinomial Softmax Regression (92.09%): Fitting a single convex linear layer $(W \in \mathbb{R}^{10 \times 784}, b \in \mathbb{R}^{10})$ using L-BFGS convergence in 16.5 seconds established the limit of linear hyperplanes.
• The Classical Latent Ensemble (97.58%): Non-linear models in raw 784-dimensional space are prohibitively slow on mobile CPUs. By projecting into an orthogonal 55-component PCA subspace (retaining 83.2% variance), we compressed the dataset by 14.25×. An ensemble blending 5-Nearest Neighbors and ExtraTrees (150 trees) on this subspace fit in 18.7 seconds and jumped 238 ranks on the leaderboard (0.92089 $\to$ 0.97578).
• The Neural Tri-Blend (98.09%): Adding a 4-layer Deep Multi-Layer Perceptron (256-128-ReLU with Adam and early stopping) to the latent ensemble pushed past 98% accuracy on Day 1.

---

5 · Day 2: Subspace Support Vector Machines & Spatial Manifolds

On Day 2, we systematically evaluated single models versus meta-ensembles and investigated spatial invariance:

Mobile Compute & Throughput Benchmark

Pipeline / Model	Architecture	Val Fit (5k CV)	Val Acc	Full Fit (Train)	Test Inference (28k)	Throughput
Deep MLP Baseline	Raw 784px $\to$ MLP(256, 128)	27.8s	98.18%	344.4s (5.7 min)	2.5s	~11,200 img/s
Raw RBF-SVM	Raw 784px $\to$ RBF-SVC(C=5)	148.6s	98.24%	~25 min (est.)	N/A	High latency
Subspace RBF-SVM (Exp 07)	PCA(55) $\to$ RBF-SVC(C=5)	7.2s	98.60%	22.5s (42k train)	19.6s	~1,425 img/s
Dual Meta-Ensemble (Exp 08)	90% Subspace SVM + 10% MLP	364.6s	98.66%	544.1s (9.1 min)	36.7s	~763 img/s
Augmented Subspace SVM (Exp 09)	2× Spatial Jitter + PCA(55) + SVM	50.8s (74k)	98.68%	51.4s (84k train)	52.9s	~529 img/s
Dual-Axis Aug SVM (Exp 10)	3× Spatial Jitter + PCA(55) + SVM	91.8s (111k)	98.78%	205.6s (3.4 min, 126k)	168.9s (2.8 min)	~166 img/s

Key Technical Insights

1. The Latent Space Denoising Win:
   Projecting onto a 55-dimensional orthogonal subspace did not just accelerate training—it acted as an optimal low-pass filter against boundary pixel noise. Subspace SVM achieved 98.60% validation accuracy versus 98.24% on uncompressed 784 pixels while training 17× faster (7.2s vs 148.6s).

2. The Dual-Axis Spatial Augmentation Breakthrough:
   Standard RBF kernels have no native translation invariance—shifting a handwritten digit by just 1 pixel changes its Euclidean distance in pixel space significantly. In Experiment 10, tripling the training partition to 126,000 samples by generating systematic horizontal (±1px on the X-axis) and vertical (±1px on the Y-axis) translational shifts provided complete 2D shift-invariance.
   Fitting the 126,000-sample pipeline took just 3.4 minutes on mobile CPU and propelled our Kaggle score to 0.98792 (Rank #406 / Top 47.05%, beating 457 teams)!

---

6 · Day 2 Milestone Achieved: Top 47.05% on Mobile Hardware (Rank #406 / 863)

With Experiment 10 scoring 0.98792, we officially crossed the Top 50% cutoff (0.98739, Rank #431) to claim Rank #406 out of 863 active teams worldwide.

• Zero GPU Resources: The entire progression—from empirical random guessing (0.10139) to classical ensembles (0.97578) and 126k dual-axis augmented subspace SVM (0.98792)—was engineered, trained, and submitted entirely on an Android smartphone CPU via Termux and Antigravity CLI.
• Looking Ahead to Day 3: With all 5 daily submissions successfully completed for Day 2, our next frontier is custom lightweight Convolutional Neural Networks (LeNet-5 & Modern Compact ResNets) built directly on PyTorch ARM64 to break into the Top 20%.

---

Pocket Data Science Series · Tested on Android 14 / Termux · Debian ARM64 PRoot · Google Antigravity CLI

All experiment scripts (exp01_random_baseline.py through exp10_dual_axis_augmented_svm.py) and submissions are archived locally in /root/digit-recognizer/.