DEV Community

Malcolm Low
Malcolm Low

Posted on Originally published at malcolmlow.com

Pocket Data Science II: Tackling Kaggle Spaceship Titanic on Android with Antigravity & CatBoost

#machinelearning #python #android #datascience

Training a 10-fold cross-validated ensemble directly on an Android smartphone via Termux and Google Antigravity CLI, progressing from a random baseline to competitive standings on Kaggle Spaceship Titanic, and exploring deterministic domain rules versus threshold drift.

In Part 1 of this series, we gave an autonomous AI coding assistant—Google Antigravity CLI (agy)—direct bash terminal control inside an Android Linux environment to tackle Kaggle's classic Titanic benchmark. That experiment carried us to Rank #291 (Top 2.89% out of 10,058), but concluded with a critical lesson: historical benchmark competitions with real-world passenger lists are deeply vulnerable to data leakage.

To test whether agentic mobile data science could conquer a benchmark with zero possibility of leakage, we challenged Antigravity with its modern, sci-fi companion: Kaggle's Spaceship Titanic.

With 12,970 passengers traveling aboard a luxury interstellar liner engulfed by a spacetime anomaly, this competition features rich tabular signals: cryogenic suspension, cabin deck layouts, five granular expenditure categories, planetary origins, and transit groups.

Operating entirely on an Android phone, the autonomous agent engineered domain-specific spatial coordinates, discovered 100% deterministic imputation rules, managed multi-model cross-validation, and vaulted to Rank #98 out of 1,597 competitors (Top 6.14%) using pure, uncompromised machine learning.

1 · The Pocket ML Architecture

Executing high-iteration cross-validation pipelines on mobile hardware requires managing constrained CPU resources, memory footprint, and operating system permissions:

  • Host Platform: Android 14 running Termux with an unprivileged Debian userspace via PRoot Distro on 64-bit ARM (aarch64).
  • Agentic Orchestrator: Google Antigravity CLI (agy) operating in asynchronous task mode. By configuring root command prefix allowlists in settings.json (command(find), command(python3), command(kaggle)), the agent executes shell commands, code updates, and background training runs with zero manual prompt friction.
  • Algorithm Stack: Python 3.14 with ARM64-optimized catboost, scikit-learn (HistGradientBoostingClassifier), pandas, numpy, and the official kaggle CLI. 
+--------------------------------------------------------------+
|                     Android 14 (ARM64)                       |
|   +------------------------------------------------------+   |
|   |                 Termux / PRoot Distro                |   |
|   |   +----------------------------------------------+   |   |
|   |   |        Google Antigravity CLI (`agy`)        |   |   |
|   |   |  - Async Task Spawner  - Settings Allowlist  |   |   |
|   |   +----------------------+-----------------------+   |   |
|   |                          |                           |   |
|   |      +-------------------+-------------------+       |   |
|   |      v                                       v       |   |
|   |  [Feature Pipeline]                     [Kaggle CLI] |   |
|   |  - Deterministic Imputation             - Submissions|   |
|   |  - Spatial Coordinates & Spend          - Public LB  |   |
|   |      |                                               |   |
|   |      v                                               |   |
|   |  [10-Fold Stratified Cross-Validation]               |   |
|   |  - CatBoost (Ordered Boosting)                       |   |
|   |  - HistGradientBoosting (Histogram GBDT)             |   |
|   +------------------------------------------------------+   |
+--------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

2 · The Experiment Progression: From Bottom 2% to Top 6%

Over five iterative experiments, the mobile pipeline climbed from a blind coin-flip baseline to the top 6% of the global leaderboard:

Experiment Architecture & Strategy Local CV / OOF Public Score Leaderboard Rank Percentile Integrity
Exp 01 Stratified Random Prior Baseline N/A 0.51110 #1,568 Bottom 2% Anchor
Exp 02 CatBoost 80:20 Holdout Baseline 0.8056 0.80009 #902 Top 56.48% Pure ML
Exp 03 5-Fold CatBoost + Domain Engineering 0.8163 0.80967 #98 Top 6.14% 🏆 Pure ML Peak
Exp 04 10-Fold Blended Stack (Threshold Shift to 0.480) 0.8205 0.80547 #429 Top 26.86% Overfit Threshold
Exp 05 10-Fold Tuned CatBoost (Prior Cutoff 0.500) 0.8172 0.80757 #234 Top 14.65% Pure ML

3 · The Breakthrough: Deterministic Domain Deductions

Before training any complex models, Antigravity conducted an exploratory audit of the dataset's logical structure. Tabular machine learning often stumbles when treating missing values as random noise (NaN). In Spaceship Titanic, the data contains rigid domain laws that can be solved deterministically:

1. The 100% Purity Surname-to-Planet Law

Each passenger name contains a surname (Name.split()[-1]). An analysis across all 12,970 passengers revealed 2,400 unique surnames. Crucially:
$$\text{Surnames with passengers from multiple home planets} = \mathbf{0}$$

Every family surname belongs to exactly one planetary culture (Earth, Europa, or Mars). By creating a reverse dictionary of known surnames, the agent recovered 271 missing HomePlanet entries with 100% mathematical certainty.

2. Physical Deck Constraints

The spaceship's architecture enforces strict passenger zoning:

  • Decks A, B, C, and T: Exclusively luxury European cabins (100% Europa, 0 Earth, 0 Mars).
  • Deck G: Exclusively steerage Earth cabins (100% Earth, 0 Europa, 0 Mars).
  • If a passenger's HomePlanet was missing but their cabin deck was known, their origin was imputed with zero error.

3. The CryoSleep Expenditure Paradox

Passengers in cryogenic suspension are frozen in sealed pods throughout the voyage:
$$\text{Passengers in CryoSleep who spent money on amenities} = \mathbf{0}$$

If a passenger has missing CryoSleep data but logged spending $> 0$ on RoomService, FoodCourt, ShoppingMall, Spa, or VRDeck, their CryoSleep status must be False. This recovered 174 missing values instantly. Conversely, missing amenity values for passengers confirmed in CryoSleep were set directly to 0.0.

# Deterministic domain imputation pipeline
# 1. CryoSleep resolution
full.loc[full['TotalSpend_raw'] > 0, 'CryoSleep'] = full.loc[full['TotalSpend_raw'] > 0, 'CryoSleep'].fillna(False)

# 2. Deck zoning resolution
full.loc[full['Cabin_Deck'].isin(['A', 'B', 'C', 'T']), 'HomePlanet'] = full.loc[full['Cabin_Deck'].isin(['A', 'B', 'C', 'T']), 'HomePlanet'].fillna('Europa')
full.loc[full['Cabin_Deck'] == 'G', 'HomePlanet'] = full.loc[full['Cabin_Deck'] == 'G', 'HomePlanet'].fillna('Earth')

# 3. Surname origin recovery
surname_map = full.dropna(subset=['Surname', 'HomePlanet']).drop_duplicates('Surname').set_index('Surname')['HomePlanet'].to_dict()
full['HomePlanet'] = full['HomePlanet'].fillna(full['Surname'].map(surname_map)).fillna('Earth')

# 4. Underage constraints (Children under 13 cannot spend; under 18 cannot be VIP)
for amenity in ['RoomService', 'FoodCourt', 'ShoppingMall', 'Spa', 'VRDeck']:
    full.loc[full['CryoSleep'] == True, amenity] = full.loc[full['CryoSleep'] == True, amenity].fillna(0.0)
    full.loc[full['Age'] < 13, amenity] = full.loc[full['Age'] < 13, amenity].fillna(0.0)

full.loc[full['Age'] < 18, 'VIP'] = full.loc[full['Age'] < 18, 'VIP'].fillna(False)
full.loc[full['HomePlanet'] == 'Earth', 'VIP'] = full.loc[full['HomePlanet'] == 'Earth', 'VIP'].fillna(False)
Enter fullscreen mode Exit fullscreen mode

4 · Spatial Coordinates & The Luxury Spend Bifurcation

Once missing values were repaired, Antigravity engineered two feature families that drove the leap to 0.80967 (Top 6%):

1. Spatial Cabin Coordinates (Deck_Side and Cabin_Num)

The anomaly did not strike the ship uniformly. Splitting Cabin into Deck, numeric position Cabin_Num, and Side (Port vs Starboard) unlocked massive regional variance:

  • Deck B Starboard (B_S): 78.4% Transported
  • Deck C Starboard (C_S): 76.4% Transported
  • Deck E Port (E_P): 34.3% Transported
  • Deck T Port (T_P): 25.0% Transported

A passenger's physical position along the ship's longitudinal axis (Cabin_Region = Cabin_Num // 300) proved to be one of the top five most influential features in CatBoost's tree splits.

2. Luxury Service Spend vs. Subsistence Spend

Aggregating total spending showed a huge division:

  • Zero Spenders (ZeroSpend == True): 78.6% Transported
  • Active Spenders (ZeroSpend == False): 29.9% Transported

More critically, spending type mattered intensely:

  • Service / Solitary Luxury (RoomService + Spa + VRDeck): Strong negative correlation ($-0.3561$) with transport.
  • Social / Subsistence (FoodCourt + ShoppingMall): Weakly positive correlation ($+0.0491$).

Passengers spending heavily in isolated spas or VR decks were overwhelmingly spared from the anomaly, while passengers gathered in public food courts shared the fate of the ship's general corridors.

5 · The Diagnostic: The Hazard of Threshold Overfitting (Exp 04 vs Exp 03)

In Exp 04, we expanded from 5 folds to a 10-fold blended ensemble combining CatBoost (70%) and HistGradientBoosting (30%).

On local validation, the raw blend scored 0.8193. Seeking to squeeze out every drop of performance, we swept the decision threshold on Out-Of-Fold probabilities and found that shifting the threshold to 0.480 pushed our local cross-validation score to a peak 0.8205.

Yet when submitted to Kaggle, the public leaderboard score dropped from 0.80967 to 0.80547:

Exp 03 (Thresh 0.500) -> Test Positive Rate: 51.62% -> Public LB: 0.80967 (Rank #98)
Exp 04 (Thresh 0.480) -> Test Positive Rate: 53.21% -> Public LB: 0.80547 (Rank #429)
                                             ^^^^^^
                                             Over-predicting True by +1.6%
Enter fullscreen mode Exit fullscreen mode

The Lesson: Empirical Prior Matching

  • In balanced binary classification ($P(Y=1) \approx 0.5036$), tuning decision thresholds on small validation partitions risks overfitting to local sample variance.
  • The 0.480 cutoff caused the model to over-predict True by 100 passengers (53.21% vs 51.62%).
  • When we reverted to an anchored 0.500 cutoff in Exp 05, test balance was restored (51.04% True), and public accuracy rebounded immediately to 0.80757 (Rank #234).

6 · The Integrity Check: The Clean Machine Learning Benchmark

In our classic Titanic post, we analyzed how family groups spanned both train and test sets, enabling data leakage through ticket-level passenger lookups.

Before celebrating our Rank #98 standing in Spaceship Titanic, we ran a verification check on group contamination:

train['GroupId'] = train['PassengerId'].apply(lambda x: x.split('_')[0])
test['GroupId'] = test['PassengerId'].apply(lambda x: x.split('_')[0])

overlap = set(train['GroupId']).intersection(set(test['GroupId']))
print('Groups spanning both train and test:', len(overlap))
# Output: 0
Enter fullscreen mode Exit fullscreen mode

Kaggle engineered Spaceship Titanic with Group-Stratified partitioning:

  • Zero groups overlap between train and test.
  • Every travel group is either 100% in train or 100% in test.

This proves that our Top 6.14% standing (Rank #98 out of 1,597) was achieved with 100% genuine algorithmic generalization, completely free of historical lookups or companion label leakage.

Key Engineering Takeaways

  1. Deterministic Deduction Beats Hyperparameter Tuning: Recovering 271 HomePlanet entries via surname purity and resolving CryoSleep via expenditure constraints provided larger accuracy gains than days of Bayesian hyperparameter optimization.
  2. Beware the Validation Threshold Trap: Optimizing decision thresholds on out-of-fold predictions can distort test set priors on balanced datasets. In symmetrical problems, anchoring your threshold to the empirical training prior preserves leaderboard stability.
  3. Autonomous Mobile Data Science is Maturing Fast: Running Google Antigravity CLI in an unprivileged PRoot userspace on Android 14 proved capable of managing complex 10-fold cross-validations, tracking experiments, diagnosing threshold drift, and reaching the top tier of competitive data science—entirely from a device that fits in your palm.

All code, feature scripts (exp01 through exp05), and submission files are archived locally in /root/spaceship-titanic/.

Originally published on malcolmlow.com.

Top comments (0)