An Offline Game Settings Optimizer Built for a Friend | TabPFN

Devansh Dubey — Sun, 04 Oct 2026 09:38:11 +0000

What I Built

I built FrameForge, an offline, AI-powered PC game settings optimizer designed for a friend who struggles with finding the perfect balance between frame rates and visual quality on their gaming rig.

PC gaming often involves tweaking dozens of graphics settings, which can be overwhelming and frustrating. FrameForge solves this by capturing real-time frame telemetry and automatically determining the optimal settings for smooth, stutter-free gameplay, completely offline.

Demo

You can check out the project landing page and download it here: [https://devanshindepth.github.io/frame-forge/]

Code

devanshindepth / frame-forge

Adaptive Game Graphics Optimizer

FrameForge — Adaptive Game Graphics Optimizer

Stop guessing graphics settings. Measure them.
FrameForge is a free Windows desktop application that measures your real frame rate while you play, tests graphics configurations automatically, and finds the fastest settings that preserve your chosen visual quality. Runs 100% offline on your own PC.

🚀 Downloads (v1.0.0)

Package	Format	Architecture	Download Link
Windows Setup (Recommended)	`.exe` (NSIS)	x86_64	FrameForge_1.0.0_x64-setup.exe
Windows Package	`.msi` (WiX)	x86_64	FrameForge_1.0.0_x64_en-US.msi

Integrity Verification (SHA-256)

# FrameForge_1.0.0_x64-setup.exe
71eeb4839a038c444b22ffcf06aa1d669c150226c5bee78479818a4cf01cfb18

# FrameForge_1.0.0_x64_en-US.msi
152695991ae4c052314779aff6bdd5625e5030416a0beee620acb3b34070930a

To verify on Windows PowerShell:

Get-FileHash -Algorithm SHA256 "path\to\FrameForge_1.0.0_x64-setup.exe"

✨ Features

Real Frame-Rate Measurement: Measures average FPS and 1% lows using Intel PresentMon via Windows Event Tracing (ETW) — never injects into game memory.
Safety First: Your game's configuration file is backed up before any test begins, and restored automatically if stopped, interrupted, or on application restart.
Applied Verification: Confirms that the game actually…

View on GitHub

How I Built It

FrameForge is built as a native Windows desktop application using Tauri (Rust core) with a vanilla HTML/JS/CSS frontend for a lightweight, snappy experience.

Its architecture consists of three components: a frontend for user interaction, a Rust-based core for game process monitoring, telemetry management, and safe configuration modification, and a Python AI sidecar packaged with PyInstaller. The Rust backend communicates with the AI engine through a bidirectional, line-delimited JSON IPC channel over stdin/stdout pipes. To ensure offline operation, the Python engine uses bundled TabPFN v2 model checkpoints, disables telemetry, and blocks outbound network connections, eliminating the need for cloud APIs or external inference services.

At the heart of FrameForge is TabPFN v2, an open-weight foundation model for tabular data, used as a surrogate model within an in-context Bayesian optimization pipeline. FrameForge captures real-time performance telemetry using Intel PresentMon (ETW) and converts graphics configurations into structured features, combining normalized settings with performance-cost priors and perceptual quality scores.

Below is the complete breakdown of results:

T1: Sample Efficiency (MAPE % vs. Sample Size n)

Lower MAPE is better.

Model	n=8	n=16	n=32	n=64	n=128	n=256
TabPFN	31.76%	20.15%	10.33%	7.63%	6.01%	4.18%
GP (Matérn-5/2)	33.87%	28.60%	13.77%	8.30%	6.90%	5.00%
CatBoost	30.88%	22.02%	14.18%	8.96%	7.71%	6.71%
XGBoost	34.67%	23.21%	15.86%	11.34%	8.14%	7.03%
Random Forest	30.35%	26.19%	20.73%	12.64%	9.56%	7.64%
MLP	32.01%	26.75%	18.62%	15.59%	11.40%	9.55%

T2: Constrained Bayesian Optimization Regret (% of Oracle Best)

Higher is better; tracks the percentage of optimal feasible objective reached.

Optimiser / Surrogate	5 evaluations	10 evaluations	15 evaluations	20 evaluations
SMAC (Random Forest)	20.0%	20.0%	56.1%	60.5%
TabPFN-BO	20.0%	20.0%	52.8%	52.8%
XGBoost + Bootstrap EI	20.0%	46.0%	47.1%	51.1%
Random Search (Baseline)	9.0%	17.1%	23.4%	43.9%
GP-BO (BoTorch qEI)	20.0%	20.0%	20.2%	24.5%

T3: Uncertainty Calibration (Empirical Coverage % vs Nominal)

Closer to Ideal coverage is better.

Model	50% Nominal	80% Nominal	90% Nominal	95% Nominal
Ideal	50.0%	80.0%	90.0%	95.0%
Random Forest	51.0%	80.2%	88.4%	94.2%
TabPFN	61.6%	87.8%	93.8%	96.2%
GP (Matérn-5/2)	38.5%	60.1%	68.3%	73.8%
XGBoost	31.0%	54.2%	63.8%	70.2%
MLP	26.1%	48.0%	58.6%	66.5%
CatBoost	7.2%	13.8%	18.2%	21.6%

T4: Measurement Noise Robustness (MAPE % vs Noise Level CV)

Lower MAPE is better; evaluated at n=32.

Model	Noise CV=0.00	CV=0.02	CV=0.05	CV=0.10	CV=0.15	CV=0.20
TabPFN	13.34%	13.29%	13.78%	15.25%	18.13%	20.30%
XGBoost	23.59%	24.10%	22.94%	25.58%	27.68%	28.67%
CatBoost	22.44%	27.70%	26.85%	25.12%	27.35%	30.75%
MLP	25.42%	25.63%	25.99%	26.74%	27.80%	28.97%
Random Forest	30.93%	30.84%	30.81%	31.63%	32.65%	32.83%
GP (Matérn-5/2)	19.27%	17.50%	21.47%	26.22%	31.21%	35.58%

T5: Cold Start / Transfer across GPUs (MAPE %)

Leave-one-GPU-out transfer with hardware features; lower MAPE is better.

Model	Average FPS MAPE	1% Low FPS MAPE
TabPFN	14.14%	23.69%
XGBoost	18.50%	24.46%
MLP	18.81%	24.09%
GP (Matérn-5/2)	19.09%	22.21%
CatBoost	19.44%	24.48%
Random Forest	25.61%	29.36%

T6: 1% Low Ranking Ability (Spearman Rank Correlation)

Higher correlation is better; measures how accurately the model orders configurations.

Model	Spearman
TabPFN	0.870
XGBoost	0.799
Random Forest	0.783
CatBoost	0.736
MLP	0.725
GP (Matérn-5/2)	0.358

T7: Robustness to Missing Telemetry (MAPE % vs Missing Fraction)

Lower MAPE is better; hardware telemetry features randomly knocked out.

Model	0% Missing	10% Missing	20% Missing	30% Missing	40% Missing
TabPFN	7.98%	14.47%	22.81%	25.85%	28.83%
XGBoost	14.36%	22.46%	28.19%	33.32%	36.41%
Random Forest	17.65%	23.08%	29.27%	33.25%	36.24%
CatBoost	14.60%	24.47%	30.36%	36.38%	37.82%
GP (Matérn-5/2)	11.99%	20.40%	28.42%	32.47%	39.00%

T9: Compute Cost (Wall-Clock Seconds for n=64, 5,000 Candidates)

Lower wall-clock latency is faster.

Model	Wall-Clock Latency
MLP	0.27 s
GP (Matérn-5/2)	0.39 s
Random Forest	0.57 s
CatBoost	6.71 s
XGBoost	9.95 s
TabPFN	30.35 s (CPU without CUDA)

T10: Large-Data Scaling (MAPE %)

Scaling test from N=256 to N=1,000.

Model	N=256	N=1,000
XGBoost	14.23%	7.95%
TabPFN	10.75%	8.00%
CatBoost	11.75%	8.20%

The optimizer generates up to 4,000 candidate configurations per iteration, filters out those that violate the user's quality requirements, and uses TabPFN's quantile predictions to estimate performance and uncertainty. An Expected Improvement acquisition function guides the search, balancing average FPS, 1% lows, and visual quality against the target refresh rate. Optimization progresses from baseline measurements and exploratory sampling to active AI-guided search, with early stopping when further improvements become unlikely.

Why Does Open Innovation Matter?

Open innovation matters because it ensures privacy, accessibility, and user control. For FrameForge, it was critical that the optimization process ran entirely offline on the local machine without sending hardware telemetry or gaming habits to the cloud.

Using an open-weight model like TabPFN made this local execution possible. A closed API would have introduced latency, required a constant internet connection, and compromised user privacy. Furthermore, open-source tools like Tauri and PresentMon provided the reliable, high-performance foundation needed to build this seamlessly.

Prize Categories

TabPFN Challenge - The benchmark results strongly validate that choosing TabPFN was the right decision for this project.

DEV Community: Devansh Dubey