DEV Community

Cover image for A Unity Build Size Check That Can Fail Your CI
HanoStudio
HanoStudio

Posted on

A Unity Build Size Check That Can Fail Your CI

#unity3d #gamedev #csharp #devops

Build Analyzer series, article 4.

In article 1, I measured which changes moved a Unity Android APK.

In article 2, I separated APK bytes, BuildReport.summary.totalSize, and packed assets.

In article 3, I opened the APK itself to find the code-size growth that PackedAssets could not explain.

This time I wanted the analyzer to stop being only a report.

After three articles, the shape of the problem changed a little.

It was useful to know why a build became larger after the fact. But in a real team, the more useful question is earlier:

Should this build be allowed to merge?

So I added a small CI gate.

It does not try to be clever. It compares the current build summary against a versioned JSON baseline and returns a non-zero exit code when a threshold is crossed.

That is the whole feature.

The input I already had

The previous reporter already writes a summary like this after each build:

artifact_bytes=17928357
artifact_mib=17.0978
build_report_total_bytes=150514033
build_report_total_mib=143.5414
packed_asset_bytes=5873204
packed_asset_mib=5.6011
Enter fullscreen mode Exit fullscreen mode

For CI, I only used the byte values:

  • artifact_bytes
  • build_report_total_bytes
  • packed_asset_bytes

The important part is that I did not compare just one number.

The APK might be stable while packed assets move. Packed assets might be stable while native code grows. The previous posts were basically me learning that lesson the slow way.

The baseline is just JSON

I wanted the baseline to be boring and reviewable in a pull request.

{
  "schemaVersion": 1,
  "name": "android-dotween-runtime",
  "metrics": [
    {
      "name": "artifact_bytes",
      "baselineBytes": 17928357,
      "warnAfterBytes": 102400,
      "failAfterBytes": 262144,
      "warnAfterPercent": 0.5,
      "failAfterPercent": 1.0
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The real baseline in the repo also includes build_report_total_bytes and packed_asset_bytes.

I like this shape because the rule is visible:

  • warn if the APK grows more than 100 KiB or 0.5%
  • fail if it grows more than 256 KiB or 1.0%
  • keep the baseline in source control
  • update it intentionally when the growth is accepted

It is not trying to tell the whole story. It is only deciding whether the story needs attention.

The CI command

The gate runs as a Unity -executeMethod:

/Applications/Unity/Hub/Editor/6000.3.7f1/Unity.app/Contents/MacOS/Unity \
  -batchmode \
  -quit \
  -projectPath "$(pwd)" \
  -executeMethod HanoStudio.BuildAnalyzer.Editor.BuildSizeGateCli.Run \
  -buildAnalyzerBaseline "$(pwd)/BuildSizeBaselines/android-dotween-runtime.json" \
  -buildAnalyzerSummary "$(pwd)/Results/raw/step-07-dotween-runtime-summary.txt" \
  -buildAnalyzerReport "$(pwd)/Builds/Reports/build-size-gate-report.txt" \
  -logFile "$(pwd)/Builds/build-size-gate.log"
Enter fullscreen mode Exit fullscreen mode

In a real project, the summary path would point at the report generated by the current CI build.

For this reproducible repo, I used the measured step 7 summary so the command can be run without rebuilding all APKs.

What the pass report looks like

Against the committed baseline, the current step 7 summary passes:

[Build Analyzer Gate] PASS
Baseline: android-dotween-runtime
Fail on warning: False

metric,status,baseline,current,delta,delta_percent,reason
artifact_bytes,Pass,17928357,17928357,0,0.0000,"within thresholds"
build_report_total_bytes,Pass,150514033,150514033,0,0.0000,"within thresholds"
packed_asset_bytes,Pass,5873204,5873204,0,0.0000,"within thresholds"
Enter fullscreen mode Exit fullscreen mode

That output is intentionally plain. CI logs are not the place for a dashboard.

When the artifact crosses the failure threshold, I want the output to be just as plain:

Build Analyzer CI failure output

The cases I tested

I added a self-test that checks four cases:

Case Current value Result
pass baseline + 20,000 bytes exit 0
warn baseline + 80,000 bytes exit 0 by default
fail baseline + 130,000 bytes exit 1
missing baseline no JSON file exit 2

Warnings can be turned into failures with:

-buildAnalyzerFailOnWarning
Enter fullscreen mode Exit fullscreen mode

I also added:

-buildAnalyzerAllowMissingBaseline
Enter fullscreen mode Exit fullscreen mode

That is useful while introducing the check to an existing project, but I would not leave it on forever. A size gate without a committed baseline is mostly theater.

What changed in my thinking

Before this, I treated the analyzer as an explanation tool.

After adding the gate, I think the product shape is clearer:

  1. collect the numbers
  2. compare them to the last accepted build
  3. fail or warn in CI
  4. then explain which layer moved

The explanation still matters. But the first useful product moment is probably not a chart.

It is the CI job saying:

This build grew by more than the team agreed to allow.

That is a small sentence, but it changes the tool from something you open after a problem to something that catches the problem while it is still cheap.

Source

Top comments (0)