DEV Community: Uray Febri

Debugging R8 Release-Only Crashes Without Disabling Optimization

Uray Febri — Sat, 19 Sep 2026 03:07:27 +0000

A debug build works. The release build installs. Then one screen crashes only after minification is enabled.

The tempting fix is immediate:

-keep class com.example.** { *; }

Or worse, turn R8 off and ship.

Both can make the symptom disappear. Neither tells you what broke.

A release-only R8 failure is usually easier to solve when you treat it as an indirect-reference debugging problem, not a ProGuard guessing contest. The goal is to identify the runtime contract that static analysis cannot see, preserve exactly that contract, and keep the rest of the optimizer working.

First Prove That Optimization Is the Boundary

Do not begin by editing keep rules.

Reproduce the failure with the same release variant that users receive. That matters because build type, flavor, dependency graph, resource shrinking, signing configuration, and generated code can all differ from debug.

A useful first comparison is:

Debug build        → works
Release, R8 on     → fails
Release, R8 off    → works

That strongly points toward optimization, shrinking, obfuscation, or a rule consumed from a dependency. It still does not prove which class needs to be kept.

If your release pipeline is already automated, this is also why Android CI should build the real release variants, not only compile debug APKs.

Retrace Before You Read the Crash

An obfuscated stack trace is evidence with its labels removed.

R8 produces mapping.txt for the optimized variant. Preserve the mapping file for every published build, because a later build can overwrite the local copy. Android's documentation recommends using retrace when the original stack trace is not automatically deobfuscated.

A manual workflow looks like this:

$ANDROID_HOME/cmdline-tools/latest/bin/retrace \
  app/build/outputs/mapping/release/mapping.txt \
  trace.txt

The important question after retracing is not simply, "Which class crashed?"

Ask:

What runtime mechanism expected this class, method, field, constructor, annotation, or name to remain discoverable?

That question usually gets you much closer to the missing rule.

Look for Indirect References

R8 is very good at following static references. Problems appear when the program reaches code in ways that static analysis cannot fully infer.

Common boundaries include reflection, classes loaded from string names, JNI, serialization frameworks that inspect members, and libraries that discover implementations dynamically.

Android's R8 guidance specifically calls out reflection-related failures such as ClassNotFoundException, NoSuchMethodException, NoSuchFieldException, NoClassDefFoundError, NoSuchMethodError, and NoSuchFieldError as useful signals.

Suppose an integration does this:

val clazz = Class.forName(className)
val instance = clazz.getDeclaredConstructor().newInstance()

To the runtime, that class is required.

To a static optimizer, a string containing a class name is not necessarily a normal code reference. If nothing else reaches the class, removing or renaming it may be perfectly logical from R8's perspective.

The bug is therefore not "R8 randomly deleted my code." The real issue is that a runtime dependency was invisible to static analysis.

Inspect the Rules R8 Actually Received

Your proguard-rules.pro is not the whole configuration.

Rules can come from the application, Android tooling, and consumer rules packaged by dependencies. Android's troubleshooting documentation points to the merged R8 configuration under the build outputs, and the Android Developers Blog also recommends printing the final configuration when you need to understand where a rule came from.

That changes the investigation from:

I think this is my R8 configuration.

into:

This is the configuration R8 actually evaluated.

This is especially useful when optimization is weaker than expected. A dependency can bring broad rules or global options that keep far more code than you intended.

Use `-whyareyoukeeping` in the Opposite Case

Not every R8 problem is missing code.

Sometimes the APK remains unexpectedly large because a class you expected R8 to remove is still present. The -whyareyoukeeping diagnostic rule asks R8 to show the reference chain responsible for retaining that code.

-whyareyoukeeping class com.example.feature.LegacyEntryPoint

Use it as a debugging instrument, not permanent configuration. Android explicitly cautions against checking it into the codebase because it can slow builds.

This gives you two complementary investigations:

Runtime crash
    ↓
What required code did R8 fail to see?

Unexpected retained code
    ↓
Why does R8 believe this code is reachable?

Both are better than adding wildcards until the build behaves.

Write the Narrowest Rule That Expresses the Runtime Contract

A broad rule is useful as a temporary experiment.

If keeping an entire package makes the crash disappear, you have learned that the missing contract probably lives inside that package. That is diagnostic progress, but it should not automatically become the final fix.

Android recommends keep rules that are as specific as possible and warns against long-lived package-wide rules such as:

-keep class com.example.feature.** { *; }

Instead, describe the actual boundary.

If only implementations of an interface are dynamically loaded, preserve those implementations and only the constructor the loader needs:

-keep class * implements com.example.runtime.StartupTask {
    <init>();
}

If reflection requires a particular member, target the member rather than freezing the entire class.

The best keep rule is not the shortest rule. It is the rule whose scope matches the runtime behavior you can explain.

Prefer Annotations When the Contract Belongs to Your Code

When your own architecture intentionally exposes code to reflection or discovery, annotations can make the boundary explicit.

Instead of maintaining a fragile list of class names, mark the code that participates in the runtime contract and write a rule against that annotation. Android's keep-rule guidance recommends this pattern because the relationship between source code and preservation becomes visible.

That is a maintainability improvement as much as an optimization improvement.

A future developer can answer "Why is this kept?" from the source instead of archaeology in a large rules file.

Do Not Test the Fix Only by Launching the App

A release build that reaches the home screen proves very little.

The verification should exercise the behavior that crossed the invisible boundary:

deserialize the affected model;
open the screen that instantiates the class dynamically;
run the JNI call;
execute the deep link or navigation route;
test the flavor where the dependency is actually included.

Then inspect the optimized artifact again.

If you fixed a crash with a narrower rule, verify both sides of the trade-off: the feature still works and R8 is still allowed to optimize unrelated code.

For multi-flavor apps, this becomes even more important because a rule that appears correct in one variant can hide a dependency-specific problem in another. The same principle applies when reviewing technical feedback across Android flavors: verify the claim against the exact variant and dependency boundary involved.

A Repeatable R8 Debugging Loop

The process can be reduced to a small loop:

Reproduce the release-only failure
              ↓
Retrace with the matching mapping file
              ↓
Identify the runtime boundary
              ↓
Inspect merged R8 configuration
              ↓
Test a temporary isolation rule
              ↓
Replace it with the narrowest valid rule
              ↓
Exercise the affected release behavior
              ↓
Verify optimization still works elsewhere

The key step is the one developers often skip: replace the temporary broad rule.

A wildcard can be a good diagnostic switch. It is rarely a good explanation.

Treat Keep Rules as Architecture Documentation

R8 failures feel mysterious when the rules file becomes a collection of copied incantations.

They become much more predictable when every rule answers three questions:

What code is reached indirectly?
Which runtime mechanism reaches it?
What is the smallest surface that mechanism requires?

Once you can answer those, the rule stops being a workaround. It becomes documentation for a boundary in your architecture.

That is the standard I would use before shipping an R8 fix: not merely "the release build no longer crashes," but "we know why the optimizer could not see this dependency, and the rule preserves exactly what runtime needs."

Originally published on RayLabs.

Native Android OTA Updates: What You Can Patch Without Shipping a New APK

Uray Febri — Sat, 19 Sep 2026 03:05:53 +0000

Over-the-air updates sound like a simple product request: fix production without waiting for users to install another app version. On native Android, however, the phrase OTA update can describe several very different mechanisms. Some are standard platform features. Some update configuration or content rather than executable code. Others cross directly into security and store-policy boundaries.

That distinction matters because a native Kotlin application does not have a direct equivalent to every code-push system available in other mobile stacks. A production Android app can change a surprising amount of behavior without a new APK, but the safe architecture is usually not to download replacement Kotlin or DEX code from your own server.

The useful question is therefore not, “How do I add OTA to Android?” It is:

What exactly needs to change after release, and which layer should own that change?

Once the requirement is classified correctly, the implementation options become much clearer.

OTA Is an Outcome, Not One Technology

Teams often group four different update problems under the same label:

Requirement	Better mechanism
Deliver a new signed Android binary	Google Play release + In-App Updates
Change feature availability or thresholds	Remote configuration / feature flags
Change copy, catalog data, forms, or server-owned content	API-driven content
Change layout within predefined components	Server-driven UI with a constrained schema
Replace arbitrary Kotlin/DEX/native code	Usually requires a new app release

This classification is more useful than starting with a code-push SDK. It keeps the update mechanism proportional to the thing that actually changes.

If a production issue can be fixed by changing a timeout, disabling a feature, switching an endpoint, or hiding a problematic flow, shipping a complete binary is unnecessarily heavy. But if the fix changes an Activity, a native SDK, a Room migration, a permission declaration, or compiled Kotlin behavior, a normal application release is usually the correct boundary.

That same risk-first thinking also applies to broader deployment work. A reliable mobile release is less about one clever update mechanism and more about executing a smooth technical rollout with explicit ownership, verification, and rollback paths.

Option 1: Use Google Play In-App Updates for New Binaries

For applications distributed through Google Play, the most straightforward native update path is the Play In-App Updates API. Google provides the API for Kotlin and Java applications through the Play Core app-update library.

The important detail is that In-App Updates does not bypass the store release process. Your new version still goes through Google Play. The API improves the user experience of discovering and installing an available version from inside the application.

Android supports two main user experiences:

Flexible updates download while the user continues using the app and can be completed later.
Immediate updates present a blocking update flow when continuing on the old version is not acceptable.

That makes In-App Updates useful for compatibility breaks, security-sensitive releases, mandatory backend migrations, or important fixes where adoption speed matters. It is not a mechanism for changing compiled code independently of the store.

A simple architecture might look like this:

App starts
   ↓
Check Play update availability
   ↓
Classify update policy
   ├── optional → flexible flow
   └── required → immediate flow
   ↓
Google Play delivers signed update

The policy decision should live outside the UI where possible. A ViewModel or use case can decide whether the current version is acceptable, while the Activity owns the Play update UI contract.

Option 2: Move Emergency Knobs Into Remote Configuration

Many requests for “native OTA” are actually requests for operational control.

Imagine a release where a new checkout path starts failing for a subset of users. If the app already has a remotely controlled feature flag, the team may be able to disable that path immediately while preparing a proper binary fix.

Useful remotely controlled values include:

feature enablement,
rollout percentages,
API timeout thresholds,
minimum supported version,
maintenance banners,
endpoint selection from a pre-approved set,
experiment variants,
kill switches for risky functionality.

The critical design rule is that remote configuration should select among capabilities already shipped in the binary. It should not become an improvised programming language.

For example:

data class RuntimePolicy(
    val newCheckoutEnabled: Boolean,
    val minimumSupportedVersion: Int,
    val requestTimeoutSeconds: Int,
)

The server can change the values, but the application still defines what those values are allowed to do.

This produces a powerful production property: some incidents can be mitigated in minutes without turning the app into a remote-code execution platform.

Option 3: Make Content Server-Driven

Text, product catalogs, FAQs, onboarding messages, campaign cards, and many business rules are data. They do not need to be hard-coded into the APK.

If the product changes these frequently, model them as server-owned content from the beginning.

The boundary can be as simple as an API response:

{
  "title": "Scheduled maintenance",
  "message": "Some transfers may be delayed.",
  "severity": "warning"
}

The native application owns rendering and allowed behaviors. The backend owns the current content.

This is technically an OTA change from the user's perspective, but no executable code has changed. It is also easier to test because the client can validate the response against a stable contract.

For resilient applications, cache the last valid payload, reject unsupported schema versions, define safe defaults, and make malformed remote content fail closed rather than crashing the screen.

Option 4: Use Constrained Server-Driven UI When Layout Must Change

Server-driven UI extends the same idea from content to composition.

Instead of the server sending arbitrary executable logic, it sends a declarative schema using components that already exist in the application:

{
  "screen": "promotion",
  "components": [
    { "type": "heading", "text": "Weekend offer" },
    { "type": "product_grid", "source": "featured" },
    { "type": "button", "action": "open_checkout" }
  ]
}

The Android client decides which component types and actions are valid. Unknown components can be ignored or replaced with a safe fallback.

This architecture can reduce release pressure for highly dynamic surfaces, but it comes with real costs:

schema versioning,
accessibility guarantees,
analytics consistency,
preview tooling,
caching and offline behavior,
backward compatibility,
more complex testing.

Server-driven UI is therefore not “free OTA.” It exchanges binary-release frequency for platform complexity.

Why Downloading New Kotlin or DEX Code Is a Different Category

Native Android code eventually becomes executable artifacts such as DEX and native libraries. Loading replacement executable code from a remote source changes the security model significantly.

Android's security guidance strongly discourages dynamically loading code from outside the application APK because it increases exposure to code injection and tampering and complicates verification and version management.

Google Play's Device and Network Abuse policy is even more important for Play-distributed applications. It states that an app distributed through Google Play may not update itself outside Google Play's update mechanism and may not download executable code such as DEX, JAR, or .so files from another source. The policy documents an exception for code running in a virtual machine or interpreter under the described conditions, but that is not a blanket permission to build an unrestricted native code-push system.

So a custom design like this deserves immediate scrutiny:

Native Android app
      ↓
download patch.dex from private server
      ↓
load classes dynamically
      ↓
replace production behavior

Even if it can be made to work technically, technical possibility is not the same as a sound distribution architecture.

There are also platform-level hardening trends to consider. Android's security guidance continues to push developers away from writable dynamically loaded code, reinforcing the principle that executable updates should remain tightly controlled.

Why Shorebird Feels Different

Flutter developers have a more visible example of production code push in Shorebird. Shorebird uses a modified Flutter engine that can download and apply patches to Dart code. Its documentation describes Android support and explains that patches can modify Dart application code while native Java/Kotlin changes, native dependencies, assets, and Flutter engine changes remain outside the patch boundary.

That architecture is possible because Flutter already introduces a runtime and engine boundary between much of the application's Dart code and the native platform.

A pure native Kotlin application does not have that same boundary by default.

This is the key comparison:

Capability	Native Android	Flutter + Shorebird
Update full application binary	Play release	Store release
Prompt/install store update in app	Play In-App Updates	Store-specific flow
Change remote flags/content	Yes	Yes
Patch Dart application code	Not applicable	Supported by Shorebird
Patch Java/Kotlin native code	New binary expected	Not supported by Shorebird patching
Patch native SDK or manifest	New binary expected	New binary expected

Shorebird's Code Push FAQ also documents the practical patch boundary and store-policy considerations that still apply to code-push workflows.

So the useful lesson from Shorebird is not “native Android needs a Shorebird clone.” The lesson is to identify a stable runtime boundary and be explicit about what can safely change inside it.

A Better Native Android Update Architecture

For most production Kotlin applications, a layered approach is more robust than arbitrary code push:

Layer 1: Store binary
Kotlin, Compose/XML, native SDKs, manifest, database migrations
                ↓
Layer 2: Runtime policy
Feature flags, kill switches, minimum version, rollout controls
                ↓
Layer 3: Remote content
Copy, catalogs, configuration, campaigns
                ↓
Layer 4: Constrained server-driven UI
Only when the product genuinely needs dynamic composition

Each layer has a different release velocity and a different risk profile.

This also makes CI/CD easier to reason about. Binary changes go through the strongest build and test gates. Configuration changes can have schema validation and staged rollout. Content changes can have editorial validation. Server-driven UI payloads can have contract tests and preview environments.

The goal is not to make every layer deploy at the same speed. The goal is to make the safest layer capable of solving the problem deploy quickly enough.

Design for Rollback Before You Design for OTA

Fast delivery without fast recovery is not a mature update system.

Before adding any remote-control mechanism, define its rollback behavior:

What happens if the configuration service is unavailable?
Is the last known-good value cached?
Can a feature be disabled independently?
Can a malformed payload crash application startup?
Does the server know which client schema versions it is targeting?
Can an emergency binary release override remote state?

For configuration, prefer typed defaults and bounded values. For server-driven UI, version schemas and reject unsupported actions. For Play releases, use staged rollouts where appropriate and monitor the version-specific failure rate.

The release mechanics should also be connected to your broader CI/CD feedback loop so a fast mitigation does not become an excuse to bypass tests for the permanent fix.

Choose the Smallest Update Surface That Solves the Problem

The strongest native Android OTA strategy is usually a collection of deliberately limited mechanisms rather than one universal patcher.

Use Play In-App Updates when users need a new signed application version. Use feature flags and remote configuration for operational switches. Move genuinely dynamic content behind APIs. Consider constrained server-driven UI only when the product benefits enough to justify the schema and testing cost. Treat arbitrary executable-code downloading as a security and policy boundary, not as a shortcut around release engineering.

That approach may sound less magical than “patch anything instantly,” but it produces a system that is easier to secure, test, roll back, and explain.

For native Android, that is often the more useful definition of OTA: not eliminating releases, but reducing how often a production problem actually requires one.