I Reverse-Engineered an APK Protected by XopProtector — and It Changed How I Look at Android
Recently, I've been spending quite a bit of time studying Android application protection.
I happened to have an APK that I had protected with XopProtector, so instead of continuing to look at it only from the perspective of the developer, I decided to switch sides for a while:
If I were an attacker or reverse engineer who had just received this APK, what could I actually see?
So I went through the whole process myself.
This isn't meant to be a "top 10 Android reverse-engineering tools" tutorial. I mainly want to record what I encountered during the process and how my understanding of Android protection changed as I dug deeper.
Project:
This article is based on security research using an APK that I own or am authorized to analyze.
The First Thing I Did: Open the APK
When I first got the APK, my reaction was pretty straightforward.
Unzip it.
An Android APK is basically a ZIP container, so the first thing I looked at was the file structure:
AndroidManifest.xml
classes.dex
classes2.dex
resources.arsc
assets/
lib/
res/
META-INF/
Then I opened it in JADX.
I've done this many times before, so initially nothing seemed particularly interesting.
With a normal Android application, I would usually see something like:
MainActivity
LoginActivity
UserManager
NetworkManager
...
The business logic is right there, and you can start following the call chain.
But with the protected APK, things looked different.
I could see the Application and some startup-related code, along with some shell and Native-related components, but the actual business classes weren't exposed in the same way as they are in a normal APK.
At that point, I started thinking:
Maybe looking at the Java layer isn't the right place to start.
Going Back to the APK Structure
So I went back to the APK itself.
I generally prefer looking at the file structure before immediately diving into the code.
For a protected APK, I pay particular attention to:
classes*.dex
assets/
lib/*/
These three areas can reveal quite a lot.
This was also where I started to think about an important question:
Where did the original DEX actually go?
After looking at how XopProtector works, I realized that the problem isn't simply "the DEX is gone."
The protection process involves DEX protection, a Native Shell, runtime recovery, and virtualization-related mechanisms.
So the more interesting question became:
The code hasn't disappeared. I just haven't figured out when and in what form it comes back.
That distinction turned out to be pretty important.
Then I Started Looking at Native
This was probably the biggest change in my thinking.
When I was mainly doing Android application development, seeing a .so file usually meant one thing to me:
JNI library.
Load it with System.loadLibrary(), call some native methods, and move on.
This time was different.
I started looking carefully at the libraries under:
lib/arm64-v8a/
and then followed the JNI calls into Native.
That's when I started to realize that if a protection system moves a significant amount of critical logic into Native, simply analyzing Java or Smali becomes much less useful.
My analysis path gradually changed from:
APK
↓
JADX
↓
Java
to:
Java
↓
JNI
↓
Native
↓
DEX
↓
ART
That change alone was one of the biggest things I learned from this exercise.
The Question That Really Got Me Interested: When Does the DEX Enter ART?
At this point, I got stuck on a question.
If the actual business DEX isn't stored in plain form inside the APK, the application still has to execute it eventually.
Android can't execute code that doesn't exist.
So there must be some process like:
Protected data
↓
Recovery / decryption
↓
DEX
↓
ClassLoader / ART
↓
Class
↓
Method
↓
Execution
That changed the question I was asking.
Instead of:
"Is the DEX encrypted?"
I started asking:
When is the DEX recovered? Where does the recovered data go? And how does it eventually reach ART?
Once I started looking into parts of the ART source code, things became much more interesting.
As an Android developer, ClassLoader had always been more of a concept to me.
Now I was paying attention to things like:
DexFile
ClassLoader
DefineClass
ClassLinker
ART
because these components ultimately determine how classes enter the runtime.
Then I Started Understanding What VMP Actually Means
XopProtector also has mechanisms such as PVM1 and PVM2.
When I first saw "VMP", my understanding was pretty simple:
Isn't this just code virtualization?
After looking deeper, I realized there was more to it.
PVM2 in particular is closer to a Native Interpreter approach.
Conceptually, instead of executing something directly through the traditional path:
Java Method
↓
ART
you can have something more like:
Method
↓
Virtual Instructions
↓
Native Interpreter
↓
Execution
Once you look at it from a reverse-engineering perspective, the problem changes completely.
You're no longer simply asking:
"How do I decompile this Java method?"
You start asking:
"What does this method actually correspond to now?"
If execution has been moved into a custom virtual instruction set, traditional Java-level analysis naturally becomes much harder.
Then Frida Came Into the Picture
When static analysis wasn't enough, the next step was to run the application and observe what was actually happening.
That's where Frida became useful.
Dynamic analysis gave me a completely different way of looking at the protection system.
Instead of guessing, I could ask concrete questions:
When is the SO loaded?
When is a file accessed?
When does decryption happen?
When is the DEX recovered?
When are classes created?
Then I could observe what actually happened at runtime.
For example, if I saw a Native function being called, I would keep following it:
Who called it?
Why was it called at this point?
What are the arguments?
What did it return?
Where does the return value go?
At this point, reverse engineering started to feel less like "finding the code" and more like debugging someone else's program.
The SO Layer Can't Be Ignored Either
As I continued, I started paying more attention to the Native libraries themselves.
This is easy to overlook.
If you only protect the DEX while leaving critical logic exposed in Native libraries, the SO files themselves can become an attack surface.
So I started looking at ELF structures:
.text
.rodata
.dynamic
symbol
relocation
JNI
I also started to understand why some protection systems need to protect the SO itself.
XopProtector isn't focused only on DEX protection. Its public project also includes Native/SO protection mechanisms.
At this point, my understanding of "Android application protection" had changed quite a bit.
It isn't simply:
Encrypt
classes.dex.
You have to think about:
DEX
Native
SO
Runtime
Integrity
Debugging environment
Hooking environment
and, more importantly, how all of these components interact.
Then I Ran Into RASP
Eventually I started looking at RASP as well.
I'd seen the concept before, but looking at it from a reverse-engineering perspective felt completely different.
From the application's point of view, things such as:
Frida
Debugger
Hooking
Modified SO
Abnormal runtime environment
can themselves be considered part of an attack.
So the application can actively inspect its environment while it is running.
The relationship becomes pretty interesting:
Reverse Engineer
↓
APK
↓
Protected Runtime
↓
RASP
↓
Continue Analysis
While you're analyzing the application, the application is also analyzing its environment.
That's one reason a protected APK can look relatively simple during static analysis but behave very differently once you actually run it.
What I Learned From the Whole Process
This time, I wasn't particularly focused on completely unpacking or restoring the APK.
By that point, the result itself wasn't the most interesting part anymore.
Previously, I would have described Android reverse engineering as:
Decompile an APK and recover the source code.
Now I see it differently.
An Android application goes through a much longer chain from a file on disk to actual CPU execution:
APK
↓
DEX
↓
ClassLoader
↓
ART
↓
JNI
↓
ELF
↓
Native
↓
CPU
Android protection works by introducing protection mechanisms at different points along this chain.
So knowing JADX and Apktool is enough to get started, but you'll eventually hit a wall.
To go further, you need to understand things like:
- DEX
- ART
- ClassLoader
- JNI
- ELF
- Linux
- ARM64
- Native debugging
- Dynamic analysis
The tools are only part of the story.
The tools answer:
"How do I look at it?"
The underlying knowledge answers:
"Why does it work this way?"
One More Thing I Got From This
There was another benefit to using XopProtector as the subject of this experiment.
Because I can study the project itself, I was able to look at the same problem from two completely different perspectives.
From the protection developer's perspective:
Why was this designed this way?
And from the reverse engineer's perspective:
If I wanted to analyze this, where would I look next?
Putting these two perspectives together made a lot of things that previously felt abstract much easier to understand.
For example, when looking at DEX protection, I no longer stop at:
"What encryption algorithm is being used?"
I immediately start asking:
Where is the key?
When does decryption happen?
How long does the plaintext exist?
Who gets access to it?
How does it eventually enter ART?
The same applies to SO protection.
I don't just ask:
"Is the SO encrypted?"
I also want to know:
When is it restored?
Who loads it?
What does the memory look like after loading?
Where does integrity verification happen?
That's what makes Android protection interesting to me.
What's Next?
This experiment gave me a good overview of the whole APK, but there are still plenty of areas I haven't explored deeply enough.
The next thing I want to focus on is the ART side of Android 15/16.
In particular:
DefineClass
DexFile
ClassLinker
V35 / V36
Native Library
Android keeps changing, and ART internals change with it.
A protection mechanism that works one way on Android 10 doesn't necessarily behave exactly the same way on Android 16.
That's also one of the areas I've been paying more attention to while studying XopProtector recently.
I started with reverse-engineering an APK, and somehow ended up reading ART source code.
I guess that's probably one of the most interesting parts of Android reverse engineering.
Top comments (0)