vmprotect

name vmprotect description VMProtect deobfuscation and unpacking for native binaries (ELF, PE, Mach-O). Use this when encountering VMProtect-packed binaries, virtualized code, mutated functions, obfuscated strings, or anti-tamper protections. Triggers: any mention of 'VMProtect', 'VMP', '.vmp0/.vmp1 sections', virtualized code with handler dispatch loops, excessive push/pop/jmp chains, or binaries with characteristic VMP section names. Covers detection heuristics, string deobfuscation, IAT reconstruction, devirtualization strategies, and anti-debug bypass. VMProtect Deobfuscation & Unpacking Comprehensive reference for identifying, analyzing, and deobfuscating VMProtect-protected binaries. VMProtect (VMP) is a commercial software protection tool that uses code virtualization, mutation, packing, and anti-debug to hinder reverse engineering. Versions covered: VMProtect 2.x and 3.x (including 3.5+). Techniques differ between versions — always identify the version first. Parent skill: ../SKILL.md — generic deobfuscation framework and reusable techniques (mprotect monitoring, API-level string capture, section dumping, deduplication). 1. Detection Heuristics Before attempting deobfuscation, confirm the target is VMProtect-protected. These heuristics range from trivial to behavioral. 1.1 Section Names VMProtect adds characteristic sections to the binary. This is the fastest indicator. .vmp0 — packed/virtualized code (primary VMP section) .vmp1 — additional VMP section (second layer or data) .vmp2 — rare, seen in heavily protected binaries .VMP — older VMProtect 2.x naming UPX0/UPX1 — VMProtect sometimes mimics UPX section names as a decoy Detection with radare2: iS~vmp # list sections matching "vmp" iS~.vmp # stricter match iS~VMP Detection with rabin2: rabin2 -S binary | grep -i vmp 1.2 Entry Point Analysis VMProtect-packed binaries have a distinctive entry point pattern: VMP 3.x: Entry point jumps into a .vmp0 section, not .text VMP 2.x: Entry point contains a push of all registers followed by a jump to the VM dispatcher The original entry point (OEP) is virtualized or redirected # Check if EP lands in a VMP section ie # show entry point iS # compare EP address against section ranges s entry0; pd 20 # disassemble first 20 instructions at EP Red flags at entry point: pushfd / pushad or push of many registers as very first instructions Immediate jump to a high/unusual address outside .text call to a location that computes the next address dynamically 1.3 VM Handler Dispatch Pattern The core of VMProtect is a bytecode interpreter (virtual machine). Look for this dispatch loop pattern: ; Typical VMP3 dispatch loop (x86-64) mov reg1, [rsi] ; fetch VM opcode from bytecode stream add rsi, 4 ; advance VM instruction pointer lea reg2, [rip + table] ; handler table base mov reg3, [reg2 + reg1*8] ; lookup handler address jmp reg3 ; dispatch to handler ; Alternative pattern (computed jump) movzx eax, byte [rbx] inc rbx jmp qword [rdi + rax*8] Heuristic signals: A single function with hundreds or thousands of basic blocks Very high cyclomatic complexity (>100) in a single function Repeated jmp reg or jmp [reg + reg*8] patterns (indirect jumps) A large jump table (>50 entries) used as a handler dispatch # Find functions with abnormally many basic blocks aflj | jq '.[] | select(.nbbs > 200) | {name, addr: .offset, blocks: .nbbs}' # Search for indirect jump patterns (handler dispatch) /c jmp rax /c jmp qword [rdi /c jmp qword [rbx 1.4 Import Table Anomalies VMProtect wraps or virtualizes imports: No imports or very few imports visible in the IAT — VMProtect resolves them at runtime via GetProcAddress / dlsym Import thunks jump through VMP stubs instead of directly to the API GetProcAddress , LoadLibraryA/W are among the few visible imports (used by VMP's own resolver) ii # list imports — suspiciously few? ii~GetProc # VMP always needs these ii~LoadLib 1.5 dlsym Mass-Resolution Pattern (Android/Linux) On Android, VMP-protected .so files resolve all their imports dynamically via dlsym at startup. This produces a distinctive burst of 100-300+ dlsym calls immediately after the library loads — covering everything from malloc and memcpy to OpenGL functions and JNI helpers. This pattern is a strong behavioral indicator: normal libraries use the ELF dynamic linker for most imports and only dlsym a handful of optional symbols. Hook dlsym and count calls originating from the target module — if you see 100+ distinct symbol resolutions at load time, VMP is almost certainly present: var dlsymCount = 0 ; Interceptor . attach ( Module . getGlobalExportByName ( 'dlsym' ), { onEnter ( args ) { if (! isFromTarget ( this . returnAddress )) return ; var name = args[ 1 ]. readCString (); if (name) { dlsymCount++; console . log ( '[dlsym #' + dlsymCount + '] ' + name); } } }); 1.6 String Obfuscation Indicators VMProtect encrypts strings referenced by virtualized code: iz (data section strings) returns very few meaningful strings izz (all strings) may reveal encrypted blobs or base64-like data API name strings are absent (resolved dynamically) String references ( axt ) point into VMP sections, not .text iz # meaningful strings are missing izz~flag # try to find something izz | wc -l # compare count — VMP binaries have far fewer readable strings 1.7 Code Mutation Indicators Mutated (but not virtualized) functions show: Junk instructions: meaningless arithmetic that cancels out (e.g., add rax, 5; sub rax, 5 ) Opaque predicates: conditional jumps where the condition is always true/false Register shuffling: excessive mov reg, reg chains Constant unfolding: simple constants split into multi-step computations Dead stores: writes to registers/memory immediately overwritten # Look for mutation patterns — high instruction count, low semantic density pdf @ fcn.addr | grep -c "nop\|xchg\|stc\|clc\|cmc" 1.8 Anti-Debug and Anti-Tamper Signatures VMProtect includes runtime checks: Calls to IsDebuggerPresent , NtQueryInformationProcess , CheckRemoteDebuggerPresent RDTSC timing checks (measure time between two points) INT 2D / INT 3 exceptions used as anti-debug CRC/hash checks over code sections (anti-tamper) TLS callbacks that run before main() On Android/Linux: reads /proc/self/status and checks TracerPid field ii~Debugger ii~NtQuery /c rdtsc /c int 0x2d /c int 3 iS~.tls # TLS section present? 1.9 mprotect Storm at Startup (Android/Linux) VMP-protected .so files call mprotect repeatedly during initialization to: Make packed sections writable ( r-x → rwx ) Unpack/decrypt code into those sections Re-protect them ( rwx → r-x ) Hook mprotect and filter for calls targeting the library's address range. Seeing 5-20+ mprotect calls on a single library at load time is a strong VMP indicator (normal libraries make 0-1 calls). See the parent skill for the hook template. 1.10 uncompress / zlib Usage During Unpacking VMProtect uses zlib's uncompress function to decompress packed code sections at runtime. If you see uncompress calls with output landing inside the target library's memory range right after load, this confirms VMP packing. Interceptor . attach ( Module . getGlobalExportByName ( 'uncompress' ), { onEnter ( args ) { this . dest = args[ 0 ]; this . srcLen = args[ 3 ]. toUInt32 (); }, onLeave ( retval ) { if (retval. toInt32 () !== 0 ) return ; if ( this . dest . compare (targetBase) >= 0 && this . dest . compare (targetEnd) < 0 ) { console . log ( '[!] uncompress wrote into target module @ ' + this . dest ); } } }); 1.11 Composite Detection Script (radare2) Run this sequence to get a confidence score: # 1. Section check iS~vmp # 2. Entry point check s entry0; pd 5 # 3. Import poverty check ii | wc -l # < 10 imports on a complex binary = suspicious # 4. String poverty check iz | wc -l # very few strings for a large binary # 5. Indirect jump frequency /c jmp rax; /c jmp rbx; /c jmp rcx # many hits = VM dispatch # 6. Anti-debug imports ii~Debugger; ii~NtQuery; ii~CheckRemote # 7. Large functions aflj | jq '[.[] | select(.nbbs > 100)] | length' Scoring: .vmp sections found → confirmed VMProtect 3+ other heuristics match → highly likely VMProtect 1–2 heuristics match → possibly VMProtect or other virtualizer (check for Themida/Code Virtualizer patterns) Behavioral confirmation (requires running the binary): dlsym mass-resolution (100+ calls at startup) → confirmed VMP import obfuscation mprotect storm (5+ calls on the library at load) → confirmed VMP packing uncompress writing into the library → confirmed VMP compression /proc/self/status TracerPid check → confirmed VMP anti-debug 2. VMProtect Architecture Understanding the VM is essential for deobfuscation. 2.1 VM Components +------------------+ +----------------+ +------------------+ | Bytecode Stream | --> | VM Dispatcher | --> | Handler Table | | (encrypted ops) | | (fetch-decode) | | (native stubs) | +------------------+ +----------------+ +------------------+ | +-----+-----+ | | +---------+ +---------+ | VM Stack| | VM Regs | | (RSP- | | (mapped | | based) | | to mem) | +---------+ +---------+ Bytecode stream: Encrypted VM opcodes stored in .vmp0 / .vmp1 Dispatcher: Fetch-decode-execute loop; fetches next opcode, decrypts, jumps to handler Handler table: Array of native code snippets, each implementing one VM opcode VM stack: Separate stack area, often pointed to by a repurposed general register VM context/registers: Virtual registers stored in memory, not mapped 1:1 to CPU registers 2.2 VMP Opcode Classes VMProtect VMs implement these broad opcode categories: Category Examples Purpose Stack ops vPush, vPop Move data on/off VM stack Arithmetic vAdd, vSub, vMul, vDiv, vNor ALU operations (NOR is used to build AND/OR/XOR/NOT) Memory vLoad, vStore Read/write native memory Control flow vJmp, vCall, vRet Branch, call native, return Flags vPushFlags, vPopFlags Manage EFLAGS/RFLAGS Context vReadReg, vWriteReg Access native CPU registers Crypto vDecrypt, vRotate Decrypt next bytecode chunk 2.3 Version Differences Feature VMP 2.x VMP 3.x Section names .VMP0 , .VMP1 .vmp0 , .vmp1 Handler encoding Direct handler addresses Delta-encoded / encrypted handler offsets Bytecode encryption Simple XOR/ADD rolling key Multi-layer, key-dependent decryption per opcode VM entry pushad; pushfd; call vm_entry More varied, often via call [rip+disp] Handler count ~30-50 ~40-80 (more granular) Mutation Light Heavy mutation + MBA (mixed boolean arithmetic) Anti-debug Basic ( IsDebuggerPresent ) Layered (timing, exception, TLS, CRC, TracerPid) 3. String Deobfuscation VMProtect encrypts strings referenced by protected functions. The most effective approach is to hook the consumers of decrypted strings rather than reversing the decryption algorithm. 3.1 Strategy: Hook API Consumers (Proven Approach) Decrypted strings must eventually flow through standard APIs — JNI calls, libc string functions, dlsym , logging. By hooking these APIs and filtering for calls originating from the target module, you capture all decrypted strings without needing to understand the decryption algorithm. This approach was proven effective on VMProtect-protected Android .so files, recovering 2000+ unique strings including class names, method signatures, dynamically resolved API names, and file paths. 3.2 JNI Vtable Hooking (Android) The most valuable source of strings on Android. Hook JNI functions by their vtable index to capture all strings crossing the Java-native boundary: Java . perform ( function ( ) { var env = Java . vm . getEnv (); var vtable = env. handle . readPointer (); // NewStringUTF — native code creating Java strings Interceptor . attach (vtable. add ( 167 * Process . pointerSize ). readPointer (), { onEnter : function ( args ) { var s = args[ 1 ]. readUtf8String (); if (s) logString ( 'JNI:NewStringUTF' , s); } }); // GetStringUTFChars — native code reading Java strings Interceptor . attach (vtable. add ( 169 * Process . pointerSize ). readPointer (), { onEnter : function ( args ) { this . jstr = args[ 1 ]; }, onLeave : function ( retval ) { var s = retval. readUtf8String (); if (s) logString ( 'JNI:GetStringUTFChars' , s); } }); // FindClass — class names being loaded Interceptor . attach (vtable. add ( 6 * Process . pointerSize ). readPointer (), { onEnter : function ( args ) { var s = args[ 1 ]. readCString (); if (s) logString ( 'JNI:FindClass' , s); } }); // GetMethodID — method lookups (name + signature) Interceptor . attach (vtable. add ( 33 * Process . pointerSize ). readPointer (), { onEnter : function ( args ) { var name = args[ 2 ]. readCString (); var sig = args[ 3 ]. readCString (); if (name) logString ( 'JNI:GetMethodID' , name, sig); } }); // GetFieldID — field lookups Interceptor . attach (vtable. add ( 36 * Process . pointerSize ). readPointer (), { onEnter : function ( args ) { var name = args[ 2 ]. readCString ();