Commit Graph
12 Commits
Author SHA1 Message Date
Claude 9fce8132d8 diag: trace_external_calls can dump visited PCs and record sentinel push chain
Two additions, both motivated by the example2-virt.bin diagnosis session:

- --dump-visited <path>: writes every unique instruction PC the emulator
  executes, in first-visit order. Diff against the lifter's 'reached
  addresses' trace (MERGEN_DIAG_LIFT_PROGRESS=1) to localise where the
  lifter's static exploration diverges from the dynamic path.

- UC_HOOK_MEM_WRITE for stack-addressed 8-byte writes whose payload is a
  sentinel. Records every such write, not just the first, because Themida
  uses push-pop swap gadgets that stage a sentinel on the stack
  transiently before the 'real' push lands it at the ret-target slot.
  The last-5-pushes summary exposes this.

Findings for example2-virt.bin @ 0x140001000:
- lifter covers emu_pos=0..1298 out of 4210 unique PCs (~30%)
- external call site is at emu_pos=4209; gap of 2911 unvisited PCs
- lifter visits 5 addresses the runtime never takes (wrong concolic branch)
- the 'final push to ret slot' is not a 'push [iat]' but rather
  'sub qword ptr [r14], <const>' — the VM decrypts a pre-staged
  stack slot in place to reconstruct the IAT pointer. Pattern-match
  recognition alone cannot handle this; concrete VM-dispatch unrolling
  is required.
2026-04-24 07:40:19 +03:00
Claude 4114bb61f7 diag: add Unicorn-based external-call tracer; document Themida transform
Adds scripts/dev/trace_external_calls.py: loads a PE into Unicorn,
patches every IAT slot with a unique unmapped-address sentinel, then
emulates from the chosen entry. When any call/jmp/ret resolves its
target to a sentinel, logs the call-site address, the mnemonic, and
the addressing form. One-shot diagnostic for answering 'what x86
instruction issues this external call at runtime.'

Using it on example2-virt.bin shows the Themida transform precisely:
- guest imports (GetStdHandle etc) remain in the IAT
- every guest call site is rewritten from 'call [rip+IAT]' to a
  VM-staged 'push target; ret' where target was loaded from the
  IAT upstream
- for example2, the first external call happens at VA 0x14017fa77
  via 'ret 0', popping the GetStdHandle IAT value off the stack
- Themida strips its own SDK markers (VirtualizerSDK64.dll#103/#503)
  from the IAT; our ignore_imports filter already accounts for this

The lifter's current recognition handles direct call-through-IAT
and register-indirect IAT calls (the non-virt binary resolves 5
imports cleanly). It does not recognize the ret-pops-IAT-loaded-
pointer pattern, which is why the virt lift surfaces zero imports.

Also annotates themida_samples.json with these properties inline
so the transform semantics live next to the test that exercises
them.
2026-04-24 07:31:39 +03:00
yusufcanislek eb49a35cc7 Pin rewrite CI clang-cl toolchain 2026-04-04 15:53:28 +03:00
yusufcanislek ce77dc514d Fix loop recovery and clang-cl CI gaps 2026-04-04 15:12:49 +03:00
yusufcanislek 555a23def5 Force pinned LLVM clang in rewrite CI 2026-04-04 09:56:03 +03:00
yusufcanislek 2fd523b4e4 Pin CI clang toolchain selection 2026-04-04 09:47:18 +03:00
yusufcanislek 67c26c9bab Fix iced CI cargo detection 2026-04-03 20:28:26 +03:00
yusufcanislek 8fba033cc6 Fix VMP gate and loop safety 2026-04-03 15:00:42 +03:00
yusufcanislek cdd52c271a feat: profile VMP samples and speed up lifting
- add nested lift diagnostics and helper-level profiling for protected381 targets
- refactor function signature specs and optimize folderBinOps fast paths
- implement the full SCAS family and document VMP 3.6 INT dispatcher findings
- add reproducible profiling scripts and root VMP testing notes
2026-03-31 22:02:57 +03:00
yusufcanislek eb10474eb8 feat: commit working-tree changes required by rewrite gates
Lifter improvements:
- PathSolver.ipp: enhanced path memoization, switch-target diagnostics
- GEPTracker.ipp: expanded value tracking, graceful bail-out paths
- Semantics_Misc.ipp: clean up CPUID handler (remove dead comments,
  simplify constant emission)

Rewrite infrastructure:
- instruction_microtests.json: add jumptable manifest entries
  (calc_jumptable, jumptable_basic, jumptable_dense) with semantic cases
- golden_ir_hashes.json: add hashes for new jumptable samples
- build_samples.cmd: support C jumptable /O2 compilation pass
- oracle vectors: regenerated (oracle_vectors.json trimmed to current
  seed set, full-handler vectors updated with new handlers)
- run_microtests.cmd / run_all_handlers.cmd: script improvements
- test.py: add jumptable semantic cases to coverage

Dev scripts:
- configure_iced/zydis.cmd, build_iced/zydis.cmd: improved toolchain
  detection and MERGEN_BUILD_JOBS support

Review automation:
- format_comment.py, invariant_guard.py, risk_map.py, shard_pr.py:
  minor fixes aligned with verify_plan public API rename

Docs:
- REWRITE_BASELINE.md: updated coverage summary and script docs
- REVIEWER_RULES.md: minor formatting
2026-03-26 07:53:43 +03:00
yusufcanislek 96f1caa06a build: make iced/zydis selection deterministic in scripts and cmake 2026-03-19 01:59:44 +03:00
yusufcanislek 567e0d7daf Add rewrite regression automation, vectors, and documentation 2026-03-03 23:04:21 +03:00