45 Commits
Author SHA1 Message Date
Vladimir Panteleev 32909f3893 Update Changelog 2013-08-21 21:11:03 +00:00
Vladimir Panteleev 472a9cec57 Add finddef instruction 2013-08-18 08:16:11 +00:00
Vladimir Panteleev 6cbb185114 disassembler: Use std.digest.md
Silences compiler deprecation notice.
2013-08-18 07:58:46 +00:00
Vladimir Panteleev f32f6fb128 abcfile: Add unimplemented {get|set|delete}propertylate instructions
Needed to handle code obfuscated by DoSWF.
Parameter list (empty) guessed judging by {as|is}typelate instructions.

Fixes issue #18.
2013-06-23 06:38:25 +00:00
Vladimir Panteleev d6d7ecf47e swflzmacompress: Add an option to update the SWF version number 2013-06-02 15:27:55 +00:00
Vladimir Panteleev 0cefc587e6 Fix DMD 2.063 slice warnings 2013-03-23 18:53:22 +00:00
Vladimir Panteleev a45b4e2bdc lzma: Work around liblzma dictionary size limitation 2013-03-18 11:25:04 +00:00
Vladimir Panteleev e383172765 lzma: Remove debugging code 2013-03-18 09:35:00 +00:00
Vladimir Panteleev c62b5438d7 Documentation: Update default DCFLAGS 2013-03-18 09:28:27 +00:00
Vladimir Panteleev a1a214ae72 build_rabcdasm: Allow overriding program list / build flags on command line 2013-03-18 09:28:09 +00:00
Vladimir Panteleev 222fa1109c Documentation: Update download link 2012-12-26 13:48:53 +02:00
Vladimir Panteleev 9fa2d122fa disassembler: Further improve refname generation by adding context priority 2012-12-02 12:06:06 +02:00
Vladimir Panteleev 93adc953c2 disassembler: Separate class and instance ref context 2012-12-02 12:05:34 +02:00
Vladimir Panteleev dc222f9e43 disassembler: Relax trait multiname kind restrictions 2012-12-01 14:05:33 +02:00
Vladimir Panteleev fb2f96b81a disassembler: Visit instance protected namespace 2012-12-01 14:01:19 +02:00
Vladimir Panteleev 14f5e669ee disassembler: Visit slot types 2012-11-27 15:23:24 +02:00
Vladimir Panteleev 4e3d43df5f 64-bit fixes 2012-09-29 14:06:39 +02:00
Vladimir Panteleev c1d6b29d94 Work around apparent liblzma bugs 2012-09-29 14:06:06 +02:00
Vladimir Panteleev 8fc7253bba Fix some warnings and -property compliance 2012-09-29 14:18:56 +03:00
Vladimir Panteleev ac2bde2550 Fix handling of truncated tags 2012-09-29 14:08:33 +03:00
Vladimir Panteleev 99dfd797ba Update Changelog 2012-09-08 20:57:34 +03:00
Vladimir Panteleev f4824cd3b4 assembler: Distinguish unlabeled and first-label homonym namespace 2012-09-07 00:52:25 +03:00
Vladimir Panteleev 87047b920c disassembler: Fix absolute UNIX paths 2012-09-07 00:50:22 +03:00
Vladimir Panteleev f17e97848b DMD 2.060 (work around issue 8526) 2012-09-07 00:01:47 +03:00
Vladimir Panteleev 336665a213 Documentation updates 2012-09-04 20:14:04 +03:00
Vladimir Panteleev fdb8489382 Method ID debug code 2012-09-04 20:13:59 +03:00
Vladimir Panteleev afe42b5d1f disassembler: improve ContextItem combining 2012-09-04 20:13:54 +03:00
Vladimir Panteleev 3a9829467f disassembler: dead code cleanup 2012-09-04 20:13:50 +03:00
Vladimir Panteleev 8e969d9e76 disassembler: Reduce memory allocations 2012-09-04 20:13:45 +03:00
Vladimir Panteleev d755065785 refid generation overhaul (support non-private homonym namespaces) 2012-09-04 20:13:40 +03:00
Vladimir Panteleev ea3dfd30b1 Documentation: rewrap Markdown to 79 columns 2012-09-03 18:25:40 +03:00
Vladimir Panteleev 86a4a2e0b8 Documentation: dos2unix 2012-09-03 18:05:49 +03:00
Vladimir Panteleev 704045594a Update Changelog 2012-09-03 16:01:00 +03:00
Vladimir Panteleev 5b4c31d2a0 Documentation: Explain #version a bit more 2012-09-03 16:00:04 +03:00
Vladimir Panteleev a56d074dad disassembler: Fix handling of recursively-nested functions 2012-09-03 14:38:11 +03:00
Vladimir Panteleev d722bdcb3c abcfile: Add coerce_b, coerce_i and coerce_d opcodes 2012-09-03 14:37:51 +03:00
Vladimir Panteleev ef9e768b07 DMD 2.060 2012-08-20 15:15:59 +03:00
Vladimir Panteleev d321be2b25 build_rabcdasm: Force rebuild of test programs 2012-08-10 06:57:23 +03:00
Vladimir Panteleev 6543d17fc4 Attempted fix for DMD 2.060 2012-08-09 08:42:18 +03:00
Vladimir Panteleev e20332aeba autodata: Fix AutoToString for strings 2012-08-09 06:36:58 +03:00
Vladimir Panteleev d77e2b679e @property fixes 2012-08-09 06:36:55 +03:00
Vladimir Panteleev 9d0d0355e5 autodata: Use .toString when present when dumping arrays 2012-08-09 06:36:38 +03:00
Vladimir Panteleev 5795d5f374 assembler: Don't accept out-of-range ubyte literals 2012-05-01 01:17:16 +03:00
Vladimir Panteleev 8e8d204269 disassembler: Relax filename sanitization for names like "controller" 2012-04-10 01:50:44 +03:00
Vladimir Panteleev 20f416d9d8 DMD 2.059 fixes 2012-04-09 10:22:33 +03:00
14 changed files with 1131 additions and 774 deletions
+44 -13
View File
@@ -1,6 +1,36 @@
RABCDAsm Changelog
==================
RABCDAsm v1.14 (2013.08.21)
---------------------------
* Improved refid generation
* Worked around liblzma dictionary size limitation, which prevented
decompression of some LZMA-compressed files
* Added an option to `swflzmacompress` to update the SWF version number
* Added `finddef` instruction
* Added unimplemented `{get|set|delete}propertylate` instructions
* Documentation updates
RABCDAsm v1.13 (2012.09.29)
---------------------------
* Fixed handling of truncated SWF tags
RABCDAsm v1.12 (2012.09.08)
---------------------------
* Relaxed filename sanitization for names like "controller"
* Added range validation for byte literals
* Fixed LZMA support misdetection caused by stale rdmd cache
* Added coerce_b, coerce_i and coerce_d opcodes
* Fixed handling of recursively-nested function literals
* Fixed handling of homonym non-private namespaces
* #privatens directives are now ignored, and no longer generated
* Improved refid generation
* Bumped #version to 3 (v1.12 disassemblies are not backwards-compatible)
* Documentation updates
RABCDAsm v1.11 (2012.03.15)
---------------------------
@@ -8,6 +38,7 @@ RABCDAsm v1.11 (2012.03.15)
* Improved refid generation for protected namespaces
* Improved handling of overflown signed integers
* Fixed metadata handling (caused by error in ABC format specification)
* Added #version directive (v1.11 disassemblies are not backwards-compatible)
* Removed path length limitation on Windows
* Improved filename sanitization
* Updated asasm.hrc
@@ -52,27 +83,27 @@ RABCDAsm v1.6 (2011.05.13)
--------------------------
* Overhaul of the structure of disassembler-generated files
* Private namespace names and `refid`s for nested classes are now generated
* Private namespace names and `refid`s for nested classes are now generated
from their referenced contexts
* This also fixes problems caused by private namespaces being referenced
* This also fixes problems caused by private namespaces being referenced
in multiple distinct contexts
* One `#include`d file per script instead of per class
* Nested classes, script initializers and other script traits are now in
* Nested classes, script initializers and other script traits are now in
the same file as their main class
* Inline methods are now in separate files, placed according to their usage
* Inline methods are now in separate files, placed according to their usage
context
* Private namespaces are sorted by their autogenerated names in the
* Private namespaces are sorted by their autogenerated names in the
`.privatens.asasm` file
* No syntax changes, output is backwards-compatible
* Improved escaping of file names
* Output correct opcode error locations
* Disassembler will now refuse to overwrite files
* `rabcdasm` will now always generate output in the same directory as the
* `rabcdasm` will now always generate output in the same directory as the
input file
* Failure to decode a method body is now not a fatal error
* Aggressively try to shorten double literals to their shortest representation
* Added the `swfbinexport` and `swfbinreplace` utilities, to aid in manipulating
contents of `DefineBinaryData` tags
* Added the `swfbinexport` and `swfbinreplace` utilities, to aid in
manipulating contents of `DefineBinaryData` tags
* The package can now be built using a simple, dedicated build tool
* Documentation updates
@@ -89,7 +120,7 @@ RABCDAsm v1.4 (2011.03.07)
* Source code ported to D2
* Add support for forward-references for TypeName-kind Multinames
* Correctly order classes by dependencies (extends/implements) and reference
* Correctly order classes by dependencies (extends/implements) and reference
count
* Finish Metadata support
* Documentation updates
@@ -98,7 +129,7 @@ RABCDAsm v1.3 (2010.11.11)
--------------------------
* Fixed double precision problem
* This also fixes problems with illegal default values for function
* This also fixes problems with illegal default values for function
parameters (default values for integer parameters are stored as doubles,
which might become out-of-range due to inadequate double precision)
* Added Changelog
@@ -107,8 +138,8 @@ RABCDAsm v1.3 (2010.11.11)
RABCDAsm v1.2 (2010.11.06)
--------------------------
* Fixed ref generation for orphan objects which were only referenced
by other orphans
* Fixed ref generation for orphan objects which were only referenced by other
orphans
* Better error handling in `abcexport`; warn when no DoABC tags found
* Documentation updates
@@ -118,7 +149,7 @@ RABCDAsm v1.1 (2010.06.30)
* Private namespaces are now referenced by auto-generated names
* Use `:` to delimit namespace and name in QNames for consistency
* Fixed relative include paths
* Add optional byte offsets to labels, which allows lossless representation
* Add optional byte offsets to labels, which allows lossless representation
of jumps inside instructions and outside the function bounds
* Documentation updates
+455 -451
View File
@@ -1,451 +1,455 @@
Robust ABC (ActionScript Bytecode) [Dis-]Assembler
==================================================
[RABCDAsm][] is a collection of utilities including an ActionScript 3
assembler/disassembler, and a few tools to manipulate SWF files.
These are:
* `rabcdasm` - ABC disassembler
* `rabcasm` - ABC assembler
* `abcexport` - extracts ABC from SWF files
* `abcreplace` - replaces ABC in SWF files
* `swfdecompress` - decompresses zlib-compressed SWF files
* `swf7zcompress` - (re-)compress the contents of a SWF using 7-Zip
* `swflzmacompress` - compress the contents of a SWF using LZMA
* `swfbinexport` / `swfbinreplace` - extract/replace contents of binary
data tags from SWF files
`abcexport` and `abcreplace` are reimplementations of similar utilities from
my [swfutilsex][] Java package, however these work faster as they do not parse
the SWF files as deeply.
`swfdecompress` is ancillary and is only useful for debugging and studying of
the SWF file format, and not required for ABC manipulation. It is functionally
equivalent to [flasm][]'s `-x` option. If you frequently work on compressed
SWF files, you may want to decompress them to speed processing up.
`swf7zcompress` is an utility to further reduce the size of SWF files. It uses
[7-Zip][] to compress the data better than the standard zlib library would. It
requires that the `7z` command-line program be installed and in `PATH`.
`swflzmacompress` compresses SWF files using the [LZMA][] algorithm, support
for which was introduced in Flash 11. It will only work with SWF files with
version 13 or higher.
`swfbinexport` and `swfbinreplace` aid in the manipulation of
`DefineBinaryData` tags in SWF files (some files may contain nested SWF files
stored in these tags).
[RABCDAsm]: http://github.com/CyberShadow/RABCDAsm
[swfutilsex]: http://github.com/CyberShadow/swfutilsex
[flasm]: http://flasm.sourceforge.net/
[7-Zip]: http://www.7-zip.org/
[LZMA]: http://en.wikipedia.org/wiki/Lempel-Ziv-Markov_chain_algorithm
Motivation and goals
--------------------
This package was created due to lack of similar software out there.
Particularly, I needed an utility which would allow me to edit ActionScript 3
bytecode with the following properties:
1. Speed. Less waiting means more productivity. `rabcasm` can assemble large
projects (>200000 LOC) in under a second on modern machines.
2. Comfortably-editable output. Each class is decompiled to its own file, with
files arranged in subdirectories representing the package hierarchy. Class
files are `#include`d from the main file.
3. Most importantly - robustness! If the Adobe AVM can load and run the file,
then it must be editable - no matter if the file is obfuscated or otherwise
mutilated to prevent reverse-engineering. RABCDAsm achieves this by using
a textual representation closer to the ABC file format, rather than to what
an ActionScript compiler would generate.
Compiling from source
---------------------
RABCDAsm is written in the [D programming language, version 2][d2].
Assuming you have [git][] and a D2 compiler, such as [dmd][] or [gdc][]
installed, compiling should be as straight-forward as:
git clone git://github.com/CyberShadow/RABCDAsm.git
cd RABCDAsm
dmd -run build_rabcdasm.d
Substitute `dmd` with `gdmd` if you're using gdc. You can use the `DC` and
`DCFLAGS` environment variables to override the detected compiler and default
compilation flags (`-w -O -inline`).
To be able to manipulate SWF files packed with LZMA compression, you'll need to
have the liblzma library and development files installed on your system.
[d2]: http://dlang.org/
[dmd]: http://www.digitalmars.com/d/download.html
[gdc]: http://bitbucket.org/goshawk/gdc/
[git]: http://git-scm.com/
Pre-compiled binaries
---------------------
You can find pre-compiled Windows binaries in the [Downloads section on
GitHub][downloads]. However, please don't expect them to be up-to-date
with the latest source versions.
[downloads]: http://github.com/CyberShadow/RABCDAsm/downloads
Usage
-----
To begin hacking on a SWF file:
abcexport file.swf
This will create `file-0.abc` ... `file-N.abc` (often just `file-0.abc`). Each
file corresponds to an ABC block inside the SWF file.
To disassemble one of the `.abc` files:
rabcdasm file-0.abc
This will create a `file-0` directory, which will contain `file-0.main.asasm`
(the main program file), `file-0.privatens.asasm` (private namespace alias
definitions), and files for ActionScript scripts, classes, and orphan and
script-level methods.
To assemble the `.asasm` files back, and update the SWF file:
rabcasm file-0/file-0.main.asasm
abcreplace file.swf 0 file-0/file-0.main.abc
The second `abcreplace` argument represents the index of the ABC block in the
SWF file, and corresponds to the number in the filename created by `abcexport`.
`swfbinexport` and `swfbinreplace` are used in the same manner as `abcexport`
and `abcreplace`.
Syntax
======
The syntax of the disassembly was designed to be very simple and allow fast
and easy parsing. It is a close representation of the `.abc` file format, and
thus it is somewhat verbose. All constant pool elements (signed/unsigned
integers, doubles, strings, namespaces, namespace sets, multinames) are always
*expanded inline*, for ease of editing. Similarly, classes, instances, methods
and method bodies are also defined inline, in the context of their "parent"
object. By-index references of classes and methods (used in the `newclass`,
`newfunction` and `callstatic` instructions) are represented via
automatically-generated unique "reference strings", declared as `refid` fields.
If you haven't yet, I strongly recommend that you look through Adobe's
[ActionScript Virtual Machine 2 (AVM2) Overview][avm2]. You will most likely
need to consult it for the instruction reference anyway (although you can also
use [this handy list][avm2i] as well). You will find it difficult to understand
the disassembly without good understanding of concepts such as namespaces and
multinames.
[avm2]: http://www.adobe.com/devnet-archive/actionscript/articles/avm2overview.pdf
[avm2i]: http://www.anotherbigidea.com/javaswf/avm2/AVM2Instructions.html
Overview
--------
In order to guarantee unambiguity and data preservation, all strings read from
the input file - including identifiers (variable/function/class names) - are
represented as string literals. Thus, the syntax does not have any "reserved
words" or such - an unrecognized word is treated as an error, not as an
identifier.
Whitespace (outside string literals, of course) is completely ignored, except
where required to separate words. Comments are Intel-assembler-style: a single
`;` demarks a comment until the next end-of-line. Control directives (such as
`#include`) are allowed anywhere where whitespace is allowed.
The syntax is comprised of hierarchical blocks. Each block contains a number
of fields - starting with a keyword specifying the field type. A block is
terminated with the `end` keyword. Some fields contain a limited number of
parameters, and others are, or contain blocks.
Hierarchy
---------
The topmost block in the hierarchy is the `program` block. This must be the
first block in the file (thus, `program` must be the first word in the file as
well). The `program` block contains `script` fields, and `class` / `method`
fields for "orphan" classes and methods (not owned by other objects in the
hierarchy). Orphan methods are usually anonymous functions. The file version is
also specified in the `program` block, using the `minorversion` and
`majorversion` fields (both unsigned integers).
`script` blocks have one mandatory `sinit` field (the script initialization
method) and `trait` fields.
A "trait" can be one of several kinds. The kind is specified right after the
`trait` keyword, followed by the trait name (a multiname). Following the name
are the trait fields, varying by trait kind:
* `slot` / `const` : `slotid` (unsigned integer), `type` (multiname), `value`
* `class` : `slotid`, `class` (the actual class block)
* `function` : `slotid`, `method` (the actual method block)
* `method` / `getter` / `setter` : `dispid` (unsigned integer), `method`
Additionally, all traits may have `flag` fields, describing the trait's
attributes (`FINAL` / `OVERRIDE` / `METADATA`), and `metadata` blocks.
`metadata` blocks (which are ignored by the AVM) consist of a name string, and
a series of `item` fields - each item having a key and value string.
`class` blocks have mandatory `instance` and `cinit` fields, defining the class
instance and the class initializer method respectively. They may also have
`trait` fields and a `refid` field (the `refid` field is not part of the file
format - it's an unique string to allow referencing the class, see above).
`instance` blocks - always declared inline of their `class` block - must
contain one `iinit` field (the instance initializer method), and may contain
one `extends` field (multiname), `implements` fields (multinames), `flag`
fields (`SEALED` / `FINAL` / `INTERFACE` / `PROTECTEDNS`), one `protectedns`
field (namespace), and `trait` fields.
`method` blocks may contain one `name` field (multiname), a `refid` field,
`param` fields (multinames - this represents the parameter types), one
`returns` field (multiname), `flag` fields (`NEED_ARGUMENTS` /
`NEED_ACTIVATION` / `NEED_REST` / `HAS_OPTIONAL` / `SET_DXNS` /
`HAS_PARAM_NAMES`), `optional` fields (values), `paramname` fields (strings),
and a `body` field (method body).
`body` blocks - always declared inline of their `method` block - must contain
the `maxstack`, `localcount`, `initscopedepth` and `maxscopedepth` fields
(unsigned integers), and a `code` field. It may also contain `try` and `trait`
fields.
`code` blocks - always declared inline of their `body` block - are somewhat
different in syntax from other blocks - mostly in that they may contain labels.
Labels follow the most common syntax - a word followed by a `:` character,
optionally followed by a relative byte offset (in case of pointers inside
instructions). Multiple instruction arguments are comma-separated. Instruction
arguments' types depend on the instruction - see the `OpcodeInfo` array in
`abcfile.d` for a reference.
`try` blocks - always declared inline of their `body` block - represent an
"exception" (try/catch) block. They contain five mandatory fields: `from`,
`to` and `target` (names of labels representing start and end of the "try"
block, and start of the "catch" block respectively), and `type` and `name`
(multinames), representing the type and name of the exception variable.
Values have the syntax *type* `(` *value* `)` . *type* can be one of `Integer`,
`UInteger`, `Double`, `Utf8`, `Namespace`, `PackageNamespace`,
`PackageInternalNs`, `ProtectedNamespace`, `ExplicitNamespace`,
`StaticProtectedNs`, `PrivateNamespace`, `True`, `False`, `Null` or
`Undefined`. The type of the value depends on *type*. Types `True`, `False`,
`Null` and `Undefined` have no value.
Constants
---------
Multinames have the syntax *type* `(` *parameters* `)` . *type* can be one of
`QName` / `QNameA`, `RTQName` / `RTQNameA`, `RTQNameL` / `RTQNameLA`,
`Multiname` / `MultinameA`, `MultinameL` / `MultinameLA`, or `TypeName`.
*parameters* depends on *type*:
* `QName` / `QNameA` `(` *namespace* `,` *string* `)`
* `RTQName` / `RTQNameA` `(` *string* `)`
* `RTQNameL` / `RTQNameLA` `(` `)`
* `Multiname` / `MultinameA` `(` *string* `,` *namespace-set* `)`
* `MultinameL` / `MultinameLA` `(` *namespace-set* `)`
* `TypeName` `(` *multiname* `<` *multiname [* `,` *multiname ... ]* `>` `)`
Namespace sets have the syntax `[` *[ namespace [* `,` *namespace ... ] ]* `]`
(that is, a comma-separated list of namespaces in square brackets). Empty
namespace sets can be specified using `[]`.
Namespaces have the syntax *type* `(` *parameters* `)` . For types other than
`PrivateNamespace` there is only one parameter - a string. `PrivateNamespace`
namespaces have a second parameter, a named alias for a particular private
namespace. Internally (the ABC file format), private namespaces are
distinguished by a numerical index - `rabcdasm` will attempt to give them
descriptive names based on their context. Aliases can be defined using the
`#privatens` directive. `rabcdasm` will create a separate file containing the
aliases (`file-0.privatens.asasm`).
Strings have a syntax similar to C string literals. Strings start and end with
a `"`. Supported escape sequences (a backslash followed by a letter) are `\n`
(generates ASCII 0x0A), `\r` (ASCII 0x0D), and `\x` followed by two hexadecimal
digits, which inserts the ASCII character with that code. Any other characters
following a backslash generate that character - thus, you can escape
backslashes using `\\` and double quotes using `\"`. When decompiling,
high-ASCII characters (usually UTF-8) are not escaped - if you see gibberish
instead of international text, configure your editor to open the files in UTF-8
encoding.
Additionally, constant pool types (signed/unsigned integers, doubles, strings,
namespaces, namespace sets and multinames) may also have the value `null`
(which represents the index 0 in the ABC file). Note that `null` is
conceptually different from zero, an empty string or empty namespace set.
Macros
------
RABCDAsm has some basic macro-like capabilities, controlled by directives and
variables. These bear some similarity to the C preprocessor, however these
are processed in-loop rather than as a separate pre-processing step.
### Directives
Directives start with a `#`, followed by a word identifying the directive:
* `#include` *string* - inserts the contents of the file by the specified
filename inline. Functionally equivalent to `#mixin #get` *string* , but
faster.
* `#mixin` *string* - inserts the contents of the specified string inline.
Not very useful on its own.
* `#call` *string* `(` *[ string [* `,` *string ... ] ]* `)` - same as
`#mixin`, however it additionally sets the special variables `$1`, `$2`
etc. to the contents of the specified arguments. When the end of the
inserted string is reached, the old values of `$1`, `$2` etc. are restored.
* `#get` *string* - inserts **a string containing** the contents of the
file by the specified filename inline. Similar to #include, but it inserts
a string (surrounded by `"` etc.) instead.
* `#set` *word* *string* - assigns the contents of the string to the
variable *word*.
* `#unset` *word* - deletes the variable *word*.
* `#privatens` defines a private namespace alias, as described above.
* `#version` specifies the version of the disassembly.
### Variables
Variables are manipulated with the `#set` and `#unset` directives, and can
be instantiated in two ways:
1. `$`*name* - this inserts the contents of the variable inline. Note that
although variables are defined using a string syntax, they are not
inserted as a string using this syntax. Thus, the code:
#set str "Hello, world!"
...
pushstring $str
will expand to `pushstring Hello, world!`, which will result in an error.
To correct the problem, add escaped quotes around the variable contents
( `#set str "\"Hello, world!\""` ), or use the second syntax:
2. `$"`*name*`"` - this inserts a string containing the contents of the
variable inline. This syntax also works for `#call` arguments (e.g.
`$"1"`).
### Example
Here's an example of how to use the above features to create a macro which
logs a string literal and the contents of a register:
#set log "
findpropstrict QName(PackageNamespace(\"\"), \"log\")
pushstring $\"1\"
getlocal $2
callpropvoid QName(PackageNamespace(\"\"), \"log\"), 2
"
; ...
pushbyte 2
pushbyte 2
add_i
setlocal1
#call $"log"("two plus two equals", "1")
Highlighting
------------
Included with the project is the file `asasm.hrc`, a simple syntax definition
for the [Colorer take5][] syntax highlighting library. It should be
straight-forward to adapt it to other syntax highlighting systems.
[Colorer take5]: http://colorer.sourceforge.net/
Hacking
=======
ABC is internally represented in two forms. The `ABCFile` class stores the raw
data structures, as they appear in the binary file. `ASProgram` uses pointers
instead of indexes, allowing easy manipulation without having to worry about
record order or constant pools. Conversion between various states is done as
follows:
file.abc
| ^
------ ABCReader | | ABCWriter ----
/ v | \
/ ABCFile \
/ | ^ \
rabcdasm---------- ABCtoAS | | AStoABC --------rabcasm
\ v | /
\ ASProgram /
\ | ^ /
--- Disassembler | | Assembler ----
v |
file.asasm
`AStoABC` will rebuild the constant pools, in a manner similar to Adobe's
compilers (reverse-sorted by reference count). The exact order will almost
surely be different, however.
Should you need to write an utility to manipulate ABC, you can use the existing
code to load the file to either an `ABCFile` or `ASProgram` instance, and
perform the necessary manipulations using those classes.
Tips
====
The following tips come from the author's experience and may be useful for
RABCDAsm users.
1. Once you have disassembled a SWF file you intend to modify, you should
immediately add the directory to a distributed source control system, such as
[Git][] or [Mercurial][]. This will allow you to easily track and undo your
changes, and easily merge your changes with new versions of SWF files.
[Git]: http://git-scm.com/
[Mercurial]: http://mercurial.selenic.com/
2. If you plan on making non-trivial changes to SWF files, you should install
the [debug Flash Player][]. This will allow you to see validation and
run-time error messages, instead of simply getting an empty window.
[debug Flash Player]: http://www.adobe.com/support/flashplayer/downloads.html
3. The [Fiddler][] Web Debugging Proxy can be very useful for analyzing
websites with SWF content. The following script fragment (which is to be
placed in the `OnBeforeResponse` function) will automatically save all SWF
files while preserving the directory structure.
if (oSession.oResponse.headers.ExistsAndContains("Content-Type",
"application/x-shockwave-flash")) {
// Set desired path here
var path:String = "C:\\Temp\\FiddlerCapture\\" +
oSession.host + oSession.PathAndQuery;
if (path.Contains('?'))
path = path.Substring(0, path.IndexOf('?'));
var dir:String = Path.GetDirectoryName(path);
if (!Directory.Exists(dir))
Directory.CreateDirectory(dir);
oSession.utilDecodeResponse();
oSession.SaveResponseBody(path);
}
A more robust version of the above snippet is available as a Fiddler plugin
[here][FiddlerAutoCapture].
Once you have edited a SWF file, you can use Fiddler's [AutoResponder][] to
replace the original file with your modified version.
[Fiddler]: http://www.fiddler2.com/fiddler2/
[AutoResponder]: http://www.fiddler2.com/fiddler2/help/AutoResponder.asp
[FiddlerAutoCapture]: https://github.com/CyberShadow/FiddlerAutoCapture
Limitations
===========
* None known.
License
=======
RABCDAsm is distributed under the terms of the GPL v3 or later, with the
exception of `murmurhash2a.d`, `zlibx.d` and LZMA components, which are in the
public domain, and `asasm.hrc`, which is tri-licensed under the MPL 1.1/GPL
2.0/LGPL 2.1. The full text of the GNU General Public License can be found in
the file `COPYING`.
Robust ABC (ActionScript Bytecode) [Dis-]Assembler
==================================================
[RABCDAsm][] is a collection of utilities including an ActionScript 3
assembler/disassembler, and a few tools to manipulate SWF files. These are:
* `rabcdasm` - ABC disassembler
* `rabcasm` - ABC assembler
* `abcexport` - extracts ABC from SWF files
* `abcreplace` - replaces ABC in SWF files
* `swfdecompress` - decompresses zlib-compressed SWF files
* `swf7zcompress` - (re-)compress the contents of a SWF using 7-Zip
* `swflzmacompress` - compress the contents of a SWF using LZMA
* `swfbinexport` / `swfbinreplace` - extract/replace contents of binary data
tags from SWF files
`abcexport` and `abcreplace` are reimplementations of similar utilities from
my [swfutilsex][] Java package, however these work faster as they do not parse
the SWF files as deeply.
`swfdecompress` is ancillary and is only useful for debugging and studying of
the SWF file format, and not required for ABC manipulation. It is functionally
equivalent to [flasm][]'s `-x` option. If you frequently work on compressed
SWF files, you may want to decompress them to speed processing up.
`swf7zcompress` is an utility to further reduce the size of SWF files. It uses
[7-Zip][] to compress the data better than the standard zlib library would. It
requires that the `7z` command-line program be installed and in `PATH`.
`swflzmacompress` compresses SWF files using the [LZMA][] algorithm, support
for which was introduced in Flash 11. It will only work with SWF files with
version 13 or higher.
`swfbinexport` and `swfbinreplace` aid in the manipulation of
`DefineBinaryData` tags in SWF files (some files may contain nested SWF files
stored in these tags).
[RABCDAsm]: http://github.com/CyberShadow/RABCDAsm
[swfutilsex]: http://github.com/CyberShadow/swfutilsex
[flasm]: http://flasm.sourceforge.net/
[7-Zip]: http://www.7-zip.org/
[LZMA]: http://en.wikipedia.org/wiki/Lempel-Ziv-Markov_chain_algorithm
Motivation and goals
--------------------
This package was created due to lack of similar software out there.
Particularly, I needed an utility which would allow me to edit ActionScript 3
bytecode with the following properties:
1. Speed. Less waiting means more productivity. `rabcasm` can assemble large
projects (>200000 LOC) in under a second on modern machines.
2. Comfortably-editable output. Each class is decompiled to its own file,
with files arranged in subdirectories representing the package hierarchy.
Class files are `#include`d from the main file.
3. Most importantly - robustness! If the Adobe AVM can load and run the file,
then it must be editable - no matter if the file is obfuscated or
otherwise mutilated to prevent reverse-engineering. RABCDAsm achieves this
by using a textual representation closer to the ABC file format, rather
than to what an ActionScript compiler would generate.
Compiling from source
---------------------
RABCDAsm is written in the [D programming language, version 2][d2].
Assuming you have [git][] and a D2 compiler, such as [dmd][] or [gdc][]
installed, compiling should be as straight-forward as:
git clone git://github.com/CyberShadow/RABCDAsm.git
cd RABCDAsm
dmd -run build_rabcdasm.d
Substitute `dmd` with `gdmd` if you're using gdc. You can use the `DC` and
`DCFLAGS` environment variables to override the detected compiler and default
compilation flags (`-O -inline`).
To be able to manipulate SWF files packed with LZMA compression, you'll need
to have the liblzma library and development files installed on your system.
[d2]: http://dlang.org/
[dmd]: http://www.digitalmars.com/d/download.html
[gdc]: http://bitbucket.org/goshawk/gdc/
[git]: http://git-scm.com/
Pre-compiled binaries
---------------------
You can find pre-compiled Windows binaries on [my website][downloads].
However, please don't expect them to be up-to-date with the latest source
versions.
[downloads]: http://files.thecybershadow.net/RABCDAsm/
Usage
-----
To begin hacking on a SWF file:
abcexport file.swf
This will create `file-0.abc` ... `file-N.abc` (often just `file-0.abc`). Each
file corresponds to an ABC block inside the SWF file.
To disassemble one of the `.abc` files:
rabcdasm file-0.abc
This will create a `file-0` directory, which will contain `file-0.main.asasm`
(the main program file) and files for ActionScript scripts, classes, and
orphan and script-level methods.
To assemble the `.asasm` files back, and update the SWF file:
rabcasm file-0/file-0.main.asasm
abcreplace file.swf 0 file-0/file-0.main.abc
The second `abcreplace` argument represents the index of the ABC block in the
SWF file, and corresponds to the number in the filename created by `abcexport`.
`swfbinexport` and `swfbinreplace` are used in the same manner as `abcexport`
and `abcreplace`.
Syntax
======
The syntax of the disassembly was designed to be very simple and allow fast
and easy parsing. It is a close representation of the `.abc` file format, and
thus it is somewhat verbose. All constant pool elements (signed/unsigned
integers, doubles, strings, namespaces, namespace sets, multinames) are always
*expanded inline*, for ease of editing. Similarly, classes, instances, methods
and method bodies are also defined inline, in the context of their "parent"
object. By-index references of classes and methods (used in the `newclass`,
`newfunction` and `callstatic` instructions) are represented via
automatically-generated unique "reference strings", declared as `refid` fields.
If you haven't yet, I strongly recommend that you look through Adobe's
[ActionScript Virtual Machine 2 (AVM2) Overview][avm2]. You will most likely
need to consult it for the instruction reference anyway (although you can also
use [this handy list][avm2i] as well). You will find it difficult to
understand the disassembly without good understanding of concepts such as
namespaces and multinames.
[avm2]: http://www.adobe.com/devnet-archive/actionscript/articles/avm2overview.pdf
[avm2i]: http://www.anotherbigidea.com/javaswf/avm2/AVM2Instructions.html
Overview
--------
In order to guarantee unambiguity and data preservation, all strings read from
the input file - including identifiers (variable/function/class names) - are
represented as string literals. Thus, the syntax does not have any "reserved
words" or such - an unrecognized word is treated as an error, not as an
identifier.
Whitespace (outside string literals, of course) is completely ignored, except
where required to separate words. Comments are Intel-assembler-style: a single
`;` demarks a comment until the next end-of-line. Control directives (such as
`#include`) are allowed anywhere where whitespace is allowed.
The syntax is comprised of hierarchical blocks. Each block contains a number
of fields - starting with a keyword specifying the field type. A block is
terminated with the `end` keyword. Some fields contain a limited number of
parameters, and others are, or contain blocks.
Hierarchy
---------
The topmost block in the hierarchy is the `program` block. This must be the
first block in the file (thus, `program` must be the first word in the file as
well). The `program` block contains `script` fields, and `class` / `method`
fields for "orphan" classes and methods (not owned by other objects in the
hierarchy). Orphan methods are usually anonymous functions. The file version
is also specified in the `program` block, using the `minorversion` and
`majorversion` fields (both unsigned integers).
`script` blocks have one mandatory `sinit` field (the script initialization
method) and `trait` fields.
A "trait" can be one of several kinds. The kind is specified right after the
`trait` keyword, followed by the trait name (a multiname). Following the name
are the trait fields, varying by trait kind:
* `slot` / `const` : `slotid` (unsigned integer), `type` (multiname), `value`
* `class` : `slotid`, `class` (the actual class block)
* `function` : `slotid`, `method` (the actual method block)
* `method` / `getter` / `setter` : `dispid` (unsigned integer), `method`
Additionally, all traits may have `flag` fields, describing the trait's
attributes (`FINAL` / `OVERRIDE` / `METADATA`), and `metadata` blocks.
`metadata` blocks (which are ignored by the AVM) consist of a name string, and
a series of `item` fields - each item having a key and value string.
`class` blocks have mandatory `instance` and `cinit` fields, defining the
class instance and the class initializer method respectively. They may also
have `trait` fields and a `refid` field (the `refid` field is not part of the
file format - it's an unique string to allow referencing the class, see above).
`instance` blocks - always declared inline of their `class` block - must
contain one `iinit` field (the instance initializer method), and may contain
one `extends` field (multiname), `implements` fields (multinames), `flag`
fields (`SEALED` / `FINAL` / `INTERFACE` / `PROTECTEDNS`), one `protectedns`
field (namespace), and `trait` fields.
`method` blocks may contain one `name` field (multiname), a `refid` field,
`param` fields (multinames - this represents the parameter types), one
`returns` field (multiname), `flag` fields (`NEED_ARGUMENTS` /
`NEED_ACTIVATION` / `NEED_REST` / `HAS_OPTIONAL` / `SET_DXNS` /
`HAS_PARAM_NAMES`), `optional` fields (values), `paramname` fields (strings),
and a `body` field (method body).
`body` blocks - always declared inline of their `method` block - must contain
the `maxstack`, `localcount`, `initscopedepth` and `maxscopedepth` fields
(unsigned integers), and a `code` field. It may also contain `try` and `trait`
fields.
`code` blocks - always declared inline of their `body` block - are somewhat
different in syntax from other blocks - mostly in that they may contain
labels. Labels follow the most common syntax - a word followed by a `:`
character, optionally followed by a relative byte offset (in case of pointers
inside instructions). Multiple instruction arguments are comma-separated.
Instruction arguments' types depend on the instruction - see the `OpcodeInfo`
array in `abcfile.d` for a reference.
`try` blocks - always declared inline of their `body` block - represent an
"exception" (try/catch) block. They contain five mandatory fields: `from`,
`to` and `target` (names of labels representing start and end of the "try"
block, and start of the "catch" block respectively), and `type` and `name`
(multinames), representing the type and name of the exception variable.
Values have the syntax *type* `(` *value* `)` . *type* can be one of
`Integer`, `UInteger`, `Double`, `Utf8`, `Namespace`, `PackageNamespace`,
`PackageInternalNs`, `ProtectedNamespace`, `ExplicitNamespace`,
`StaticProtectedNs`, `PrivateNamespace`, `True`, `False`, `Null` or
`Undefined`. The type of the value depends on *type*. Types `True`, `False`,
`Null` and `Undefined` have no value.
Constants
---------
Multinames have the syntax *type* `(` *parameters* `)` . *type* can be one of
`QName` / `QNameA`, `RTQName` / `RTQNameA`, `RTQNameL` / `RTQNameLA`,
`Multiname` / `MultinameA`, `MultinameL` / `MultinameLA`, or `TypeName`.
*parameters* depends on *type*:
* `QName` / `QNameA` `(` *namespace* `,` *string* `)`
* `RTQName` / `RTQNameA` `(` *string* `)`
* `RTQNameL` / `RTQNameLA` `(` `)`
* `Multiname` / `MultinameA` `(` *string* `,` *namespace-set* `)`
* `MultinameL` / `MultinameLA` `(` *namespace-set* `)`
* `TypeName` `(` *multiname* `<` *multiname [* `,` *multiname ... ]* `>` `)`
Namespace sets have the syntax `[` *[ namespace [* `,` *namespace ... ] ]* `]`
(that is, a comma-separated list of namespaces in square brackets). Empty
namespace sets can be specified using `[]`.
Namespaces have the syntax *type* `(` *string [* `,` *string ]* `)` . The
first string indicates the namespace name. In the case that there are multiple
distinct namespaces with the same type and name (as `PrivateNamespace`
namespaces usually are), a second parameter may be present to uniquely
distinguish them. Internally (the ABC file format), namespaces are
distinguished by their numerical index. When disassembling, `rabcdasm` will
attempt to assign descriptive labels to homonym namespaces based on their
context.
Strings have a syntax similar to C string literals. Strings start and end with
a `"`. Supported escape sequences (a backslash followed by a letter) are `\n`
(generates ASCII 0x0A), `\r` (ASCII 0x0D), and `\x` followed by two
hexadecimal digits, which inserts the ASCII character with that code. Any
other characters following a backslash generate that character - thus, you can
escape backslashes using `\\` and double quotes using `\"`. When decompiling,
high-ASCII characters (usually UTF-8) are not escaped - if you see gibberish
instead of international text, configure your editor to open the files in
UTF-8 encoding.
Additionally, constant pool types (signed/unsigned integers, doubles, strings,
namespaces, namespace sets and multinames) may also have the value `null`
(which represents the index 0 in the ABC file). Note that `null` is
conceptually different from zero, an empty string or empty namespace set.
Macros
------
RABCDAsm has some basic macro-like capabilities, controlled by directives and
variables. These bear some similarity to the C preprocessor, however these are
processed in-loop rather than as a separate pre-processing step.
### Directives
Directives start with a `#`, followed by a word identifying the directive:
* `#include` *string* - inserts the contents of the file by the specified
filename inline. Functionally equivalent to `#mixin #get` *string* , but
faster.
* `#mixin` *string* - inserts the contents of the specified string inline.
Not very useful on its own.
* `#call` *string* `(` *[ string [* `,` *string ... ] ]* `)` - same as
`#mixin`, however it additionally sets the special variables `$1`, `$2`
etc. to the contents of the specified arguments. When the end of the
inserted string is reached, the old values of `$1`, `$2` etc. are restored.
* `#get` *string* - inserts **a string containing** the contents of the file
by the specified filename inline. Similar to #include, but it inserts a
string (surrounded by `"` etc.) instead.
* `#set` *word* *string* - assigns the contents of the string to the
variable *word*.
* `#unset` *word* - deletes the variable *word*.
* `#privatens` *number* *string* - deprecated, currently ignored.
* `#version` specifies the syntax version of the disassembly. Newer RABCDAsm
versions may emit disassembly output that is not backwards-compatible, but
should still understand older disassemblies. The versions are:
1. The first version.
2. Introduced in v1.11 to work around error in ABC format specification.
3. Introduced in v1.12 to support multiple non-private namespaces with
the same name. This is the current version.
### Variables
Variables are manipulated with the `#set` and `#unset` directives, and can be
instantiated in two ways:
1. `$`*name* - this inserts the contents of the variable inline. Note that
although variables are defined using a string syntax, they are not
inserted as a string using this syntax. Thus, the code:
#set str "Hello, world!"
...
pushstring $str
will expand to `pushstring Hello, world!`, which will result in an error.
To correct the problem, add escaped quotes around the variable contents
( `#set str "\"Hello, world!\""` ), or use the second syntax:
2. `$"`*name*`"` - this inserts a string containing the contents of the
variable inline. This syntax also works for `#call` arguments (e.g.
`$"1"`).
### Example
Here's an example of how to use the above features to create a macro which
logs a string literal and the contents of a register:
#set log "
findpropstrict QName(PackageNamespace(\"\"), \"log\")
pushstring $\"1\"
getlocal $2
callpropvoid QName(PackageNamespace(\"\"), \"log\"), 2
"
; ...
pushbyte 2
pushbyte 2
add_i
setlocal1
#call $"log"("two plus two equals", "1")
Highlighting
------------
Included with the project is the file `asasm.hrc`, a simple syntax definition
for the [Colorer take5][] syntax highlighting library. It should be
straight-forward to adapt it to other syntax highlighting systems.
[Colorer take5]: http://colorer.sourceforge.net/
Hacking
=======
ABC is internally represented in two forms. The `ABCFile` class stores the raw
data structures, as they appear in the binary file. `ASProgram` uses pointers
instead of indexes, allowing easy manipulation without having to worry about
record order or constant pools. Conversion between various states is done as
follows:
file.abc
| ^
------ ABCReader | | ABCWriter ----
/ v | \
/ ABCFile \
/ | ^ \
rabcdasm---------- ABCtoAS | | AStoABC --------rabcasm
\ v | /
\ ASProgram /
\ | ^ /
--- Disassembler | | Assembler ----
v |
file.asasm
`AStoABC` will rebuild the constant pools, in a manner similar to Adobe's
compilers (reverse-sorted by reference count). The exact order will almost
surely be different, however.
Should you need to write an utility to manipulate ABC, you can use the
existing code to load the file to either an `ABCFile` or `ASProgram` instance,
and perform the necessary manipulations using those classes.
Tips
====
The following tips come from the author's experience and may be useful for
RABCDAsm users.
1. Once you have disassembled a SWF file you intend to modify, you should
immediately add the directory to a distributed source control system, such
as [Git][] or [Mercurial][]. This will allow you to easily track and undo
your changes, and easily merge your changes with new versions of SWF files.
[Git]: http://git-scm.com/
[Mercurial]: http://mercurial.selenic.com/
2. If you plan on making non-trivial changes to SWF files, you should install
the [debug Flash Player][]. This will allow you to see validation and
run-time error messages, instead of simply getting an empty window.
[debug Flash Player]: http://www.adobe.com/support/flashplayer/downloads.html
3. The [Fiddler][] Web Debugging Proxy can be very useful for analyzing
websites with SWF content. The following script fragment (which is to be
placed in the `OnBeforeResponse` function) will automatically save all SWF
files while preserving the directory structure.
if (oSession.oResponse.headers.ExistsAndContains("Content-Type",
"application/x-shockwave-flash")) {
// Set desired path here
var path:String = "C:\\Temp\\FiddlerCapture\\" +
oSession.host + oSession.PathAndQuery;
if (path.Contains('?'))
path = path.Substring(0, path.IndexOf('?'));
var dir:String = Path.GetDirectoryName(path);
if (!Directory.Exists(dir))
Directory.CreateDirectory(dir);
oSession.utilDecodeResponse();
oSession.SaveResponseBody(path);
}
A more robust version of the above snippet is available as a Fiddler plugin
[here][FiddlerAutoCapture].
Once you have edited a SWF file, you can use Fiddler's [AutoResponder][] to
replace the original file with your modified version.
[Fiddler]: http://www.fiddler2.com/fiddler2/
[AutoResponder]: http://www.fiddler2.com/fiddler2/help/AutoResponder.asp
[FiddlerAutoCapture]: https://github.com/CyberShadow/FiddlerAutoCapture
Limitations
===========
* None known.
License
=======
RABCDAsm is distributed under the terms of the GPL v3 or later, with the
exception of `murmurhash2a.d`, `zlibx.d` and LZMA components, which are in the
public domain, and `asasm.hrc`, which is tri-licensed under the MPL 1.1/GPL
2.0/LGPL 2.1. The full text of the GNU General Public License can be found in
the file `COPYING`.
+13 -13
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -177,12 +177,12 @@ class ABCFile
}
uint[] metadata;
TraitKind kind() { return cast(TraitKind)(kindAttr&0xF); }
void kind(TraitKind value) { kindAttr = (kindAttr&0xF0) | value; }
@property TraitKind kind() { return cast(TraitKind)(kindAttr&0xF); }
@property void kind(TraitKind value) { kindAttr = (kindAttr&0xF0) | value; }
// TraitAttributes bitmask
ubyte attr() { return cast(ubyte)(kindAttr >> 4); }
void attr(ubyte value) { kindAttr = cast(ubyte)((kindAttr&0xF) | (value<<4)); }
@property ubyte attr() { return cast(ubyte)(kindAttr >> 4); }
@property void attr(ubyte value) { kindAttr = cast(ubyte)((kindAttr&0xF) | (value<<4)); }
}
struct Class
@@ -754,7 +754,7 @@ const OpcodeInfo[256] opcodeInfo = [
/* 0x5C */ {"0x5C", [OpcodeArgumentType.Unknown]},
/* 0x5D */ {"findpropstrict", [OpcodeArgumentType.Multiname]},
/* 0x5E */ {"findproperty", [OpcodeArgumentType.Multiname]},
/* 0x5F */ {"finddef", [OpcodeArgumentType.Unknown]},
/* 0x5F */ {"finddef", [OpcodeArgumentType.Multiname]},
/* 0x60 */ {"getlex", [OpcodeArgumentType.Multiname]},
/* 0x61 */ {"setproperty", [OpcodeArgumentType.Multiname]},
/* 0x62 */ {"getlocal", [OpcodeArgumentType.UIntLiteral]},
@@ -762,11 +762,11 @@ const OpcodeInfo[256] opcodeInfo = [
/* 0x64 */ {"getglobalscope", []},
/* 0x65 */ {"getscopeobject", [OpcodeArgumentType.UByteLiteral]},
/* 0x66 */ {"getproperty", [OpcodeArgumentType.Multiname]},
/* 0x67 */ {"getpropertylate", [OpcodeArgumentType.Unknown]},
/* 0x67 */ {"getpropertylate", []},
/* 0x68 */ {"initproperty", [OpcodeArgumentType.Multiname]},
/* 0x69 */ {"setpropertylate", [OpcodeArgumentType.Unknown]},
/* 0x69 */ {"setpropertylate", []},
/* 0x6A */ {"deleteproperty", [OpcodeArgumentType.Multiname]},
/* 0x6B */ {"deletepropertylate", [OpcodeArgumentType.Unknown]},
/* 0x6B */ {"deletepropertylate", []},
/* 0x6C */ {"getslot", [OpcodeArgumentType.UIntLiteral]},
/* 0x6D */ {"setslot", [OpcodeArgumentType.UIntLiteral]},
/* 0x6E */ {"getglobalslot", [OpcodeArgumentType.UIntLiteral]},
@@ -788,10 +788,10 @@ const OpcodeInfo[256] opcodeInfo = [
/* 0x7E */ {"0x7E", [OpcodeArgumentType.Unknown]},
/* 0x7F */ {"0x7F", [OpcodeArgumentType.Unknown]},
/* 0x80 */ {"coerce", [OpcodeArgumentType.Multiname]},
/* 0x81 */ {"coerce_b", [OpcodeArgumentType.Unknown]},
/* 0x81 */ {"coerce_b", []},
/* 0x82 */ {"coerce_a", []},
/* 0x83 */ {"coerce_i", [OpcodeArgumentType.Unknown]},
/* 0x84 */ {"coerce_d", [OpcodeArgumentType.Unknown]},
/* 0x83 */ {"coerce_i", []},
/* 0x84 */ {"coerce_d", []},
/* 0x85 */ {"coerce_s", []},
/* 0x86 */ {"astype", [OpcodeArgumentType.Multiname]},
/* 0x87 */ {"astypelate", []},
@@ -1325,7 +1325,7 @@ private final class ABCReader
size_t start = pos;
size_t end = pos + len;
size_t offset() { return pos - start; }
@property size_t offset() { return pos - start; }
try
{
+2 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -41,6 +41,7 @@ void main(string[] args)
while (*p++) {} // skip name
tag.data = tag.data[0..p-tag.data.ptr] ~ abc;
}
tag.length = cast(uint)tag.data.length;
write(args[1], swf.write());
return;
}
+11 -8
View File
@@ -40,7 +40,7 @@ final class ASProgram
ASType kind;
string name;
uint privateIndex; // unique index for private namespaces
uint id; // unique index for private/homonym namespaces
mixin AutoCompare;
mixin AutoToString;
@@ -83,7 +83,7 @@ final class ASProgram
mixin AutoCompare;
mixin AutoToString;
R processData(R, string prolog, string epilog, H)(ref H handler) const pure nothrow
R processData(R, string prolog, string epilog, H)(ref H handler) const
{
mixin(prolog);
mixin(addAutoField("kind"));
@@ -115,7 +115,7 @@ final class ASProgram
mixin(addAutoField("vTypeName.params"));
break;
default:
assert(false, "Unknown Multiname kind");
throw new .Exception("Unknown Multiname kind");
}
mixin(epilog);
}
@@ -154,6 +154,8 @@ final class ASProgram
Value[] options;
string[] paramNames;
uint id; // file index
MethodBody vbody;
}
@@ -371,12 +373,12 @@ private final class ABCtoAS
return o;
}
ASProgram.Namespace convertNamespace(ref ABCFile.Namespace namespace, int privateIndex)
ASProgram.Namespace convertNamespace(ref ABCFile.Namespace namespace, int id)
{
auto n = new ASProgram.Namespace();
n.kind = namespace.kind;
n.name = abc.strings[namespace.name];
n.privateIndex = privateIndex;
n.id = id;
return n;
}
@@ -447,7 +449,7 @@ private final class ABCtoAS
}
}
ASProgram.Method convertMethod(ref ABCFile.MethodInfo method)
ASProgram.Method convertMethod(ref ABCFile.MethodInfo method, uint id)
{
auto n = new ASProgram.Method();
n.paramTypes.length = method.paramTypes.length;
@@ -462,6 +464,7 @@ private final class ABCtoAS
n.paramNames.length = method.paramNames.length;
foreach (j, name; method.paramNames)
n.paramNames[j] = abc.strings[name];
n.id = id;
return n;
}
@@ -665,8 +668,8 @@ private final class ABCtoAS
postConvertMultiname(multiname, multinames[i]);
methods.length = methodAdded.length = abc.methods.length;
foreach (i, ref method; abc.methods)
methods[i] = convertMethod(method);
foreach (uint i, ref method; abc.methods)
methods[i] = convertMethod(method, i);
metadata.length = abc.metadata.length;
foreach (i, ref md; abc.metadata)
+23 -20
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -67,7 +67,7 @@ final class Assembler
return File(name, data, data.ptr, data.ptr + data.length, arguments, basePath);
}
Position position()
@property Position position()
{
Position p;
p.filename = filename;
@@ -75,7 +75,7 @@ final class Assembler
return p;
}
string positionStr()
@property string positionStr()
{
auto lines = splitLines(buf);
foreach (i, line; lines)
@@ -104,7 +104,7 @@ final class Assembler
foreach (ref c; buf)
if (c == '\\')
c = '/';
return buildPath(getBasePath, buf);
return buildPath(getBasePath(), buf);
}
void skipWhitespace()
@@ -133,7 +133,7 @@ final class Assembler
}
string[string] vars;
uint[string] privateNamespaces;
uint[string] namespaceLabels; // for homonym namespaces
uint sourceVersion = 1;
void handlePreprocessor()
@@ -162,11 +162,13 @@ final class Assembler
vars.remove(readWord());
break;
case "privatens":
uint index = cast(uint)readUInt();
privateNamespaces[readString()] = index;
enforce(sourceVersion < 3, "#privatens is deprecated");
readUInt();
readString();
break;
case "version":
sourceVersion = cast(uint)readUInt();
enforce(sourceVersion >= 1 && sourceVersion <= 3, "Invalid/unknown #version");
break;
default:
files[0].pos -= word.length;
@@ -264,7 +266,7 @@ final class Assembler
{
if (fileCount == files.length)
throw new Exception("Recursion limit exceeded");
files[1..fileCount+1] = files[0..fileCount].dup;
files[1..fileCount+1] = files[0..fileCount].dup[];
fileCount++;
files[0] = file;
}
@@ -282,7 +284,7 @@ final class Assembler
if (fileCount==1)
throw new Exception("Unexpected end of file");
fileCount--;
files[0..fileCount] = files[1..fileCount+1].dup;
files[0..fileCount] = files[1..fileCount+1].dup[];
}
void expectWord(string expected)
@@ -563,14 +565,15 @@ final class Assembler
n.kind = toASType(word);
expectChar('(');
n.name = readString();
if (n.kind == ASType.PrivateNamespace)
if (peekChar() == ',')
{
expectChar(',');
skipChar();
string name = readString();
auto pindex = name in privateNamespaces;
if (pindex is null)
throw new Exception("Unknown private namespace name");
n.privateIndex = *pindex;
auto pindex = name in namespaceLabels;
if (pindex)
n.id = *pindex;
else
n.id = namespaceLabels[name] = cast(uint)namespaceLabels.length+1;
}
expectChar(')');
return n;
@@ -1042,7 +1045,7 @@ final class Assembler
throw new Exception("Don't know how to assemble OP_" ~ opcodeInfo[instruction.opcode].name);
case OpcodeArgumentType.UByteLiteral:
instruction.arguments[i].ubytev = cast(ubyte)readUInt();
instruction.arguments[i].ubytev = to!ubyte(readUInt());
break;
case OpcodeArgumentType.IntLiteral:
instruction.arguments[i].intv = readInt();
@@ -1097,13 +1100,13 @@ final class Assembler
foreach (ref f; jumpFixups)
{
scope(failure) setFile(f.where.load);
scope(failure) setFile(f.where.load());
instructions[f.ii].arguments[f.ai].jumpTarget = parseLabel(f.name, labels);
}
foreach (ref f; switchFixups)
{
scope(failure) setFile(f.where.load);
scope(failure) setFile(f.where.load());
instructions[f.ii].arguments[f.ai].switchTargets[f.si] = parseLabel(f.name, labels);
}
@@ -1208,7 +1211,7 @@ final class Assembler
auto cp = f.name in classesByID;
if (cp is null)
{
setFile(f.where.load);
setFile(f.where.load());
throw new Exception("Unknown class refid: " ~ f.name);
}
*f.ptr = *cp;
@@ -1222,7 +1225,7 @@ final class Assembler
auto mp = f.name in methodsByID;
if (mp is null)
{
setFile(f.where.load);
setFile(f.where.load());
throw new Exception("Unknown method refid: " ~ f.name);
}
*f.ptr = *mp;
+50 -26
View File
@@ -22,7 +22,7 @@ import murmurhash2a;
import std.traits;
public import std.conv;
string addAutoField(string name, bool reverseSort = false) pure
string addAutoField(string name, bool reverseSort = false)
{
return `mixin(typeof(handler).getMixin!(typeof(` ~ name ~ `), "` ~ name ~ `", ` ~ (reverseSort ? "true" : "false") ~`));`;
}
@@ -34,8 +34,8 @@ template AutoCompare()
alias typeof(this) _AutoDataTypeReference;
alias Object _AutoDataOtherTypeReference;
override hash_t toHash() const { return _AutoDataHash(); }
override bool opEquals(Object o) const pure { return _AutoDataEquals(o); }
override hash_t toHash() const { try { return _AutoDataHash(); } catch(object.Exception e) { assert(0, e.msg); } }
override bool opEquals(Object o) const { return _AutoDataEquals(o); }
override int opCmp(Object o) const { return _AutoDataCmp(o); }
}
else // struct
@@ -43,12 +43,12 @@ template AutoCompare()
alias const(typeof(this)*) _AutoDataTypeReference;
alias const(typeof(this)*) _AutoDataOtherTypeReference;
hash_t toHash() const pure nothrow @safe { return _AutoDataHash(); }
bool opEquals(ref const typeof(this) s) const pure { return _AutoDataEquals(&s); }
hash_t toHash() const { return _AutoDataHash(); }
bool opEquals(ref const typeof(this) s) const { return _AutoDataEquals(&s); }
int opCmp(ref const typeof(this) s) const { return _AutoDataCmp(&s); }
}
private hash_t _AutoDataHash() const pure nothrow @trusted
@trusted private hash_t _AutoDataHash() const
{
HashDataHandler handler;
handler.hasher.Begin();
@@ -56,7 +56,7 @@ template AutoCompare()
return handler.hasher.End();
}
private bool _AutoDataEquals(_AutoDataOtherTypeReference other) const pure
private bool _AutoDataEquals(_AutoDataOtherTypeReference other) const
{
auto handler = EqualsDataHandler!_AutoDataTypeReference(cast(_AutoDataTypeReference) other);
if (handler.other is null)
@@ -78,7 +78,7 @@ template AutoToString()
static if (is(typeof(this)==class))
override string toString() const { return _AutoDataToString(); }
else // struct
string toString() { return _AutoDataToString(); }
string toString() const { return _AutoDataToString(); }
string _AutoDataToString() const
{
@@ -89,7 +89,7 @@ template AutoToString()
template ProcessAllData()
{
R processData(R, string prolog, string epilog, H)(ref H handler) const pure nothrow
R processData(R, string prolog, string epilog, H)(ref H handler) const
{
mixin(prolog);
foreach (i, T; this.tupleof)
@@ -138,7 +138,7 @@ struct HashDataHandler
template getRawMixin(string ptr, string len)
{
enum getRawMixin = "handler.hasher.Add(" ~ ptr ~ ", " ~ len ~ ");";
enum getRawMixin = "handler.hasher.Add(" ~ ptr ~ ", to!int(" ~ len ~ "));";
}
}
@@ -202,7 +202,7 @@ struct CmpDataHandler(O)
else
static if (is(typeof(T.opCmp)))
enum dataCode = nullCheck!(T, name, reverseStr)
~ "{ int _AutoDataCmp = this." ~ name ~ ".opCmp(_AutoDataOther." ~ name ~ "); if (_AutoDataCmp != 0) return " ~ reverseStr ~ "_AutoDataCmp; }";
~ "{ int _AutoDataCmp = this." ~ name ~ ".opCmp(cast()_AutoDataOther." ~ name ~ "); if (_AutoDataCmp != 0) return " ~ reverseStr ~ "_AutoDataCmp; }";
else
enum dataCode = "if (this." ~ name ~ " < _AutoDataOther." ~ name ~ ") return " ~ reverseStr ~ "(-1);" ~
"if (this." ~ name ~ " > _AutoDataOther." ~ name ~ ") return " ~ reverseStr ~ "( 1);";
@@ -212,22 +212,46 @@ struct CmpDataHandler(O)
struct ToStringDataHandler
{
template getMixinSingle(T, string name)
{
/*
enum getMixinSingle = "
static if (is(typeof(_AutoDataResult ~= " ~ name ~ ".toString())))
_AutoDataResult ~= " ~ name ~ ".toString();
else
_AutoDataResult ~= to!string(" ~ name ~ ");
";
*/
static if (is(typeof(T.init.toString())))
enum getMixinSingle = "_AutoDataResult ~= " ~ name ~ ".toString();";
else
enum getMixinSingle = "_AutoDataResult ~= to!string(" ~ name ~ ");";
}
template getMixinBody(T, string name)
{
// TODO: arrays of arrays
static if (is(T U : U[]) && !is(T : const(char)[]))
{
enum getMixinBody = "
_AutoDataResult ~= ` [ `;
foreach (_AutoDataArrayIndex, _AutoDataArrayItem; " ~ name ~ ")
{
if (_AutoDataArrayIndex) _AutoDataResult ~= ` , `;
" ~ getMixinSingle!(U, "_AutoDataArrayItem") ~ "
}
_AutoDataResult ~= ` ] `;
";
}
else
enum getMixinBody = getMixinSingle!(T, name);
}
template getMixin(T, string name, bool reverseSort)
{
//enum getMixin = "_AutoDataResult ~= `" ~ name ~ " = ` ~ text(this." ~ name ~ ") ~ ` `;";
enum getMixin = ""; // FIXME!!!
}
}
private void test()
{
// Test instantiation
struct S
{
int i;
mixin ProcessAllData;
mixin AutoCompare;
mixin AutoToString;
enum getMixin =
"_AutoDataResult ~= `" ~ name ~ " = `;" ~
getMixinBody!(T, name) ~
"_AutoDataResult ~= ` `;";
}
}
+17 -5
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -18,6 +18,7 @@
/// A simple tool to build RABCDAsm in one command.
/// You can use the DC and DCFLAGS environment variables to override the detected compiler and compilation flags.
/// You can also pass program names or compilation options on the command-line to override the default ones.
module build_rabcdasm;
@@ -33,7 +34,7 @@ version(DigitalMars)
else
const DEFAULT_COMPILER = "gdmd";
const DEFAULT_FLAGS = "-w -O -inline";
const DEFAULT_FLAGS = "-O -inline";
const LZMA_FLAGS = "-version=HAVE_LZMA";
import std.exception;
@@ -55,14 +56,16 @@ void test(string code, string extraFlags=null)
const FN = "test.d";
std.file.write(FN, code);
scope(exit) remove(FN);
enforce(system(format("rdmd --compiler=%s %s %s %s", compiler, flags, extraFlags, FN)) == 0, "Test failed");
enforce(system(format("rdmd --force --compiler=%s %s %s %s", compiler, flags, extraFlags, FN)) == 0, "Test failed");
stderr.writeln(" >>> OK");
}
int main()
int main(string[] args)
{
try
{
auto programs = ["rabcasm", "rabcdasm", "abcexport", "abcreplace", "swfbinexport", "swfbinreplace", "swfdecompress", "swf7zcompress"];
compiler = getenv("DC");
if (compiler is null)
compiler = DEFAULT_COMPILER;
@@ -71,6 +74,15 @@ int main()
if (flags is null)
flags = DEFAULT_FLAGS;
string[] optionArgs, programArgs;
foreach (arg; args[1..$])
(arg.startsWith("-") ? optionArgs : programArgs) ~= arg;
if (optionArgs.length)
flags = optionArgs.join(" ");
if (programArgs.length)
programs = programArgs;
stderr.writeln("* Checking for working compiler...");
test(`
void main() {}
@@ -103,7 +115,7 @@ int main()
if (haveLZMA)
flags ~= " " ~ LZMA_FLAGS;
foreach (program; ["rabcasm", "rabcdasm", "abcexport", "abcreplace", "swfbinexport", "swfbinreplace", "swfdecompress", "swf7zcompress"])
foreach (program; programs)
compile(program);
if (haveLZMA)
+430 -203
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -25,7 +25,7 @@ import std.conv;
import std.exception;
import std.algorithm;
import std.path;
import std.md5;
import std.digest.md;
import abcfile;
import asprogram;
import autodata;
@@ -53,7 +53,7 @@ final class StringBuilder
auto end = pos + s.length;
while (buf.length < end)
buf.length = buf.length*2;
buf[pos..end] = s;
buf[pos..end] = s[];
pos = end;
}
@@ -70,7 +70,7 @@ final class StringBuilder
for (int l=0; l<dirSegments.length-1; l++)
{
auto subdir = join(dirSegments[0..l+1], "/");
if (!exists(longPath(subdir)))
if (subdir.length && !exists(longPath(subdir)))
mkdir(longPath(subdir));
}
write(longPath(filename), buf[0..pos]);
@@ -106,9 +106,35 @@ final class StringBuilder
final class RefBuilder : ASTraitsVisitor
{
bool[uint][string][ASType.Max] homonyms;
debug bool homonymsBuilt;
bool hasHomonyms(ASProgram.Namespace ns)
{
debug assert(homonymsBuilt);
auto pnsHomonyms = ns.name in homonyms[ns.kind];
auto nsHomonyms = pnsHomonyms ? *pnsHomonyms : null;
return nsHomonyms.length > 1;
}
void addHomonym(ASProgram.Namespace ns)
{
debug assert(!homonymsBuilt);
homonyms[ns.kind][ns.name][ns.id] = true;
}
/// Represents a link in a "context chain", which represents the context in which an object is encountered.
/// The common root of contexts is used to build ref strings, namespace labels, file paths etc.
/// Expansion is recursive and delayed until decompilation (an object may be encountered inside a private namespace,
/// all contexts of which are unknown until the whole program is scanned).
struct ContextItem
{
enum Type { Multiname, String }
enum Type
{
Multiname, /// An ASProgram.Multiname (may be a private namespace, etc.)
String, /// Fixed string
Group, /// Multiple ContextItem[]s (which are expanded and the common root is taken as the result)
}
Type type;
union
@@ -119,6 +145,11 @@ final class RefBuilder : ASTraitsVisitor
string str;
bool filenameSuffix;
}
struct
{
ContextItem[] group;
string groupFallback;
}
}
struct Segment
@@ -127,60 +158,140 @@ final class RefBuilder : ASTraitsVisitor
string str;
}
Segment[] toSegments(bool filename)
ContextItem[] reduceGroup(RefBuilder refs)
{
assert(type==Type.Group);
ContextItem[][] contexts;
foreach (context; group)
contexts ~= ContextItem.expand(refs, [context]);
ContextItem[] context;
if (contexts.length)
context = reduce!contextRoot(contexts);
if (!context.length)
context = /*null*/[ContextItem(groupFallback)];
return context;
}
Segment[] toSegments(RefBuilder refs, bool filename)
out(result)
{
debug(CONTEXTS) std.stdio.writefln("Segmented:\n\t%s\nto\n\t%s\n", this, result);
}
body
{
final switch(type)
{
case Type.Multiname:
assert(multiname.kind == ASType.QName);
auto nsName = multiname.vQName.ns.name;
if (multiname.vQName.ns.kind == ASType.PrivateNamespace) // unexpanded private namespace
nsName ~= '#' ~ to!string(multiname.vQName.ns.privateIndex);
if (nsName.length)
if (multiname.vQName.name.length)
return [Segment('/', nsName), Segment(filename ? '/' : ':', multiname.vQName.name)];
case Type.Multiname:
{
assert(multiname.kind == ASType.QName);
auto ns = multiname.vQName.ns;
auto nsName = ns.name;
// if (refs.hasHomonyms(ns))
// nsName ~= '#' ~ to!string(ns.id);
if (nsName.length)
if (multiname.vQName.name.length)
return [Segment('/', nsName), Segment(filename ? '/' : ':', multiname.vQName.name)];
else
return [Segment('/', nsName)];
else
return [Segment('/', nsName)];
else
if (multiname.vQName.name.length)
return [Segment('/', multiname.vQName.name)];
else
assert(0);
case Type.String:
return [Segment(filenameSuffix ? '.' : '/', str)];
if (multiname.vQName.name.length)
return [Segment('/', multiname.vQName.name)];
else
assert(0);
}
case Type.String:
return [Segment((filename && filenameSuffix) ? '.' : '/', str)];
case Type.Group:
{
Segment[] segments;
foreach (context; reduceGroup(refs))
segments ~= context.toSegments(refs, filename);
return segments;
}
}
}
static ContextItem[] expand(RefBuilder refs, ContextItem[] context, bool abortOnUnknown) /// recursively expand all private namespaces
static ContextItem[] expand(RefBuilder refs, ContextItem[] context)
{
ContextItem[] newContext;
foreach (ref c; context)
{
auto cEx = c.expand(refs, abortOnUnknown);
if (cEx is null) return null;
newContext ~= cEx;
auto cEx = c.expand(refs);
if (cEx)
newContext ~= cEx;
}
return newContext;
}
ContextItem[] expand(RefBuilder refs, bool abortOnUnknown)
bool expanding;
/// Recursively expand contexts (lower to Type.String ContextItems)
ContextItem[] expand(RefBuilder refs)
in
{
if (type==Type.Multiname && multiname.vQName.ns.kind == ASType.PrivateNamespace)
{
auto pcontext = multiname.vQName.ns.privateIndex in refs.privateNamespaces.contexts;
if (pcontext is null)
debug(CONTEXTS) std.stdio.writefln("Expanding:\n\t%s\n", this);
}
out(result)
{
debug(CONTEXTS) std.stdio.writefln("Expanded:\n\t%s\nto\n\t%s\n", this, result);
}
body
{
if (expanding)
final switch (type)
{
if (abortOnUnknown)
return null;
else
case Type.String:
assert(0);
case Type.Multiname:
return (&this)[0..1];
case Type.Group:
return [ContextItem(groupFallback)];
}
auto expanded = expand(refs, *pcontext, abortOnUnknown);
if (expanded is null) return null;
return expanded ~ (multiname.vQName.name.length ? [ContextItem(multiname.vQName.name)] : null); // hack
assert(!expanding);
expanding = true;
scope(exit) expanding = false;
final switch (type)
{
case Type.Multiname:
switch (multiname.kind)
{
case ASType.QName:
{
auto ns = multiname.vQName.ns;
if (ns.kind == ASType.PrivateNamespace)
{
// auto pcontext = ns.id in refs.namespaces[ns.kind].contexts;
// if (pcontext is null)
// return (&this)[0..1];
// assert(pcontext);
auto context = refs.namespaces[ns.kind].getContext(refs, ns.id);
debug(CONTEXTS) std.stdio.writefln("Context of namespace %s is:\n\t%s\n", ns, context);
// auto expanded = expand(refs, context);
// if (expanded is null) return null;
return /*expanded*/context ~ (multiname.vQName.name.length ? [ContextItem(multiname.vQName.name)] : null); // hack
}
break;
}
case ASType.Multiname:
return multiname.vMultiname.name.length ? [ContextItem(multiname.vMultiname.name)] : null;
default:
debug
assert(false, text(multiname.kind));
else
break;
}
break;
case Type.String:
break;
case Type.Group:
return reduceGroup(refs);
}
else
return (&this)[0..1];
return (&this)[0..1];
}
this(ASProgram.Multiname m)
@@ -196,6 +307,13 @@ final class RefBuilder : ASTraitsVisitor
this.filenameSuffix = filenameSuffix;
}
this(ContextItem[] group, string groupFallback)
{
this.type = Type.Group;
this.group = group;
this.groupFallback = groupFallback;
}
mixin AutoCompare;
mixin AutoToString;
@@ -205,13 +323,17 @@ final class RefBuilder : ASTraitsVisitor
mixin(addAutoField("type"));
final switch (type)
{
case Type.Multiname:
mixin(addAutoField("multiname"));
break;
case Type.String:
mixin(addAutoField("str"));
mixin(addAutoField("filenameSuffix"));
break;
case Type.Multiname:
mixin(addAutoField("multiname"));
break;
case Type.String:
mixin(addAutoField("str"));
mixin(addAutoField("filenameSuffix"));
break;
case Type.Group:
mixin(addAutoField("group"));
mixin(addAutoField("groupFallback"));
break;
}
mixin(epilog);
}
@@ -219,10 +341,17 @@ final class RefBuilder : ASTraitsVisitor
static bool similar(ref ContextItem i1, ref ContextItem i2)
{
if (i1.type != i2.type) return false;
if (i1.type == ContextItem.Type.String)
return i1.str == i2.str;
if (i1.multiname.vQName.name != i2.multiname.vQName.name) return false;
return nsSimilar(i1.multiname.vQName.ns, i2.multiname.vQName.ns);
final switch (i1.type)
{
case ContextItem.Type.String:
return i1.str == i2.str;
case ContextItem.Type.Multiname:
assert(i1.multiname.kind == ASType.QName && i2.multiname.kind == ASType.QName);
if (i1.multiname.vQName.name != i2.multiname.vQName.name) return false;
return nsSimilar(i1.multiname.vQName.ns, i2.multiname.vQName.ns);
case ContextItem.Type.Group:
return i1.group == i2.group;
}
}
// truncate=true -> return partial ContextItem
@@ -237,8 +366,10 @@ final class RefBuilder : ASTraitsVisitor
if (c1.multiname.kind != ASType.QName || c2.multiname.kind != ASType.QName)
return null;
auto ns1 = c1.multiname.vQName.ns;
auto ns2 = c2.multiname.vQName.ns;
auto name1 = c1.multiname.vQName.name;
auto name2 = c2.multiname.vQName.name;
auto ns1 = c1.multiname.vQName.ns;
auto ns2 = c2.multiname.vQName.ns;
if (nsSimilar(ns1, ns2) && ns1.name.length && truncate)
{
@@ -248,13 +379,14 @@ final class RefBuilder : ASTraitsVisitor
return [ContextItem(m)];
}
if (c1.multiname.vQName.name && !c2.multiname.vQName.name && truncate)
if (name1 && !name2 && truncate)
{
swap(c1, c2);
swap(ns1, ns2);
swap(name1, name2);
}
if (!c1.multiname.vQName.name && c2.multiname.vQName.name && nsSimilar(ns1, ns2))
if (!name1 && name2 && nsSimilar(ns1, ns2))
{
if (truncate)
{
@@ -269,14 +401,29 @@ final class RefBuilder : ASTraitsVisitor
if (ns1.name.length && ns2.name.length)
{
if (nsSimilar(ns1, ns2))
{
assert(name1 != name2); // handled by similar() check
static if (truncate)
assert(false); // handled above
else
{
if (name2.length)
return [c1, ContextItem(name2)];
else
return [c1];
}
}
if (ns1.name.length > ns2.name.length && truncate)
{
swap(c1, c2);
swap(ns1, ns2);
swap(name1, name2);
}
auto fullName1 = ns1.name ~ (c1.multiname.vQName.name ? ':' ~ c1.multiname.vQName.name : "");
auto fullName2 = ns2.name ~ (c2.multiname.vQName.name ? ':' ~ c2.multiname.vQName.name : "");
auto fullName1 = ns1.name ~ (name1 ? ':' ~ name1 : "");
auto fullName2 = ns2.name ~ (name2 ? ':' ~ name2 : "");
if (fullName2.startsWith(fullName1 ~ ":"))
return [truncate ? c1 : c2];
}
@@ -293,39 +440,64 @@ final class RefBuilder : ASTraitsVisitor
void pushContext(T...)(T v) { context ~= ContextItem(v); }
void popContext() { context = context[0..$-1]; }
enum ContextPriority
{
declaration,
usage,
orphan,
max
}
struct ContextSet(T, bool ALLOW_DUPLICATES)
{
ContextItem[][T] contexts;
string[T] names, filenames;
ContextItem[][][ContextPriority.max][T] contextSets;
debug bool contextsSealed;
void add(U)(U obj, ContextItem[] context)
string[T] names, filenames;
debug bool coagulated;
bool add(U)(U obj, ContextItem[] context, ContextPriority priority)
{
debug assert(!coagulated);
debug assert(!contextsSealed);
auto p = cast(T)obj;
static if (ALLOW_DUPLICATES)
auto pset = p in contextSets;
if (!pset)
{
auto pexisting = p in contexts;
if (pexisting)
{
auto rootContext = contextRoot(*pexisting, context);
contexts[p] = rootContext;
}
else
contexts[p] = context;
contextSets[p] = contextSets[p].init;
contextSets[p][priority] ~= context.dup;
return true;
}
else
{
if (p in contexts)
contexts[p] = [ContextItem("multireferenced")];
else
contexts[p] = context.dup;
if ((*pset)[priority].length==0 || (*pset)[priority][$-1] != context) // Optimization: don't add contexts identical to the last added
(*pset)[priority] ~= context.dup;
return false;
}
}
bool addIfNew(U)(U obj, ContextItem[] context, ContextPriority priority)
{
if (isAdded(obj))
return false;
else
return add(obj, context, priority);
}
void coagulate(RefBuilder refs)
{
debug assert(!coagulated);
int[string] collisionCounter;
T[string] first;
foreach (obj, set; contextSets)
if (obj !in contexts)
getContext(refs, obj);
foreach (obj; contexts.keys.sort)
{
auto context = contexts[obj];
@@ -349,14 +521,49 @@ final class RefBuilder : ASTraitsVisitor
filenames[obj] = bfilename ~ suffix;
collisionCounter[bname] = counter+1;
}
debug coagulated = true;
}
bool isAdded(U)(U obj) { return (cast(T)obj in contexts) ? true : false; }
ContextItem[] getContext(U)(U obj) { return contexts[cast(T)obj]; }
bool isAdded(U)(U obj) { return (cast(T)obj in contextSets) ? true : false; }
ContextItem[] getContext(U)(RefBuilder refs, U obj)
{
// debug assert(coagulated);
debug contextsSealed = true;
auto p = cast(T)obj;
auto pcontext = p in contexts;
if (pcontext)
return *pcontext;
ContextItem[][] set;
foreach (prioritySet; contextSets[p])
if (prioritySet)
set = prioritySet;
static if (ALLOW_DUPLICATES)
{
auto context = ContextItem.expand(refs, set[0]);
foreach (setContext; set[1..$])
context = contextRoot(context, ContextItem.expand(refs, setContext));
return contexts[p] = context;
}
else
{
if (set.length > 1)
return contexts[p] = [ContextItem("multireferenced")];
else
return contexts[p] = ContextItem.expand(refs, set[0]);
}
}
string getName(U)(U obj)
{
debug assert(coagulated);
auto pname = cast(T)obj in names;
// return pname?*pname:"##" ~ format(obj);
assert(pname, format("Unscanned object: ", obj));
return *pname;
}
@@ -365,6 +572,8 @@ final class RefBuilder : ASTraitsVisitor
string getFilename(U)(U obj, string suffix)
{
debug assert(coagulated);
auto pname = cast(T)obj in filenames;
assert(pname, format("Unscanned object: ", obj));
auto filename = *pname;
@@ -389,7 +598,7 @@ final class RefBuilder : ASTraitsVisitor
}
}
ContextSet!(uint, true) privateNamespaces;
ContextSet!(uint, true)[ASType.Max] namespaces;
ContextSet!(void*, false) objects, scripts;
this(ASProgram as)
@@ -409,94 +618,121 @@ final class RefBuilder : ASTraitsVisitor
addOrphan(method);
super.run();
foreach (i, v; as.scripts)
{
ContextItem[][] classContexts;
ContextItem[] classContexts;
foreach (trait; v.traits)
{
auto c = ContextItem.expand(this, [ContextItem(trait.name)], true);
if (c)
classContexts ~= c;
if (trait.name.kind == ASType.QName && trait.name.vQName.ns.kind != ASType.PrivateNamespace)
classContexts ~= ContextItem(trait.name);
}
if (classContexts.length)
context = reduce!contextRoot(classContexts);
else
context = null;
if (context.length==0)
pushContext("script_" ~ to!string(i));
scripts.add(v, context);
pushContext("sinit", true);
addMethod(v.sinit);
if (!classContexts.length)
foreach (trait; v.traits)
classContexts ~= ContextItem(trait.name);
context = [ContextItem(classContexts, "script_" ~ to!string(i))];
scripts.add(v, context, ContextPriority.declaration);
pushContext("init", true);
addMethod(v.sinit, ContextPriority.declaration);
context = null;
}
foreach (i, vclass; as.orphanClasses)
if (!objects.isAdded(vclass))
{
pushContext("orphan_class_" ~ to!string(i));
addClass(vclass);
addClass(vclass, ContextPriority.orphan);
popContext();
}
foreach (i, method; as.orphanMethods)
if (!objects.isAdded(method))
{
pushContext("orphan_method_" ~ to!string(i));
addMethod(method);
addMethod(method, ContextPriority.orphan);
popContext();
}
// add private namespaces referenced only at script level
scripts.coagulate(this);
// add namespaces referenced only at script level
foreach (v; as.scripts)
foreach (trait; v.traits)
if (trait.name.kind == ASType.QName && trait.name.vQName.ns.kind == ASType.PrivateNamespace && !privateNamespaces.isAdded(trait.name.vQName.ns.privateIndex))
privateNamespaces.add(trait.name.vQName.ns.privateIndex, scripts.getContext(v));
if (trait.name.kind == ASType.QName)
namespaces[trait.name.vQName.ns.kind].addIfNew(trait.name.vQName.ns.id, scripts.getContext(this, v), ContextPriority.declaration);
foreach (privateIndex, b; possibleOrphanPrivateNamespaces)
if (!privateNamespaces.isAdded(privateIndex))
foreach (id, b; possibleOrphanPrivateNamespaces)
if (!namespaces[ASType.PrivateNamespace].isAdded(id))
{
pushContext("private_namespace_" ~ to!string(privateIndex));
privateNamespaces.add(privateIndex, context);
pushContext("orphan_namespace_" ~ to!string(id));
namespaces[ASType.PrivateNamespace].add(id, context, ContextPriority.orphan);
popContext();
}
privateNamespaces.coagulate(this);
debug homonymsBuilt = true;
foreach (ref namespace; namespaces)
namespace.coagulate(this);
objects.coagulate(this);
scripts.coagulate(this);
}
override void visitTrait(ref ASProgram.Trait trait)
{
auto m = trait.name;
if (m.kind != ASType.QName)
throw new Exception("Trait name is not a QName");
// if (m.kind != ASType.QName)
// throw new Exception("Trait name is not a QName");
pushContext(m);
visitMultiname(m);
visitMultiname(m, ContextPriority.declaration);
switch (trait.kind)
{
case TraitKind.Slot:
case TraitKind.Const:
visitMultiname(trait.vSlot.typeName, ContextPriority.usage);
super.visitTrait(trait);
break;
case TraitKind.Class:
addClass(trait.vClass.vclass);
addClass(trait.vClass.vclass, ContextPriority.declaration);
pushContext("class", true);
visitTraits(trait.vClass.vclass.traits);
popContext();
pushContext("instance", true);
visitTraits(trait.vClass.vclass.instance.traits);
popContext();
break;
case TraitKind.Function:
addMethod(trait.vFunction.vfunction);
addMethod(trait.vFunction.vfunction, ContextPriority.declaration);
super.visitTrait(trait);
break;
case TraitKind.Method:
addMethod(trait.vMethod.vmethod);
addMethod(trait.vMethod.vmethod, ContextPriority.declaration);
super.visitTrait(trait);
break;
case TraitKind.Getter:
pushContext("getter");
addMethod(trait.vMethod.vmethod);
addMethod(trait.vMethod.vmethod, ContextPriority.declaration);
popContext();
super.visitTrait(trait);
break;
case TraitKind.Setter:
pushContext("setter");
addMethod(trait.vMethod.vmethod);
addMethod(trait.vMethod.vmethod, ContextPriority.declaration);
popContext();
super.visitTrait(trait);
break;
default:
super.visitTrait(trait);
break;
}
super.visitTrait(trait);
popContext();
}
@@ -504,6 +740,10 @@ final class RefBuilder : ASTraitsVisitor
{
static bool uninteresting(ContextItem[] c)
{
// Do some validation while we're at it
foreach (cc; c)
assert(cc.type != ContextItem.Type.Group, "Groups should be expanded by now");
return
(c.length==1 && c[0].type==ContextItem.Type.String && c[0].str.startsWith("script_") && c[0].str.endsWith("_sinit")) ||
(c.length==1 && c[0].type==ContextItem.Type.String && c[0].str.startsWith("orphan_method_")) ||
@@ -528,39 +768,42 @@ final class RefBuilder : ASTraitsVisitor
bool[uint] possibleOrphanPrivateNamespaces;
void visitNamespace(ASProgram.Namespace ns)
void visitNamespace(ASProgram.Namespace ns, ContextPriority priority)
{
if (ns is null) return;
if (ns.kind == ASType.PrivateNamespace)
{
assert(context.length > 0, "No context");
//assert(ns.name is null, "Named private namespace");
auto myPos = context.length;
foreach (i, ref item; context)
if (item.type == ContextItem.Type.Multiname && item.multiname.vQName.ns == ns)
{
myPos = i;
break;
}
if (myPos == 0)
// Add to homonyms
addHomonym(ns);
// Add to namespaces (for context)
assert(context.length > 0, "No context");
auto myPos = context.length;
foreach (i, ref item; context)
if (item.type == ContextItem.Type.Multiname && item.multiname.kind == ASType.QName && item.multiname.vQName.ns == ns)
{
possibleOrphanPrivateNamespaces[ns.privateIndex] = true;
return;
myPos = i;
break;
}
auto myContext = context[0..myPos].dup;
privateNamespaces.add(ns.privateIndex, myContext);
if (ns.kind == ASType.PrivateNamespace && myPos == 0)
{
possibleOrphanPrivateNamespaces[ns.id] = true;
return;
}
auto myContext = context[0..myPos].dup;
namespaces[ns.kind].add(ns.id, myContext, priority);
}
void visitNamespaceSet(ASProgram.Namespace[] nsSet)
void visitNamespaceSet(ASProgram.Namespace[] nsSet, ContextPriority priority)
{
foreach (ns; nsSet)
visitNamespace(ns);
visitNamespace(ns, priority);
}
void visitMultiname(ASProgram.Multiname m)
void visitMultiname(ASProgram.Multiname m, ContextPriority priority)
{
if (m is null) return;
with (m)
@@ -568,20 +811,20 @@ final class RefBuilder : ASTraitsVisitor
{
case ASType.QName:
case ASType.QNameA:
visitNamespace(vQName.ns);
visitNamespace(vQName.ns, priority);
break;
case ASType.Multiname:
case ASType.MultinameA:
visitNamespaceSet(vMultiname.nsSet);
visitNamespaceSet(vMultiname.nsSet, priority);
break;
case ASType.MultinameL:
case ASType.MultinameLA:
visitNamespaceSet(vMultinameL.nsSet);
visitNamespaceSet(vMultinameL.nsSet, priority);
break;
case ASType.TypeName:
visitMultiname(vTypeName.name);
visitMultiname(vTypeName.name, priority);
foreach (param; vTypeName.params)
visitMultiname(param);
visitMultiname(param, priority);
break;
default:
break;
@@ -595,21 +838,21 @@ final class RefBuilder : ASTraitsVisitor
switch (type)
{
case OpcodeArgumentType.Namespace:
visitNamespace(instruction.arguments[i].namespacev);
visitNamespace(instruction.arguments[i].namespacev, ContextPriority.usage);
break;
case OpcodeArgumentType.Multiname:
visitMultiname(instruction.arguments[i].multinamev);
visitMultiname(instruction.arguments[i].multinamev, ContextPriority.usage);
break;
case OpcodeArgumentType.Class:
pushContext("inline_class");
if (isOrphan(instruction.arguments[i].classv))
addClass(instruction.arguments[i].classv);
addClass(instruction.arguments[i].classv, ContextPriority.usage);
popContext();
break;
case OpcodeArgumentType.Method:
pushContext("inline_method");
if (isOrphan(instruction.arguments[i].methodv))
addMethod(instruction.arguments[i].methodv);
addMethod(instruction.arguments[i].methodv, ContextPriority.usage);
popContext();
break;
default:
@@ -618,8 +861,17 @@ final class RefBuilder : ASTraitsVisitor
}
string contextToString(ContextItem[] context, bool filename)
in
{
context = ContextItem.expand(this, context, false);
debug(CONTEXTS) std.stdio.writefln("Stringizing:\n\t%s\n", context);
}
out(result)
{
debug(CONTEXTS) std.stdio.writefln("Stringized:\n\t%s\nto\n\t%s\n------------------\n", context, result);
}
body
{
context = ContextItem.expand(this, context);
if (!context.length)
return null;
@@ -632,7 +884,7 @@ final class RefBuilder : ASTraitsVisitor
ContextItem.Segment[] segments;
foreach (ci; context)
segments ~= ci.toSegments(filename);
segments ~= ci.toSegments(this, filename);
string escape(string s)
{
@@ -660,14 +912,14 @@ final class RefBuilder : ASTraitsVisitor
static const reservedNames = ["CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9"];
auto pathSegmentU = pathSegment.toUpper();
foreach (reservedName; reservedNames)
if (pathSegmentU.startsWith(reservedName))
if (pathSegmentU.startsWith(reservedName) && (pathSegmentU.length == reservedName.length || pathSegmentU[reservedName.length]=='.'))
{
pathSegment = "%" ~ pathSegment;
break;
}
if (pathSegment.length > 240)
pathSegment = pathSegment[0..200] ~ '-' ~ getDigestString(pathSegment);
pathSegment = assumeUnique(pathSegment[0..200] ~ '-' ~ toHexString(md5Of(pathSegment)));
}
return arrayJoin(pathSegments, "/");
@@ -680,33 +932,41 @@ final class RefBuilder : ASTraitsVisitor
return arrayJoin(strings);
}
void addObject(T)(T obj) { objects.add(obj, context); }
bool addObject(T)(T obj, ContextPriority priority) { return objects.add(obj, context, priority); }
void addClass(ASProgram.Class vclass)
void addClass(ASProgram.Class vclass, ContextPriority priority)
{
addObject(vclass);
addObject(vclass, priority);
pushContext("cinit");
addMethod(vclass.cinit);
popContext();
pushContext("class", true);
pushContext("init", true);
addMethod(vclass.cinit, ContextPriority.declaration);
popContext(); // init
popContext(); // class
pushContext("iinit");
addMethod(vclass.instance.iinit);
popContext();
pushContext("instance", true);
pushContext("init", true);
addMethod(vclass.instance.iinit, ContextPriority.declaration);
popContext(); // init
pushContext("instance");
visitMultiname(vclass.instance.name);
visitMultiname(vclass.instance.superName);
visitMultiname(vclass.instance.name, ContextPriority.declaration);
visitMultiname(vclass.instance.superName, ContextPriority.usage);
visitNamespace(vclass.instance.protectedNs, ContextPriority.declaration);
foreach (iface; vclass.instance.interfaces)
visitMultiname(iface);
popContext();
visitMultiname(iface, ContextPriority.usage);
popContext(); // instance
}
void addMethod(ASProgram.Method method)
void addMethod(ASProgram.Method method, ContextPriority priority)
{
addObject(method);
if (method.vbody)
visitMethodBody(method.vbody);
if (addObject(method, priority))
{
foreach (paramType; method.paramTypes)
visitMultiname(paramType, ContextPriority.usage);
visitMultiname(method.returnType, ContextPriority.usage);
if (method.vbody)
visitMethodBody(method.vbody);
}
}
}
@@ -753,10 +1013,7 @@ final class Disassembler
StringBuilder sb = new StringBuilder(dir ~ "/" ~ name ~ ".main.asasm");
sb ~= "#version 2";
sb.newLine();
sb ~= "#include ";
dumpString(sb, name ~ ".privatens.asasm");
sb ~= "#version 3";
sb.newLine();
sb ~= "program";
@@ -772,8 +1029,9 @@ final class Disassembler
foreach (uint i, script; as.scripts)
{
auto ii = i; // http://d.puremagic.com/issues/show_bug.cgi?id=8526
newInclude(sb, refs.scripts.getFilename(script, "script"), (StringBuilder sb) {
dumpScript(sb, script, i);
dumpScript(sb, script, ii);
});
}
sb.newLine();
@@ -810,39 +1068,6 @@ final class Disassembler
sb ~= "end ; program"; sb.newLine();
sb.save();
// now dump the private namespace indices
sb = new StringBuilder(dir ~ "/" ~ name ~ ".privatens.asasm");
uint[] indices = refs.privateNamespaces.names.keys;
bool alphaSortDelegate(uint a, uint b) { return refs.privateNamespaces.names[a] < refs.privateNamespaces.names[b]; }
sort!alphaSortDelegate(indices);
foreach (index; indices)
{
void dumpContext(RefBuilder.ContextItem[] context)
{
sb ~= "; ";
foreach (i, c; context)
{
if (c.type == RefBuilder.ContextItem.Type.Multiname)
dumpMultiname(sb, c.multiname);
else
sb ~= c.str;
if (i < context.length-1)
sb ~= " -> ";
}
sb.newLine();
}
auto context = refs.privateNamespaces.contexts[index];
dumpContext(context);
/*auto contextEx = refs.ContextItem.expand(refs, context);
if (context != contextEx)
dumpContext(contextEx);*/
sb ~= format("#privatens %4d ", index);
dumpString(sb, refs.privateNamespaces.names[index]);
sb.newLine();
}
sb.save();
}
void dumpInt(StringBuilder sb, long v)
@@ -921,15 +1146,16 @@ final class Disassembler
if (namespace is null)
sb ~= "null";
else
with (namespace)
{
sb ~= ASTypeNames[kind];
sb ~= ASTypeNames[namespace.kind];
sb ~= '(';
dumpString(sb, name);
if (kind == ASType.PrivateNamespace)
dumpString(sb, namespace.name);
if (refs.hasHomonyms(namespace))
{
sb ~= ", ";
dumpString(sb, refs.privateNamespaces.getName(privateIndex));
auto label = refs.namespaces[namespace.kind].getName(namespace.id);
// label ~= format(" (%d)", namespace.id);
dumpString(sb, label);
}
sb ~= ')';
}
@@ -1170,6 +1396,7 @@ final class Disassembler
void dumpMethod(StringBuilder sb, ASProgram.Method method, string label)
{
sb ~= label;
// sb ~= format(" ; %d", method.id);
sb.indent++; sb.newLine();
if (method.name !is null)
{
@@ -1474,7 +1701,7 @@ private:
bool nsSimilar(ASProgram.Namespace ns1, ASProgram.Namespace ns2)
{
if (ns1.kind==ASType.PrivateNamespace || ns2.kind==ASType.PrivateNamespace)
return ns1.kind==ns2.kind && ns1.privateIndex==ns2.privateIndex;
return ns1.kind==ns2.kind && ns1.id==ns2.id;
// ignore ns kind in other cases
return ns1.name == ns2.name;
}
+33 -17
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -29,9 +29,9 @@ version (Windows)
else
{ pragma(lib, "lzma"); }
align(1)
struct LZMAHeader
align(1) struct LZMAHeader
{
align(1):
ubyte compressionParameters;
uint dictionarySize;
long decompressedSize = -1;
@@ -40,9 +40,9 @@ static assert(LZMAHeader.sizeof == 13);
ubyte[] lzmaDecompress(LZMAHeader header, in ubyte[] compressedData)
{
enforce(header.decompressedSize > 0, "Decompression with unknown size is unsupported");
enforce(header.decompressedSize > 0, "Decompression with unknown size is unsupported");
lzma_stream strm;
lzma_stream strm;
lzmaEnforce(lzma_alone_decoder(&strm, ulong.max), "lzma_alone_decoder");
scope(exit) lzma_end(&strm);
@@ -54,15 +54,16 @@ ubyte[] lzmaDecompress(LZMAHeader header, in ubyte[] compressedData)
{
strm.next_in = chunk.ptr;
strm.avail_in = chunk.length;
lzmaEnforce(lzma_code(&strm, lzma_action.LZMA_RUN), "lzma_code");
lzmaEnforce!true(lzma_code(&strm, lzma_action.LZMA_RUN), "lzma_code (LZMA_RUN)");
enforce(strm.avail_in == 0, "Not all data was read");
}
header.decompressedSize = -1; // Required as Flash uses End-of-Stream marker
fixDictSize(header.dictionarySize);
decompress(cast(ubyte[])(&header)[0..1]);
decompress(compressedData);
lzmaEnforce(lzma_code(&strm, lzma_action.LZMA_FINISH), "lzma_code");
lzmaEnforce!true(lzma_code(&strm, lzma_action.LZMA_FINISH), "lzma_code (LZMA_FINISH)");
enforce(strm.avail_out == 0, "Decompressed size mismatch");
@@ -71,30 +72,45 @@ ubyte[] lzmaDecompress(LZMAHeader header, in ubyte[] compressedData)
ubyte[] lzmaCompress(in ubyte[] decompressedData, LZMAHeader* header)
{
lzma_options_lzma opts;
enforce(lzma_lzma_preset(&opts, 9 | LZMA_PRESET_EXTREME) == false, "lzma_lzma_preset error");
lzma_options_lzma opts;
enforce(lzma_lzma_preset(&opts, 9 | LZMA_PRESET_EXTREME) == false, "lzma_lzma_preset error");
lzma_stream strm;
lzma_stream strm;
lzmaEnforce(lzma_alone_encoder(&strm, &opts), "lzma_alone_encoder");
scope(exit) lzma_end(&strm);
auto outBuf = new ubyte[decompressedData.length];
auto outBuf = new ubyte[decompressedData.length + 1024];
strm.next_out = outBuf.ptr;
strm.avail_out = outBuf.length;
strm.next_in = decompressedData.ptr;
strm.avail_in = decompressedData.length;
lzmaEnforce(lzma_code(&strm, lzma_action.LZMA_RUN), "lzma_code");
scope(failure) { import std.stdio; writeln("avail_in=", strm.avail_in); }
lzmaEnforce(lzma_code(&strm, lzma_action.LZMA_RUN), "lzma_code (LZMA_RUN)");
enforce(strm.avail_in == 0, "Not all data was read");
enforce(strm.avail_out != 0, "Ran out of compression space");
lzmaEnforce(lzma_code(&strm, lzma_action.LZMA_FINISH), "lzma_code");
lzmaEnforce!true(lzma_code(&strm, lzma_action.LZMA_FINISH), "lzma_code (LZMA_FINISH)");
*header = *cast(LZMAHeader*)outBuf.ptr;
return outBuf[LZMAHeader.sizeof..to!size_t(strm.total_out)];
}
private void lzmaEnforce(lzma_ret v, string f)
private void lzmaEnforce(bool STREAM_END_OK=false)(lzma_ret v, string f)
{
if (v != lzma_ret.LZMA_OK && v != lzma_ret.LZMA_STREAM_END)
throw new Exception(text(f, " error: ", v));
if (v != lzma_ret.LZMA_OK && (!STREAM_END_OK || v != lzma_ret.LZMA_STREAM_END))
throw new Exception(text(f, " error: ", v));
}
/// Work around an artificial lzma_alone_decoder limitation in liblzma
/// which prevents it from accepting any streams with a dictionary size
/// that is not 2^n or 2^n + 2^(n-1).
/// See xz\src\liblzma\common\alone_decoder.c (git commit e7b424d267), line 87
private void fixDictSize(ref uint d)
{
--d;
d |= d >> 2;
d |= d >> 3;
d |= d >> 4;
d |= d >> 8;
d |= d >> 16;
++d;
}
+5 -8
View File
@@ -20,12 +20,12 @@ import std.conv;
struct MurmurHash2A
{
private static string mmix(string h, string k) pure { return "{ "~k~" *= m; "~k~" ^= "~k~" >> r; "~k~" *= m; "~h~" *= m; "~h~" ^= "~k~"; }"; }
private static string mmix(string h, string k) { return "{ "~k~" *= m; "~k~" ^= "~k~" >> r; "~k~" *= m; "~h~" *= m; "~h~" ^= "~k~"; }"; }
public:
void Begin ( uint seed = 0 ) pure nothrow
void Begin ( uint seed = 0 )
{
m_hash = seed;
m_tail = 0;
@@ -33,11 +33,8 @@ public:
m_size = 0;
}
void Add ( const(void) * vdata, size_t len_s ) pure nothrow
void Add ( const(void) * vdata, int len )
{
assert(len_s <= int.min);
int len = cast(int)len_s;
ubyte * data = cast(ubyte*)vdata;
m_size += len;
@@ -56,7 +53,7 @@ public:
MixTail(data,len);
}
uint End ( ) pure nothrow
uint End ( )
{
mixin(mmix("m_hash","m_tail"));
mixin(mmix("m_hash","m_size"));
@@ -80,7 +77,7 @@ private:
static const uint m = 0x5bd1e995;
static const int r = 24;
void MixTail ( ref ubyte * data, ref int len ) pure nothrow
void MixTail ( ref ubyte * data, ref int len )
{
while( len && ((len<4) || m_count) )
{
+2 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -33,6 +33,7 @@ void main(string[] args)
{
auto bin = cast(ubyte[])read(args[3]);
tag.data = tag.data[0..6] ~ bin;
tag.length = cast(uint)tag.data.length;
write(args[1], swf.write());
return;
}
+11 -5
View File
@@ -39,6 +39,7 @@ final class SWFFile
align(1) struct Header
{
align(1):
char[3] signature;
ubyte ver;
uint fileLength;
@@ -47,6 +48,7 @@ final class SWFFile
align(1) struct LZMAHeader
{
align(1):
uint compressedLength;
ubyte compressionParameters;
uint dictionarySize;
@@ -63,6 +65,7 @@ final class SWFFile
{
ushort type;
ubyte[] data;
uint length; // may be >data.length if file is truncated
bool forceLongLength;
}
@@ -127,10 +130,12 @@ private final class SWFReader
pos += raw.length;
}
/// May read less than len on EOF
void[] readRaw(size_t len)
{
auto data = buf[pos..pos+len];
pos += len;
auto end = pos+len;
auto data = buf[pos..end<$?end:$];
pos = end;
return data;
}
@@ -172,6 +177,7 @@ private final class SWFReader
if (length < 0x3F)
t.forceLongLength = true;
}
t.length = length;
t.data = cast(ubyte[])readRaw(length);
return t;
}
@@ -261,16 +267,16 @@ private final class SWFWriter
foreach (ref tag; swf.tags)
{
ushort u = cast(ushort)(tag.type << 6);
if (tag.data.length < 0x3F && !tag.forceLongLength)
if (tag.length < 0x3F && !tag.forceLongLength)
{
u |= tag.data.length;
u |= tag.length;
buf ~= toArray(u);
}
else
{
u |= 0x3F;
buf ~= toArray(u);
uint l = to!uint(tag.data.length);
uint l = to!uint(tag.length);
buf ~= toArray(l);
}
buf ~= tag.data;
+35 -3
View File
@@ -1,5 +1,5 @@
/*
* Copyright 2010, 2011, 2012 Vladimir Panteleev <vladimir@thecybershadow.net>
* Copyright 2010, 2011, 2012, 2013 Vladimir Panteleev <vladimir@thecybershadow.net>
* This file is part of RABCDAsm.
*
* RABCDAsm is free software: you can redistribute it and/or modify
@@ -18,18 +18,50 @@
module swflzmacompress;
import std.exception;
import std.file;
import std.getopt;
import std.string;
import swffile;
void main(string[] args)
{
bool force, updateVersion;
getopt(args,
"--force", &force,
"--update-version", &updateVersion,
);
if (args.length == 1)
throw new Exception("No file specified");
enum MIN_LZMA_VER = 13;
foreach (arg; args[1..$])
{
auto swf = SWFFile.read(cast(ubyte[])read(arg));
if (swf.header.signature[0] == cast(ubyte)'Z')
throw new Exception("Already LZMA-compressed");
enforce(swf.header.signature[0] != cast(ubyte)'Z', "Already LZMA-compressed");
if (swf.header.ver < MIN_LZMA_VER)
{
if (updateVersion)
{
if (swf.header.ver < 8 && !force)
throw new Exception(format(
"SWF version %d has different file format than version %d, " ~
"required for LZMA. Resulting file may not work. " ~
"Use --force to override and update version anyway.",
swf.header.ver, MIN_LZMA_VER
));
swf.header.ver = MIN_LZMA_VER;
}
else
if (!force)
throw new Exception(format(
"SWF version %d is too old to support SWF LZMA compression, " ~
"which requires version %d. " ~
"Use --update-version to update the version number, " ~
"or --force to compress anyway without updating it.",
swf.header.ver, MIN_LZMA_VER
));
}
swf.header.signature[0] = cast(ubyte)'Z'; // LZMA
write(arg, swf.write());
}