24 Commits
1.0 ... d1
Author SHA1 Message Date
Vladimir Panteleev 2e48e8deef Update Changelog 2011-02-20 20:44:31 +02:00
Vladimir Panteleev f49fe820b6 Whitespace 2011-02-20 20:44:05 +02:00
Vladimir Panteleev 52a2cde1b0 Finish metadata support 2011-02-17 06:48:40 +02:00
Vladimir Panteleev 608a8ca51d Correctly order classes by dependencies (extends/implements) and reference count 2011-02-17 05:25:49 +02:00
Vladimir Panteleev d11270a553 A bit of debugging code 2011-02-17 05:25:01 +02:00
Vladimir Panteleev 68a0845c53 Support forward references of typenames 2011-02-17 04:31:04 +02:00
Vladimir Panteleev 824a945247 asprogram: Refactor constant/reference pool code 2011-02-17 02:52:00 +02:00
Vladimir Panteleev c89f2bbe70 Documentation: update link to AVM2 reference PDF 2010-12-20 19:28:49 +02:00
Vladimir Panteleev e2dbee2e2a Add Changelog 2010-11-11 23:42:54 +02:00
Vladimir Panteleev 66bccc6a6f Disassemble doubles with adequate precision 2010-11-11 23:25:00 +02:00
Vladimir Panteleev 22194e5baf Documentation: Markdown fixes 2010-11-09 03:41:21 +02:00
Vladimir Panteleev b95f0ece09 Generate refs for orphan objects properly 2010-11-06 00:54:20 +02:00
Vladimir Panteleev b691796c4e Better error handling; warn when no DoABC tags found 2010-09-03 18:00:57 +03:00
Vladimir Panteleev 54349c3a55 Documentation: clarify abcreplace usage 2010-07-22 19:14:35 +03:00
Vladimir Panteleev f0302dcfca Documentation updates 2010-07-22 19:11:51 +03:00
Vladimir Panteleev fdc9246665 Add debug Flash Player tip 2010-06-30 16:58:09 +03:00
Vladimir Panteleev acbd85f4da Fix type conversion warning 2010-06-30 16:53:55 +03:00
Vladimir Panteleev bd8826c5df Add byte offset to labels
This gets rid of previous limitations of labels.
2010-06-30 16:52:34 +03:00
Vladimir Panteleev b54b1cada4 Fix relative include paths 2010-06-04 10:45:52 +03:00
Vladimir Panteleev 60684436a7 Documentation: work around bug in Github's Markdown parser 2010-06-02 21:05:30 +03:00
Vladimir Panteleev 26f3fb5b53 Use : to delimit namespace and name in QNames for consistency
(for consistency with protected namespace names)
2010-06-02 19:37:57 +03:00
Vladimir Panteleev c0a9e75c65 Private namespaces are now referenced by auto-generated names 2010-06-02 19:01:39 +03:00
Vladimir Panteleev 23c5d1ba6f Documentation: use unordered list for limitations 2010-06-02 18:57:46 +03:00
Vladimir Panteleev 9892caceab Document floating point precision limitation 2010-06-02 11:10:36 +03:00
9 changed files with 944 additions and 442 deletions
+45
View File
@@ -0,0 +1,45 @@
RABCDAsm Changelog
==================
RABCDAsm v1.4 (2011.02.xx)
--------------------------
* Add support for forward-references for TypeName-kind Multinames
* Correctly order classes by dependencies (extends/implements) and reference
count
* Finish Metadata support
* Documentation updates
RABCDAsm v1.3 (2010.11.11)
--------------------------
* Fixed double precision problem
* This also fixes problems with illegal default values for function
parameters (default values for integer parameters are stored as doubles,
which might become out-of-range due to inadequate double precision)
* Added Changelog
* Documentation markdown fixes
RABCDAsm v1.2 (2010.11.06)
--------------------------
* Fixed ref generation for orphan objects which were only referenced
by other orphans
* Better error handling in `abcexport`; warn when no DoABC tags found
* Documentation updates
RABCDAsm v1.1 (2010.06.30)
--------------------------
* Private namespaces are now referenced by auto-generated names
* Use `:` to delimit namespace and name in QNames for consistency
* Warning: this breaks compatibility with v1.0 disassemblies
* Fixed relative include paths
* Add optional byte offsets to labels, which allows lossless representation
of jumps inside instructions and outside the function bounds
* Documentation updates
RABCDAsm v1.0 (2010.05.05)
--------------------------
* Initial release.
+76 -81
View File
@@ -1,7 +1,7 @@
Robust ABC (ActionScript Bytecode) [Dis-]Assembler
==================================================
[RABCDAsm][] is a collection of utilities including an ActionScript
[RABCDAsm][] is a collection of utilities including an ActionScript 3
assembler/disassembler, and a few tools to manipulate SWF files.
These are:
@@ -31,7 +31,7 @@ Motivation and goals
--------------------
This package was created due to lack of similar software out there.
Particularly, I needed an utility which would allow me to edit ActionScript
Particularly, I needed an utility which would allow me to edit ActionScript 3
bytecode with the following properties:
1. Speed. Less waiting means more productivity. `rabcasm` can assemble large
@@ -94,13 +94,17 @@ To disassemble one of the `.abc` files:
rabcdasm file0.abc
This will create a `file0` directory, which will contain `file0.main.asasm`
(the main program file) and a file per ActionScript class.
(the main program file), `file0.privatens.asasm` (private namespace alias
definitions) and a file per ActionScript class.
To assemble the `.asasm` files back, and update the SWF file:
rabcasm file0/file0.main.asasm
abcreplace file0.swf 0 file0/file0.main.abc
The second `abcreplace` argument represents the index of the ABC block in the
SWF file, and corresponds to the number in the filename created by `abcexport`.
Syntax
======
@@ -121,7 +125,7 @@ use [this handy list][avm2i] as well). You will find it difficult to understand
the disassembly without good understanding of concepts such as namespaces and
multinames.
[avm2]: http://www.adobe.com/devnet/actionscript/articles/avm2overview.pdf
[avm2]: http://www.adobe.com/devnet-archive/actionscript/articles/avm2overview.pdf
[avm2i]: http://www.anotherbigidea.com/javaswf/avm2/AVM2Instructions.html
Overview
@@ -167,7 +171,10 @@ are the trait fields, varying by trait kind:
* `method` / `getter` / `setter` : `dispid` (unsigned integer), `method`
Additionally, all traits may have `flag` fields, describing the trait's
attributes (`FINAL` / `OVERRIDE` / `METADATA`).
attributes (`FINAL` / `OVERRIDE` / `METADATA`), and `metadata` blocks.
`metadata` blocks (which are ignored by the AVM) consist of a name string, and
a series of `item` fields - each item having a key and value string.
`class` blocks have mandatory `instance` and `cinit` fields, defining the class
instance and the class initializer method respectively. They may also have
@@ -194,10 +201,11 @@ fields.
`code` blocks - always declared inline of their `body` block - are somewhat
different in syntax from other blocks - mostly in that they may contain labels.
Labels follow the most common syntax - a word followed by a `:` character.
Multiple instruction arguments are comma-separated. Instruction arguments'
types depend on the instruction - see the `OpcodeInfo` array in `abcfile.d`
for a reference.
Labels follow the most common syntax - a word followed by a `:` character,
optionally followed by a relative byte offset (in case of pointers inside
instructions). Multiple instruction arguments are comma-separated. Instruction
arguments' types depend on the instruction - see the `OpcodeInfo` array in
`abcfile.d` for a reference.
`try` blocks - always declared inline of their `body` block - represent an
"exception" (try/catch) block. They contain five mandatory fields: `from`,
@@ -233,8 +241,12 @@ namespace sets can be specified using `[]`.
Namespaces have the syntax *type* `(` *parameters* `)` . For types other than
`PrivateNamespace` there is only one parameter - a string. `PrivateNamespace`
namespaces have a second parameter - an integer to distinguish this private
namespace from others.
namespaces have a second parameter, a named alias for a particular private
namespace. Internally (the ABC file format), private namespaces are
distinguished by a numerical index - `rabcdasm` will attempt to give them
descriptive names based on their context. Aliases can be defined using the
`#privatens` directive. `rabcdasm` will create a separate file containing the
aliases (`file0.privatens.asasm`).
Strings have a syntax similar to C string literals. Strings start and end with
a `"`. Supported escape sequences (a backslash followed by a letter) are `\n`
@@ -277,6 +289,7 @@ Directives start with a `#`, followed by a word identifying the directive:
* `#set` *word* *string* - assigns the contents of the string to the
variable *word*.
* `#unset` *word* - deletes the variable *word*.
* `#privatens` defines a private namespace alias, as described above.
### Variables
@@ -287,9 +300,9 @@ be instantiated in two ways:
although variables are defined using a string syntax, they are not
inserted as a string using this syntax. Thus, the code:
#set str "Hello, world!"
...
pushstring $str
#set str "Hello, world!"
...
pushstring $str
will expand to `pushstring Hello, world!`, which will result in an error.
To correct the problem, add escaped quotes around the variable contents
@@ -304,20 +317,20 @@ be instantiated in two ways:
Here's an example of how to use the above features to create a macro which
logs a string literal and the contents of a register:
#set log "
findpropstrict QName(PackageNamespace(\"\"), \"log\")
pushstring $\"1\"
getlocal $2
callpropvoid QName(PackageNamespace(\"\"), \"log\"), 2
"
; ...
pushbyte 2
pushbyte 2
add_i
setlocal1
#call $"log"("two plus two equals", "1")
#set log "
findpropstrict QName(PackageNamespace(\"\"), \"log\")
pushstring $\"1\"
getlocal $2
callpropvoid QName(PackageNamespace(\"\"), \"log\"), 2
"
; ...
pushbyte 2
pushbyte 2
add_i
setlocal1
#call $"log"("two plus two equals", "1")
Highlighting
------------
@@ -337,26 +350,23 @@ instead of indexes, allowing easy manipulation without having to worry about
record order or constant pools. Conversion between various states is done as
follows:
file.abc
| ^
ABCReader | | ABCWriter
v |
ABCFile
| ^
ABCtoAS | | AStoABC
v |
ASProgram
| ^
Disassembler | | Assembler
v |
file.asasm
file.abc
| ^
------ ABCReader | | ABCWriter ----
/ v | \
/ ABCFile \
/ | ^ \
rabcdasm---------- ABCtoAS | | AStoABC --------rabcasm
\ v | /
\ ASProgram /
\ | ^ /
--- Disassembler | | Assembler ----
v |
file.asasm
`AStoABC` will rebuild the constant pools, in a manner similar to Adobe's
compilers (reverse-sorted by reference count). The exact order will almost
surely be different, however. Also, some records (classes and methods) are
currently not sorted by reference count, which may cause `rabcasm` to generate
slightly larger files than the originals (due to variable-length encoding of
integers).
surely be different, however.
Should you need to write an utility to manipulate ABC, you can use the existing
code to load the file to either an `ABCFile` or `ASProgram` instance, and
@@ -376,24 +386,30 @@ RABCDAsm users.
[Git]: http://git-scm.com/
[Mercurial]: http://mercurial.selenic.com/
2. The [Fiddler][] Web Debugging Proxy can be very useful for analysing
2. If you plan on making non-trivial changes to SWF files, you should install
the [debug Flash Player][]. This will allow you to see validation and
run-time error messages, instead of simply getting a blank rectangle.
[debug Flash Player]: http://www.adobe.com/support/flashplayer/downloads.html
3. The [Fiddler][] Web Debugging Proxy can be very useful for analysing
websites with SWF content. The following script fragment (which is to be
placed in the `OnBeforeResponse` function) will automatically save all SWF
files while preserving the directory structure.
if (oSession.oResponse.headers.ExistsAndContains("Content-Type",
"application/x-shockwave-flash")) {
// Set desired path here
var path:String = "C:\\Temp\\FiddlerCapture\\" +
oSession.host + oSession.PathAndQuery;
if (path.Contains('?'))
path = path.Substring(0, path.IndexOf('?'));
var dir:String = Path.GetDirectoryName(path);
if (!Directory.Exists(dir))
Directory.CreateDirectory(dir);
oSession.utilDecodeResponse();
oSession.SaveResponseBody(path);
}
if (oSession.oResponse.headers.ExistsAndContains("Content-Type",
"application/x-shockwave-flash")) {
// Set desired path here
var path:String = "C:\\Temp\\FiddlerCapture\\" +
oSession.host + oSession.PathAndQuery;
if (path.Contains('?'))
path = path.Substring(0, path.IndexOf('?'));
var dir:String = Path.GetDirectoryName(path);
if (!Directory.Exists(dir))
Directory.CreateDirectory(dir);
oSession.utilDecodeResponse();
oSession.SaveResponseBody(path);
}
Once you have edited a SWF file, you can use Fiddler's [AutoResponder][] to
replace the original file with your modified version.
@@ -404,28 +420,7 @@ RABCDAsm users.
Limitations
===========
1. Metadata is currently ignored. I haven't noticed any metadata blocks in any
SWF files I've disassembled.
2. Private namespaces are currently represented by an automatically-assigned
integer. This causes problems when comparing disassemblies from two
versions of a file, since those numbers are prone to change when classes
are added or removed.
3. `rabcasm` may create a broken file due to not ordering classes by ancestry.
The problem originates from the fact that a class's ancestors (extended
class and implemented interfaces) are stored as multinames, and not as
class indices. (This makes sense, since classes may extend objects outside
the current ABC file.) Since `rabcasm` currently doesn't decode multinames,
it is unaware of the class dependencies, and may thus write the classes out
of order. This results in a file that, when opened, will fail to load with
an error message similar to:
VerifyError: Error #1014: Class AncestorClassName could not be found.
The simple work-around is to re-order the classes as they are declared in
the `.main.asasm` file, and place ancestors before descendants.
* None known.
License
=======
+22 -16
View File
@@ -21,6 +21,7 @@ module abcexport;
import std.file;
import std.path;
import std.string;
import std.stdio;
import swffile;
void main(string[] args)
@@ -28,22 +29,27 @@ void main(string[] args)
if (args.length == 1)
throw new Exception("No file specified");
foreach (arg; args[1..$])
{
scope swf = SWFFile.read(cast(ubyte[])read(arg));
uint count;
foreach (ref tag; swf.tags)
if ((tag.type == TagType.DoABC || tag.type == TagType.DoABC2))
{
ubyte[] abc;
if (tag.type == TagType.DoABC)
abc = tag.data;
else
try
{
scope swf = SWFFile.read(cast(ubyte[])read(arg));
uint count = 0;
foreach (ref tag; swf.tags)
if ((tag.type == TagType.DoABC || tag.type == TagType.DoABC2))
{
auto p = tag.data.ptr+4; // skip flags
while (*p++) {} // skip name
abc = tag.data[p-tag.data.ptr..$];
ubyte[] abc;
if (tag.type == TagType.DoABC)
abc = tag.data;
else
{
auto p = tag.data.ptr+4; // skip flags
while (*p++) {} // skip name
abc = tag.data[p-tag.data.ptr..$];
}
write(getName(arg) ~ .toString(count++) ~ ".abc", abc);
}
write(getName(arg) ~ .toString(count++) ~ ".abc", abc);
}
}
if (count == 0)
throw new Exception("No DoABC tags found");
}
catch (Object o)
writefln("Error while processing %s: %s", arg, o);
}
+65 -48
View File
@@ -23,12 +23,7 @@ import std.string : format; // exception formatting
/**
* Implements a shallow representation of an .abc file.
* Loading and saving an .abc file using this class should produce
* output identical to the input - with one exception:
*
* exception_info's "to" field may point inside an instruction.
* This is apparently valid according to the implementation, however
* the exact offset inside the instruction is not preserved - instead,
* the field will point to the beginning of the next instruction.
* output identical to the input.
*/
class ABCFile
@@ -213,6 +208,20 @@ class ABCFile
TraitsInfo[] traits;
}
/// Destination for a jump or exception block boundary
struct Label
{
union
{
struct
{
uint index; /// instruction index
int offset; /// signed offset relative to said instruction
}
private int absoluteOffset; /// internal temporary value used during reading and writing
}
}
struct Instruction
{
Opcode opcode;
@@ -223,15 +232,15 @@ class ABCFile
ulong uintv;
uint index;
uint jumpTarget;
uint[] switchTargets;
Label jumpTarget;
Label[] switchTargets;
}
Argument[] arguments;
}
struct ExceptionInfo
{
uint from, to, target;
Label from, to, target;
uint excType;
uint varName;
}
@@ -1268,25 +1277,32 @@ private final class ABCReader
size_t len = readU30();
uint[] instructionAtOffset = new uint[len];
void offsetToIndex(ref uint x, bool relaxed = false)
void translateLabel(ref ABCFile.Label label)
{
if (x >= len)
int absoluteOffset = label.absoluteOffset;
int instructionOffset = absoluteOffset;
while (true)
{
//throw new Exception(format("Jump out of bounds (by %d bytes)", x - len));
// Unreachable OOB jumps seem to be "valid"
x = 0;
return;
}
if (relaxed)
while (instructionAtOffset[x] == uint.max)
if (instructionOffset >= cast(int)len)
{
x++;
if (x >= len)
throw new Exception("Relaxed jump inside last instruction");
label.index = r.instructions.length;
instructionOffset = len;
break;
}
x = instructionAtOffset[x];
if (x == uint.max)
throw new Exception("Jump inside instruction");
if (instructionOffset <= 0)
{
label.index = 0;
instructionOffset = 0;
break;
}
if (instructionAtOffset[instructionOffset] != uint.max)
{
label.index = instructionAtOffset[instructionOffset];
break;
}
instructionOffset--;
}
label.offset = absoluteOffset-instructionOffset;
}
{
@@ -1333,17 +1349,17 @@ private final class ABCReader
case OpcodeArgumentType.JumpTarget:
int delta = readS24();
instruction.arguments[i].jumpTarget = offset + delta;
instruction.arguments[i].jumpTarget.absoluteOffset = offset + delta;
break;
case OpcodeArgumentType.SwitchDefaultTarget:
instruction.arguments[i].jumpTarget = instructionOffset + readS24();
instruction.arguments[i].jumpTarget.absoluteOffset = instructionOffset + readS24();
break;
case OpcodeArgumentType.SwitchTargets:
instruction.arguments[i].switchTargets.length = readU30()+1;
foreach (ref off; instruction.arguments[i].switchTargets)
off = instructionOffset + readS24();
foreach (ref label; instruction.arguments[i].switchTargets)
label.absoluteOffset = instructionOffset + readS24();
break;
default:
@@ -1363,13 +1379,11 @@ private final class ABCReader
{
case OpcodeArgumentType.JumpTarget:
case OpcodeArgumentType.SwitchDefaultTarget:
pos = start + instructionOffsets[ii];
offsetToIndex(instruction.arguments[i].jumpTarget);
translateLabel(instruction.arguments[i].jumpTarget);
break;
case OpcodeArgumentType.SwitchTargets:
pos = start + instructionOffsets[ii];
foreach (ref x; instruction.arguments[i].switchTargets)
offsetToIndex(x);
translateLabel(x);
break;
default:
break;
@@ -1381,9 +1395,9 @@ private final class ABCReader
foreach (ref value; r.exceptions)
{
value = readExceptionInfo();
offsetToIndex(value.from);
offsetToIndex(value.to, true);
offsetToIndex(value.target);
translateLabel(value.from);
translateLabel(value.to);
translateLabel(value.target);
}
r.traits.length = readU30();
foreach (ref value; r.traits)
@@ -1394,9 +1408,9 @@ private final class ABCReader
ABCFile.ExceptionInfo readExceptionInfo()
{
ABCFile.ExceptionInfo r;
r.from = readU30();
r.to = readU30();
r.target = readU30();
r.from.absoluteOffset = readU30();
r.to.absoluteOffset = readU30();
r.target.absoluteOffset = readU30();
r.excType = readU30();
r.varName = readU30();
return r;
@@ -1702,7 +1716,7 @@ private final class ABCWriter
if (v.attr & TraitAttributes.Metadata)
{
writeU30(v.metadata.length);
foreach (ref value; v.metadata)
foreach (value; v.metadata)
writeU30(value);
}
}
@@ -1731,7 +1745,9 @@ private final class ABCWriter
writeU30(v.initScopeDepth);
writeU30(v.maxScopeDepth);
uint[] instructionOffsets = new uint[v.instructions.length];
uint[] instructionOffsets = new uint[v.instructions.length+1];
uint resolveLabel(ref ABCFile.Label label) { return instructionOffsets[label.index]+label.offset; }
{
// we don't know the length before writing all the instructions - swap buffer with a temporary one
@@ -1740,7 +1756,7 @@ private final class ABCWriter
buf = new ubyte[1024];
pos = 0;
struct Fixup { uint target, pos, base; }
struct Fixup { ABCFile.Label target; uint pos, base; }
Fixup[] fixups;
foreach (ii, ref instruction; v.instructions)
@@ -1807,11 +1823,12 @@ private final class ABCWriter
}
buf.length = pos;
instructionOffsets[v.instructions.length] = pos;
foreach (ref fixup; fixups)
{
pos = fixup.pos;
writeS24(instructionOffsets[fixup.target]-fixup.base);
writeS24(resolveLabel(fixup.target)-fixup.base);
}
auto code = buf;
@@ -1825,9 +1842,9 @@ private final class ABCWriter
writeU30(v.exceptions.length);
foreach (ref value; v.exceptions)
{
value.from = instructionOffsets[value.from];
value.to = instructionOffsets[value.to];
value.target = instructionOffsets[value.target];
value.from.absoluteOffset = resolveLabel(value.from);
value.to.absoluteOffset = resolveLabel(value.to);
value.target.absoluteOffset = resolveLabel(value.target);
writeExceptionInfo(value);
}
writeU30(v.traits.length);
@@ -1837,9 +1854,9 @@ private final class ABCWriter
void writeExceptionInfo(ABCFile.ExceptionInfo v)
{
writeU30(v.from);
writeU30(v.to);
writeU30(v.target);
writeU30(v.from.absoluteOffset);
writeU30(v.to.absoluteOffset);
writeU30(v.target.absoluteOffset);
writeU30(v.excType);
writeU30(v.varName);
}
+3
View File
@@ -52,6 +52,7 @@
<word name="#get"/>
<word name="#include"/>
<word name="#mixin"/>
<word name="#privatens"/>
<word name="#set"/>
<word name="#unset"/>
</keywords>
@@ -73,10 +74,12 @@
<word name="implements"/>
<word name="initscopedepth"/>
<word name="instance"/>
<word name="item"/>
<word name="localcount"/>
<word name="majorversion"/>
<word name="maxscopedepth"/>
<word name="maxstack"/>
<word name="metadata"/>
<word name="method"/>
<word name="minorversion"/>
<word name="name"/>
+369 -160
View File
@@ -87,6 +87,7 @@ final class ASProgram
}
mixin AutoCompare;
mixin AutoToString;
R processData(R, string prolog, string epilog, H)(ref H handler)
{
@@ -124,6 +125,30 @@ final class ASProgram
}
mixin(epilog);
}
private Multiname[] toQNames()
{
switch (kind)
{
case ASType.QName:
case ASType.QNameA:
return [this];
case ASType.Multiname:
case ASType.MultinameA:
Multiname[] result;
foreach (ns; vMultiname.nsSet)
{
auto n = new Multiname;
n.kind = kind==ASType.Multiname ? ASType.QName : ASType.QNameA;
n.vQName.ns = ns;
n.vQName.name = vMultiname.name;
result ~= n;
}
return result;
default:
throw new .Exception("Can't expand Multiname of this kind");
}
}
}
static class Method
@@ -143,10 +168,16 @@ final class ASProgram
struct Item
{
string key, value;
mixin AutoCompare;
mixin ProcessAllData;
}
string name;
Item[] items;
mixin AutoCompare;
mixin ProcessAllData;
}
static class Instance
@@ -212,6 +243,11 @@ final class ASProgram
Trait[] traits;
Instance instance;
string toString()
{
return instance.name.toString();
}
}
static class Script
@@ -250,15 +286,15 @@ final class ASProgram
Class classv;
Method methodv;
uint jumpTarget;
uint[] switchTargets;
ABCFile.Label jumpTarget;
ABCFile.Label[] switchTargets;
}
Argument[] arguments;
}
struct Exception
{
uint from, to, target;
ABCFile.Label from, to, target;
Multiname excType;
Multiname varName;
}
@@ -371,7 +407,7 @@ private final class ABCtoAS
return n;
}
ASProgram.Multiname convertMultiname(ref ABCFile.Multiname multiname, uint i)
ASProgram.Multiname convertMultiname(ref ABCFile.Multiname multiname)
{
auto n = new ASProgram.Multiname();
n.kind = multiname.kind;
@@ -399,16 +435,7 @@ private final class ABCtoAS
n.vMultinameL.nsSet = namespaceSets[multiname.MultinameL.nsSet];
break;
case ASType.TypeName:
if (multiname.TypeName.name >= i)
throw new Exception("Forward Multiname/TypeName reference");
n.vTypeName.name = multinames[multiname.TypeName.name];
n.vTypeName.params.length = multiname.TypeName.params.length;
foreach (j, param; multiname.TypeName.params)
{
if (param >= i)
throw new Exception("Forward Multiname/TypeName parameter reference");
n.vTypeName.params[j] = multinames[param];
}
// handled in postConvertMultiname
break;
default:
throw new Exception("Unknown Multiname kind");
@@ -416,6 +443,20 @@ private final class ABCtoAS
return n;
}
void postConvertMultiname(ref ABCFile.Multiname multiname, ASProgram.Multiname n)
{
switch (multiname.kind)
{
case ASType.TypeName:
n.vTypeName.name = multinames[multiname.TypeName.name];
n.vTypeName.params.length = multiname.TypeName.params.length;
foreach (j, param; multiname.TypeName.params)
n.vTypeName.params[j] = multinames[param];
default:
break;
}
}
ASProgram.Method convertMethod(ref ABCFile.MethodInfo method)
{
auto n = new ASProgram.Method();
@@ -482,6 +523,9 @@ private final class ABCtoAS
default:
throw new Exception("Unknown trait kind");
}
r[i].metadata.length = trait.metadata.length;
foreach (j, index; trait.metadata)
r[i].metadata[j] = metadata[index];
}
return r;
}
@@ -626,7 +670,10 @@ private final class ABCtoAS
multinames.length = abc.multinames.length;
foreach (i, ref multiname; abc.multinames)
if (i)
multinames[i] = convertMultiname(multiname, i);
multinames[i] = convertMultiname(multiname);
foreach (i, ref multiname; abc.multinames)
if (i)
postConvertMultiname(multiname, multinames[i]);
methods.length = methodAdded.length = abc.methods.length;
foreach (i, ref method; abc.methods)
@@ -686,19 +733,36 @@ private final class AStoABC
}
/// Maintain an unordered set of values; sort/index by usage count
struct Constant(T)
struct ConstantPool(T, bool haveNull = true)
{
static Constant!(T)[T] pool;
static T[] values;
static bool add(T value) // return true if added
struct Entry
{
if (isNull(value))
uint hits;
T value;
uint index;
mixin AutoCompare;
R processData(R, string prolog, string epilog, H)(ref H handler)
{
mixin(prolog);
mixin(addAutoField("hits", true));
mixin(addAutoField("value"));
mixin(epilog);
}
}
Entry[T] pool;
T[] values;
bool add(T value) // return true if added
{
if (haveNull && isNull(value))
return false;
auto cp = value in pool;
if (cp is null)
{
pool[value] = Constant!(T)(1, value);
pool[value] = Entry(1, value);
return true;
}
else
@@ -708,113 +772,196 @@ private final class AStoABC
}
}
static bool notAdded(T value)
bool notAdded(T value)
{
return !(isNull(value) || value in pool);
auto ep = value in pool;
if (ep)
ep.hits++;
return !((haveNull && isNull(value)) || ep);
}
static void sort()
void finalize()
{
auto all = pool.values;
all.sort;
values.length = all.length+1;
enum { NullOffset = haveNull ? 1 : 0 }
values.length = all.length + NullOffset;
foreach (i, ref c; all)
{
pool[c.value].index = i+1;
values[i+1] = c.value;
pool[c.value].index = i + NullOffset;
values[i + NullOffset] = c.value;
}
}
static uint get(T value)
uint get(T value)
{
if (isNull(value))
if (haveNull && isNull(value))
return 0;
return pool[value].index;
}
mixin AutoCompare;
R processData(R, string prolog, string epilog, H)(ref H handler)
{
mixin(prolog);
mixin(addAutoField("hits", true));
mixin(addAutoField("value"));
mixin(epilog);
}
uint hits;
T value;
uint index;
}
/// Pair an index with class instances
struct Reference(T)
struct ReferencePool(T : Object)
{
static Reference[void*] references;
static T[] objects;
struct Entry
{
uint hits;
void* object;
uint addIndex, index;
void*[] parents;
static bool add(T obj) // return true if added
int opCmp(Entry* o)
{
if (this.parents.contains(o.object))
return 1;
if (o.parents.contains(this.object))
return -1;
if (o.hits==hits)
return addIndex-o.addIndex;
else
return o.hits-hits;
}
}
Entry[void*] pool;
T[] objects;
bool add(T obj) // return true if added
{
if (obj is null)
return false;
auto p = cast(void*)obj;
auto rp = p in references;
auto rp = p in pool;
if (rp is null)
{
references[p] = Reference!(T)(1); //, objects.length
objects ~= obj;
pool[p] = Entry(1, p, pool.length);
return true;
}
else
{
rp.hits++;
return false;
}
}
static bool notAdded(T obj)
bool notAdded(T obj)
{
return !(obj is null || (cast(void*)obj) in references);
auto ep = (cast(void*)obj) in pool;
if (ep)
ep.hits++;
return !(obj is null || ep);
}
static void reindex()
void registerDependency(T from, T to)
{
foreach (i, o; objects)
references[cast(void*)o].index = i;
auto pfrom = (cast(void*)from) in pool;
assert(pfrom, "Unknown dependency source");
auto vto = cast(void*)to;
auto pto = vto in pool;
assert(pto, "Unknown dependency target");
assert(!pfrom.parents.contains(vto), "Dependency already set");
pfrom.parents ~= vto;
}
static uint get(T obj)
T[] getPreliminaryObjects()
{
return cast(T[])pool.keys;
}
void finalize()
{
// assign unique index
int i=0;
foreach (ref e; pool)
e.index = i++;
// topographical sort
bool done;
while (!done)
{
done = true;
foreach (ref a; pool)
foreach (parent; a.parents)
{
auto pb = parent in pool;
assert(pb !is null, "Can't find referenced object");
if (pb.index > a.index)
{
auto t = pb.index;
pb.index = a.index;
a.index = t;
done = false;
break;
}
}
}
// copy to array
auto all = new Entry[i];
foreach (ref e; pool)
all[e.index] = e;
// sort
//all.sort; // sort preserving topological integrity demands compared items to always be adjacent
done = false;
while (!done)
{
done = true;
for (int j=0; j<all.length-1; j++)
if (all[j] > &all[j+1])
{
auto t = all[j];
all[j] = all[j+1];
all[j+1] = t;
done = false;
}
}
objects.length = i;
foreach (j, ref e; all)
{
pool[e.object].index = j;
objects[j] = cast(T)e.object;
}
}
uint get(T obj)
{
assert(obj !is null, "Trying to get index of null object");
return references[cast(void*)obj].index;
return pool[cast(void*)obj].index;
}
uint index;
}
typedef Constant!(long) IntC;
typedef Constant!(ulong) UIntC;
typedef Constant!(double) DoubleC;
typedef Constant!(string) StringC;
typedef Constant!(ASProgram.Namespace) NamespaceC;
typedef Constant!(ASProgram.Namespace[]) NamespaceSetC;
typedef Constant!(ASProgram.Multiname) MultinameC;
typedef Reference!(ASProgram.Class) ClassR;
typedef Reference!(ASProgram.Method) MethodR;
ConstantPool!(long) ints;
ConstantPool!(ulong) uints;
ConstantPool!(double) doubles;
ConstantPool!(string) strings;
ConstantPool!(ASProgram.Namespace) namespaces;
ConstantPool!(ASProgram.Namespace[]) namespaceSets;
ConstantPool!(ASProgram.Multiname) multinames;
ConstantPool!(ASProgram.Metadata, false) metadatas;
ReferencePool!(ASProgram.Class) classes;
ReferencePool!(ASProgram.Method) methods;
void visitNamespace(ASProgram.Namespace ns)
{
if (NamespaceC.add(ns))
StringC.add(ns.name);
if (namespaces.add(ns))
strings.add(ns.name);
}
void visitNamespaceSet(ASProgram.Namespace[] nsSet)
{
if (NamespaceSetC.add(nsSet))
if (namespaceSets.add(nsSet))
foreach (ns; nsSet)
visitNamespace(ns);
}
void visitMultiname(ASProgram.Multiname multiname)
{
if (MultinameC.notAdded(multiname))
if (multinames.notAdded(multiname))
{
with (multiname)
switch (kind)
@@ -822,18 +969,18 @@ private final class AStoABC
case ASType.QName:
case ASType.QNameA:
visitNamespace(vQName.ns);
StringC.add(vQName.name);
strings.add(vQName.name);
break;
case ASType.RTQName:
case ASType.RTQNameA:
StringC.add(vRTQName.name);
strings.add(vRTQName.name);
break;
case ASType.RTQNameL:
case ASType.RTQNameLA:
break;
case ASType.Multiname:
case ASType.MultinameA:
StringC.add(vMultiname.name);
strings.add(vMultiname.name);
visitNamespaceSet(vMultiname.nsSet);
break;
case ASType.MultinameL:
@@ -848,7 +995,7 @@ private final class AStoABC
default:
throw new .Exception("Unknown Multiname kind");
}
bool r = MultinameC.add(multiname);
bool r = multinames.add(multiname);
assert(r);
}
}
@@ -885,6 +1032,21 @@ private final class AStoABC
default:
throw new Exception("Unknown trait kind");
}
foreach (metadata; trait.metadata)
visitMetadata(metadata);
}
}
void visitMetadata(ASProgram.Metadata metadata)
{
if (metadatas.add(metadata))
{
strings.add(metadata.name);
foreach (ref item; metadata.items)
{
strings.add(item.key);
strings.add(item.value);
}
}
}
@@ -893,16 +1055,16 @@ private final class AStoABC
switch (value.vkind)
{
case ASType.Integer:
IntC.add(value.vint);
ints.add(value.vint);
break;
case ASType.UInteger:
UIntC.add(value.vuint);
uints.add(value.vuint);
break;
case ASType.Double:
DoubleC.add(value.vdouble);
doubles.add(value.vdouble);
break;
case ASType.Utf8:
StringC.add(value.vstring);
strings.add(value.vstring);
break;
case ASType.Namespace:
case ASType.PackageNamespace:
@@ -925,31 +1087,31 @@ private final class AStoABC
void visitClass(ASProgram.Class vclass)
{
if (ClassR.notAdded(vclass))
if (classes.notAdded(vclass))
{
visitMethod(vclass.cinit);
visitTraits(vclass.traits);
visitInstance(vclass.instance);
bool r = ClassR.add(vclass);
bool r = classes.add(vclass);
assert(r);
}
}
void visitMethod(ASProgram.Method method)
{
if (MethodR.add(method))
if (methods.add(method))
with (method)
{
foreach (type; paramTypes)
visitMultiname(type);
visitMultiname(returnType);
StringC.add(name);
strings.add(name);
foreach (ref value; options)
visitValue(value);
foreach (name; paramNames)
StringC.add(name);
strings.add(name);
if (vbody)
visitMethodBody(vbody);
@@ -971,16 +1133,16 @@ private final class AStoABC
break;
case OpcodeArgumentType.Int:
IntC.add(instruction.arguments[i].intv);
ints.add(instruction.arguments[i].intv);
break;
case OpcodeArgumentType.UInt:
UIntC.add(instruction.arguments[i].uintv);
uints.add(instruction.arguments[i].uintv);
break;
case OpcodeArgumentType.Double:
DoubleC.add(instruction.arguments[i].doublev);
doubles.add(instruction.arguments[i].doublev);
break;
case OpcodeArgumentType.String:
StringC.add(instruction.arguments[i].stringv);
strings.add(instruction.arguments[i].stringv);
break;
case OpcodeArgumentType.Namespace:
visitNamespace(instruction.arguments[i].namespacev);
@@ -1010,6 +1172,7 @@ private final class AStoABC
visitMultiname(exception.varName);
}
visitMethod(vbody.method);
visitTraits(vbody.traits);
}
@@ -1029,13 +1192,13 @@ private final class AStoABC
switch (value.vkind)
{
case ASType.Integer:
return IntC.get(value.vint);
return ints.get(value.vint);
case ASType.UInteger:
return UIntC.get(value.vuint);
return uints.get(value.vuint);
case ASType.Double:
return DoubleC.get(value.vdouble);
return doubles.get(value.vdouble);
case ASType.Utf8:
return StringC.get(value.vstring);
return strings.get(value.vstring);
case ASType.Namespace:
case ASType.PackageNamespace:
case ASType.PackageInternalNs:
@@ -1043,7 +1206,7 @@ private final class AStoABC
case ASType.ExplicitNamespace:
case ASType.StaticProtectedNs:
case ASType.PrivateNamespace:
return NamespaceC.get(value.vnamespace);
return namespaces.get(value.vnamespace);
case ASType.True:
case ASType.False:
case ASType.Null:
@@ -1054,6 +1217,28 @@ private final class AStoABC
}
}
void registerClassDependencies()
{
ASProgram.Class[ASProgram.Multiname] classByName;
ASProgram.Class[] classObjects = classes.getPreliminaryObjects();
foreach (c; classObjects)
{
assert(!(c.instance.name in classByName), "Duplicate class name " ~ c.instance.name.toString());
classByName[c.instance.name] = c;
}
foreach (c; classObjects)
foreach (dependency; [c.instance.superName] ~ c.instance.interfaces)
if (dependency)
foreach (dependencyName; dependency.toQNames())
{
auto pp = dependencyName in classByName;
if (pp)
classes.registerDependency(c, *pp);
}
}
this(ASProgram as)
{
this.abc = new ABCFile();
@@ -1069,43 +1254,43 @@ private final class AStoABC
foreach (method; as.orphanMethods)
visitMethod(method);
IntC.sort();
UIntC.sort();
DoubleC.sort();
StringC.sort();
NamespaceC.sort();
NamespaceSetC.sort();
MultinameC.sort();
//MultinameC.flip();
//MultinameC.reindex();
//ClassR.flip();
ClassR.reindex();
MethodR.reindex();
registerClassDependencies();
abc.ints = IntC.values;
abc.uints = UIntC.values;
abc.doubles = DoubleC.values;
abc.strings = StringC.values;
ints.finalize();
uints.finalize();
doubles.finalize();
strings.finalize();
namespaces.finalize();
namespaceSets.finalize();
multinames.finalize();
metadatas.finalize();
classes.finalize();
methods.finalize();
abc.namespaces.length = NamespaceC.values.length;
foreach (i, v; NamespaceC.values[1..$])
abc.ints = ints.values;
abc.uints = uints.values;
abc.doubles = doubles.values;
abc.strings = strings.values;
abc.namespaces.length = namespaces.values.length;
foreach (i, v; namespaces.values[1..$])
{
auto n = &abc.namespaces[i+1];
n.kind = v.kind;
n.name = StringC.get(v.name);
n.name = strings.get(v.name);
}
abc.namespaceSets.length = NamespaceSetC.values.length;
foreach (i, v; NamespaceSetC.values[1..$])
abc.namespaceSets.length = namespaceSets.values.length;
foreach (i, v; namespaceSets.values[1..$])
{
auto n = new uint[v.length];
foreach (j, ns; v)
n[j] = NamespaceC.get(ns);
n[j] = namespaces.get(ns);
abc.namespaceSets[i+1] = n;
}
abc.multinames.length = MultinameC.values.length;
foreach (i, v; MultinameC.values[1..$])
abc.multinames.length = multinames.values.length;
foreach (i, v; multinames.values[1..$])
{
auto n = &abc.multinames[i+1];
n.kind = v.kind;
@@ -1113,47 +1298,60 @@ private final class AStoABC
{
case ASType.QName:
case ASType.QNameA:
n.QName.ns = NamespaceC.get(v.vQName.ns);
n.QName.name = StringC.get(v.vQName.name);
n.QName.ns = namespaces.get(v.vQName.ns);
n.QName.name = strings.get(v.vQName.name);
break;
case ASType.RTQName:
case ASType.RTQNameA:
n.RTQName.name = StringC.get(v.vRTQName.name);
n.RTQName.name = strings.get(v.vRTQName.name);
break;
case ASType.RTQNameL:
case ASType.RTQNameLA:
break;
case ASType.Multiname:
case ASType.MultinameA:
n.Multiname.name = StringC.get(v.vMultiname.name);
n.Multiname.nsSet = NamespaceSetC.get(v.vMultiname.nsSet);
n.Multiname.name = strings.get(v.vMultiname.name);
n.Multiname.nsSet = namespaceSets.get(v.vMultiname.nsSet);
break;
case ASType.MultinameL:
case ASType.MultinameLA:
n.MultinameL.nsSet = NamespaceSetC.get(v.vMultinameL.nsSet);
n.MultinameL.nsSet = namespaceSets.get(v.vMultinameL.nsSet);
break;
case ASType.TypeName:
n.TypeName.name = MultinameC.get(v.vTypeName.name);
n.TypeName.name = multinames.get(v.vTypeName.name);
n.TypeName.params.length = v.vTypeName.params.length;
foreach (j, param; v.vTypeName.params)
n.TypeName.params[j] = MultinameC.get(param);
n.TypeName.params[j] = multinames.get(param);
break;
default:
throw new Exception("Unknown Multiname kind");
}
}
abc.metadata.length = metadatas.values.length;
foreach (i, m; metadatas.values)
{
auto n = &abc.metadata[i];
n.name = strings.get(m.name);
n.items.length = m.items.length;
foreach (j, ref item; m.items)
{
n.items[j].key = strings.get(m.items[j].key);
n.items[j].value = strings.get(m.items[j].value);
}
}
ASProgram.MethodBody[] bodies;
abc.methods.length = MethodR.objects.length;
foreach (i, o; MethodR.objects)
abc.methods.length = methods.objects.length;
foreach (i, o; methods.objects)
{
auto n = &abc.methods[i];
n.paramTypes.length = o.paramTypes.length;
foreach (j, p; o.paramTypes)
n.paramTypes[j] = MultinameC.get(p);
n.returnType = MultinameC.get(o.returnType);
n.name = StringC.get(o.name);
n.paramTypes[j] = multinames.get(p);
n.returnType = multinames.get(o.returnType);
n.name = strings.get(o.name);
n.flags = o.flags;
n.options.length = o.options.length;
foreach (j, ref value; o.options)
@@ -1163,34 +1361,34 @@ private final class AStoABC
}
n.paramNames.length = o.paramNames.length;
foreach (j, name; o.paramNames)
n.paramNames[j] = StringC.get(name);
n.paramNames[j] = strings.get(name);
if (o.vbody)
bodies ~= o.vbody;
}
abc.instances.length = ClassR.objects.length;
foreach (i, c; ClassR.objects)
abc.instances.length = classes.objects.length;
foreach (i, c; classes.objects)
{
auto o = c.instance;
auto n = &abc.instances[i];
n.name = MultinameC.get(o.name);
n.superName = MultinameC.get(o.superName);
n.name = multinames.get(o.name);
n.superName = multinames.get(o.superName);
n.flags = o.flags;
n.protectedNs = NamespaceC.get(o.protectedNs);
n.protectedNs = namespaces.get(o.protectedNs);
n.interfaces.length = o.interfaces.length;
foreach (j, intf; o.interfaces)
n.interfaces[j] = MultinameC.get(intf);
n.iinit = MethodR.get(o.iinit);
n.interfaces[j] = multinames.get(intf);
n.iinit = methods.get(o.iinit);
n.traits = convertTraits(o.traits);
}
abc.classes.length = ClassR.objects.length;
foreach (i, o; ClassR.objects)
abc.classes.length = classes.objects.length;
foreach (i, o; classes.objects)
{
auto n = &abc.classes[i];
n.cinit = MethodR.get(o.cinit);
n.cinit = methods.get(o.cinit);
n.traits = convertTraits(o.traits);
}
@@ -1198,7 +1396,7 @@ private final class AStoABC
foreach (i, o; as.scripts)
{
auto n = &abc.scripts[i];
n.sinit = MethodR.get(o.sinit);
n.sinit = methods.get(o.sinit);
n.traits = convertTraits(o.traits);
}
@@ -1206,7 +1404,7 @@ private final class AStoABC
foreach (i, o; bodies)
{
auto n = &abc.bodies[i];
n.method = MethodR.get(o.method);
n.method = methods.get(o.method);
n.maxStack = o.maxStack;
n.localCount = o.localCount;
n.initScopeDepth = o.initScopeDepth;
@@ -1221,8 +1419,8 @@ private final class AStoABC
ne.from = oe.from;
ne.to = oe.to;
ne.target = oe.target;
ne.excType = MultinameC.get(oe.excType);
ne.varName = MultinameC.get(oe.varName);
ne.excType = multinames.get(oe.excType);
ne.varName = multinames.get(oe.varName);
}
n.traits = convertTraits(o.traits);
}
@@ -1233,7 +1431,7 @@ private final class AStoABC
auto r = new ABCFile.TraitsInfo[traits.length];
foreach (i, ref trait; traits)
{
r[i].name = MultinameC.get(trait.name);
r[i].name = multinames.get(trait.name);
r[i].kind = trait.kind;
r[i].attr = trait.attr;
switch (trait.kind)
@@ -1241,27 +1439,30 @@ private final class AStoABC
case TraitKind.Slot:
case TraitKind.Const:
r[i].Slot.slotId = trait.vSlot.slotId;
r[i].Slot.typeName = MultinameC.get(trait.vSlot.typeName);
r[i].Slot.typeName = multinames.get(trait.vSlot.typeName);
r[i].Slot.vkind = trait.vSlot.value.vkind;
r[i].Slot.vindex = getValueIndex(trait.vSlot.value);
break;
case TraitKind.Class:
r[i].Class.slotId = trait.vClass.slotId;
r[i].Class.classi = ClassR.get(trait.vClass.vclass);
r[i].Class.classi = classes.get(trait.vClass.vclass);
break;
case TraitKind.Function:
r[i].Function.slotId = trait.vFunction.slotId;
r[i].Function.functioni = MethodR.get(trait.vFunction.vfunction);
r[i].Function.functioni = methods.get(trait.vFunction.vfunction);
break;
case TraitKind.Method:
case TraitKind.Getter:
case TraitKind.Setter:
r[i].Method.dispId = trait.vMethod.dispId;
r[i].Method.method = MethodR.get(trait.vMethod.vmethod);
r[i].Method.method = methods.get(trait.vMethod.vmethod);
break;
default:
throw new Exception("Unknown trait kind");
}
r[i].metadata.length = trait.metadata.length;
foreach (j, ref m; trait.metadata)
r[i].metadata[j] = metadatas.get(m);
}
return r;
}
@@ -1289,28 +1490,28 @@ private final class AStoABC
break;
case OpcodeArgumentType.Int:
r.arguments[i].index = IntC.get(instruction.arguments[i].intv);
r.arguments[i].index = ints.get(instruction.arguments[i].intv);
break;
case OpcodeArgumentType.UInt:
r.arguments[i].index = UIntC.get(instruction.arguments[i].uintv);
r.arguments[i].index = uints.get(instruction.arguments[i].uintv);
break;
case OpcodeArgumentType.Double:
r.arguments[i].index = DoubleC.get(instruction.arguments[i].doublev);
r.arguments[i].index = doubles.get(instruction.arguments[i].doublev);
break;
case OpcodeArgumentType.String:
r.arguments[i].index = StringC.get(instruction.arguments[i].stringv);
r.arguments[i].index = strings.get(instruction.arguments[i].stringv);
break;
case OpcodeArgumentType.Namespace:
r.arguments[i].index = NamespaceC.get(instruction.arguments[i].namespacev);
r.arguments[i].index = namespaces.get(instruction.arguments[i].namespacev);
break;
case OpcodeArgumentType.Multiname:
r.arguments[i].index = MultinameC.get(instruction.arguments[i].multinamev);
r.arguments[i].index = multinames.get(instruction.arguments[i].multinamev);
break;
case OpcodeArgumentType.Class:
r.arguments[i].index = ClassR.get(instruction.arguments[i].classv);
r.arguments[i].index = classes.get(instruction.arguments[i].classv);
break;
case OpcodeArgumentType.Method:
r.arguments[i].index = MethodR.get(instruction.arguments[i].methodv);
r.arguments[i].index = methods.get(instruction.arguments[i].methodv);
break;
case OpcodeArgumentType.JumpTarget:
@@ -1376,3 +1577,11 @@ class ASTraitsVisitor
}
}
}
private bool contains(T)(T[] arr, T val)
{
foreach (v; arr)
if (v == val)
return true;
return false;
}
+93 -31
View File
@@ -21,6 +21,7 @@ module assembler;
import std.file;
import std.string;
import std.conv;
import std.path;
import abcfile;
import asprogram;
@@ -46,16 +47,22 @@ final class Assembler
char* pos;
char* end;
string[] arguments;
string basePath;
static File load(string filename, string[] arguments = null)
{
return fromData(filename, cast(string)read(filename), arguments);
return fromFile(filename, cast(string)read(filename), arguments);
}
static File fromFile(string filename, string data, string[] arguments = null)
{
return fromData(filename, data, arguments, getDirName(filename));
}
static File fromData(string name, string data, string[] arguments = null)
static File fromData(string name, string data, string[] arguments = null, string basePath = null)
{
data ~= \0; data = data[0..$-1]; // hack to prevent readWord etc. from checking for end-of-file on every character
return File(name, data, data.ptr, data.ptr + data.length, arguments);
return File(name, data, data.ptr, data.ptr + data.length, arguments, basePath);
}
Position position()
@@ -79,8 +86,14 @@ final class Assembler
File[64] files;
int fileCount; /// recursion depth
string basePath;
string getBasePath()
{
foreach (ref file; files[0..fileCount])
if (file.basePath !is null)
return file.basePath;
return null;
}
string convertFilename(string filename)
{
if (filename.length == 0)
@@ -89,18 +102,9 @@ final class Assembler
foreach (ref c; filename)
if (c == '\\')
c = '/';
version(Windows)
{
if (filename.length > 2 && filename[1] == ':')
return filename;
}
if (filename[0] == '/')
return filename;
return basePath ~ filename;
return std.path.join(getBasePath, filename);
}
string[string] vars;
void skipWhitespace()
{
while (true)
@@ -126,6 +130,9 @@ final class Assembler
}
}
string[string] vars;
uint[string] privateNamespaces;
void handlePreprocessor()
{
skipChar(); // #
@@ -143,7 +150,7 @@ final class Assembler
break;
case "get":
auto filename = convertFilename(readString());
pushFile(File.fromData(filename, toStringLiteral(cast(string)read(filename))));
pushFile(File.fromFile(filename, toStringLiteral(cast(string)read(filename))));
break;
case "set":
vars[readWord()] = readString();
@@ -151,6 +158,10 @@ final class Assembler
case "unset":
vars.remove(readWord());
break;
case "privatens":
uint index = cast(uint)readUInt();
privateNamespaces[readString()] = index;
break;
default:
files[0].pos -= word.length;
throw new Exception("Unknown preprocessor declaration: " ~ word);
@@ -545,7 +556,11 @@ final class Assembler
if (n.kind == ASType.PrivateNamespace)
{
expectChar(',');
n.privateIndex = cast(uint)readUInt();
string name = readString();
auto pindex = name in privateNamespaces;
if (pindex is null)
throw new Exception("Unknown private namespace name");
n.privateIndex = *pindex;
}
expectChar(')');
return n;
@@ -641,6 +656,9 @@ final class Assembler
case "value":
t.vSlot.value = readValue();
break;
case "metadata":
t.metadata ~= readMetadata();
break;
case "end":
return t;
default:
@@ -663,6 +681,9 @@ final class Assembler
mustBeNull(t.vClass.vclass);
t.vClass.vclass = readClass();
break;
case "metadata":
t.metadata ~= readMetadata();
break;
case "end":
return t;
default:
@@ -685,6 +706,9 @@ final class Assembler
mustBeNull(t.vFunction.vfunction);
t.vFunction.vfunction = readMethod();
break;
case "metadata":
t.metadata ~= readMetadata();
break;
case "end":
return t;
default:
@@ -709,6 +733,9 @@ final class Assembler
mustBeNull(t.vMethod.vmethod);
t.vMethod.vmethod = readMethod();
break;
case "metadata":
t.metadata ~= readMetadata();
break;
case "end":
return t;
default:
@@ -720,6 +747,23 @@ final class Assembler
}
}
ASProgram.Metadata readMetadata()
{
auto metadata = new ASProgram.Metadata;
metadata.name = readString();
while (true)
switch (readWord())
{
case "item":
metadata.items ~= ASProgram.Metadata.Item(readString(), readString());
break;
case "end":
return metadata;
default:
throw new Exception("Expected item or end");
}
}
ASProgram.Method readMethod()
{
ASProgram.Method m = new ASProgram.Method;
@@ -915,6 +959,24 @@ final class Assembler
}
}
ABCFile.Label parseLabel(string label, uint[string] labels)
{
string name = label;
int offset = 0;
foreach (i, c; label)
if (c=='-' || c=='+')
{
name = label[0..i];
offset = .toInt(label[i..$]);
break;
}
auto lp = name in labels;
if (lp is null)
throw new Exception("Unknown label " ~ name);
return ABCFile.Label(*lp, offset);
}
ASProgram.Instruction[] readInstructions(ref uint[string] _labels)
{
ASProgram.Instruction[] instructions;
@@ -994,7 +1056,7 @@ final class Assembler
case OpcodeArgumentType.SwitchTargets:
string[] switchTargetLabels = readList!('[', ']', readWord, false)();
instruction.arguments[i].switchTargets = new uint[switchTargetLabels.length];
instruction.arguments[i].switchTargets.length = switchTargetLabels.length;
foreach (li, s; switchTargetLabels)
switchFixups ~= LocalFixup(files[0].position, instructions.length, i, s, li);
break;
@@ -1011,24 +1073,24 @@ final class Assembler
foreach (ref f; jumpFixups)
{
auto lp = f.name in labels;
if (lp is null)
try
instructions[f.ii].arguments[f.ai].jumpTarget = parseLabel(f.name, labels);
catch (Object o)
{
setFile(f.where.load);
throw new Exception("Unknown label " ~ f.name);
throw o;
}
instructions[f.ii].arguments[f.ai].jumpTarget = *lp;
}
foreach (ref f; switchFixups)
{
auto lp = f.name in labels;
if (lp is null)
try
instructions[f.ii].arguments[f.ai].switchTargets[f.si] = parseLabel(f.name, labels);
catch (Object o)
{
setFile(f.where.load);
throw new Exception("Unknown label " ~ f.name);
throw o;
}
instructions[f.ii].arguments[f.ai].switchTargets[f.si] = *lp;
}
foreach (ref f; localClassFixups)
@@ -1042,16 +1104,16 @@ final class Assembler
ASProgram.Exception readException(uint[string] labels)
{
uint readLabel()
ABCFile.Label readLabel()
{
auto word = readWord();
auto plabel = word in labels;
if (plabel is null)
try
return parseLabel(word, labels);
catch (Object o)
{
backpedal(word.length);
throw new Exception("Unknown label " ~ word);
throw o;
}
return *plabel;
}
ASProgram.Exception e;
+204 -39
View File
@@ -95,6 +95,9 @@ final class RefBuilder : ASTraitsVisitor
ASProgram.Class[string] classByName;
ASProgram.Method[string] methodByName;
string[uint] privateNamespaceNames;
uint[string] privateNamespaceByName;
ASProgram.Multiname[] context;
this(ASProgram as)
@@ -106,12 +109,23 @@ final class RefBuilder : ASTraitsVisitor
{
foreach (i, ref v; as.scripts)
addMethod(v.sinit, "script" ~ .toString(i) ~ "_sinit");
foreach (vclass; as.orphanClasses)
addClass(vclass, "orphan");
foreach (method; as.orphanMethods)
addMethod(method, "orphan");
super.run();
}
override void visitTrait(ref ASProgram.Trait trait)
{
context ~= trait.name;
auto m = trait.name;
if (m.kind != ASType.QName)
throw new Exception("Trait name is not a QName");
visitMultiname(m);
context ~= m;
switch (trait.kind)
{
case TraitKind.Class:
@@ -136,16 +150,96 @@ final class RefBuilder : ASTraitsVisitor
context = context[0..$-1];
}
string addPrivateNamespace(uint index, string bname)
{
string name = bname;
{
int n = 0;
uint* pindex;
while ((pindex = name in privateNamespaceByName) !is null && *pindex != index)
name = bname ~ .toString(++n);
}
auto pname = index in privateNamespaceNames;
if (pname)
{
if (*pname != name)
throw new Exception("Ambiguous private namespace: " ~ *pname ~ " and " ~ name);
}
else
{
privateNamespaceNames[index] = name;
privateNamespaceByName[name] = index;
}
return name;
}
void visitNamespace(ASProgram.Namespace ns)
{
if (ns.kind == ASType.PrivateNamespace && context.length>0 && context[0].vQName.ns.kind != ASType.PrivateNamespace)
addPrivateNamespace(ns.privateIndex, qNameToString(context[0]));
}
void visitNamespaceSet(ASProgram.Namespace[] nsSet)
{
foreach (ns; nsSet)
visitNamespace(ns);
}
void visitMultiname(ASProgram.Multiname m)
{
with (m)
switch (kind)
{
case ASType.QName:
case ASType.QNameA:
visitNamespace(vQName.ns);
break;
case ASType.Multiname:
case ASType.MultinameA:
visitNamespaceSet(vMultiname.nsSet);
break;
case ASType.MultinameL:
case ASType.MultinameLA:
visitNamespaceSet(vMultinameL.nsSet);
break;
case ASType.TypeName:
visitMultiname(vTypeName.name);
foreach (param; vTypeName.params)
visitMultiname(param);
break;
default:
break;
}
}
void visitMethodBody(ASProgram.MethodBody b)
{
foreach (ref instruction; b.instructions)
foreach (i, type; opcodeInfo[instruction.opcode].argumentTypes)
switch (type)
{
case OpcodeArgumentType.Namespace:
visitNamespace(instruction.arguments[i].namespacev);
break;
case OpcodeArgumentType.Multiname:
visitMultiname(instruction.arguments[i].multinamev);
break;
default:
break;
}
}
static string qNameToString(ASProgram.Multiname m)
{
assert(m.kind == ASType.QName);
return (m.vQName.ns.name.length ? m.vQName.ns.name ~ ":" : "") ~ m.vQName.name;
}
string contextToString(string field)
{
string[] strings = new string[context.length + (field ? 1 : 0)];
foreach (i, m; context)
{
// should this check ever fail, it's easy to fix it - just build any unique-ish string from the context
if (m.kind != ASType.QName)
throw new Exception("Trait name is not a QName");
strings[i] = (m.vQName.ns.name.length ? m.vQName.ns.name ~ "." : "") ~ m.vQName.name;
}
strings[i] = qNameToString(m);
if (field)
strings[$-1] = field;
string s = join(strings, "/");
@@ -167,9 +261,9 @@ final class RefBuilder : ASTraitsVisitor
return uniqueName;
}
void addClass(ASProgram.Class vclass)
void addClass(ASProgram.Class vclass, string field = null)
{
addObject(vclass, classByName, string.init);
addObject(vclass, classByName, field);
addMethod(vclass.cinit, "cinit");
addMethod(vclass.instance.iinit, "iinit");
}
@@ -177,6 +271,8 @@ final class RefBuilder : ASTraitsVisitor
void addMethod(ASProgram.Method method, string field = null)
{
addObject(method, methodByName, field);
if (method.vbody)
visitMethodBody(method.vbody);
}
string getObjectName(T)(T obj, ref T[string] objByName)
@@ -197,6 +293,16 @@ final class RefBuilder : ASTraitsVisitor
{
return getObjectName(method, methodByName);
}
string getPrivateNamespaceName(uint index)
{
auto pname = index in privateNamespaceNames;
if (pname)
return *pname;
else
//throw new Exception("Nameless private namespace: " ~ .toString(index));
return addPrivateNamespace(index, "OrphanPrivateNamespace");
}
}
final class Disassembler
@@ -221,6 +327,10 @@ final class Disassembler
StringBuilder sb = new StringBuilder(name ~ "/" ~ name ~ ".main.asasm");
sb ~= "#include ";
dumpString(sb, name ~ ".privatens.asasm");
sb.newLine();
sb ~= "program";
sb.indent++; sb.newLine();
@@ -272,6 +382,17 @@ final class Disassembler
sb ~= "end ; program"; sb.newLine();
sb.save();
// now dump the private namespace indices
sb = new StringBuilder(name ~ "/" ~ name ~ ".privatens.asasm");
uint[] indices = refs.privateNamespaceNames.keys.sort;
foreach (index; indices)
{
sb ~= "#privatens " ~ .toString(index) ~ " ";
dumpString(sb, refs.privateNamespaceNames[index]);
sb.newLine();
}
sb.save();
}
void dumpInt(StringBuilder sb, long v)
@@ -295,7 +416,7 @@ final class Disassembler
if (v == ABCFile.NULL_DOUBLE)
sb ~= "null";
else
sb ~= .toString(v);
sb ~= format("%.18g", v);
}
void dumpString(StringBuilder sb, string str)
@@ -345,7 +466,7 @@ final class Disassembler
if (kind == ASType.PrivateNamespace)
{
sb ~= ", ";
dumpUInt(sb, privateIndex);
dumpString(sb, refs.getPrivateNamespaceName(privateIndex));
}
sb ~= ')';
}
@@ -430,6 +551,7 @@ final class Disassembler
dumpMultiname(sb, trait.name);
if (trait.attr)
dumpFlags!(true)(sb, trait.attr, TraitAttributeNames);
bool inLine = false;
switch (trait.kind)
{
case TraitKind.Slot:
@@ -449,8 +571,7 @@ final class Disassembler
sb ~= " value ";
dumpValue(sb, trait.vSlot.value);
}
sb ~= " end";
sb.newLine();
inLine = true;
break;
case TraitKind.Class:
if (trait.vClass.slotId)
@@ -461,7 +582,6 @@ final class Disassembler
sb.indent++; sb.newLine();
sb ~= "class";
dumpClass(sb, trait.vClass.vclass);
sb.indent--; sb ~= "end ; trait"; sb.newLine();
break;
case TraitKind.Function:
if (trait.vFunction.slotId)
@@ -472,7 +592,6 @@ final class Disassembler
sb.indent++; sb.newLine();
sb ~= "method";
dumpMethod(sb, trait.vFunction.vfunction);
sb.indent--; sb ~= "end ; trait"; sb.newLine();
break;
case TraitKind.Method:
case TraitKind.Getter:
@@ -485,14 +604,44 @@ final class Disassembler
sb.indent++; sb.newLine();
sb ~= "method";
dumpMethod(sb, trait.vMethod.vmethod);
sb.indent--; sb ~= "end ; trait"; sb.newLine();
break;
default:
throw new Exception("Unknown trait kind");
}
foreach (metadata; trait.metadata)
{
if (inLine)
{
sb.indent++; sb.newLine();
inLine = false;
}
dumpMetadata(sb, metadata);
}
if (inLine)
{ sb ~= " end"; sb.newLine(); }
else
{ sb.indent--; sb ~= "end ; trait"; sb.newLine(); }
}
}
void dumpMetadata(StringBuilder sb, ASProgram.Metadata metadata)
{
sb ~= "metadata ";
dumpString(sb, metadata.name);
sb.indent++; sb.newLine();
foreach (ref item; metadata.items)
{
sb ~= "item ";
dumpString(sb, item.key);
sb ~= " ";
dumpString(sb, item.value);
sb.newLine();
}
sb.indent--; sb ~= "end ; metadata"; sb.newLine();
}
void dumpFlags(bool oneLine = false)(StringBuilder sb, ubyte flags, string[] names)
{
for (int i=0; flags; i++, flags>>=1)
@@ -600,10 +749,10 @@ final class Disassembler
{
string filename = refid.dup;
foreach (ref c; filename)
if (c == '.')
if (c == '.' || c == ':')
c = '/';
else
if (c == '\\' || c == ':' || c == '*' || c == '?' || c == '"' || c == '<' || c == '>' || c == '|')
if (c == '\\' || c == '*' || c == '?' || c == '"' || c == '<' || c == '>' || c == '|')
c = '_';
version (Windows)
@@ -701,6 +850,18 @@ final class Disassembler
sb.newLine();
}
void dumpLabel(StringBuilder sb, ref ABCFile.Label label)
{
sb ~= 'L';
sb ~= .toString(label.index);
if (label.offset != 0)
{
if (label.offset > 0)
sb ~= '+';
sb ~= .toString(label.offset);
}
}
void dumpMethodBody(StringBuilder sb, ASProgram.MethodBody mbody)
{
sb ~= "body";
@@ -712,22 +873,22 @@ final class Disassembler
sb ~= "code";
sb.newLine();
bool[] labels = new bool[mbody.instructions.length];
bool[] labels = new bool[mbody.instructions.length+1];
// reserve exception labels
foreach (ref e; mbody.exceptions)
labels[e.from] = labels[e.to] = labels[e.target] = true;
labels[e.from.index] = labels[e.to.index] = labels[e.target.index] = true;
dumpInstructions(sb, mbody.instructions, labels);
sb ~= "end ; code";
sb.newLine();
foreach (ref e; mbody.exceptions)
{
sb ~= "try from L";
sb ~= .toString(e.from);
sb ~= " to L";
sb ~= .toString(e.to);
sb ~= " target L";
sb ~= .toString(e.target);
sb ~= "try from ";
dumpLabel(sb, e.from);
sb ~= " to ";
dumpLabel(sb, e.to);
sb ~= " target ";
dumpLabel(sb, e.target);
sb ~= " type ";
dumpMultiname(sb, e.excType);
sb ~= " name ";
@@ -748,22 +909,18 @@ final class Disassembler
{
case OpcodeArgumentType.JumpTarget:
case OpcodeArgumentType.SwitchDefaultTarget:
labels[instruction.arguments[i].jumpTarget] = true;
labels[instruction.arguments[i].jumpTarget.index] = true;
break;
case OpcodeArgumentType.SwitchTargets:
foreach (ref x; instruction.arguments[i].switchTargets)
labels[x] = true;
foreach (ref label; instruction.arguments[i].switchTargets)
labels[label.index] = true;
break;
default:
break;
}
bool extraNewLine = false;
foreach (ii, ref instruction; instructions)
void checkLabel(uint ii)
{
if (extraNewLine)
sb.newLine();
extraNewLine = newLineAfter[instruction.opcode];
if (labels[ii])
{
sb.noIndent();
@@ -772,6 +929,15 @@ final class Disassembler
sb ~= ':';
sb.newLine();
}
}
bool extraNewLine = false;
foreach (ii, ref instruction; instructions)
{
if (extraNewLine)
sb.newLine();
extraNewLine = newLineAfter[instruction.opcode];
checkLabel(ii);
sb ~= opcodeInfo[instruction.opcode].name;
auto argTypes = opcodeInfo[instruction.opcode].argumentTypes;
@@ -823,8 +989,7 @@ final class Disassembler
case OpcodeArgumentType.JumpTarget:
case OpcodeArgumentType.SwitchDefaultTarget:
sb ~= 'L';
sb ~= .toString(instruction.arguments[i].jumpTarget);
dumpLabel(sb, instruction.arguments[i].jumpTarget);
break;
case OpcodeArgumentType.SwitchTargets:
@@ -832,8 +997,7 @@ final class Disassembler
auto targets = instruction.arguments[i].switchTargets;
foreach (ti, t; targets)
{
sb ~= 'L';
sb ~= .toString(t);
dumpLabel(sb, t);
if (ti < targets.length-1)
sb ~= ", ";
}
@@ -849,6 +1013,7 @@ final class Disassembler
}
sb.newLine();
}
checkLabel(instructions.length);
sb.indent--;
}
}
+67 -67
View File
@@ -152,73 +152,73 @@ private final class SWFReader
enum TagType
{
End = 0,
ShowFrame = 1,
DefineShape = 2,
FreeCharacter = 3,
PlaceObject = 4,
RemoveObject = 5,
DefineBits = 6,
DefineButton = 7,
JPEGTables = 8,
SetBackgroundColor = 9,
DefineFont = 10,
DefineText = 11,
DoAction = 12,
DefineFontInfo = 13,
DefineSound = 14,
StartSound = 15,
DefineButtonSound = 17,
SoundStreamHead = 18,
SoundStreamBlock = 19,
DefineBitsLossless = 20,
DefineBitsJPEG2 = 21,
DefineShape2 = 22,
DefineButtonCxform = 23,
Protect = 24,
PathsArePostScript = 25,
PlaceObject2 = 26,
RemoveObject2 = 28,
DefineShape3 = 32,
DefineText2 = 33,
DefineButton2 = 34,
DefineBitsJPEG3 = 35,
DefineBitsLossless2 = 36,
DefineSprite = 39,
ProductInfo = 41,
FrameLabel = 43,
SoundStreamHead2 = 45,
DefineMorphShape = 46,
DefineFont2 = 48,
DefineEditText = 37,
ExportAssets = 56,
ImportAssets = 57,
EnableDebugger = 58,
DoInitAction = 59,
DefineVideoStream = 60,
VideoFrame = 61,
DefineFontInfo2 = 62,
DebugID = 63,
EnableDebugger2 = 64,
ScriptLimits = 65,
SetTabIndex = 66,
FileAttributes = 69,
PlaceObject3 = 70,
ImportAssets2 = 71,
DoABC = 72,
DefineFontAlignZones = 73,
CSMTextSettings = 74,
DefineFont3 = 75,
SymbolClass = 76,
Metadata = 77,
DefineScalingGrid = 78,
DoABC2 = 82,
DefineShape4 = 83,
DefineMorphShape2 = 84,
DefineSceneAndFrameLabelData = 86,
DefineBinaryData = 87,
DefineFontName = 88,
DefineFont4 = 91
End = 0,
ShowFrame = 1,
DefineShape = 2,
FreeCharacter = 3,
PlaceObject = 4,
RemoveObject = 5,
DefineBits = 6,
DefineButton = 7,
JPEGTables = 8,
SetBackgroundColor = 9,
DefineFont = 10,
DefineText = 11,
DoAction = 12,
DefineFontInfo = 13,
DefineSound = 14,
StartSound = 15,
DefineButtonSound = 17,
SoundStreamHead = 18,
SoundStreamBlock = 19,
DefineBitsLossless = 20,
DefineBitsJPEG2 = 21,
DefineShape2 = 22,
DefineButtonCxform = 23,
Protect = 24,
PathsArePostScript = 25,
PlaceObject2 = 26,
RemoveObject2 = 28,
DefineShape3 = 32,
DefineText2 = 33,
DefineButton2 = 34,
DefineBitsJPEG3 = 35,
DefineBitsLossless2 = 36,
DefineSprite = 39,
ProductInfo = 41,
FrameLabel = 43,
SoundStreamHead2 = 45,
DefineMorphShape = 46,
DefineFont2 = 48,
DefineEditText = 37,
ExportAssets = 56,
ImportAssets = 57,
EnableDebugger = 58,
DoInitAction = 59,
DefineVideoStream = 60,
VideoFrame = 61,
DefineFontInfo2 = 62,
DebugID = 63,
EnableDebugger2 = 64,
ScriptLimits = 65,
SetTabIndex = 66,
FileAttributes = 69,
PlaceObject3 = 70,
ImportAssets2 = 71,
DoABC = 72,
DefineFontAlignZones = 73,
CSMTextSettings = 74,
DefineFont3 = 75,
SymbolClass = 76,
Metadata = 77,
DefineScalingGrid = 78,
DoABC2 = 82,
DefineShape4 = 83,
DefineMorphShape2 = 84,
DefineSceneAndFrameLabelData = 86,
DefineBinaryData = 87,
DefineFontName = 88,
DefineFont4 = 91
}
private final class SWFWriter