The LZMA dictionary size was being read from only 3 bytes (indices 1-3)
instead of the full 4 bytes (indices 1-4) specified in the LZMA format.
This caused decompression to fail with LZMAError.notEnoughToRepeat for
archives using dictionary sizes >= 16MB (e.g., LZMA:24 = 2^24 = 16MB),
because the high byte was not read and the dictionary size defaulted
to the minimum 4096 bytes.
Change: `for i in 1..<4` → `for i in 1..<5`
This seems to have a positive impact on the performance of Deflate and BZip2 decompression and compression. It is unclear to me why these two were declared as classes in the first place.
These are related to situation described in issue #57. We should now either properly throw an error, or in rare cases (when only literals are inside the block) produce an output.
In a dynamic Huffman block if either previous or zero code length is copied too many times, it could cause a crash or lead to inconsistent state. These situations are checked now.
This symbol in the dynamic Huffman block encodes copy of the previous code length. If this symbol is the first, then there is nothing to copy. Previously, this could cause a crash.
The idea here is that BZip2.decompress should not crash for any possible truncation. It should either throw an error or produce an output (if stars align such that truncated input is still a valid BZip2 "archive").
These functions weren't called anyway since 5e8f568f64 commit where Benchmark protocol requirement for this function acquired Double return type. In any event, skipping of warmup iteration is better controlled by the corresponding option of "benchmark run" command.
According to LZMA specification this property is ignored by the reference implementation. In addition, specification suggests that other LZMA decoders should also ignore it. We weren't using this property for anything anyway, so its removal does not really change anything, except for better compliance with LZMA specification.
In addition, range decoder initialization error is now thrown in accordance with reference implementation.
First, write(unsignedNumber:) is better than write(number:) when the argument is already UInt, since it does one fewer precondition check and does not do additional integer conversion. Secondly, using write(number:) to write one bit was very inefficient.
This allowed to rewrite implementation in a much clearer way. One user-visible change, however, is that NEW/BASE is now always printed with an index even if there is only one (implementation complexity to accommodate empty index is not worth it).
Ultimately, it turned out that the approach of the old save file format of separating runs metadatas and results only leads to super complicated code. Unfortunately, it didn't help with append-updates to save files (they are still loaded fully, their content is modified in memory, and the actual file is then overwritten).
Timofey Solomko
nekonyaandTimofey Solomko