6d6a32b7ae
[LZ4] Produce output as an arrary when decoding blocks
...
This change significantly improves performance.
2021-10-22 19:40:17 +03:00
e860b9d4a9
[LZ4] Slightly optimize match calculation in compression
2021-10-22 19:27:23 +03:00
100002cf8f
[LZ4] Simplify removing dictionary from the output in decompression
2021-10-21 22:31:34 +03:00
9e7faebf90
[LZ4] Don't use crc32 for matches identification
2021-10-21 22:31:00 +03:00
de7864c6db
[Deflate] Replace crc32 of a match with bytes contacenated into UInt32
...
Previously, we were using crc32 to identify matches when doing compression. This not only was an overkill computationally (the new version has boosted the compression speed), but it could also lead to incorrect results in some rare cases (potentially, only theoretical situtations) due to collisions of crc32 algorithm.
2021-10-21 22:30:23 +03:00
c1315b5727
[swcomp] Rename the --no-content-checksums option of lz4 command to --no-content-checksum
...
There can be at most one content checksum in a frame.
2021-10-20 15:01:56 +03:00
acb041cbf4
[LZ4] Fix incorrect encoding of small inputs
2021-10-20 15:00:57 +03:00
bb9574cefd
[LZ4] Fix incorrect checks for the location of the last match
2021-10-20 14:48:43 +03:00
1324744785
[swcomp] Print compression ratio in appropriate benchmarks
2021-10-18 21:38:31 +03:00
da8f4f7f5c
[Deflate] Rename some variables in lengthEncode
2021-10-18 18:35:06 +03:00
268aaaa356
[Deflate] Use start/endIndex instead of count property of data input
2021-10-18 18:31:40 +03:00
113972bcce
[Deflate] Reduce nesting in lengthEncode
...
This should be a purely refactoring change with no effect on performance or functionality.
2021-10-18 18:30:15 +03:00
3a54c54ecc
[LZ4] Fix incorrect matching
...
We erroneously used match index as a cyclical index which was producing non well-formed/incorrect results in some rare cases.
2021-10-17 15:45:11 +03:00
743b83e4cb
[LZ4] Add default values for dictionary and blockSize arguments of LZ4.compress
2021-10-17 15:45:11 +03:00
b64c5b81e9
[LZ4] Fix more potential issues with indices when resetting dictionary in compression
2021-10-16 21:59:18 +03:00
6cdfc14f54
[7-Zip] Add support for LZ4
2021-10-16 21:57:46 +03:00
68f1428355
[swcomp] Add various compression options for LZ4
...
Also add a check that dict id is not present without a dictionary.
2021-10-16 21:33:03 +03:00
212e1808e7
[swcomp] Add benchmark command for LZ4 compression with dependent blocks
2021-10-16 20:31:32 +03:00
2491d77bfc
[LZ4] Fix incorrect usage of dictionaries in compression
...
Previously, the dictionary was effectively ignored, since it was never used to populate the table of matches. Additionally, we fixed another issue with dictionaries, which was hidden, because we never really used them.
2021-10-16 20:21:59 +03:00
9915e8a1d7
[LZ4] Account for the min match length when checking for end-of-block condition
2021-10-16 19:31:58 +03:00
dac1814c8d
[LZ4] Simplify last sequence creation
2021-10-16 16:26:41 +03:00
20544178a0
[LZ4] Fix potential index issues when setting dictionary for block decoding
2021-10-16 16:25:58 +03:00
be899bc355
[LZ4] Fix incorrect block checksums in case of incompressible blocks
2021-10-16 16:25:15 +03:00
254fe0c848
[LZ4] Strictly check that the block is compressible
2021-10-16 16:24:36 +03:00
1a49f0543d
[swcomp] Add benchmark command for LZ4 compression with independent blocks
...
LZ4 compression with block dependency will be added in the future (after SWCompression is fixed with regards to dependent blocks).
2021-10-15 22:44:55 +03:00
377344197b
[swcomp] Implement --compress option for the lz4 command
2021-10-15 22:40:55 +03:00
9176d2fa35
[LZ4] Implement block encoding
2021-10-15 22:28:10 +03:00
f6c23987c4
[LZ4] Fix incorrect truncation checks in block decoding
2021-10-15 20:39:28 +03:00
cac1d635b9
[BZip2] Change BurrowsWheeler and SuffixArray from classes to enums
2021-10-15 20:14:21 +03:00
6c063cd522
[LZ4] Fix compilation issues on Swift 5.1-5.3
2021-10-02 18:23:26 +03:00
d5d0abeff2
[LZ4] Implement a no-compression version of LZ4 compression
...
Basically, the frame format with all features has been implemented.
2021-10-02 15:24:11 +03:00
68e46728af
[LZ4] Slightly optimize xxHash32
2021-10-02 13:49:29 +03:00
fc0dfdf43a
[LZ4] Add an empty implementation for compression
2021-10-02 11:22:45 +03:00
ba15f96312
[LZ4] Implement decompression of multiple concatenated frames
2021-10-01 14:41:16 +03:00
86c31f7462
[swcomp] Add options to lz4 command to support dictionary decompression
2021-09-27 14:29:30 +03:00
8e31f60b10
[LZ4] Check that dictionary is supplied if dictID is present in the frame
2021-09-27 10:46:27 +03:00
4df0d5feb2
[LZ4] Change dictionaryID type to UInt32?
...
This allows us to verify the value on type level, instead of by using the precondition. It is also okay to use UInt32 instead of Int ("currency type"), since this API is supposed to be used by advanced users only.
2021-09-27 10:43:26 +03:00
4bc7d37d7d
[LZ4] Implement dictionary decompression
2021-09-26 19:49:35 +03:00
d8f7b8a8f5
[swcomp] Add benchmark command for LZ4 decompression
2021-09-26 16:06:09 +03:00
8b0715b5a2
[swcomp] Add lz4 command
2021-09-26 16:03:56 +03:00
8f1a1e0517
[Tests] Add tests for LZ4 frames with various block sizes
2021-09-26 15:55:36 +03:00
8e32397e7c
[LZ4] Implement legacy frame processing
2021-09-26 15:18:57 +03:00
d430e939f8
[LZ4] Implement skipping of skippable frames
2021-09-26 14:37:31 +03:00
cfe0416547
[LZ4] Reorganize implementation to support other frame types
2021-09-26 14:24:59 +03:00
490d5ad816
Make DataError conform to Equatable
2021-09-26 14:03:38 +03:00
cb89e77857
[LZ4] Add truncation checks in block processing
2021-09-26 13:52:34 +03:00
c4946a00a3
[LZ4] Add checks for end of block restrictions
2021-09-26 13:43:47 +03:00
0b2dd279a9
[LZ4] Implement dependent blocks
2021-09-26 13:32:32 +03:00
c6addda952
[LZ4] Implement decoding of independent blocks
2021-09-25 22:56:51 +03:00
39d36a6805
[LZ4] Implement block loop and some checks for truncated data
2021-09-25 21:44:53 +03:00
Timofey Solomko