1LZMA SDK 24.09 2-------------- 3 4LZMA SDK provides the documentation, samples, header files, 5libraries, and tools you need to develop applications that 6use 7z / LZMA / LZMA2 / XZ compression. 7 8LZMA is an improved version of famous LZ77 compression algorithm. 9It was improved in way of maximum increasing of compression ratio, 10keeping high decompression speed and low memory requirements for 11decompressing. 12 13LZMA2 is a LZMA based compression method. LZMA2 provides better 14multithreading support for compression than LZMA and some other improvements. 15 167z is a file format for data compression and file archiving. 177z is a main file format for 7-Zip compression program (www.7-zip.org). 187z format supports different compression methods: LZMA, LZMA2 and others. 197z also supports AES-256 based encryption. 20 21XZ is a file format for data compression that uses LZMA2 compression. 22XZ format provides additional features: SHA/CRC check, filters for 23improved compression ratio, splitting to blocks and streams, 24 25 26 27LICENSE 28------- 29 30LZMA SDK is written and placed in the public domain by Igor Pavlov. 31 32Some code in LZMA SDK is based on public domain code from another developers: 33 1) PPMd var.H (2001): Dmitry Shkarin 34 2) SHA-256: Wei Dai (Crypto++ library) 35 36Anyone is free to copy, modify, publish, use, compile, sell, or distribute the 37original LZMA SDK code, either in source code form or as a compiled binary, for 38any purpose, commercial or non-commercial, and by any means. 39 40LZMA SDK code is compatible with open source licenses, for example, you can 41include it to GNU GPL or GNU LGPL code. 42 43 44LZMA SDK Contents 45----------------- 46 47 Source code: 48 49 - C / C++ / C# / Java - LZMA compression and decompression 50 - C / C++ - LZMA2 compression and decompression 51 - C / C++ - XZ compression and decompression 52 - C - 7z decompression 53 - C++ - 7z compression and decompression 54 - C - small SFXs for installers (7z decompression) 55 - C++ - SFXs and SFXs for installers (7z decompression) 56 57 Precomiled binaries: 58 59 - console programs for lzma / 7z / xz compression and decompression 60 - SFX modules for installers. 61 62 63UNIX/Linux version 64------------------ 65There are several otpions to compile 7-Zip with different compilers: gcc and clang. 66Also 7-Zip code contains two versions for some critical parts of code: in C and in Assembeler. 67So if you compile the version with Assembeler code, you will get faster 7-Zip binary. 68 697-Zip's assembler code uses the following syntax for different platforms: 70 711) x86 and x86-64 (AMD64): MASM syntax. 72 There are 2 programs that supports MASM syntax in Linux. 73' 'Asmc Macro Assembler and JWasm. But JWasm now doesn't support some 74 cpu instructions used in 7-Zip. 75 So you must install Asmc Macro Assembler in Linux, if you want to compile fastest version 76 of 7-Zip x86 and x86-64: 77 https://github.com/nidud/asmc 78 792) arm64: GNU assembler for ARM64 with preprocessor. 80 That systax of that arm64 assembler code in 7-Zip is supported by GCC and CLANG for ARM64. 81 82There are different binaries that can be compiled from 7-Zip source. 83There are 2 main files in folder for compiling: 84 makefile - that can be used for compiling Windows version of 7-Zip with nmake command 85 makefile.gcc - that can be used for compiling Linux/macOS versions of 7-Zip with make command 86 87At first you must change the current folder to folder that contains `makefile.gcc`: 88 89 cd CPP/7zip/Bundles/Alone7z 90 91Then you can compile `makefile.gcc` with the command: 92 93 make -j -f makefile.gcc 94 95Also there are additional "*.mak" files in folder "CPP/7zip/" that can be used to compile 967-Zip binaries with optimized code and optimzing options. 97 98To compile with GCC without assembler: 99 cd CPP/7zip/Bundles/Alone7z 100 make -j -f ../../cmpl_gcc.mak 101 102To compile with CLANG without assembler: 103 make -j -f ../../cmpl_clang.mak 104 105To compile 7-Zip for x86-64 with asmc assembler: 106 make -j -f ../../cmpl_gcc_x64.mak 107 108To compile 7-Zip for arm64 with assembler: 109 make -j -f ../../cmpl_gcc_arm64.mak 110 111To compile 7-Zip for arm64 for macOS: 112 make -j -f ../../cmpl_mac_arm64.mak 113 114Also you can change some compiler options in the mak files: 115 cmpl_gcc.mak 116 var_gcc.mak 117 warn_gcc.mak 118 119 120 121Also you can use p7zip (port of 7-Zip for POSIX systems like Unix or Linux): 122 123 http://p7zip.sourceforge.net/ 124 125 126Files 127----- 128 129DOC/7zC.txt - 7z ANSI-C Decoder description 130DOC/7zFormat.txt - 7z Format description 131DOC/installer.txt - information about 7-Zip for installers 132DOC/lzma.txt - LZMA compression description 133DOC/lzma-sdk.txt - LZMA SDK description (this file) 134DOC/lzma-history.txt - history of LZMA SDK 135DOC/lzma-specification.txt - Specification of LZMA 136DOC/Methods.txt - Compression method IDs for .7z 137 138bin/installer/ - example script to create installer that uses SFX module, 139 140bin/7zdec.exe - simplified 7z archive decoder (x86 32-bit version) 141bin/7zr.exe - 7-Zip console program (reduced version) (x86 32-bit version) 142bin/x64/7zr.exe - 7-Zip console program (reduced version) (x64 version) 143bin/x64/7zdec.exe - simplified 7z archive decoder (x64 version) 144bin/arm64/7zr.exe - 7-Zip console program (reduced version) (arm64 version) 145bin/arm64/7zdec.exe - simplified 7z archive decoder (arm64 version) 146bin/lzma.exe - file->file LZMA encoder/decoder for Windows 147bin/7zS2.sfx - small SFX module for installers (GUI version) 148bin/7zS2con.sfx - small SFX module for installers (Console version) 149bin/7zSD.sfx - SFX module for installers. 150 151 1527zDec.exe 153--------- 1547zDec.exe is simplified 7z archive decoder. 155It supports only LZMA, LZMA2, and PPMd methods. 1567zDec decodes whole solid block from 7z archive to RAM. 157The RAM consumption can be high. 158 159 160 161 162Source code structure 163--------------------- 164 165 166Asm/ - asm files (optimized code for CRC calculation and Intel-AES encryption) 167 168C/ - C files (compression / decompression and other) 169 Util/ 170 7z - 7z decoder program (decoding 7z files) 171 Lzma - LZMA program (file->file LZMA encoder/decoder). 172 LzmaLib - LZMA library (.DLL for Windows) 173 SfxSetup - small SFX module for installers 174 175CPP/ -- CPP files 176 177 Common - common files for C++ projects 178 Windows - common files for Windows related code 179 180 7zip - files related to 7-Zip 181 182 Archive - files related to archiving 183 184 Common - common files for archive handling 185 7z - 7z C++ Encoder/Decoder 186 187 Bundles - Modules that are bundles of other modules (files) 188 189 Alone7z - 7zr.exe: Standalone 7-Zip console program (reduced version) 190 Format7zExtractR - 7zxr.dll: Reduced version of 7z DLL: extracting from 7z/LZMA/BCJ/BCJ2. 191 Format7zR - 7zr.dll: Reduced version of 7z DLL: extracting/compressing to 7z/LZMA/BCJ/BCJ2 192 LzmaCon - lzma.exe: LZMA compression/decompression 193 LzmaSpec - example code for LZMA Specification 194 SFXCon - 7zCon.sfx: Console 7z SFX module 195 SFXSetup - 7zS.sfx: 7z SFX module for installers 196 SFXWin - 7z.sfx: GUI 7z SFX module 197 198 Common - common files for 7-Zip 199 200 Compress - files for compression/decompression 201 202 Crypto - files for encryption / decompression 203 204 UI - User Interface files 205 206 Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll 207 Common - Common UI files 208 Console - Code for console program (7z.exe) 209 Explorer - Some code from 7-Zip Shell extension 210 FileManager - Some GUI code from 7-Zip File Manager 211 GUI - Some GUI code from 7-Zip 212 213 214CS/ - C# files 215 7zip 216 Common - some common files for 7-Zip 217 Compress - files related to compression/decompression 218 LZ - files related to LZ (Lempel-Ziv) compression algorithm 219 LZMA - LZMA compression/decompression 220 LzmaAlone - file->file LZMA compression/decompression 221 RangeCoder - Range Coder (special code of compression/decompression) 222 223Java/ - Java files 224 SevenZip 225 Compression - files related to compression/decompression 226 LZ - files related to LZ (Lempel-Ziv) compression algorithm 227 LZMA - LZMA compression/decompression 228 RangeCoder - Range Coder (special code of compression/decompression) 229 230 231Note: 232 Asm / C / C++ source code of LZMA SDK is part of 7-Zip's source code. 233 7-Zip's source code can be downloaded from 7-Zip's SourceForge page: 234 235 http://sourceforge.net/projects/sevenzip/ 236 237 238 239LZMA features 240------------- 241 - Variable dictionary size (up to 4 GB) 242 - Estimated compressing speed: about 2 MB/s on 2 GHz CPU 243 - Estimated decompressing speed: 244 - 20-30 MB/s on modern 2 GHz cpu 245 - 1-2 MB/s on 200 MHz simple RISC cpu: (ARM, MIPS, PowerPC) 246 - Small memory requirements for decompressing (16 KB + DictionarySize) 247 - Small code size for decompressing: 5-8 KB 248 249LZMA decoder uses only integer operations and can be 250implemented in any modern 32-bit CPU (or on 16-bit CPU with some conditions). 251 252Some critical operations that affect the speed of LZMA decompression: 253 1) 32*16 bit integer multiply 254 2) Mispredicted branches (penalty mostly depends from pipeline length) 255 3) 32-bit shift and arithmetic operations 256 257The speed of LZMA decompressing mostly depends from CPU speed. 258Memory speed has no big meaning. But if your CPU has small data cache, 259overall weight of memory speed will slightly increase. 260 261 262How To Use 263---------- 264 265Using LZMA encoder/decoder executable 266-------------------------------------- 267 268Usage: LZMA <e|d> inputFile outputFile [<switches>...] 269 270 e: encode file 271 272 d: decode file 273 274 b: Benchmark. There are two tests: compressing and decompressing 275 with LZMA method. Benchmark shows rating in MIPS (million 276 instructions per second). Rating value is calculated from 277 measured speed and it is normalized with Intel's Core 2 results. 278 Also Benchmark checks possible hardware errors (RAM 279 errors in most cases). Benchmark uses these settings: 280 (-a1, -d21, -fb32, -mfbt4). You can change only -d parameter. 281 Also you can change the number of iterations. Example for 30 iterations: 282 LZMA b 30 283 Default number of iterations is 10. 284 285<Switches> 286 287 288 -a{N}: set compression mode 0 = fast, 1 = normal 289 default: 1 (normal) 290 291 d{N}: Sets Dictionary size - [0, 31], default: N=24 (32 MB) 292 The maximum value for dictionary size is N=31 (2 GB). 293 Dictionary size is calculated as DictionarySize = 2^N bytes. 294 For decompressing file compressed by LZMA method with dictionary 295 size D = 2^N you need about D bytes of memory (RAM). 296 297 -fb{N}: set number of fast bytes - [5, 273], default: 128 298 Usually big number gives a little bit better compression ratio 299 and slower compression process. 300 301 -lc{N}: set number of literal context bits - [0, 8], default: 3 302 Sometimes lc=4 gives gain for big files. 303 304 -lp{N}: set number of literal pos bits - [0, 4], default: 0 305 lp switch is intended for periodical data when period is 306 equal 2^N. For example, for 32-bit (4 bytes) 307 periodical data you can use lp=2. Often it's better to set lc0, 308 if you change lp switch. 309 310 -pb{N}: set number of pos bits - [0, 4], default: 2 311 pb switch is intended for periodical data 312 when period is equal 2^N. 313 314 -mf{MF_ID}: set Match Finder. Default: bt4. 315 Algorithms from hc* group doesn't provide good compression 316 ratio, but they often works pretty fast in combination with 317 fast mode (-a0). 318 319 Memory requirements depend from dictionary size 320 (parameter "d" in table below). 321 322 MF_ID Memory Description 323 324 bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing. 325 bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing. 326 bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing. 327 bt5 d * 11.5 + 4MB Binary Tree with 5 bytes hashing. 328 hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing. 329 hc5 d * 7.5 + 4MB Hash Chain with 5 bytes hashing. 330 331 -eos: write End Of Stream marker. By default LZMA doesn't write 332 eos marker, since LZMA decoder knows uncompressed size 333 stored in .lzma file header. 334 335 -si: Read data from stdin (it will write End Of Stream marker). 336 -so: Write data to stdout 337 338 339Examples: 340 3411) LZMA e file.bin file.lzma -d16 -lc0 342 343compresses file.bin to file.lzma with 64 KB dictionary (2^16=64K) 344and 0 literal context bits. -lc0 allows to reduce memory requirements 345for decompression. 346 347 3482) LZMA e file.bin file.lzma -lc0 -lp2 349 350compresses file.bin to file.lzma with settings suitable 351for 32-bit periodical data (for example, ARM or MIPS code). 352 3533) LZMA d file.lzma file.bin 354 355decompresses file.lzma to file.bin. 356 357 358Compression ratio hints 359----------------------- 360 361Recommendations 362--------------- 363 364To increase the compression ratio for LZMA compressing it's desirable 365to have aligned data (if it's possible) and also it's desirable to locate 366data in such order, where code is grouped in one place and data is 367grouped in other place (it's better than such mixing: code, data, code, 368data, ...). 369 370 371Filters 372------- 373You can increase the compression ratio for some data types, using 374special filters before compressing. For example, it's possible to 375increase the compression ratio on 5-10% for code for those CPU ISAs: 376x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC. 377 378You can find C source code of such filters in C/Bra*.* files 379 380You can check the compression ratio gain of these filters with such 3817-Zip commands (example for ARM code): 382No filter: 383 7z a a1.7z a.bin -m0=lzma 384 385With filter for little-endian ARM code: 386 7z a a2.7z a.bin -m0=arm -m1=lzma 387 388It works in such manner: 389Compressing = Filter_encoding + LZMA_encoding 390Decompressing = LZMA_decoding + Filter_decoding 391 392Compressing and decompressing speed of such filters is very high, 393so it will not increase decompressing time too much. 394Moreover, it reduces decompression time for LZMA_decoding, 395since compression ratio with filtering is higher. 396 397These filters convert CALL (calling procedure) instructions 398from relative offsets to absolute addresses, so such data becomes more 399compressible. 400 401For some ISAs (for example, for MIPS) it's impossible to get gain from such filter. 402 403 404 405--- 406 407http://www.7-zip.org 408http://www.7-zip.org/sdk.html 409http://www.7-zip.org/support.html 410