4 LZMA SDK Copyright (C) 1999-2007 Igor Pavlov
\r
6 LZMA SDK provides the documentation, samples, header files, libraries,
\r
7 and tools you need to develop applications that use LZMA compression.
\r
9 LZMA is default and general compression method of 7z format
\r
10 in 7-Zip compression program (www.7-zip.org). LZMA provides high
\r
11 compression ratio and very fast decompression.
\r
13 LZMA is an improved version of famous LZ77 compression algorithm.
\r
14 It was improved in way of maximum increasing of compression ratio,
\r
15 keeping high decompression speed and low memory requirements for
\r
23 LZMA SDK is available under any of the following licenses:
\r
25 1) GNU Lesser General Public License (GNU LGPL)
\r
26 2) Common Public License (CPL)
\r
27 3) Simplified license for unmodified code (read SPECIAL EXCEPTION)
\r
28 4) Proprietary license
\r
30 It means that you can select one of these four options and follow rules of that license.
\r
33 1,2) GNU LGPL and CPL licenses are pretty similar and both these
\r
34 licenses are classified as
\r
35 - "Free software licenses" at http://www.gnu.org/
\r
36 - "OSI-approved" at http://www.opensource.org/
\r
39 3) SPECIAL EXCEPTION
\r
41 Igor Pavlov, as the author of this code, expressly permits you
\r
42 to statically or dynamically link your code (or bind by name)
\r
43 to the files from LZMA SDK without subjecting your linked
\r
44 code to the terms of the CPL or GNU LGPL.
\r
45 Any modifications or additions to files from LZMA SDK, however,
\r
46 are subject to the GNU LGPL or CPL terms.
\r
48 SPECIAL EXCEPTION allows you to use LZMA SDK in applications with closed code,
\r
49 while you keep LZMA SDK code unmodified.
\r
52 SPECIAL EXCEPTION #2: Igor Pavlov, as the author of this code, expressly permits
\r
53 you to use this code under the same terms and conditions contained in the License
\r
54 Agreement you have for any previous version of LZMA SDK developed by Igor Pavlov.
\r
56 SPECIAL EXCEPTION #2 allows owners of proprietary licenses to use latest version
\r
57 of LZMA SDK as update for previous versions.
\r
60 SPECIAL EXCEPTION #3: Igor Pavlov, as the author of this code, expressly permits
\r
61 you to use code of the following files:
\r
62 BranchTypes.h, LzmaTypes.h, LzmaTest.c, LzmaStateTest.c, LzmaAlone.cpp,
\r
63 LzmaAlone.cs, LzmaAlone.java
\r
64 as public domain code.
\r
67 4) Proprietary license
\r
69 LZMA SDK also can be available under a proprietary license which
\r
72 1) Right to modify code without subjecting modified code to the
\r
73 terms of the CPL or GNU LGPL
\r
74 2) Technical support for code
\r
76 To request such proprietary license or any additional consultations,
\r
77 send email message from that page:
\r
78 http://www.7-zip.org/support.html
\r
81 You should have received a copy of the GNU Lesser General Public
\r
82 License along with this library; if not, write to the Free Software
\r
83 Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
\r
85 You should have received a copy of the Common Public License
\r
86 along with this library.
\r
94 - C++ source code of LZMA compressing and decompressing
\r
95 - ANSI-C compatible source code for LZMA decompressing
\r
96 - C# source code for LZMA compressing and decompressing
\r
97 - Java source code for LZMA compressing and decompressing
\r
98 - Compiled file->file LZMA compressing/decompressing program for Windows system
\r
100 ANSI-C LZMA decompression code was ported from original C++ sources to C.
\r
101 Also it was simplified and optimized for code size.
\r
102 But it is fully compatible with LZMA from 7-Zip.
\r
105 UNIX/Linux version
\r
107 To compile C++ version of file->file LZMA, go to directory
\r
108 C/7zip/Compress/LZMA_Alone
\r
109 and type "make" or "make clean all" to recompile all.
\r
111 In some UNIX/Linux versions you must compile LZMA with static libraries.
\r
112 To compile with static libraries, change string in makefile
\r
119 ---------------------
\r
121 CPP - CPP source code
\r
122 CS - C# source code
\r
123 Java - Java source code
\r
124 lzma.txt - LZMA SDK description (this file)
\r
125 7zFormat.txt - 7z Format description
\r
126 7zC.txt - 7z ANSI-C Decoder description (this file)
\r
127 methods.txt - Compression method IDs for .7z
\r
128 LGPL.txt - GNU Lesser General Public License
\r
129 CPL.html - Common Public License
\r
130 lzma.exe - Compiled file->file LZMA encoder/decoder for Windows
\r
131 history.txt - history of the LZMA SDK
\r
134 Source code structure
\r
135 ---------------------
\r
138 Compress - files related to compression/decompression
\r
139 Lz - files related to LZ (Lempel-Ziv) compression algorithm
\r
140 Lzma - ANSI-C compatible LZMA decompressor
\r
142 LzmaDecode.h - interface for LZMA decoding on ANSI-C
\r
143 LzmaDecode.c - LZMA decoding on ANSI-C (new fastest version)
\r
144 LzmaDecodeSize.c - LZMA decoding on ANSI-C (old size-optimized version)
\r
145 LzmaTest.c - test application that decodes LZMA encoded file
\r
146 LzmaTypes.h - basic types for LZMA Decoder
\r
147 LzmaStateDecode.h - interface for LZMA decoding (State version)
\r
148 LzmaStateDecode.c - LZMA decoding on ANSI-C (State version)
\r
149 LzmaStateTest.c - test application (State version)
\r
151 Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code
\r
153 Archive - files related to archiving
\r
154 7z_C - 7z ANSI-C Decoder
\r
159 Common - common files for C++ projects
\r
160 Windows - common files for Windows related code
\r
161 7zip - files related to 7-Zip Project
\r
163 Common - common files for 7-Zip
\r
165 Compress - files related to compression/decompression
\r
167 LZ - files related to LZ (Lempel-Ziv) compression algorithm
\r
170 RangeCoder - Range Coder (special code of compression/decompression)
\r
171 LZMA - LZMA compression/decompression on C++
\r
172 LZMA_Alone - file->file LZMA compression/decompression
\r
174 Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code
\r
176 Archive - files related to archiving
\r
178 Common - common files for archive handling
\r
179 7z - 7z C++ Encoder/Decoder
\r
181 Bundles - Modules that are bundles of other modules
\r
183 Alone7z - 7zr.exe: Standalone version of 7z.exe that supports only 7z/LZMA/BCJ/BCJ2
\r
184 Format7zR - 7zr.dll: Reduced version of 7za.dll: extracting/compressing to 7z/LZMA/BCJ/BCJ2
\r
185 Format7zExtractR - 7zxr.dll: Reduced version of 7zxa.dll: extracting from 7z/LZMA/BCJ/BCJ2.
\r
187 UI - User Interface files
\r
189 Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll
\r
190 Common - Common UI files
\r
191 Console - Code for console archiver
\r
197 Common - some common files for 7-Zip
\r
198 Compress - files related to compression/decompression
\r
199 LZ - files related to LZ (Lempel-Ziv) compression algorithm
\r
200 LZMA - LZMA compression/decompression
\r
201 LzmaAlone - file->file LZMA compression/decompression
\r
202 RangeCoder - Range Coder (special code of compression/decompression)
\r
206 Compression - files related to compression/decompression
\r
207 LZ - files related to LZ (Lempel-Ziv) compression algorithm
\r
208 LZMA - LZMA compression/decompression
\r
209 RangeCoder - Range Coder (special code of compression/decompression)
\r
211 C/C++ source code of LZMA SDK is part of 7-Zip project.
\r
213 You can find ANSI-C LZMA decompressing code at folder
\r
214 C/7zip/Compress/Lzma
\r
215 7-Zip doesn't use that ANSI-C LZMA code and that code was developed
\r
216 specially for this SDK. And files from C/7zip/Compress/Lzma do not need
\r
217 files from other directories of SDK for compiling.
\r
219 7-Zip source code can be downloaded from 7-Zip's SourceForge page:
\r
221 http://sourceforge.net/projects/sevenzip/
\r
226 - Variable dictionary size (up to 1 GB)
\r
227 - Estimated compressing speed: about 1 MB/s on 1 GHz CPU
\r
228 - Estimated decompressing speed:
\r
229 - 8-12 MB/s on 1 GHz Intel Pentium 3 or AMD Athlon
\r
230 - 500-1000 KB/s on 100 MHz ARM, MIPS, PowerPC or other simple RISC
\r
231 - Small memory requirements for decompressing (8-32 KB + DictionarySize)
\r
232 - Small code size for decompressing: 2-8 KB (depending from
\r
233 speed optimizations)
\r
235 LZMA decoder uses only integer operations and can be
\r
236 implemented in any modern 32-bit CPU (or on 16-bit CPU with some conditions).
\r
238 Some critical operations that affect to speed of LZMA decompression:
\r
239 1) 32*16 bit integer multiply
\r
240 2) Misspredicted branches (penalty mostly depends from pipeline length)
\r
241 3) 32-bit shift and arithmetic operations
\r
243 Speed of LZMA decompressing mostly depends from CPU speed.
\r
244 Memory speed has no big meaning. But if your CPU has small data cache,
\r
245 overall weight of memory speed will slightly increase.
\r
251 Using LZMA encoder/decoder executable
\r
252 --------------------------------------
\r
254 Usage: LZMA <e|d> inputFile outputFile [<switches>...]
\r
260 b: Benchmark. There are two tests: compressing and decompressing
\r
261 with LZMA method. Benchmark shows rating in MIPS (million
\r
262 instructions per second). Rating value is calculated from
\r
263 measured speed and it is normalized with AMD Athlon 64 X2 CPU
\r
264 results. Also Benchmark checks possible hardware errors (RAM
\r
265 errors in most cases). Benchmark uses these settings:
\r
266 (-a1, -d21, -fb32, -mfbt4). You can change only -d. Also you
\r
267 can change number of iterations. Example for 30 iterations:
\r
269 Default number of iterations is 10.
\r
274 -a{N}: set compression mode 0 = fast, 1 = normal
\r
275 default: 1 (normal)
\r
277 d{N}: Sets Dictionary size - [0, 30], default: 23 (8MB)
\r
278 The maximum value for dictionary size is 1 GB = 2^30 bytes.
\r
279 Dictionary size is calculated as DictionarySize = 2^N bytes.
\r
280 For decompressing file compressed by LZMA method with dictionary
\r
281 size D = 2^N you need about D bytes of memory (RAM).
\r
283 -fb{N}: set number of fast bytes - [5, 273], default: 128
\r
284 Usually big number gives a little bit better compression ratio
\r
285 and slower compression process.
\r
287 -lc{N}: set number of literal context bits - [0, 8], default: 3
\r
288 Sometimes lc=4 gives gain for big files.
\r
290 -lp{N}: set number of literal pos bits - [0, 4], default: 0
\r
291 lp switch is intended for periodical data when period is
\r
292 equal 2^N. For example, for 32-bit (4 bytes)
\r
293 periodical data you can use lp=2. Often it's better to set lc0,
\r
294 if you change lp switch.
\r
296 -pb{N}: set number of pos bits - [0, 4], default: 2
\r
297 pb switch is intended for periodical data
\r
298 when period is equal 2^N.
\r
300 -mf{MF_ID}: set Match Finder. Default: bt4.
\r
301 Algorithms from hc* group doesn't provide good compression
\r
302 ratio, but they often works pretty fast in combination with
\r
305 Memory requirements depend from dictionary size
\r
306 (parameter "d" in table below).
\r
308 MF_ID Memory Description
\r
310 bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing.
\r
311 bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing.
\r
312 bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing.
\r
313 hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing.
\r
315 -eos: write End Of Stream marker. By default LZMA doesn't write
\r
316 eos marker, since LZMA decoder knows uncompressed size
\r
317 stored in .lzma file header.
\r
319 -si: Read data from stdin (it will write End Of Stream marker).
\r
320 -so: Write data to stdout
\r
325 1) LZMA e file.bin file.lzma -d16 -lc0
\r
327 compresses file.bin to file.lzma with 64 KB dictionary (2^16=64K)
\r
328 and 0 literal context bits. -lc0 allows to reduce memory requirements
\r
332 2) LZMA e file.bin file.lzma -lc0 -lp2
\r
334 compresses file.bin to file.lzma with settings suitable
\r
335 for 32-bit periodical data (for example, ARM or MIPS code).
\r
337 3) LZMA d file.lzma file.bin
\r
339 decompresses file.lzma to file.bin.
\r
342 Compression ratio hints
\r
343 -----------------------
\r
348 To increase compression ratio for LZMA compressing it's desirable
\r
349 to have aligned data (if it's possible) and also it's desirable to locate
\r
350 data in such order, where code is grouped in one place and data is
\r
351 grouped in other place (it's better than such mixing: code, data, code,
\r
357 You can increase compression ratio for some data types, using
\r
358 special filters before compressing. For example, it's possible to
\r
359 increase compression ratio on 5-10% for code for those CPU ISAs:
\r
360 x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC.
\r
362 You can find C/C++ source code of such filters in folder "7zip/Compress/Branch"
\r
364 You can check compression ratio gain of these filters with such
\r
365 7-Zip commands (example for ARM code):
\r
367 7z a a1.7z a.bin -m0=lzma
\r
369 With filter for little-endian ARM code:
\r
370 7z a a2.7z a.bin -m0=bc_arm -m1=lzma
\r
372 With filter for big-endian ARM code (using additional Swap4 filter):
\r
373 7z a a3.7z a.bin -m0=swap4 -m1=bc_arm -m2=lzma
\r
375 It works in such manner:
\r
376 Compressing = Filter_encoding + LZMA_encoding
\r
377 Decompressing = LZMA_decoding + Filter_decoding
\r
379 Compressing and decompressing speed of such filters is very high,
\r
380 so it will not increase decompressing time too much.
\r
381 Moreover, it reduces decompression time for LZMA_decoding,
\r
382 since compression ratio with filtering is higher.
\r
384 These filters convert CALL (calling procedure) instructions
\r
385 from relative offsets to absolute addresses, so such data becomes more
\r
386 compressible. Source code of these CALL filters is pretty simple
\r
387 (about 20 lines of C++), so you can convert it from C++ version yourself.
\r
389 For some ISAs (for example, for MIPS) it's impossible to get gain from such filter.
\r
392 LZMA compressed file format
\r
393 ---------------------------
\r
394 Offset Size Description
\r
395 0 1 Special LZMA properties for compressed data
\r
396 1 4 Dictionary size (little endian)
\r
397 5 8 Uncompressed size (little endian). -1 means unknown size
\r
401 ANSI-C LZMA Decoder
\r
402 ~~~~~~~~~~~~~~~~~~~
\r
404 To compile ANSI-C LZMA Decoder you can use one of the following files sets:
\r
405 1) LzmaDecode.h + LzmaDecode.c + LzmaTest.c (fastest version)
\r
406 2) LzmaDecode.h + LzmaDecodeSize.c + LzmaTest.c (old size-optimized version)
\r
407 3) LzmaStateDecode.h + LzmaStateDecode.c + LzmaStateTest.c (zlib-like interface)
\r
410 Memory requirements for LZMA decoding
\r
411 -------------------------------------
\r
413 LZMA decoder doesn't allocate memory itself, so you must
\r
414 allocate memory and send it to LZMA.
\r
416 Stack usage of LZMA decoding function for local variables is not
\r
417 larger than 200 bytes.
\r
419 How To decompress data
\r
420 ----------------------
\r
422 LZMA Decoder (ANSI-C version) now supports 5 interfaces:
\r
423 1) Single-call Decompressing
\r
424 2) Single-call Decompressing with input stream callback
\r
425 3) Multi-call Decompressing with output buffer
\r
426 4) Multi-call Decompressing with input callback and output buffer
\r
427 5) Multi-call State Decompressing (zlib-like interface)
\r
429 Variant-5 is similar to Variant-4, but Variant-5 doesn't use callback functions.
\r
431 Decompressing steps
\r
432 -------------------
\r
434 1) read LZMA properties (5 bytes):
\r
435 unsigned char properties[LZMA_PROPERTIES_SIZE];
\r
437 2) read uncompressed size (8 bytes, little-endian)
\r
439 3) Decode properties:
\r
441 CLzmaDecoderState state; /* it's 24-140 bytes structure, if int is 32-bit */
\r
443 if (LzmaDecodeProperties(&state.Properties, properties, LZMA_PROPERTIES_SIZE) != LZMA_RESULT_OK)
\r
444 return PrintError(rs, "Incorrect stream properties");
\r
446 4) Allocate memory block for internal Structures:
\r
448 state.Probs = (CProb *)malloc(LzmaGetNumProbs(&state.Properties) * sizeof(CProb));
\r
449 if (state.Probs == 0)
\r
450 return PrintError(rs, kCantAllocateMessage);
\r
452 LZMA decoder uses array of CProb variables as internal structure.
\r
453 By default, CProb is unsigned_short. But you can define _LZMA_PROB32 to make
\r
454 it unsigned_int. It can increase speed on some 32-bit CPUs, but memory
\r
455 usage will be doubled in that case.
\r
458 5) Main Decompressing
\r
460 You must use one of the following interfaces:
\r
462 5.1 Single-call Decompressing
\r
463 -----------------------------
\r
464 When to use: RAM->RAM decompressing
\r
465 Compile files: LzmaDecode.h, LzmaDecode.c
\r
466 Compile defines: no defines
\r
467 Memory Requirements:
\r
468 - Input buffer: compressed size
\r
469 - Output buffer: uncompressed size
\r
470 - LZMA Internal Structures (~16 KB for default settings)
\r
473 int res = LzmaDecode(&state,
\r
474 inStream, compressedSize, &inProcessed,
\r
475 outStream, outSize, &outProcessed);
\r
478 5.2 Single-call Decompressing with input stream callback
\r
479 --------------------------------------------------------
\r
480 When to use: File->RAM or Flash->RAM decompressing.
\r
481 Compile files: LzmaDecode.h, LzmaDecode.c
\r
482 Compile defines: _LZMA_IN_CB
\r
483 Memory Requirements:
\r
484 - Buffer for input stream: any size (for example, 16 KB)
\r
485 - Output buffer: uncompressed size
\r
486 - LZMA Internal Structures (~16 KB for default settings)
\r
489 typedef struct _CBuffer
\r
491 ILzmaInCallback InCallback;
\r
493 unsigned char Buffer[kInBufferSize];
\r
496 int LzmaReadCompressed(void *object, const unsigned char **buffer, SizeT *size)
\r
498 CBuffer *bo = (CBuffer *)object;
\r
499 *buffer = bo->Buffer;
\r
500 *size = MyReadFile(bo->File, bo->Buffer, kInBufferSize);
\r
501 return LZMA_RESULT_OK;
\r
504 CBuffer g_InBuffer;
\r
506 g_InBuffer.File = inFile;
\r
507 g_InBuffer.InCallback.Read = LzmaReadCompressed;
\r
508 int res = LzmaDecode(&state,
\r
509 &g_InBuffer.InCallback,
\r
510 outStream, outSize, &outProcessed);
\r
513 5.3 Multi-call decompressing with output buffer
\r
514 -----------------------------------------------
\r
515 When to use: RAM->File decompressing
\r
516 Compile files: LzmaDecode.h, LzmaDecode.c
\r
517 Compile defines: _LZMA_OUT_READ
\r
518 Memory Requirements:
\r
519 - Input buffer: compressed size
\r
520 - Buffer for output stream: any size (for example, 16 KB)
\r
521 - LZMA Internal Structures (~16 KB for default settings)
\r
522 - LZMA dictionary (dictionary size is encoded in stream properties)
\r
526 state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize);
\r
528 LzmaDecoderInit(&state);
\r
532 inBuffer, inAvail, &inProcessed,
\r
533 g_OutBuffer, outAvail, &outProcessed);
\r
534 inAvail -= inProcessed;
\r
535 inBuffer += inProcessed;
\r
537 while you need more bytes
\r
539 see LzmaTest.c for more details.
\r
542 5.4 Multi-call decompressing with input callback and output buffer
\r
543 ------------------------------------------------------------------
\r
544 When to use: File->File decompressing
\r
545 Compile files: LzmaDecode.h, LzmaDecode.c
\r
546 Compile defines: _LZMA_IN_CB, _LZMA_OUT_READ
\r
547 Memory Requirements:
\r
548 - Buffer for input stream: any size (for example, 16 KB)
\r
549 - Buffer for output stream: any size (for example, 16 KB)
\r
550 - LZMA Internal Structures (~16 KB for default settings)
\r
551 - LZMA dictionary (dictionary size is encoded in stream properties)
\r
555 state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize);
\r
557 LzmaDecoderInit(&state);
\r
562 g_OutBuffer, outAvail, &outProcessed);
\r
564 while you need more bytes
\r
566 see LzmaTest.c for more details:
\r
569 5.5 Multi-call State Decompressing (zlib-like interface)
\r
570 ------------------------------------------------------------------
\r
571 When to use: file->file decompressing
\r
572 Compile files: LzmaStateDecode.h, LzmaStateDecode.c
\r
574 Memory Requirements:
\r
575 - Buffer for input stream: any size (for example, 16 KB)
\r
576 - Buffer for output stream: any size (for example, 16 KB)
\r
577 - LZMA Internal Structures (~16 KB for default settings)
\r
578 - LZMA dictionary (dictionary size is encoded in stream properties)
\r
582 state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize);
\r
585 LzmaDecoderInit(&state);
\r
588 res = LzmaDecode(&state,
\r
589 inBuffer, inAvail, &inProcessed,
\r
590 g_OutBuffer, outAvail, &outProcessed,
\r
592 inAvail -= inProcessed;
\r
593 inBuffer += inProcessed;
\r
595 while you need more bytes
\r
597 see LzmaStateTest.c for more details:
\r
600 6) Free all allocated blocks
\r
605 LzmaDecodeSize.c is size-optimized version of LzmaDecode.c.
\r
606 But compiled code of LzmaDecodeSize.c can be larger than
\r
607 compiled code of LzmaDecode.c. So it's better to use
\r
608 LzmaDecode.c in most cases.
\r
614 LZMA decoder can return one of the following codes:
\r
616 #define LZMA_RESULT_OK 0
\r
617 #define LZMA_RESULT_DATA_ERROR 1
\r
619 If you use callback function for input data and you return some
\r
620 error code, LZMA Decoder also returns that code.
\r
627 _LZMA_IN_CB - Use callback for input data
\r
629 _LZMA_OUT_READ - Use read function for output data
\r
631 _LZMA_LOC_OPT - Enable local speed optimizations inside code.
\r
632 _LZMA_LOC_OPT is only for LzmaDecodeSize.c (size-optimized version).
\r
633 _LZMA_LOC_OPT doesn't affect LzmaDecode.c (speed-optimized version)
\r
634 and LzmaStateDecode.c
\r
636 _LZMA_PROB32 - It can increase speed on some 32-bit CPUs,
\r
637 but memory usage will be doubled in that case
\r
639 _LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler
\r
640 and long is 32-bit.
\r
642 _LZMA_SYSTEM_SIZE_T - Define it if you want to use system's size_t.
\r
643 You can use it to enable 64-bit sizes supporting
\r
647 C++ LZMA Encoder/Decoder
\r
648 ~~~~~~~~~~~~~~~~~~~~~~~~
\r
649 C++ LZMA code use COM-like interfaces. So if you want to use it,
\r
650 you can study basics of COM/OLE.
\r
652 By default, LZMA Encoder contains all Match Finders.
\r
653 But for compressing it's enough to have just one of them.
\r
654 So for reducing size of compressing code you can define:
\r
655 #define COMPRESS_MF_BT
\r
656 #define COMPRESS_MF_BT4
\r
657 and it will use only bt4 match finder.
\r
662 http://www.7-zip.org
\r
663 http://www.7-zip.org/support.html
\r