package.xml0000664000175000017500000001754412667627750011334 0ustar janjan horde_lz4 pear.horde.org Horde LZ4 Compression Extension PHP extension that implements the LZ4 compression algorithm - an extremely fast lossless compression algorithm. Michael Slusarz slusarz slusarz@horde.org yes 2016-03-08 1.0.10 1.0.0 stable stable MIT (Expat) * [jan] Fix module version number (Bug #14240). 5.0.0 8.0.0alpha1 8.0.0alpha1 1.7.0 horde_lz4 2013-03-03 1.0.0beta1 1.0.0 beta beta PHP 3.01 * [mms] Initial release of package. 1.0.0beta2 1.0.0 beta beta 2013-03-04 PHP 3.01 * [mms] More robust header checking: eliminates the need to explicitly pass decompression size to horde_lz4_uncompress(). * [mms] Remove maxsize option to horde_lz4_uncompress(). * [mms] Remove 'prefix' option to horde_lz4_compress(). * [mms] Be silent when encountering decompression errors. 1.0.0 1.0.0 stable stable 2013-05-06 PHP 3.01 * First stable release. * [mms] Convert unit tests to PHPUnit. 1.0.1 1.0.0 stable stable 2013-07-08 MIT (Expat) * [mms] Change license from PHP -> MIT. 1.0.2 1.0.0 stable stable 2013-08-02 MIT (Expat) * [mms] Update LZ4 source to r99. 1.0.3 1.0.0 stable stable 2014-02-11 MIT (Expat) * [mms] Update lz4 source to r108. 1.0.4 1.0.0 stable stable 2014-07-01 MIT (Expat) * [jan] Update lz4 source to r118 (CVE-2014-4611). 1.0.5 1.0.0 stable stable 2014-07-03 MIT (Expat) * [mms] Fix compilation on ancient versions of PHP 5.3 (Bug #13310). 1.0.6 1.0.0 stable stable 2014-07-03 MIT (Expat) * [jan] Update lz4 source to r119 (CVE-2014-4715). 1.0.7 1.0.0 stable stable 2014-09-16 MIT (Expat) * [mms] Allow extension to be built with system lz4 library (Remi Collet <fedora@famillecollet.com>). * [mms] Add --with-liblz4 option to use system liblz4 (Remi) 1.0.8 1.0.0 stable stable 2015-04-13 MIT (Expat) * [jan] Fix PHP 7 compatibility (Remi Collet <fedora@famillecollet.com>). 1.0.9 1.0.0 stable stable 2016-02-01 MIT (Expat) * [jan] Mark PHP 7 as supported. 1.0.10 1.0.0 stable stable 2016-03-08 MIT (Expat) * [jan] Fix module version number (Bug #14240). horde_lz4-1.0.10/test/horde/lz4/fixtures/data.txt0000664000175000017500000000673412667627750017737 0ustar janjanTo be or not to be, that is the question; Whether 'tis nobler in the mind to suffer The Slings and Arrows of outrageous Fortune Or to take arms against a sea of troubles, And by opposing, end them. To die, to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to 'tis a consummation Devoutly to be wish'd. To die, to sleep; To sleep, perchance to dream. Ay, there's the rub, For in that sleep of death what dreams may come, When we have shuffled off this mortal coil, Must give us pause. There's the respect That makes calamity of so long life, For who would bear the whips and scorns of time, Th'oppressor's wrong, the proud man's contumely, The pangs of dispriz'd love, the law's delay, The insolence of office, and the spurns That patient merit of th'unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, But that the dread of something after death, The undiscovered country from whose bourn No traveller returns, puzzles the will, And makes us rather bear those ills we have Than fly to others that we know not of? Thus conscience does make cowards of us all, And thus the native hue of resolution Is sicklied o'er with the pale cast of thought, And enterprises of great pitch and moment With this regard their currents turn away, And lose the name of action. Hath not a Jew eyes? Hath not a Jew hands, organs, dimensions, senses, affections, passions; fed with the same food, hurt with the same weapons, subject to the same diseases, heal'd by the same means, warm'd and cool'd by the same winter and summer as a Christian is? If you prick us, do we not bleed? If you tickle us, do we not laugh? If you poison us, do we not die? And if you wrong us, shall we not revenge? If we are like you in the rest, we will resemble you in that. If a Jew wrong a Christian, what is his humility? Revenge. If a Christian wrong a Jew, what should his sufferance be by Christian example? Why, revenge. The villainy you teach me, I will execute, and it shall go hard but I will better the instruction. Is this a dagger which I see before me, The handle toward my hand? Come, let me clutch thee. I have thee not, and yet I see thee still. Art thou not, fatal vision, sensible To feeling as to sight? or art thou but A dagger of the mind, a false creation, Proceeding from the heat-oppress'd brain? I see thee yet, in form as palpable As this which now I draw. Thou marshall'st me the way that I was going; And such an instrument I was to use. Mine eyes are made the fools o' the other senses, Or else worth all the rest; I see thee still, And on thy blade and dudgeon gouts of blood, Which was not so before. There's no such thing: It is the bloody business which informs Thus to mine eyes. Now o'er the one halfworld Nature seems dead, and wicked dreams abuse The curtain'd sleep; witchcraft celebrates Pale Hecate's offerings, and wither'd murder, Alarum'd by his sentinel, the wolf, Whose howl's his watch, thus with his stealthy pace. With Tarquin's ravishing strides, towards his design Moves like a ghost. Thou sure and firm-set earth, Hear not my steps, which way they walk, for fear Thy very stones prate of my whereabout, And take the present horror from the time, Which now suits with it. Whiles I threat, he lives: Words to the heat of deeds too cold breath gives. I go, and it is done; the bell invites me. Hear it not, Duncan; for it is a knell That summons thee to heaven or to hell. horde_lz4-1.0.10/test/horde/lz4/AllTests.php0000664000175000017500000000013212667627750016642 0ustar janjanrun(); horde_lz4-1.0.10/test/horde/lz4/bootstrap.php0000664000175000017500000000014312667627750017126 0ustar janjan horde_lz4-1.0.10/test/horde/lz4/UnitTest.php0000664000175000017500000001203112667627750016667 0ustar janjan * @category Horde * @copyright 2013 Horde LLC * @ignore * @license http://www.php.net/license/3_01.txt PHP 3.01 * @package horde_lz4 * @subpackage UnitTests */ class horde_lz4_UnitTest extends PHPUnit_Framework_TestCase { private $data; public function setUp() { if (!extension_loaded('horde_lz4')) { $this->markTestSkipped('horde_lz4 extension not installed.'); } $this->data = file_get_contents(__DIR__ . '/fixtures/data.txt'); } // Test horde_lz4_compress() function : basic functionality public function testCompressBasic() { // Compressing a big string $output = horde_lz4_compress($this->data); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $this->data) ); // Compressing a smaller string $smallstring = "A small string to compress\n"; $output = horde_lz4_compress($smallstring); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $smallstring) ); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionOne() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Zero arguments horde_lz4_compress(); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionTwo() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Test horde_lz4_compress with one more than the expected number // of arguments $data = 'string_val'; $extra_arg = 10; horde_lz4_compress($data, false, $extra_arg); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionThree() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); horde_lz4_compress(new stdClass); } // Test horde_lz4_compress() function : variation public function testCompressVariation() { $output = horde_lz4_compress($this->data); $this->assertNotEquals( md5($output), md5(horde_lz4_compress($output)) ); } // Test horde_lz4_uncompress() function : basic functionality public function testUncompressBasic() { $compressed = horde_lz4_compress($this->data); $this->assertEquals( 0, strcmp($this->data, horde_lz4_uncompress($compressed)) ); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionOne() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Zero arguments horde_lz4_uncompress(); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionTwo() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Test horde_lz4_uncompress with one more than the expected number // of arguments $data = 'string_val'; $extra_arg = 10; horde_lz4_uncompress($data, $extra_arg); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionThree() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); horde_lz4_uncompress(new stdClass); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionFour() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Testing with incorrect arguments horde_lz4_uncompress(123); } // Test horde_lz4_compress() function : high compression public function testHighCompression() { // Compressing a big string $output = horde_lz4_compress($this->data, true); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $this->data) ); // Compressing a smaller string $smallstring = "A small string to compress\n"; $output = horde_lz4_compress($smallstring, true); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $smallstring) ); } // Test horde_lz4_uncompress() function : bad input (non-lz4 data) public function testUncompressBadInput() { // Bad data is missing the Horde-LZ4 header and is not LZ4 data. $bad_data = "12345678"; $this->assertFalse(horde_lz4_uncompress($bad_data)); } } horde_lz4-1.0.10/lib/LICENSE0000664000175000017500000000243612667627750013371 0ustar janjanLZ4 Library Copyright (c) 2011-2014, Yann Collet All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.horde_lz4-1.0.10/lib/lz4.c0000664000175000017500000013046712667627750013247 0ustar janjan/* LZ4 - Fast LZ compression algorithm Copyright (C) 2011-2014, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 source repository : http://code.google.com/p/lz4/ - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c */ /************************************** Tuning parameters **************************************/ /* * HEAPMODE : * Select how default compression functions will allocate memory for their hash table, * in memory stack (0:default, fastest), or in memory heap (1:requires memory allocation (malloc)). */ #define HEAPMODE 0 /************************************** CPU Feature Detection **************************************/ /* 32 or 64 bits ? */ #if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \ || defined(__powerpc64__) || defined(__powerpc64le__) \ || defined(__ppc64__) || defined(__ppc64le__) \ || defined(__PPC64__) || defined(__PPC64LE__) \ || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */ # define LZ4_ARCH64 1 #else # define LZ4_ARCH64 0 #endif /* * Little Endian or Big Endian ? * Overwrite the #define below if you know your architecture endianess */ #include /* Apparently required to detect endianess */ #if defined (__GLIBC__) # include # if (__BYTE_ORDER == __BIG_ENDIAN) # define LZ4_BIG_ENDIAN 1 # endif #elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN)) # define LZ4_BIG_ENDIAN 1 #elif defined(__sparc) || defined(__sparc__) \ || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \ || defined(__hpux) || defined(__hppa) \ || defined(_MIPSEB) || defined(__s390__) # define LZ4_BIG_ENDIAN 1 #else /* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */ #endif /* * Unaligned memory access is automatically enabled for "common" CPU, such as x86. * For others CPU, such as ARM, the compiler may be more cautious, inserting unnecessary extra code to ensure aligned access property * If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance */ #if defined(__ARM_FEATURE_UNALIGNED) # define LZ4_FORCE_UNALIGNED_ACCESS 1 #endif /* Define this parameter if your target system or compiler does not support hardware bit count */ #if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */ # define LZ4_FORCE_SW_BITCOUNT #endif /* * BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE : * This option may provide a small boost to performance for some big endian cpu, although probably modest. * You may set this option to 1 if data will remain within closed environment. * This option is useless on Little_Endian CPU (such as x86) */ /* #define BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE 1 */ /************************************** Compiler Options **************************************/ #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ /* "restrict" is a known keyword */ #else # define restrict /* Disable restrict */ #endif #ifdef _MSC_VER /* Visual Studio */ # define FORCE_INLINE static __forceinline # include /* For Visual 2005 */ # if LZ4_ARCH64 /* 64-bits */ # pragma intrinsic(_BitScanForward64) /* For Visual 2005 */ # pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */ # else /* 32-bits */ # pragma intrinsic(_BitScanForward) /* For Visual 2005 */ # pragma intrinsic(_BitScanReverse) /* For Visual 2005 */ # endif # pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ #else # ifdef __GNUC__ # define FORCE_INLINE static inline __attribute__((always_inline)) # else # define FORCE_INLINE static inline # endif #endif #ifdef _MSC_VER /* Visual Studio */ # define lz4_bswap16(x) _byteswap_ushort(x) #else # define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8))) #endif #define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) #if (GCC_VERSION >= 302) || (__INTEL_COMPILER >= 800) || defined(__clang__) # define expect(expr,value) (__builtin_expect ((expr),(value)) ) #else # define expect(expr,value) (expr) #endif #define likely(expr) expect((expr) != 0, 1) #define unlikely(expr) expect((expr) != 0, 0) /************************************** Memory routines **************************************/ #include /* malloc, calloc, free */ #define ALLOCATOR(n,s) calloc(n,s) #define FREEMEM free #include /* memset, memcpy */ #define MEM_INIT memset /************************************** Includes **************************************/ #include "lz4.h" /************************************** Basic Types **************************************/ #if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ # include typedef uint8_t BYTE; typedef uint16_t U16; typedef uint32_t U32; typedef int32_t S32; typedef uint64_t U64; #else typedef unsigned char BYTE; typedef unsigned short U16; typedef unsigned int U32; typedef signed int S32; typedef unsigned long long U64; #endif #if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS) # define _PACKED __attribute__ ((packed)) #else # define _PACKED #endif #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # if defined(__IBMC__) || defined(__SUNPRO_C) || defined(__SUNPRO_CC) # pragma pack(1) # else # pragma pack(push, 1) # endif #endif typedef struct { U16 v; } _PACKED U16_S; typedef struct { U32 v; } _PACKED U32_S; typedef struct { U64 v; } _PACKED U64_S; typedef struct {size_t v;} _PACKED size_t_S; #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # if defined(__SUNPRO_C) || defined(__SUNPRO_CC) # pragma pack(0) # else # pragma pack(pop) # endif #endif #define A16(x) (((U16_S *)(x))->v) #define A32(x) (((U32_S *)(x))->v) #define A64(x) (((U64_S *)(x))->v) #define AARCH(x) (((size_t_S *)(x))->v) /************************************** Constants **************************************/ #define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2) #define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE) #define HASH_SIZE_U32 (1 << LZ4_HASHLOG) #define MINMATCH 4 #define COPYLENGTH 8 #define LASTLITERALS 5 #define MFLIMIT (COPYLENGTH+MINMATCH) static const int LZ4_minLength = (MFLIMIT+1); #define KB *(1U<<10) #define MB *(1U<<20) #define GB *(1U<<30) #define LZ4_64KLIMIT ((64 KB) + (MFLIMIT-1)) #define SKIPSTRENGTH 6 /* Increasing this value will make the compression run slower on incompressible data */ #define MAXD_LOG 16 #define MAX_DISTANCE ((1 << MAXD_LOG) - 1) #define ML_BITS 4 #define ML_MASK ((1U<=e; */ #else # define LZ4_WILDCOPY(d,s,e) { if (likely(e-d <= 8)) LZ4_COPY8(d,s) else do { LZ4_COPY8(d,s) } while (d>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clzll(val) >> 3); # else int r; if (!(val>>32)) { r=4; } else { r=0; val>>=32; } if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } r += (!val); return r; # endif # else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanForward64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctzll(val) >> 3); # else static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 }; return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; # endif # endif } #else int LZ4_NbCommonBytes (register U32 val) { # if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanReverse( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clz(val) >> 3); # else int r; if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } r += (!val); return r; # endif # else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanForward( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctz(val) >> 3); # else static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 }; return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; # endif # endif } #endif /******************************** Compression functions ********************************/ int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); } static int LZ4_hashSequence(U32 sequence, tableType_t tableType) { if (tableType == byU16) return (((sequence) * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1))); else return (((sequence) * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG)); } static int LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return LZ4_hashSequence(A32(p), tableType); } static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, tableType_t tableType, const BYTE* srcBase) { switch (tableType) { case byPtr: { const BYTE** hashTable = (const BYTE**) tableBase; hashTable[h] = p; break; } case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); break; } case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); break; } } } static void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase) { U32 h = LZ4_hashPosition(p, tableType); LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase); } static const BYTE* LZ4_getPositionOnHash(U32 h, void* tableBase, tableType_t tableType, const BYTE* srcBase) { if (tableType == byPtr) { const BYTE** hashTable = (const BYTE**) tableBase; return hashTable[h]; } if (tableType == byU32) { U32* hashTable = (U32*) tableBase; return hashTable[h] + srcBase; } { U16* hashTable = (U16*) tableBase; return hashTable[h] + srcBase; } /* default, to ensure a return */ } static const BYTE* LZ4_getPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase) { U32 h = LZ4_hashPosition(p, tableType); return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase); } static unsigned LZ4_count(const BYTE* pIn, const BYTE* pRef, const BYTE* pInLimit) { const BYTE* const pStart = pIn; while (likely(pIndictSize; const BYTE* const dictionary = dictPtr->dictionary; const BYTE* const dictEnd = dictionary + dictPtr->dictSize; const size_t dictDelta = dictEnd - (const BYTE*)source; const BYTE* anchor = (const BYTE*) source; const BYTE* const iend = ip + inputSize; const BYTE* const mflimit = iend - MFLIMIT; const BYTE* const matchlimit = iend - LASTLITERALS; BYTE* op = (BYTE*) dest; BYTE* const olimit = op + maxOutputSize; const int skipStrength = SKIPSTRENGTH; U32 forwardH; size_t refDelta=0; /* Init conditions */ if ((U32)inputSize > (U32)LZ4_MAX_INPUT_SIZE) return 0; /* Unsupported input size, too large (or negative) */ switch(dict) { case noDict: default: base = (const BYTE*)source; lowLimit = (const BYTE*)source; break; case withPrefix64k: base = (const BYTE*)source - dictPtr->currentOffset; lowLimit = (const BYTE*)source - dictPtr->dictSize; break; case usingExtDict: base = (const BYTE*)source - dictPtr->currentOffset; lowLimit = (const BYTE*)source; break; } if ((tableType == byU16) && (inputSize>=(int)LZ4_64KLIMIT)) return 0; /* Size too large (not within 64K limit) */ if (inputSize> skipStrength; //if (step>8) step=8; // required for valid forwardIp ; slows down uncompressible data a bit if (unlikely(forwardIp > mflimit)) goto _last_literals; ref = LZ4_getPositionOnHash(h, ctx, tableType, base); if (dict==usingExtDict) { if (ref<(const BYTE*)source) { refDelta = dictDelta; lowLimit = dictionary; } else { refDelta = 0; lowLimit = (const BYTE*)source; } } forwardH = LZ4_hashPosition(forwardIp, tableType); LZ4_putPositionOnHash(ip, h, ctx, tableType, base); } while ( ((dictIssue==dictSmall) ? (ref < lowRefLimit) : 0) || ((tableType==byU16) ? 0 : (ref + MAX_DISTANCE < ip)) || (A32(ref+refDelta) != A32(ip)) ); } /* Catch up */ while ((ip>anchor) && (ref+refDelta > lowLimit) && (unlikely(ip[-1]==ref[refDelta-1]))) { ip--; ref--; } { /* Encode Literal length */ unsigned litLength = (unsigned)(ip - anchor); token = op++; if ((outputLimited) && (unlikely(op + litLength + (2 + 1 + LASTLITERALS) + (litLength/255) > olimit))) return 0; /* Check output limit */ if (litLength>=RUN_MASK) { int len = (int)litLength-RUN_MASK; *token=(RUN_MASK<= 255 ; len-=255) *op++ = 255; *op++ = (BYTE)len; } else *token = (BYTE)(litLength< matchlimit) limit = matchlimit; matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, limit); ip += MINMATCH + matchLength; if (ip==limit) { unsigned more = LZ4_count(ip, (const BYTE*)source, matchlimit); matchLength += more; ip += more; } } else { matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, matchlimit); ip += MINMATCH + matchLength; } if (matchLength>=ML_MASK) { if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) + (matchLength>>8) > olimit))) return 0; /* Check output limit */ *token += ML_MASK; matchLength -= ML_MASK; for (; matchLength >= 510 ; matchLength-=510) { *op++ = 255; *op++ = 255; } if (matchLength >= 255) { matchLength-=255; *op++ = 255; } *op++ = (BYTE)matchLength; } else *token += (BYTE)(matchLength); } anchor = ip; /* Test end of chunk */ if (ip > mflimit) break; /* Fill table */ LZ4_putPosition(ip-2, ctx, tableType, base); /* Test next position */ ref = LZ4_getPosition(ip, ctx, tableType, base); if (dict==usingExtDict) { if (ref<(const BYTE*)source) { refDelta = dictDelta; lowLimit = dictionary; } else { refDelta = 0; lowLimit = (const BYTE*)source; } } LZ4_putPosition(ip, ctx, tableType, base); if ( ((dictIssue==dictSmall) ? (ref>=lowRefLimit) : 1) && (ref+MAX_DISTANCE>=ip) && (A32(ref+refDelta)==A32(ip)) ) { token=op++; *token=0; goto _next_match; } /* Prepare next loop */ forwardH = LZ4_hashPosition(++ip, tableType); } _last_literals: /* Encode Last Literals */ { int lastRun = (int)(iend - anchor); if ((outputLimited) && (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize)) return 0; /* Check output limit */ if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK<= 255 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; } else *op++ = (BYTE)(lastRun<= sizeof(LZ4_stream_t_internal)); /* A compilation error here means LZ4_STREAMSIZE is not large enough */ if (dict->initCheck) MEM_INIT(dict, 0, sizeof(LZ4_stream_t_internal)); /* Uninitialized structure detected */ if (dictSize < MINMATCH) { dict->dictionary = NULL; dict->dictSize = 0; return 1; } if (p <= dictEnd - 64 KB) p = dictEnd - 64 KB; base = p - dict->currentOffset; dict->dictionary = p; dict->dictSize = (U32)(dictEnd - p); dict->currentOffset += dict->dictSize; while (p <= dictEnd-MINMATCH) { LZ4_putPosition(p, dict, byU32, base); p+=3; } return 1; } void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src) { if ((LZ4_dict->currentOffset > 0x80000000) || ((size_t)LZ4_dict->currentOffset > (size_t)src)) /* address space overflow */ { /* rescale hash table */ U32 delta = LZ4_dict->currentOffset - 64 KB; const BYTE* dictEnd = LZ4_dict->dictionary + LZ4_dict->dictSize; int i; for (i=0; ihashTable[i] < delta) LZ4_dict->hashTable[i]=0; else LZ4_dict->hashTable[i] -= delta; } LZ4_dict->currentOffset = 64 KB; if (LZ4_dict->dictSize > 64 KB) LZ4_dict->dictSize = 64 KB; LZ4_dict->dictionary = dictEnd - LZ4_dict->dictSize; } } FORCE_INLINE int LZ4_compress_continue_generic (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize, limitedOutput_directive limit) { LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_stream; const BYTE* const dictEnd = streamPtr->dictionary + streamPtr->dictSize; const BYTE* smallest = (const BYTE*) source; if (streamPtr->initCheck) return 0; /* Uninitialized structure detected */ if ((streamPtr->dictSize>0) && (smallest>dictEnd)) smallest = dictEnd; LZ4_renormDictT(streamPtr, smallest); /* Check overlapping input/dictionary space */ { const BYTE* sourceEnd = (const BYTE*) source + inputSize; if ((sourceEnd > streamPtr->dictionary) && (sourceEnd < dictEnd)) { streamPtr->dictSize = (U32)(dictEnd - sourceEnd); if (streamPtr->dictSize > 64 KB) streamPtr->dictSize = 64 KB; if (streamPtr->dictSize < 4) streamPtr->dictSize = 0; streamPtr->dictionary = dictEnd - streamPtr->dictSize; } } /* prefix mode : source data follows dictionary */ if (dictEnd == (const BYTE*)source) { int result; if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset)) result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, withPrefix64k, dictSmall); else result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, withPrefix64k, noDictIssue); streamPtr->dictSize += (U32)inputSize; streamPtr->currentOffset += (U32)inputSize; return result; } /* external dictionary mode */ { int result; if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset)) result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, usingExtDict, dictSmall); else result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, usingExtDict, noDictIssue); streamPtr->dictionary = (const BYTE*)source; streamPtr->dictSize = (U32)inputSize; streamPtr->currentOffset += (U32)inputSize; return result; } } int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize) { return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, 0, notLimited); } int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize) { return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limitedOutput); } // Hidden debug function, to force separate dictionary mode int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int inputSize) { LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_dict; int result; const BYTE* const dictEnd = streamPtr->dictionary + streamPtr->dictSize; const BYTE* smallest = dictEnd; if (smallest > (const BYTE*) source) smallest = (const BYTE*) source; LZ4_renormDictT((LZ4_stream_t_internal*)LZ4_dict, smallest); result = LZ4_compress_generic(LZ4_dict, source, dest, inputSize, 0, notLimited, byU32, usingExtDict, noDictIssue); streamPtr->dictionary = (const BYTE*)source; streamPtr->dictSize = (U32)inputSize; streamPtr->currentOffset += (U32)inputSize; return result; } int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int dictSize) { LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict; const BYTE* previousDictEnd = dict->dictionary + dict->dictSize; if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a dictionary > 64 KB */ if ((U32)dictSize > dict->dictSize) dictSize = dict->dictSize; memcpy(safeBuffer, previousDictEnd - dictSize, dictSize); dict->dictionary = (const BYTE*)safeBuffer; dict->dictSize = (U32)dictSize; return 1; } /**************************** Decompression functions ****************************/ /* * This generic decompression function cover all use cases. * It shall be instanciated several times, using different sets of directives * Note that it is essential this generic function is really inlined, * in order to remove useless branches during compilation optimisation. */ FORCE_INLINE int LZ4_decompress_generic( const char* source, char* dest, int inputSize, int outputSize, /* If endOnInput==endOnInputSize, this value is the max size of Output Buffer. */ int endOnInput, /* endOnOutputSize, endOnInputSize */ int partialDecoding, /* full, partial */ int targetOutputSize, /* only used if partialDecoding==partial */ int dict, /* noDict, withPrefix64k, usingExtDict */ const char* dictStart, /* only if dict==usingExtDict */ int dictSize /* note : = 0 if noDict */ ) { /* Local Variables */ const BYTE* restrict ip = (const BYTE*) source; const BYTE* ref; const BYTE* const iend = ip + inputSize; BYTE* op = (BYTE*) dest; BYTE* const oend = op + outputSize; BYTE* cpy; BYTE* oexit = op + targetOutputSize; const BYTE* const lowLimit = (const BYTE*)dest - dictSize; const BYTE* const dictEnd = (const BYTE*)dictStart + dictSize; //#define OLD #ifdef OLD const size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */ #else const size_t dec32table[] = {4-0, 4-3, 4-2, 4-3, 4-0, 4-0, 4-0, 4-0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */ #endif static const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3}; const int checkOffset = (endOnInput) && (dictSize < (int)(64 KB)); /* Special cases */ if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT; /* targetOutputSize too high => decode everything */ if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) && (*ip==0)) ? 0 : -1; /* Empty output buffer */ if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0?1:-1); /* Main Loop */ while (1) { unsigned token; size_t length; /* get runlength */ token = *ip++; if ((length=(token>>ML_BITS)) == RUN_MASK) { unsigned s; do { s = *ip++; length += s; } while (likely((endOnInput)?ipLZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */ if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)(op))) goto _output_error; /* quickfix issue 134 */ if ((endOnInput) && (sizeof(void*)==4) && unlikely((size_t)(ip+length)<(size_t)(ip))) goto _output_error; /* quickfix issue 134 */ } /* copy literals */ cpy = op+length; if (((endOnInput) && ((cpy>(partialDecoding?oexit:oend-MFLIMIT)) || (ip+length>iend-(2+1+LASTLITERALS))) ) || ((!endOnInput) && (cpy>oend-COPYLENGTH))) { if (partialDecoding) { if (cpy > oend) goto _output_error; /* Error : write attempt beyond end of output buffer */ if ((endOnInput) && (ip+length > iend)) goto _output_error; /* Error : read attempt beyond end of input buffer */ } else { if ((!endOnInput) && (cpy != oend)) goto _output_error; /* Error : block decoding must stop exactly there */ if ((endOnInput) && ((ip+length != iend) || (cpy > oend))) goto _output_error; /* Error : input must be consumed */ } memcpy(op, ip, length); ip += length; op += length; break; /* Necessarily EOF, due to parsing restrictions */ } LZ4_WILDCOPY(op, ip, cpy); ip -= (op-cpy); op = cpy; /* get offset */ LZ4_READ_LITTLEENDIAN_16(ref,cpy,ip); ip+=2; if ((checkOffset) && (unlikely(ref < lowLimit))) goto _output_error; /* Error : offset outside destination buffer */ /* get matchlength */ if ((length=(token&ML_MASK)) == ML_MASK) { unsigned s; do { if ((endOnInput) && (ip > iend-LASTLITERALS)) goto _output_error; s = *ip++; length += s; } while (s==255); //if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */ if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)op)) goto _output_error; /* quickfix issue 134 */ } /* check external dictionary */ if ((dict==usingExtDict) && (ref < (BYTE* const)dest)) { if (unlikely(op+length+MINMATCH > oend-LASTLITERALS)) goto _output_error; if (length+MINMATCH <= (size_t)(dest-(char*)ref)) { ref = dictEnd - (dest-(char*)ref); memcpy(op, ref, length+MINMATCH); op += length+MINMATCH; } else { size_t copySize = (size_t)(dest-(char*)ref); memcpy(op, dictEnd - copySize, copySize); op += copySize; copySize = length+MINMATCH - copySize; if (copySize > (size_t)((char*)op-dest)) /* overlap */ { BYTE* const cpy = op + copySize; const BYTE* ref = (BYTE*)dest; while (op < cpy) *op++ = *ref++; } else { memcpy(op, dest, copySize); op += copySize; } } continue; } /* copy repeated sequence */ if (unlikely((op-ref)<(int)STEPSIZE)) { const size_t dec64 = dec64table[(sizeof(void*)==4) ? 0 : op-ref]; op[0] = ref[0]; op[1] = ref[1]; op[2] = ref[2]; op[3] = ref[3]; #ifdef OLD op += 4, ref += 4; ref -= dec32table[op-ref]; A32(op) = A32(ref); op += STEPSIZE-4; ref -= dec64; #else ref += dec32table[op-ref]; A32(op+4) = A32(ref); op += STEPSIZE; ref -= dec64; #endif } else { LZ4_COPYSTEP(op,ref); } cpy = op + length - (STEPSIZE-4); if (unlikely(cpy>oend-COPYLENGTH-(STEPSIZE-4))) { if (cpy > oend-LASTLITERALS) goto _output_error; /* Error : last 5 bytes must be literals */ if (opdictionary = dictionary; lz4sd->dictSize = dictSize; return 1; } /* *_continue() : These decoding functions allow decompression of multiple blocks in "streaming" mode. Previously decoded blocks must still be available at the memory position where they were decoded. If it's not possible, save the relevant part of decoded data into a safe buffer, and indicate where it stands using LZ4_setDictDecode() */ int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize) { LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode; int result; result = LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize); if (result <= 0) return result; if (lz4sd->dictionary + lz4sd->dictSize == dest) { lz4sd->dictSize += result; } else { lz4sd->dictionary = dest; lz4sd->dictSize = result; } return result; } int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize) { LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode; int result; result = LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize); if (result <= 0) return result; if (lz4sd->dictionary + lz4sd->dictSize == dest) { lz4sd->dictSize += result; } else { lz4sd->dictionary = dest; lz4sd->dictSize = result; } return result; } /* Advanced decoding functions : *_usingDict() : These decoding functions work the same as "_continue" ones, the dictionary must be explicitly provided within parameters */ int LZ4_decompress_safe_usingDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize) { return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, dictStart, dictSize); } int LZ4_decompress_fast_usingDict(const char* source, char* dest, int originalSize, const char* dictStart, int dictSize) { return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, dictStart, dictSize); } /*************************************************** Obsolete Functions ***************************************************/ /* These function names are deprecated and should no longer be used. They are only provided here for compatibility with older user programs. - LZ4_uncompress is totally equivalent to LZ4_decompress_fast - LZ4_uncompress_unknownOutputSize is totally equivalent to LZ4_decompress_safe */ int LZ4_uncompress (const char* source, char* dest, int outputSize) { return LZ4_decompress_fast(source, dest, outputSize); } int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize) { return LZ4_decompress_safe(source, dest, isize, maxOutputSize); } /* Obsolete Streaming functions */ int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; } void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base) { MEM_INIT(lz4ds, 0, LZ4_STREAMSIZE); lz4ds->bufferStart = base; } int LZ4_resetStreamState(void* state, const char* inputBuffer) { if ((((size_t)state) & 3) != 0) return 1; /* Error : pointer is not aligned on 4-bytes boundary */ LZ4_init((LZ4_stream_t_internal*)state, (const BYTE*)inputBuffer); return 0; } void* LZ4_create (const char* inputBuffer) { void* lz4ds = ALLOCATOR(4, LZ4_STREAMSIZE_U32); LZ4_init ((LZ4_stream_t_internal*)lz4ds, (const BYTE*)inputBuffer); return lz4ds; } char* LZ4_slideInputBuffer (void* LZ4_Data) { LZ4_stream_t_internal* lz4ds = (LZ4_stream_t_internal*)LZ4_Data; LZ4_saveDict((LZ4_stream_t*)LZ4_Data, (char*)lz4ds->bufferStart, 64 KB); return (char*)(lz4ds->bufferStart + 64 KB); } /* Obsolete compresson functions using User-allocated state */ int LZ4_sizeofState() { return LZ4_STREAMSIZE; } int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize) { if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */ MEM_INIT(state, 0, LZ4_STREAMSIZE); if (inputSize < (int)LZ4_64KLIMIT) return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue); else return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); } int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize) { if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */ MEM_INIT(state, 0, LZ4_STREAMSIZE); if (inputSize < (int)LZ4_64KLIMIT) return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue); else return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue); } /* Obsolete streaming decompression functions */ int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int compressedSize, int maxOutputSize) { return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, withPrefix64k, NULL, 64 KB); } int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int originalSize) { return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, NULL, 64 KB); } horde_lz4-1.0.10/lib/lz4.h0000664000175000017500000003247512667627750013254 0ustar janjan/* LZ4 - Fast LZ compression algorithm Header File Copyright (C) 2011-2014, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 source repository : http://code.google.com/p/lz4/ - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c */ #pragma once #if defined (__cplusplus) extern "C" { #endif /************************************** Version **************************************/ #define LZ4_VERSION_MAJOR 1 /* for major interface/format changes */ #define LZ4_VERSION_MINOR 2 /* for minor interface/format changes */ #define LZ4_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */ /************************************** Tuning parameter **************************************/ /* * LZ4_MEMORY_USAGE : * Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) * Increasing memory usage improves compression ratio * Reduced memory usage can improve speed, due to cache effect * Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache */ #define LZ4_MEMORY_USAGE 14 /************************************** Simple Functions **************************************/ int LZ4_compress (const char* source, char* dest, int inputSize); int LZ4_decompress_safe (const char* source, char* dest, int compressedSize, int maxOutputSize); /* LZ4_compress() : Compresses 'inputSize' bytes from 'source' into 'dest'. Destination buffer must be already allocated, and must be sized to handle worst cases situations (input data not compressible) Worst case size evaluation is provided by function LZ4_compressBound() inputSize : Max supported value is LZ4_MAX_INPUT_VALUE return : the number of bytes written in buffer dest or 0 if the compression fails LZ4_decompress_safe() : compressedSize : is obviously the source size maxOutputSize : is the size of the destination buffer, which must be already allocated. return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize) If the destination buffer is not large enough, decoding will stop and output an error code (<0). If the source stream is detected malformed, the function will stop decoding and return a negative result. This function is protected against buffer overflow exploits : it never writes outside of output buffer, and never reads outside of input buffer. Therefore, it is protected against malicious data packets. */ /* Note : Should you prefer to explicitly allocate compression-table memory using your own allocation method, use the streaming functions provided below, simply reset the memory area between each call to LZ4_compress_continue() */ /************************************** Advanced Functions **************************************/ #define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */ #define LZ4_COMPRESSBOUND(isize) ((unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16) /* LZ4_compressBound() : Provides the maximum size that LZ4 may output in a "worst case" scenario (input data not compressible) primarily useful for memory allocation of output buffer. macro is also provided when result needs to be evaluated at compilation (such as stack memory allocation). isize : is the input size. Max supported value is LZ4_MAX_INPUT_SIZE return : maximum output size in a "worst case" scenario or 0, if input size is too large ( > LZ4_MAX_INPUT_SIZE) */ int LZ4_compressBound(int isize); /* LZ4_compress_limitedOutput() : Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. If it cannot achieve it, compression will stop, and result of the function will be zero. This function never writes outside of provided output buffer. inputSize : Max supported value is LZ4_MAX_INPUT_VALUE maxOutputSize : is the size of the destination buffer (which must be already allocated) return : the number of bytes written in buffer 'dest' or 0 if the compression fails */ int LZ4_compress_limitedOutput (const char* source, char* dest, int inputSize, int maxOutputSize); /* LZ4_decompress_fast() : originalSize : is the original and therefore uncompressed size return : the number of bytes read from the source buffer (in other words, the compressed size) If the source stream is malformed, the function will stop decoding and return a negative result. Destination buffer must be already allocated. Its size must be a minimum of 'originalSize' bytes. note : This function is a bit faster than LZ4_decompress_safe() It provides fast decompression and fully respect memory boundaries for properly formed compressed data. It does not provide full protection against intentionnally modified data stream. Use this function in a trusted environment (data to decode comes from a trusted source). */ int LZ4_decompress_fast (const char* source, char* dest, int originalSize); /* LZ4_decompress_safe_partial() : This function decompress a compressed block of size 'compressedSize' at position 'source' into output buffer 'dest' of size 'maxOutputSize'. The function tries to stop decompressing operation as soon as 'targetOutputSize' has been reached, reducing decompression time. return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize) Note : this number can be < 'targetOutputSize' should the compressed block to decode be smaller. Always control how many bytes were decoded. If the source stream is detected malformed, the function will stop decoding and return a negative result. This function never writes outside of output buffer, and never reads outside of input buffer. It is therefore protected against malicious data packets */ int LZ4_decompress_safe_partial (const char* source, char* dest, int compressedSize, int targetOutputSize, int maxOutputSize); /*********************************************** Experimental Streaming Compression Functions ***********************************************/ #define LZ4_STREAMSIZE_U32 ((1 << (LZ4_MEMORY_USAGE-2)) + 8) #define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U32 * sizeof(unsigned int)) /* * LZ4_stream_t * information structure to track an LZ4 stream. * important : set this structure content to zero before first use ! */ typedef struct { unsigned int table[LZ4_STREAMSIZE_U32]; } LZ4_stream_t; /* * If you prefer dynamic allocation methods, * LZ4_createStream * provides a pointer (void*) towards an initialized LZ4_stream_t structure. * LZ4_free just frees it. */ void* LZ4_createStream(); int LZ4_free (void* LZ4_stream); /* * LZ4_loadDict * Use this function to load a static dictionary into LZ4_stream. * Any previous data will be forgotten, only 'dictionary' will remain in memory. * Loading a size of 0 is allowed (same effect as init). * Return : 1 if OK, 0 if error */ int LZ4_loadDict (void* LZ4_stream, const char* dictionary, int dictSize); /* * LZ4_compress_continue * Compress data block 'source', using blocks compressed before as dictionary to improve compression ratio * Previous data blocks are assumed to still be present at their previous location. */ int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize); /* * LZ4_compress_limitedOutput_continue * Same as before, but also specify a maximum target compressed size (maxOutputSize) * If objective cannot be met, compression exits, and returns a zero. */ int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize); /* * LZ4_saveDict * If previously compressed data block is not guaranteed to remain at its previous memory location * save it into a safe place (char* safeBuffer) * Note : you don't need to call LZ4_loadDict() afterwards, * dictionary is immediately usable, you can therefore call again LZ4_compress_continue() * Return : 1 if OK, 0 if error * Note : any dictSize > 64 KB will be interpreted as 64KB. */ int LZ4_saveDict (void* LZ4_stream, char* safeBuffer, int dictSize); /************************************************ Experimental Streaming Decompression Functions ************************************************/ #define LZ4_STREAMDECODESIZE_U32 4 #define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned int)) /* * LZ4_streamDecode_t * information structure to track an LZ4 stream. * important : set this structure content to zero before first use ! */ typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; } LZ4_streamDecode_t; /* * If you prefer dynamic allocation methods, * LZ4_createStreamDecode() * provides a pointer (void*) towards an initialized LZ4_streamDecode_t structure. * LZ4_free just frees it. */ void* LZ4_createStreamDecode(); int LZ4_free (void* LZ4_stream); /* yes, it's the same one as for compression */ /* *_continue() : These decoding functions allow decompression of multiple blocks in "streaming" mode. Previously decoded blocks must still be available at the memory position where they were decoded. If it's not possible, save the relevant part of decoded data into a safe buffer, and indicate where it stands using LZ4_setDictDecode() */ int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize); int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize); /* * LZ4_setDictDecode * Use this function to instruct where to find the dictionary. * This function can be used to specify a static dictionary, * or to instruct where to find some previously decoded data saved into a different memory space. * Setting a size of 0 is allowed (same effect as no dictionary). * Return : 1 if OK, 0 if error */ int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int dictSize); /* Advanced decoding functions : *_usingDict() : These decoding functions work the same as a combination of LZ4_setDictDecode() followed by LZ4_decompress_x_continue() all together into a single function call. It doesn't use nor update an LZ4_streamDecode_t structure. */ int LZ4_decompress_safe_usingDict (const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize); int LZ4_decompress_fast_usingDict (const char* source, char* dest, int originalSize, const char* dictStart, int dictSize); /************************************** Obsolete Functions **************************************/ /* Obsolete decompression functions These function names are deprecated and should no longer be used. They are only provided here for compatibility with older user programs. - LZ4_uncompress is the same as LZ4_decompress_fast - LZ4_uncompress_unknownOutputSize is the same as LZ4_decompress_safe */ int LZ4_uncompress (const char* source, char* dest, int outputSize); int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize); /* Obsolete functions for externally allocated state; use streaming interface instead */ int LZ4_sizeofState(void); int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize); int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize); /* Obsolete streaming functions; use new streaming interface whenever possible */ void* LZ4_create (const char* inputBuffer); int LZ4_sizeofStreamState(void); int LZ4_resetStreamState(void* state, const char* inputBuffer); char* LZ4_slideInputBuffer (void* state); /* Obsolete streaming decoding functions */ int LZ4_decompress_safe_withPrefix64k (const char* source, char* dest, int compressedSize, int maxOutputSize); int LZ4_decompress_fast_withPrefix64k (const char* source, char* dest, int originalSize); #if defined (__cplusplus) } #endif horde_lz4-1.0.10/lib/lz4hc.c0000664000175000017500000007205012667627750013553 0ustar janjan/* LZ4 HC - High Compression Mode of LZ4 Copyright (C) 2011-2014, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html - LZ4 source repository : http://code.google.com/p/lz4/ */ /************************************** Tuning Parameter **************************************/ #define LZ4HC_DEFAULT_COMPRESSIONLEVEL 8 /************************************** Memory routines **************************************/ #include /* calloc, free */ #define ALLOCATOR(s) calloc(1,s) #define FREEMEM free #include /* memset, memcpy */ #define MEM_INIT memset /************************************** CPU Feature Detection **************************************/ /* 32 or 64 bits ? */ #if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \ || defined(__powerpc64__) || defined(__powerpc64le__) \ || defined(__ppc64__) || defined(__ppc64le__) \ || defined(__PPC64__) || defined(__PPC64LE__) \ || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */ # define LZ4_ARCH64 1 #else # define LZ4_ARCH64 0 #endif /* * Little Endian or Big Endian ? * Overwrite the #define below if you know your architecture endianess */ #include /* Apparently required to detect endianess */ #if defined (__GLIBC__) # include # if (__BYTE_ORDER == __BIG_ENDIAN) # define LZ4_BIG_ENDIAN 1 # endif #elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN)) # define LZ4_BIG_ENDIAN 1 #elif defined(__sparc) || defined(__sparc__) \ || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \ || defined(__hpux) || defined(__hppa) \ || defined(_MIPSEB) || defined(__s390__) # define LZ4_BIG_ENDIAN 1 #else /* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */ #endif /* * Unaligned memory access is automatically enabled for "common" CPU, such as x86. * For others CPU, the compiler will be more cautious, and insert extra code to ensure aligned access is respected * If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance */ #if defined(__ARM_FEATURE_UNALIGNED) # define LZ4_FORCE_UNALIGNED_ACCESS 1 #endif /* Define this parameter if your target system or compiler does not support hardware bit count */ #if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */ # define LZ4_FORCE_SW_BITCOUNT #endif /************************************** Compiler Options **************************************/ #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ /* "restrict" is a known keyword */ #else # define restrict /* Disable restrict */ #endif #ifdef _MSC_VER /* Visual Studio */ # define FORCE_INLINE static __forceinline # include /* For Visual 2005 */ # if LZ4_ARCH64 /* 64-bits */ # pragma intrinsic(_BitScanForward64) /* For Visual 2005 */ # pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */ # else /* 32-bits */ # pragma intrinsic(_BitScanForward) /* For Visual 2005 */ # pragma intrinsic(_BitScanReverse) /* For Visual 2005 */ # endif # pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ # pragma warning(disable : 4701) /* disable: C4701: potentially uninitialized local variable used */ #else # ifdef __GNUC__ # define FORCE_INLINE static inline __attribute__((always_inline)) # else # define FORCE_INLINE static inline # endif #endif #ifdef _MSC_VER /* Visual Studio */ # define lz4_bswap16(x) _byteswap_ushort(x) #else # define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8))) #endif /************************************** Includes **************************************/ #include "lz4hc.h" #include "lz4.h" /************************************** Basic Types **************************************/ #if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */ # include typedef uint8_t BYTE; typedef uint16_t U16; typedef uint32_t U32; typedef int32_t S32; typedef uint64_t U64; #else typedef unsigned char BYTE; typedef unsigned short U16; typedef unsigned int U32; typedef signed int S32; typedef unsigned long long U64; #endif #if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS) # define _PACKED __attribute__ ((packed)) #else # define _PACKED #endif #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # ifdef __IBMC__ # pragma pack(1) # else # pragma pack(push, 1) # endif #endif typedef struct _U16_S { U16 v; } _PACKED U16_S; typedef struct _U32_S { U32 v; } _PACKED U32_S; typedef struct _U64_S { U64 v; } _PACKED U64_S; #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # pragma pack(pop) #endif #define A64(x) (((U64_S *)(x))->v) #define A32(x) (((U32_S *)(x))->v) #define A16(x) (((U16_S *)(x))->v) /************************************** Constants **************************************/ #define MINMATCH 4 #define DICTIONARY_LOGSIZE 16 #define MAXD (1<> ((MINMATCH*8)-HASH_LOG)) #define HASH_VALUE(p) HASH_FUNCTION(A32(p)) #define HASH_POINTER(p) (HashTable[HASH_VALUE(p)] + base) #define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK] #define GETNEXT(p) ((p) - (size_t)DELTANEXT(p)) /************************************** Private functions **************************************/ #if LZ4_ARCH64 FORCE_INLINE int LZ4_NbCommonBytes (register U64 val) { #if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanReverse64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clzll(val) >> 3); # else int r; if (!(val>>32)) { r=4; } else { r=0; val>>=32; } if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } r += (!val); return r; # endif #else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanForward64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctzll(val) >> 3); # else static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 }; return DeBruijnBytePos[((U64)((val & -val) * 0x0218A392CDABBD3F)) >> 58]; # endif #endif } #else FORCE_INLINE int LZ4_NbCommonBytes (register U32 val) { #if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanReverse( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clz(val) >> 3); # else int r; if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } r += (!val); return r; # endif #else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanForward( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctz(val) >> 3); # else static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 }; return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; # endif #endif } #endif int LZ4_sizeofStreamStateHC() { return sizeof(LZ4HC_Data_Structure); } FORCE_INLINE void LZ4_initHC (LZ4HC_Data_Structure* hc4, const BYTE* base) { MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable)); MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable)); hc4->nextToUpdate = base + 1; hc4->base = base; hc4->inputBuffer = base; hc4->end = base; } int LZ4_resetStreamStateHC(void* state, const char* inputBuffer) { if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error : pointer is not aligned for pointer (32 or 64 bits) */ LZ4_initHC((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer); return 0; } void* LZ4_createHC (const char* inputBuffer) { void* hc4 = ALLOCATOR(sizeof(LZ4HC_Data_Structure)); LZ4_initHC ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer); return hc4; } int LZ4_freeHC (void* LZ4HC_Data) { FREEMEM(LZ4HC_Data); return (0); } /* Update chains up to ip (excluded) */ FORCE_INLINE void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip) { U16* chainTable = hc4->chainTable; HTYPE* HashTable = hc4->hashTable; INITBASE(base,hc4->base); while(hc4->nextToUpdate < ip) { const BYTE* const p = hc4->nextToUpdate; size_t delta = (p) - HASH_POINTER(p); if (delta>MAX_DISTANCE) delta = MAX_DISTANCE; DELTANEXT(p) = (U16)delta; HashTable[HASH_VALUE(p)] = (HTYPE)((p) - base); hc4->nextToUpdate++; } } char* LZ4_slideInputBufferHC(void* LZ4HC_Data) { LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data; U32 distance = (U32)(hc4->end - hc4->inputBuffer) - 64 KB; distance = (distance >> 16) << 16; /* Must be a multiple of 64 KB */ LZ4HC_Insert(hc4, hc4->end - MINMATCH); memcpy((void*)(hc4->end - 64 KB - distance), (const void*)(hc4->end - 64 KB), 64 KB); hc4->nextToUpdate -= distance; hc4->base -= distance; if ((U32)(hc4->inputBuffer - hc4->base) > 1 GB + 64 KB) /* Avoid overflow */ { int i; hc4->base += 1 GB; for (i=0; ihashTable[i] -= 1 GB; } hc4->end -= distance; return (char*)(hc4->end); } FORCE_INLINE size_t LZ4HC_CommonLength (const BYTE* p1, const BYTE* p2, const BYTE* const matchlimit) { const BYTE* p1t = p1; while (p1tchainTable; HTYPE* const HashTable = hc4->hashTable; const BYTE* ref; INITBASE(base,hc4->base); int nbAttempts=maxNbAttempts; size_t repl=0, ml=0; U16 delta=0; /* useless assignment, to remove an uninitialization warning */ /* HC4 match finder */ LZ4HC_Insert(hc4, ip); ref = HASH_POINTER(ip); #define REPEAT_OPTIMIZATION #ifdef REPEAT_OPTIMIZATION /* Detect repetitive sequences of length <= 4 */ if ((U32)(ip-ref) <= 4) /* potential repetition */ { if (A32(ref) == A32(ip)) /* confirmed */ { delta = (U16)(ip-ref); repl = ml = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH; *matchpos = ref; } ref = GETNEXT(ref); } #endif while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) { nbAttempts--; if (*(ref+ml) == *(ip+ml)) if (A32(ref) == A32(ip)) { size_t mlt = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH; if (mlt > ml) { ml = mlt; *matchpos = ref; } } ref = GETNEXT(ref); } #ifdef REPEAT_OPTIMIZATION /* Complete table */ if (repl) { const BYTE* ptr = ip; const BYTE* end; end = ip + repl - (MINMATCH-1); while(ptr < end-delta) { DELTANEXT(ptr) = delta; /* Pre-Load */ ptr++; } do { DELTANEXT(ptr) = delta; HashTable[HASH_VALUE(ptr)] = (HTYPE)((ptr) - base); /* Head of chain */ ptr++; } while(ptr < end); hc4->nextToUpdate = end; } #endif return (int)ml; } FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (LZ4HC_Data_Structure* hc4, const BYTE* ip, const BYTE* startLimit, const BYTE* matchlimit, int longest, const BYTE** matchpos, const BYTE** startpos, const int maxNbAttempts) { U16* const chainTable = hc4->chainTable; HTYPE* const HashTable = hc4->hashTable; INITBASE(base,hc4->base); const BYTE* ref; int nbAttempts = maxNbAttempts; int delta = (int)(ip-startLimit); /* First Match */ LZ4HC_Insert(hc4, ip); ref = HASH_POINTER(ip); while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) { nbAttempts--; if (*(startLimit + longest) == *(ref - delta + longest)) if (A32(ref) == A32(ip)) { #if 1 const BYTE* reft = ref+MINMATCH; const BYTE* ipt = ip+MINMATCH; const BYTE* startt = ip; while (iptstartLimit) && (reft > hc4->inputBuffer) && (startt[-1] == reft[-1])) {startt--; reft--;} if ((ipt-startt) > longest) { longest = (int)(ipt-startt); *matchpos = reft; *startpos = startt; } } ref = GETNEXT(ref); } return longest; } typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive; FORCE_INLINE int LZ4HC_encodeSequence ( const BYTE** ip, BYTE** op, const BYTE** anchor, int matchLength, const BYTE* ref, limitedOutput_directive limitedOutputBuffer, BYTE* oend) { int length; BYTE* token; /* Encode Literal length */ length = (int)(*ip - *anchor); token = (*op)++; if ((limitedOutputBuffer) && ((*op + length + (2 + 1 + LASTLITERALS) + (length>>8)) > oend)) return 1; /* Check output limit */ if (length>=(int)RUN_MASK) { int len; *token=(RUN_MASK< 254 ; len-=255) *(*op)++ = 255; *(*op)++ = (BYTE)len; } else *token = (BYTE)(length<>8) > oend)) return 1; /* Check output limit */ if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length > 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) { length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; } else *token += (BYTE)(length); /* Prepare next loop */ *ip += matchLength; *anchor = *ip; return 0; } #define MAX_COMPRESSION_LEVEL 16 static int LZ4HC_compress_generic ( void* ctxvoid, const char* source, char* dest, int inputSize, int maxOutputSize, int compressionLevel, limitedOutput_directive limit ) { LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid; const BYTE* ip = (const BYTE*) source; const BYTE* anchor = ip; const BYTE* const iend = ip + inputSize; const BYTE* const mflimit = iend - MFLIMIT; const BYTE* const matchlimit = (iend - LASTLITERALS); BYTE* op = (BYTE*) dest; BYTE* const oend = op + maxOutputSize; const int maxNbAttempts = compressionLevel > MAX_COMPRESSION_LEVEL ? 1 << MAX_COMPRESSION_LEVEL : compressionLevel ? 1<<(compressionLevel-1) : 1<end) return 0; ctx->end += inputSize; ip++; /* Main Loop */ while (ip < mflimit) { ml = LZ4HC_InsertAndFindBestMatch (ctx, ip, matchlimit, (&ref), maxNbAttempts); if (!ml) { ip++; continue; } /* saved, in case we would skip too much */ start0 = ip; ref0 = ref; ml0 = ml; _Search2: if (ip+ml < mflimit) ml2 = LZ4HC_InsertAndGetWiderMatch(ctx, ip + ml - 2, ip + 1, matchlimit, ml, &ref2, &start2, maxNbAttempts); else ml2 = ml; if (ml2 == ml) /* No better match */ { if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0; continue; } if (start0 < ip) { if (start2 < ip + ml0) /* empirical */ { ip = start0; ref = ref0; ml = ml0; } } /* Here, start0==ip */ if ((start2 - ip) < 3) /* First Match too small : removed */ { ml = ml2; ip = start2; ref =ref2; goto _Search2; } _Search3: /* * Currently we have : * ml2 > ml1, and * ip1+3 <= ip2 (usually < ip1+ml1) */ if ((start2 - ip) < OPTIMAL_ML) { int correction; int new_ml = ml; if (new_ml > OPTIMAL_ML) new_ml = OPTIMAL_ML; if (ip+new_ml > start2 + ml2 - MINMATCH) new_ml = (int)(start2 - ip) + ml2 - MINMATCH; correction = new_ml - (int)(start2 - ip); if (correction > 0) { start2 += correction; ref2 += correction; ml2 -= correction; } } /* Now, we have start2 = ip+new_ml, with new_ml = min(ml, OPTIMAL_ML=18) */ if (start2 + ml2 < mflimit) ml3 = LZ4HC_InsertAndGetWiderMatch(ctx, start2 + ml2 - 3, start2, matchlimit, ml2, &ref3, &start3, maxNbAttempts); else ml3 = ml2; if (ml3 == ml2) /* No better match : 2 sequences to encode */ { /* ip & ref are known; Now for ml */ if (start2 < ip+ml) ml = (int)(start2 - ip); /* Now, encode 2 sequences */ if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0; ip = start2; if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml2, ref2, limit, oend)) return 0; continue; } if (start3 < ip+ml+3) /* Not enough space for match 2 : remove it */ { if (start3 >= (ip+ml)) /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */ { if (start2 < ip+ml) { int correction = (int)(ip+ml - start2); start2 += correction; ref2 += correction; ml2 -= correction; if (ml2 < MINMATCH) { start2 = start3; ref2 = ref3; ml2 = ml3; } } if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0; ip = start3; ref = ref3; ml = ml3; start0 = start2; ref0 = ref2; ml0 = ml2; goto _Search2; } start2 = start3; ref2 = ref3; ml2 = ml3; goto _Search3; } /* * OK, now we have 3 ascending matches; let's write at least the first one * ip & ref are known; Now for ml */ if (start2 < ip+ml) { if ((start2 - ip) < (int)ML_MASK) { int correction; if (ml > OPTIMAL_ML) ml = OPTIMAL_ML; if (ip + ml > start2 + ml2 - MINMATCH) ml = (int)(start2 - ip) + ml2 - MINMATCH; correction = ml - (int)(start2 - ip); if (correction > 0) { start2 += correction; ref2 += correction; ml2 -= correction; } } else { ml = (int)(start2 - ip); } } if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0; ip = start2; ref = ref2; ml = ml2; start2 = start3; ref2 = ref3; ml2 = ml3; goto _Search3; } /* Encode Last Literals */ { int lastRun = (int)(iend - anchor); if ((limit) && (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize)) return 0; /* Check output limit */ if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK< 254 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; } else *op++ = (BYTE)(lastRun< The memory position where the next input data block must start is provided as the result of the function. Compression can then resume, using LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue(), as usual. When compression is completed, a call to LZ4_freeHC() will release the memory used by the LZ4HC Data Structure. */ int LZ4_sizeofStreamStateHC(void); int LZ4_resetStreamStateHC(void* state, const char* inputBuffer); /* These functions achieve the same result as : void* LZ4_createHC (const char* inputBuffer); They are provided here to allow the user program to allocate memory using its own routines. To know how much space must be allocated, use LZ4_sizeofStreamStateHC(); Note also that space must be aligned for pointers (32 or 64 bits). Once space is allocated, you must initialize it using : LZ4_resetStreamStateHC(void* state, const char* inputBuffer); void* state is a pointer to the space allocated. It must be aligned for pointers (32 or 64 bits), and be large enough. The parameter 'const char* inputBuffer' must, obviously, point at the beginning of input buffer. The input buffer must be already allocated, and size at least 192KB. 'inputBuffer' will also be the 'const char* source' of the first block. The same space can be re-used multiple times, just by initializing it each time with LZ4_resetStreamState(). return value of LZ4_resetStreamStateHC() must be 0 is OK. Any other value means there was an error (typically, state is not aligned for pointers (32 or 64 bits)). */ #if defined (__cplusplus) } #endif horde_lz4-1.0.10/config.m40000664000175000017500000000452312667627750013324 0ustar janjandnl config.m4 for extension horde_lz4 dnl Check PHP version: AC_MSG_CHECKING(PHP version) if test ! -z "$phpincludedir"; then PHP_VERSION=`grep 'PHP_VERSION ' $phpincludedir/main/php_version.h | sed -e 's/.*"\([[0-9\.]]*\).*".*/\1/g' 2>/dev/null` elif test ! -z "$PHP_CONFIG"; then PHP_VERSION=`$PHP_CONFIG --version 2>/dev/null` fi if test x"$PHP_VERSION" = "x"; then AC_MSG_WARN([none]) else PHP_MAJOR_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\1/g' 2>/dev/null` PHP_MINOR_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\2/g' 2>/dev/null` PHP_RELEASE_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\3/g' 2>/dev/null` AC_MSG_RESULT([$PHP_VERSION]) fi if test $PHP_MAJOR_VERSION -lt 5; then AC_MSG_ERROR([need at least PHP 5 or newer]) fi PHP_ARG_ENABLE(horde_lz4, whether to enable horde_lz4 support, [ --enable-horde_lz4 Enable horde_lz4 support]) PHP_ARG_WITH(liblz4, whether to use system liblz4, [ --with-liblz4 Use system liblz4], no, no) if test "$PHP_HORDE_LZ4" != "no"; then sources=horde_lz4.c if test "$PHP_LIBLZ4" != "no"; then AC_PATH_PROG(PKG_CONFIG, pkg-config, no) AC_MSG_CHECKING(liblz4 version) if test -x "$PKG_CONFIG" && $PKG_CONFIG --exists liblz4; then LIBLZ4_INCLUDE=`$PKG_CONFIG liblz4 --cflags` LIBLZ4_LIBRARY=`$PKG_CONFIG liblz4 --libs` LIBLZ4_VERSION=`$PKG_CONFIG liblz4 --modversion` fi if test -z "$LIBLZ4_VERSION"; then AC_MSG_RESULT(liblz4.pc not found) AC_CHECK_HEADERS([lz4.h]) PHP_CHECK_LIBRARY(lz4, LZ4_decompress_fast, [PHP_ADD_LIBRARY(lz4, 1, HORDE_LZ4_SHARED_LIBADD)], [AC_MSG_ERROR(lz4 library not found)]) else AC_MSG_RESULT($LIBLZ4_VERSION) PHP_EVAL_INCLINE($LIBLZ4_INCLUDE) PHP_EVAL_LIBLINE($LIBLZ4_LIBRARY, HORDE_LZ4_SHARED_LIBADD) fi PHP_NEW_EXTENSION(horde_lz4, $sources, $ext_shared) PHP_SUBST(HORDE_LZ4_SHARED_LIBADD) else sources="$sources lib/lz4.c lib/lz4hc.c" PHP_NEW_EXTENSION(horde_lz4, $sources, $ext_shared) PHP_ADD_BUILD_DIR($ext_builddir/lib, 1) PHP_ADD_INCLUDE([$ext_srcdir/lib]) fi ifdef([PHP_INSTALL_HEADERS], [ PHP_INSTALL_HEADERS([ext/horde_lz4/], [horde_lz4.h]) ], [ PHP_ADD_MAKEFILE_FRAGMENT ]) fi horde_lz4-1.0.10/CREDITS0000664000175000017500000000060412667627750012631 0ustar janjanhorde_lz4 extension This package is maintained by The Horde Project (http://www.horde.org/). Copyright 2013-2016 Horde LLC The original PHP extension package was created by kamijo https://github.com/kjdev/php-ext-lz4 The php-ext-lz4 package is released under the MIT (Expat) License. The LZ4 C source was created by Yann Collet and released under the BSD license horde_lz4-1.0.10/horde_lz4.c0000664000175000017500000000721612667627750013655 0ustar janjan#ifdef HAVE_CONFIG_H #include "config.h" #endif #include "php.h" #include "php_ini.h" #include "ext/standard/info.h" #include "horde_lz4.h" /* lz4 */ #include #include ZEND_BEGIN_ARG_INFO_EX(arginfo_horde_lz4_compress, 0, 0, 1) ZEND_ARG_INFO(0, data) ZEND_ARG_INFO(0, high) ZEND_END_ARG_INFO() ZEND_BEGIN_ARG_INFO_EX(arginfo_horde_lz4_uncompress, 0, 0, 1) ZEND_ARG_INFO(0, data) ZEND_END_ARG_INFO() const zend_function_entry horde_lz4_functions[] = { PHP_FE(horde_lz4_compress, arginfo_horde_lz4_compress) PHP_FE(horde_lz4_uncompress, arginfo_horde_lz4_uncompress) /* Fix for PHP < 5.3.10 */ #ifdef PHP_FE_END PHP_FE_END #else { NULL, NULL, NULL, 0, 0 } #endif }; zend_module_entry horde_lz4_module_entry = { #if ZEND_MODULE_API_NO >= 20010901 STANDARD_MODULE_HEADER, #endif "horde_lz4", horde_lz4_functions, NULL, NULL, NULL, NULL, PHP_MINFO(horde_lz4), #if ZEND_MODULE_API_NO >= 20010901 HORDE_LZ4_EXT_VERSION, #endif STANDARD_MODULE_PROPERTIES }; #ifdef COMPILE_DL_HORDE_LZ4 ZEND_GET_MODULE(horde_lz4) #endif PHP_MINFO_FUNCTION(horde_lz4) { php_info_print_table_start(); php_info_print_table_row(2, "Horde LZ4 support", "enabled"); php_info_print_table_row(2, "Extension Version", HORDE_LZ4_EXT_VERSION); php_info_print_table_end(); } static char horde_lz4_headerid = 'H'; PHP_FUNCTION(horde_lz4_compress) { zval *data; char *output; int data_len, output_len; int header_offset = (sizeof(horde_lz4_headerid) + sizeof(data_len)); zend_bool high = 0; if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z|b", &data, &high) == FAILURE) { RETURN_FALSE; } if (Z_TYPE_P(data) != IS_STRING) { zend_error(E_WARNING, "horde_lz4_compress: uncompressed data must be a string."); RETURN_FALSE; } data_len = (int)Z_STRLEN_P(data); output = (char *)emalloc(LZ4_compressBound(data_len) + header_offset); if (!output) { zend_error(E_WARNING, "horde_lz4_compress: memory error"); RETURN_FALSE; } output[0] = horde_lz4_headerid; memcpy(output + sizeof(horde_lz4_headerid), &data_len, sizeof(data_len)); if (high) { output_len = LZ4_compressHC(Z_STRVAL_P(data), output + header_offset, data_len); } else { output_len = LZ4_compress(Z_STRVAL_P(data), output + header_offset, data_len); } if (output_len <= 0) { RETVAL_FALSE; } else { HORDE_LZ4_RETSTRL(output, output_len + header_offset); } efree(output); } PHP_FUNCTION(horde_lz4_uncompress) { zval *data; int data_len = 0; int output_len; int header_offset = (sizeof(horde_lz4_headerid) + sizeof(data_len)); char *output, *p; if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z", &data) == FAILURE) { RETURN_FALSE; } if (Z_TYPE_P(data) != IS_STRING) { zend_error(E_WARNING, "horde_lz4_uncompress: compressed data must be a string."); RETURN_FALSE; } p = Z_STRVAL_P(data); /* Check for header information. */ if (p[0] == horde_lz4_headerid) { memcpy(&data_len, p + sizeof(horde_lz4_headerid), sizeof(data_len)); } /* Header information not found. */ if (data_len <= 0) { RETURN_FALSE; } output = (char *)emalloc(data_len + 1); if (!output) { RETURN_FALSE; } output_len = LZ4_decompress_fast(p + header_offset, output, data_len); if (output_len <= 0) { RETVAL_FALSE; } else { HORDE_LZ4_RETSTRL(output, data_len); } efree(output); } horde_lz4-1.0.10/horde_lz4.h0000664000175000017500000000135712667627750013662 0ustar janjan#ifndef PHP_HORDE_LZ4_H #define PHP_HORDE_LZ4_H extern zend_module_entry horde_lz4_module_entry; #define phpext_horde_lz4_ptr &horde_lz4_module_entry #define HORDE_LZ4_EXT_VERSION "1.0.10" #ifdef PHP_WIN32 # define PHP_HORDE_LZ4_API __declspec(dllexport) #elif defined(__GNUC__) && __GNUC__ >= 4 # define PHP_HORDE_LZ4_API __attribute__ ((visibility("default"))) #else # define PHP_HORDE_LZ4_API #endif #ifdef ZTS #include "TSRM.h" #endif PHP_MINFO_FUNCTION(horde_lz4); PHP_FUNCTION(horde_lz4_compress); PHP_FUNCTION(horde_lz4_uncompress); #if PHP_MAJOR_VERSION < 7 #define HORDE_LZ4_RETSTRL(a,l) RETURN_STRINGL(a,l,1) #else typedef size_t strsize; #define HORDE_LZ4_RETSTRL(a,l) RETURN_STRINGL(a,l) #endif #endif /* PHP_HORDE_LZ4_H */ horde_lz4-1.0.10/LICENSE0000664000175000017500000000203112667627750012612 0ustar janjanCopyright (c) 2013 kjdev Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. horde_lz4-1.0.10/README.md0000664000175000017500000000211412667627750013066 0ustar janjan# Horde LZ4 Extension for PHP # This extension allows LZ4 compression. Documentation for LZ4 can be found at [» http://code.google.com/p/lz4/](http://code.google.com/p/lz4/). ## Configration ## php.ini: extension=horde_lz4.so ## Function ## * horde\_lz4\_compress — LZ4 compression * horde\_lz4\_uncompress — LZ4 decompression ### horde\_lz4\_compress — LZ4 compression ### #### Description #### string **horde\_lz4\_compress** (string _$data_ [, bool _$high_ = false]) LZ4 compression. #### Pameters #### * _data_ The string to compress. * _high_ High Compression Mode. #### Return Values #### Returns the compressed data or FALSE if an error occurred. ### horde\_lz4\_uncompress — LZ4 decompression ### #### Description #### string **horde\_lz4\_uncompress** (string _$data_) LZ4 decompression. #### Pameters #### * _data_ The compressed string. #### Return Values #### Returns the decompressed data or FALSE if an error occurred. ## Examples ## $compressed = horde_lz4_compress('test'); $uncompressed = horde_lz4_uncompress($compressed);