package.xml0000644000175000007640000001176012177042237013201 0ustar slusarzslusarz horde_lz4 pear.horde.org Horde LZ4 Compression Extension PHP extension that implements the LZ4 compression algorithm - an extremely fast lossless compression algorithm. Michael Slusarz slusarz slusarz@horde.org yes 2013-08-02 1.0.2 1.0.0 stable stable MIT (Expat) * [mms] Update LZ4 source to r99. 5.0.0 1.7.0 horde_lz4 2013-03-03 1.0.0beta1 1.0.0 beta beta PHP 3.01 * [mms] Initial release of package. 1.0.0beta2 1.0.0 beta beta 2013-03-04 PHP 3.01 * [mms] More robust header checking: eliminates the need to explicitly pass decompression size to horde_lz4_uncompress(). * [mms] Remove maxsize option to horde_lz4_uncompress(). * [mms] Remove 'prefix' option to horde_lz4_compress(). * [mms] Be silent when encountering decompression errors. 1.0.0 1.0.0 stable stable 2013-05-06 PHP 3.01 * First stable release. * [mms] Convert unit tests to PHPUnit. 1.0.1 1.0.0 stable stable 2013-07-08 MIT (Expat) * [mms] Change license from PHP -> MIT. 1.0.2 1.0.0 stable stable 2013-08-02 MIT (Expat) * [mms] Update LZ4 source to r99. horde_lz4-1.0.2/test/horde/lz4/fixtures/data.txt0000644000175000007640000000673412177042237021533 0ustar slusarzslusarzTo be or not to be, that is the question; Whether 'tis nobler in the mind to suffer The Slings and Arrows of outrageous Fortune Or to take arms against a sea of troubles, And by opposing, end them. To die, to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to 'tis a consummation Devoutly to be wish'd. To die, to sleep; To sleep, perchance to dream. Ay, there's the rub, For in that sleep of death what dreams may come, When we have shuffled off this mortal coil, Must give us pause. There's the respect That makes calamity of so long life, For who would bear the whips and scorns of time, Th'oppressor's wrong, the proud man's contumely, The pangs of dispriz'd love, the law's delay, The insolence of office, and the spurns That patient merit of th'unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, But that the dread of something after death, The undiscovered country from whose bourn No traveller returns, puzzles the will, And makes us rather bear those ills we have Than fly to others that we know not of? Thus conscience does make cowards of us all, And thus the native hue of resolution Is sicklied o'er with the pale cast of thought, And enterprises of great pitch and moment With this regard their currents turn away, And lose the name of action. Hath not a Jew eyes? Hath not a Jew hands, organs, dimensions, senses, affections, passions; fed with the same food, hurt with the same weapons, subject to the same diseases, heal'd by the same means, warm'd and cool'd by the same winter and summer as a Christian is? If you prick us, do we not bleed? If you tickle us, do we not laugh? If you poison us, do we not die? And if you wrong us, shall we not revenge? If we are like you in the rest, we will resemble you in that. If a Jew wrong a Christian, what is his humility? Revenge. If a Christian wrong a Jew, what should his sufferance be by Christian example? Why, revenge. The villainy you teach me, I will execute, and it shall go hard but I will better the instruction. Is this a dagger which I see before me, The handle toward my hand? Come, let me clutch thee. I have thee not, and yet I see thee still. Art thou not, fatal vision, sensible To feeling as to sight? or art thou but A dagger of the mind, a false creation, Proceeding from the heat-oppress'd brain? I see thee yet, in form as palpable As this which now I draw. Thou marshall'st me the way that I was going; And such an instrument I was to use. Mine eyes are made the fools o' the other senses, Or else worth all the rest; I see thee still, And on thy blade and dudgeon gouts of blood, Which was not so before. There's no such thing: It is the bloody business which informs Thus to mine eyes. Now o'er the one halfworld Nature seems dead, and wicked dreams abuse The curtain'd sleep; witchcraft celebrates Pale Hecate's offerings, and wither'd murder, Alarum'd by his sentinel, the wolf, Whose howl's his watch, thus with his stealthy pace. With Tarquin's ravishing strides, towards his design Moves like a ghost. Thou sure and firm-set earth, Hear not my steps, which way they walk, for fear Thy very stones prate of my whereabout, And take the present horror from the time, Which now suits with it. Whiles I threat, he lives: Words to the heat of deeds too cold breath gives. I go, and it is done; the bell invites me. Hear it not, Duncan; for it is a knell That summons thee to heaven or to hell. horde_lz4-1.0.2/test/horde/lz4/AllTests.php0000644000175000007640000000013212177042237020436 0ustar slusarzslusarzrun(); horde_lz4-1.0.2/test/horde/lz4/bootstrap.php0000644000175000007640000000014312177042237020722 0ustar slusarzslusarz horde_lz4-1.0.2/test/horde/lz4/UnitTest.php0000644000175000007640000001202412177042237020465 0ustar slusarzslusarz * @category Horde * @copyright 2013 Horde LLC * @ignore * @license http://www.php.net/license/3_01.txt PHP 3.01 * @package horde_lz4 * @subpackage UnitTests */ class horde_lz4_UnitTest extends PHPUnit_Framework_TestCase { private $data; public function setUp() { if (!extension_loaded('horde_lz4')) { $this->markTestSkipped('horde_lz4 extension not installed.'); } $this->data = file_get_contents(__DIR__ . '/fixtures/data.txt'); } // Test horde_lz4_compress() function : basic functionality public function testCompressBasic() { // Compressing a big string $output = horde_lz4_compress($this->data); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $this->data) ); // Compressing a smaller string $smallstring = "A small string to compress\n"; $output = horde_lz4_compress($smallstring); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $smallstring) ); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionOne() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Zero arguments horde_lz4_compress(); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionTwo() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Test horde_lz4_compress with one more than the expected number // of arguments $data = 'string_val'; $extra_arg = 10; horde_lz4_compress($data, false, $extra_arg); } // Test horde_lz4_compress() function : error conditions public function testCompressErrorConditionThree() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); horde_lz4_compress(new stdClass); } // Test horde_lz4_compress() function : variation public function testCompressVariation() { $output = horde_lz4_compress($this->data); $this->assertNotEquals( md5($output), md5(horde_lz4_compress($output)) ); } // Test horde_lz4_uncompress() function : basic functionality public function testUncompressBasic() { $compressed = horde_lz4_compress($this->data); $this->assertEquals( 0, strcmp($this->data, horde_lz4_uncompress($compressed)) ); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionOne() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Zero arguments horde_lz4_uncompress(); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionTwo() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Test horde_lz4_uncompress with one more than the expected number // of arguments $data = 'string_val'; $extra_arg = 10; horde_lz4_uncompress($data, $extra_arg); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionThree() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); horde_lz4_uncompress(new stdClass); } // Test horde_lz4_uncompress() function : error conditions public function testUncompressErrorConditionFour() { $this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Testing with incorrect arguments horde_lz4_uncompress(123); } // Test horde_lz4_compress() function : high compression public function testHighCompression() { // Compressing a big string $output = horde_lz4_compress($this->data, true); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $this->data) ); // Compressing a smaller string $smallstring = "A small string to compress\n"; $output = horde_lz4_compress($smallstring, true); $this->assertEquals( 0, strcmp(horde_lz4_uncompress($output), $smallstring) ); } // Test horde_lz4_uncompress() function : bad input (non-lz4 data) public function testUncompressBadInput() { // Bad data is missing the Horde-LZ4 header and is not LZ4 data. $bad_data = "12345678"; $this->assertFalse(horde_lz4_uncompress($bad_data)); } } horde_lz4-1.0.2/config.m40000644000175000007640000000234712177042237015122 0ustar slusarzslusarzdnl config.m4 for extension horde_lz4 dnl Check PHP version: AC_MSG_CHECKING(PHP version) if test ! -z "$phpincludedir"; then PHP_VERSION=`grep 'PHP_VERSION ' $phpincludedir/main/php_version.h | sed -e 's/.*"\([[0-9\.]]*\)".*/\1/g' 2>/dev/null` elif test ! -z "$PHP_CONFIG"; then PHP_VERSION=`$PHP_CONFIG --version 2>/dev/null` fi if test x"$PHP_VERSION" = "x"; then AC_MSG_WARN([none]) else PHP_MAJOR_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\1/g' 2>/dev/null` PHP_MINOR_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\2/g' 2>/dev/null` PHP_RELEASE_VERSION=`echo $PHP_VERSION | sed -e 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\3/g' 2>/dev/null` AC_MSG_RESULT([$PHP_VERSION]) fi if test $PHP_MAJOR_VERSION -lt 5; then AC_MSG_ERROR([need at least PHP 5 or newer]) fi PHP_ARG_ENABLE(horde_lz4, whether to enable horde_lz4 support, [ --enable-horde_lz4 Enable horde_lz4 support]) if test "$PHP_HORDE_LZ4" != "no"; then PHP_NEW_EXTENSION(horde_lz4, horde_lz4.c lz4.c lz4hc.c, $ext_shared) ifdef([PHP_INSTALL_HEADERS], [ PHP_INSTALL_HEADERS([ext/horde_lz4/], [horde_lz4.h]) ], [ PHP_ADD_MAKEFILE_FRAGMENT ]) fi horde_lz4-1.0.2/CREDITS0000644000175000007640000000057712177042237014436 0ustar slusarzslusarzhorde_lz4 extension This package is maintained by The Horde Project (http://www.horde.org/). Copyright 2013 Horde LLC The original PHP extension package was created by kamijo https://github.com/kjdev/php-ext-lz4 The php-ext-lz4 package is released under the MIT (Expat) License. The LZ4 C source was created by Yann Collet and released under the BSD license horde_lz4-1.0.2/horde_lz4.c0000644000175000007640000000710612177042237015447 0ustar slusarzslusarz#ifdef HAVE_CONFIG_H #include "config.h" #endif #include "php.h" #include "php_ini.h" #include "ext/standard/info.h" #include "php_verdep.h" #include "horde_lz4.h" /* lz4 */ #include "lz4.h" #include "lz4hc.h" static ZEND_FUNCTION(horde_lz4_compress); static ZEND_FUNCTION(horde_lz4_uncompress); ZEND_BEGIN_ARG_INFO_EX(arginfo_horde_lz4_compress, 0, 0, 1) ZEND_ARG_INFO(0, data) ZEND_ARG_INFO(0, high) ZEND_END_ARG_INFO() ZEND_BEGIN_ARG_INFO_EX(arginfo_horde_lz4_uncompress, 0, 0, 1) ZEND_ARG_INFO(0, data) ZEND_END_ARG_INFO() static zend_function_entry horde_lz4_functions[] = { ZEND_FE(horde_lz4_compress, arginfo_horde_lz4_compress) ZEND_FE(horde_lz4_uncompress, arginfo_horde_lz4_uncompress) ZEND_FE_END }; ZEND_MINFO_FUNCTION(horde_lz4) { php_info_print_table_start(); php_info_print_table_row(2, "Horde LZ4 support", "enabled"); php_info_print_table_row(2, "Extension Version", HORDE_LZ4_EXT_VERSION); php_info_print_table_end(); } zend_module_entry horde_lz4_module_entry = { #if ZEND_MODULE_API_NO >= 20010901 STANDARD_MODULE_HEADER, #endif "horde_lz4", horde_lz4_functions, NULL, NULL, NULL, NULL, ZEND_MINFO(horde_lz4), #if ZEND_MODULE_API_NO >= 20010901 HORDE_LZ4_EXT_VERSION, #endif STANDARD_MODULE_PROPERTIES }; #ifdef COMPILE_DL_HORDE_LZ4 ZEND_GET_MODULE(horde_lz4) #endif char headerid = 'H'; static ZEND_FUNCTION(horde_lz4_compress) { zval *data; char *output; int header_offset = (sizeof(headerid) + sizeof(int)); int output_len, data_len; zend_bool high = 0; if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z|b", &data, &high) == FAILURE) { RETURN_FALSE; } if (Z_TYPE_P(data) != IS_STRING) { zend_error(E_WARNING, "horde_lz4_compress: uncompressed data must be a string."); RETURN_FALSE; } data_len = Z_STRLEN_P(data); output = (char *)emalloc(LZ4_compressBound(data_len) + header_offset); if (!output) { zend_error(E_WARNING, "horde_lz4_compress: memory error"); RETURN_FALSE; } *output = headerid; memcpy(output + sizeof(headerid), &data_len, sizeof(int)); if (high) { output_len = LZ4_compressHC(Z_STRVAL_P(data), output + header_offset, data_len); } else { output_len = LZ4_compress(Z_STRVAL_P(data), output + header_offset, data_len); } if (output_len <= 0) { RETVAL_FALSE; } else { RETVAL_STRINGL(output, output_len + header_offset, 1); } efree(output); } static ZEND_FUNCTION(horde_lz4_uncompress) { zval *data; int data_len = 0; int header_offset = (sizeof(headerid) + sizeof(int)); int output_len; char *output, *p; if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z", &data) == FAILURE) { RETURN_FALSE; } if (Z_TYPE_P(data) != IS_STRING) { zend_error(E_WARNING, "horde_lz4_uncompress: compressed data must be a string."); RETURN_FALSE; } p = Z_STRVAL_P(data); /* Check for header information. */ if (p[0] == headerid) { memcpy(&data_len, p + sizeof(headerid), sizeof(int)); } /* Header information not found. */ if (data_len <= 0) { RETURN_FALSE; } output = (char *)emalloc(data_len + 1); if (!output) { RETURN_FALSE; } output_len = LZ4_uncompress(p + header_offset, output, data_len); if (output_len <= 0) { RETVAL_FALSE; } else { RETVAL_STRINGL(output, data_len, 1); } efree(output); } horde_lz4-1.0.2/horde_lz4.h0000644000175000007640000000121312177042237015445 0ustar slusarzslusarz#ifndef PHP_HORDE_LZ4_H #define PHP_HORDE_LZ4_H #define HORDE_LZ4_EXT_VERSION "1.0.2" extern char headerid; extern zend_module_entry horde_lz4_module_entry; #define phpext_horde_lz4_ptr &horde_lz4_module_entry #ifdef PHP_WIN32 # define PHP_HORDE_LZ4_API __declspec(dllexport) #elif defined(__GNUC__) && __GNUC__ >= 4 # define PHP_HORDE_LZ4_API __attribute__ ((visibility("default"))) #else # define PHP_HORDE_LZ4_API #endif #ifdef ZTS #include "TSRM.h" #endif #ifdef ZTS #define HORDE_LZ4_G(v) TSRMG(horde_lz4_globals_id, zend_horde_lz4_globals *, v) #else #define HORDE_LZ4_G(v) (horde_lz4_globals.v) #endif #endif /* PHP_HORDE_LZ4_H */ horde_lz4-1.0.2/LICENSE0000644000175000007640000000203112177042237014406 0ustar slusarzslusarzCopyright (c) 2013 kjdev Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. horde_lz4-1.0.2/LICENSE.lz40000644000175000007640000000300612177042237015121 0ustar slusarzslusarzLZ4 - Fast LZ compression algorithm Copyright (C) 2011-2012, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html - LZ4 source repository : http://code.google.com/p/lz4/ horde_lz4-1.0.2/lz4.c0000644000175000007640000006071612177042237014274 0ustar slusarzslusarz/* LZ4 - Fast LZ compression algorithm Copyright (C) 2011-2013, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html - LZ4 source repository : http://code.google.com/p/lz4/ */ /* Note : this source file requires "lz4_encoder.h" */ //************************************** // Tuning parameters //************************************** // MEMORY_USAGE : // Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) // Increasing memory usage improves compression ratio // Reduced memory usage can improve speed, due to cache effect // Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache #define MEMORY_USAGE 14 // HEAPMODE : // Select how default compression function will allocate memory for its hash table, // in memory stack (0:default, fastest), or in memory heap (1:requires memory allocation (malloc)). // Default allocation strategy is to use stack (HEAPMODE 0) // Note : explicit functions *_stack* and *_heap* are unaffected by this setting #define HEAPMODE 0 //************************************** // CPU Feature Detection //************************************** // 32 or 64 bits ? #if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \ || defined(__powerpc64__) || defined(__ppc64__) || defined(__PPC64__) \ || defined(__64BIT__) || defined(_LP64) || defined(__LP64__) \ || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) // Detects 64 bits mode # define LZ4_ARCH64 1 #else # define LZ4_ARCH64 0 #endif // Little Endian or Big Endian ? // Overwrite the #define below if you know your architecture endianess #if defined (__GLIBC__) # include # if (__BYTE_ORDER == __BIG_ENDIAN) # define LZ4_BIG_ENDIAN 1 # endif #elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN)) # define LZ4_BIG_ENDIAN 1 #elif defined(__sparc) || defined(__sparc__) \ || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \ || defined(__hpux) || defined(__hppa) \ || defined(_MIPSEB) || defined(__s390__) # define LZ4_BIG_ENDIAN 1 #else // Little Endian assumed. PDP Endian and other very rare endian format are unsupported. #endif // Unaligned memory access is automatically enabled for "common" CPU, such as x86. // For others CPU, such as ARM, the compiler may be more cautious, inserting unnecessary extra code to ensure aligned access property // If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance #if defined(__ARM_FEATURE_UNALIGNED) # define LZ4_FORCE_UNALIGNED_ACCESS 1 #endif // Define this parameter if your target system or compiler does not support hardware bit count #if defined(_MSC_VER) && defined(_WIN32_WCE) // Visual Studio for Windows CE does not support Hardware bit count # define LZ4_FORCE_SW_BITCOUNT #endif // BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE : // This option may provide a small boost to performance for some big endian cpu, although probably modest. // You may set this option to 1 if data will remain within closed environment. // This option is useless on Little_Endian CPU (such as x86) //#define BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE 1 //************************************** // Compiler Options //************************************** #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) // C99 /* "restrict" is a known keyword */ #else # define restrict // Disable restrict #endif #ifdef _MSC_VER // Visual Studio # define forceinline static __forceinline # include // For Visual 2005 # if LZ4_ARCH64 // 64-bits # pragma intrinsic(_BitScanForward64) // For Visual 2005 # pragma intrinsic(_BitScanReverse64) // For Visual 2005 # else // 32-bits # pragma intrinsic(_BitScanForward) // For Visual 2005 # pragma intrinsic(_BitScanReverse) // For Visual 2005 # endif # pragma warning(disable : 4127) // disable: C4127: conditional expression is constant #else # ifdef __GNUC__ # define forceinline static inline __attribute__((always_inline)) # else # define forceinline static inline # endif #endif #ifdef _MSC_VER # define lz4_bswap16(x) _byteswap_ushort(x) #else # define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8))) #endif #define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) #if (GCC_VERSION >= 302) || (__INTEL_COMPILER >= 800) || defined(__clang__) # define expect(expr,value) (__builtin_expect ((expr),(value)) ) #else # define expect(expr,value) (expr) #endif #define likely(expr) expect((expr) != 0, 1) #define unlikely(expr) expect((expr) != 0, 0) //************************************** // Includes //************************************** #include // for malloc #include // for memset #include "lz4.h" //************************************** // Basic Types //************************************** #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L // C99 # include typedef uint8_t BYTE; typedef uint16_t U16; typedef uint32_t U32; typedef int32_t S32; typedef uint64_t U64; #else typedef unsigned char BYTE; typedef unsigned short U16; typedef unsigned int U32; typedef signed int S32; typedef unsigned long long U64; #endif #if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS) # define _PACKED __attribute__ ((packed)) #else # define _PACKED #endif #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # ifdef __IBMC__ # pragma pack(1) # else # pragma pack(push, 1) # endif #endif typedef struct { U16 v; } _PACKED U16_S; typedef struct { U32 v; } _PACKED U32_S; typedef struct { U64 v; } _PACKED U64_S; typedef struct {size_t v;} _PACKED size_t_S; #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # pragma pack(pop) #endif #define A16(x) (((U16_S *)(x))->v) #define A32(x) (((U32_S *)(x))->v) #define A64(x) (((U64_S *)(x))->v) #define AARCH(x) (((size_t_S *)(x))->v) //************************************** // Constants //************************************** #define HASHTABLESIZE (1 << MEMORY_USAGE) #define MINMATCH 4 #define COPYLENGTH 8 #define LASTLITERALS 5 #define MFLIMIT (COPYLENGTH+MINMATCH) #define MINLENGTH (MFLIMIT+1) #define LZ4_64KLIMIT ((1<<16) + (MFLIMIT-1)) #define SKIPSTRENGTH 6 // Increasing this value will make the compression run slower on incompressible data #define MAXD_LOG 16 #define MAX_DISTANCE ((1 << MAXD_LOG) - 1) #define ML_BITS 4 #define ML_MASK ((1U<>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clzll(val) >> 3); # else int r; if (!(val>>32)) { r=4; } else { r=0; val>>=32; } if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } r += (!val); return r; # endif # else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanForward64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctzll(val) >> 3); # else static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 }; return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; # endif # endif } #else forceinline int LZ4_NbCommonBytes (register U32 val) { # if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanReverse( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clz(val) >> 3); # else int r; if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } r += (!val); return r; # endif # else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanForward( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctz(val) >> 3); # else static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 }; return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; # endif # endif } #endif //****************************** // Compression functions //****************************** /* int LZ4_compress_stack( const char* source, char* dest, int inputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest'. Destination buffer must be already allocated, and sized at a minimum of LZ4_compressBound(inputSize). return : the number of bytes written in buffer 'dest' */ #define FUNCTION_NAME LZ4_compress_stack #include "lz4_encoder.h" /* int LZ4_compress_stack_limitedOutput( const char* source, char* dest, int inputSize, int maxOutputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. If it cannot achieve it, compression will stop, and result of the function will be zero. return : the number of bytes written in buffer 'dest', or 0 if the compression fails */ #define FUNCTION_NAME LZ4_compress_stack_limitedOutput #define LIMITED_OUTPUT #include "lz4_encoder.h" /* int LZ4_compress64k_stack( const char* source, char* dest, int inputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest'. This function compresses better than LZ4_compress_stack(), on the condition that 'inputSize' must be < to LZ4_64KLIMIT, or the function will fail. Destination buffer must be already allocated, and sized at a minimum of LZ4_compressBound(inputSize). return : the number of bytes written in buffer 'dest', or 0 if compression fails */ #define FUNCTION_NAME LZ4_compress64k_stack #define COMPRESS_64K #include "lz4_encoder.h" /* int LZ4_compress64k_stack_limitedOutput( const char* source, char* dest, int inputSize, int maxOutputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. This function compresses better than LZ4_compress_stack_limitedOutput(), on the condition that 'inputSize' must be < to LZ4_64KLIMIT, or the function will fail. If it cannot achieve it, compression will stop, and result of the function will be zero. return : the number of bytes written in buffer 'dest', or 0 if the compression fails */ #define FUNCTION_NAME LZ4_compress64k_stack_limitedOutput #define COMPRESS_64K #define LIMITED_OUTPUT #include "lz4_encoder.h" /* void* LZ4_createHeapMemory(); int LZ4_freeHeapMemory(void* ctx); Used to allocate and free hashTable memory to be used by the LZ4_compress_heap* family of functions. LZ4_createHeapMemory() returns NULL is memory allocation fails. */ void* LZ4_create() { return malloc(HASHTABLESIZE); } int LZ4_free(void* ctx) { free(ctx); return 0; } /* int LZ4_compress_heap( void* ctx, const char* source, char* dest, int inputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest'. The memory used for compression must be created by LZ4_createHeapMemory() and provided by pointer 'ctx'. Destination buffer must be already allocated, and sized at a minimum of LZ4_compressBound(inputSize). return : the number of bytes written in buffer 'dest' */ #define FUNCTION_NAME LZ4_compress_heap #define USE_HEAPMEMORY #include "lz4_encoder.h" /* int LZ4_compress_heap_limitedOutput( void* ctx, const char* source, char* dest, int inputSize, int maxOutputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. If it cannot achieve it, compression will stop, and result of the function will be zero. The memory used for compression must be created by LZ4_createHeapMemory() and provided by pointer 'ctx'. return : the number of bytes written in buffer 'dest', or 0 if the compression fails */ #define FUNCTION_NAME LZ4_compress_heap_limitedOutput #define LIMITED_OUTPUT #define USE_HEAPMEMORY #include "lz4_encoder.h" /* int LZ4_compress64k_heap( void* ctx, const char* source, char* dest, int inputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest'. The memory used for compression must be created by LZ4_createHeapMemory() and provided by pointer 'ctx'. 'inputSize' must be < to LZ4_64KLIMIT, or the function will fail. Destination buffer must be already allocated, and sized at a minimum of LZ4_compressBound(inputSize). return : the number of bytes written in buffer 'dest' */ #define FUNCTION_NAME LZ4_compress64k_heap #define COMPRESS_64K #define USE_HEAPMEMORY #include "lz4_encoder.h" /* int LZ4_compress64k_heap_limitedOutput( void* ctx, const char* source, char* dest, int inputSize, int maxOutputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. If it cannot achieve it, compression will stop, and result of the function will be zero. The memory used for compression must be created by LZ4_createHeapMemory() and provided by pointer 'ctx'. 'inputSize' must be < to LZ4_64KLIMIT, or the function will fail. return : the number of bytes written in buffer 'dest', or 0 if the compression fails */ #define FUNCTION_NAME LZ4_compress64k_heap_limitedOutput #define COMPRESS_64K #define LIMITED_OUTPUT #define USE_HEAPMEMORY #include "lz4_encoder.h" int LZ4_compress(const char* source, char* dest, int inputSize) { #if HEAPMODE void* ctx = LZ4_create(); int result; if (ctx == NULL) return 0; // Failed allocation => compression not done if (inputSize < LZ4_64KLIMIT) result = LZ4_compress64k_heap(ctx, source, dest, inputSize); else result = LZ4_compress_heap(ctx, source, dest, inputSize); LZ4_free(ctx); return result; #else if (inputSize < (int)LZ4_64KLIMIT) return LZ4_compress64k_stack(source, dest, inputSize); return LZ4_compress_stack(source, dest, inputSize); #endif } int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize) { #if HEAPMODE void* ctx = LZ4_create(); int result; if (ctx == NULL) return 0; // Failed allocation => compression not done if (inputSize < LZ4_64KLIMIT) result = LZ4_compress64k_heap_limitedOutput(ctx, source, dest, inputSize, maxOutputSize); else result = LZ4_compress_heap_limitedOutput(ctx, source, dest, inputSize, maxOutputSize); LZ4_free(ctx); return result; #else if (inputSize < (int)LZ4_64KLIMIT) return LZ4_compress64k_stack_limitedOutput(source, dest, inputSize, maxOutputSize); return LZ4_compress_stack_limitedOutput(source, dest, inputSize, maxOutputSize); #endif } //**************************** // Decompression functions //**************************** typedef enum { noPrefix = 0, withPrefix = 1 } prefix64k_directive; typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive; typedef enum { full = 0, partial = 1 } earlyEnd_directive; // This generic decompression function cover all use cases. // It shall be instanciated several times, using different sets of directives // Note that it is essential this generic function is really inlined, // in order to remove useless branches during compilation optimisation. forceinline int LZ4_decompress_generic( const char* source, char* dest, int inputSize, // int outputSize, // OutputSize must be != 0; if endOnInput==endOnInputSize, this value is the max size of Output Buffer. int endOnInput, // endOnOutputSize, endOnInputSize int prefix64k, // noPrefix, withPrefix int partialDecoding, // full, partial int targetOutputSize // only used if partialDecoding==partial ) { // Local Variables const BYTE* restrict ip = (const BYTE*) source; const BYTE* ref; const BYTE* const iend = ip + inputSize; BYTE* op = (BYTE*) dest; BYTE* const oend = op + outputSize; BYTE* cpy; BYTE* oexit = op + targetOutputSize; size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; #if LZ4_ARCH64 size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3}; #endif // Special cases if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT; // targetOutputSize too high => decode everything if ((endOnInput) && unlikely(outputSize==0)) return ((inputSize==1) && (*ip==0)) ? 0 : -1; // Empty output buffer if ((!endOnInput) && unlikely(outputSize==0)) return (*ip==0?1:-1); // Main Loop while (1) { unsigned token; size_t length; // get runlength token = *ip++; if ((length=(token>>ML_BITS)) == RUN_MASK) { unsigned s=255; while (((endOnInput)?ip(partialDecoding?oexit:oend-MFLIMIT)) || (ip+length>iend-(2+1+LASTLITERALS))) ) || ((!endOnInput) && (cpy>oend-COPYLENGTH))) { if (partialDecoding) { if (cpy > oend) goto _output_error; // Error : write attempt beyond end of output buffer if ((endOnInput) && (ip+length > iend)) goto _output_error; // Error : read attempt beyond end of input buffer } else { if ((!endOnInput) && (cpy != oend)) goto _output_error; // Error : block decoding must stop exactly there if ((endOnInput) && ((ip+length != iend) || (cpy > oend))) goto _output_error; // Error : input must be consumed } memcpy(op, ip, length); ip += length; op += length; break; // Necessarily EOF, due to parsing restrictions } LZ4_WILDCOPY(ip, op, cpy); ip -= (op-cpy); op = cpy; // get offset LZ4_READ_LITTLEENDIAN_16(ref,cpy,ip); ip+=2; if ((prefix64k==noPrefix) && unlikely(ref < (BYTE* const)dest)) goto _output_error; // Error : offset outside destination buffer // get matchlength if ((length=(token&ML_MASK)) == ML_MASK) { for ( ; (!endOnInput) || (ipoend-(COPYLENGTH)-(STEPSIZE-4)) { if (cpy > oend-LASTLITERALS) goto _output_error; // Error : last 5 bytes must be literals LZ4_SECURECOPY(ref, op, (oend-COPYLENGTH)); while(op> ((MINMATCH*8)-HASHLOG)) #define LZ4_HASHVALUE(p) LZ4_HASH(A32(p)) //**************************** // Function code //**************************** int FUNCTION_NAME( #ifdef USE_HEAPMEMORY void* ctx, #endif const char* source, char* dest, int inputSize #ifdef LIMITED_OUTPUT ,int maxOutputSize #endif ) { #ifdef USE_HEAPMEMORY CURRENT_H_TYPE* HashTable = (CURRENT_H_TYPE*)ctx; #else CURRENT_H_TYPE HashTable[HASHTABLE_NBCELLS] = {0}; #endif const BYTE* ip = (BYTE*) source; CURRENTBASE(base); const BYTE* anchor = ip; const BYTE* const iend = ip + inputSize; const BYTE* const mflimit = iend - MFLIMIT; #define matchlimit (iend - LASTLITERALS) BYTE* op = (BYTE*) dest; #ifdef LIMITED_OUTPUT BYTE* const oend = op + maxOutputSize; #endif int length; const int skipStrength = SKIPSTRENGTH; U32 forwardH; // Init if (inputSize=LZ4_64KLIMIT) return 0; // Size too large (not within 64K limit) #endif #ifdef USE_HEAPMEMORY memset((void*)HashTable, 0, HASHTABLESIZE); #endif // First Byte HashTable[LZ4_HASHVALUE(ip)] = (CURRENT_H_TYPE)(ip - base); ip++; forwardH = LZ4_HASHVALUE(ip); // Main Loop for ( ; ; ) { int findMatchAttempts = (1U << skipStrength) + 3; const BYTE* forwardIp = ip; const BYTE* ref; BYTE* token; // Find a match do { U32 h = forwardH; int step = findMatchAttempts++ >> skipStrength; ip = forwardIp; forwardIp = ip + step; if unlikely(forwardIp > mflimit) { goto _last_literals; } forwardH = LZ4_HASHVALUE(forwardIp); ref = base + HashTable[h]; HashTable[h] = (CURRENT_H_TYPE)(ip - base); } while ((ref < ip - MAX_DISTANCE) || (A32(ref) != A32(ip))); // Catch up while ((ip>anchor) && (ref>(BYTE*)source) && unlikely(ip[-1]==ref[-1])) { ip--; ref--; } // Encode Literal length length = (int)(ip - anchor); token = op++; #ifdef LIMITED_OUTPUT if unlikely(op + length + (2 + 1 + LASTLITERALS) + (length>>8) > oend) return 0; // Check output limit #endif if (length>=(int)RUN_MASK) { int len = length-RUN_MASK; *token=(RUN_MASK<= 255 ; len-=255) *op++ = 255; *op++ = (BYTE)len; } else *token = (BYTE)(length<>8) > oend) return 0; // Check output limit #endif if (length>=(int)ML_MASK) { *token += ML_MASK; length -= ML_MASK; for (; length > 509 ; length-=510) { *op++ = 255; *op++ = 255; } if (length >= 255) { length-=255; *op++ = 255; } *op++ = (BYTE)length; } else *token += (BYTE)(length); // Test end of chunk if (ip > mflimit) { anchor = ip; break; } // Fill table HashTable[LZ4_HASHVALUE(ip-2)] = (CURRENT_H_TYPE)(ip - 2 - base); // Test next position ref = base + HashTable[LZ4_HASHVALUE(ip)]; HashTable[LZ4_HASHVALUE(ip)] = (CURRENT_H_TYPE)(ip - base); if ((ref >= ip - MAX_DISTANCE) && (A32(ref) == A32(ip))) { token = op++; *token=0; goto _next_match; } // Prepare next loop anchor = ip++; forwardH = LZ4_HASHVALUE(ip); } _last_literals: // Encode Last Literals { int lastRun = (int)(iend - anchor); #ifdef LIMITED_OUTPUT if (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize) return 0; // Check output limit #endif if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK<= 255 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; } else *op++ = (BYTE)(lastRun< // calloc, free #define ALLOCATOR(s) calloc(1,s) #define FREEMEM free #include // memset, memcpy #define MEM_INIT memset //************************************** // CPU Feature Detection //************************************** // 32 or 64 bits ? #if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \ || defined(__powerpc64__) || defined(__ppc64__) || defined(__PPC64__) \ || defined(__64BIT__) || defined(_LP64) || defined(__LP64__) \ || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) // Detects 64 bits mode # define LZ4_ARCH64 1 #else # define LZ4_ARCH64 0 #endif // Little Endian or Big Endian ? // Overwrite the #define below if you know your architecture endianess #if defined (__GLIBC__) # include # if (__BYTE_ORDER == __BIG_ENDIAN) # define LZ4_BIG_ENDIAN 1 # endif #elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN)) # define LZ4_BIG_ENDIAN 1 #elif defined(__sparc) || defined(__sparc__) \ || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \ || defined(__hpux) || defined(__hppa) \ || defined(_MIPSEB) || defined(__s390__) # define LZ4_BIG_ENDIAN 1 #else // Little Endian assumed. PDP Endian and other very rare endian format are unsupported. #endif // Unaligned memory access is automatically enabled for "common" CPU, such as x86. // For others CPU, the compiler will be more cautious, and insert extra code to ensure aligned access is respected // If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance #if defined(__ARM_FEATURE_UNALIGNED) # define LZ4_FORCE_UNALIGNED_ACCESS 1 #endif // Define this parameter if your target system or compiler does not support hardware bit count #if defined(_MSC_VER) && defined(_WIN32_WCE) // Visual Studio for Windows CE does not support Hardware bit count # define LZ4_FORCE_SW_BITCOUNT #endif //************************************** // Compiler Options //************************************** #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L // C99 /* "restrict" is a known keyword */ #else # define restrict // Disable restrict #endif #ifdef _MSC_VER // Visual Studio # define forceinline static __forceinline # include // For Visual 2005 # if LZ4_ARCH64 // 64-bits # pragma intrinsic(_BitScanForward64) // For Visual 2005 # pragma intrinsic(_BitScanReverse64) // For Visual 2005 # else // 32-bits # pragma intrinsic(_BitScanForward) // For Visual 2005 # pragma intrinsic(_BitScanReverse) // For Visual 2005 # endif # pragma warning(disable : 4127) // disable: C4127: conditional expression is constant # pragma warning(disable : 4701) // disable: C4701: potentially uninitialized local variable used #else # ifdef __GNUC__ # define forceinline static inline __attribute__((always_inline)) # else # define forceinline static inline # endif #endif #ifdef _MSC_VER // Visual Studio # define lz4_bswap16(x) _byteswap_ushort(x) #else # define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8))) #endif //************************************** // Includes //************************************** #include "lz4hc.h" #include "lz4.h" //************************************** // Basic Types //************************************** #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L // C99 # include typedef uint8_t BYTE; typedef uint16_t U16; typedef uint32_t U32; typedef int32_t S32; typedef uint64_t U64; #else typedef unsigned char BYTE; typedef unsigned short U16; typedef unsigned int U32; typedef signed int S32; typedef unsigned long long U64; #endif #if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS) # define _PACKED __attribute__ ((packed)) #else # define _PACKED #endif #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # ifdef __IBMC__ # pragma pack(1) # else # pragma pack(push, 1) # endif #endif typedef struct _U16_S { U16 v; } _PACKED U16_S; typedef struct _U32_S { U32 v; } _PACKED U32_S; typedef struct _U64_S { U64 v; } _PACKED U64_S; #if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__) # pragma pack(pop) #endif #define A64(x) (((U64_S *)(x))->v) #define A32(x) (((U32_S *)(x))->v) #define A16(x) (((U16_S *)(x))->v) //************************************** // Constants //************************************** #define MINMATCH 4 #define DICTIONARY_LOGSIZE 16 #define MAXD (1<> ((MINMATCH*8)-HASH_LOG)) #define HASH_VALUE(p) HASH_FUNCTION(A32(p)) #define HASH_POINTER(p) (HashTable[HASH_VALUE(p)] + base) #define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK] #define GETNEXT(p) ((p) - (size_t)DELTANEXT(p)) //************************************** // Private functions //************************************** #if LZ4_ARCH64 forceinline int LZ4_NbCommonBytes (register U64 val) { #if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanReverse64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clzll(val) >> 3); # else int r; if (!(val>>32)) { r=4; } else { r=0; val>>=32; } if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } r += (!val); return r; # endif #else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r = 0; _BitScanForward64( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctzll(val) >> 3); # else static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 }; return DeBruijnBytePos[((U64)((val & -val) * 0x0218A392CDABBD3F)) >> 58]; # endif #endif } #else forceinline int LZ4_NbCommonBytes (register U32 val) { #if defined(LZ4_BIG_ENDIAN) # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanReverse( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_clz(val) >> 3); # else int r; if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } r += (!val); return r; # endif #else # if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) unsigned long r; _BitScanForward( &r, val ); return (int)(r>>3); # elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT) return (__builtin_ctz(val) >> 3); # else static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 }; return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; # endif #endif } #endif forceinline int LZ4_InitHC (LZ4HC_Data_Structure* hc4, const BYTE* base) { MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable)); MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable)); hc4->nextToUpdate = base + 1; hc4->base = base; hc4->inputBuffer = base; hc4->end = base; return 1; } void* LZ4_createHC (const char* slidingInputBuffer) { void* hc4 = ALLOCATOR(sizeof(LZ4HC_Data_Structure)); LZ4_InitHC ((LZ4HC_Data_Structure*)hc4, (const BYTE*)slidingInputBuffer); return hc4; } int LZ4_freeHC (void* LZ4HC_Data) { FREEMEM(LZ4HC_Data); return (0); } // Update chains up to ip (excluded) forceinline void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip) { U16* chainTable = hc4->chainTable; HTYPE* HashTable = hc4->hashTable; INITBASE(base,hc4->base); while(hc4->nextToUpdate < ip) { const BYTE* const p = hc4->nextToUpdate; size_t delta = (p) - HASH_POINTER(p); if (delta>MAX_DISTANCE) delta = MAX_DISTANCE; DELTANEXT(p) = (U16)delta; HashTable[HASH_VALUE(p)] = (HTYPE)((p) - base); hc4->nextToUpdate++; } } char* LZ4_slideInputBufferHC(void* LZ4HC_Data) { LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data; U32 distance = (U32)(hc4->end - hc4->inputBuffer) - 64 KB; distance = (distance >> 16) << 16; // Must be a multiple of 64 KB LZ4HC_Insert(hc4, hc4->end - MINMATCH); memcpy((void*)(hc4->end - 64 KB - distance), (const void*)(hc4->end - 64 KB), 64 KB); hc4->nextToUpdate -= distance; hc4->base -= distance; if ((U32)(hc4->inputBuffer - hc4->base) > 1 GB + 64 KB) // Avoid overflow { int i; hc4->base += 1 GB; for (i=0; ihashTable[i] -= 1 GB; } hc4->end -= distance; return (char*)(hc4->end); } forceinline size_t LZ4HC_CommonLength (const BYTE* p1, const BYTE* p2, const BYTE* const matchlimit) { const BYTE* p1t = p1; while (p1tchainTable; HTYPE* const HashTable = hc4->hashTable; const BYTE* ref; INITBASE(base,hc4->base); int nbAttempts=MAX_NB_ATTEMPTS; size_t repl=0, ml=0; U16 delta=0; // useless assignment, to remove an uninitialization warning // HC4 match finder LZ4HC_Insert(hc4, ip); ref = HASH_POINTER(ip); #define REPEAT_OPTIMIZATION #ifdef REPEAT_OPTIMIZATION // Detect repetitive sequences of length <= 4 if ((U32)(ip-ref) <= 4) // potential repetition { if (A32(ref) == A32(ip)) // confirmed { delta = (U16)(ip-ref); repl = ml = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH; *matchpos = ref; } ref = GETNEXT(ref); } #endif while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) { nbAttempts--; if (*(ref+ml) == *(ip+ml)) if (A32(ref) == A32(ip)) { size_t mlt = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH; if (mlt > ml) { ml = mlt; *matchpos = ref; } } ref = GETNEXT(ref); } #ifdef REPEAT_OPTIMIZATION // Complete table if (repl) { const BYTE* ptr = ip; const BYTE* end; end = ip + repl - (MINMATCH-1); while(ptr < end-delta) { DELTANEXT(ptr) = delta; // Pre-Load ptr++; } do { DELTANEXT(ptr) = delta; HashTable[HASH_VALUE(ptr)] = (HTYPE)((ptr) - base); // Head of chain ptr++; } while(ptr < end); hc4->nextToUpdate = end; } #endif return (int)ml; } forceinline int LZ4HC_InsertAndGetWiderMatch (LZ4HC_Data_Structure* hc4, const BYTE* ip, const BYTE* startLimit, const BYTE* matchlimit, int longest, const BYTE** matchpos, const BYTE** startpos) { U16* const chainTable = hc4->chainTable; HTYPE* const HashTable = hc4->hashTable; INITBASE(base,hc4->base); const BYTE* ref; int nbAttempts = MAX_NB_ATTEMPTS; int delta = (int)(ip-startLimit); // First Match LZ4HC_Insert(hc4, ip); ref = HASH_POINTER(ip); while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts)) { nbAttempts--; if (*(startLimit + longest) == *(ref - delta + longest)) if (A32(ref) == A32(ip)) { #if 1 const BYTE* reft = ref+MINMATCH; const BYTE* ipt = ip+MINMATCH; const BYTE* startt = ip; while (iptstartLimit) && (reft > hc4->inputBuffer) && (startt[-1] == reft[-1])) {startt--; reft--;} if ((ipt-startt) > longest) { longest = (int)(ipt-startt); *matchpos = reft; *startpos = startt; } } ref = GETNEXT(ref); } return longest; } //************************************** // Compression functions //************************************** /* int LZ4_compressHC( const char* source, char* dest, int inputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest'. Destination buffer must be already allocated, and sized at a minimum of LZ4_compressBound(inputSize). return : the number of bytes written in buffer 'dest' */ #define FUNCTION_NAME LZ4_compressHC #include "lz4hc_encoder.h" /* int LZ4_compressHC_limitedOutput( const char* source, char* dest, int inputSize, int maxOutputSize) Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'. If it cannot achieve it, compression will stop, and result of the function will be zero. return : the number of bytes written in buffer 'dest', or 0 if the compression fails */ #define FUNCTION_NAME LZ4_compressHC_limitedOutput #define LIMITED_OUTPUT #include "lz4hc_encoder.h" horde_lz4-1.0.2/lz4hc_encoder.h0000644000175000007640000002463712177042237016315 0ustar slusarzslusarz/* LZ4 HC Encoder - Part of LZ4 HC algorithm Copyright (C) 2011-2013, Yann Collet. BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You can contact the author at : - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html - LZ4 source repository : http://code.google.com/p/lz4/ */ /* lz4hc_encoder.h must be included into lz4hc.c The objective of this file is to create a single LZ4 compression function source which will be instanciated multiple times with minor variations depending on a set of #define. */ //**************************** // Check required defines //**************************** #ifndef FUNCTION_NAME # error "FUNTION_NAME is not defined" #endif //**************************** // Local definitions //**************************** #define COMBINED_NAME_RAW(n1,n2) n1 ## n2 #define COMBINED_NAME(n1,n2) COMBINED_NAME_RAW(n1,n2) #define ENCODE_SEQUENCE_NAME COMBINED_NAME(FUNCTION_NAME,_encodeSequence) #ifdef LIMITED_OUTPUT # define ENCODE_SEQUENCE(i,o,a,m,r,d) if (ENCODE_SEQUENCE_NAME(i,o,a,m,r,d)) return 0; #else # define ENCODE_SEQUENCE(i,o,a,m,r,d) ENCODE_SEQUENCE_NAME(i,o,a,m,r) #endif //**************************** // Function code //**************************** forceinline int ENCODE_SEQUENCE_NAME ( const BYTE** ip, BYTE** op, const BYTE** anchor, int matchLength, const BYTE* ref #ifdef LIMITED_OUTPUT ,BYTE* oend #endif ) { int length, len; BYTE* token; // Encode Literal length length = (int)(*ip - *anchor); token = (*op)++; #ifdef LIMITED_OUTPUT if ((*op + length + (2 + 1 + LASTLITERALS) + (length>>8)) > oend) return 1; // Check output limit #endif if (length>=(int)RUN_MASK) { *token=(RUN_MASK< 254 ; len-=255) *(*op)++ = 255; *(*op)++ = (BYTE)len; } else *token = (BYTE)(length<>8) > oend) return 1; // Check output limit #endif if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length > 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) { length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; } else *token += (BYTE)(length); // Prepare next loop *ip += matchLength; *anchor = *ip; return 0; } int COMBINED_NAME(FUNCTION_NAME,_continue) ( void* ctxvoid, const char* source, char* dest, int inputSize #ifdef LIMITED_OUTPUT ,int maxOutputSize #endif ) { LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid; const BYTE* ip = (const BYTE*) source; const BYTE* anchor = ip; const BYTE* const iend = ip + inputSize; const BYTE* const mflimit = iend - MFLIMIT; const BYTE* const matchlimit = (iend - LASTLITERALS); BYTE* op = (BYTE*) dest; #ifdef LIMITED_OUTPUT BYTE* const oend = op + maxOutputSize; #endif int ml, ml2, ml3, ml0; const BYTE* ref=NULL; const BYTE* start2=NULL; const BYTE* ref2=NULL; const BYTE* start3=NULL; const BYTE* ref3=NULL; const BYTE* start0; const BYTE* ref0; // Ensure blocks follow each other if (ip != ctx->end) return 0; ctx->end += inputSize; ip++; // Main Loop while (ip < mflimit) { ml = LZ4HC_InsertAndFindBestMatch (ctx, ip, matchlimit, (&ref)); if (!ml) { ip++; continue; } // saved, in case we would skip too much start0 = ip; ref0 = ref; ml0 = ml; _Search2: if (ip+ml < mflimit) ml2 = LZ4HC_InsertAndGetWiderMatch(ctx, ip + ml - 2, ip + 1, matchlimit, ml, &ref2, &start2); else ml2 = ml; if (ml2 == ml) // No better match { ENCODE_SEQUENCE(&ip, &op, &anchor, ml, ref, oend); continue; } if (start0 < ip) { if (start2 < ip + ml0) // empirical { ip = start0; ref = ref0; ml = ml0; } } // Here, start0==ip if ((start2 - ip) < 3) // First Match too small : removed { ml = ml2; ip = start2; ref =ref2; goto _Search2; } _Search3: // Currently we have : // ml2 > ml1, and // ip1+3 <= ip2 (usually < ip1+ml1) if ((start2 - ip) < OPTIMAL_ML) { int correction; int new_ml = ml; if (new_ml > OPTIMAL_ML) new_ml = OPTIMAL_ML; if (ip+new_ml > start2 + ml2 - MINMATCH) new_ml = (int)(start2 - ip) + ml2 - MINMATCH; correction = new_ml - (int)(start2 - ip); if (correction > 0) { start2 += correction; ref2 += correction; ml2 -= correction; } } // Now, we have start2 = ip+new_ml, with new_ml = min(ml, OPTIMAL_ML=18) if (start2 + ml2 < mflimit) ml3 = LZ4HC_InsertAndGetWiderMatch(ctx, start2 + ml2 - 3, start2, matchlimit, ml2, &ref3, &start3); else ml3 = ml2; if (ml3 == ml2) // No better match : 2 sequences to encode { // ip & ref are known; Now for ml if (start2 < ip+ml) ml = (int)(start2 - ip); // Now, encode 2 sequences ENCODE_SEQUENCE(&ip, &op, &anchor, ml, ref, oend); ip = start2; ENCODE_SEQUENCE(&ip, &op, &anchor, ml2, ref2, oend); continue; } if (start3 < ip+ml+3) // Not enough space for match 2 : remove it { if (start3 >= (ip+ml)) // can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 { if (start2 < ip+ml) { int correction = (int)(ip+ml - start2); start2 += correction; ref2 += correction; ml2 -= correction; if (ml2 < MINMATCH) { start2 = start3; ref2 = ref3; ml2 = ml3; } } ENCODE_SEQUENCE(&ip, &op, &anchor, ml, ref, oend); ip = start3; ref = ref3; ml = ml3; start0 = start2; ref0 = ref2; ml0 = ml2; goto _Search2; } start2 = start3; ref2 = ref3; ml2 = ml3; goto _Search3; } // OK, now we have 3 ascending matches; let's write at least the first one // ip & ref are known; Now for ml if (start2 < ip+ml) { if ((start2 - ip) < (int)ML_MASK) { int correction; if (ml > OPTIMAL_ML) ml = OPTIMAL_ML; if (ip + ml > start2 + ml2 - MINMATCH) ml = (int)(start2 - ip) + ml2 - MINMATCH; correction = ml - (int)(start2 - ip); if (correction > 0) { start2 += correction; ref2 += correction; ml2 -= correction; } } else { ml = (int)(start2 - ip); } } ENCODE_SEQUENCE(&ip, &op, &anchor, ml, ref, oend); ip = start2; ref = ref2; ml = ml2; start2 = start3; ref2 = ref3; ml2 = ml3; goto _Search3; } // Encode Last Literals { int lastRun = (int)(iend - anchor); #ifdef LIMITED_OUTPUT if (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize) return 0; // Check output limit #endif if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK< 254 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; } else *op++ = (BYTE)(lastRun< The memory position where the next input data block must start is provided as the result of the function. Compression can then resume, using LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue(), as usual. When compression is completed, a call to LZ4_freeHC() will release the memory used by the LZ4HC Data Structure. */ #if defined (__cplusplus) } #endif horde_lz4-1.0.2/php_verdep.h0000644000175000007640000000247512177042237015722 0ustar slusarzslusarz#ifndef PHP_VERDEP_H #define PHP_VERDEP_H #ifndef ZED_FE_END #define ZEND_FE_END { NULL, NULL, NULL, 0, 0 } #endif #ifndef Z_SET_REFCOUNT_P #define Z_SET_REFCOUNT_P(pz, rc) ((pz)->refcount = rc) #endif #ifndef Z_UNSET_ISREF_PP #define Z_UNSET_ISREF_PP(ppz) Z_UNSET_ISREF_P(*(ppz)) #endif #ifndef Z_UNSET_ISREF_P #define Z_UNSET_ISREF_P(pz) ((pz)->is_ref = 0) #endif #ifndef Z_ISREF_PP #define Z_ISREF_PP(ppz) Z_ISREF_P(*(ppz)) #endif #ifndef Z_ISREF_P #if ZEND_MODULE_API_NO >= 20090626 #define Z_ISREF_P(pz) zval_isref_p(pz) #else #define Z_ISREF_P(pz) ((pz)->is_ref) #endif #endif #ifndef Z_ADDREF_PP #define Z_ADDREF_PP(ppz) Z_ADDREF_P(*(ppz)) #endif #ifndef Z_ADDREF_P #if ZEND_MODULE_API_NO >= 20090626 #define Z_ADDREF_P(pz) zval_addref_p(pz) #else #define Z_ADDREF_P(pz) (++(pz)->refcount) #endif #endif #ifndef Z_SET_ISREF_PP #define Z_SET_ISREF_PP(ppz) Z_SET_ISREF_P(*(ppz)) #endif #ifndef Z_SET_ISREF_P #if ZEND_MODULE_API_NO >= 20090626 #define Z_SET_ISREF_P(pz) zval_set_isref_p(pz) #else #define Z_SET_ISREF_P(pz) ((pz)->is_ref = 1) #endif #endif #ifndef array_init_size #define array_init_size(arg, size) _array_init((arg) ZEND_FILE_LINE_CC) #endif #ifndef zend_parse_parameters_none #define zend_parse_parameters_none() \ zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "") #endif #endif /* PHP_VERDEP_H */ horde_lz4-1.0.2/README.md0000644000175000007640000000211412177042237014662 0ustar slusarzslusarz# Horde LZ4 Extension for PHP # This extension allows LZ4 compression. Documentation for LZ4 can be found at [» http://code.google.com/p/lz4/](http://code.google.com/p/lz4/). ## Configration ## php.ini: extension=horde_lz4.so ## Function ## * horde\_lz4\_compress — LZ4 compression * horde\_lz4\_uncompress — LZ4 decompression ### horde\_lz4\_compress — LZ4 compression ### #### Description #### string **horde\_lz4\_compress** (string _$data_ [, bool _$high_ = false]) LZ4 compression. #### Pameters #### * _data_ The string to compress. * _high_ High Compression Mode. #### Return Values #### Returns the compressed data or FALSE if an error occurred. ### horde\_lz4\_uncompress — LZ4 decompression ### #### Description #### string **horde\_lz4\_uncompress** (string _$data_) LZ4 decompression. #### Pameters #### * _data_ The compressed string. #### Return Values #### Returns the decompressed data or FALSE if an error occurred. ## Examples ## $compressed = horde_lz4_compress('test'); $uncompressed = horde_lz4_uncompress($compressed);