package.xml 0000664 0001750 0001750 00000016634 12653745320 011320 0 ustar jan jan
horde_lz4pear.horde.orgHorde LZ4 Compression ExtensionPHP extension that implements the LZ4 compression algorithm - an extremely fast lossless compression algorithm.Michael Slusarzslusarzslusarz@horde.orgyes2016-02-011.0.91.0.0stablestableMIT (Expat)
* [jan] Mark PHP 7 as supported.
5.0.08.0.0alpha18.0.0alpha11.7.0horde_lz42013-03-031.0.0beta11.0.0betabetaPHP 3.01
* [mms] Initial release of package.
1.0.0beta21.0.0betabeta2013-03-04PHP 3.01
* [mms] More robust header checking: eliminates the need to explicitly pass decompression size to horde_lz4_uncompress().
* [mms] Remove maxsize option to horde_lz4_uncompress().
* [mms] Remove 'prefix' option to horde_lz4_compress().
* [mms] Be silent when encountering decompression errors.
1.0.01.0.0stablestable2013-05-06PHP 3.01
* First stable release.
* [mms] Convert unit tests to PHPUnit.
1.0.11.0.0stablestable2013-07-08MIT (Expat)
* [mms] Change license from PHP -> MIT.
1.0.21.0.0stablestable2013-08-02MIT (Expat)
* [mms] Update LZ4 source to r99.
1.0.31.0.0stablestable2014-02-11MIT (Expat)
* [mms] Update lz4 source to r108.
1.0.41.0.0stablestable2014-07-01MIT (Expat)
* [jan] Update lz4 source to r118 (CVE-2014-4611).
1.0.51.0.0stablestable2014-07-03MIT (Expat)
* [mms] Fix compilation on ancient versions of PHP 5.3 (Bug #13310).
1.0.61.0.0stablestable2014-07-03MIT (Expat)
* [jan] Update lz4 source to r119 (CVE-2014-4715).
1.0.71.0.0stablestable2014-09-16MIT (Expat)
* [mms] Allow extension to be built with system lz4 library (Remi Collet <fedora@famillecollet.com>).
* [mms] Add --with-liblz4 option to use system liblz4 (Remi)
1.0.81.0.0stablestable2015-04-13MIT (Expat)
* [jan] Fix PHP 7 compatibility (Remi Collet <fedora@famillecollet.com>).
1.0.91.0.0stablestable2016-02-01MIT (Expat)
* [jan] Mark PHP 7 as supported.
horde_lz4-1.0.9/test/horde/lz4/fixtures/data.txt 0000664 0001750 0001750 00000006734 12653745320 017654 0 ustar jan jan To be or not to be, that is the question;
Whether 'tis nobler in the mind to suffer
The Slings and Arrows of outrageous Fortune
Or to take arms against a sea of troubles,
And by opposing, end them. To die, to sleep;
No more; and by a sleep to say we end
The heart-ache and the thousand natural shocks
That flesh is heir to 'tis a consummation
Devoutly to be wish'd. To die, to sleep;
To sleep, perchance to dream. Ay, there's the rub,
For in that sleep of death what dreams may come,
When we have shuffled off this mortal coil,
Must give us pause. There's the respect
That makes calamity of so long life,
For who would bear the whips and scorns of time,
Th'oppressor's wrong, the proud man's contumely,
The pangs of dispriz'd love, the law's delay,
The insolence of office, and the spurns
That patient merit of th'unworthy takes,
When he himself might his quietus make
With a bare bodkin? who would fardels bear,
To grunt and sweat under a weary life,
But that the dread of something after death,
The undiscovered country from whose bourn
No traveller returns, puzzles the will,
And makes us rather bear those ills we have
Than fly to others that we know not of?
Thus conscience does make cowards of us all,
And thus the native hue of resolution
Is sicklied o'er with the pale cast of thought,
And enterprises of great pitch and moment
With this regard their currents turn away,
And lose the name of action.
Hath not a Jew eyes? Hath not a Jew hands, organs, dimensions,
senses, affections, passions; fed with the same food, hurt with
the same weapons, subject to the same diseases, heal'd by
the same means, warm'd and cool'd by the same winter and summer
as a Christian is? If you prick us, do we not bleed? If you
tickle us, do we not laugh? If you poison us, do we not die?
And if you wrong us, shall we not revenge? If we are like you
in the rest, we will resemble you in that. If a Jew wrong a
Christian, what is his humility? Revenge. If a Christian wrong
a Jew, what should his sufferance be by Christian example? Why,
revenge. The villainy you teach me, I will execute, and it
shall go hard but I will better the instruction.
Is this a dagger which I see before me,
The handle toward my hand? Come, let me clutch thee.
I have thee not, and yet I see thee still.
Art thou not, fatal vision, sensible
To feeling as to sight? or art thou but
A dagger of the mind, a false creation,
Proceeding from the heat-oppress'd brain?
I see thee yet, in form as palpable
As this which now I draw.
Thou marshall'st me the way that I was going;
And such an instrument I was to use.
Mine eyes are made the fools o' the other senses,
Or else worth all the rest; I see thee still,
And on thy blade and dudgeon gouts of blood,
Which was not so before.
There's no such thing:
It is the bloody business which informs Thus to mine eyes.
Now o'er the one halfworld Nature seems dead,
and wicked dreams abuse The curtain'd sleep; witchcraft celebrates
Pale Hecate's offerings, and wither'd murder,
Alarum'd by his sentinel, the wolf,
Whose howl's his watch, thus with his stealthy pace.
With Tarquin's ravishing strides, towards his design
Moves like a ghost. Thou sure and firm-set earth,
Hear not my steps, which way they walk,
for fear Thy very stones prate of my whereabout,
And take the present horror from the time,
Which now suits with it.
Whiles I threat, he lives:
Words to the heat of deeds too cold breath gives.
I go, and it is done; the bell invites me.
Hear it not, Duncan; for it is a knell
That summons thee to heaven or to hell.
horde_lz4-1.0.9/test/horde/lz4/AllTests.php 0000664 0001750 0001750 00000000132 12653745320 016557 0 ustar jan jan run();
horde_lz4-1.0.9/test/horde/lz4/bootstrap.php 0000664 0001750 0001750 00000000143 12653745320 017043 0 ustar jan jan
horde_lz4-1.0.9/test/horde/lz4/UnitTest.php 0000664 0001750 0001750 00000012031 12653745320 016604 0 ustar jan jan
* @category Horde
* @copyright 2013 Horde LLC
* @ignore
* @license http://www.php.net/license/3_01.txt PHP 3.01
* @package horde_lz4
* @subpackage UnitTests
*/
class horde_lz4_UnitTest extends PHPUnit_Framework_TestCase
{
private $data;
public function setUp()
{
if (!extension_loaded('horde_lz4')) {
$this->markTestSkipped('horde_lz4 extension not installed.');
}
$this->data = file_get_contents(__DIR__ . '/fixtures/data.txt');
}
// Test horde_lz4_compress() function : basic functionality
public function testCompressBasic()
{
// Compressing a big string
$output = horde_lz4_compress($this->data);
$this->assertEquals(
0,
strcmp(horde_lz4_uncompress($output), $this->data)
);
// Compressing a smaller string
$smallstring = "A small string to compress\n";
$output = horde_lz4_compress($smallstring);
$this->assertEquals(
0,
strcmp(horde_lz4_uncompress($output), $smallstring)
);
}
// Test horde_lz4_compress() function : error conditions
public function testCompressErrorConditionOne()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
// Zero arguments
horde_lz4_compress();
}
// Test horde_lz4_compress() function : error conditions
public function testCompressErrorConditionTwo()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
// Test horde_lz4_compress with one more than the expected number
// of arguments
$data = 'string_val';
$extra_arg = 10;
horde_lz4_compress($data, false, $extra_arg);
}
// Test horde_lz4_compress() function : error conditions
public function testCompressErrorConditionThree()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
horde_lz4_compress(new stdClass);
}
// Test horde_lz4_compress() function : variation
public function testCompressVariation()
{
$output = horde_lz4_compress($this->data);
$this->assertNotEquals(
md5($output),
md5(horde_lz4_compress($output))
);
}
// Test horde_lz4_uncompress() function : basic functionality
public function testUncompressBasic()
{
$compressed = horde_lz4_compress($this->data);
$this->assertEquals(
0,
strcmp($this->data, horde_lz4_uncompress($compressed))
);
}
// Test horde_lz4_uncompress() function : error conditions
public function testUncompressErrorConditionOne()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
// Zero arguments
horde_lz4_uncompress();
}
// Test horde_lz4_uncompress() function : error conditions
public function testUncompressErrorConditionTwo()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
// Test horde_lz4_uncompress with one more than the expected number
// of arguments
$data = 'string_val';
$extra_arg = 10;
horde_lz4_uncompress($data, $extra_arg);
}
// Test horde_lz4_uncompress() function : error conditions
public function testUncompressErrorConditionThree()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
horde_lz4_uncompress(new stdClass);
}
// Test horde_lz4_uncompress() function : error conditions
public function testUncompressErrorConditionFour()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning');
// Testing with incorrect arguments
horde_lz4_uncompress(123);
}
// Test horde_lz4_compress() function : high compression
public function testHighCompression()
{
// Compressing a big string
$output = horde_lz4_compress($this->data, true);
$this->assertEquals(
0,
strcmp(horde_lz4_uncompress($output), $this->data)
);
// Compressing a smaller string
$smallstring = "A small string to compress\n";
$output = horde_lz4_compress($smallstring, true);
$this->assertEquals(
0,
strcmp(horde_lz4_uncompress($output), $smallstring)
);
}
// Test horde_lz4_uncompress() function : bad input (non-lz4 data)
public function testUncompressBadInput()
{
// Bad data is missing the Horde-LZ4 header and is not LZ4 data.
$bad_data = "12345678";
$this->assertFalse(horde_lz4_uncompress($bad_data));
}
}
horde_lz4-1.0.9/lib/LICENSE 0000664 0001750 0001750 00000002436 12653745320 013306 0 ustar jan jan LZ4 Library
Copyright (c) 2011-2014, Yann Collet
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. horde_lz4-1.0.9/lib/lz4.c 0000664 0001750 0001750 00000130467 12653745320 013164 0 ustar jan jan /*
LZ4 - Fast LZ compression algorithm
Copyright (C) 2011-2014, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- LZ4 source repository : http://code.google.com/p/lz4/
- LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
/**************************************
Tuning parameters
**************************************/
/*
* HEAPMODE :
* Select how default compression functions will allocate memory for their hash table,
* in memory stack (0:default, fastest), or in memory heap (1:requires memory allocation (malloc)).
*/
#define HEAPMODE 0
/**************************************
CPU Feature Detection
**************************************/
/* 32 or 64 bits ? */
#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
|| defined(__powerpc64__) || defined(__powerpc64le__) \
|| defined(__ppc64__) || defined(__ppc64le__) \
|| defined(__PPC64__) || defined(__PPC64LE__) \
|| defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */
# define LZ4_ARCH64 1
#else
# define LZ4_ARCH64 0
#endif
/*
* Little Endian or Big Endian ?
* Overwrite the #define below if you know your architecture endianess
*/
#include /* Apparently required to detect endianess */
#if defined (__GLIBC__)
# include
# if (__BYTE_ORDER == __BIG_ENDIAN)
# define LZ4_BIG_ENDIAN 1
# endif
#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN))
# define LZ4_BIG_ENDIAN 1
#elif defined(__sparc) || defined(__sparc__) \
|| defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
|| defined(__hpux) || defined(__hppa) \
|| defined(_MIPSEB) || defined(__s390__)
# define LZ4_BIG_ENDIAN 1
#else
/* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */
#endif
/*
* Unaligned memory access is automatically enabled for "common" CPU, such as x86.
* For others CPU, such as ARM, the compiler may be more cautious, inserting unnecessary extra code to ensure aligned access property
* If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance
*/
#if defined(__ARM_FEATURE_UNALIGNED)
# define LZ4_FORCE_UNALIGNED_ACCESS 1
#endif
/* Define this parameter if your target system or compiler does not support hardware bit count */
#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */
# define LZ4_FORCE_SW_BITCOUNT
#endif
/*
* BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE :
* This option may provide a small boost to performance for some big endian cpu, although probably modest.
* You may set this option to 1 if data will remain within closed environment.
* This option is useless on Little_Endian CPU (such as x86)
*/
/* #define BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE 1 */
/**************************************
Compiler Options
**************************************/
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
/* "restrict" is a known keyword */
#else
# define restrict /* Disable restrict */
#endif
#ifdef _MSC_VER /* Visual Studio */
# define FORCE_INLINE static __forceinline
# include /* For Visual 2005 */
# if LZ4_ARCH64 /* 64-bits */
# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
# else /* 32-bits */
# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
# endif
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
#else
# ifdef __GNUC__
# define FORCE_INLINE static inline __attribute__((always_inline))
# else
# define FORCE_INLINE static inline
# endif
#endif
#ifdef _MSC_VER /* Visual Studio */
# define lz4_bswap16(x) _byteswap_ushort(x)
#else
# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8)))
#endif
#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
#if (GCC_VERSION >= 302) || (__INTEL_COMPILER >= 800) || defined(__clang__)
# define expect(expr,value) (__builtin_expect ((expr),(value)) )
#else
# define expect(expr,value) (expr)
#endif
#define likely(expr) expect((expr) != 0, 1)
#define unlikely(expr) expect((expr) != 0, 0)
/**************************************
Memory routines
**************************************/
#include /* malloc, calloc, free */
#define ALLOCATOR(n,s) calloc(n,s)
#define FREEMEM free
#include /* memset, memcpy */
#define MEM_INIT memset
/**************************************
Includes
**************************************/
#include "lz4.h"
/**************************************
Basic Types
**************************************/
#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
# include
typedef uint8_t BYTE;
typedef uint16_t U16;
typedef uint32_t U32;
typedef int32_t S32;
typedef uint64_t U64;
#else
typedef unsigned char BYTE;
typedef unsigned short U16;
typedef unsigned int U32;
typedef signed int S32;
typedef unsigned long long U64;
#endif
#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS)
# define _PACKED __attribute__ ((packed))
#else
# define _PACKED
#endif
#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
# if defined(__IBMC__) || defined(__SUNPRO_C) || defined(__SUNPRO_CC)
# pragma pack(1)
# else
# pragma pack(push, 1)
# endif
#endif
typedef struct { U16 v; } _PACKED U16_S;
typedef struct { U32 v; } _PACKED U32_S;
typedef struct { U64 v; } _PACKED U64_S;
typedef struct {size_t v;} _PACKED size_t_S;
#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
# if defined(__SUNPRO_C) || defined(__SUNPRO_CC)
# pragma pack(0)
# else
# pragma pack(pop)
# endif
#endif
#define A16(x) (((U16_S *)(x))->v)
#define A32(x) (((U32_S *)(x))->v)
#define A64(x) (((U64_S *)(x))->v)
#define AARCH(x) (((size_t_S *)(x))->v)
/**************************************
Constants
**************************************/
#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
#define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
#define HASH_SIZE_U32 (1 << LZ4_HASHLOG)
#define MINMATCH 4
#define COPYLENGTH 8
#define LASTLITERALS 5
#define MFLIMIT (COPYLENGTH+MINMATCH)
static const int LZ4_minLength = (MFLIMIT+1);
#define KB *(1U<<10)
#define MB *(1U<<20)
#define GB *(1U<<30)
#define LZ4_64KLIMIT ((64 KB) + (MFLIMIT-1))
#define SKIPSTRENGTH 6 /* Increasing this value will make the compression run slower on incompressible data */
#define MAXD_LOG 16
#define MAX_DISTANCE ((1 << MAXD_LOG) - 1)
#define ML_BITS 4
#define ML_MASK ((1U<=e; */
#else
# define LZ4_WILDCOPY(d,s,e) { if (likely(e-d <= 8)) LZ4_COPY8(d,s) else do { LZ4_COPY8(d,s) } while (d>3);
# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_clzll(val) >> 3);
# else
int r;
if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
r += (!val);
return r;
# endif
# else
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r = 0;
_BitScanForward64( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_ctzll(val) >> 3);
# else
static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 };
return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58];
# endif
# endif
}
#else
int LZ4_NbCommonBytes (register U32 val)
{
# if defined(LZ4_BIG_ENDIAN)
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r = 0;
_BitScanReverse( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_clz(val) >> 3);
# else
int r;
if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
r += (!val);
return r;
# endif
# else
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r;
_BitScanForward( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && (GCC_VERSION >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_ctz(val) >> 3);
# else
static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
# endif
# endif
}
#endif
/********************************
Compression functions
********************************/
int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); }
static int LZ4_hashSequence(U32 sequence, tableType_t tableType)
{
if (tableType == byU16)
return (((sequence) * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1)));
else
return (((sequence) * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG));
}
static int LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return LZ4_hashSequence(A32(p), tableType); }
static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, tableType_t tableType, const BYTE* srcBase)
{
switch (tableType)
{
case byPtr: { const BYTE** hashTable = (const BYTE**) tableBase; hashTable[h] = p; break; }
case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); break; }
case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); break; }
}
}
static void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase)
{
U32 h = LZ4_hashPosition(p, tableType);
LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase);
}
static const BYTE* LZ4_getPositionOnHash(U32 h, void* tableBase, tableType_t tableType, const BYTE* srcBase)
{
if (tableType == byPtr) { const BYTE** hashTable = (const BYTE**) tableBase; return hashTable[h]; }
if (tableType == byU32) { U32* hashTable = (U32*) tableBase; return hashTable[h] + srcBase; }
{ U16* hashTable = (U16*) tableBase; return hashTable[h] + srcBase; } /* default, to ensure a return */
}
static const BYTE* LZ4_getPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase)
{
U32 h = LZ4_hashPosition(p, tableType);
return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
}
static unsigned LZ4_count(const BYTE* pIn, const BYTE* pRef, const BYTE* pInLimit)
{
const BYTE* const pStart = pIn;
while (likely(pIndictSize;
const BYTE* const dictionary = dictPtr->dictionary;
const BYTE* const dictEnd = dictionary + dictPtr->dictSize;
const size_t dictDelta = dictEnd - (const BYTE*)source;
const BYTE* anchor = (const BYTE*) source;
const BYTE* const iend = ip + inputSize;
const BYTE* const mflimit = iend - MFLIMIT;
const BYTE* const matchlimit = iend - LASTLITERALS;
BYTE* op = (BYTE*) dest;
BYTE* const olimit = op + maxOutputSize;
const int skipStrength = SKIPSTRENGTH;
U32 forwardH;
size_t refDelta=0;
/* Init conditions */
if ((U32)inputSize > (U32)LZ4_MAX_INPUT_SIZE) return 0; /* Unsupported input size, too large (or negative) */
switch(dict)
{
case noDict:
default:
base = (const BYTE*)source;
lowLimit = (const BYTE*)source;
break;
case withPrefix64k:
base = (const BYTE*)source - dictPtr->currentOffset;
lowLimit = (const BYTE*)source - dictPtr->dictSize;
break;
case usingExtDict:
base = (const BYTE*)source - dictPtr->currentOffset;
lowLimit = (const BYTE*)source;
break;
}
if ((tableType == byU16) && (inputSize>=(int)LZ4_64KLIMIT)) return 0; /* Size too large (not within 64K limit) */
if (inputSize> skipStrength;
//if (step>8) step=8; // required for valid forwardIp ; slows down uncompressible data a bit
if (unlikely(forwardIp > mflimit)) goto _last_literals;
ref = LZ4_getPositionOnHash(h, ctx, tableType, base);
if (dict==usingExtDict)
{
if (ref<(const BYTE*)source)
{
refDelta = dictDelta;
lowLimit = dictionary;
}
else
{
refDelta = 0;
lowLimit = (const BYTE*)source;
}
}
forwardH = LZ4_hashPosition(forwardIp, tableType);
LZ4_putPositionOnHash(ip, h, ctx, tableType, base);
} while ( ((dictIssue==dictSmall) ? (ref < lowRefLimit) : 0)
|| ((tableType==byU16) ? 0 : (ref + MAX_DISTANCE < ip))
|| (A32(ref+refDelta) != A32(ip)) );
}
/* Catch up */
while ((ip>anchor) && (ref+refDelta > lowLimit) && (unlikely(ip[-1]==ref[refDelta-1]))) { ip--; ref--; }
{
/* Encode Literal length */
unsigned litLength = (unsigned)(ip - anchor);
token = op++;
if ((outputLimited) && (unlikely(op + litLength + (2 + 1 + LASTLITERALS) + (litLength/255) > olimit)))
return 0; /* Check output limit */
if (litLength>=RUN_MASK)
{
int len = (int)litLength-RUN_MASK;
*token=(RUN_MASK<= 255 ; len-=255) *op++ = 255;
*op++ = (BYTE)len;
}
else *token = (BYTE)(litLength< matchlimit) limit = matchlimit;
matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, limit);
ip += MINMATCH + matchLength;
if (ip==limit)
{
unsigned more = LZ4_count(ip, (const BYTE*)source, matchlimit);
matchLength += more;
ip += more;
}
}
else
{
matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, matchlimit);
ip += MINMATCH + matchLength;
}
if (matchLength>=ML_MASK)
{
if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) + (matchLength>>8) > olimit)))
return 0; /* Check output limit */
*token += ML_MASK;
matchLength -= ML_MASK;
for (; matchLength >= 510 ; matchLength-=510) { *op++ = 255; *op++ = 255; }
if (matchLength >= 255) { matchLength-=255; *op++ = 255; }
*op++ = (BYTE)matchLength;
}
else *token += (BYTE)(matchLength);
}
anchor = ip;
/* Test end of chunk */
if (ip > mflimit) break;
/* Fill table */
LZ4_putPosition(ip-2, ctx, tableType, base);
/* Test next position */
ref = LZ4_getPosition(ip, ctx, tableType, base);
if (dict==usingExtDict)
{
if (ref<(const BYTE*)source)
{
refDelta = dictDelta;
lowLimit = dictionary;
}
else
{
refDelta = 0;
lowLimit = (const BYTE*)source;
}
}
LZ4_putPosition(ip, ctx, tableType, base);
if ( ((dictIssue==dictSmall) ? (ref>=lowRefLimit) : 1)
&& (ref+MAX_DISTANCE>=ip)
&& (A32(ref+refDelta)==A32(ip)) )
{ token=op++; *token=0; goto _next_match; }
/* Prepare next loop */
forwardH = LZ4_hashPosition(++ip, tableType);
}
_last_literals:
/* Encode Last Literals */
{
int lastRun = (int)(iend - anchor);
if ((outputLimited) && (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize))
return 0; /* Check output limit */
if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK<= 255 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; }
else *op++ = (BYTE)(lastRun<= sizeof(LZ4_stream_t_internal)); /* A compilation error here means LZ4_STREAMSIZE is not large enough */
if (dict->initCheck) MEM_INIT(dict, 0, sizeof(LZ4_stream_t_internal)); /* Uninitialized structure detected */
if (dictSize < MINMATCH)
{
dict->dictionary = NULL;
dict->dictSize = 0;
return 1;
}
if (p <= dictEnd - 64 KB) p = dictEnd - 64 KB;
base = p - dict->currentOffset;
dict->dictionary = p;
dict->dictSize = (U32)(dictEnd - p);
dict->currentOffset += dict->dictSize;
while (p <= dictEnd-MINMATCH)
{
LZ4_putPosition(p, dict, byU32, base);
p+=3;
}
return 1;
}
void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src)
{
if ((LZ4_dict->currentOffset > 0x80000000) ||
((size_t)LZ4_dict->currentOffset > (size_t)src)) /* address space overflow */
{
/* rescale hash table */
U32 delta = LZ4_dict->currentOffset - 64 KB;
const BYTE* dictEnd = LZ4_dict->dictionary + LZ4_dict->dictSize;
int i;
for (i=0; ihashTable[i] < delta) LZ4_dict->hashTable[i]=0;
else LZ4_dict->hashTable[i] -= delta;
}
LZ4_dict->currentOffset = 64 KB;
if (LZ4_dict->dictSize > 64 KB) LZ4_dict->dictSize = 64 KB;
LZ4_dict->dictionary = dictEnd - LZ4_dict->dictSize;
}
}
FORCE_INLINE int LZ4_compress_continue_generic (void* LZ4_stream, const char* source, char* dest, int inputSize,
int maxOutputSize, limitedOutput_directive limit)
{
LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_stream;
const BYTE* const dictEnd = streamPtr->dictionary + streamPtr->dictSize;
const BYTE* smallest = (const BYTE*) source;
if (streamPtr->initCheck) return 0; /* Uninitialized structure detected */
if ((streamPtr->dictSize>0) && (smallest>dictEnd)) smallest = dictEnd;
LZ4_renormDictT(streamPtr, smallest);
/* Check overlapping input/dictionary space */
{
const BYTE* sourceEnd = (const BYTE*) source + inputSize;
if ((sourceEnd > streamPtr->dictionary) && (sourceEnd < dictEnd))
{
streamPtr->dictSize = (U32)(dictEnd - sourceEnd);
if (streamPtr->dictSize > 64 KB) streamPtr->dictSize = 64 KB;
if (streamPtr->dictSize < 4) streamPtr->dictSize = 0;
streamPtr->dictionary = dictEnd - streamPtr->dictSize;
}
}
/* prefix mode : source data follows dictionary */
if (dictEnd == (const BYTE*)source)
{
int result;
if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset))
result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, withPrefix64k, dictSmall);
else
result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, withPrefix64k, noDictIssue);
streamPtr->dictSize += (U32)inputSize;
streamPtr->currentOffset += (U32)inputSize;
return result;
}
/* external dictionary mode */
{
int result;
if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset))
result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, usingExtDict, dictSmall);
else
result = LZ4_compress_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limit, byU32, usingExtDict, noDictIssue);
streamPtr->dictionary = (const BYTE*)source;
streamPtr->dictSize = (U32)inputSize;
streamPtr->currentOffset += (U32)inputSize;
return result;
}
}
int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize)
{
return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, 0, notLimited);
}
int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize)
{
return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize, maxOutputSize, limitedOutput);
}
// Hidden debug function, to force separate dictionary mode
int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int inputSize)
{
LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_dict;
int result;
const BYTE* const dictEnd = streamPtr->dictionary + streamPtr->dictSize;
const BYTE* smallest = dictEnd;
if (smallest > (const BYTE*) source) smallest = (const BYTE*) source;
LZ4_renormDictT((LZ4_stream_t_internal*)LZ4_dict, smallest);
result = LZ4_compress_generic(LZ4_dict, source, dest, inputSize, 0, notLimited, byU32, usingExtDict, noDictIssue);
streamPtr->dictionary = (const BYTE*)source;
streamPtr->dictSize = (U32)inputSize;
streamPtr->currentOffset += (U32)inputSize;
return result;
}
int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int dictSize)
{
LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict;
const BYTE* previousDictEnd = dict->dictionary + dict->dictSize;
if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a dictionary > 64 KB */
if ((U32)dictSize > dict->dictSize) dictSize = dict->dictSize;
memcpy(safeBuffer, previousDictEnd - dictSize, dictSize);
dict->dictionary = (const BYTE*)safeBuffer;
dict->dictSize = (U32)dictSize;
return 1;
}
/****************************
Decompression functions
****************************/
/*
* This generic decompression function cover all use cases.
* It shall be instanciated several times, using different sets of directives
* Note that it is essential this generic function is really inlined,
* in order to remove useless branches during compilation optimisation.
*/
FORCE_INLINE int LZ4_decompress_generic(
const char* source,
char* dest,
int inputSize,
int outputSize, /* If endOnInput==endOnInputSize, this value is the max size of Output Buffer. */
int endOnInput, /* endOnOutputSize, endOnInputSize */
int partialDecoding, /* full, partial */
int targetOutputSize, /* only used if partialDecoding==partial */
int dict, /* noDict, withPrefix64k, usingExtDict */
const char* dictStart, /* only if dict==usingExtDict */
int dictSize /* note : = 0 if noDict */
)
{
/* Local Variables */
const BYTE* restrict ip = (const BYTE*) source;
const BYTE* ref;
const BYTE* const iend = ip + inputSize;
BYTE* op = (BYTE*) dest;
BYTE* const oend = op + outputSize;
BYTE* cpy;
BYTE* oexit = op + targetOutputSize;
const BYTE* const lowLimit = (const BYTE*)dest - dictSize;
const BYTE* const dictEnd = (const BYTE*)dictStart + dictSize;
//#define OLD
#ifdef OLD
const size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */
#else
const size_t dec32table[] = {4-0, 4-3, 4-2, 4-3, 4-0, 4-0, 4-0, 4-0}; /* static reduces speed for LZ4_decompress_safe() on GCC64 */
#endif
static const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3};
const int checkOffset = (endOnInput) && (dictSize < (int)(64 KB));
/* Special cases */
if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT; /* targetOutputSize too high => decode everything */
if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) && (*ip==0)) ? 0 : -1; /* Empty output buffer */
if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0?1:-1);
/* Main Loop */
while (1)
{
unsigned token;
size_t length;
/* get runlength */
token = *ip++;
if ((length=(token>>ML_BITS)) == RUN_MASK)
{
unsigned s;
do
{
s = *ip++;
length += s;
}
while (likely((endOnInput)?ipLZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */
if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)(op))) goto _output_error; /* quickfix issue 134 */
if ((endOnInput) && (sizeof(void*)==4) && unlikely((size_t)(ip+length)<(size_t)(ip))) goto _output_error; /* quickfix issue 134 */
}
/* copy literals */
cpy = op+length;
if (((endOnInput) && ((cpy>(partialDecoding?oexit:oend-MFLIMIT)) || (ip+length>iend-(2+1+LASTLITERALS))) )
|| ((!endOnInput) && (cpy>oend-COPYLENGTH)))
{
if (partialDecoding)
{
if (cpy > oend) goto _output_error; /* Error : write attempt beyond end of output buffer */
if ((endOnInput) && (ip+length > iend)) goto _output_error; /* Error : read attempt beyond end of input buffer */
}
else
{
if ((!endOnInput) && (cpy != oend)) goto _output_error; /* Error : block decoding must stop exactly there */
if ((endOnInput) && ((ip+length != iend) || (cpy > oend))) goto _output_error; /* Error : input must be consumed */
}
memcpy(op, ip, length);
ip += length;
op += length;
break; /* Necessarily EOF, due to parsing restrictions */
}
LZ4_WILDCOPY(op, ip, cpy); ip -= (op-cpy); op = cpy;
/* get offset */
LZ4_READ_LITTLEENDIAN_16(ref,cpy,ip); ip+=2;
if ((checkOffset) && (unlikely(ref < lowLimit))) goto _output_error; /* Error : offset outside destination buffer */
/* get matchlength */
if ((length=(token&ML_MASK)) == ML_MASK)
{
unsigned s;
do
{
if ((endOnInput) && (ip > iend-LASTLITERALS)) goto _output_error;
s = *ip++;
length += s;
} while (s==255);
//if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE)) goto _output_error; /* overflow detection */
if ((sizeof(void*)==4) && unlikely((size_t)(op+length)<(size_t)op)) goto _output_error; /* quickfix issue 134 */
}
/* check external dictionary */
if ((dict==usingExtDict) && (ref < (BYTE* const)dest))
{
if (unlikely(op+length+MINMATCH > oend-LASTLITERALS)) goto _output_error;
if (length+MINMATCH <= (size_t)(dest-(char*)ref))
{
ref = dictEnd - (dest-(char*)ref);
memcpy(op, ref, length+MINMATCH);
op += length+MINMATCH;
}
else
{
size_t copySize = (size_t)(dest-(char*)ref);
memcpy(op, dictEnd - copySize, copySize);
op += copySize;
copySize = length+MINMATCH - copySize;
if (copySize > (size_t)((char*)op-dest)) /* overlap */
{
BYTE* const cpy = op + copySize;
const BYTE* ref = (BYTE*)dest;
while (op < cpy) *op++ = *ref++;
}
else
{
memcpy(op, dest, copySize);
op += copySize;
}
}
continue;
}
/* copy repeated sequence */
if (unlikely((op-ref)<(int)STEPSIZE))
{
const size_t dec64 = dec64table[(sizeof(void*)==4) ? 0 : op-ref];
op[0] = ref[0];
op[1] = ref[1];
op[2] = ref[2];
op[3] = ref[3];
#ifdef OLD
op += 4, ref += 4; ref -= dec32table[op-ref];
A32(op) = A32(ref);
op += STEPSIZE-4; ref -= dec64;
#else
ref += dec32table[op-ref];
A32(op+4) = A32(ref);
op += STEPSIZE; ref -= dec64;
#endif
} else { LZ4_COPYSTEP(op,ref); }
cpy = op + length - (STEPSIZE-4);
if (unlikely(cpy>oend-COPYLENGTH-(STEPSIZE-4)))
{
if (cpy > oend-LASTLITERALS) goto _output_error; /* Error : last 5 bytes must be literals */
if (opdictionary = dictionary;
lz4sd->dictSize = dictSize;
return 1;
}
/*
*_continue() :
These decoding functions allow decompression of multiple blocks in "streaming" mode.
Previously decoded blocks must still be available at the memory position where they were decoded.
If it's not possible, save the relevant part of decoded data into a safe buffer,
and indicate where it stands using LZ4_setDictDecode()
*/
int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize)
{
LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode;
int result;
result = LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize);
if (result <= 0) return result;
if (lz4sd->dictionary + lz4sd->dictSize == dest)
{
lz4sd->dictSize += result;
}
else
{
lz4sd->dictionary = dest;
lz4sd->dictSize = result;
}
return result;
}
int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize)
{
LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*) LZ4_streamDecode;
int result;
result = LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize);
if (result <= 0) return result;
if (lz4sd->dictionary + lz4sd->dictSize == dest)
{
lz4sd->dictSize += result;
}
else
{
lz4sd->dictionary = dest;
lz4sd->dictSize = result;
}
return result;
}
/*
Advanced decoding functions :
*_usingDict() :
These decoding functions work the same as "_continue" ones,
the dictionary must be explicitly provided within parameters
*/
int LZ4_decompress_safe_usingDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize)
{
return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, usingExtDict, dictStart, dictSize);
}
int LZ4_decompress_fast_usingDict(const char* source, char* dest, int originalSize, const char* dictStart, int dictSize)
{
return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, usingExtDict, dictStart, dictSize);
}
/***************************************************
Obsolete Functions
***************************************************/
/*
These function names are deprecated and should no longer be used.
They are only provided here for compatibility with older user programs.
- LZ4_uncompress is totally equivalent to LZ4_decompress_fast
- LZ4_uncompress_unknownOutputSize is totally equivalent to LZ4_decompress_safe
*/
int LZ4_uncompress (const char* source, char* dest, int outputSize) { return LZ4_decompress_fast(source, dest, outputSize); }
int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize) { return LZ4_decompress_safe(source, dest, isize, maxOutputSize); }
/* Obsolete Streaming functions */
int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; }
void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base)
{
MEM_INIT(lz4ds, 0, LZ4_STREAMSIZE);
lz4ds->bufferStart = base;
}
int LZ4_resetStreamState(void* state, const char* inputBuffer)
{
if ((((size_t)state) & 3) != 0) return 1; /* Error : pointer is not aligned on 4-bytes boundary */
LZ4_init((LZ4_stream_t_internal*)state, (const BYTE*)inputBuffer);
return 0;
}
void* LZ4_create (const char* inputBuffer)
{
void* lz4ds = ALLOCATOR(4, LZ4_STREAMSIZE_U32);
LZ4_init ((LZ4_stream_t_internal*)lz4ds, (const BYTE*)inputBuffer);
return lz4ds;
}
char* LZ4_slideInputBuffer (void* LZ4_Data)
{
LZ4_stream_t_internal* lz4ds = (LZ4_stream_t_internal*)LZ4_Data;
LZ4_saveDict((LZ4_stream_t*)LZ4_Data, (char*)lz4ds->bufferStart, 64 KB);
return (char*)(lz4ds->bufferStart + 64 KB);
}
/* Obsolete compresson functions using User-allocated state */
int LZ4_sizeofState() { return LZ4_STREAMSIZE; }
int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize)
{
if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */
MEM_INIT(state, 0, LZ4_STREAMSIZE);
if (inputSize < (int)LZ4_64KLIMIT)
return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, byU16, noDict, noDictIssue);
else
return LZ4_compress_generic(state, source, dest, inputSize, 0, notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue);
}
int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize)
{
if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned on 4-bytes boundary */
MEM_INIT(state, 0, LZ4_STREAMSIZE);
if (inputSize < (int)LZ4_64KLIMIT)
return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue);
else
return LZ4_compress_generic(state, source, dest, inputSize, maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue);
}
/* Obsolete streaming decompression functions */
int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int compressedSize, int maxOutputSize)
{
return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, endOnInputSize, full, 0, withPrefix64k, NULL, 64 KB);
}
int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int originalSize)
{
return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, NULL, 64 KB);
}
horde_lz4-1.0.9/lib/lz4.h 0000664 0001750 0001750 00000032475 12653745320 013171 0 ustar jan jan /*
LZ4 - Fast LZ compression algorithm
Header File
Copyright (C) 2011-2014, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- LZ4 source repository : http://code.google.com/p/lz4/
- LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
#pragma once
#if defined (__cplusplus)
extern "C" {
#endif
/**************************************
Version
**************************************/
#define LZ4_VERSION_MAJOR 1 /* for major interface/format changes */
#define LZ4_VERSION_MINOR 2 /* for minor interface/format changes */
#define LZ4_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */
/**************************************
Tuning parameter
**************************************/
/*
* LZ4_MEMORY_USAGE :
* Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
* Increasing memory usage improves compression ratio
* Reduced memory usage can improve speed, due to cache effect
* Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache
*/
#define LZ4_MEMORY_USAGE 14
/**************************************
Simple Functions
**************************************/
int LZ4_compress (const char* source, char* dest, int inputSize);
int LZ4_decompress_safe (const char* source, char* dest, int compressedSize, int maxOutputSize);
/*
LZ4_compress() :
Compresses 'inputSize' bytes from 'source' into 'dest'.
Destination buffer must be already allocated,
and must be sized to handle worst cases situations (input data not compressible)
Worst case size evaluation is provided by function LZ4_compressBound()
inputSize : Max supported value is LZ4_MAX_INPUT_VALUE
return : the number of bytes written in buffer dest
or 0 if the compression fails
LZ4_decompress_safe() :
compressedSize : is obviously the source size
maxOutputSize : is the size of the destination buffer, which must be already allocated.
return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize)
If the destination buffer is not large enough, decoding will stop and output an error code (<0).
If the source stream is detected malformed, the function will stop decoding and return a negative result.
This function is protected against buffer overflow exploits :
it never writes outside of output buffer, and never reads outside of input buffer.
Therefore, it is protected against malicious data packets.
*/
/*
Note :
Should you prefer to explicitly allocate compression-table memory using your own allocation method,
use the streaming functions provided below, simply reset the memory area between each call to LZ4_compress_continue()
*/
/**************************************
Advanced Functions
**************************************/
#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
#define LZ4_COMPRESSBOUND(isize) ((unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16)
/*
LZ4_compressBound() :
Provides the maximum size that LZ4 may output in a "worst case" scenario (input data not compressible)
primarily useful for memory allocation of output buffer.
macro is also provided when result needs to be evaluated at compilation (such as stack memory allocation).
isize : is the input size. Max supported value is LZ4_MAX_INPUT_SIZE
return : maximum output size in a "worst case" scenario
or 0, if input size is too large ( > LZ4_MAX_INPUT_SIZE)
*/
int LZ4_compressBound(int isize);
/*
LZ4_compress_limitedOutput() :
Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of maximum size 'maxOutputSize'.
If it cannot achieve it, compression will stop, and result of the function will be zero.
This function never writes outside of provided output buffer.
inputSize : Max supported value is LZ4_MAX_INPUT_VALUE
maxOutputSize : is the size of the destination buffer (which must be already allocated)
return : the number of bytes written in buffer 'dest'
or 0 if the compression fails
*/
int LZ4_compress_limitedOutput (const char* source, char* dest, int inputSize, int maxOutputSize);
/*
LZ4_decompress_fast() :
originalSize : is the original and therefore uncompressed size
return : the number of bytes read from the source buffer (in other words, the compressed size)
If the source stream is malformed, the function will stop decoding and return a negative result.
Destination buffer must be already allocated. Its size must be a minimum of 'originalSize' bytes.
note : This function is a bit faster than LZ4_decompress_safe()
It provides fast decompression and fully respect memory boundaries for properly formed compressed data.
It does not provide full protection against intentionnally modified data stream.
Use this function in a trusted environment (data to decode comes from a trusted source).
*/
int LZ4_decompress_fast (const char* source, char* dest, int originalSize);
/*
LZ4_decompress_safe_partial() :
This function decompress a compressed block of size 'compressedSize' at position 'source'
into output buffer 'dest' of size 'maxOutputSize'.
The function tries to stop decompressing operation as soon as 'targetOutputSize' has been reached,
reducing decompression time.
return : the number of bytes decoded in the destination buffer (necessarily <= maxOutputSize)
Note : this number can be < 'targetOutputSize' should the compressed block to decode be smaller.
Always control how many bytes were decoded.
If the source stream is detected malformed, the function will stop decoding and return a negative result.
This function never writes outside of output buffer, and never reads outside of input buffer. It is therefore protected against malicious data packets
*/
int LZ4_decompress_safe_partial (const char* source, char* dest, int compressedSize, int targetOutputSize, int maxOutputSize);
/***********************************************
Experimental Streaming Compression Functions
***********************************************/
#define LZ4_STREAMSIZE_U32 ((1 << (LZ4_MEMORY_USAGE-2)) + 8)
#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U32 * sizeof(unsigned int))
/*
* LZ4_stream_t
* information structure to track an LZ4 stream.
* important : set this structure content to zero before first use !
*/
typedef struct { unsigned int table[LZ4_STREAMSIZE_U32]; } LZ4_stream_t;
/*
* If you prefer dynamic allocation methods,
* LZ4_createStream
* provides a pointer (void*) towards an initialized LZ4_stream_t structure.
* LZ4_free just frees it.
*/
void* LZ4_createStream();
int LZ4_free (void* LZ4_stream);
/*
* LZ4_loadDict
* Use this function to load a static dictionary into LZ4_stream.
* Any previous data will be forgotten, only 'dictionary' will remain in memory.
* Loading a size of 0 is allowed (same effect as init).
* Return : 1 if OK, 0 if error
*/
int LZ4_loadDict (void* LZ4_stream, const char* dictionary, int dictSize);
/*
* LZ4_compress_continue
* Compress data block 'source', using blocks compressed before as dictionary to improve compression ratio
* Previous data blocks are assumed to still be present at their previous location.
*/
int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest, int inputSize);
/*
* LZ4_compress_limitedOutput_continue
* Same as before, but also specify a maximum target compressed size (maxOutputSize)
* If objective cannot be met, compression exits, and returns a zero.
*/
int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source, char* dest, int inputSize, int maxOutputSize);
/*
* LZ4_saveDict
* If previously compressed data block is not guaranteed to remain at its previous memory location
* save it into a safe place (char* safeBuffer)
* Note : you don't need to call LZ4_loadDict() afterwards,
* dictionary is immediately usable, you can therefore call again LZ4_compress_continue()
* Return : 1 if OK, 0 if error
* Note : any dictSize > 64 KB will be interpreted as 64KB.
*/
int LZ4_saveDict (void* LZ4_stream, char* safeBuffer, int dictSize);
/************************************************
Experimental Streaming Decompression Functions
************************************************/
#define LZ4_STREAMDECODESIZE_U32 4
#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned int))
/*
* LZ4_streamDecode_t
* information structure to track an LZ4 stream.
* important : set this structure content to zero before first use !
*/
typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; } LZ4_streamDecode_t;
/*
* If you prefer dynamic allocation methods,
* LZ4_createStreamDecode()
* provides a pointer (void*) towards an initialized LZ4_streamDecode_t structure.
* LZ4_free just frees it.
*/
void* LZ4_createStreamDecode();
int LZ4_free (void* LZ4_stream); /* yes, it's the same one as for compression */
/*
*_continue() :
These decoding functions allow decompression of multiple blocks in "streaming" mode.
Previously decoded blocks must still be available at the memory position where they were decoded.
If it's not possible, save the relevant part of decoded data into a safe buffer,
and indicate where it stands using LZ4_setDictDecode()
*/
int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize);
int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source, char* dest, int originalSize);
/*
* LZ4_setDictDecode
* Use this function to instruct where to find the dictionary.
* This function can be used to specify a static dictionary,
* or to instruct where to find some previously decoded data saved into a different memory space.
* Setting a size of 0 is allowed (same effect as no dictionary).
* Return : 1 if OK, 0 if error
*/
int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int dictSize);
/*
Advanced decoding functions :
*_usingDict() :
These decoding functions work the same as
a combination of LZ4_setDictDecode() followed by LZ4_decompress_x_continue()
all together into a single function call.
It doesn't use nor update an LZ4_streamDecode_t structure.
*/
int LZ4_decompress_safe_usingDict (const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize);
int LZ4_decompress_fast_usingDict (const char* source, char* dest, int originalSize, const char* dictStart, int dictSize);
/**************************************
Obsolete Functions
**************************************/
/*
Obsolete decompression functions
These function names are deprecated and should no longer be used.
They are only provided here for compatibility with older user programs.
- LZ4_uncompress is the same as LZ4_decompress_fast
- LZ4_uncompress_unknownOutputSize is the same as LZ4_decompress_safe
*/
int LZ4_uncompress (const char* source, char* dest, int outputSize);
int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize);
/* Obsolete functions for externally allocated state; use streaming interface instead */
int LZ4_sizeofState(void);
int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize);
int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize);
/* Obsolete streaming functions; use new streaming interface whenever possible */
void* LZ4_create (const char* inputBuffer);
int LZ4_sizeofStreamState(void);
int LZ4_resetStreamState(void* state, const char* inputBuffer);
char* LZ4_slideInputBuffer (void* state);
/* Obsolete streaming decoding functions */
int LZ4_decompress_safe_withPrefix64k (const char* source, char* dest, int compressedSize, int maxOutputSize);
int LZ4_decompress_fast_withPrefix64k (const char* source, char* dest, int originalSize);
#if defined (__cplusplus)
}
#endif
horde_lz4-1.0.9/lib/lz4hc.c 0000664 0001750 0001750 00000072050 12653745320 013470 0 ustar jan jan /*
LZ4 HC - High Compression Mode of LZ4
Copyright (C) 2011-2014, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
- LZ4 source repository : http://code.google.com/p/lz4/
*/
/**************************************
Tuning Parameter
**************************************/
#define LZ4HC_DEFAULT_COMPRESSIONLEVEL 8
/**************************************
Memory routines
**************************************/
#include /* calloc, free */
#define ALLOCATOR(s) calloc(1,s)
#define FREEMEM free
#include /* memset, memcpy */
#define MEM_INIT memset
/**************************************
CPU Feature Detection
**************************************/
/* 32 or 64 bits ? */
#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
|| defined(__powerpc64__) || defined(__powerpc64le__) \
|| defined(__ppc64__) || defined(__ppc64le__) \
|| defined(__PPC64__) || defined(__PPC64LE__) \
|| defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /* Detects 64 bits mode */
# define LZ4_ARCH64 1
#else
# define LZ4_ARCH64 0
#endif
/*
* Little Endian or Big Endian ?
* Overwrite the #define below if you know your architecture endianess
*/
#include /* Apparently required to detect endianess */
#if defined (__GLIBC__)
# include
# if (__BYTE_ORDER == __BIG_ENDIAN)
# define LZ4_BIG_ENDIAN 1
# endif
#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) || defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) || defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN))
# define LZ4_BIG_ENDIAN 1
#elif defined(__sparc) || defined(__sparc__) \
|| defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
|| defined(__hpux) || defined(__hppa) \
|| defined(_MIPSEB) || defined(__s390__)
# define LZ4_BIG_ENDIAN 1
#else
/* Little Endian assumed. PDP Endian and other very rare endian format are unsupported. */
#endif
/*
* Unaligned memory access is automatically enabled for "common" CPU, such as x86.
* For others CPU, the compiler will be more cautious, and insert extra code to ensure aligned access is respected
* If you know your target CPU supports unaligned memory access, you want to force this option manually to improve performance
*/
#if defined(__ARM_FEATURE_UNALIGNED)
# define LZ4_FORCE_UNALIGNED_ACCESS 1
#endif
/* Define this parameter if your target system or compiler does not support hardware bit count */
#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE does not support Hardware bit count */
# define LZ4_FORCE_SW_BITCOUNT
#endif
/**************************************
Compiler Options
**************************************/
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
/* "restrict" is a known keyword */
#else
# define restrict /* Disable restrict */
#endif
#ifdef _MSC_VER /* Visual Studio */
# define FORCE_INLINE static __forceinline
# include /* For Visual 2005 */
# if LZ4_ARCH64 /* 64-bits */
# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
# else /* 32-bits */
# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
# endif
# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */
# pragma warning(disable : 4701) /* disable: C4701: potentially uninitialized local variable used */
#else
# ifdef __GNUC__
# define FORCE_INLINE static inline __attribute__((always_inline))
# else
# define FORCE_INLINE static inline
# endif
#endif
#ifdef _MSC_VER /* Visual Studio */
# define lz4_bswap16(x) _byteswap_ushort(x)
#else
# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) & 0xffu) << 8)))
#endif
/**************************************
Includes
**************************************/
#include "lz4hc.h"
#include "lz4.h"
/**************************************
Basic Types
**************************************/
#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
# include
typedef uint8_t BYTE;
typedef uint16_t U16;
typedef uint32_t U32;
typedef int32_t S32;
typedef uint64_t U64;
#else
typedef unsigned char BYTE;
typedef unsigned short U16;
typedef unsigned int U32;
typedef signed int S32;
typedef unsigned long long U64;
#endif
#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS)
# define _PACKED __attribute__ ((packed))
#else
# define _PACKED
#endif
#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
# ifdef __IBMC__
# pragma pack(1)
# else
# pragma pack(push, 1)
# endif
#endif
typedef struct _U16_S { U16 v; } _PACKED U16_S;
typedef struct _U32_S { U32 v; } _PACKED U32_S;
typedef struct _U64_S { U64 v; } _PACKED U64_S;
#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
# pragma pack(pop)
#endif
#define A64(x) (((U64_S *)(x))->v)
#define A32(x) (((U32_S *)(x))->v)
#define A16(x) (((U16_S *)(x))->v)
/**************************************
Constants
**************************************/
#define MINMATCH 4
#define DICTIONARY_LOGSIZE 16
#define MAXD (1<> ((MINMATCH*8)-HASH_LOG))
#define HASH_VALUE(p) HASH_FUNCTION(A32(p))
#define HASH_POINTER(p) (HashTable[HASH_VALUE(p)] + base)
#define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK]
#define GETNEXT(p) ((p) - (size_t)DELTANEXT(p))
/**************************************
Private functions
**************************************/
#if LZ4_ARCH64
FORCE_INLINE int LZ4_NbCommonBytes (register U64 val)
{
#if defined(LZ4_BIG_ENDIAN)
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r = 0;
_BitScanReverse64( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_clzll(val) >> 3);
# else
int r;
if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
r += (!val);
return r;
# endif
#else
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r = 0;
_BitScanForward64( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_ctzll(val) >> 3);
# else
static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7 };
return DeBruijnBytePos[((U64)((val & -val) * 0x0218A392CDABBD3F)) >> 58];
# endif
#endif
}
#else
FORCE_INLINE int LZ4_NbCommonBytes (register U32 val)
{
#if defined(LZ4_BIG_ENDIAN)
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r;
_BitScanReverse( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_clz(val) >> 3);
# else
int r;
if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
r += (!val);
return r;
# endif
#else
# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
unsigned long r;
_BitScanForward( &r, val );
return (int)(r>>3);
# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) && !defined(LZ4_FORCE_SW_BITCOUNT)
return (__builtin_ctz(val) >> 3);
# else
static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
# endif
#endif
}
#endif
int LZ4_sizeofStreamStateHC()
{
return sizeof(LZ4HC_Data_Structure);
}
FORCE_INLINE void LZ4_initHC (LZ4HC_Data_Structure* hc4, const BYTE* base)
{
MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable));
MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable));
hc4->nextToUpdate = base + 1;
hc4->base = base;
hc4->inputBuffer = base;
hc4->end = base;
}
int LZ4_resetStreamStateHC(void* state, const char* inputBuffer)
{
if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error : pointer is not aligned for pointer (32 or 64 bits) */
LZ4_initHC((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer);
return 0;
}
void* LZ4_createHC (const char* inputBuffer)
{
void* hc4 = ALLOCATOR(sizeof(LZ4HC_Data_Structure));
LZ4_initHC ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer);
return hc4;
}
int LZ4_freeHC (void* LZ4HC_Data)
{
FREEMEM(LZ4HC_Data);
return (0);
}
/* Update chains up to ip (excluded) */
FORCE_INLINE void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip)
{
U16* chainTable = hc4->chainTable;
HTYPE* HashTable = hc4->hashTable;
INITBASE(base,hc4->base);
while(hc4->nextToUpdate < ip)
{
const BYTE* const p = hc4->nextToUpdate;
size_t delta = (p) - HASH_POINTER(p);
if (delta>MAX_DISTANCE) delta = MAX_DISTANCE;
DELTANEXT(p) = (U16)delta;
HashTable[HASH_VALUE(p)] = (HTYPE)((p) - base);
hc4->nextToUpdate++;
}
}
char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
{
LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data;
U32 distance = (U32)(hc4->end - hc4->inputBuffer) - 64 KB;
distance = (distance >> 16) << 16; /* Must be a multiple of 64 KB */
LZ4HC_Insert(hc4, hc4->end - MINMATCH);
memcpy((void*)(hc4->end - 64 KB - distance), (const void*)(hc4->end - 64 KB), 64 KB);
hc4->nextToUpdate -= distance;
hc4->base -= distance;
if ((U32)(hc4->inputBuffer - hc4->base) > 1 GB + 64 KB) /* Avoid overflow */
{
int i;
hc4->base += 1 GB;
for (i=0; ihashTable[i] -= 1 GB;
}
hc4->end -= distance;
return (char*)(hc4->end);
}
FORCE_INLINE size_t LZ4HC_CommonLength (const BYTE* p1, const BYTE* p2, const BYTE* const matchlimit)
{
const BYTE* p1t = p1;
while (p1tchainTable;
HTYPE* const HashTable = hc4->hashTable;
const BYTE* ref;
INITBASE(base,hc4->base);
int nbAttempts=maxNbAttempts;
size_t repl=0, ml=0;
U16 delta=0; /* useless assignment, to remove an uninitialization warning */
/* HC4 match finder */
LZ4HC_Insert(hc4, ip);
ref = HASH_POINTER(ip);
#define REPEAT_OPTIMIZATION
#ifdef REPEAT_OPTIMIZATION
/* Detect repetitive sequences of length <= 4 */
if ((U32)(ip-ref) <= 4) /* potential repetition */
{
if (A32(ref) == A32(ip)) /* confirmed */
{
delta = (U16)(ip-ref);
repl = ml = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH;
*matchpos = ref;
}
ref = GETNEXT(ref);
}
#endif
while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts))
{
nbAttempts--;
if (*(ref+ml) == *(ip+ml))
if (A32(ref) == A32(ip))
{
size_t mlt = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH, matchlimit) + MINMATCH;
if (mlt > ml) { ml = mlt; *matchpos = ref; }
}
ref = GETNEXT(ref);
}
#ifdef REPEAT_OPTIMIZATION
/* Complete table */
if (repl)
{
const BYTE* ptr = ip;
const BYTE* end;
end = ip + repl - (MINMATCH-1);
while(ptr < end-delta)
{
DELTANEXT(ptr) = delta; /* Pre-Load */
ptr++;
}
do
{
DELTANEXT(ptr) = delta;
HashTable[HASH_VALUE(ptr)] = (HTYPE)((ptr) - base); /* Head of chain */
ptr++;
} while(ptr < end);
hc4->nextToUpdate = end;
}
#endif
return (int)ml;
}
FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (LZ4HC_Data_Structure* hc4, const BYTE* ip, const BYTE* startLimit, const BYTE* matchlimit, int longest, const BYTE** matchpos, const BYTE** startpos, const int maxNbAttempts)
{
U16* const chainTable = hc4->chainTable;
HTYPE* const HashTable = hc4->hashTable;
INITBASE(base,hc4->base);
const BYTE* ref;
int nbAttempts = maxNbAttempts;
int delta = (int)(ip-startLimit);
/* First Match */
LZ4HC_Insert(hc4, ip);
ref = HASH_POINTER(ip);
while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts))
{
nbAttempts--;
if (*(startLimit + longest) == *(ref - delta + longest))
if (A32(ref) == A32(ip))
{
#if 1
const BYTE* reft = ref+MINMATCH;
const BYTE* ipt = ip+MINMATCH;
const BYTE* startt = ip;
while (iptstartLimit) && (reft > hc4->inputBuffer) && (startt[-1] == reft[-1])) {startt--; reft--;}
if ((ipt-startt) > longest)
{
longest = (int)(ipt-startt);
*matchpos = reft;
*startpos = startt;
}
}
ref = GETNEXT(ref);
}
return longest;
}
typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive;
FORCE_INLINE int LZ4HC_encodeSequence (
const BYTE** ip,
BYTE** op,
const BYTE** anchor,
int matchLength,
const BYTE* ref,
limitedOutput_directive limitedOutputBuffer,
BYTE* oend)
{
int length;
BYTE* token;
/* Encode Literal length */
length = (int)(*ip - *anchor);
token = (*op)++;
if ((limitedOutputBuffer) && ((*op + length + (2 + 1 + LASTLITERALS) + (length>>8)) > oend)) return 1; /* Check output limit */
if (length>=(int)RUN_MASK) { int len; *token=(RUN_MASK< 254 ; len-=255) *(*op)++ = 255; *(*op)++ = (BYTE)len; }
else *token = (BYTE)(length<>8) > oend)) return 1; /* Check output limit */
if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length > 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) { length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; }
else *token += (BYTE)(length);
/* Prepare next loop */
*ip += matchLength;
*anchor = *ip;
return 0;
}
#define MAX_COMPRESSION_LEVEL 16
static int LZ4HC_compress_generic (
void* ctxvoid,
const char* source,
char* dest,
int inputSize,
int maxOutputSize,
int compressionLevel,
limitedOutput_directive limit
)
{
LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid;
const BYTE* ip = (const BYTE*) source;
const BYTE* anchor = ip;
const BYTE* const iend = ip + inputSize;
const BYTE* const mflimit = iend - MFLIMIT;
const BYTE* const matchlimit = (iend - LASTLITERALS);
BYTE* op = (BYTE*) dest;
BYTE* const oend = op + maxOutputSize;
const int maxNbAttempts = compressionLevel > MAX_COMPRESSION_LEVEL ? 1 << MAX_COMPRESSION_LEVEL : compressionLevel ? 1<<(compressionLevel-1) : 1<end) return 0;
ctx->end += inputSize;
ip++;
/* Main Loop */
while (ip < mflimit)
{
ml = LZ4HC_InsertAndFindBestMatch (ctx, ip, matchlimit, (&ref), maxNbAttempts);
if (!ml) { ip++; continue; }
/* saved, in case we would skip too much */
start0 = ip;
ref0 = ref;
ml0 = ml;
_Search2:
if (ip+ml < mflimit)
ml2 = LZ4HC_InsertAndGetWiderMatch(ctx, ip + ml - 2, ip + 1, matchlimit, ml, &ref2, &start2, maxNbAttempts);
else ml2 = ml;
if (ml2 == ml) /* No better match */
{
if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0;
continue;
}
if (start0 < ip)
{
if (start2 < ip + ml0) /* empirical */
{
ip = start0;
ref = ref0;
ml = ml0;
}
}
/* Here, start0==ip */
if ((start2 - ip) < 3) /* First Match too small : removed */
{
ml = ml2;
ip = start2;
ref =ref2;
goto _Search2;
}
_Search3:
/*
* Currently we have :
* ml2 > ml1, and
* ip1+3 <= ip2 (usually < ip1+ml1)
*/
if ((start2 - ip) < OPTIMAL_ML)
{
int correction;
int new_ml = ml;
if (new_ml > OPTIMAL_ML) new_ml = OPTIMAL_ML;
if (ip+new_ml > start2 + ml2 - MINMATCH) new_ml = (int)(start2 - ip) + ml2 - MINMATCH;
correction = new_ml - (int)(start2 - ip);
if (correction > 0)
{
start2 += correction;
ref2 += correction;
ml2 -= correction;
}
}
/* Now, we have start2 = ip+new_ml, with new_ml = min(ml, OPTIMAL_ML=18) */
if (start2 + ml2 < mflimit)
ml3 = LZ4HC_InsertAndGetWiderMatch(ctx, start2 + ml2 - 3, start2, matchlimit, ml2, &ref3, &start3, maxNbAttempts);
else ml3 = ml2;
if (ml3 == ml2) /* No better match : 2 sequences to encode */
{
/* ip & ref are known; Now for ml */
if (start2 < ip+ml) ml = (int)(start2 - ip);
/* Now, encode 2 sequences */
if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0;
ip = start2;
if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml2, ref2, limit, oend)) return 0;
continue;
}
if (start3 < ip+ml+3) /* Not enough space for match 2 : remove it */
{
if (start3 >= (ip+ml)) /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */
{
if (start2 < ip+ml)
{
int correction = (int)(ip+ml - start2);
start2 += correction;
ref2 += correction;
ml2 -= correction;
if (ml2 < MINMATCH)
{
start2 = start3;
ref2 = ref3;
ml2 = ml3;
}
}
if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0;
ip = start3;
ref = ref3;
ml = ml3;
start0 = start2;
ref0 = ref2;
ml0 = ml2;
goto _Search2;
}
start2 = start3;
ref2 = ref3;
ml2 = ml3;
goto _Search3;
}
/*
* OK, now we have 3 ascending matches; let's write at least the first one
* ip & ref are known; Now for ml
*/
if (start2 < ip+ml)
{
if ((start2 - ip) < (int)ML_MASK)
{
int correction;
if (ml > OPTIMAL_ML) ml = OPTIMAL_ML;
if (ip + ml > start2 + ml2 - MINMATCH) ml = (int)(start2 - ip) + ml2 - MINMATCH;
correction = ml - (int)(start2 - ip);
if (correction > 0)
{
start2 += correction;
ref2 += correction;
ml2 -= correction;
}
}
else
{
ml = (int)(start2 - ip);
}
}
if (LZ4HC_encodeSequence(&ip, &op, &anchor, ml, ref, limit, oend)) return 0;
ip = start2;
ref = ref2;
ml = ml2;
start2 = start3;
ref2 = ref3;
ml2 = ml3;
goto _Search3;
}
/* Encode Last Literals */
{
int lastRun = (int)(iend - anchor);
if ((limit) && (((char*)op - dest) + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > (U32)maxOutputSize)) return 0; /* Check output limit */
if (lastRun>=(int)RUN_MASK) { *op++=(RUN_MASK< 254 ; lastRun-=255) *op++ = 255; *op++ = (BYTE) lastRun; }
else *op++ = (BYTE)(lastRun<