././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1602804847.5743518 kaitaistruct-0.9/0000777000000000000000000000000000000000000012224 5ustar0000000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1571508468.0 kaitaistruct-0.9/MANIFEST.in0000666000000000000000000000002400000000000013756 0ustar0000000000000000include README.rst ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1602804847.5743518 kaitaistruct-0.9/PKG-INFO0000666000000000000000000000425200000000000013324 0ustar0000000000000000Metadata-Version: 1.2 Name: kaitaistruct Version: 0.9 Summary: Kaitai Struct declarative parser generator for binary data: runtime library for Python Home-page: http://kaitai.io Author: Kaitai Project Author-email: greycat@kaitai.io License: MIT Description: Kaitai Struct: runtime library for Python =========================================== This library implements Kaitai Struct API for Python. `Kaitai Struct `_ is a declarative language used for describe various binary data structures, laid out in files or in memory: i.e. binary file formats, network stream packet formats, etc. It is similar to `Python's Construct 2.9 `_ but it is language-agnostic. The format description is done in YAML-based .ksy format, which then can be compiled into a wide range of target languages. Further reading: * `About Kaitai Struct `_ * `About API implemented in this library `_ * `Python-specific notes `_ in KS documentation discuss installation and usage of this runtime Keywords: kaitai,struct,construct,ksy,declarative,data structure,data format,file format,packet format,binary,parser,parsing,unpack,development Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Build Tools Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.* ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1581703961.0 kaitaistruct-0.9/README.rst0000666000000000000000000000156200000000000013717 0ustar0000000000000000Kaitai Struct: runtime library for Python =========================================== This library implements Kaitai Struct API for Python. `Kaitai Struct `_ is a declarative language used for describe various binary data structures, laid out in files or in memory: i.e. binary file formats, network stream packet formats, etc. It is similar to `Python's Construct 2.9 `_ but it is language-agnostic. The format description is done in YAML-based .ksy format, which then can be compiled into a wide range of target languages. Further reading: * `About Kaitai Struct `_ * `About API implemented in this library `_ * `Python-specific notes `_ in KS documentation discuss installation and usage of this runtime ././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1602804847.572352 kaitaistruct-0.9/kaitaistruct.egg-info/0000777000000000000000000000000000000000000016425 5ustar0000000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1602804847.0 kaitaistruct-0.9/kaitaistruct.egg-info/PKG-INFO0000666000000000000000000000425200000000000017525 0ustar0000000000000000Metadata-Version: 1.2 Name: kaitaistruct Version: 0.9 Summary: Kaitai Struct declarative parser generator for binary data: runtime library for Python Home-page: http://kaitai.io Author: Kaitai Project Author-email: greycat@kaitai.io License: MIT Description: Kaitai Struct: runtime library for Python =========================================== This library implements Kaitai Struct API for Python. `Kaitai Struct `_ is a declarative language used for describe various binary data structures, laid out in files or in memory: i.e. binary file formats, network stream packet formats, etc. It is similar to `Python's Construct 2.9 `_ but it is language-agnostic. The format description is done in YAML-based .ksy format, which then can be compiled into a wide range of target languages. Further reading: * `About Kaitai Struct `_ * `About API implemented in this library `_ * `Python-specific notes `_ in KS documentation discuss installation and usage of this runtime Keywords: kaitai,struct,construct,ksy,declarative,data structure,data format,file format,packet format,binary,parser,parsing,unpack,development Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Build Tools Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Requires-Python: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.* ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1602804847.0 kaitaistruct-0.9/kaitaistruct.egg-info/SOURCES.txt0000666000000000000000000000035000000000000020307 0ustar0000000000000000MANIFEST.in README.rst kaitaistruct.py setup.cfg setup.py kaitaistruct.egg-info/PKG-INFO kaitaistruct.egg-info/SOURCES.txt kaitaistruct.egg-info/dependency_links.txt kaitaistruct.egg-info/top_level.txt kaitaistruct.egg-info/zip-safe././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1602804847.0 kaitaistruct-0.9/kaitaistruct.egg-info/dependency_links.txt0000666000000000000000000000000100000000000022473 0ustar0000000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1602804847.0 kaitaistruct-0.9/kaitaistruct.egg-info/top_level.txt0000666000000000000000000000001500000000000021153 0ustar0000000000000000kaitaistruct ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1602802769.0 kaitaistruct-0.9/kaitaistruct.egg-info/zip-safe0000666000000000000000000000000200000000000020056 0ustar0000000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1590099076.0 kaitaistruct-0.9/kaitaistruct.py0000666000000000000000000004101200000000000015303 0ustar0000000000000000import itertools import sys import struct from io import open, BytesIO, SEEK_CUR, SEEK_END # noqa PY2 = sys.version_info[0] == 2 # Kaitai Struct runtime streaming API version, defined as per PEP-0396 # standard. Used for two purposes: # # * .py files generated by ksc from .ksy check that they import proper # KS runtime library by this version number; # * distribution utils (setup.py) use this when packaging for PyPI # __version__ = '0.9' class KaitaiStruct(object): def __init__(self, stream): self._io = stream def __enter__(self): return self def __exit__(self, *args, **kwargs): self.close() def close(self): self._io.close() @classmethod def from_file(cls, filename): f = open(filename, 'rb') try: return cls(KaitaiStream(f)) except Exception: # close file descriptor, then reraise the exception f.close() raise @classmethod def from_bytes(cls, buf): return cls(KaitaiStream(BytesIO(buf))) @classmethod def from_io(cls, io): return cls(KaitaiStream(io)) class KaitaiStream(object): def __init__(self, io): self._io = io self.align_to_byte() def __enter__(self): return self def __exit__(self, *args, **kwargs): self.close() def close(self): self._io.close() # ======================================================================== # Stream positioning # ======================================================================== def is_eof(self): if self.bits_left > 0: return False io = self._io t = io.read(1) if t == b'': return True else: io.seek(-1, SEEK_CUR) return False def seek(self, n): self._io.seek(n) def pos(self): return self._io.tell() def size(self): # Python has no internal File object API function to get # current file / StringIO size, thus we use the following # trick. io = self._io # Remember our current position cur_pos = io.tell() # Seek to the end of the File object io.seek(0, SEEK_END) # Remember position, which is equal to the full length full_size = io.tell() # Seek back to the current position io.seek(cur_pos) return full_size # ======================================================================== # Integer numbers # ======================================================================== packer_s1 = struct.Struct('b') packer_s2be = struct.Struct('>h') packer_s4be = struct.Struct('>i') packer_s8be = struct.Struct('>q') packer_s2le = struct.Struct('H') packer_u4be = struct.Struct('>I') packer_u8be = struct.Struct('>Q') packer_u2le = struct.Struct('f') packer_f8be = struct.Struct('>d') packer_f4le = struct.Struct(' 0: # 1 bit => 1 byte # 8 bits => 1 byte # 9 bits => 2 bytes bytes_needed = ((bits_needed - 1) // 8) + 1 buf = self.read_bytes(bytes_needed) for byte in buf: byte = KaitaiStream.int_from_byte(byte) self.bits <<= 8 self.bits |= byte self.bits_left += 8 # raw mask with required number of 1s, starting from lowest bit mask = (1 << n) - 1 # shift self.bits to align the highest bits with the mask & derive reading result shift_bits = self.bits_left - n res = (self.bits >> shift_bits) & mask # clear top bits that we've just read => AND with 1s self.bits_left -= n mask = (1 << self.bits_left) - 1 self.bits &= mask return res # Unused since Kaitai Struct Compiler v0.9+ - compatibility with # older versions. def read_bits_int(self, n): return self.read_bits_int_be(n) def read_bits_int_le(self, n): bits_needed = n - self.bits_left if bits_needed > 0: # 1 bit => 1 byte # 8 bits => 1 byte # 9 bits => 2 bytes bytes_needed = ((bits_needed - 1) // 8) + 1 buf = self.read_bytes(bytes_needed) for byte in buf: byte = KaitaiStream.int_from_byte(byte) self.bits |= (byte << self.bits_left) self.bits_left += 8 # raw mask with required number of 1s, starting from lowest bit mask = (1 << n) - 1 # derive reading result res = self.bits & mask # remove bottom bits that we've just read by shifting self.bits >>= n self.bits_left -= n return res # ======================================================================== # Byte arrays # ======================================================================== def read_bytes(self, n): if n < 0: raise ValueError( "requested invalid %d amount of bytes" % (n,) ) r = self._io.read(n) if len(r) < n: raise EOFError( "requested %d bytes, but got only %d bytes" % (n, len(r)) ) return r def read_bytes_full(self): return self._io.read() def read_bytes_term(self, term, include_term, consume_term, eos_error): r = b'' while True: c = self._io.read(1) if c == b'': if eos_error: raise Exception( "end of stream reached, but no terminator %d found" % (term,) ) else: return r elif ord(c) == term: if include_term: r += c if not consume_term: self._io.seek(-1, SEEK_CUR) return r else: r += c def ensure_fixed_contents(self, expected): actual = self._io.read(len(expected)) if actual != expected: raise Exception( "unexpected fixed contents: got %r, was waiting for %r" % (actual, expected) ) return actual @staticmethod def bytes_strip_right(data, pad_byte): new_len = len(data) if PY2: # data[...] must yield an integer, to compare with integer pad_byte data = bytearray(data) while new_len > 0 and data[new_len - 1] == pad_byte: new_len -= 1 return data[:new_len] @staticmethod def bytes_terminate(data, term, include_term): new_len = 0 max_len = len(data) if PY2: # data[...] must yield an integer, to compare with integer term data = bytearray(data) while new_len < max_len and data[new_len] != term: new_len += 1 if include_term and new_len < max_len: new_len += 1 return data[:new_len] # ======================================================================== # Byte array processing # ======================================================================== @staticmethod def process_xor_one(data, key): if PY2: return bytes(bytearray(v ^ key for v in bytearray(data))) else: return bytes(v ^ key for v in data) @staticmethod def process_xor_many(data, key): if PY2: return bytes(bytearray(a ^ b for a, b in zip(bytearray(data), itertools.cycle(bytearray(key))))) else: return bytes(a ^ b for a, b in zip(data, itertools.cycle(key))) @staticmethod def process_rotate_left(data, amount, group_size): if group_size != 1: raise Exception( "unable to rotate group of %d bytes yet" % (group_size,) ) mask = group_size * 8 - 1 anti_amount = -amount & mask r = bytearray(data) for i in range(len(r)): r[i] = (r[i] << amount) & 0xff | (r[i] >> anti_amount) return bytes(r) # ======================================================================== # Misc # ======================================================================== @staticmethod def int_from_byte(v): if PY2: return ord(v) return v @staticmethod def byte_array_index(data, i): return KaitaiStream.int_from_byte(data[i]) @staticmethod def byte_array_min(b): return KaitaiStream.int_from_byte(min(b)) @staticmethod def byte_array_max(b): return KaitaiStream.int_from_byte(max(b)) @staticmethod def resolve_enum(enum_obj, value): """Resolves value using enum: if the value is not found in the map, we'll just use literal value per se. Works around problem with Python enums throwing an exception when encountering unknown value. """ try: return enum_obj(value) except ValueError: return value class KaitaiStructError(BaseException): """Common ancestor for all error originating from Kaitai Struct usage. Stores KSY source path, pointing to an element supposedly guilty of an error. """ def __init__(self, msg, src_path): super(KaitaiStructError, self).__init__("%s: %s" % (src_path, msg)) self.src_path = src_path class UndecidedEndiannessError(KaitaiStructError): """Error that occurs when default endianness should be decided with switch, but nothing matches (although using endianness expression implies that there should be some positive result). """ def __init__(self, src_path): super(KaitaiStructError, self).__init__("unable to decide on endianness for a type", src_path) class ValidationFailedError(KaitaiStructError): """Common ancestor for all validation failures. Stores pointer to KaitaiStream IO object which was involved in an error. """ def __init__(self, msg, io, src_path): super(ValidationFailedError, self).__init__("at pos %d: validation failed: %s" % (io.pos(), msg), src_path) self.io = io class ValidationNotEqualError(ValidationFailedError): """Signals validation failure: we required "actual" value to be equal to "expected", but it turned out that it's not. """ def __init__(self, expected, actual, io, src_path): super(ValidationNotEqualError, self).__init__("not equal, expected %s, but got %s" % (repr(expected), repr(actual)), io, src_path) self.expected = expected self.actual = actual class ValidationLessThanError(ValidationFailedError): """Signals validation failure: we required "actual" value to be greater than or equal to "min", but it turned out that it's not. """ def __init__(self, min, actual, io, src_path): super(ValidationLessThanError, self).__init__("not in range, min %s, but got %s" % (repr(min), repr(actual)), io, src_path) self.min = min self.actual = actual class ValidationGreaterThanError(ValidationFailedError): """Signals validation failure: we required "actual" value to be less than or equal to "max", but it turned out that it's not. """ def __init__(self, max, actual, io, src_path): super(ValidationGreaterThanError, self).__init__("not in range, max %s, but got %s" % (repr(max), repr(actual)), io, src_path) self.max = max self.actual = actual class ValidationNotAnyOfError(ValidationFailedError): """Signals validation failure: we required "actual" value to be from the list, but it turned out that it's not. """ def __init__(self, actual, io, src_path): super(ValidationNotAnyOfError, self).__init__("not any of the list, got %s" % (repr(actual)), io, src_path) self.actual = actual class ValidationExprError(ValidationFailedError): """Signals validation failure: we required "actual" value to match the expression, but it turned out that it doesn't. """ def __init__(self, actual, io, src_path): super(ValidationExprError, self).__init__("not matching the expression, got %s" % (repr(actual)), io, src_path) self.actual = actual ././@PaxHeader0000000000000000000000000000003300000000000011451 xustar000000000000000027 mtime=1602804847.576354 kaitaistruct-0.9/setup.cfg0000666000000000000000000000253300000000000014050 0ustar0000000000000000[metadata] name = kaitaistruct version = attr: kaitaistruct.__version__ author = Kaitai Project author_email = greycat@kaitai.io url = http://kaitai.io description = Kaitai Struct declarative parser generator for binary data: runtime library for Python long_description = file: README.rst license = MIT keywords = kaitai, struct, construct, ksy, declarative, data structure, data format, file format, packet format, binary, parser, parsing, unpack, development classifiers = Development Status :: 4 - Beta Intended Audience :: Developers Topic :: Software Development :: Build Tools License :: OSI Approved :: MIT License Programming Language :: Python :: 2 Programming Language :: Python :: 2.7 Programming Language :: Python :: 3 Programming Language :: Python :: 3.4 Programming Language :: Python :: 3.5 Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 Programming Language :: Python :: Implementation :: CPython Programming Language :: Python :: Implementation :: PyPy [options] zip_safe = True include_package_data = True py_modules = kaitaistruct python_requires = >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.* [bdist_wheel] universal = 1 [build-system] requires = ["setuptools", "wheel"] [pycodestyle] max-line-length = 140 statistics = True [egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1589667259.0 kaitaistruct-0.9/setup.py0000666000000000000000000000005100000000000013732 0ustar0000000000000000from setuptools import setup setup()