pax_global_header00006660000000000000000000000064134507131510014512gustar00rootroot0000000000000052 comment=551bb71eb19e174b6c12f97acd90b37017c71676 cyarray-1.1/000077500000000000000000000000001345071315100130255ustar00rootroot00000000000000cyarray-1.1/.gitignore000066400000000000000000000001101345071315100150050ustar00rootroot00000000000000*.pyc *.o *.c *.cpp *~ *.so build/ dist/ cyarray.egg-info/ .pytest_cachecyarray-1.1/.travis.yml000066400000000000000000000003301345071315100151320ustar00rootroot00000000000000language: python python: - "2.7" - "3.6" # command to install dependencies install: - pip install -r requirements.txt - python setup.py install # command to run tests script: - pytest -v --pyargs cyarray cyarray-1.1/LICENSE.txt000066400000000000000000000031111345071315100146440ustar00rootroot00000000000000Unless otherwise specified by LICENSE.txt files in individual directories, all code is Copyright (c) 2009-2018, the PySPH developers All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. cyarray-1.1/MANIFEST.in000066400000000000000000000002321345071315100145600ustar00rootroot00000000000000include MANIFEST.in Makefile *.py *.rst *.yml *.toml recursive-exclude cyarray *.cpp recursive-include cyarray *.pxd *.h *.mako recursive-include docs *.*cyarray-1.1/README.rst000066400000000000000000000041241345071315100145150ustar00rootroot00000000000000cyarray: a typed, re-sizable Cython array ------------------------------------------ |Travis Status| |Appveyor Status| |Documentation Status| .. |Travis Status| image:: https://travis-ci.org/pypr/cyarray.svg?branch=master :target: https://travis-ci.org/pypr/cyarray .. |Appveyor Status| image:: https://ci.appveyor.com/api/projects/status/rbl358nj2876dap1?svg=true :target: https://ci.appveyor.com/project/prabhuramachandran/cyarray .. |Documentation Status| image:: https://readthedocs.org/projects/cyarray/badge/?version=latest :target: https://cyarray.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status The cyarray package provides a fast, typed, re-sizable, Cython array. It currently provides the following arrays: ``IntArray, UIntArray, LongArray, FloatArray, DoubleArray``. All arrays provide for the following operations: - access by indexing. - access through get/set function. - resizing the array. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. If you are writing Cython code this is a convenient array to use as it exposes the raw underlying pointer to the data. For example if you use a ``FloatArray`` and access its ``data`` attribute it will be a ``float*``. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. Installation ------------ cyarray can be installed using pip_:: $ pip install cyarray The package requires ``Cython``, ``numpy``, and ``mako`` to be installed and also requires a suitably configured C/C++ compiler. .. _pip: http://www.pip-installer.org Usage ----- In Python one may import and use the package as:: from cyarray.api import IntArray a = IntArray(10) Here ``a`` is an array of 10 integers. cyarray-1.1/appveyor.yml000066400000000000000000000013711345071315100154170ustar00rootroot00000000000000environment: matrix: - PYTHON: "C:\\Python27" - PYTHON: "C:\\Python36-x64" - PYTHON: "C:\\Python37-x64" install: # We need wheel installed to build wheels - "%PYTHON%\\python.exe -m pip install -r requirements.txt" - "%PYTHON%\\python.exe setup.py develop" build: off test_script: # Put your test command here. # If you don't need to build C extensions on 64-bit Python 3.3 or 3.4, # you can remove "build.cmd" from the front of the command, as it's # only needed to support those cases. # Note that you must use the environment variable %PYTHON% to refer to # the interpreter you're using - Appveyor does not do anything special # to put the Python version you want to use on PATH. - "%PYTHON%\\python.exe -m pytest -v" cyarray-1.1/cyarray/000077500000000000000000000000001345071315100144775ustar00rootroot00000000000000cyarray-1.1/cyarray/__init__.py000066400000000000000000000000241345071315100166040ustar00rootroot00000000000000__version__ = '1.1' cyarray-1.1/cyarray/api.py000066400000000000000000000001141345071315100156160ustar00rootroot00000000000000from .carray import IntArray, UIntArray, LongArray, FloatArray, DoubleArray cyarray-1.1/cyarray/carray.pxd000066400000000000000000000166221345071315100165040ustar00rootroot00000000000000# This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate """ Implementation of resizeable arrays of different types in Cython. Declaration File. """ # numpy import cimport numpy as np cdef long aligned(long n, int item_size) nogil cdef void* aligned_malloc(size_t bytes) nogil cdef void* aligned_realloc(void* existing, size_t bytes, size_t old_size) nogil cdef void aligned_free(void* p) nogil # forward declaration cdef class BaseArray cdef class LongArray(BaseArray) cdef class BaseArrayIter: cdef BaseArray arr cdef long i cdef class BaseArray: """Base class for managed C-arrays.""" cdef public long length, alloc cdef np.ndarray _npy_array cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_reserve(self, long size) nogil cdef void c_reset(self) nogil cdef void c_resize(self, long size) nogil cdef void c_squeeze(self) nogil cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef align_array(self, LongArray new_indices, int stride=*) cpdef str get_c_type(self) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=*, int start=*) cpdef copy_subset(self, BaseArray source, long start_index=*, long end_index=*, int stride=*) cpdef update_min_max(self) # ########################################################################### # `IntArray` class. # ########################################################################### cdef class IntArray(BaseArray): """This class defines a managed array of ints. """ cdef int *data cdef int *_old_data cdef public int minimum, maximum cdef IntArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, int value) nogil cdef void c_set_view(self, int *array, long length) nogil cdef int* get_data_ptr(self) cpdef int get(self, long idx) cpdef set(self, long idx, int value) cpdef append(self, int value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, IntArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, int value) # ########################################################################### # `UIntArray` class. # ########################################################################### cdef class UIntArray(BaseArray): """This class defines a managed array of unsigned ints. """ cdef unsigned int *data cdef unsigned int *_old_data cdef public unsigned int minimum, maximum cdef UIntArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, unsigned int value) nogil cdef void c_set_view(self, unsigned int *array, long length) nogil cdef unsigned int* get_data_ptr(self) cpdef unsigned int get(self, long idx) cpdef set(self, long idx, unsigned int value) cpdef append(self, unsigned int value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, UIntArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, unsigned int value) # ########################################################################### # `LongArray` class. # ########################################################################### cdef class LongArray(BaseArray): """This class defines a managed array of longs. """ cdef long *data cdef long *_old_data cdef public long minimum, maximum cdef LongArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, long value) nogil cdef void c_set_view(self, long *array, long length) nogil cdef long* get_data_ptr(self) cpdef long get(self, long idx) cpdef set(self, long idx, long value) cpdef append(self, long value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, LongArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, long value) # ########################################################################### # `FloatArray` class. # ########################################################################### cdef class FloatArray(BaseArray): """This class defines a managed array of floats. """ cdef float *data cdef float *_old_data cdef public float minimum, maximum cdef FloatArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, float value) nogil cdef void c_set_view(self, float *array, long length) nogil cdef float* get_data_ptr(self) cpdef float get(self, long idx) cpdef set(self, long idx, float value) cpdef append(self, float value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, FloatArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, float value) # ########################################################################### # `DoubleArray` class. # ########################################################################### cdef class DoubleArray(BaseArray): """This class defines a managed array of doubles. """ cdef double *data cdef double *_old_data cdef public double minimum, maximum cdef DoubleArray _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, double value) nogil cdef void c_set_view(self, double *array, long length) nogil cdef double* get_data_ptr(self) cpdef double get(self, long idx) cpdef set(self, long idx, double value) cpdef append(self, double value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, DoubleArray, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, double value) cyarray-1.1/cyarray/carray.pxd.mako000066400000000000000000000062301345071315100174240ustar00rootroot00000000000000<% type_info = [ ('int', 'IntArray', 'NPY_INT'), ('unsigned int', 'UIntArray', 'NPY_UINT'), ('long', 'LongArray', 'NPY_LONG'), ('float', 'FloatArray', 'NPY_FLOAT'), ('double', 'DoubleArray', 'NPY_DOUBLE'), ] %># This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate """ Implementation of resizeable arrays of different types in Cython. Declaration File. """ # numpy import cimport numpy as np cdef long aligned(long n, int item_size) nogil cdef void* aligned_malloc(size_t bytes) nogil cdef void* aligned_realloc(void* existing, size_t bytes, size_t old_size) nogil cdef void aligned_free(void* p) nogil # forward declaration cdef class BaseArray cdef class LongArray(BaseArray) cdef class BaseArrayIter: cdef BaseArray arr cdef long i cdef class BaseArray: """Base class for managed C-arrays.""" cdef public long length, alloc cdef np.ndarray _npy_array cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_reserve(self, long size) nogil cdef void c_reset(self) nogil cdef void c_resize(self, long size) nogil cdef void c_squeeze(self) nogil cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef align_array(self, LongArray new_indices, int stride=*) cpdef str get_c_type(self) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=*, int start=*) cpdef copy_subset(self, BaseArray source, long start_index=*, long end_index=*, int stride=*) cpdef update_min_max(self) % for ARRAY_TYPE, CLASSNAME, NUMPY_TYPENAME in type_info: # ########################################################################### # `${CLASSNAME}` class. # ########################################################################### cdef class ${CLASSNAME}(BaseArray): """This class defines a managed array of ${ARRAY_TYPE}s. """ cdef ${ARRAY_TYPE} *data cdef ${ARRAY_TYPE} *_old_data cdef public ${ARRAY_TYPE} minimum, maximum cdef ${CLASSNAME} _parent cdef _setup_npy_array(self) cdef void c_align_array(self, LongArray new_indices, int stride=*) nogil cdef void c_append(self, ${ARRAY_TYPE} value) nogil cdef void c_set_view(self, ${ARRAY_TYPE} *array, long length) nogil cdef ${ARRAY_TYPE}* get_data_ptr(self) cpdef ${ARRAY_TYPE} get(self, long idx) cpdef set(self, long idx, ${ARRAY_TYPE} value) cpdef append(self, ${ARRAY_TYPE} value) cpdef reserve(self, long size) cpdef resize(self, long size) cpdef np.ndarray get_npy_array(self) cpdef set_data(self, np.ndarray) cpdef set_view(self, ${CLASSNAME}, long start, long end) cpdef squeeze(self) cpdef remove(self, np.ndarray index_list, bint input_sorted=*, int stride=*) cpdef extend(self, np.ndarray in_array) cpdef reset(self) cpdef long index(self, ${ARRAY_TYPE} value) % endfor cyarray-1.1/cyarray/carray.pyx000066400000000000000000002453641345071315100165400ustar00rootroot00000000000000# This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate # distutils: language=c++ # cython: embedsignature=True """ Implementation of resizeable arrays of different types in Cython. All arrays provide for the following operations: - access by indexing. - access through get/set function. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. """ # For malloc etc. from libc.stdlib cimport * IF UNAME_SYSNAME == "Windows": cdef extern from "msstdint.h" nogil: ctypedef unsigned int uintptr_t ELSE: from libc.stdint cimport uintptr_t cimport numpy as np import numpy as np # logging imports import logging logger = logging.getLogger() # 'importing' some Numpy C-api functions. cdef extern from "numpy/arrayobject.h": cdef void _import_array() ctypedef struct PyArrayObject: char *data np.npy_intp *dimensions cdef enum NPY_TYPES: NPY_INT, NPY_UINT, NPY_LONG, NPY_FLOAT, NPY_DOUBLE np.ndarray PyArray_SimpleNewFromData(int, np.npy_intp*, int, void*) # memcpy cdef extern from "stdlib.h": void *memcpy(void *dst, void *src, long n) nogil # numpy module initialization call _import_array() cdef inline long aligned(long n, int item_size) nogil: """Align `n` items each having size (in bytes) `item_size` to 64 bytes and return the appropriate number of items that would be aligned to 64 bytes. """ if n*item_size%64 == 0: return n else: if 64%item_size == 0: return (n*item_size/64 + 1)*64/item_size else: return (n*item_size/64 + 1)*64 cpdef long py_aligned(long n, int item_size): """Align `n` items each having size (in bytes) `item_size` to 64 bits and return the appropriate number of items that would be aligned to 64 bytes. """ return aligned(n, item_size) cdef void* _aligned_malloc(size_t bytes) nogil: """Allocates block of memory starting on a cache line. Algorithm from: http://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410 """ cdef size_t cache_size = 64 cdef char* base = malloc(cache_size + bytes) # Round pointer up to next line cdef char* result = ((base+cache_size)&-(cache_size)) # Record where block actually starts. (result)[-1] = base return result cdef void* _aligned_realloc(void *existing, size_t bytes, size_t old_size) nogil: """Allocates block of memory starting on a cache line. """ cdef void* result = _aligned_malloc(bytes) cdef size_t copy_size = min(bytes, old_size) # Copy everything from the old to the new and free the old. memcpy(result, existing, copy_size) aligned_free(existing) return result cdef void* _deref_base(void* ptr) nogil: cdef size_t cache_size = 64 # Recover where block actually starts cdef char* base = (ptr)[-1] if ((base+cache_size)&-(cache_size)) != ptr: with gil: raise MemoryError("Passed pointer is not aligned.") return base cdef void* aligned_malloc(size_t bytes) nogil: return _aligned_malloc(bytes) cdef void* aligned_realloc(void* p, size_t bytes, size_t old_size) nogil: return _aligned_realloc(p, bytes, old_size) cdef void aligned_free(void* p) nogil: """Free block allocated by alligned_malloc. """ free(_deref_base(p)) cdef class BaseArray: """Base class for managed C-arrays. """ cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ pass cdef void c_reserve(self, long size) nogil: pass cdef void c_reset(self) nogil: cdef PyArrayObject* arr = self._npy_array self.length = 0 arr.dimensions[0] = self.length cdef void c_resize(self, long size) nogil: pass cdef void c_squeeze(self) nogil: pass cpdef str get_c_type(self): """Return the c data type of this array. """ raise NotImplementedError, 'BaseArray::get_c_type' cpdef reserve(self, long size): """Resizes the internal data to required size. """ raise NotImplementedError, 'BaseArray::reserve' cpdef resize(self, long size): """Resizes the array to the new size. """ raise NotImplementedError, 'BaseArray::resize' cpdef np.ndarray get_npy_array(self): """Returns a numpy array of the data: do not keep its reference. """ return self._npy_array cpdef set_data(self, np.ndarray nparr): """Set data from the given numpy array. If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained. """ cdef PyArrayObject* sarr = nparr cdef PyArrayObject* darr = self._npy_array if sarr.data == darr.data: return elif sarr.dimensions[0] <= darr.dimensions[0]: self._npy_array[:sarr.dimensions[0]] = nparr else: raise ValueError, 'array size mismatch' cpdef squeeze(self): """Release any unused memory. """ raise NotImplementedError, 'BaseArray::squeeze' cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. """ raise NotImplementedError, 'BaseArray::remove' cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. """ raise NotImplementedError, 'BaseArray::extend' cpdef align_array(self, LongArray new_indices, int stride=1): """Rearrange the array contents according to the new indices. """ if new_indices.length != self.length//stride: raise ValueError('Unequal array lengths') self.c_align_array(new_indices, stride) cpdef reset(self): """Reset the length of the array to 0. """ raise NotImplementedError, 'BaseArray::reset' cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copy values of indexed particles from self to dest. """ raise NotImplementedError, 'BaseArray::copy_values' cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy subset of values from source to self. """ raise NotImplementedError, 'BaseArray::copy_subset' cpdef update_min_max(self): """Update the min and max values of the array. """ raise NotImplementedError, 'BaseArray::update_min_max' def __len__(self): return self.length def __iter__(self): """ Support the iteration protocol""" return BaseArrayIter(self) cdef class BaseArrayIter: """ Iteration object to support iteration over BaseArray. """ def __init__(self, BaseArray arr): self.arr = arr self.i = -1 def __next__(self): self.i = self.i+1 if self.i < self.arr.length: return self.arr[self.i] else: raise StopIteration def __iter__(self): return self # ########################################################################### # `IntArray` class. # ########################################################################### cdef class IntArray(BaseArray): """Represents an array of `ints` Mallocs a memory buffer of size (n*sizeof(int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = IntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = IntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef int *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(int)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, int value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, int value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, int value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (IntArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_INT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef int *temp n_bytes = sizeof(int)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, int value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(int), self.alloc*sizeof(int) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, int *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(int), self.alloc*sizeof(int) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'int' cdef int* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef int get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, int value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, int value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(int)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(int)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, IntArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : IntArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef IntArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef IntArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef int min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `UIntArray` class. # ########################################################################### cdef class UIntArray(BaseArray): """Represents an array of `unsigned ints` Mallocs a memory buffer of size (n*sizeof(unsigned int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = UIntArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = UIntArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef unsigned int *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(unsigned int)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, unsigned int value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, unsigned int value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, unsigned int value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (UIntArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_UINT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef unsigned int *temp n_bytes = sizeof(unsigned int)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, unsigned int value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(unsigned int), self.alloc*sizeof(unsigned int) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, unsigned int *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(unsigned int), self.alloc*sizeof(unsigned int) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'unsigned int' cdef unsigned int* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef unsigned int get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, unsigned int value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, unsigned int value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(unsigned int)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(unsigned int)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, UIntArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : UIntArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef UIntArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef UIntArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef unsigned int min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `LongArray` class. # ########################################################################### cdef class LongArray(BaseArray): """Represents an array of `longs` Mallocs a memory buffer of size (n*sizeof(long)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = LongArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = LongArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef long *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(long)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, long value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, long value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, long value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (LongArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_LONG, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef long *temp n_bytes = sizeof(long)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, long value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(long), self.alloc*sizeof(long) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, long *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(long), self.alloc*sizeof(long) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'long' cdef long* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef long get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, long value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, long value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(long)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(long)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, LongArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : LongArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef LongArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef LongArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef long min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `FloatArray` class. # ########################################################################### cdef class FloatArray(BaseArray): """Represents an array of `floats` Mallocs a memory buffer of size (n*sizeof(float)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = FloatArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = FloatArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef float *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(float)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, float value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, float value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, float value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (FloatArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_FLOAT, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef float *temp n_bytes = sizeof(float)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, float value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(float), self.alloc*sizeof(float) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, float *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(float), self.alloc*sizeof(float) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'float' cdef float* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef float get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, float value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, float value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(float)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(float)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, FloatArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : FloatArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef FloatArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef FloatArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef float min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val # ########################################################################### # `DoubleArray` class. # ########################################################################### cdef class DoubleArray(BaseArray): """Represents an array of `doubles` Mallocs a memory buffer of size (n*sizeof(double)) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = DoubleArray() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = DoubleArray(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef double *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = aligned_malloc(n*sizeof(double)) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, double value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, double value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, double value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (DoubleArray, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, NPY_DOUBLE, self.data ) cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef double *temp n_bytes = sizeof(double)*length temp = aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, double value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = aligned_realloc( self.data, size*sizeof(double), self.alloc*sizeof(double) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, double *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = aligned_realloc( self.data, size*sizeof(double), self.alloc*sizeof(double) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = data self.alloc = size arr.data = self.data cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return 'double' cdef double* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef double get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, double value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, double value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(double)`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(double)`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, DoubleArray parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : DoubleArray The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef DoubleArray dest_array = dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef DoubleArray src = source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef double min_val, max_val if self.length == 0: self.minimum = 0 self.maximum = 0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val cyarray-1.1/cyarray/carray.pyx.mako000066400000000000000000000573301345071315100174600ustar00rootroot00000000000000<% type_info = [ ('int', 'IntArray', 'NPY_INT'), ('unsigned int', 'UIntArray', 'NPY_UINT'), ('long', 'LongArray', 'NPY_LONG'), ('float', 'FloatArray', 'NPY_FLOAT'), ('double', 'DoubleArray', 'NPY_DOUBLE'), ] %># This file (carray.pxd) has been generated automatically. # DO NOT modify this file # To make changes modify the source templates (carray.pxd.mako) and regenerate # distutils: language=c++ # cython: embedsignature=True """ Implementation of resizeable arrays of different types in Cython. All arrays provide for the following operations: - access by indexing. - access through get/set function. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. """ # For malloc etc. from libc.stdlib cimport * IF UNAME_SYSNAME == "Windows": cdef extern from "msstdint.h" nogil: ctypedef unsigned int uintptr_t ELSE: from libc.stdint cimport uintptr_t cimport numpy as np import numpy as np # logging imports import logging logger = logging.getLogger() # 'importing' some Numpy C-api functions. cdef extern from "numpy/arrayobject.h": cdef void _import_array() ctypedef struct PyArrayObject: char *data np.npy_intp *dimensions cdef enum NPY_TYPES: NPY_INT, NPY_UINT, NPY_LONG, NPY_FLOAT, NPY_DOUBLE np.ndarray PyArray_SimpleNewFromData(int, np.npy_intp*, int, void*) # memcpy cdef extern from "stdlib.h": void *memcpy(void *dst, void *src, long n) nogil # numpy module initialization call _import_array() cdef inline long aligned(long n, int item_size) nogil: """Align `n` items each having size (in bytes) `item_size` to 64 bytes and return the appropriate number of items that would be aligned to 64 bytes. """ if n*item_size%64 == 0: return n else: if 64%item_size == 0: return (n*item_size/64 + 1)*64/item_size else: return (n*item_size/64 + 1)*64 cpdef long py_aligned(long n, int item_size): """Align `n` items each having size (in bytes) `item_size` to 64 bits and return the appropriate number of items that would be aligned to 64 bytes. """ return aligned(n, item_size) cdef void* _aligned_malloc(size_t bytes) nogil: """Allocates block of memory starting on a cache line. Algorithm from: http://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410 """ cdef size_t cache_size = 64 cdef char* base = malloc(cache_size + bytes) # Round pointer up to next line cdef char* result = ((base+cache_size)&-(cache_size)) # Record where block actually starts. (result)[-1] = base return result cdef void* _aligned_realloc(void *existing, size_t bytes, size_t old_size) nogil: """Allocates block of memory starting on a cache line. """ cdef void* result = _aligned_malloc(bytes) cdef size_t copy_size = min(bytes, old_size) # Copy everything from the old to the new and free the old. memcpy(result, existing, copy_size) aligned_free(existing) return result cdef void* _deref_base(void* ptr) nogil: cdef size_t cache_size = 64 # Recover where block actually starts cdef char* base = (ptr)[-1] if ((base+cache_size)&-(cache_size)) != ptr: with gil: raise MemoryError("Passed pointer is not aligned.") return base cdef void* aligned_malloc(size_t bytes) nogil: return _aligned_malloc(bytes) cdef void* aligned_realloc(void* p, size_t bytes, size_t old_size) nogil: return _aligned_realloc(p, bytes, old_size) cdef void aligned_free(void* p) nogil: """Free block allocated by alligned_malloc. """ free(_deref_base(p)) cdef class BaseArray: """Base class for managed C-arrays. """ #### Cython interface ################################################# cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ pass cdef void c_reserve(self, long size) nogil: pass cdef void c_reset(self) nogil: cdef PyArrayObject* arr = self._npy_array self.length = 0 arr.dimensions[0] = self.length cdef void c_resize(self, long size) nogil: pass cdef void c_squeeze(self) nogil: pass #### Python interface ################################################# cpdef str get_c_type(self): """Return the c data type of this array. """ raise NotImplementedError, 'BaseArray::get_c_type' cpdef reserve(self, long size): """Resizes the internal data to required size. """ raise NotImplementedError, 'BaseArray::reserve' cpdef resize(self, long size): """Resizes the array to the new size. """ raise NotImplementedError, 'BaseArray::resize' cpdef np.ndarray get_npy_array(self): """Returns a numpy array of the data: do not keep its reference. """ return self._npy_array cpdef set_data(self, np.ndarray nparr): """Set data from the given numpy array. If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained. """ cdef PyArrayObject* sarr = nparr cdef PyArrayObject* darr = self._npy_array if sarr.data == darr.data: return elif sarr.dimensions[0] <= darr.dimensions[0]: self._npy_array[:sarr.dimensions[0]] = nparr else: raise ValueError, 'array size mismatch' cpdef squeeze(self): """Release any unused memory. """ raise NotImplementedError, 'BaseArray::squeeze' cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. """ raise NotImplementedError, 'BaseArray::remove' cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. """ raise NotImplementedError, 'BaseArray::extend' cpdef align_array(self, LongArray new_indices, int stride=1): """Rearrange the array contents according to the new indices. """ if new_indices.length != self.length//stride: raise ValueError('Unequal array lengths') self.c_align_array(new_indices, stride) cpdef reset(self): """Reset the length of the array to 0. """ raise NotImplementedError, 'BaseArray::reset' cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copy values of indexed particles from self to dest. """ raise NotImplementedError, 'BaseArray::copy_values' cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy subset of values from source to self. """ raise NotImplementedError, 'BaseArray::copy_subset' cpdef update_min_max(self): """Update the min and max values of the array. """ raise NotImplementedError, 'BaseArray::update_min_max' def __len__(self): return self.length def __iter__(self): """ Support the iteration protocol""" return BaseArrayIter(self) cdef class BaseArrayIter: """ Iteration object to support iteration over BaseArray. """ def __init__(self, BaseArray arr): self.arr = arr self.i = -1 def __next__(self): self.i = self.i+1 if self.i < self.arr.length: return self.arr[self.i] else: raise StopIteration def __iter__(self): return self % for ARRAY_TYPE, CLASSNAME, NUMPY_TYPENAME in type_info: # ########################################################################### # `${CLASSNAME}` class. # ########################################################################### cdef class ${CLASSNAME}(BaseArray): """Represents an array of `${ARRAY_TYPE}s` Mallocs a memory buffer of size (n*sizeof(${ARRAY_TYPE})) and sets up the numpy array. The memory is aligned to 64 byte boundaries. Parameters ---------- n : long Length of the array. Attributes ---------- data: pointer Pointer to an integer array. length: long Size of the array itself. alloc: long Size of the data buffer allocated. Examples -------- >>> x = ${CLASSNAME}() >>> x.resize(5) >>> x.set_data(np.arange(5)) >>> x[0] 0 >>> x = ${CLASSNAME}(5) >>> xnp = x.get_npy_array() >>> xnp[:] = np.arange(5) >>> x[0], x[4] (0.0, 4.0) """ #cdef public long length, alloc #cdef ${ARRAY_TYPE} *data #cdef np.ndarray _npy_array def __cinit__(self, long n=0): """Constructor for the class. """ self.length = n self._parent = None self._old_data = NULL if n == 0: n = 16 self.alloc = n self.data = <${ARRAY_TYPE}*>aligned_malloc(n*sizeof(${ARRAY_TYPE})) self._setup_npy_array() def __dealloc__(self): """Frees the array. """ if self._old_data == NULL: aligned_free(self.data) else: aligned_free(self._old_data) def __getitem__(self, long idx): """Get item at position idx. """ return self.data[idx] def __setitem__(self, long idx, ${ARRAY_TYPE} value): """Set location idx to value. """ self.data[idx] = value cpdef long index(self, ${ARRAY_TYPE} value): """Returns the index at which value is in self, else -1. """ cdef long i for i in range(self.length): if self.data[i] == value: return i return -1 def __contains__(self, ${ARRAY_TYPE} value): """Returns True if value is in self. """ return (self.index(value) >= 0) def __reduce__(self): """Implemented to facilitate pickling. """ d = {} d['data'] = self.get_npy_array() return (${CLASSNAME}, (), d) def __setstate__(self, d): """Load the carray from the dictionary d. """ cdef np.ndarray arr = d['data'] self.resize(arr.size) self.set_data(arr) cdef _setup_npy_array(self): """Create the numpy array. """ cdef int nd = 1 cdef np.npy_intp dims = self.length self._npy_array = PyArray_SimpleNewFromData( nd, &dims, ${NUMPY_TYPENAME}, self.data ) ##### Cython protocol ###################################### cdef void c_align_array(self, LongArray new_indices, int stride=1) nogil: """Rearrange the array contents according to the new indices. """ cdef long i, j, new_index cdef long length = self.length cdef long n_bytes cdef ${ARRAY_TYPE} *temp n_bytes = sizeof(${ARRAY_TYPE})*length temp = <${ARRAY_TYPE}*>aligned_malloc(n_bytes) memcpy(temp, self.data, n_bytes) # copy the data from the resized portion to the actual positions. if stride == 1: for i in range(length): new_index = new_indices.data[i] if i != new_index: self.data[i] = temp[new_index] else: for i in range(length//stride): new_index = new_indices.data[i] if i != new_index: for j in range(stride): self.data[i*stride + j] = temp[new_index*stride + j] aligned_free(temp) cdef void c_append(self, ${ARRAY_TYPE} value) nogil: cdef long l = self.length cdef PyArrayObject* arr = self._npy_array if l >= self.alloc: self.c_reserve(l*2) self.data[l] = value self.length += 1 # update the numpy arrays length arr.dimensions[0] = self.length cdef void c_reserve(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL if size > self.alloc: data = <${ARRAY_TYPE}*>aligned_realloc( self.data, size*sizeof(${ARRAY_TYPE}), self.alloc*sizeof(${ARRAY_TYPE}) ) if data == NULL: aligned_free(self.data) with gil: raise MemoryError self.data = <${ARRAY_TYPE}*>data self.alloc = size arr.data = self.data cdef void c_reset(self) nogil: BaseArray.c_reset(self) if self._old_data != NULL: self.data = self._old_data self._old_data = NULL self._npy_array.data = self.data cdef void c_resize(self, long size) nogil: cdef PyArrayObject* arr = self._npy_array # reserve memory self.c_reserve(size) # update the lengths self.length = size arr.dimensions[0] = self.length cdef void c_set_view(self, ${ARRAY_TYPE} *array, long length) nogil: """Create a view of a given raw data pointer with given length. """ if self._old_data == NULL: self._old_data = self.data self.data = array self.length = length cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cdef void c_squeeze(self) nogil: cdef PyArrayObject* arr = self._npy_array cdef void* data = NULL cdef size_t size = max(self.length, 16) data = <${ARRAY_TYPE}*>aligned_realloc( self.data, size*sizeof(${ARRAY_TYPE}), self.alloc*sizeof(${ARRAY_TYPE}) ) if data == NULL: # free original data aligned_free(self.data) with gil: raise MemoryError self.data = <${ARRAY_TYPE}*>data self.alloc = size arr.data = self.data ##### Python protocol ###################################### cpdef str get_c_type(self): """Return the c data type for this array as a string. """ return '${ARRAY_TYPE}' cdef ${ARRAY_TYPE}* get_data_ptr(self): """Return the internal data pointer. """ return self.data cpdef ${ARRAY_TYPE} get(self, long idx): """Gets value stored at position `idx`. """ return self.data[idx] cpdef set(self, long idx, ${ARRAY_TYPE} value): """Sets location `idx` to `value`. """ self.data[idx] = value cpdef append(self, ${ARRAY_TYPE} value): """Appends `value` to the end of the array. """ self.c_append(value) cpdef reserve(self, long size): """Resizes the internal data to ``size*sizeof(${ARRAY_TYPE})`` bytes. """ self.c_reserve(size) cpdef reset(self): """Reset the length of the array to 0. """ self.c_reset() if self._old_data != NULL: self._parent = None cpdef resize(self, long size): """Resizes internal data to ``size*sizeof(${ARRAY_TYPE})`` bytes and sets the length to the new size. """ if self._old_data != NULL: raise RuntimeError('Cannot reize array which is a view.') self.c_resize(size) cpdef set_view(self, ${CLASSNAME} parent, long start, long end): """Create a view of a given a `parent` array from start to end. Note that this excludes the end index. Parameters ---------- parent : ${CLASSNAME} The parent array of which this is a view. start : long The starting index to start the view from. end : long The ending index to end the view at, excludes the end itself. """ if self._parent is None: self._old_data = self.data self._parent = parent self.data = parent.data + start self.length = end - start cdef PyArrayObject* arr = self._npy_array arr.data = self.data arr.dimensions[0] = self.length cpdef squeeze(self): """Release any unused memory. """ if self._old_data != NULL: raise RuntimeError('Cannot squeeze array which is a view.') self.c_squeeze() cpdef remove(self, np.ndarray index_list, bint input_sorted=0, int stride=1): """Remove the particles with indices in index_list. Parameters ---------- index_list : ndarray a list of indices which should be removed. input_sorted : bool indicates if the input is sorted in ascending order. if not, the array will be sorted internally. stride : int indicates the stride size for the indices. Notes ----- If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array. If stride is 3, then the indices are multiplied by 3 and chunks of 3 elements are removed. """ if self._old_data != NULL: raise RuntimeError('Cannot remove elements from view array.') cdef long i cdef int j cdef long inlength = index_list.size cdef np.ndarray sorted_indices cdef long id cdef PyArrayObject* arr = self._npy_array if inlength > self.length: return if input_sorted != 1: sorted_indices = np.sort(index_list) else: sorted_indices = index_list if stride == 1: for i in range(inlength): id = sorted_indices[inlength-(i+1)] if id < self.length: self.data[id] = self.data[self.length-1] self.length = self.length - 1 else: for i in range(inlength): id = sorted_indices[inlength-(i+1)]*stride if id < self.length: for j in range(stride): self.data[id + j] = self.data[self.length - stride + j] self.length = self.length - stride arr.dimensions[0] = self.length cpdef extend(self, np.ndarray in_array): """Extend the array with data from in_array. Parameters ---------- in_array : ndarray a numpy array with data to be added to the current array. Notes ----- - accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file. """ if self._old_data != NULL: raise RuntimeError('Cannot extend array which is a view.') cdef long len = in_array.size cdef long i for i in range(len): self.append(in_array[i]) cpdef copy_values(self, LongArray indices, BaseArray dest, int stride=1, int start=0): """Copies values of indices in indices from self to `dest`. No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices. """ cdef ${CLASSNAME} dest_array = <${CLASSNAME}>dest cdef long i, num_values cdef int j num_values = indices.length if stride == 1: for i in range(num_values): dest_array.data[start + i] = self.data[indices.data[i]] else: for i in range(num_values): for j in range(stride): dest_array.data[start + i*stride + j] = self.data[indices.data[i]*stride + j] cpdef copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1, int stride=1): """Copy a subset of values from src to self. Parameters ---------- start_index : long the first index in self that corresponds to the 0th index in source end_index : long the first index in self from start_index that is not copied stride : int the stride along indices, basically copy over chunks of the given stride. """ cdef long si, ei, s_length, d_length, i, j cdef int k cdef ${CLASSNAME} src = <${CLASSNAME}>source s_length = src.length d_length = self.length if end_index < 0: if start_index < 0: if s_length != d_length: msg = 'Source length should be same as dest length' logger.error(msg) raise ValueError, msg si = 0 ei = self.length else: # meaning we copy from the specified start index to the end of # self. make sure the sizes are consistent. si = start_index ei = d_length if start_index > (d_length-1): msg = 'start_index beyond array length' logger.error(msg) raise ValueError, msg if (ei - si) > s_length: msg = 'Not enough values in source' logger.error(msg) raise ValueError, msg else: if start_index < 0: msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg else: if (start_index > (d_length-1) or end_index > d_length or start_index > end_index): msg = 'start_index : %d, end_index : %d'%(start_index, end_index) logger.error(msg) raise ValueError, msg si = start_index ei = end_index # we have valid start and end indices now. can start copying now. if stride == 1: j = 0 for i in range(si, ei): self.data[i] = src.data[j] j += 1 else: j = 0 for i in range(si, ei): for k in range(stride): self.data[i*stride + k] = src.data[j*stride + k] j += 1 cpdef update_min_max(self): """Updates the min and max values of the array. """ cdef long i = 0 cdef ${ARRAY_TYPE} min_val, max_val if self.length == 0: self.minimum = <${ARRAY_TYPE}>0 self.maximum = <${ARRAY_TYPE}>0 return min_val = self.data[0] max_val = self.data[0] for i in range(self.length): if min_val > self.data[i]: min_val = self.data[i] if max_val < self.data[i]: max_val = self.data[i] self.minimum = min_val self.maximum = max_val % endfor cyarray-1.1/cyarray/generator.py000077500000000000000000000052611345071315100170460ustar00rootroot00000000000000#!/usr/bin/env python ''' Module to collect and generate source files from mako template files. When used to locate source files as a main program: The template files must have an extension '.mako'. The generated files have the name same as the mako file but with the '.mako' removed. Example: `carray.pyx.mako` is generated into `carray.pyx` ''' from __future__ import print_function import glob import os from os.path import abspath, dirname, exists, join import sys def is_modified_later(filename1, filename2): '''Return `True` if the file1 is modified later than file2''' return os.stat(filename1).st_mtime > os.stat(filename2).st_mtime def _inject_paths_in_sys_path(outfile): # Inject the directory of the output file into the path, # so that local imports will work. sys.path.insert(0, dirname(outfile)) # inject the path to pysph if pysph cannot be imported. try: import pysph except ImportError: sys.path.insert(0, dirname(dirname(dirname(__file__)))) def generate_files(dirname, if_modified=True): '''Generates source files from the template files with extension `.mako` If `if_modified` is True (default), the source file will be created only if the template has been modified later than the source ''' for filename in glob.glob(join(dirname, '*.mako')): outfile = abspath(filename[:-5]) message = 'generating file {outfile} from {filename}'.format( outfile=outfile, filename=filename ) if not exists(outfile) or \ (if_modified and is_modified_later(filename, outfile)): _inject_paths_in_sys_path(outfile) from mako.template import Template print(message) template = Template(filename=filename) with open(outfile, 'w') as fp: fp.write(template.render()) else: print('Not ' + message) def main(paths=None): '''Generates source files using mako template files. Parameters ----------- - paths: is a list of directories to convert. If None, all files in current file's directory are converted. ''' if not paths: generate_files(dirname(__file__)) else: for pth in paths: generate_files(pth) if __name__ == '__main__': import sys if '--help' in sys.argv or '-h' in sys.argv: print('usage:') print(' generator.py [filenames]') print() print(' Convert template files with extension .mako into ' 'source files') print(' If filenames is omitted all .mako files in current ' 'directory will be converted') else: main(sys.argv[1:]) cyarray-1.1/cyarray/msstdint.h000066400000000000000000000176451345071315100165320ustar00rootroot00000000000000// ISO C9x compliant stdint.h for Microsoft Visual Studio // Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124 // // Copyright (c) 2006-2013 Alexander Chemeris // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are met: // // 1. Redistributions of source code must retain the above copyright notice, // this list of conditions and the following disclaimer. // // 2. Redistributions in binary form must reproduce the above copyright // notice, this list of conditions and the following disclaimer in the // documentation and/or other materials provided with the distribution. // // 3. Neither the name of the product nor the names of its contributors may // be used to endorse or promote products derived from this software // without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED // WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF // MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO // EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, // PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; // OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, // WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR // OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF // ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // /////////////////////////////////////////////////////////////////////////////// #ifndef _MSC_VER // [ #error "Use this header only with Microsoft Visual C++ compilers!" #endif // _MSC_VER ] #ifndef _MSC_STDINT_H_ // [ #define _MSC_STDINT_H_ #if _MSC_VER > 1000 #pragma once #endif #if _MSC_VER >= 1600 // [ #include #else // ] _MSC_VER >= 1600 [ #include // For Visual Studio 6 in C++ mode and for many Visual Studio versions when // compiling for ARM we should wrap include with 'extern "C++" {}' // or compiler give many errors like this: // error C2733: second C linkage of overloaded function 'wmemchr' not allowed #ifdef __cplusplus extern "C" { #endif # include #ifdef __cplusplus } #endif // Define _W64 macros to mark types changing their size, like intptr_t. #ifndef _W64 # if !defined(__midl) && (defined(_X86_) || defined(_M_IX86)) && _MSC_VER >= 1300 # define _W64 __w64 # else # define _W64 # endif #endif // 7.18.1 Integer types // 7.18.1.1 Exact-width integer types // Visual Studio 6 and Embedded Visual C++ 4 doesn't // realize that, e.g. char has the same size as __int8 // so we give up on __intX for them. #if (_MSC_VER < 1300) typedef signed char int8_t; typedef signed short int16_t; typedef signed int int32_t; typedef unsigned char uint8_t; typedef unsigned short uint16_t; typedef unsigned int uint32_t; #else typedef signed __int8 int8_t; typedef signed __int16 int16_t; typedef signed __int32 int32_t; typedef unsigned __int8 uint8_t; typedef unsigned __int16 uint16_t; typedef unsigned __int32 uint32_t; #endif typedef signed __int64 int64_t; typedef unsigned __int64 uint64_t; // 7.18.1.2 Minimum-width integer types typedef int8_t int_least8_t; typedef int16_t int_least16_t; typedef int32_t int_least32_t; typedef int64_t int_least64_t; typedef uint8_t uint_least8_t; typedef uint16_t uint_least16_t; typedef uint32_t uint_least32_t; typedef uint64_t uint_least64_t; // 7.18.1.3 Fastest minimum-width integer types typedef int8_t int_fast8_t; typedef int16_t int_fast16_t; typedef int32_t int_fast32_t; typedef int64_t int_fast64_t; typedef uint8_t uint_fast8_t; typedef uint16_t uint_fast16_t; typedef uint32_t uint_fast32_t; typedef uint64_t uint_fast64_t; // 7.18.1.4 Integer types capable of holding object pointers #ifdef _WIN64 // [ typedef signed __int64 intptr_t; typedef unsigned __int64 uintptr_t; #else // _WIN64 ][ typedef _W64 signed int intptr_t; typedef _W64 unsigned int uintptr_t; #endif // _WIN64 ] // 7.18.1.5 Greatest-width integer types typedef int64_t intmax_t; typedef uint64_t uintmax_t; // 7.18.2 Limits of specified-width integer types #if !defined(__cplusplus) || defined(__STDC_LIMIT_MACROS) // [ See footnote 220 at page 257 and footnote 221 at page 259 // 7.18.2.1 Limits of exact-width integer types #define INT8_MIN ((int8_t)_I8_MIN) #define INT8_MAX _I8_MAX #define INT16_MIN ((int16_t)_I16_MIN) #define INT16_MAX _I16_MAX #define INT32_MIN ((int32_t)_I32_MIN) #define INT32_MAX _I32_MAX #define INT64_MIN ((int64_t)_I64_MIN) #define INT64_MAX _I64_MAX #define UINT8_MAX _UI8_MAX #define UINT16_MAX _UI16_MAX #define UINT32_MAX _UI32_MAX #define UINT64_MAX _UI64_MAX // 7.18.2.2 Limits of minimum-width integer types #define INT_LEAST8_MIN INT8_MIN #define INT_LEAST8_MAX INT8_MAX #define INT_LEAST16_MIN INT16_MIN #define INT_LEAST16_MAX INT16_MAX #define INT_LEAST32_MIN INT32_MIN #define INT_LEAST32_MAX INT32_MAX #define INT_LEAST64_MIN INT64_MIN #define INT_LEAST64_MAX INT64_MAX #define UINT_LEAST8_MAX UINT8_MAX #define UINT_LEAST16_MAX UINT16_MAX #define UINT_LEAST32_MAX UINT32_MAX #define UINT_LEAST64_MAX UINT64_MAX // 7.18.2.3 Limits of fastest minimum-width integer types #define INT_FAST8_MIN INT8_MIN #define INT_FAST8_MAX INT8_MAX #define INT_FAST16_MIN INT16_MIN #define INT_FAST16_MAX INT16_MAX #define INT_FAST32_MIN INT32_MIN #define INT_FAST32_MAX INT32_MAX #define INT_FAST64_MIN INT64_MIN #define INT_FAST64_MAX INT64_MAX #define UINT_FAST8_MAX UINT8_MAX #define UINT_FAST16_MAX UINT16_MAX #define UINT_FAST32_MAX UINT32_MAX #define UINT_FAST64_MAX UINT64_MAX // 7.18.2.4 Limits of integer types capable of holding object pointers #ifdef _WIN64 // [ # define INTPTR_MIN INT64_MIN # define INTPTR_MAX INT64_MAX # define UINTPTR_MAX UINT64_MAX #else // _WIN64 ][ # define INTPTR_MIN INT32_MIN # define INTPTR_MAX INT32_MAX # define UINTPTR_MAX UINT32_MAX #endif // _WIN64 ] // 7.18.2.5 Limits of greatest-width integer types #define INTMAX_MIN INT64_MIN #define INTMAX_MAX INT64_MAX #define UINTMAX_MAX UINT64_MAX // 7.18.3 Limits of other integer types #ifdef _WIN64 // [ # define PTRDIFF_MIN _I64_MIN # define PTRDIFF_MAX _I64_MAX #else // _WIN64 ][ # define PTRDIFF_MIN _I32_MIN # define PTRDIFF_MAX _I32_MAX #endif // _WIN64 ] #define SIG_ATOMIC_MIN INT_MIN #define SIG_ATOMIC_MAX INT_MAX #ifndef SIZE_MAX // [ # ifdef _WIN64 // [ # define SIZE_MAX _UI64_MAX # else // _WIN64 ][ # define SIZE_MAX _UI32_MAX # endif // _WIN64 ] #endif // SIZE_MAX ] // WCHAR_MIN and WCHAR_MAX are also defined in #ifndef WCHAR_MIN // [ # define WCHAR_MIN 0 #endif // WCHAR_MIN ] #ifndef WCHAR_MAX // [ # define WCHAR_MAX _UI16_MAX #endif // WCHAR_MAX ] #define WINT_MIN 0 #define WINT_MAX _UI16_MAX #endif // __STDC_LIMIT_MACROS ] // 7.18.4 Limits of other integer types #if !defined(__cplusplus) || defined(__STDC_CONSTANT_MACROS) // [ See footnote 224 at page 260 // 7.18.4.1 Macros for minimum-width integer constants #define INT8_C(val) val##i8 #define INT16_C(val) val##i16 #define INT32_C(val) val##i32 #define INT64_C(val) val##i64 #define UINT8_C(val) val##ui8 #define UINT16_C(val) val##ui16 #define UINT32_C(val) val##ui32 #define UINT64_C(val) val##ui64 // 7.18.4.2 Macros for greatest-width integer constants // These #ifndef's are needed to prevent collisions with . // Check out Issue 9 for the details. #ifndef INTMAX_C // [ # define INTMAX_C INT64_C #endif // INTMAX_C ] #ifndef UINTMAX_C // [ # define UINTMAX_C UINT64_C #endif // UINTMAX_C ] #endif // __STDC_CONSTANT_MACROS ] #endif // _MSC_VER >= 1600 ] #endif // _MSC_STDINT_H_ ] cyarray-1.1/cyarray/tests/000077500000000000000000000000001345071315100156415ustar00rootroot00000000000000cyarray-1.1/cyarray/tests/__init__.py000066400000000000000000000000001345071315100177400ustar00rootroot00000000000000cyarray-1.1/cyarray/tests/test_carray.py000066400000000000000000000327301345071315100205400ustar00rootroot00000000000000"""Tests for the carray module. Only the LongArray is tested. As the code in carray.pyx is auto-generated, tests for one class hould suffice. """ # standard imports import unittest import numpy # local imports from cyarray.carray import LongArray, py_aligned class TestAligned(unittest.TestCase): def test_aligned_to_64_bits(self): self.assertEqual(py_aligned(12, 1), 64) self.assertEqual(py_aligned(1, 1), 64) self.assertEqual(py_aligned(64, 1), 64) self.assertEqual(py_aligned(120, 1), 128) self.assertEqual(py_aligned(1, 2), 32) self.assertEqual(py_aligned(12, 2), 32) self.assertEqual(py_aligned(32, 2), 32) self.assertEqual(py_aligned(33, 2), 64) self.assertEqual(py_aligned(1, 3), 64) self.assertEqual(py_aligned(65, 3), 256) self.assertEqual(py_aligned(1, 4), 16) self.assertEqual(py_aligned(16, 4), 16) self.assertEqual(py_aligned(21, 4), 32) self.assertEqual(py_aligned(1, 5), 64) self.assertEqual(py_aligned(13, 5), 128) self.assertEqual(py_aligned(1, 8), 8) self.assertEqual(py_aligned(8, 8), 8) self.assertEqual(py_aligned(11, 8), 16) class TestLongArray(unittest.TestCase): """ Tests for the LongArray class. """ def test_constructor(self): """ Test the constructor. """ la = LongArray(10) self.assertEqual(la.length, 10) self.assertEqual(la.alloc, 10) self.assertEqual(len(la.get_npy_array()), 10) la = LongArray() self.assertEqual(la.length, 0) self.assertEqual(la.alloc, 16) self.assertEqual(len(la.get_npy_array()), 0) def test_get_set_indexing(self): """ Test get/set and [] operator. """ la = LongArray(10) la.set(0, 10) la.set(9, 1) self.assertEqual(la.get(0), 10) self.assertEqual(la.get(9), 1) la[9] = 2 self.assertEqual(la[9], 2) def test_append(self): """ Test the append function. """ la = LongArray(0) la.append(1) la.append(2) la.append(3) self.assertEqual(la.length, 3) self.assertEqual(la[0], 1) self.assertEqual(la[1], 2) self.assertEqual(la[2], 3) def test_reserve(self): """ Tests the reserve function. """ la = LongArray(0) la.reserve(10) self.assertEqual(la.alloc, 16) self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) la.reserve(20) self.assertEqual(la.alloc, 20) self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) def test_resize(self): """ Tests the resize function. """ la = LongArray(0) la.resize(20) self.assertEqual(la.length, 20) self.assertEqual(len(la.get_npy_array()), 20) self.assertEqual(la.alloc >= la.length, True) def test_get_npy_array(self): """ Tests the get_npy_array array. """ la = LongArray(3) la[0] = 1 la[1] = 2 la[2] = 3 nparray = la.get_npy_array() self.assertEqual(len(nparray), 3) for i in range(3): self.assertEqual(nparray[0], la[0]) def test_set_data(self): """ Tests the set_data function. """ la = LongArray(5) np = numpy.arange(5) la.set_data(np) for i in range(5): self.assertEqual(la[i], np[i]) self.assertRaises(ValueError, la.set_data, numpy.arange(10)) def test_squeeze(self): la = LongArray(5) la.append(4) self.assertEqual(la.alloc > la.length, True) la.squeeze() self.assertEqual(la.length, 6) self.assertEqual(la.alloc >= la.length, True) self.assertEqual(len(la.get_npy_array()), 6) def test_squeeze_for_zero_length_array(self): # Given. la = LongArray() # When la.squeeze() # Then self.assertEqual(la.length, 0) self.assertEqual(len(la.get_npy_array()), 0) self.assertEqual(la.alloc >= la.length, True) del la # This should work and not segfault. def test_squeeze_large_array_should_not_segfault(self): # Given la = LongArray(10) la.set_data(numpy.zeros(10, dtype=int)) la.reserve(100000) # When la.squeeze() la.reserve(1000) # Then self.assertEqual(la.length, 10) numpy.testing.assert_array_almost_equal(la.get_npy_array(), 0) self.assertEqual(la.alloc >= la.length, True) def test_reset(self): """ Tests the reset function. """ la = LongArray(5) la.reset() self.assertEqual(la.length, 0) self.assertEqual(la.alloc, 5) self.assertEqual(len(la.get_npy_array()), 0) def test_extend(self): """ Tests the extend function. """ l1 = LongArray(5) for i in range(5): l1[i] = i l2 = LongArray(5) for i in range(5): l2[i] = 5 + i l1.extend(l2.get_npy_array()) self.assertEqual(l1.length, 10) self.assertEqual( numpy.allclose( l1.get_npy_array(), numpy.arange(10)), True) def test_remove(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) rem = [0, 4, 3] l1.remove(numpy.array(rem, dtype=numpy.int)) self.assertEqual(l1.length, 7) self.assertEqual(numpy.allclose([7, 1, 2, 8, 9, 5, 6], l1.get_npy_array()), True) l1.remove(numpy.array(rem, dtype=numpy.int)) self.assertEqual(l1.length, 4) self.assertEqual(numpy.allclose( [6, 1, 2, 5], l1.get_npy_array()), True) rem = [0, 1, 3] l1.remove(numpy.array(rem, dtype=numpy.int)) self.assertEqual(l1.length, 1) self.assertEqual(numpy.allclose([2], l1.get_npy_array()), True) l1.remove(numpy.array([0], dtype=numpy.int)) self.assertEqual(l1.length, 0) self.assertEqual(len(l1.get_npy_array()), 0) def test_remove_with_strides(self): # Given l1 = LongArray(12) l1.set_data(numpy.arange(12)) # When rem = [3, 1] l1.remove(numpy.array(rem, dtype=numpy.int), stride=3) # Then self.assertEqual(l1.length, 6) self.assertEqual(numpy.allclose([0, 1, 2, 6, 7, 8], l1.get_npy_array()), True) # Given l1 = LongArray(12) l1.set_data(numpy.arange(12)) # When rem = [0, 2] l1.remove(numpy.array(rem, dtype=numpy.int), stride=3) # Then self.assertEqual(l1.length, 6) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] self.assertEqual(numpy.allclose([9, 10, 11, 3, 4, 5], l1.get_npy_array()), True) def test_align_array(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) new_indices = LongArray(10) new_indices.set_data(numpy.asarray([1, 5, 3, 2, 4, 7, 8, 6, 9, 0])) l1.align_array(new_indices) self.assertEqual(numpy.allclose([1, 5, 3, 2, 4, 7, 8, 6, 9, 0], l1.get_npy_array()), True) # Test case with strides. l1 = LongArray(6) l1.set_data(numpy.arange(6)) new_indices = LongArray(3) new_indices.set_data(numpy.asarray([2, 1, 0])) l1.align_array(new_indices, 2) self.assertEqual(numpy.allclose([4, 5, 2, 3, 0, 1], l1.get_npy_array()), True) def test_copy_subset(self): l1 = LongArray(10) l1.set_data(numpy.arange(10)) l2 = LongArray(4) l2[0] = 4 l2[1] = 3 l2[2] = 2 l2[3] = 1 # a valid copy. l1.copy_subset(l2, 5, 9) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 4, 3, 2, 1, 9], l1.get_npy_array()), True) # try to copy different sized arrays without any index specification. l1.set_data(numpy.arange(10)) # copy to the last k values of source array. l1.copy_subset(l2, start_index=6) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 5, 4, 3, 2, 1], l1.get_npy_array()), True) l1.set_data(numpy.arange(10)) l1.copy_subset(l2, start_index=7) self.assertEqual(numpy.allclose([0, 1, 2, 3, 4, 5, 6, 4, 3, 2], l1.get_npy_array()), True) # some invalid operations. l1.set_data(numpy.arange(10)) self.assertRaises(ValueError, l1.copy_subset, l2, -1, 1) self.assertRaises(ValueError, l1.copy_subset, l2, 3, 2) self.assertRaises(ValueError, l1.copy_subset, l2, 0, 11) self.assertRaises(ValueError, l1.copy_subset, l2, 10, 20) self.assertRaises(ValueError, l1.copy_subset, l2, -1, -1) def test_copy_subset_works_with_strides(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(4) l2.set_data(numpy.arange(10, 14)) # When l1.copy_subset(l2, 2, 3, stride=2) # Then numpy.testing.assert_array_equal( l1.get_npy_array(), [0, 1, 2, 3, 10, 11, 6, 7] ) # When l1.copy_subset(l2, 2, 4, stride=2) # Then numpy.testing.assert_array_equal( l1.get_npy_array(), [0, 1, 2, 3, 10, 11, 12, 13] ) def test_copy_values(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(8) l2.set_data(numpy.zeros(8, dtype=int)) # When indices = LongArray(3) indices.set_data(numpy.array([2, 4, 6])) l1.copy_values(indices, l2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [2, 4, 6] + [0] * 5 ) # When l2.set_data(numpy.zeros(8, dtype=int)) indices.set_data(numpy.array([1, 2, 3])) l1.copy_values(indices, l2, stride=2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [2, 3, 4, 5, 6, 7, 0, 0] ) def test_copy_values_with_start_index(self): # Given l1 = LongArray(8) l1.set_data(numpy.arange(8)) l2 = LongArray(8) l2.set_data(numpy.zeros(8, dtype=int)) # When indices = LongArray(3) indices.set_data(numpy.array([2, 4, 6])) l1.copy_values(indices, l2, start=5) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [0] * 5 + [2, 4, 6] ) # When l2.set_data(numpy.zeros(8, dtype=int)) indices.set_data(numpy.array([1, 2, 3])) l1.copy_values(indices, l2, stride=2, start=2) # Then numpy.testing.assert_array_equal( l2.get_npy_array(), [0, 0, 2, 3, 4, 5, 6, 7] ) def test_update_min_max(self): """ Tests the update_min_max function. """ l1 = LongArray(10) l1.set_data(numpy.arange(10)) l1.update_min_max() self.assertEqual(l1.minimum, 0) self.assertEqual(l1.maximum, 9) l1[9] = -1 l1[0] = -20 l1[4] = 200 l1.update_min_max() self.assertEqual(l1.minimum, -20) self.assertEqual(l1.maximum, 200) def test_pickling(self): """ Tests the __reduce__ and __setstate__ functions. """ l1 = LongArray(10) l1.set_data(numpy.arange(10)) import pickle l1_dump = pickle.dumps(l1) l1_load = pickle.loads(l1_dump) self.assertEqual( (l1_load.get_npy_array() == l1.get_npy_array()).all(), True) def test_set_view(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. view = LongArray() view.set_view(src, 1, 4) # Then. self.assertEqual(view.length, 3) expect = list(range(1, 4)) self.assertListEqual(view.get_npy_array().tolist(), expect) def test_set_view_for_empty_array(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When. view = LongArray() view.set_view(src, 1, 1) # Then. self.assertEqual(view.length, 0) expect = [] self.assertListEqual(view.get_npy_array().tolist(), expect) def test_set_view_stores_reference_to_parent(self): # Given src = LongArray() src.extend(numpy.arange(5)) # When view = LongArray() view.set_view(src, 1, 4) del src # Then. self.assertEqual(view.length, 3) expect = list(range(1, 4)) self.assertListEqual(view.get_npy_array().tolist(), expect) def test_reset_works_after_set_view(self): # Given src = LongArray() src.extend(numpy.arange(5)) view = LongArray() view.set_view(src, 1, 3) # When. view.reset() view.extend(numpy.arange(3) * 10) # Then. self.assertEqual(view.length, 3) expect = (numpy.arange(3) * 10).tolist() self.assertListEqual(view.get_npy_array().tolist(), expect) if __name__ == '__main__': unittest.main() cyarray-1.1/docs/000077500000000000000000000000001345071315100137555ustar00rootroot00000000000000cyarray-1.1/docs/Makefile000066400000000000000000000011401345071315100154110ustar00rootroot00000000000000# Minimal makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build SPHINXPROJ = cyarray SOURCEDIR = source BUILDDIR = build # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)cyarray-1.1/docs/make.bat000066400000000000000000000014571345071315100153710ustar00rootroot00000000000000@ECHO OFF pushd %~dp0 REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set SOURCEDIR=source set BUILDDIR=build set SPHINXPROJ=cyarray if "%1" == "" goto help %SPHINXBUILD% >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% goto end :help %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% :end popd cyarray-1.1/docs/source/000077500000000000000000000000001345071315100152555ustar00rootroot00000000000000cyarray-1.1/docs/source/conf.py000066400000000000000000000121401345071315100165520ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # cyarray documentation build configuration file, created by # sphinx-quickstart on Thu Nov 29 20:49:39 2018. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.mathjax', 'sphinx.ext.viewcode', 'sphinx.ext.napoleon'] autodoc_default_flags = ['show-inheritance'] autoclass_content = "both" napoleon_google_docstring = True napoleon_numpy_docstring = True napoleon_include_private_with_doc = False napoleon_include_special_with_doc = True # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'cyarray' copyright = '2018, PySPH developers' author = 'PySPH developers' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '1.0' # The full version, including alpha/beta/rc tags. release = '1.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = [] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'cyarraydoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # # 'preamble': '', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'cyarray.tex', 'cyarray Documentation', 'PySPH developers', 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'cyarray', 'cyarray Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'cyarray', 'cyarray Documentation', author, 'cyarray', 'One line description of project.', 'Miscellaneous'), ] # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = {'https://docs.python.org/': None} cyarray-1.1/docs/source/index.rst000066400000000000000000000013101345071315100171110ustar00rootroot00000000000000.. cyarray documentation master file, created by sphinx-quickstart on Thu Nov 29 20:49:39 2018. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to cyarray's documentation! =================================== The cyarray package provides a fast, typed, re-sizable, Cython array. .. toctree:: :maxdepth: 2 :caption: Contents: overview.rst ************************ Reference documentation ************************ Autogenerated from doc strings using sphinx's autodoc feature. .. toctree:: :maxdepth: 2 reference/index Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` cyarray-1.1/docs/source/overview.rst000066400000000000000000000033361345071315100176620ustar00rootroot00000000000000========= Overview ========= The cyarray package provides a fast, typed, re-sizable, Cython_ array. It currently provides the following arrays: ``IntArray, UIntArray, LongArray, FloatArray, DoubleArray``. All arrays provide for the following operations: - access by indexing. - access through get/set function. - resizing the array. - appending values at the end of the array. - reserving space for future appends. - access to internal data through a numpy array. If you are writing Cython code this is a convenient array to use as it exposes the raw underlying pointer to the data. For example if you use a ``FloatArray`` and access its ``data`` attribute it will be a ``float*``. Each array also provides an interface to its data through a numpy array. This is done through the ``get_npy_array`` function. The returned numpy array can be used just like any other numpy array but for the following restrictions: - the array may not be resized. - references of this array should not be kept. - slices of this array may not be made. The numpy array may however be copied and used in any manner. Installation ------------ cyarray can be installed using pip_:: $ pip install cyarray The package requires ``Cython``, ``numpy``, and ``mako`` to be installed and also requires a suitably configured C/C++ compiler. .. _pip: http://www.pip-installer.org .. _Cython: https://cython.org Usage ----- In Python one may import and use the package as:: from cyarray.api import IntArray a = IntArray(10) Here ``a`` is an array of 10 integers. For more usage information, see the simple test cases in `test_carray.py `_. Also see the reference documentation included here. cyarray-1.1/docs/source/reference/000077500000000000000000000000001345071315100172135ustar00rootroot00000000000000cyarray-1.1/docs/source/reference/carray.rst000066400000000000000000000001361345071315100212260ustar00rootroot00000000000000Module carray =============== .. automodule:: cyarray.carray :members: :undoc-members: cyarray-1.1/docs/source/reference/index.rst000066400000000000000000000002541345071315100210550ustar00rootroot00000000000000cyarray reference documentation ================================ Autogenerated from doc strings using sphinx’s autodoc feature. .. toctree:: :maxdepth: 3 carray cyarray-1.1/requirements.txt000066400000000000000000000000571345071315100163130ustar00rootroot00000000000000Cython>=0.20 setuptools>=6.0 numpy mako pytest cyarray-1.1/setup.py000066400000000000000000000121061345071315100145370ustar00rootroot00000000000000import os import sys from subprocess import check_output if len(os.environ.get('COVERAGE', '')) > 0: MACROS = [("CYTHON_TRACE", "1"), ("CYTHON_TRACE_NOGIL", "1")] COMPILER_DIRECTIVES = {"linetrace": True} print("-" * 80) print("Enabling linetracing for cython and setting CYTHON_TRACE = 1") print("-" * 80) else: MACROS = [] COMPILER_DIRECTIVES = {} MODE = 'normal' if len(sys.argv) >= 2 and \ ('--help' in sys.argv[1:] or sys.argv[1] in ('--help-commands', 'egg_info', '--version', 'clean', 'sdist')): MODE = 'info' def get_basic_extensions(): if MODE == 'info': try: from Cython.Distutils import Extension except ImportError: from distutils.core import Extension try: import numpy except ImportError: include_dirs = [] else: include_dirs = [numpy.get_include()] else: from Cython.Distutils import Extension import numpy include_dirs = [numpy.get_include()] ext_modules = [ Extension( name="cyarray.carray", sources=["cyarray/carray.pyx"], include_dirs=include_dirs, ), ] return ext_modules def create_sources(): argv = sys.argv if 'build_ext' in argv or 'develop' in sys.argv or 'install' in argv: generator = os.path.join('cyarray', 'generator.py') cmd = [sys.executable, generator, os.path.abspath('cyarray')] print(check_output(cmd).decode()) def _is_cythonize_default(): import warnings result = True with warnings.catch_warnings(): warnings.simplefilter("ignore") try: # old_build_ext was introduced in Cython 0.25 and this is when # cythonize was made the default. from Cython.Distutils import old_build_ext # noqa: F401 except ImportError: result = False return result def setup_package(): from setuptools import find_packages, setup if MODE == 'info': cmdclass = {} else: from Cython.Distutils import build_ext cmdclass = {'build_ext': build_ext} create_sources() # Extract the version information from pysph/__init__.py info = {} module = os.path.join('cyarray', '__init__.py') exec(compile(open(module).read(), module, 'exec'), info) # The requirements. install_requires = [ 'numpy', 'mako', 'Cython>=0.20', 'setuptools>=6.0' ] tests_require = ["pytest"] docs_require = ["sphinx"] ext_modules = get_basic_extensions() if MODE != 'info' and _is_cythonize_default(): # Cython >= 0.25 uses cythonize to compile the extensions. This # requires the compile_time_env to be set explicitly to work. compile_env = {} include_path = set() for mod in ext_modules: compile_env.update(mod.cython_compile_time_env or {}) include_path.update(mod.include_dirs) from Cython.Build import cythonize ext_modules = cythonize( ext_modules, compile_time_env=compile_env, include_path=list(include_path), compiler_directives=COMPILER_DIRECTIVES, ) setup(name='cyarray', version=info['__version__'], author='Cyarray Developers', author_email='pysph-dev@googlegroups.com', description='A fast, typed, resizable, Cython array.', long_description=open('README.rst').read(), url='http://github.com/pypr/cyarray', license="BSD", keywords="Cython array resizable", packages=find_packages(), package_data={ '': ['*.pxd', '*.mako', '*.rst'] }, # exclude package data in installation. exclude_package_data={ '': ['Makefile', '*.bat', '*.cfg', '*.rst', '*.sh', '*.yml'], }, ext_modules=ext_modules, include_package_data=True, cmdclass=cmdclass, install_requires=install_requires, extras_require={ "docs": docs_require, "tests": tests_require, "dev": docs_require + tests_require, }, zip_safe=False, platforms=['Linux', 'Mac OS-X', 'Unix', 'Windows'], classifiers=[c.strip() for c in """\ Development Status :: 5 - Production/Stable Environment :: Console Intended Audience :: Developers Intended Audience :: Science/Research License :: OSI Approved :: BSD License Natural Language :: English Operating System :: MacOS :: MacOS X Operating System :: Microsoft :: Windows Operating System :: POSIX Operating System :: Unix Programming Language :: Python Programming Language :: Python :: 3 Topic :: Scientific/Engineering Topic :: Scientific/Engineering :: Physics Topic :: Software Development :: Libraries """.splitlines() if len(c.split()) > 0], ) if __name__ == '__main__': setup_package()