portalocker-1.5.1/0000755000076500000240000000000013521711277014124 5ustar rickstaff00000000000000portalocker-1.5.1/CHANGELOG.rst0000644000076500000240000000221413521706531016141 0ustar rickstaff000000000000001.5: * Moved tests to prevent collisions with other packages 1.4: * Added optional file open parameters 1.3: * Improved documentation * Added file handle to locking exceptions 1.2: * Added signed releases and tags to PyPI and Git 1.1: * Added support for Python 3.6+ * Using real time to calculate timeout 1.0: * Complete code refactor. - Splitting of code in logical classes - 100% test coverage and change in API behaviour - The default behavior of the `Lock` class has changed to append instead of write/truncate. 0.6: * Added msvcrt support for Windows 0.5: * Python 3 support 0.4: * Fixing a few bugs, added coveralls support, switched to py.test and added 100% test coverage. - Fixing exception thrown when fail_when_locked is true - Fixing exception "Lock object has no attribute '_release_lock'" when fail_when_locked is true due to the call to Lock._release_lock() which fails because _release_lock is not defined. 0.3: * Now actually returning the file descriptor from the `Lock` class 0.2: * Added `Lock` class to help prevent cache race conditions 0.1: * Initial release portalocker-1.5.1/LICENSE0000644000076500000240000000456313250036236015133 0ustar rickstaff00000000000000PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 -------------------------------------------- 1. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation. 2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python. 4. PSF is making Python available to Licensee on an "AS IS" basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 6. This License Agreement will automatically terminate upon a material breach of its terms and conditions. 7. Nothing in this License Agreement shall be deemed to create any relationship of agency, partnership, or joint venture between PSF and Licensee. This License Agreement does not grant permission to use PSF trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee, or any third party. 8. By copying, installing or otherwise using Python, Licensee agrees to be bound by the terms and conditions of this License Agreement. portalocker-1.5.1/MANIFEST.in0000644000076500000240000000014213521706531015654 0ustar rickstaff00000000000000include CHANGELOG.rst include README.rst include LICENSE recursive-include portalocker_tests *.py portalocker-1.5.1/PKG-INFO0000644000076500000240000001152313521711277015223 0ustar rickstaff00000000000000Metadata-Version: 2.1 Name: portalocker Version: 1.5.1 Summary: Wraps the portalocker recipe for easy usage Home-page: https://github.com/WoLpH/portalocker Author: Rick van Hattem Author-email: wolph@wol.ph License: PSF Description: ############################################ portalocker - Cross-platform locking library ############################################ .. image:: https://travis-ci.org/WoLpH/portalocker.svg?branch=master :alt: Linux Test Status :target: https://travis-ci.org/WoLpH/portalocker .. image:: https://ci.appveyor.com/api/projects/status/mgqry98hgpy4prhh?svg=true :alt: Windows Tests Status :target: https://ci.appveyor.com/project/WoLpH/portalocker .. image:: https://coveralls.io/repos/WoLpH/portalocker/badge.svg?branch=master :alt: Coverage Status :target: https://coveralls.io/r/WoLpH/portalocker?branch=master Overview -------- Portalocker is a library to provide an easy API to file locking. An important detail to note is that on Linux and Unix systems the locks are advisory by default. By specifying the `-o mand` option to the mount command it is possible to enable mandatory file locking on Linux. This is generally not recommended however. For more information about the subject: - https://en.wikipedia.org/wiki/File_locking - http://stackoverflow.com/questions/39292051/portalocker-does-not-seem-to-lock - https://stackoverflow.com/questions/12062466/mandatory-file-lock-on-linux The module is currently maintained by Rick van Hattem . The project resides at https://github.com/WoLpH/portalocker . Bugs and feature requests can be submitted there. Patches are also very welcome. Tips ---- On some networked filesystems it might be needed to force a `os.fsync()` before closing the file so it's actually written before another client reads the file. Effectively this comes down to: :: with portalocker.Lock('some_file', 'rb+', timeout=60) as fh: # do what you need to do ... # flush and sync to filesystem fh.flush() os.fsync(fh.fileno()) Links ----- * Documentation - http://portalocker.readthedocs.org/en/latest/ * Source - https://github.com/WoLpH/portalocker * Bug reports - https://github.com/WoLpH/portalocker/issues * Package homepage - https://pypi.python.org/pypi/portalocker * My blog - http://w.wol.ph/ Examples -------- To make sure your cache generation scripts don't race, use the `Lock` class: >>> import portalocker >>> with portalocker.Lock('somefile', timeout=1) as fh: print >>fh, 'writing some stuff to my cache...' To customize the opening and locking a manual approach is also possible: >>> import portalocker >>> file = open('somefile', 'r+') >>> portalocker.lock(file, portalocker.LOCK_EX) >>> file.seek(12) >>> file.write('foo') >>> file.close() There is no explicit need to unlock the file as it is automatically unlocked after `file.close()`. If you still feel the need to manually unlock a file than you can do it like this: >>> portalocker.unlock(file) Do note that your data might still be in a buffer so it is possible that your data is not available until you `flush()` or `close()`. More examples can be found in the `tests `_. Changelog --------- See the `changelog `_ page. License ------- See the `LICENSE `_ file. Keywords: locking,locks,with statement,windows,linux,unix Platform: any Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Provides-Extra: docs Provides-Extra: tests portalocker-1.5.1/README.rst0000644000076500000240000000634013405602421015605 0ustar rickstaff00000000000000############################################ portalocker - Cross-platform locking library ############################################ .. image:: https://travis-ci.org/WoLpH/portalocker.svg?branch=master :alt: Linux Test Status :target: https://travis-ci.org/WoLpH/portalocker .. image:: https://ci.appveyor.com/api/projects/status/mgqry98hgpy4prhh?svg=true :alt: Windows Tests Status :target: https://ci.appveyor.com/project/WoLpH/portalocker .. image:: https://coveralls.io/repos/WoLpH/portalocker/badge.svg?branch=master :alt: Coverage Status :target: https://coveralls.io/r/WoLpH/portalocker?branch=master Overview -------- Portalocker is a library to provide an easy API to file locking. An important detail to note is that on Linux and Unix systems the locks are advisory by default. By specifying the `-o mand` option to the mount command it is possible to enable mandatory file locking on Linux. This is generally not recommended however. For more information about the subject: - https://en.wikipedia.org/wiki/File_locking - http://stackoverflow.com/questions/39292051/portalocker-does-not-seem-to-lock - https://stackoverflow.com/questions/12062466/mandatory-file-lock-on-linux The module is currently maintained by Rick van Hattem . The project resides at https://github.com/WoLpH/portalocker . Bugs and feature requests can be submitted there. Patches are also very welcome. Tips ---- On some networked filesystems it might be needed to force a `os.fsync()` before closing the file so it's actually written before another client reads the file. Effectively this comes down to: :: with portalocker.Lock('some_file', 'rb+', timeout=60) as fh: # do what you need to do ... # flush and sync to filesystem fh.flush() os.fsync(fh.fileno()) Links ----- * Documentation - http://portalocker.readthedocs.org/en/latest/ * Source - https://github.com/WoLpH/portalocker * Bug reports - https://github.com/WoLpH/portalocker/issues * Package homepage - https://pypi.python.org/pypi/portalocker * My blog - http://w.wol.ph/ Examples -------- To make sure your cache generation scripts don't race, use the `Lock` class: >>> import portalocker >>> with portalocker.Lock('somefile', timeout=1) as fh: print >>fh, 'writing some stuff to my cache...' To customize the opening and locking a manual approach is also possible: >>> import portalocker >>> file = open('somefile', 'r+') >>> portalocker.lock(file, portalocker.LOCK_EX) >>> file.seek(12) >>> file.write('foo') >>> file.close() There is no explicit need to unlock the file as it is automatically unlocked after `file.close()`. If you still feel the need to manually unlock a file than you can do it like this: >>> portalocker.unlock(file) Do note that your data might still be in a buffer so it is possible that your data is not available until you `flush()` or `close()`. More examples can be found in the `tests `_. Changelog --------- See the `changelog `_ page. License ------- See the `LICENSE `_ file. portalocker-1.5.1/portalocker/0000755000076500000240000000000013521711277016451 5ustar rickstaff00000000000000portalocker-1.5.1/portalocker/__about__.py0000644000076500000240000000034713521711212020722 0ustar rickstaff00000000000000__package_name__ = 'portalocker' __author__ = 'Rick van Hattem' __email__ = 'wolph@wol.ph' __version__ = '1.5.1' __description__ = '''Wraps the portalocker recipe for easy usage''' __url__ = 'https://github.com/WoLpH/portalocker' portalocker-1.5.1/portalocker/__init__.py0000644000076500000240000000325113521711212020550 0ustar rickstaff00000000000000from . import __about__ from . import constants from . import exceptions from . import portalocker from . import utils #: The package name on Pypi __package_name__ = __about__.__package_name__ #: Current author and maintainer, view the git history for the previous ones __author__ = __about__.__author__ #: Current author's email address __email__ = __about__.__email__ #: Version number __version__ = '1.5.1' #: Package description for Pypi __description__ = __about__.__description__ #: Package homepage __url__ = __about__.__url__ #: Exception thrown when the file is already locked by someone else AlreadyLocked = exceptions.AlreadyLocked #: Exception thrown if an error occurred during locking LockException = exceptions.LockException #: Lock a file. Note that this is an advisory lock on Linux/Unix systems lock = portalocker.lock #: Unlock a file unlock = portalocker.unlock #: Place an exclusive lock. #: Only one process may hold an exclusive lock for a given file at a given #: time. LOCK_EX = constants.LOCK_EX #: Place a shared lock. #: More than one process may hold a shared lock for a given file at a given #: time. LOCK_SH = constants.LOCK_SH #: Acquire the lock in a non-blocking fashion. LOCK_NB = constants.LOCK_NB #: Remove an existing lock held by this process. LOCK_UN = constants.LOCK_UN #: Locking utility class to automatically handle opening with timeouts and #: context wrappers Lock = utils.Lock RLock = utils.RLock TemporaryFileLock = utils.TemporaryFileLock open_atomic = utils.open_atomic __all__ = [ 'lock', 'unlock', 'LOCK_EX', 'LOCK_SH', 'LOCK_NB', 'LOCK_UN', 'LockException', 'Lock', 'AlreadyLocked', 'open_atomic', ] portalocker-1.5.1/portalocker/constants.py0000644000076500000240000000154713405602255021042 0ustar rickstaff00000000000000''' Locking constants Lock types: - `LOCK_EX` exclusive lock - `LOCK_SH` shared lock Lock flags: - `LOCK_NB` non-blocking Manually unlock, only needed internally - `LOCK_UN` unlock ''' import os # The actual tests will execute the code anyhow so the following code can # safely be ignored from the coverage tests if os.name == 'nt': # pragma: no cover import msvcrt LOCK_EX = 0x1 #: exclusive lock LOCK_SH = 0x2 #: shared lock LOCK_NB = 0x4 #: non-blocking LOCK_UN = msvcrt.LK_UNLCK #: unlock elif os.name == 'posix': # pragma: no cover import fcntl LOCK_EX = fcntl.LOCK_EX #: exclusive lock LOCK_SH = fcntl.LOCK_SH #: shared lock LOCK_NB = fcntl.LOCK_NB #: non-blocking LOCK_UN = fcntl.LOCK_UN #: unlock else: # pragma: no cover raise RuntimeError('PortaLocker only defined for nt and posix platforms') portalocker-1.5.1/portalocker/exceptions.py0000644000076500000240000000054713405602421021201 0ustar rickstaff00000000000000class BaseLockException(Exception): # Error codes: LOCK_FAILED = 1 def __init__(self, *args, **kwargs): self.fh = kwargs.pop('fh', None) Exception.__init__(self, *args, **kwargs) class LockException(BaseLockException): pass class AlreadyLocked(BaseLockException): pass class FileToLarge(BaseLockException): pass portalocker-1.5.1/portalocker/portalocker.py0000644000076500000240000001271513405602421021345 0ustar rickstaff00000000000000import os import sys from . import exceptions from . import constants if os.name == 'nt': # pragma: no cover import win32con import win32file import pywintypes import winerror import msvcrt __overlapped = pywintypes.OVERLAPPED() if sys.version_info.major == 2: lock_length = -1 else: lock_length = int(2**31 - 1) def lock(file_, flags): if flags & constants.LOCK_SH: if sys.version_info.major == 2: if flags & constants.LOCK_NB: mode = win32con.LOCKFILE_FAIL_IMMEDIATELY else: mode = 0 else: if flags & constants.LOCK_NB: mode = msvcrt.LK_NBRLCK else: mode = msvcrt.LK_RLCK # is there any reason not to reuse the following structure? hfile = win32file._get_osfhandle(file_.fileno()) try: win32file.LockFileEx(hfile, mode, 0, -0x10000, __overlapped) except pywintypes.error as exc_value: # error: (33, 'LockFileEx', 'The process cannot access the file # because another process has locked a portion of the file.') if exc_value.winerror == winerror.ERROR_LOCK_VIOLATION: raise exceptions.LockException( exceptions.LockException.LOCK_FAILED, exc_value.strerror, fh=file_) else: # Q: Are there exceptions/codes we should be dealing with # here? raise else: mode = win32con.LOCKFILE_EXCLUSIVE_LOCK if flags & constants.LOCK_NB: mode |= win32con.LOCKFILE_FAIL_IMMEDIATELY if flags & constants.LOCK_NB: mode = msvcrt.LK_NBLCK else: mode = msvcrt.LK_LOCK # windows locks byte ranges, so make sure to lock from file start try: savepos = file_.tell() if savepos: # [ ] test exclusive lock fails on seek here # [ ] test if shared lock passes this point file_.seek(0) # [x] check if 0 param locks entire file (not documented in # Python) # [x] fails with "IOError: [Errno 13] Permission denied", # but -1 seems to do the trick try: msvcrt.locking(file_.fileno(), mode, lock_length) except IOError as exc_value: # [ ] be more specific here raise exceptions.LockException( exceptions.LockException.LOCK_FAILED, exc_value.strerror, fh=file_) finally: if savepos: file_.seek(savepos) except IOError as exc_value: raise exceptions.LockException( exceptions.LockException.LOCK_FAILED, exc_value.strerror, fh=file_) def unlock(file_): try: savepos = file_.tell() if savepos: file_.seek(0) try: msvcrt.locking(file_.fileno(), constants.LOCK_UN, lock_length) except IOError as exc_value: if exc_value.strerror == 'Permission denied': hfile = win32file._get_osfhandle(file_.fileno()) try: win32file.UnlockFileEx( hfile, 0, -0x10000, __overlapped) except pywintypes.error as exc_value: if exc_value.winerror == winerror.ERROR_NOT_LOCKED: # error: (158, 'UnlockFileEx', # 'The segment is already unlocked.') # To match the 'posix' implementation, silently # ignore this error pass else: # Q: Are there exceptions/codes we should be # dealing with here? raise else: raise exceptions.LockException( exceptions.LockException.LOCK_FAILED, exc_value.strerror, fh=file_) finally: if savepos: file_.seek(savepos) except IOError as exc_value: raise exceptions.LockException( exceptions.LockException.LOCK_FAILED, exc_value.strerror, fh=file_) elif os.name == 'posix': # pragma: no cover import fcntl def lock(file_, flags): locking_exceptions = IOError, try: # pragma: no cover locking_exceptions += BlockingIOError, except NameError: # pragma: no cover pass try: fcntl.flock(file_.fileno(), flags) except locking_exceptions as exc_value: # The exception code varies on different systems so we'll catch # every IO error raise exceptions.LockException(exc_value, fh=file_) def unlock(file_): fcntl.flock(file_.fileno(), constants.LOCK_UN) else: # pragma: no cover raise RuntimeError('PortaLocker only defined for nt and posix platforms') portalocker-1.5.1/portalocker/utils.py0000644000076500000240000001716513430266672020200 0ustar rickstaff00000000000000import os import time import atexit import tempfile import contextlib from . import exceptions from . import constants from . import portalocker current_time = getattr(time, "monotonic", time.time) DEFAULT_TIMEOUT = 5 DEFAULT_CHECK_INTERVAL = 0.25 LOCK_METHOD = constants.LOCK_EX | constants.LOCK_NB __all__ = [ 'Lock', 'open_atomic', ] @contextlib.contextmanager def open_atomic(filename, binary=True): '''Open a file for atomic writing. Instead of locking this method allows you to write the entire file and move it to the actual location. Note that this makes the assumption that a rename is atomic on your platform which is generally the case but not a guarantee. http://docs.python.org/library/os.html#os.rename >>> filename = 'test_file.txt' >>> if os.path.exists(filename): ... os.remove(filename) >>> with open_atomic(filename) as fh: ... written = fh.write(b'test') >>> assert os.path.exists(filename) >>> os.remove(filename) ''' assert not os.path.exists(filename), '%r exists' % filename path, name = os.path.split(filename) # Create the parent directory if it doesn't exist if path and not os.path.isdir(path): # pragma: no cover os.makedirs(path) temp_fh = tempfile.NamedTemporaryFile( mode=binary and 'wb' or 'w', dir=path, delete=False, ) yield temp_fh temp_fh.flush() os.fsync(temp_fh.fileno()) temp_fh.close() try: os.rename(temp_fh.name, filename) finally: try: os.remove(temp_fh.name) except Exception: pass class Lock(object): def __init__( self, filename, mode='a', timeout=DEFAULT_TIMEOUT, check_interval=DEFAULT_CHECK_INTERVAL, fail_when_locked=False, flags=LOCK_METHOD, **file_open_kwargs): '''Lock manager with build-in timeout filename -- filename mode -- the open mode, 'a' or 'ab' should be used for writing truncate -- use truncate to emulate 'w' mode, None is disabled, 0 is truncate to 0 bytes timeout -- timeout when trying to acquire a lock check_interval -- check interval while waiting fail_when_locked -- after the initial lock failed, return an error or lock the file **file_open_kwargs -- The kwargs for the `open(...)` call fail_when_locked is useful when multiple threads/processes can race when creating a file. If set to true than the system will wait till the lock was acquired and then return an AlreadyLocked exception. Note that the file is opened first and locked later. So using 'w' as mode will result in truncate _BEFORE_ the lock is checked. ''' if 'w' in mode: truncate = True mode = mode.replace('w', 'a') else: truncate = False self.fh = None self.filename = filename self.mode = mode self.truncate = truncate self.timeout = timeout self.check_interval = check_interval self.fail_when_locked = fail_when_locked self.flags = flags self.file_open_kwargs = file_open_kwargs def acquire( self, timeout=None, check_interval=None, fail_when_locked=None): '''Acquire the locked filehandle''' if timeout is None: timeout = self.timeout if timeout is None: timeout = 0 if check_interval is None: check_interval = self.check_interval if fail_when_locked is None: fail_when_locked = self.fail_when_locked # If we already have a filehandle, return it fh = self.fh if fh: return fh # Get a new filehandler fh = self._get_fh() try: # Try to lock fh = self._get_lock(fh) except exceptions.LockException as exception: # Try till the timeout has passed timeoutend = current_time() + timeout while timeoutend > current_time(): # Wait a bit time.sleep(check_interval) # Try again try: # We already tried to the get the lock # If fail_when_locked is true, then stop trying if fail_when_locked: raise exceptions.AlreadyLocked(exception) else: # pragma: no cover # We've got the lock fh = self._get_lock(fh) break except exceptions.LockException: pass else: # We got a timeout... reraising raise exceptions.LockException(exception) # Prepare the filehandle (truncate if needed) fh = self._prepare_fh(fh) self.fh = fh return fh def release(self): '''Releases the currently locked file handle''' if self.fh: self.fh.close() self.fh = None def _get_fh(self): '''Get a new filehandle''' return open(self.filename, self.mode, **self.file_open_kwargs) def _get_lock(self, fh): ''' Try to lock the given filehandle returns LockException if it fails''' portalocker.lock(fh, self.flags) return fh def _prepare_fh(self, fh): ''' Prepare the filehandle for usage If truncate is a number, the file will be truncated to that amount of bytes ''' if self.truncate: fh.seek(0) fh.truncate(0) return fh def __enter__(self): return self.acquire() def __exit__(self, type_, value, tb): self.release() def __delete__(self, instance): # pragma: no cover instance.release() class RLock(Lock): """ A reentrant lock, functions in a similar way to threading.RLock in that it can be acquired multiple times. When the corresponding number of release() calls are made the lock will finally release the underlying file lock. """ def __init__( self, filename, mode='a', timeout=DEFAULT_TIMEOUT, check_interval=DEFAULT_CHECK_INTERVAL, fail_when_locked=False, flags=LOCK_METHOD): super(RLock, self).__init__(filename, mode, timeout, check_interval, fail_when_locked, flags) self._acquire_count = 0 def acquire( self, timeout=None, check_interval=None, fail_when_locked=None): if self._acquire_count >= 1: fh = self.fh else: fh = super(RLock, self).acquire(timeout, check_interval, fail_when_locked) self._acquire_count += 1 return fh def release(self): if self._acquire_count == 0: raise exceptions.LockException( "Cannot release more times than acquired") if self._acquire_count == 1: super(RLock, self).release() self._acquire_count -= 1 class TemporaryFileLock(Lock): def __init__(self, filename='.lock', timeout=DEFAULT_TIMEOUT, check_interval=DEFAULT_CHECK_INTERVAL, fail_when_locked=True, flags=LOCK_METHOD): Lock.__init__(self, filename=filename, mode='w', timeout=timeout, check_interval=check_interval, fail_when_locked=fail_when_locked, flags=flags) atexit.register(self.release) def release(self): Lock.release(self) if os.path.isfile(self.filename): # pragma: no branch os.unlink(self.filename) portalocker-1.5.1/portalocker.egg-info/0000755000076500000240000000000013521711277020143 5ustar rickstaff00000000000000portalocker-1.5.1/portalocker.egg-info/PKG-INFO0000644000076500000240000001152313521711276021241 0ustar rickstaff00000000000000Metadata-Version: 2.1 Name: portalocker Version: 1.5.1 Summary: Wraps the portalocker recipe for easy usage Home-page: https://github.com/WoLpH/portalocker Author: Rick van Hattem Author-email: wolph@wol.ph License: PSF Description: ############################################ portalocker - Cross-platform locking library ############################################ .. image:: https://travis-ci.org/WoLpH/portalocker.svg?branch=master :alt: Linux Test Status :target: https://travis-ci.org/WoLpH/portalocker .. image:: https://ci.appveyor.com/api/projects/status/mgqry98hgpy4prhh?svg=true :alt: Windows Tests Status :target: https://ci.appveyor.com/project/WoLpH/portalocker .. image:: https://coveralls.io/repos/WoLpH/portalocker/badge.svg?branch=master :alt: Coverage Status :target: https://coveralls.io/r/WoLpH/portalocker?branch=master Overview -------- Portalocker is a library to provide an easy API to file locking. An important detail to note is that on Linux and Unix systems the locks are advisory by default. By specifying the `-o mand` option to the mount command it is possible to enable mandatory file locking on Linux. This is generally not recommended however. For more information about the subject: - https://en.wikipedia.org/wiki/File_locking - http://stackoverflow.com/questions/39292051/portalocker-does-not-seem-to-lock - https://stackoverflow.com/questions/12062466/mandatory-file-lock-on-linux The module is currently maintained by Rick van Hattem . The project resides at https://github.com/WoLpH/portalocker . Bugs and feature requests can be submitted there. Patches are also very welcome. Tips ---- On some networked filesystems it might be needed to force a `os.fsync()` before closing the file so it's actually written before another client reads the file. Effectively this comes down to: :: with portalocker.Lock('some_file', 'rb+', timeout=60) as fh: # do what you need to do ... # flush and sync to filesystem fh.flush() os.fsync(fh.fileno()) Links ----- * Documentation - http://portalocker.readthedocs.org/en/latest/ * Source - https://github.com/WoLpH/portalocker * Bug reports - https://github.com/WoLpH/portalocker/issues * Package homepage - https://pypi.python.org/pypi/portalocker * My blog - http://w.wol.ph/ Examples -------- To make sure your cache generation scripts don't race, use the `Lock` class: >>> import portalocker >>> with portalocker.Lock('somefile', timeout=1) as fh: print >>fh, 'writing some stuff to my cache...' To customize the opening and locking a manual approach is also possible: >>> import portalocker >>> file = open('somefile', 'r+') >>> portalocker.lock(file, portalocker.LOCK_EX) >>> file.seek(12) >>> file.write('foo') >>> file.close() There is no explicit need to unlock the file as it is automatically unlocked after `file.close()`. If you still feel the need to manually unlock a file than you can do it like this: >>> portalocker.unlock(file) Do note that your data might still be in a buffer so it is possible that your data is not available until you `flush()` or `close()`. More examples can be found in the `tests `_. Changelog --------- See the `changelog `_ page. License ------- See the `LICENSE `_ file. Keywords: locking,locks,with statement,windows,linux,unix Platform: any Classifier: Intended Audience :: Developers Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Provides-Extra: docs Provides-Extra: tests portalocker-1.5.1/portalocker.egg-info/SOURCES.txt0000644000076500000240000000104413521711276022025 0ustar rickstaff00000000000000CHANGELOG.rst LICENSE MANIFEST.in README.rst setup.cfg setup.py portalocker/__about__.py portalocker/__init__.py portalocker/constants.py portalocker/exceptions.py portalocker/portalocker.py portalocker/utils.py portalocker.egg-info/PKG-INFO portalocker.egg-info/SOURCES.txt portalocker.egg-info/dependency_links.txt portalocker.egg-info/requires.txt portalocker.egg-info/top_level.txt portalocker_tests/__init__.py portalocker_tests/conftest.py portalocker_tests/temporary_file_lock.py portalocker_tests/test_combined.py portalocker_tests/tests.pyportalocker-1.5.1/portalocker.egg-info/dependency_links.txt0000644000076500000240000000000113521711276024210 0ustar rickstaff00000000000000 portalocker-1.5.1/portalocker.egg-info/requires.txt0000644000076500000240000000030013521711276022533 0ustar rickstaff00000000000000 [:platform_system == "Windows"] pypiwin32 [docs] sphinx>=1.7.1 [tests] flake8>=3.5.0 pytest>=3.4.0 pytest-cache>=1.0 pytest-cov>=2.5.1 pytest-flakes>=2.0.0 pytest-pep8>=1.0.6 sphinx>=1.7.1 portalocker-1.5.1/portalocker.egg-info/top_level.txt0000644000076500000240000000001413521711276022667 0ustar rickstaff00000000000000portalocker portalocker-1.5.1/portalocker_tests/0000755000076500000240000000000013521711277017673 5ustar rickstaff00000000000000portalocker-1.5.1/portalocker_tests/__init__.py0000644000076500000240000000000013521711212021757 0ustar rickstaff00000000000000portalocker-1.5.1/portalocker_tests/conftest.py0000644000076500000240000000044113512440116022060 0ustar rickstaff00000000000000import py import pytest @pytest.fixture def tmpfile(tmpdir_factory): tmpdir = tmpdir_factory.mktemp('temp') filename = tmpdir.join('tmpfile') yield str(filename) try: filename.remove(ignore_errors=True) except (py.error.EBUSY, py.error.ENOENT): pass portalocker-1.5.1/portalocker_tests/temporary_file_lock.py0000644000076500000240000000040013512440116024257 0ustar rickstaff00000000000000import os import portalocker def test_temporary_file_lock(tmpfile): with portalocker.TemporaryFileLock(tmpfile): pass assert not os.path.isfile(tmpfile) lock = portalocker.TemporaryFileLock(tmpfile) lock.acquire() del lock portalocker-1.5.1/portalocker_tests/test_combined.py0000644000076500000240000000050613512440116023054 0ustar rickstaff00000000000000import sys def test_combined(tmpdir): from distutils import dist import setup output_file = tmpdir.join('combined.py') combine = setup.Combine(dist.Distribution()) combine.output_file = str(output_file) combine.run() sys.path.append(output_file.dirname) import combined assert combined portalocker-1.5.1/portalocker_tests/tests.py0000644000076500000240000001323313512440116021400 0ustar rickstaff00000000000000from __future__ import print_function from __future__ import with_statement import pytest import portalocker def test_exceptions(tmpfile): # Open the file 2 times a = open(tmpfile, 'a') b = open(tmpfile, 'a') # Lock exclusive non-blocking lock_flags = portalocker.LOCK_EX | portalocker.LOCK_NB # First lock file a portalocker.lock(a, lock_flags) # Now see if we can lock file b with pytest.raises(portalocker.LockException): portalocker.lock(b, lock_flags) # Cleanup a.close() b.close() def test_with_timeout(tmpfile): # Open the file 2 times with pytest.raises(portalocker.AlreadyLocked): with portalocker.Lock(tmpfile, timeout=0.1) as fh: print('writing some stuff to my cache...', file=fh) with portalocker.Lock(tmpfile, timeout=0.1, mode='wb', fail_when_locked=True): pass print('writing more stuff to my cache...', file=fh) def test_without_timeout(tmpfile): # Open the file 2 times with pytest.raises(portalocker.LockException): with portalocker.Lock(tmpfile, timeout=None) as fh: print('writing some stuff to my cache...', file=fh) with portalocker.Lock(tmpfile, timeout=None, mode='w'): pass print('writing more stuff to my cache...', file=fh) def test_without_fail(tmpfile): # Open the file 2 times with pytest.raises(portalocker.LockException): with portalocker.Lock(tmpfile, timeout=0.1) as fh: print('writing some stuff to my cache...', file=fh) lock = portalocker.Lock(tmpfile, timeout=0.1) lock.acquire(check_interval=0.05, fail_when_locked=False) def test_simple(tmpfile): with open(tmpfile, 'w') as fh: fh.write('spam and eggs') fh = open(tmpfile, 'r+') portalocker.lock(fh, portalocker.LOCK_EX) fh.seek(13) fh.write('foo') # Make sure we didn't overwrite the original text fh.seek(0) assert fh.read(13) == 'spam and eggs' portalocker.unlock(fh) fh.close() def test_truncate(tmpfile): with open(tmpfile, 'w') as fh: fh.write('spam and eggs') with portalocker.Lock(tmpfile, mode='a+') as fh: # Make sure we didn't overwrite the original text fh.seek(0) assert fh.read(13) == 'spam and eggs' with portalocker.Lock(tmpfile, mode='w+') as fh: # Make sure we truncated the file assert fh.read() == '' def test_class(tmpfile): lock = portalocker.Lock(tmpfile) lock2 = portalocker.Lock(tmpfile, fail_when_locked=False, timeout=0.01) with lock: lock.acquire() with pytest.raises(portalocker.LockException): with lock2: pass with lock2: pass def test_acquire_release(tmpfile): lock = portalocker.Lock(tmpfile) lock2 = portalocker.Lock(tmpfile, fail_when_locked=False) lock.acquire() # acquire lock when nobody is using it with pytest.raises(portalocker.LockException): # another party should not be able to acquire the lock lock2.acquire(timeout=0.01) # re-acquire a held lock is a no-op lock.acquire() lock.release() # release the lock lock.release() # second release does nothing def test_rlock_acquire_release_count(tmpfile): lock = portalocker.RLock(tmpfile) # Twice acquire h = lock.acquire() assert not h.closed lock.acquire() assert not h.closed # Two release lock.release() assert not h.closed lock.release() assert h.closed def test_rlock_acquire_release(tmpfile): lock = portalocker.RLock(tmpfile) lock2 = portalocker.RLock(tmpfile, fail_when_locked=False) lock.acquire() # acquire lock when nobody is using it with pytest.raises(portalocker.LockException): # another party should not be able to acquire the lock lock2.acquire(timeout=0.01) # Now acquire again lock.acquire() lock.release() # release the lock lock.release() # second release does nothing def test_release_unacquired(tmpfile): with pytest.raises(portalocker.LockException): portalocker.RLock(tmpfile).release() def test_exlusive(tmpfile): with open(tmpfile, 'w') as fh: fh.write('spam and eggs') fh = open(tmpfile, 'r') portalocker.lock(fh, portalocker.LOCK_EX | portalocker.LOCK_NB) # Make sure we can't read the locked file with pytest.raises(portalocker.LockException): with open(tmpfile, 'r') as fh2: portalocker.lock(fh2, portalocker.LOCK_EX | portalocker.LOCK_NB) fh2.read() # Make sure we can't write the locked file with pytest.raises(portalocker.LockException): with open(tmpfile, 'w+') as fh2: portalocker.lock(fh2, portalocker.LOCK_EX | portalocker.LOCK_NB) fh2.write('surprise and fear') # Make sure we can explicitly unlock the file portalocker.unlock(fh) fh.close() def test_shared(tmpfile): with open(tmpfile, 'w') as fh: fh.write('spam and eggs') f = open(tmpfile, 'r') portalocker.lock(f, portalocker.LOCK_SH | portalocker.LOCK_NB) # Make sure we can read the locked file with open(tmpfile, 'r') as fh2: portalocker.lock(fh2, portalocker.LOCK_SH | portalocker.LOCK_NB) assert fh2.read() == 'spam and eggs' # Make sure we can't write the locked file with pytest.raises(portalocker.LockException): with open(tmpfile, 'w+') as fh2: portalocker.lock(fh2, portalocker.LOCK_EX | portalocker.LOCK_NB) fh2.write('surprise and fear') # Make sure we can explicitly unlock the file portalocker.unlock(f) f.close() portalocker-1.5.1/setup.cfg0000644000076500000240000000034513521711277015747 0ustar rickstaff00000000000000[metadata] description-file = README.rst [build_sphinx] source-dir = docs/ build-dir = docs/_build all_files = 1 [upload_sphinx] upload-dir = docs/_build/html [bdist_wheel] universal = 1 [egg_info] tag_build = tag_date = 0 portalocker-1.5.1/setup.py0000644000076500000240000001100413521706531015627 0ustar rickstaff00000000000000from __future__ import print_function import re import os import sys import setuptools from setuptools.command.test import test as TestCommand from distutils.version import StrictVersion from setuptools import __version__ as setuptools_version if StrictVersion(setuptools_version) < StrictVersion('38.3.0'): raise SystemExit( 'Your `setuptools` version is old. ' 'Please upgrade setuptools by running `pip install -U setuptools` ' 'and try again.' ) # To prevent importing about and thereby breaking the coverage info we use this # exec hack about = {} with open('portalocker/__about__.py') as fp: exec(fp.read(), about) tests_require = [ 'flake8>=3.5.0', 'pytest>=3.4.0', 'pytest-cache>=1.0', 'pytest-cov>=2.5.1', 'pytest-flakes>=2.0.0', 'pytest-pep8>=1.0.6', 'sphinx>=1.7.1', ] class PyTest(TestCommand): user_options = [('pytest-args=', 'a', "Arguments to pass to pytest")] def initialize_options(self): TestCommand.initialize_options(self) self.pytest_args = '' def run_tests(self): import shlex # import here, cause outside the eggs aren't loaded import pytest errno = pytest.main(shlex.split(self.pytest_args)) sys.exit(errno) class Combine(setuptools.Command): description = 'Build single combined portalocker file' relative_import_re = re.compile(r'^from \. import (?P.+)$', re.MULTILINE) user_options = [ ('output-file=', 'o', 'Path to the combined output file'), ] def initialize_options(self): self.output_file = os.path.join( 'dist', '%(package_name)s_%(version)s.py' % dict( package_name=about['__package_name__'], version=about['__version__'].replace('.', '-'), )) def finalize_options(self): pass def run(self): dirname = os.path.dirname(self.output_file) if dirname and not os.path.isdir(dirname): os.makedirs(dirname) output = open(self.output_file, 'w') print("'''", file=output) with open('README.rst') as fh: output.write(fh.read().rstrip()) print('', file=output) print('', file=output) with open('LICENSE') as fh: output.write(fh.read().rstrip()) print('', file=output) print("'''", file=output) names = set() lines = [] for line in open('portalocker/__init__.py'): match = self.relative_import_re.match(line) if match: names.add(match.group('name')) with open('portalocker/%(name)s.py' % match.groupdict()) as fh: line = fh.read() line = self.relative_import_re.sub('', line) lines.append(line) import_attributes = re.compile(r'\b(%s)\.' % '|'.join(names)) for line in lines[:]: line = import_attributes.sub('', line) output.write(line) print('Wrote combined file to %r' % self.output_file) if __name__ == '__main__': setuptools.setup( name=about['__package_name__'], version=about['__version__'], description=about['__description__'], long_description=open('README.rst').read(), classifiers=[ 'Intended Audience :: Developers', 'Programming Language :: Python', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: Implementation :: CPython', 'Programming Language :: Python :: Implementation :: PyPy', ], keywords='locking, locks, with statement, windows, linux, unix', author=about['__author__'], author_email=about['__email__'], url=about['__url__'], license='PSF', packages=setuptools.find_packages(exclude=['ez_setup', 'examples', 'portalocker_tests']), # zip_safe=False, platforms=['any'], cmdclass={ 'combine': Combine, 'test': PyTest, }, install_requires=[ 'pypiwin32; platform_system == "Windows"', ], tests_require=tests_require, extras_require=dict( docs=[ 'sphinx>=1.7.1', ], tests=tests_require, ), )