os-brick-2.3.0/0000775000175100017510000000000013230233405013244 5ustar zuulzuul00000000000000os-brick-2.3.0/PKG-INFO0000664000175100017510000000523413230233405014345 0ustar zuulzuul00000000000000Metadata-Version: 1.1 Name: os-brick Version: 2.3.0 Summary: OpenStack Cinder brick library for managing local volume attaches Home-page: https://docs.openstack.org/os-brick/ Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description-Content-Type: UNKNOWN Description: ======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on =============================== brick =============================== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ * License: Apache License, Version 2.0 * Source: https://git.openstack.org/cgit/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 os-brick-2.3.0/CONTRIBUTING.rst0000666000175100017510000000103513230233223015704 0ustar zuulzuul00000000000000If you would like to contribute to the development of OpenStack, you must follow the steps in this page: https://docs.openstack.org/infra/manual/developers.html Once those steps have been completed, changes to OpenStack should be submitted for review via the Gerrit tool, following the workflow documented at: https://docs.openstack.org/infra/manual/developers.html#development-workflow Pull requests submitted through GitHub will be ignored. Bugs should be filed on Launchpad, not GitHub: https://bugs.launchpad.net/os-brick os-brick-2.3.0/LICENSE0000666000175100017510000002363713230233223014264 0ustar zuulzuul00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. os-brick-2.3.0/releasenotes/0000775000175100017510000000000013230233405015735 5ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/notes/0000775000175100017510000000000013230233405017065 5ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/notes/bug-1722432-2408dab55c903c5b.yaml0000666000175100017510000000046313230233223023560 0ustar zuulzuul00000000000000--- fixes: - | [`bug 1722432 `_] Changes the supported_transports to support tcp transport. With this change, we can define an custom iface with tcp transport to limit the storage traffic only be transimitted via storage NIC we specified. os-brick-2.3.0/releasenotes/notes/refactor_iscsi_disconnect-557f4173bc1ae4ed.yaml0000666000175100017510000000077313230233223027464 0ustar zuulzuul00000000000000--- features: - | New parameters on `disconnect_volume` named `force` and `ignore_errors` can be used to let OS-Brick know that data loss is secondary to leaving a clean system with no leftover devices. If `force` is not set, or set to False, preventing data loss will take priority. Currently only iSCSI implements these new parameters. fixes: - | iSCSI disconnect refactoring improves reliability, speed, and thoroughness on leaving a cleaner system after disconnection. os-brick-2.3.0/releasenotes/notes/iscsi_manual_scan_support-d64a1c3c8e1986b4.yaml0000666000175100017510000000062413230233223027443 0ustar zuulzuul00000000000000--- features: - | Support for setting the scan mode on the Open-iSCSI initiator. If installed iscsiadm supports this feature OS-Brick will set all it's new sessions to manual scan. fixes: - | On systems with scan mode support on open-iSCSI we'll no longer see unwanted devices polluting our system due to the automatic initiator scan or to AEN/AER messages from the backend. os-brick-2.3.0/releasenotes/notes/local-attach-in-rbd-connector-c06347fb164b084a.yaml0000666000175100017510000000021513230233223027572 0ustar zuulzuul00000000000000--- features: - Local attach feature in RBD connector. We use RBD kernel module to attach and detach volumes locally without Nova. os-brick-2.3.0/releasenotes/notes/add_custom_keyring_for_rbd_connection-eccbaae9ee5f3491.yaml0000666000175100017510000000022613230233223032257 0ustar zuulzuul00000000000000--- fixes: - Add support to use custom Ceph keyring files (previously os-brick hardcoded using /etc/ceph/.client..keyring file). os-brick-2.3.0/releasenotes/notes/start-using-reno-23e8d5f1a30851a1.yaml0000666000175100017510000000007113230233223025313 0ustar zuulzuul00000000000000--- other: - Start using reno to manage release notes. ././@LongLink0000000000000000000000000000015200000000000011213 Lustar 00000000000000os-brick-2.3.0/releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd.yamlos-brick-2.3.0/releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd0000666000175100017510000000056213230233223033231 0ustar zuulzuul00000000000000--- deprecations: - | The direct use of the encryption provider classes such as os_brick.encryptors.luks.LuksEncryptor continues to be deprecated and will now be blocked in the Queens release of os-brick. The use of out of tree encryption provider classes also continues to be deprecated and will also be blocked in the Queens release of os-brick. os-brick-2.3.0/releasenotes/notes/veritas-hyperscale-connector-fe56cec68b1947cd.yaml0000666000175100017510000000007313230233223030145 0ustar zuulzuul00000000000000--- features: - Add Veritas HyperScale connector support os-brick-2.3.0/releasenotes/notes/refactor_iscsi_connect-dfbb24305a954783.yaml0000666000175100017510000000014713230233223026621 0ustar zuulzuul00000000000000--- fixes: - | iSCSI connect mechanism refactoring to be faster, more robust, more reliable. os-brick-2.3.0/releasenotes/notes/add-windows-smbfs-d86edaa003130a31.yaml0000666000175100017510000000006713230233223025472 0ustar zuulzuul00000000000000--- features: - Add Windows SMBFS connector support. os-brick-2.3.0/releasenotes/notes/vmware-vmdk-connector-19e6999e6cae43cd.yaml0000666000175100017510000000021013230233223026511 0ustar zuulzuul00000000000000--- features: - Added initiator connector 'VmdkConnector' to support backup and restore of vmdk volumes by Cinder backup service. os-brick-2.3.0/releasenotes/notes/multipath-improvements-596c2c6eadfba6ea.yaml0000666000175100017510000000006313230233223027223 0ustar zuulzuul00000000000000--- fixes: - Improved multipath device handling. os-brick-2.3.0/releasenotes/notes/introduce-encryption-provider-constants-a7cd0ce58da2bae8.yaml0000666000175100017510000000121413230233223032505 0ustar zuulzuul00000000000000--- features: - | Encryption provider constants have been introduced detailing the supported encryption formats such as LUKs along with their associated in-tree provider implementations. These constants should now be used to identify an encryption provider implementation for a given encryption format. deprecations: - | The direct use of the encryption provider classes such as os_brick.encryptors.luks.LuksEncryptor is now deprecated and will be blocked in the Pike release of os-brick. The use of out of tree encryption provider classes is also deprecated and will be blocked in the Pike release of os-brick. os-brick-2.3.0/releasenotes/notes/remove-old-constants-20021f5b30bde890.yaml0000666000175100017510000000047413230233223026157 0ustar zuulzuul00000000000000--- upgrade: - | The location for connector constants was moved in the 1.6.0 release, but their old location was kept for backwards compatibility. These legacy constants are now being removed and any out of tree code should be updated to use the latest location (os_brick.initiator.CONSTANT_NAME). os-brick-2.3.0/releasenotes/notes/add-windows-fibre-channel-030c095c149da321.yaml0000666000175100017510000000007713230233223026725 0ustar zuulzuul00000000000000--- features: - Add Windows Fibre Channel connector support. os-brick-2.3.0/releasenotes/notes/add-vstorage-protocol-b536f4e21d764801.yaml0000666000175100017510000000011413230233223026241 0ustar zuulzuul00000000000000--- features: - Added vStorage protocol support for RemoteFS connections. os-brick-2.3.0/releasenotes/notes/add-windows-iscsi-15d6b1392695f978.yaml0000666000175100017510000000006713230233223025324 0ustar zuulzuul00000000000000--- features: - Add Windows iSCSI connector support. os-brick-2.3.0/releasenotes/source/0000775000175100017510000000000013230233405017235 5ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/source/newton.rst0000666000175100017510000000021613230233223021300 0ustar zuulzuul00000000000000============================= Newton Series Release Notes ============================= .. release-notes:: :branch: origin/stable/newton os-brick-2.3.0/releasenotes/source/index.rst0000666000175100017510000000023413230233223021075 0ustar zuulzuul00000000000000======================== os-brick Release Notes ======================== .. toctree:: :maxdepth: 1 unreleased pike ocata newton mitaka os-brick-2.3.0/releasenotes/source/mitaka.rst0000666000175100017510000000023213230233223021232 0ustar zuulzuul00000000000000=================================== Mitaka Series Release Notes =================================== .. release-notes:: :branch: origin/stable/mitaka os-brick-2.3.0/releasenotes/source/ocata.rst0000666000175100017510000000021413230233223021053 0ustar zuulzuul00000000000000============================= Ocata Series Release Notes ============================= .. release-notes:: :branch: origin/stable/ocata os-brick-2.3.0/releasenotes/source/_templates/0000775000175100017510000000000013230233405021372 5ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/source/_templates/.placeholder0000666000175100017510000000000013230233223023643 0ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/source/conf.py0000666000175100017510000000354013230233223020536 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # # os-brick Release Notes documentation build configuration file # # Refer to the Sphinx documentation for advice on configuring this file: # # http://www.sphinx-doc.org/en/stable/config.html # -- General configuration ------------------------------------------------ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'reno.sphinxext', 'openstackdocstheme', ] # The suffix of source filenames. source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = u'OS-Brick Release Notes' copyright = u'2015, Cinder Developers' # Release notes are unversioned, so we don't need to set version and release version = '' release = '' # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'openstackdocs' # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. html_last_updated_fmt = '%Y-%m-%d %H:%M' # -- Options for openstackdocstheme ------------------------------------------- repository_name = 'openstack/os-brick' bug_project = 'os-brick' bug_tag = '' os-brick-2.3.0/releasenotes/source/pike.rst0000666000175100017510000000021713230233223020717 0ustar zuulzuul00000000000000=================================== Pike Series Release Notes =================================== .. release-notes:: :branch: stable/pike os-brick-2.3.0/releasenotes/source/_static/0000775000175100017510000000000013230233405020663 5ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/source/_static/.placeholder0000666000175100017510000000000013230233223023134 0ustar zuulzuul00000000000000os-brick-2.3.0/releasenotes/source/unreleased.rst0000666000175100017510000000016013230233223022113 0ustar zuulzuul00000000000000============================== Current Series Release Notes ============================== .. release-notes:: os-brick-2.3.0/os_brick.egg-info/0000775000175100017510000000000013230233405016531 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick.egg-info/PKG-INFO0000664000175100017510000000523413230233404017631 0ustar zuulzuul00000000000000Metadata-Version: 1.1 Name: os-brick Version: 2.3.0 Summary: OpenStack Cinder brick library for managing local volume attaches Home-page: https://docs.openstack.org/os-brick/ Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description-Content-Type: UNKNOWN Description: ======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on =============================== brick =============================== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ * License: Apache License, Version 2.0 * Source: https://git.openstack.org/cgit/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 os-brick-2.3.0/os_brick.egg-info/pbr.json0000664000175100017510000000005613230233404020207 0ustar zuulzuul00000000000000{"git_version": "7dd2076", "is_release": true}os-brick-2.3.0/os_brick.egg-info/requires.txt0000664000175100017510000000043013230233404021125 0ustar zuulzuul00000000000000pbr!=2.1.0,>=2.0.0 Babel!=2.4.0,>=2.3.4 eventlet!=0.18.3,!=0.20.1,<0.21.0,>=0.18.2 oslo.concurrency>=3.25.0 oslo.log>=3.36.0 oslo.i18n>=3.15.3 oslo.privsep>=1.23.0 oslo.service!=1.28.1,>=1.24.0 oslo.utils>=3.33.0 requests>=2.14.2 retrying!=1.3.0,>=1.2.3 six>=1.10.0 os-win>=2.0.0 os-brick-2.3.0/os_brick.egg-info/dependency_links.txt0000664000175100017510000000000113230233404022576 0ustar zuulzuul00000000000000 os-brick-2.3.0/os_brick.egg-info/SOURCES.txt0000664000175100017510000001473313230233405020425 0ustar zuulzuul00000000000000.coveragerc .mailmap .stestr.conf AUTHORS CONTRIBUTING.rst ChangeLog HACKING.rst LICENSE README.rst babel.cfg bindep.txt pylintrc requirements.txt setup.cfg setup.py test-requirements.txt tox.ini doc/source/conf.py doc/source/index.rst doc/source/install/index.rst doc/source/reference/index.rst doc/source/reference/os_brick/exception.rst doc/source/reference/os_brick/index.rst doc/source/reference/os_brick/initiator/connector.rst doc/source/reference/os_brick/initiator/index.rst doc/source/user/tutorial.rst etc/os-brick/rootwrap.d/os-brick.filters os_brick/__init__.py os_brick/exception.py os_brick/executor.py os_brick/i18n.py os_brick/utils.py os_brick/version.py os_brick.egg-info/PKG-INFO os_brick.egg-info/SOURCES.txt os_brick.egg-info/dependency_links.txt os_brick.egg-info/not-zip-safe os_brick.egg-info/pbr.json os_brick.egg-info/requires.txt os_brick.egg-info/top_level.txt os_brick/encryptors/__init__.py os_brick/encryptors/base.py os_brick/encryptors/cryptsetup.py os_brick/encryptors/luks.py os_brick/encryptors/nop.py os_brick/initiator/__init__.py os_brick/initiator/connector.py os_brick/initiator/host_driver.py os_brick/initiator/initiator_connector.py os_brick/initiator/linuxfc.py os_brick/initiator/linuxrbd.py os_brick/initiator/linuxscsi.py os_brick/initiator/linuxsheepdog.py os_brick/initiator/connectors/__init__.py os_brick/initiator/connectors/aoe.py os_brick/initiator/connectors/base.py os_brick/initiator/connectors/base_iscsi.py os_brick/initiator/connectors/disco.py os_brick/initiator/connectors/drbd.py os_brick/initiator/connectors/fake.py os_brick/initiator/connectors/fibre_channel.py os_brick/initiator/connectors/fibre_channel_ppc64.py os_brick/initiator/connectors/fibre_channel_s390x.py os_brick/initiator/connectors/gpfs.py os_brick/initiator/connectors/hgst.py os_brick/initiator/connectors/huawei.py os_brick/initiator/connectors/iscsi.py os_brick/initiator/connectors/local.py os_brick/initiator/connectors/nvme.py os_brick/initiator/connectors/rbd.py os_brick/initiator/connectors/remotefs.py os_brick/initiator/connectors/scaleio.py os_brick/initiator/connectors/sheepdog.py os_brick/initiator/connectors/storpool.py os_brick/initiator/connectors/vmware.py os_brick/initiator/connectors/vrtshyperscale.py os_brick/initiator/windows/__init__.py os_brick/initiator/windows/base.py os_brick/initiator/windows/fibre_channel.py os_brick/initiator/windows/iscsi.py os_brick/initiator/windows/smbfs.py os_brick/local_dev/__init__.py os_brick/local_dev/lvm.py os_brick/privileged/__init__.py os_brick/privileged/rootwrap.py os_brick/remotefs/__init__.py os_brick/remotefs/remotefs.py os_brick/remotefs/windows_remotefs.py os_brick/tests/__init__.py os_brick/tests/base.py os_brick/tests/test_brick.py os_brick/tests/test_exception.py os_brick/tests/test_executor.py os_brick/tests/test_utils.py os_brick/tests/encryptors/__init__.py os_brick/tests/encryptors/test_base.py os_brick/tests/encryptors/test_cryptsetup.py os_brick/tests/encryptors/test_luks.py os_brick/tests/encryptors/test_nop.py os_brick/tests/initiator/__init__.py os_brick/tests/initiator/test_connector.py os_brick/tests/initiator/test_host_driver.py os_brick/tests/initiator/test_linuxfc.py os_brick/tests/initiator/test_linuxrbd.py os_brick/tests/initiator/test_linuxscsi.py os_brick/tests/initiator/test_linuxsheepdog.py os_brick/tests/initiator/connectors/__init__.py os_brick/tests/initiator/connectors/test_aoe.py os_brick/tests/initiator/connectors/test_base_iscsi.py os_brick/tests/initiator/connectors/test_disco.py os_brick/tests/initiator/connectors/test_drbd.py os_brick/tests/initiator/connectors/test_fibre_channel.py os_brick/tests/initiator/connectors/test_fibre_channel_ppc64.py os_brick/tests/initiator/connectors/test_fibre_channel_s390x.py os_brick/tests/initiator/connectors/test_gpfs.py os_brick/tests/initiator/connectors/test_hgst.py os_brick/tests/initiator/connectors/test_huawei.py os_brick/tests/initiator/connectors/test_iscsi.py os_brick/tests/initiator/connectors/test_iser.py os_brick/tests/initiator/connectors/test_local.py os_brick/tests/initiator/connectors/test_nvme.py os_brick/tests/initiator/connectors/test_rbd.py os_brick/tests/initiator/connectors/test_remotefs.py os_brick/tests/initiator/connectors/test_scaleio.py os_brick/tests/initiator/connectors/test_sheepdog.py os_brick/tests/initiator/connectors/test_storpool.py os_brick/tests/initiator/connectors/test_vmware.py os_brick/tests/initiator/connectors/test_vrtshyperscale.py os_brick/tests/local_dev/__init__.py os_brick/tests/local_dev/fake_lvm.py os_brick/tests/local_dev/test_brick_lvm.py os_brick/tests/privileged/__init__.py os_brick/tests/privileged/test_rootwrap.py os_brick/tests/remotefs/__init__.py os_brick/tests/remotefs/test_remotefs.py os_brick/tests/remotefs/test_windows_remotefs.py os_brick/tests/windows/__init__.py os_brick/tests/windows/fake_win_conn.py os_brick/tests/windows/test_base.py os_brick/tests/windows/test_base_connector.py os_brick/tests/windows/test_factory.py os_brick/tests/windows/test_fibre_channel.py os_brick/tests/windows/test_iscsi.py os_brick/tests/windows/test_smbfs.py releasenotes/notes/add-vstorage-protocol-b536f4e21d764801.yaml releasenotes/notes/add-windows-fibre-channel-030c095c149da321.yaml releasenotes/notes/add-windows-iscsi-15d6b1392695f978.yaml releasenotes/notes/add-windows-smbfs-d86edaa003130a31.yaml releasenotes/notes/add_custom_keyring_for_rbd_connection-eccbaae9ee5f3491.yaml releasenotes/notes/bug-1722432-2408dab55c903c5b.yaml releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd.yaml releasenotes/notes/introduce-encryption-provider-constants-a7cd0ce58da2bae8.yaml releasenotes/notes/iscsi_manual_scan_support-d64a1c3c8e1986b4.yaml releasenotes/notes/local-attach-in-rbd-connector-c06347fb164b084a.yaml releasenotes/notes/multipath-improvements-596c2c6eadfba6ea.yaml releasenotes/notes/refactor_iscsi_connect-dfbb24305a954783.yaml releasenotes/notes/refactor_iscsi_disconnect-557f4173bc1ae4ed.yaml releasenotes/notes/remove-old-constants-20021f5b30bde890.yaml releasenotes/notes/start-using-reno-23e8d5f1a30851a1.yaml releasenotes/notes/veritas-hyperscale-connector-fe56cec68b1947cd.yaml releasenotes/notes/vmware-vmdk-connector-19e6999e6cae43cd.yaml releasenotes/source/conf.py releasenotes/source/index.rst releasenotes/source/mitaka.rst releasenotes/source/newton.rst releasenotes/source/ocata.rst releasenotes/source/pike.rst releasenotes/source/unreleased.rst releasenotes/source/_static/.placeholder releasenotes/source/_templates/.placeholder tools/fast8.sh tools/lintstack.py tools/lintstack.shos-brick-2.3.0/os_brick.egg-info/top_level.txt0000664000175100017510000000001113230233404021252 0ustar zuulzuul00000000000000os_brick os-brick-2.3.0/os_brick.egg-info/not-zip-safe0000664000175100017510000000000113230233364020763 0ustar zuulzuul00000000000000 os-brick-2.3.0/setup.py0000666000175100017510000000200613230233223014754 0ustar zuulzuul00000000000000# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT import setuptools # In python < 2.7.4, a lazy loading of package `pbr` will break # setuptools if some other modules registered functions in `atexit`. # solution from: http://bugs.python.org/issue15881#msg170215 try: import multiprocessing # noqa except ImportError: pass setuptools.setup( setup_requires=['pbr>=2.0.0'], pbr=True) os-brick-2.3.0/.stestr.conf0000666000175100017510000000010113230233223015505 0ustar zuulzuul00000000000000[DEFAULT] test_path=${OS_TEST_PATH:-./os_brick/tests} top_dir=./ os-brick-2.3.0/etc/0000775000175100017510000000000013230233405014017 5ustar zuulzuul00000000000000os-brick-2.3.0/etc/os-brick/0000775000175100017510000000000013230233405015530 5ustar zuulzuul00000000000000os-brick-2.3.0/etc/os-brick/rootwrap.d/0000775000175100017510000000000013230233405017627 5ustar zuulzuul00000000000000os-brick-2.3.0/etc/os-brick/rootwrap.d/os-brick.filters0000666000175100017510000000072713230233223022740 0ustar zuulzuul00000000000000# os-brick command filters # This file should be owned by (and only-writeable by) the root user [Filters] # privileged/__init__.py: priv_context.PrivContext(default) # This line ties the superuser privs with the config files, context name, # and (implicitly) the actual python code invoked. privsep-rootwrap: RegExpFilter, privsep-helper, root, privsep-helper, --config-file, /etc/(?!\.\.).*, --privsep_context, os_brick.privileged.default, --privsep_sock_path, /tmp/.* os-brick-2.3.0/doc/0000775000175100017510000000000013230233405014011 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/0000775000175100017510000000000013230233405015311 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/index.rst0000666000175100017510000000055613230233223017160 0ustar zuulzuul00000000000000======== os-brick ======== `os-brick` is a Python package containing classes that help with volume discovery and removal from a host. Installation Guide ------------------ .. toctree:: :maxdepth: 2 install/index Usage Guide ----------- .. toctree:: :maxdepth: 2 user/tutorial Reference --------- .. toctree:: :maxdepth: 2 reference/index os-brick-2.3.0/doc/source/user/0000775000175100017510000000000013230233405016267 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/user/tutorial.rst0000666000175100017510000000221013230233223020657 0ustar zuulzuul00000000000000======== Tutorial ======== This tutorial is intended as an introduction to working with **os-brick**. Prerequisites ------------- Before we start, make sure that you have the **os-brick** distribution :doc:`installed `. In the Python shell, the following should run without raising an exception: .. code-block:: bash >>> import os_brick Fetch all of the initiator information from the host ---------------------------------------------------- An example of how to collect the initiator information that is needed to export a volume to this host. .. code-block:: python from os_brick.initiator import connector # what helper do you want to use to get root access? root_helper = "sudo" # The ip address of the host you are running on my_ip = "192.168.1.1" # Do you want to support multipath connections? multipath = True # Do you want to enforce that multipath daemon is running? enforce_multipath = False initiator = connector.get_connector_properties(root_helper, my_ip, multipath, enforce_multipath) os-brick-2.3.0/doc/source/conf.py0000777000175100017510000000356113230233223016620 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', 'reno.sphinxext', 'openstackdocstheme', ] # The master toctree document. master_doc = 'index' # General information about the project. project = u'os-brick' copyright = u'2015, OpenStack Foundation' # If true, '()' will be appended to :func: etc. cross-reference text. add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). add_module_names = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. html_theme = 'openstackdocs' # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. html_last_updated_fmt = '%Y-%m-%d %H:%M' # -- Options for openstackdocstheme ------------------------------------------- repository_name = 'openstack/os-brick' bug_project = 'os-brick' bug_tag = '' os-brick-2.3.0/doc/source/install/0000775000175100017510000000000013230233405016757 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/install/index.rst0000666000175100017510000000056713230233223020630 0ustar zuulzuul00000000000000============ Installation ============ At the command line: .. code-block:: shell $ pip install os-brick Or, if you have virtualenvwrapper installed: .. code-block:: shell $ mkvirtualenv os-brick $ pip install os-brick Or, from source: .. code-block:: shell $ git clone https://github.com/openstack/os-brick $ cd os-brick $ python setup.py install os-brick-2.3.0/doc/source/reference/0000775000175100017510000000000013230233405017247 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/reference/index.rst0000666000175100017510000000035313230233223021111 0ustar zuulzuul00000000000000API Documentation ================= The **os-brick** package provides the ability to collect host initiator information as well as discovery volumes and removal of volumes from a host. .. toctree:: :maxdepth: 2 os_brick/index os-brick-2.3.0/doc/source/reference/os_brick/0000775000175100017510000000000013230233405021042 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/reference/os_brick/exception.rst0000666000175100017510000000112313230233223023567 0ustar zuulzuul00000000000000:mod:`exception` -- Exceptions ============================== .. automodule:: os_brick.exception :synopsis: Exceptions generated by os-brick .. autoclass:: os_brick.exception.BrickException .. autoclass:: os_brick.exception.NotFound .. autoclass:: os_brick.exception.Invalid .. autoclass:: os_brick.exception.InvalidParameterValue .. autoclass:: os_brick.exception.NoFibreChannelHostsFound .. autoclass:: os_brick.exception.NoFibreChannelVolumeDeviceFound .. autoclass:: os_brick.exception.VolumeDeviceNotFound .. autoclass:: os_brick.exception.ProtocolNotSupported os-brick-2.3.0/doc/source/reference/os_brick/index.rst0000666000175100017510000000034413230233223022704 0ustar zuulzuul00000000000000:mod:`os_brick` -- OpenStack Brick library ========================================== .. automodule:: os_brick :synopsis: OpenStack Brick library Sub-modules: .. toctree:: :maxdepth: 2 initiator/index exception os-brick-2.3.0/doc/source/reference/os_brick/initiator/0000775000175100017510000000000013230233405023044 5ustar zuulzuul00000000000000os-brick-2.3.0/doc/source/reference/os_brick/initiator/index.rst0000666000175100017510000000027213230233223024706 0ustar zuulzuul00000000000000:mod:`initiator` -- Initiator ============================= .. automodule:: os_brick.initiator :synopsis: Initiator module Sub-modules: .. toctree:: :maxdepth: 2 connector os-brick-2.3.0/doc/source/reference/os_brick/initiator/connector.rst0000666000175100017510000000171013230233223025567 0ustar zuulzuul00000000000000:mod:`connector` -- Connector ============================= .. automodule:: os_brick.initiator.connector :synopsis: Connector module for os-brick .. autoclass:: os_brick.initiator.connector.InitiatorConnector .. automethod:: factory .. autoclass:: os_brick.initiator.connector.ISCSIConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.FibreChannelConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.AoEConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.LocalConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.HuaweiStorHyperConnector .. automethod:: connect_volume .. automethod:: disconnect_volume os-brick-2.3.0/ChangeLog0000664000175100017510000003743113230233404015025 0ustar zuulzuul00000000000000CHANGES ======= 2.3.0 ----- * adding iSER connector for PPC64 * adding VERITAS\_HYPERSCALE connector for PPC64 * Updated from global requirements * adding VZSTORAGE connector for PPC64 * Updated from global requirements * Update supported transports for iscsi connector * Remove the unnecessary pv\_list assign during LVM object init 2.2.0 ----- * Remove requirement on oslo.serialization * Cleanup test-requirements * Updated from global requirements * Avoid tox\_install.sh for constraints support * Recover node.startup values after discovering * Add the StorPool brick connector 2.1.1 ----- * set vg\_thin\_pool\_size to float type * Fix a typographical error in a release notes entry 2.1.0 ----- * Make close on cryptsetup volumes idempotent * Make close on luks volumes idempotent * Remove setting of version/release from releasenotes * Adding NVMEoF for initiator CLI * Updated from global requirements * Fixing FC scanning * Updated from global requirements * Always set ignoreskipactivation on snapshot creation 2.0.0 ----- * Enable Python hash randomization for tests * Remove legacy connector constants * Add .stestr.conf configuration * Fix \_remove\_scsi\_symlinks\_no\_links test * Protect against race within os.path.realpath * rescan fails for hba missing target wwn * Updated from global requirements * Updated from global requirements * Fix vmware migrate available volume bug * FC PPC64 device discovery issue * Add attribute 'name' to class RBDVolume * Updated from global requirements * Updated from global requirements * Fix iSCSI volume attachment over RDMA transport * doc: Restructure docs for doc-migration * doc: Remove cruft from conf.py * doc: Switch from oslosphinx to openstackdocstheme * Update reno for stable/pike * Fix ISCSIConnector.\_get\_potential\_volume\_paths logic * FC connector logs number of attempts incorrectly * Enable some off-by-default checks * Update and replace http with https for doc links * Get the right portal from output of iscsiadm command * Update and optimize documentation links * Updated from global requirements * Add client connect exception unit test for rbd 1.15.1 ------ * Don't obscure logs on iSCSI sendtargets failure * Return WWN in multipath\_id * Return symlinks for encrypted volumes 1.15.0 ------ * Revert "Don't use ignoreskipactivation for thin LVM" * Don't use ignoreskipactivation for thin LVM * Fix iSCSI cleanup fix on discovery backends * Fix manual scan for discovery type backends * Fix ceph incremental backup fail * Updated from global requirements 1.14.0 ------ * iSCSI multipath: improve logging on connect * Fix iSCSI cleanup issue when using discovery * Updated from global requirements * Add open-iscsi manual scan support * Refactor iSCSI connect * Fix slow test\_connect\_volume\_device\_not\_valid test * Add libssl to bindep * Refactor iSCSI disconnect * Updated from global requirements * Force LUN\_ID to an int 1.13.1 ------ * Fix supported connectors for Power platform * Removed invalid comments in tox.ini [flake8] * Stop ignoring H904 hacking rule in tox * Stop ignoring H405 hacking rule in tox * Stop ignoring E265 pycodestyle rule in tox * Stop ignoring E123 and E125 pycodestyle rules * Update hacking version to align with Cinder * Fixed the veritas connector path * Change code to be more Pythonic 1.13.0 ------ * Return correct device path from Veritas connector * Prevent rbd map again if it's already mapped * Check host device alive before multipath id discovery * Updated from global requirements * Changed way of providing RBD keyring from keyring\_path to client token * encryptors: Delay removal of legacy provider names * Change log level on \_get\_hba\_channel\_scsi\_target * Veritas os-brick connector should use privsep * Adding support for FibreChannelConnector for PPC64 * Updated from global requirements 1.12.0 ------ * Fixed generated temp file problem for RBD backend * Replace random uuid with fake uuid in unit tests * Updated from global requirements * Mask logging of connection info for iSCSI connector * Include identity information in rbd commands * os-brick connector for Veritas HyperScale * Move vzstorage related code out of RemoteFsClient * Updated from global requirements * RBD: consider a custom keyring in connection info * Add Ocata release notes page * Using assertIsNone(x) instead of assertEqual(None, x) * Remove log translations * Updated from global requirements * Fix iSCSI multipath rescan * Retry multipath flush when map is in use * Fix multipath flush when using friendly names * Fix unittest run on s390x host 1.11.0 ------ * Updated from global requirements * Encryptors: Fix compat with Nova encryptors for Ocata * Add Python 3.5 classifier and venv 1.10.0 ------ * Fix a wrong indentation * s390 FC device path fix for Ubuntu 1.9.0 ----- * Updated from global requirements * encryptors: Introduce encryption provider constants * Add debug to tox environment * Windows connectors: add device\_scan\_interval arg * Add curl to bindep * Removes unnecessary utf-8 encoding * Replace assertDictMatch with assertDictEqual * Add Constraints support and missing bindep.txt dependencies * Move castellan to test-reqs * Fix import method to follow community guideline * Remove the duplicate calls to rescan * Code cleanup in initiator/linuxfc.py * Updated from global requirements * linuxfc: log path when HBA not found * RBD: ensure temporary config gets deleted * os-brick: Add bindep support * Show team and repo badges on README * Updated from global requirements * Add developer docs url in README.rst(trivial) * encryptors: Workaround mangled passphrases * encryptors: Mock os\_brick.executor correctly * RBD: enclose ipv6 addresses in square brackets * Updated from global requirements * Mask passwords in utils.trace for func params 1.8.0 ----- * Updated from global requirements * Raise specific exception for an invalid protocol connector * Updated from global requirements * Multipath device keeps old size when extending volume * Updated from global requirements * Delete deprecated Hacking in tox.ini 1.7.0 ----- * Delete MANIFEST.in in os-brick * Drop py33 support * Windows remotefs: create mountpoints at the expected location * linuxrbd: remove obsolete comment on close() * Enable release notes translation * Detect if Fibre Channel support exists * Close connection to ceph after cinder bakcup * Updated from global requirements * Updated from global requirements * Replace 'assertTrue(a not in b)' with 'assertNotIn(a, b)' * s390x iscsi support enablement * Docstrings should not start with a space * Use assertEqual() instead of assertDictEqual() * Stop calling multipath -r when attaching/detaching iSCSI volumes * DISCO: Log init message as debug * Change warning to info logging for connected volume rescans * standardize release note page ordering * Mock time.sleep for test\_lv\_deactivate\_timeout test * Update reno for stable/newton * Change assertTrue(isinstance()) with optimal assert * Remove self.\_\_dict\_\_ for formatting strings * Create connector aliases to the new connectors refactor * TrivialFix: Remove logging import unused 1.6.0 ----- * Fix cmd execution stderr, stdout unicode errors * Mask out passwords when tracing * RBD: Fix typo in rados timeout assignment * Fixes with customized ceph cluster name * Add retries to iSCSI connect\_volume * Add connector for GPFS volumes * Add missing %s in print message * Fix linuxrbd to work with Python 3 * Add tracing unit tests * Wrong param makes exception message throws inaccurate * Fix the typo in the file * Add connector for vmdk volumes * Fix iSCSI discovery with ISER transport * RemoteFsClient extend Executor * Add Windows Fibre Channel connector * Add Windows SMBFS connector * Fix FC multipath cleanup * Fix weak test\_vzstorage\_with\_mds\_list * Fix the mocking mess * Fix FC multipath rescan * Update the home-page info with the developer documentation * Splitting Out Connectors from connector.py * Remove race condition from lvextend 1.5.0 ----- * Updated from global requirements * Mock write and read operations to filesystem * Local attach feature in RBD connector * Remove useless info logging in check\_valid\_device * ScaleIO to get volume name from connection properties * Add ignore for . directories * Upgrade tox to 2.0 * Add trace facility * Fix string interpolation to delayed to be handled by the logging code * Replace assertEqual(None, \*) with assertIsNone in tests * Fix wrong path used in iscsi "multipath -l" * Updated from global requirements * Remove unused LOG to keep code clean * Fix multipath iSCSI encrypted volume attach failure * Updated from global requirements * release note for windows iSCSI * Add Windows iSCSI connector * Make code line length less than 79 characters * Updated from global requirements * Replace ip with portal to express more accurately * Fix argument order for assertEqual to (expected, observed) * Add fast8 to quickly test pep8 changes * Make RBDImageMetadata and RBDVolumeIOWrapper re-usable 1.4.0 ----- * Copy encryptors from Nova to os-brick * Disconnect multipath iscsi may logout session * Add support for processutils.execute * Updated from global requirements * Mock time.sleep in ISCSIConnectorTestCase * Updated from global requirements * Updated from global requirements * Updated from global requirements * Ensure that the base connector is platform independent * Updated from global requirements * os-brick refactor get\_connector\_properties * Handle exception case with only target\_portals * Retire ISERConnector from documentation * LVM: Create thin pool with 100%FREE * Fix coverage generation * Trivial rootwrap -> privsep replacement * Updated from global requirements * Updated from global requirements 1.3.0 ----- * LVM: Call supports\_thin\_provisioning as static * Add pylint tox env * Don't use oslo-incubator stuff * Update reno for stable/mitaka * Replace \_get\_multipath\_device\_name with \_discover\_mpath\_device * Fixes get\_all\_available\_volumes return value * Updated from global requirements * Fix Scality SOFS support * Actually run the RemoteFSClient unit tests * Mock time.sleep() in 3 unit tests 1.1.0 ----- * Fix setting the multipath\_id * Updated from global requirements * Add sheepdog support * Include multipath -ll output in failed to parse warning 1.0.0 ----- * Fix iSCSI Multipath * Add missing release notes * Lun id's > 255 should be converted to hex * Updated from global requirements * Fix output returned from get\_all\_available\_volumes * Raise exception in find\_multipath\_device * Updated from global requirements * Remove multipath -l logic from ISCSI connector * Add vzstorage protocol for remotefs connections * Add reno for release notes management * Fix get\_device\_size with newlines * Updated from global requirements 0.8.0 ----- * Add connector for ITRI DISCO cinder driver * os-brick add extend\_volume API * os-brick add cinder local\_dev lvm code * Revert "Use assertTrue/False instead of assertEqual(T/F)" * Fix another unit test failure * Use assertTrue/False instead of assertEqual(T/F) * Actually log the command used in \_run\_iscsiadm * Updated from global requirements * remove python 2.6 trove classifier 0.7.0 ----- * DRBD connector class * Updated from global requirements * Deprecated tox -downloadcache option removed * ScaleIO could connect wrong volume to VM * Allow RBDClient to be used from a with-statement * Updated from global requirements * Remove brackets from portal * Minor documentation fixes for the method parameters 0.6.0 ----- * Add requests to project requirements * Add quobyte protocol for remotefs connections * Correct a log message * Brick add param documentation to connectors * Updated from global requirements * Multipath Device Action Being Parsed as Name * Fix iopsLimit parameter in ScaleIO connector * Parse FCoE sysfs device paths * Add new Connector APIs for path validation * Updated from global requirements * Fix test\_connect\_volume when skip is bypassed * Fetch and return SCSI WWN * Update minimum tox version to 1.8 * Updated from global requirements * Wait for FC multipath devices to become writable * Check RBDConnector.disconnect\_volume device\_info argument * Updated from global requirements * Fix silent iSCSI login failures * Change os-brick to use ostestr * Updated from global requirements * Fix iSCSI multipath cleanup * Removed use of deprecated LOG.warn * Fix typo in vgc-cluster command in rootwrap file 0.5.0 ----- * Change ignore-errors to ignore\_errors * Updated from global requirements * Add fancy pypi version and download images * iSCSI fix misleading Log warning on connect fail * Fix missing value types for log message * Log a message when can’t find multipath device * Removed unused dependency: discover * Use 'device' instead of 'volume\_path' 0.4.0 ----- * Add support for --interface option in iscsiadm * FC Stop calling multipath command line * Updated from global requirements * Add rootwrap filters * Handle FC LUN IDs greater 255 correctly on s390x architectures * Fix incorrect comments in FibreChannelConnector * Adding CHAP discovery logic to os-brick * Updated from global requirements * Remove the iSCSI rescan during disconnect * Remotefs: add ScalityFS support * Updated from global requirements * Updated from global requirements * Change SCSI device removal backoff rate * Changed connector protocols to use constants * Updated from global requirements * Fix race in check and access of /dev/disk/by-path/ * Updated from global requirements 0.3.2 ----- * remotefs: add virtuozzo storage support * Perform port\_rescan on s390x platforms * FC discover existing devices for removal 0.3.1 ----- * Use pbr's automatically generated changelog 0.3.0 ----- * Updated from global requirements * Updated from global requirements * Update changelog to 0.3.0 being latest * Fix mock==1.1.0 break unit tests * Cleanup Python 3 changes * Prep for 0.2.1 release * Add connector driver for the ScaleIO cinder driver * Added ABCMeta class to the InitiatorConnector * Remove unused oslo incubator files * update os-brick to pass python3 tests * Updated from global requirements * FC Eliminate the need to return devices list * Switch to oslo.service * Add RBD connector * Add HGST Solutions connector * Support host type specific block volume attachment * Updated from global requirements * optimize multipath call to identify IQN * Updated from global requirements * Trivial exception parameter name fix for Huawei * Fix connecting unnecessary iSCSI sessions issue * Fix disconnecting necessary iSCSI sessions issue * Add retry to iSCSI delete * Updated from global requirements * Add missing connectors to factory test * Fix local connector test case inheritance 0.2.0 ----- * Allow overriding the host field * Assign the platform after declaration * Added a unit test for masking iscsiadm passwords * Preparing for the 0.1.1 release * ISCSI be careful parsing iscsiadm output * Updated from global requirements * Drop use of 'oslo' namespace package 0.1.0 ----- * Update README to work with release tools * Brick: Fix race in removing iSCSI device * Update os-brick requirements * Mask passwords with iscsiadm commands * Sync latest \_i18n module for os\_brick * Use oslo\_log instead of openstack.common.log * Sync loopingcall from oslo-incubator for os-brick * Fix wrong command for \_rescan\_multipath * Fix multipath device discovery when UFN is enabled * Use six.text\_type instead of unicode * Fix missing translations for log messages * Remove error messages from multipath command output before parsing * Remove mocks after each unit test finished * Correct project name in .gitreview * Adjust os-brick to support FCP on System z systems * Use target\_portals/iqns/luns for alternative target information * Fix comments style according to Hacking rules * Update the documentation for os-brick * Failover to alternative iSCSI portals on login failure * Remove some unused exceptions from Cinder * Brick os-brick up to par with cinder brick * renamed the project to os-brick * Created the Brick library from Cinder os-brick-2.3.0/setup.cfg0000666000175100017510000000243213230233405015070 0ustar zuulzuul00000000000000[metadata] name = os-brick summary = OpenStack Cinder brick library for managing local volume attaches description-file = README.rst author = OpenStack author-email = openstack-dev@lists.openstack.org home-page = https://docs.openstack.org/os-brick/ classifier = Environment :: OpenStack Intended Audience :: Information Technology Intended Audience :: System Administrators License :: OSI Approved :: Apache Software License Operating System :: POSIX :: Linux Programming Language :: Python Programming Language :: Python :: 2 Programming Language :: Python :: 2.7 Programming Language :: Python :: 3 Programming Language :: Python :: 3.4 Programming Language :: Python :: 3.5 [global] setup-hooks = pbr.hooks.setup_hook [files] packages = os_brick data_files = etc/ = etc/* [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 [build_sphinx] warning-is-error = 1 source-dir = doc/source build-dir = doc/build all_files = 1 [upload_sphinx] upload-dir = doc/build/html [compile_catalog] directory = os_brick/locale domain = os_brick [update_catalog] domain = os_brick output_dir = os_brick/locale input_file = os_brick/locale/os-brick.pot [extract_messages] keywords = _ gettext ngettext l_ lazy_gettext mapping_file = babel.cfg output_file = os_brick/locale/os-brick.pot os-brick-2.3.0/bindep.txt0000666000175100017510000000075713230233223015257 0ustar zuulzuul00000000000000# This is a cross-platform list tracking distribution packages needed for # install and tests # see https://docs.openstack.org/infra/bindep/ for additional information. curl multipath-utils [platform:dpkg rpm] sg3-utils [platform:dpkg] sg3_utils [platform:rpm] libxml2-devel [platform:rpm] libxml2-dev [platform:dpkg] libxslt-devel [platform:rpm] libxslt1-dev [platform:dpkg] libssl-dev [platform:dpkg] openssl-devel [platform:rpm !platform:suse] libopenssl-devel [platform:suse !platform:rpm] os-brick-2.3.0/README.rst0000666000175100017510000000265413230233223014742 0ustar zuulzuul00000000000000======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on =============================== brick =============================== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.python.org/pypi/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ * License: Apache License, Version 2.0 * Source: https://git.openstack.org/cgit/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick os-brick-2.3.0/babel.cfg0000666000175100017510000000002113230233223014763 0ustar zuulzuul00000000000000[python: **.py] os-brick-2.3.0/.coveragerc0000666000175100017510000000013513230233223015364 0ustar zuulzuul00000000000000[run] branch = True source = os_brick omit = os_brick/tests/* [report] ignore_errors = True os-brick-2.3.0/tools/0000775000175100017510000000000013230233405014404 5ustar zuulzuul00000000000000os-brick-2.3.0/tools/lintstack.sh0000777000175100017510000000420613230233223016741 0ustar zuulzuul00000000000000#!/usr/bin/env bash # Copyright (c) 2012-2013, AT&T Labs, Yun Mao # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # Use lintstack.py to compare pylint errors. # We run pylint twice, once on HEAD, once on the code before the latest # commit for review. set -e TOOLS_DIR=$(cd $(dirname "$0") && pwd) # Get the current branch name. GITHEAD=`git rev-parse --abbrev-ref HEAD` if [[ "$GITHEAD" == "HEAD" ]]; then # In detached head mode, get revision number instead GITHEAD=`git rev-parse HEAD` echo "Currently we are at commit $GITHEAD" else echo "Currently we are at branch $GITHEAD" fi cp -f $TOOLS_DIR/lintstack.py $TOOLS_DIR/lintstack.head.py if git rev-parse HEAD^2 2>/dev/null; then # The HEAD is a Merge commit. Here, the patch to review is # HEAD^2, the master branch is at HEAD^1, and the patch was # written based on HEAD^2~1. PREV_COMMIT=`git rev-parse HEAD^2~1` git checkout HEAD~1 # The git merge is necessary for reviews with a series of patches. # If not, this is a no-op so won't hurt either. git merge $PREV_COMMIT else # The HEAD is not a merge commit. This won't happen on gerrit. # Most likely you are running against your own patch locally. # We assume the patch to examine is HEAD, and we compare it against # HEAD~1 git checkout HEAD~1 fi # First generate tools/pylint_exceptions from HEAD~1 $TOOLS_DIR/lintstack.head.py generate # Then use that as a reference to compare against HEAD git checkout $GITHEAD $TOOLS_DIR/lintstack.head.py echo "Check passed. FYI: the pylint exceptions are:" cat $TOOLS_DIR/pylint_exceptions os-brick-2.3.0/tools/fast8.sh0000777000175100017510000000045313230233223015772 0ustar zuulzuul00000000000000#!/bin/bash cd $(dirname "$0")/.. CHANGED=$(git diff --name-only HEAD~1 | tr '\n' ' ') # Skip files that don't exist # (have been git rm'd) CHECK="" for FILE in $CHANGED; do if [ -f "$FILE" ]; then CHECK="$CHECK $FILE" fi done diff -u --from-file /dev/null $CHECK | flake8 --diff os-brick-2.3.0/tools/lintstack.py0000777000175100017510000001530413230233223016760 0ustar zuulzuul00000000000000#!/usr/bin/env python # Copyright (c) 2013, AT&T Labs, Yun Mao # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """pylint error checking.""" from __future__ import print_function import json import re import sys from pylint import lint from pylint.reporters import text from six.moves import cStringIO as StringIO ignore_codes = [ # Note(maoy): E1103 is error code related to partial type inference "E1103" ] ignore_messages = [ # Note(fengqian): this message is the pattern of [E0611]. # It should be ignored because use six module to keep py3.X compatibility. "No name 'urllib' in module '_MovedItems'", # Note(xyang): these error messages are for the code [E1101]. # They should be ignored because 'sha256' and 'sha224' are functions in # 'hashlib'. "Module 'hashlib' has no 'sha256' member", "Module 'hashlib' has no 'sha224' member", ] ignore_modules = ["os_brick/tests/", "tools/lintstack.head.py"] KNOWN_PYLINT_EXCEPTIONS_FILE = "tools/pylint_exceptions" class LintOutput(object): _cached_filename = None _cached_content = None def __init__(self, filename, lineno, line_content, code, message, lintoutput): self.filename = filename self.lineno = lineno self.line_content = line_content self.code = code self.message = message self.lintoutput = lintoutput @classmethod def from_line(cls, line): m = re.search(r"(\S+):(\d+): \[(\S+)(, \S+)?] (.*)", line) matched = m.groups() filename, lineno, code, message = (matched[0], int(matched[1]), matched[2], matched[-1]) if cls._cached_filename != filename: with open(filename) as f: cls._cached_content = list(f.readlines()) cls._cached_filename = filename line_content = cls._cached_content[lineno - 1].rstrip() return cls(filename, lineno, line_content, code, message, line.rstrip()) @classmethod def from_msg_to_dict(cls, msg): """Converts pytlint message to a unique-error dictionary. From the output of pylint msg, to a dict, where each key is a unique error identifier, value is a list of LintOutput """ result = {} for line in msg.splitlines(): obj = cls.from_line(line) if obj.is_ignored(): continue key = obj.key() if key not in result: result[key] = [] result[key].append(obj) return result def is_ignored(self): if self.code in ignore_codes: return True if any(self.filename.startswith(name) for name in ignore_modules): return True return False def key(self): if self.code in ["E1101", "E1103"]: # These two types of errors are like Foo class has no member bar. # We discard the source code so that the error will be ignored # next time another Foo.bar is encountered. return self.message, "" return self.message, self.line_content.strip() def json(self): return json.dumps(self.__dict__) def review_str(self): return ("File %(filename)s\nLine %(lineno)d:%(line_content)s\n" "%(code)s: %(message)s" % {'filename': self.filename, 'lineno': self.lineno, 'line_content': self.line_content, 'code': self.code, 'message': self.message}) class ErrorKeys(object): @classmethod def print_json(cls, errors, output=sys.stdout): print("# automatically generated by tools/lintstack.py", file=output) for i in sorted(errors.keys()): print(json.dumps(i), file=output) @classmethod def from_file(cls, filename): keys = set() for line in open(filename): if line and line[0] != "#": d = json.loads(line) keys.add(tuple(d)) return keys def run_pylint(): buff = StringIO() reporter = text.ParseableTextReporter(output=buff) args = ["--include-ids=y", "-E", "os_brick"] lint.Run(args, reporter=reporter, exit=False) val = buff.getvalue() buff.close() return val def generate_error_keys(msg=None): print("Generating", KNOWN_PYLINT_EXCEPTIONS_FILE) if msg is None: msg = run_pylint() errors = LintOutput.from_msg_to_dict(msg) with open(KNOWN_PYLINT_EXCEPTIONS_FILE, "w") as f: ErrorKeys.print_json(errors, output=f) def validate(newmsg=None): print("Loading", KNOWN_PYLINT_EXCEPTIONS_FILE) known = ErrorKeys.from_file(KNOWN_PYLINT_EXCEPTIONS_FILE) if newmsg is None: print("Running pylint. Be patient...") newmsg = run_pylint() errors = LintOutput.from_msg_to_dict(newmsg) print("Unique errors reported by pylint: was %d, now %d." % (len(known), len(errors))) passed = True for err_key, err_list in errors.items(): for err in err_list: if err_key not in known: print(err.lintoutput) print() passed = False if passed: print("Congrats! pylint check passed.") redundant = known - set(errors.keys()) if redundant: print("Extra credit: some known pylint exceptions disappeared.") for i in sorted(redundant): print(json.dumps(i)) print("Consider regenerating the exception file if you will.") else: print("Please fix the errors above. If you believe they are false " "positives, run 'tools/lintstack.py generate' to overwrite.") sys.exit(1) def usage(): print("""Usage: tools/lintstack.py [generate|validate] To generate pylint_exceptions file: tools/lintstack.py generate To validate the current commit: tools/lintstack.py """) def main(): option = "validate" if len(sys.argv) > 1: option = sys.argv[1] if option == "generate": generate_error_keys() elif option == "validate": validate() else: usage() if __name__ == "__main__": main() os-brick-2.3.0/HACKING.rst0000666000175100017510000000023313230233223015040 0ustar zuulzuul00000000000000brick Style Commandments =============================================== Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/ os-brick-2.3.0/requirements.txt0000666000175100017510000000117713230233223016536 0ustar zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. pbr!=2.1.0,>=2.0.0 # Apache-2.0 Babel!=2.4.0,>=2.3.4 # BSD eventlet!=0.18.3,!=0.20.1,<0.21.0,>=0.18.2 # MIT oslo.concurrency>=3.25.0 # Apache-2.0 oslo.log>=3.36.0 # Apache-2.0 oslo.i18n>=3.15.3 # Apache-2.0 oslo.privsep>=1.23.0 # Apache-2.0 oslo.service!=1.28.1,>=1.24.0 # Apache-2.0 oslo.utils>=3.33.0 # Apache-2.0 requests>=2.14.2 # Apache-2.0 retrying!=1.3.0,>=1.2.3 # Apache-2.0 six>=1.10.0 # MIT os-win>=2.0.0 # Apache-2.0 os-brick-2.3.0/pylintrc0000666000175100017510000000220313230233223015030 0ustar zuulzuul00000000000000# The format of this file isn't really documented; just use --generate-rcfile [Messages Control] # NOTE(justinsb): We might want to have a 2nd strict pylintrc in future # C0111: Don't require docstrings on every method # W0511: TODOs in code comments are fine. # W0142: *args and **kwargs are fine. # W0622: Redefining id is fine. disable=C0111,W0511,W0142,W0622 [Basic] # Variable names can be 1 to 31 characters long, with lowercase and underscores variable-rgx=[a-z_][a-z0-9_]{0,30}$ # Argument names can be 2 to 31 characters long, with lowercase and underscores argument-rgx=[a-z_][a-z0-9_]{1,30}$ # Method names should be at least 3 characters long # and be lowercased with underscores method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|tearDown)$ # Don't require docstrings on tests. no-docstring-rgx=((__.*__)|([tT]est.*)|setUp|tearDown)$ [Design] max-public-methods=100 min-public-methods=0 max-args=6 [Variables] dummy-variables-rgx=_ [Typecheck] # Disable warnings on the HTTPSConnection classes because pylint doesn't # support importing from six.moves yet, see: # https://bitbucket.org/logilab/pylint/issue/550/ ignored-classes=HTTPSConnection os-brick-2.3.0/tox.ini0000666000175100017510000000516013230233223014561 0ustar zuulzuul00000000000000[tox] minversion = 2.0 envlist = py35,py27,pep8 skipsdist = True [testenv] usedevelop = True setenv = VIRTUAL_ENV={envdir} OS_TEST_PATH=./os_brick/tests OS_TEST_TIMEOUT=60 OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 install_command = pip install {opts} {packages} deps = -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt # By default ostestr will set concurrency # to ncpu, to specify something else use # the concurrency= option. # call ie: 'tox -epy27 -- --concurrency=4' commands = ostestr {posargs} whitelist_externals = bash find passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY [testenv:debug] commands = find . -type f -name "*.pyc" -delete oslo_debug_helper {posargs} [testenv:pep8] commands = flake8 {posargs} [testenv:fast8] envdir = {toxworkdir}/pep8 commands = {toxinidir}/tools/fast8.sh [testenv:pylint] deps = -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} -r{toxinidir}/requirements.txt pylint==0.26.0 commands = bash tools/lintstack.sh [testenv:venv] commands = {posargs} [testenv:cover] # To see the report of missing coverage add to commands # coverage report --show-missing setenv = {[testenv]setenv} PYTHON=coverage run --source os_brick --parallel-mode commands = stestr run {posargs} coverage combine coverage html -d cover coverage xml -o cover/coverage/xml [testenv:docs] commands = python setup.py build_sphinx [testenv:releasenotes] commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html [flake8] # Following checks are ignored on purpose. # # E251 unexpected spaces around keyword / parameter equals # reason: no improvement in readability show-source = True ignore = E251 enable-extensions=H106,H203 builtins = _ exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build max-complexity=30 [hacking] import_exceptions = os_brick.i18n [testenv:bindep] # Do not install any requirements. We want this to be fast and work even if # system dependencies are missing, since it's used to tell you what system # dependencies are missing! This also means that bindep must be installed # separately, outside of the requirements files, and develop mode disabled # explicitly to avoid unnecessarily installing the checked-out repo too (this # further relies on "tox.skipsdist = True" above). deps = bindep commands = bindep test usedevelop = False os-brick-2.3.0/test-requirements.txt0000666000175100017510000000115313230233241017505 0ustar zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 coverage!=4.4,>=4.0 # Apache-2.0 ddt>=1.0.1 # MIT reno>=2.5.0 # Apache-2.0 sphinx!=1.6.6,>=1.6.2 # BSD openstackdocstheme>=1.17.0 # Apache-2.0 oslotest>=3.2.0 # Apache-2.0 testrepository>=0.0.18 # Apache-2.0/BSD testscenarios>=0.4 # Apache-2.0/BSD testtools>=2.2.0 # MIT os-testr>=1.0.0 # Apache-2.0 oslo.vmware>=2.17.0 # Apache-2.0 castellan>=0.16.0 # Apache-2.0 os-brick-2.3.0/os_brick/0000775000175100017510000000000013230233405015037 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/executor.py0000666000175100017510000000561413230233223017255 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic exec utility that allows us to set the execute and root_helper attributes for putils. Some projects need their own execute wrapper and root_helper settings, so this provides that hook. """ import threading from oslo_concurrency import processutils as putils from oslo_context import context as context_utils from oslo_utils import encodeutils from os_brick.privileged import rootwrap as priv_rootwrap class Executor(object): def __init__(self, root_helper, execute=None, *args, **kwargs): if execute is None: execute = priv_rootwrap.execute self.set_execute(execute) self.set_root_helper(root_helper) @staticmethod def safe_decode(string): return string and encodeutils.safe_decode(string, errors='ignore') @classmethod def make_putils_error_safe(cls, exc): """Converts ProcessExecutionError string attributes to unicode.""" for field in ('stderr', 'stdout', 'cmd', 'description'): value = getattr(exc, field, None) if value: setattr(exc, field, cls.safe_decode(value)) def _execute(self, *args, **kwargs): try: result = self.__execute(*args, **kwargs) if result: result = (self.safe_decode(result[0]), self.safe_decode(result[1])) return result except putils.ProcessExecutionError as e: self.make_putils_error_safe(e) raise def set_execute(self, execute): self.__execute = execute def set_root_helper(self, helper): self._root_helper = helper class Thread(threading.Thread): """Thread class that inherits the parent's context. This is useful when you are spawning a thread and want LOG entries to display the right context information, such as the request. """ def __init__(self, *args, **kwargs): # Store the caller's context as a private variable shared among threads self.__context__ = context_utils.get_current() super(Thread, self).__init__(*args, **kwargs) def run(self): # Store the context in the current thread's request store if self.__context__: self.__context__.update_store() super(Thread, self).run() os-brick-2.3.0/os_brick/__init__.py0000666000175100017510000000000013230233223017136 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/remotefs/0000775000175100017510000000000013230233405016663 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/remotefs/remotefs.py0000666000175100017510000002404513230233223021066 0ustar zuulzuul00000000000000# Copyright (c) 2013 OpenStack Foundation # All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Remote filesystem client utilities.""" import hashlib import os import re import tempfile from oslo_log import log as logging import six from os_brick import exception from os_brick import executor from os_brick.i18n import _ LOG = logging.getLogger(__name__) class RemoteFsClient(executor.Executor): def __init__(self, mount_type, root_helper, execute=None, *args, **kwargs): super(RemoteFsClient, self).__init__(root_helper, execute=execute, *args, **kwargs) mount_type_to_option_prefix = { 'nfs': 'nfs', 'cifs': 'smbfs', 'glusterfs': 'glusterfs', 'vzstorage': 'vzstorage', 'quobyte': 'quobyte', 'scality': 'scality' } if mount_type not in mount_type_to_option_prefix: raise exception.ProtocolNotSupported(protocol=mount_type) self._mount_type = mount_type option_prefix = mount_type_to_option_prefix[mount_type] self._mount_base = kwargs.get(option_prefix + '_mount_point_base') if not self._mount_base: raise exception.InvalidParameterValue( err=_('%s_mount_point_base required') % option_prefix) self._mount_options = kwargs.get(option_prefix + '_mount_options') if mount_type == "nfs": self._check_nfs_options() def get_mount_base(self): return self._mount_base def _get_hash_str(self, base_str): """Return a string that represents hash of base_str (hex format).""" if isinstance(base_str, six.text_type): base_str = base_str.encode('utf-8') return hashlib.md5(base_str).hexdigest() def get_mount_point(self, device_name): """Get Mount Point. :param device_name: example 172.18.194.100:/var/nfs """ return os.path.join(self._mount_base, self._get_hash_str(device_name)) def _read_mounts(self): (out, _err) = self._execute('mount', check_exit_code=0) lines = out.split('\n') mounts = {} for line in lines: tokens = line.split() if 2 < len(tokens): device = tokens[0] mnt_point = tokens[2] mounts[mnt_point] = device return mounts def mount(self, share, flags=None): """Mount given share.""" mount_path = self.get_mount_point(share) if mount_path in self._read_mounts(): LOG.info('Already mounted: %s', mount_path) return self._execute('mkdir', '-p', mount_path, check_exit_code=0) if self._mount_type == 'nfs': self._mount_nfs(share, mount_path, flags) else: self._do_mount(self._mount_type, share, mount_path, self._mount_options, flags) def _do_mount(self, mount_type, share, mount_path, mount_options=None, flags=None): """Mounts share based on the specified params.""" mnt_cmd = ['mount', '-t', mount_type] if mount_options is not None: mnt_cmd.extend(['-o', mount_options]) if flags is not None: mnt_cmd.extend(flags) mnt_cmd.extend([share, mount_path]) self._execute(*mnt_cmd, root_helper=self._root_helper, run_as_root=True, check_exit_code=0) def _mount_nfs(self, nfs_share, mount_path, flags=None): """Mount nfs share using present mount types.""" mnt_errors = {} # This loop allows us to first try to mount with NFS 4.1 for pNFS # support but falls back to mount NFS 4 or NFS 3 if either the client # or server do not support it. for mnt_type in sorted(self._nfs_mount_type_opts.keys(), reverse=True): options = self._nfs_mount_type_opts[mnt_type] try: self._do_mount('nfs', nfs_share, mount_path, options, flags) LOG.debug('Mounted %(sh)s using %(mnt_type)s.', {'sh': nfs_share, 'mnt_type': mnt_type}) return except Exception as e: mnt_errors[mnt_type] = six.text_type(e) LOG.debug('Failed to do %s mount.', mnt_type) raise exception.BrickException(_("NFS mount failed for share %(sh)s. " "Error - %(error)s") % {'sh': nfs_share, 'error': mnt_errors}) def _check_nfs_options(self): """Checks and prepares nfs mount type options.""" self._nfs_mount_type_opts = {'nfs': self._mount_options} nfs_vers_opt_patterns = ['^nfsvers', '^vers', '^v[\d]'] for opt in nfs_vers_opt_patterns: if self._option_exists(self._mount_options, opt): return # pNFS requires NFS 4.1. The mount.nfs4 utility does not automatically # negotiate 4.1 support, we have to ask for it by specifying two # options: vers=4 and minorversion=1. pnfs_opts = self._update_option(self._mount_options, 'vers', '4') pnfs_opts = self._update_option(pnfs_opts, 'minorversion', '1') self._nfs_mount_type_opts['pnfs'] = pnfs_opts def _option_exists(self, options, opt_pattern): """Checks if the option exists in nfs options and returns position.""" options = [x.strip() for x in options.split(',')] if options else [] pos = 0 for opt in options: pos = pos + 1 if re.match(opt_pattern, opt, flags=0): return pos return 0 def _update_option(self, options, option, value=None): """Update option if exists else adds it and returns new options.""" opts = [x.strip() for x in options.split(',')] if options else [] pos = self._option_exists(options, option) if pos: opts.pop(pos - 1) opt = '%s=%s' % (option, value) if value else option opts.append(opt) return ",".join(opts) if len(opts) > 1 else opts[0] class ScalityRemoteFsClient(RemoteFsClient): def __init__(self, mount_type, root_helper, execute=None, *args, **kwargs): super(ScalityRemoteFsClient, self).__init__(mount_type, root_helper, execute=execute, *args, **kwargs) self._mount_type = mount_type self._mount_base = kwargs.get( 'scality_mount_point_base', "").rstrip('/') if not self._mount_base: raise exception.InvalidParameterValue( err=_('scality_mount_point_base required')) self._mount_options = None def get_mount_point(self, device_name): return os.path.join(self._mount_base, device_name, "00") def mount(self, share, flags=None): """Mount the Scality ScaleOut FS. The `share` argument is ignored because you can't mount several SOFS at the same type on a single server. But we want to keep the same method signature for class inheritance purpose. """ if self._mount_base in self._read_mounts(): LOG.info('Already mounted: %s', self._mount_base) return self._execute('mkdir', '-p', self._mount_base, check_exit_code=0) super(ScalityRemoteFsClient, self)._do_mount( 'sofs', '/etc/sfused.conf', self._mount_base) class VZStorageRemoteFSClient(RemoteFsClient): def _vzstorage_write_mds_list(self, cluster_name, mdss): tmp_dir = tempfile.mkdtemp(prefix='vzstorage-') tmp_bs_path = os.path.join(tmp_dir, 'bs_list') with open(tmp_bs_path, 'w') as f: for mds in mdss: f.write(mds + "\n") conf_dir = os.path.join('/etc/pstorage/clusters', cluster_name) if os.path.exists(conf_dir): bs_path = os.path.join(conf_dir, 'bs_list') self._execute('cp', '-f', tmp_bs_path, bs_path, root_helper=self._root_helper, run_as_root=True) else: self._execute('cp', '-rf', tmp_dir, conf_dir, root_helper=self._root_helper, run_as_root=True) self._execute('chown', '-R', 'root:root', conf_dir, root_helper=self._root_helper, run_as_root=True) def _do_mount(self, mount_type, vz_share, mount_path, mount_options=None, flags=None): m = re.search("(?:(\S+):\/)?([a-zA-Z0-9_-]+)(?::(\S+))?", vz_share) if not m: msg = (_("Invalid Virtuozzo Storage share specification: %r." "Must be: [MDS1[,MDS2],...:/][:PASSWORD].") % vz_share) raise exception.BrickException(msg) mdss = m.group(1) cluster_name = m.group(2) passwd = m.group(3) if mdss: mdss = mdss.split(',') self._vzstorage_write_mds_list(cluster_name, mdss) if passwd: self._execute('pstorage', '-c', cluster_name, 'auth-node', '-P', process_input=passwd, root_helper=self._root_helper, run_as_root=True) mnt_cmd = ['pstorage-mount', '-c', cluster_name] if flags: mnt_cmd.extend(flags) mnt_cmd.extend([mount_path]) self._execute(*mnt_cmd, root_helper=self._root_helper, run_as_root=True, check_exit_code=0) os-brick-2.3.0/os_brick/remotefs/__init__.py0000666000175100017510000000000013230233223020762 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/remotefs/windows_remotefs.py0000666000175100017510000001147513230233223022643 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Windows remote filesystem client utilities.""" import os import re from oslo_log import log as logging from os_win import utilsfactory from os_brick import exception from os_brick.i18n import _ from os_brick.remotefs import remotefs LOG = logging.getLogger(__name__) class WindowsRemoteFsClient(remotefs.RemoteFsClient): _username_regex = re.compile(r'user(?:name)?=([^, ]+)') _password_regex = re.compile(r'pass(?:word)?=([^, ]+)') _loopback_share_map = {} def __init__(self, mount_type, root_helper=None, execute=None, *args, **kwargs): mount_type_to_option_prefix = { 'cifs': 'smbfs', 'smbfs': 'smbfs', } self._local_path_for_loopback = kwargs.get('local_path_for_loopback', False) if mount_type not in mount_type_to_option_prefix: raise exception.ProtocolNotSupported(protocol=mount_type) self._mount_type = mount_type option_prefix = mount_type_to_option_prefix[mount_type] self._mount_base = kwargs.get(option_prefix + '_mount_point_base') self._mount_options = kwargs.get(option_prefix + '_mount_options') self._smbutils = utilsfactory.get_smbutils() self._pathutils = utilsfactory.get_pathutils() def get_local_share_path(self, share, expect_existing=True): local_share_path = self._smbutils.get_smb_share_path(share) if not local_share_path and expect_existing: err_msg = _("Could not find the local " "share path for %(share)s.") raise exception.VolumePathsNotFound(err_msg % dict(share=share)) return local_share_path def _get_share_norm_path(self, share): return share.replace('/', '\\') def get_share_name(self, share): return self._get_share_norm_path(share).lstrip('\\').split('\\', 1)[1] def mount(self, share, flags=None): share_norm_path = self._get_share_norm_path(share) use_local_path = (self._local_path_for_loopback and self._smbutils.is_local_share(share_norm_path)) if use_local_path: LOG.info("Skipping mounting local share %(share_path)s.", dict(share_path=share_norm_path)) else: mount_options = " ".join( [self._mount_options or '', flags or '']) username, password = self._parse_credentials(mount_options) if not self._smbutils.check_smb_mapping( share_norm_path): self._smbutils.mount_smb_share(share_norm_path, username=username, password=password) if self._mount_base: self._create_mount_point(share, use_local_path) def unmount(self, share): self._smbutils.unmount_smb_share(self._get_share_norm_path(share)) def _create_mount_point(self, share, use_local_path): # The mount point will contain a hash of the share so we're # intentionally preserving the original share path as this is # what the caller will expect. mnt_point = self.get_mount_point(share) share_norm_path = self._get_share_norm_path(share) share_name = self.get_share_name(share) symlink_dest = (share_norm_path if not use_local_path else self.get_local_share_path(share_name)) if not os.path.isdir(self._mount_base): os.makedirs(self._mount_base) if os.path.exists(mnt_point): if not self._pathutils.is_symlink(mnt_point): raise exception.BrickException(_("Link path already exists " "and it's not a symlink")) else: self._pathutils.create_sym_link(mnt_point, symlink_dest) def _parse_credentials(self, opts_str): if not opts_str: return None, None match = self._username_regex.findall(opts_str) username = match[0] if match and match[0] != 'guest' else None match = self._password_regex.findall(opts_str) password = match[0] if match else None return username, password os-brick-2.3.0/os_brick/local_dev/0000775000175100017510000000000013230233405016767 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/local_dev/lvm.py0000666000175100017510000007725513230233241020157 0ustar zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ LVM class for performing LVM operations. """ import math import os import re from os_brick import exception from os_brick import executor from os_brick.privileged import rootwrap as priv_rootwrap from os_brick import utils from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import excutils from six import moves LOG = logging.getLogger(__name__) class LVM(executor.Executor): """LVM object to enable various LVM related operations.""" LVM_CMD_PREFIX = ['env', 'LC_ALL=C'] def __init__(self, vg_name, root_helper, create_vg=False, physical_volumes=None, lvm_type='default', executor=None, lvm_conf=None): """Initialize the LVM object. The LVM object is based on an LVM VolumeGroup, one instantiation for each VolumeGroup you have/use. :param vg_name: Name of existing VG or VG to create :param root_helper: Execution root_helper method to use :param create_vg: Indicates the VG doesn't exist and we want to create it :param physical_volumes: List of PVs to build VG on :param lvm_type: VG and Volume type (default, or thin) :param executor: Execute method to use, None uses oslo_concurrency.processutils """ super(LVM, self).__init__(execute=executor, root_helper=root_helper) self.vg_name = vg_name self.pv_list = [] self.vg_size = 0.0 self.vg_free_space = 0.0 self.vg_lv_count = 0 self.vg_uuid = None self.vg_thin_pool = None self.vg_thin_pool_size = 0.0 self.vg_thin_pool_free_space = 0.0 self._supports_snapshot_lv_activation = None self._supports_lvchange_ignoreskipactivation = None self.vg_provisioned_capacity = 0.0 # Ensure LVM_SYSTEM_DIR has been added to LVM.LVM_CMD_PREFIX # before the first LVM command is executed, and use the directory # where the specified lvm_conf file is located as the value. if lvm_conf and os.path.isfile(lvm_conf): lvm_sys_dir = os.path.dirname(lvm_conf) LVM.LVM_CMD_PREFIX = ['env', 'LC_ALL=C', 'LVM_SYSTEM_DIR=' + lvm_sys_dir] if create_vg and physical_volumes is not None: try: self._create_vg(physical_volumes) except putils.ProcessExecutionError as err: LOG.exception('Error creating Volume Group') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise exception.VolumeGroupCreationFailed(vg_name=self.vg_name) if self._vg_exists() is False: LOG.error('Unable to locate Volume Group %s', vg_name) raise exception.VolumeGroupNotFound(vg_name=vg_name) # NOTE: we assume that the VG has been activated outside of Cinder if lvm_type == 'thin': pool_name = "%s-pool" % self.vg_name if self.get_volume(pool_name) is None: try: self.create_thin_pool(pool_name) except putils.ProcessExecutionError: # Maybe we just lost the race against another copy of # this driver being in init in parallel - e.g. # cinder-volume and cinder-backup starting in parallel if self.get_volume(pool_name) is None: raise self.vg_thin_pool = pool_name self.activate_lv(self.vg_thin_pool) self.pv_list = self.get_all_physical_volumes(root_helper, vg_name) def _vg_exists(self): """Simple check to see if VG exists. :returns: True if vg specified in object exists, else False """ exists = False cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '-o', 'name', self.vg_name] (out, _err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: volume_groups = out.split() if self.vg_name in volume_groups: exists = True return exists def _create_vg(self, pv_list): cmd = ['vgcreate', self.vg_name, ','.join(pv_list)] self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) def _get_vg_uuid(self): cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '-o', 'uuid', self.vg_name] (out, _err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: return out.split() else: return [] def _get_thin_pool_free_space(self, vg_name, thin_pool_name): """Returns available thin pool free space. :param vg_name: the vg where the pool is placed :param thin_pool_name: the thin pool to gather info for :returns: Free space in GB (float), calculated using data_percent """ cmd = LVM.LVM_CMD_PREFIX + ['lvs', '--noheadings', '--unit=g', '-o', 'size,data_percent', '--separator', ':', '--nosuffix'] # NOTE(gfidente): data_percent only applies to some types of LV so we # make sure to append the actual thin pool name cmd.append("/dev/%s/%s" % (vg_name, thin_pool_name)) free_space = 0.0 try: (out, err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: out = out.strip() data = out.split(':') pool_size = float(data[0]) data_percent = float(data[1]) consumed_space = pool_size / 100 * data_percent free_space = pool_size - consumed_space free_space = round(free_space, 2) except putils.ProcessExecutionError as err: LOG.exception('Error querying thin pool about data_percent') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) return free_space @staticmethod def get_lvm_version(root_helper): """Static method to get LVM version from system. :param root_helper: root_helper to use for execute :returns: version 3-tuple """ cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--version'] (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) lines = out.split('\n') for line in lines: if 'LVM version' in line: version_list = line.split() # NOTE(gfidente): version is formatted as follows: # major.minor.patchlevel(library API version)[-customisation] version = version_list[2] version_filter = r"(\d+)\.(\d+)\.(\d+).*" r = re.search(version_filter, version) version_tuple = tuple(map(int, r.group(1, 2, 3))) return version_tuple @staticmethod def supports_thin_provisioning(root_helper): """Static method to check for thin LVM support on a system. :param root_helper: root_helper to use for execute :returns: True if supported, False otherwise """ return LVM.get_lvm_version(root_helper) >= (2, 2, 95) @property def supports_snapshot_lv_activation(self): """Property indicating whether snap activation changes are supported. Check for LVM version >= 2.02.91. (LVM2 git: e8a40f6 Allow to activate snapshot) :returns: True/False indicating support """ if self._supports_snapshot_lv_activation is not None: return self._supports_snapshot_lv_activation self._supports_snapshot_lv_activation = ( self.get_lvm_version(self._root_helper) >= (2, 2, 91)) return self._supports_snapshot_lv_activation @property def supports_lvchange_ignoreskipactivation(self): """Property indicating whether lvchange can ignore skip activation. Check for LVM version >= 2.02.99. (LVM2 git: ab789c1bc add --ignoreactivationskip to lvchange) """ if self._supports_lvchange_ignoreskipactivation is not None: return self._supports_lvchange_ignoreskipactivation self._supports_lvchange_ignoreskipactivation = ( self.get_lvm_version(self._root_helper) >= (2, 2, 99)) return self._supports_lvchange_ignoreskipactivation @property def supports_full_pool_create(self): """Property indicating whether 100% pool creation is supported. Check for LVM version >= 2.02.115. Ref: https://bugzilla.redhat.com/show_bug.cgi?id=998347 """ if self.get_lvm_version(self._root_helper) >= (2, 2, 115): return True else: return False @staticmethod def get_lv_info(root_helper, vg_name=None, lv_name=None): """Retrieve info about LVs (all, in a VG, or a single LV). :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :param lv_name: optional, gathers info for only the specified LV :returns: List of Dictionaries with LV info """ cmd = LVM.LVM_CMD_PREFIX + ['lvs', '--noheadings', '--unit=g', '-o', 'vg_name,name,size', '--nosuffix'] if lv_name is not None and vg_name is not None: cmd.append("%s/%s" % (vg_name, lv_name)) elif vg_name is not None: cmd.append(vg_name) try: (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) except putils.ProcessExecutionError as err: with excutils.save_and_reraise_exception(reraise=True) as ctx: if "not found" in err.stderr or "Failed to find" in err.stderr: ctx.reraise = False LOG.info("Logical Volume not found when querying " "LVM info. (vg_name=%(vg)s, lv_name=%(lv)s", {'vg': vg_name, 'lv': lv_name}) out = None lv_list = [] if out is not None: volumes = out.split() iterator = moves.zip(*[iter(volumes)] * 3) # pylint: disable=E1101 for vg, name, size in iterator: lv_list.append({"vg": vg, "name": name, "size": size}) return lv_list def get_volumes(self, lv_name=None): """Get all LV's associated with this instantiation (VG). :returns: List of Dictionaries with LV info """ return self.get_lv_info(self._root_helper, self.vg_name, lv_name) def get_volume(self, name): """Get reference object of volume specified by name. :returns: dict representation of Logical Volume if exists """ ref_list = self.get_volumes(name) for r in ref_list: if r['name'] == name: return r return None @staticmethod def get_all_physical_volumes(root_helper, vg_name=None): """Static method to get all PVs on a system. :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :returns: List of Dictionaries with PV info """ field_sep = '|' cmd = LVM.LVM_CMD_PREFIX + ['pvs', '--noheadings', '--unit=g', '-o', 'vg_name,name,size,free', '--separator', field_sep, '--nosuffix'] (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) pvs = out.split() if vg_name is not None: pvs = [pv for pv in pvs if vg_name == pv.split(field_sep)[0]] pv_list = [] for pv in pvs: fields = pv.split(field_sep) pv_list.append({'vg': fields[0], 'name': fields[1], 'size': float(fields[2]), 'available': float(fields[3])}) return pv_list def get_physical_volumes(self): """Get all PVs associated with this instantiation (VG). :returns: List of Dictionaries with PV info """ self.pv_list = self.get_all_physical_volumes(self._root_helper, self.vg_name) return self.pv_list @staticmethod def get_all_volume_groups(root_helper, vg_name=None): """Static method to get all VGs on a system. :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :returns: List of Dictionaries with VG info """ cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '--unit=g', '-o', 'name,size,free,lv_count,uuid', '--separator', ':', '--nosuffix'] if vg_name is not None: cmd.append(vg_name) (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) vg_list = [] if out is not None: vgs = out.split() for vg in vgs: fields = vg.split(':') vg_list.append({'name': fields[0], 'size': float(fields[1]), 'available': float(fields[2]), 'lv_count': int(fields[3]), 'uuid': fields[4]}) return vg_list def update_volume_group_info(self): """Update VG info for this instantiation. Used to update member fields of object and provide a dict of info for caller. :returns: Dictionaries of VG info """ vg_list = self.get_all_volume_groups(self._root_helper, self.vg_name) if len(vg_list) != 1: LOG.error('Unable to find VG: %s', self.vg_name) raise exception.VolumeGroupNotFound(vg_name=self.vg_name) self.vg_size = float(vg_list[0]['size']) self.vg_free_space = float(vg_list[0]['available']) self.vg_lv_count = int(vg_list[0]['lv_count']) self.vg_uuid = vg_list[0]['uuid'] total_vols_size = 0.0 if self.vg_thin_pool is not None: # NOTE(xyang): If providing only self.vg_name, # get_lv_info will output info on the thin pool and all # individual volumes. # get_lv_info(self._root_helper, 'stack-vg') # sudo lvs --noheadings --unit=g -o vg_name,name,size # --nosuffix stack-vg # stack-vg stack-pool 9.51 # stack-vg volume-13380d16-54c3-4979-9d22-172082dbc1a1 1.00 # stack-vg volume-629e13ab-7759-46a5-b155-ee1eb20ca892 1.00 # stack-vg volume-e3e6281c-51ee-464c-b1a7-db6c0854622c 1.00 # # If providing both self.vg_name and self.vg_thin_pool, # get_lv_info will output only info on the thin pool, but not # individual volumes. # get_lv_info(self._root_helper, 'stack-vg', 'stack-pool') # sudo lvs --noheadings --unit=g -o vg_name,name,size # --nosuffix stack-vg/stack-pool # stack-vg stack-pool 9.51 # # We need info on both the thin pool and the volumes, # therefore we should provide only self.vg_name, but not # self.vg_thin_pool here. for lv in self.get_lv_info(self._root_helper, self.vg_name): lvsize = lv['size'] # get_lv_info runs "lvs" command with "--nosuffix". # This removes "g" from "1.00g" and only outputs "1.00". # Running "lvs" command without "--nosuffix" will output # "1.00g" if "g" is the unit. # Remove the unit if it is in lv['size']. if not lv['size'][-1].isdigit(): lvsize = lvsize[:-1] if lv['name'] == self.vg_thin_pool: self.vg_thin_pool_size = float(lvsize) tpfs = self._get_thin_pool_free_space(self.vg_name, self.vg_thin_pool) self.vg_thin_pool_free_space = tpfs else: total_vols_size = total_vols_size + float(lvsize) total_vols_size = round(total_vols_size, 2) self.vg_provisioned_capacity = total_vols_size def _calculate_thin_pool_size(self): """Calculates the correct size for a thin pool. Ideally we would use 100% of the containing volume group and be done. But the 100%VG notation to lvcreate is not implemented and thus cannot be used. See https://bugzilla.redhat.com/show_bug.cgi?id=998347 Further, some amount of free space must remain in the volume group for metadata for the contained logical volumes. The exact amount depends on how much volume sharing you expect. :returns: An lvcreate-ready string for the number of calculated bytes. """ # make sure volume group information is current self.update_volume_group_info() if LVM.supports_full_pool_create: return ["-l", "100%FREE"] # leave 5% free for metadata return ["-L", "%sg" % (self.vg_free_space * 0.95)] def create_thin_pool(self, name=None): """Creates a thin provisioning pool for this VG. The syntax here is slightly different than the default lvcreate -T, so we'll just write a custom cmd here and do it. :param name: Name to use for pool, default is "-pool" :returns: The size string passed to the lvcreate command """ if not LVM.supports_thin_provisioning(self._root_helper): LOG.error('Requested to setup thin provisioning, ' 'however current LVM version does not ' 'support it.') return None if name is None: name = '%s-pool' % self.vg_name vg_pool_name = '%s/%s' % (self.vg_name, name) size_args = self._calculate_thin_pool_size() cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-T'] cmd.extend(size_args) cmd.append(vg_pool_name) LOG.debug("Creating thin pool '%(pool)s' with size %(size)s of " "total %(free)sg", {'pool': vg_pool_name, 'size': size_args, 'free': self.vg_free_space}) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) self.vg_thin_pool = name return def create_volume(self, name, size_str, lv_type='default', mirror_count=0): """Creates a logical volume on the object's VG. :param name: Name to use when creating Logical Volume :param size_str: Size to use when creating Logical Volume :param lv_type: Type of Volume (default or thin) :param mirror_count: Use LVM mirroring with specified count """ if lv_type == 'thin': pool_path = '%s/%s' % (self.vg_name, self.vg_thin_pool) cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-T', '-V', size_str, '-n', name, pool_path] else: cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-n', name, self.vg_name, '-L', size_str] if mirror_count > 0: cmd.extend(['-m', mirror_count, '--nosync', '--mirrorlog', 'mirrored']) terras = int(size_str[:-1]) / 1024.0 if terras >= 1.5: rsize = int(2 ** math.ceil(math.log(terras) / math.log(2))) # NOTE(vish): Next power of two for region size. See: # http://red.ht/U2BPOD cmd.extend(['-R', str(rsize)]) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error creating Volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise @utils.retry(putils.ProcessExecutionError) def create_lv_snapshot(self, name, source_lv_name, lv_type='default'): """Creates a snapshot of a logical volume. :param name: Name to assign to new snapshot :param source_lv_name: Name of Logical Volume to snapshot :param lv_type: Type of LV (default or thin) """ source_lvref = self.get_volume(source_lv_name) if source_lvref is None: LOG.error("Trying to create snapshot by non-existent LV: %s", source_lv_name) raise exception.VolumeDeviceNotFound(device=source_lv_name) cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '--name', name, '-k', 'y', '--snapshot', '%s/%s' % (self.vg_name, source_lv_name)] if lv_type != 'thin': size = source_lvref['size'] cmd.extend(['-L', '%sg' % (size)]) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error creating snapshot') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise def _mangle_lv_name(self, name): # Linux LVM reserves name that starts with snapshot, so that # such volume name can't be created. Mangle it. if not name.startswith('snapshot'): return name return '_' + name def _lv_is_active(self, name): cmd = LVM.LVM_CMD_PREFIX + ['lvdisplay', '--noheading', '-C', '-o', 'Attr', '%s/%s' % (self.vg_name, name)] out, _err = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out: out = out.strip() # An example output might be '-wi-a----'; the 4th index specifies # the status of the volume. 'a' for active, '-' for inactive. if (out[4] == 'a'): return True return False def deactivate_lv(self, name): lv_path = self.vg_name + '/' + self._mangle_lv_name(name) cmd = ['lvchange', '-a', 'n'] cmd.append(lv_path) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error deactivating LV') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise # Wait until lv is deactivated to return in # order to prevent a race condition. self._wait_for_volume_deactivation(name) @utils.retry(exceptions=exception.VolumeNotDeactivated, retries=3, backoff_rate=1) def _wait_for_volume_deactivation(self, name): LOG.debug("Checking to see if volume %s has been deactivated.", name) if self._lv_is_active(name): LOG.debug("Volume %s is still active.", name) raise exception.VolumeNotDeactivated(name=name) else: LOG.debug("Volume %s has been deactivated.", name) def activate_lv(self, name, is_snapshot=False, permanent=False): """Ensure that logical volume/snapshot logical volume is activated. :param name: Name of LV to activate :param is_snapshot: whether LV is a snapshot :param permanent: whether we should drop skipactivation flag :raises: putils.ProcessExecutionError """ # This is a no-op if requested for a snapshot on a version # of LVM that doesn't support snapshot activation. # (Assume snapshot LV is always active.) if is_snapshot and not self.supports_snapshot_lv_activation: return lv_path = self.vg_name + '/' + self._mangle_lv_name(name) # Must pass --yes to activate both the snap LV and its origin LV. # Otherwise lvchange asks if you would like to do this interactively, # and fails. cmd = ['lvchange', '-a', 'y', '--yes'] if self.supports_lvchange_ignoreskipactivation: cmd.append('-K') # If permanent=True is specified, drop the skipactivation flag in # order to make this LV automatically activated after next reboot. if permanent: cmd += ['-k', 'n'] cmd.append(lv_path) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error activating LV') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise @utils.retry(putils.ProcessExecutionError) def delete(self, name): """Delete logical volume or snapshot. :param name: Name of LV to delete """ def run_udevadm_settle(): self._execute('udevadm', 'settle', root_helper=self._root_helper, run_as_root=True, check_exit_code=False) # LV removal seems to be a race with other writers or udev in # some cases (see LP #1270192), so we enable retry deactivation LVM_CONFIG = 'activation { retry_deactivation = 1} ' try: self._execute( 'lvremove', '--config', LVM_CONFIG, '-f', '%s/%s' % (self.vg_name, name), root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.debug('Error reported running lvremove: CMD: %(command)s, ' 'RESPONSE: %(response)s', {'command': err.cmd, 'response': err.stderr}) LOG.debug('Attempting udev settle and retry of lvremove...') run_udevadm_settle() # The previous failing lvremove -f might leave behind # suspended devices; when lvmetad is not available, any # further lvm command will block forever. # Therefore we need to skip suspended devices on retry. LVM_CONFIG += 'devices { ignore_suspended_devices = 1}' self._execute( 'lvremove', '--config', LVM_CONFIG, '-f', '%s/%s' % (self.vg_name, name), root_helper=self._root_helper, run_as_root=True) LOG.debug('Successfully deleted volume: %s after ' 'udev settle.', name) def revert(self, snapshot_name): """Revert an LV from snapshot. :param snapshot_name: Name of snapshot to revert """ self._execute('lvconvert', '--merge', snapshot_name, root_helper=self._root_helper, run_as_root=True) def lv_has_snapshot(self, name): cmd = LVM.LVM_CMD_PREFIX + ['lvdisplay', '--noheading', '-C', '-o', 'Attr', '%s/%s' % (self.vg_name, name)] out, _err = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out: out = out.strip() if (out[0] == 'o') or (out[0] == 'O'): return True return False def extend_volume(self, lv_name, new_size): """Extend the size of an existing volume.""" # Volumes with snaps have attributes 'o' or 'O' and will be # deactivated, but Thin Volumes with snaps have attribute 'V' # and won't be deactivated because the lv_has_snapshot method looks # for 'o' or 'O' if self.lv_has_snapshot(lv_name): self.deactivate_lv(lv_name) try: cmd = LVM.LVM_CMD_PREFIX + ['lvextend', '-L', new_size, '%s/%s' % (self.vg_name, lv_name)] self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error extending Volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise def vg_mirror_free_space(self, mirror_count): free_capacity = 0.0 disks = [] for pv in self.pv_list: disks.append(float(pv['available'])) while True: disks = sorted([a for a in disks if a > 0.0], reverse=True) if len(disks) <= mirror_count: break # consume the smallest disk disk = disks[-1] disks = disks[:-1] # match extents for each mirror on the largest disks for index in list(range(mirror_count)): disks[index] -= disk free_capacity += disk return free_capacity def vg_mirror_size(self, mirror_count): return (self.vg_free_space / (mirror_count + 1)) def rename_volume(self, lv_name, new_name): """Change the name of an existing volume.""" try: self._execute('lvrename', self.vg_name, lv_name, new_name, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error renaming logical volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise os-brick-2.3.0/os_brick/local_dev/__init__.py0000666000175100017510000000000013230233223021066 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/version.py0000666000175100017510000000131313230233223017074 0ustar zuulzuul00000000000000# All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # import pbr.version version_info = pbr.version.VersionInfo('os-brick') __version__ = version_info.version_string() os-brick-2.3.0/os_brick/encryptors/0000775000175100017510000000000013230233405017247 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/encryptors/nop.py0000666000175100017510000000276313230233223020425 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import base class NoOpEncryptor(base.VolumeEncryptor): """A VolumeEncryptor that does nothing. This class exists solely to wrap regular (i.e., unencrypted) volumes so that they do not require special handling with respect to an encrypted volume. This implementation performs no action when a volume is attached or detached. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(NoOpEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) def attach_volume(self, context): pass def detach_volume(self): pass os-brick-2.3.0/os_brick/encryptors/__init__.py0000666000175100017510000001204013230233223021355 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import nop from oslo_log import log as logging from oslo_utils import importutils from oslo_utils import strutils LOG = logging.getLogger(__name__) LUKS = "luks" PLAIN = "plain" FORMAT_TO_FRONTEND_ENCRYPTOR_MAP = { LUKS: 'os_brick.encryptors.luks.LuksEncryptor', PLAIN: 'os_brick.encryptors.cryptsetup.CryptsetupEncryptor' } LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP = { "nova.volume.encryptors.luks.LuksEncryptor": LUKS, "nova.volume.encryptors.cryptsetup.CryptsetupEncryptor": PLAIN, "nova.volume.encryptors.nop.NoopEncryptor": None, "os_brick.encryptors.luks.LuksEncryptor": LUKS, "os_brick.encryptors.cryptsetup.CryptsetupEncryptor": PLAIN, "os_brick.encryptors.nop.NoopEncryptor": None, "LuksEncryptor": LUKS, "CryptsetupEncryptor": PLAIN, "NoOpEncryptor": None, } def get_volume_encryptor(root_helper, connection_info, keymgr, execute=None, *args, **kwargs): """Creates a VolumeEncryptor used to encrypt the specified volume. :param: the connection information used to attach the volume :returns VolumeEncryptor: the VolumeEncryptor for the volume """ encryptor = nop.NoOpEncryptor(root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) location = kwargs.get('control_location', None) if location and location.lower() == 'front-end': # case insensitive provider = kwargs.get('provider') # TODO(lyarwood): Remove the following in Queens and raise an # ERROR if provider is not a key in SUPPORTED_ENCRYPTION_PROVIDERS. # Until then continue to allow both the class name and path to be used. if provider in LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP: LOG.warning("Use of the in tree encryptor class %(provider)s" " by directly referencing the implementation class" " will be blocked in the Queens release of" " os-brick.", {'provider': provider}) provider = LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP[provider] if provider in FORMAT_TO_FRONTEND_ENCRYPTOR_MAP: provider = FORMAT_TO_FRONTEND_ENCRYPTOR_MAP[provider] elif provider is None: provider = "os_brick.encryptors.nop.NoOpEncryptor" else: LOG.warning("Use of the out of tree encryptor class " "%(provider)s will be blocked with the Queens " "release of os-brick.", {'provider': provider}) try: encryptor = importutils.import_object( provider, root_helper, connection_info, keymgr, execute, **kwargs) except Exception as e: LOG.error("Error instantiating %(provider)s: %(exception)s", {'provider': provider, 'exception': e}) raise msg = ("Using volume encryptor '%(encryptor)s' for connection: " "%(connection_info)s" % {'encryptor': encryptor, 'connection_info': connection_info}) LOG.debug(strutils.mask_password(msg)) return encryptor def get_encryption_metadata(context, volume_api, volume_id, connection_info): metadata = {} if ('data' in connection_info and connection_info['data'].get('encrypted', False)): try: metadata = volume_api.get_volume_encryption_metadata(context, volume_id) if not metadata: LOG.warning('Volume %s should be encrypted but there is no ' 'encryption metadata.', volume_id) except Exception as e: LOG.error("Failed to retrieve encryption metadata for " "volume %(volume_id)s: %(exception)s", {'volume_id': volume_id, 'exception': e}) raise if metadata: msg = ("Using volume encryption metadata '%(metadata)s' for " "connection: %(connection_info)s" % {'metadata': metadata, 'connection_info': connection_info}) LOG.debug(strutils.mask_password(msg)) return metadata os-brick-2.3.0/os_brick/encryptors/base.py0000666000175100017510000000420513230233223020534 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import abc from os_brick import executor import six @six.add_metaclass(abc.ABCMeta) class VolumeEncryptor(executor.Executor): """Base class to support encrypted volumes. A VolumeEncryptor provides hooks for attaching and detaching volumes, which are called immediately prior to attaching the volume to an instance and immediately following detaching the volume from an instance. This class performs no actions for either hook. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(VolumeEncryptor, self).__init__(root_helper, execute=execute, *args, **kwargs) self._key_manager = keymgr self.encryption_key_id = kwargs.get('encryption_key_id') def _get_key(self, context): """Retrieves the encryption key for the specified volume. :param: the connection information used to attach the volume """ return self._key_manager.get(context, self.encryption_key_id) @abc.abstractmethod def attach_volume(self, context, **kwargs): """Hook called immediately prior to attaching a volume to an instance. """ pass @abc.abstractmethod def detach_volume(self, **kwargs): """Hook called immediately after detaching a volume from an instance. """ pass os-brick-2.3.0/os_brick/encryptors/luks.py0000666000175100017510000002053113230233223020600 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import cryptsetup from os_brick.privileged import rootwrap as priv_rootwrap from oslo_concurrency import processutils as putils from oslo_log import log as logging LOG = logging.getLogger(__name__) def is_luks(root_helper, device, execute=None): """Checks if the specified device uses LUKS for encryption. :param device: the device to check :returns: true if the specified device uses LUKS; false otherwise """ try: # check to see if the device uses LUKS: exit status is 0 # if the device is a LUKS partition and non-zero if not if execute is None: execute = priv_rootwrap.execute execute('cryptsetup', 'isLuks', '--verbose', device, run_as_root=True, root_helper=root_helper, check_exit_code=True) return True except putils.ProcessExecutionError as e: LOG.warning("isLuks exited abnormally (status %(exit_code)s): " "%(stderr)s", {"exit_code": e.exit_code, "stderr": e.stderr}) return False class LuksEncryptor(cryptsetup.CryptsetupEncryptor): """A VolumeEncryptor based on LUKS. This VolumeEncryptor uses dm-crypt to encrypt the specified volume. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(LuksEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) def _format_volume(self, passphrase, **kwargs): """Creates a LUKS header on the volume. :param passphrase: the passphrase used to access the volume """ LOG.debug("formatting encrypted volume %s", self.dev_path) # NOTE(joel-coffman): cryptsetup will strip trailing newlines from # input specified on stdin unless --key-file=- is specified. cmd = ["cryptsetup", "--batch-mode", "luksFormat", "--key-file=-"] cipher = kwargs.get("cipher", None) if cipher is not None: cmd.extend(["--cipher", cipher]) key_size = kwargs.get("key_size", None) if key_size is not None: cmd.extend(["--key-size", key_size]) cmd.extend([self.dev_path]) self._execute(*cmd, process_input=passphrase, check_exit_code=True, run_as_root=True, root_helper=self._root_helper, attempts=3) def _open_volume(self, passphrase, **kwargs): """Opens the LUKS partition on the volume using passphrase. :param passphrase: the passphrase used to access the volume """ LOG.debug("opening encrypted volume %s", self.dev_path) self._execute('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=passphrase, run_as_root=True, check_exit_code=True, root_helper=self._root_helper) def _unmangle_volume(self, key, passphrase, **kwargs): """Workaround for bug#1633518 First identify if a mangled passphrase is used and if found then replace with the correct unmangled version of the passphrase. """ mangled_passphrase = self._get_mangled_passphrase(key) self._open_volume(mangled_passphrase, **kwargs) self._close_volume(**kwargs) LOG.debug("%s correctly opened with a mangled passphrase, replacing " "this with the original passphrase", self.dev_path) # NOTE(lyarwood): Now that we are sure that the mangled passphrase is # used attempt to add the correct passphrase before removing the # mangled version from the volume. # luksAddKey currently prompts for the following input : # Enter any existing passphrase: # Enter new passphrase for key slot: # Verify passphrase: self._execute('cryptsetup', 'luksAddKey', self.dev_path, process_input=''.join([mangled_passphrase, '\n', passphrase, '\n', passphrase]), run_as_root=True, check_exit_code=True, root_helper=self._root_helper) # Verify that we can open the volume with the current passphrase # before removing the mangled passphrase. self._open_volume(passphrase, **kwargs) self._close_volume(**kwargs) # luksRemoveKey only prompts for the key to remove. self._execute('cryptsetup', 'luksRemoveKey', self.dev_path, process_input=mangled_passphrase, run_as_root=True, check_exit_code=True, root_helper=self._root_helper) LOG.debug("%s mangled passphrase successfully replaced", self.dev_path) def attach_volume(self, context, **kwargs): """Shadow the device and pass an unencrypted version to the instance. Transparent disk encryption is achieved by mounting the volume via dm-crypt and passing the resulting device to the instance. The instance is unaware of the underlying encryption due to modifying the original symbolic link to refer to the device mounted by dm-crypt. """ key = self._get_key(context).get_encoded() passphrase = self._get_passphrase(key) try: self._open_volume(passphrase, **kwargs) except putils.ProcessExecutionError as e: if e.exit_code == 1 and not is_luks(self._root_helper, self.dev_path, execute=self._execute): # the device has never been formatted; format it and try again LOG.info("%s is not a valid LUKS device;" " formatting device for first use", self.dev_path) self._format_volume(passphrase, **kwargs) self._open_volume(passphrase, **kwargs) elif e.exit_code == 2: # NOTE(lyarwood): Workaround bug#1633518 by replacing any # mangled passphrases that are found on the volume. # TODO(lyarwood): Remove workaround during R. LOG.warning("%s is not usable with the current " "passphrase, attempting to use a mangled " "passphrase to open the volume.", self.dev_path) self._unmangle_volume(key, passphrase, **kwargs) self._open_volume(passphrase, **kwargs) else: raise # modify the original symbolic link to refer to the decrypted device self._execute('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self._root_helper, run_as_root=True, check_exit_code=True) def _close_volume(self, **kwargs): """Closes the device (effectively removes the dm-crypt mapping).""" LOG.debug("closing encrypted volume %s", self.dev_path) # NOTE(mdbooth): luksClose will return 4 (wrong device specified) if # the device doesn't exist. We assume here that the caller hasn't # specified the wrong device, and that it doesn't exist because it # isn't open. We don't fail in this case in order to make this # operation idempotent. self._execute('cryptsetup', 'luksClose', self.dev_name, run_as_root=True, check_exit_code=[0, 4], root_helper=self._root_helper, attempts=3) os-brick-2.3.0/os_brick/encryptors/cryptsetup.py0000666000175100017510000001754413230233223022056 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import array import binascii import os from os_brick.encryptors import base from os_brick import exception from oslo_concurrency import processutils from oslo_log import log as logging LOG = logging.getLogger(__name__) class CryptsetupEncryptor(base.VolumeEncryptor): """A VolumeEncryptor based on dm-crypt. This VolumeEncryptor uses dm-crypt to encrypt the specified volume. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(CryptsetupEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) # Fail if no device_path was set when connecting the volume, e.g. in # the case of libvirt network volume drivers. data = connection_info['data'] if not data.get('device_path'): volume_id = data.get('volume_id') or connection_info.get('serial') raise exception.VolumeEncryptionNotSupported( volume_id=volume_id, volume_type=connection_info['driver_volume_type']) # the device's path as given to libvirt -- e.g., /dev/disk/by-path/... self.symlink_path = connection_info['data']['device_path'] # a unique name for the volume -- e.g., the iSCSI participant name self.dev_name = 'crypt-%s' % os.path.basename(self.symlink_path) # NOTE(lixiaoy1): This is to import fix for 1439869 from Nova. # NOTE(tsekiyama): In older version of nova, dev_name was the same # as the symlink name. Now it has 'crypt-' prefix to avoid conflict # with multipath device symlink. To enable rolling update, we use the # old name when the encrypted volume already exists. old_dev_name = os.path.basename(self.symlink_path) wwn = data.get('multipath_id') if self._is_crypt_device_available(old_dev_name): self.dev_name = old_dev_name LOG.debug("Using old encrypted volume name: %s", self.dev_name) elif wwn and wwn != old_dev_name: # FibreChannel device could be named '/dev/mapper/'. if self._is_crypt_device_available(wwn): self.dev_name = wwn LOG.debug("Using encrypted volume name from wwn: %s", self.dev_name) # the device's actual path on the compute host -- e.g., /dev/sd_ self.dev_path = os.path.realpath(self.symlink_path) def _is_crypt_device_available(self, dev_name): if not os.path.exists('/dev/mapper/%s' % dev_name): return False try: self._execute('cryptsetup', 'status', dev_name, run_as_root=True) except processutils.ProcessExecutionError as e: # If /dev/mapper/ is a non-crypt block device (such as a # normal disk or multipath device), exit_code will be 1. In the # case, we will omit the warning message. if e.exit_code != 1: LOG.warning('cryptsetup status %(dev_name)s exited ' 'abnormally (status %(exit_code)s): %(err)s', {"dev_name": dev_name, "exit_code": e.exit_code, "err": e.stderr}) return False return True def _get_passphrase(self, key): """Convert raw key to string.""" return binascii.hexlify(key).decode('utf-8') def _open_volume(self, passphrase, **kwargs): """Open the LUKS partition on the volume using passphrase. :param passphrase: the passphrase used to access the volume """ LOG.debug("opening encrypted volume %s", self.dev_path) # NOTE(joel-coffman): cryptsetup will strip trailing newlines from # input specified on stdin unless --key-file=- is specified. cmd = ["cryptsetup", "create", "--key-file=-"] cipher = kwargs.get("cipher", None) if cipher is not None: cmd.extend(["--cipher", cipher]) key_size = kwargs.get("key_size", None) if key_size is not None: cmd.extend(["--key-size", key_size]) cmd.extend([self.dev_name, self.dev_path]) self._execute(*cmd, process_input=passphrase, check_exit_code=True, run_as_root=True, root_helper=self._root_helper) def _get_mangled_passphrase(self, key): """Convert the raw key into a list of unsigned int's and then a string """ # NOTE(lyarwood): This replicates the methods used prior to Newton to # first encode the passphrase as a list of unsigned int's before # decoding back into a string. This method strips any leading 0's # of the resulting hex digit pairs, resulting in a different # passphrase being returned. encoded_key = array.array('B', key).tolist() return ''.join(hex(x).replace('0x', '') for x in encoded_key) def attach_volume(self, context, **kwargs): """Shadow the device and pass an unencrypted version to the instance. Transparent disk encryption is achieved by mounting the volume via dm-crypt and passing the resulting device to the instance. The instance is unaware of the underlying encryption due to modifying the original symbolic link to refer to the device mounted by dm-crypt. """ key = self._get_key(context).get_encoded() passphrase = self._get_passphrase(key) try: self._open_volume(passphrase, **kwargs) except processutils.ProcessExecutionError as e: if e.exit_code == 2: # NOTE(lyarwood): Workaround bug#1633518 by attempting to use # a mangled passphrase to open the device.. LOG.info("Unable to open %s with the current passphrase, " "attempting to use a mangled passphrase to open " "the volume.", self.dev_path) self._open_volume(self._get_mangled_passphrase(key), **kwargs) # modify the original symbolic link to refer to the decrypted device self._execute('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self._root_helper, run_as_root=True, check_exit_code=True) def _close_volume(self, **kwargs): """Closes the device (effectively removes the dm-crypt mapping).""" LOG.debug("closing encrypted volume %s", self.dev_path) # NOTE(mdbooth): remove will return 4 (wrong device specified) if # the device doesn't exist. We assume here that the caller hasn't # specified the wrong device, and that it doesn't exist because it # isn't open. We don't fail in this case in order to make this # operation idempotent. self._execute('cryptsetup', 'remove', self.dev_name, run_as_root=True, check_exit_code=[0, 4], root_helper=self._root_helper) def detach_volume(self, **kwargs): """Removes the dm-crypt mapping for the device.""" self._close_volume(**kwargs) os-brick-2.3.0/os_brick/initiator/0000775000175100017510000000000013230233405017041 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/initiator/initiator_connector.py0000666000175100017510000001577513230233223023506 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import abc import six from os_brick import exception from os_brick import executor from os_brick import initiator @six.add_metaclass(abc.ABCMeta) class InitiatorConnector(executor.Executor): # This object can be used on any platform (x86, S390) platform = initiator.PLATFORM_ALL # This object can be used on any os type (linux, windows) os_type = initiator.OS_TYPE_ALL def __init__(self, root_helper, driver=None, execute=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(InitiatorConnector, self).__init__(root_helper, execute=execute, *args, **kwargs) self.device_scan_attempts = device_scan_attempts def set_driver(self, driver): """The driver is used to find used LUNs.""" self.driver = driver @abc.abstractmethod def get_connector_properties(root_helper, *args, **kwargs): """The generic connector properties.""" pass @abc.abstractmethod def check_valid_device(self, path, run_as_root=True): """Test to see if the device path is a real device. :param path: The file system path for the device. :type path: str :param run_as_root: run the tests as root user? :type run_as_root: bool :returns: bool """ pass @abc.abstractmethod def connect_volume(self, connection_properties): """Connect to a volume. The connection_properties describes the information needed by the specific protocol to use to make the connection. The connection_properties is a dictionary that describes the target volume. It varies slightly by protocol type (iscsi, fibre_channel), but the structure is usually the same. An example for iSCSI: {'driver_volume_type': 'iscsi', 'data': { 'target_luns': [0, 2], 'target_iqns': ['iqn.2000-05.com.3pardata:20810002ac00383d', 'iqn.2000-05.com.3pardata:21810002ac00383d'], 'target_discovered': True, 'encrypted': False, 'qos_specs': None, 'target_portals': ['10.52.1.11:3260', '10.52.2.11:3260'], 'access_mode': 'rw', }} An example for fibre_channel: {'driver_volume_type': 'fibre_channel', 'data': { 'initiator_target_map': {'100010604b010459': ['21230002AC00383D'], '100010604b01045d': ['21230002AC00383D'] }, 'target_discovered': True, 'encrypted': False, 'qos_specs': None, 'target_lun': 1, 'access_mode': 'rw', 'target_wwn': [ '20210002AC00383D', '20220002AC00383D', ], }} :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ pass @abc.abstractmethod def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. The connection_properties are the same as from connect_volume. The device_info is returned from connect_volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :type ignore_errors: bool """ pass @abc.abstractmethod def get_volume_paths(self, connection_properties): """Return the list of existing paths for a volume. The job of this method is to find out what paths in the system are associated with a volume as described by the connection_properties. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ pass @abc.abstractmethod def get_search_path(self): """Return the directory where a Connector looks for volumes. Some Connectors need the information in the connection_properties to determine the search path. """ pass @abc.abstractmethod def extend_volume(self, connection_properties): """Update the attached volume's size. This method will attempt to update the local hosts's volume after the volume has been extended on the remote system. The new volume size in bytes will be returned. If there is a failure to update, then None will be returned. :param connection_properties: The volume connection properties. :returns: new size of the volume. """ pass @abc.abstractmethod def get_all_available_volumes(self, connection_properties=None): """Return all volumes that exist in the search directory. At connect_volume time, a Connector looks in a specific directory to discover a volume's paths showing up. This method's job is to return all paths in the directory that connect_volume uses to find a volume. This method is used in coordination with get_volume_paths() to verify that volumes have gone away after disconnect_volume has been called. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ pass def check_IO_handle_valid(self, handle, data_type, protocol): """Check IO handle has correct data type.""" if (handle and not isinstance(handle, data_type)): raise exception.InvalidIOHandleObject( protocol=protocol, actual_type=type(handle)) os-brick-2.3.0/os_brick/initiator/__init__.py0000666000175100017510000000321513230233223021153 0ustar zuulzuul00000000000000# Copyright 2015 OpenStack Foundation # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Brick's Initiator module. The initator module contains the capabilities for discovering the initiator information as well as discovering and removing volumes from a host. """ import re DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 MULTIPATH_ERROR_REGEX = re.compile("\w{3} \d+ \d\d:\d\d:\d\d \|.*$") MULTIPATH_PATH_CHECK_REGEX = re.compile("\s+\d+:\d+:\d+:\d+\s+") PLATFORM_ALL = 'ALL' PLATFORM_x86 = 'X86' PLATFORM_S390 = 'S390' PLATFORM_PPC64 = 'PPC64' OS_TYPE_ALL = 'ALL' OS_TYPE_LINUX = 'LINUX' OS_TYPE_WINDOWS = 'WIN' S390X = "s390x" S390 = "s390" PPC64 = "ppc64" PPC64LE = "ppc64le" ISCSI = "ISCSI" ISER = "ISER" FIBRE_CHANNEL = "FIBRE_CHANNEL" AOE = "AOE" DRBD = "DRBD" NFS = "NFS" SMBFS = 'SMBFS' GLUSTERFS = "GLUSTERFS" LOCAL = "LOCAL" HUAWEISDSHYPERVISOR = "HUAWEISDSHYPERVISOR" HGST = "HGST" RBD = "RBD" SCALEIO = "SCALEIO" SCALITY = "SCALITY" QUOBYTE = "QUOBYTE" DISCO = "DISCO" VZSTORAGE = "VZSTORAGE" SHEEPDOG = "SHEEPDOG" VMDK = "VMDK" GPFS = "GPFS" VERITAS_HYPERSCALE = "VERITAS_HYPERSCALE" STORPOOL = "STORPOOL" NVME = "NVME" os-brick-2.3.0/os_brick/initiator/linuxfc.py0000666000175100017510000003347613230233223021100 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic linux Fibre Channel utilities.""" import errno import os from oslo_concurrency import processutils as putils from oslo_log import log as logging import six from os_brick.initiator import linuxscsi LOG = logging.getLogger(__name__) class LinuxFibreChannel(linuxscsi.LinuxSCSI): def has_fc_support(self): FC_HOST_SYSFS_PATH = '/sys/class/fc_host' if os.path.isdir(FC_HOST_SYSFS_PATH): return True else: return False def _get_hba_channel_scsi_target(self, hba, conn_props): """Try to get the HBA channel and SCSI target for an HBA. This method only works for Fibre Channel targets that implement a single WWNN for all ports, so caller should expect us to return either None or an empty list. :returns: List or None """ # We want the target's WWPNs, so we use the initiator_target_map if # present for this hba or default to target_wwns if not present. wwpns = conn_props['target_wwn'] if 'initiator_target_map' in conn_props: wwpns = conn_props['initiator_target_map'].get(hba['port_name'], wwpns) # If it's not a string then it's an iterable (most likely a list), # so we need to create a BRE for the grep query. if not isinstance(wwpns, six.string_types): wwpns = '\|'.join(wwpns) # Leave only the number from the host_device field (ie: host6) host_device = hba['host_device'] if host_device and len(host_device) > 4: host_device = host_device[4:] path = '/sys/class/fc_transport/target%s:' % host_device # Since we'll run the command in a shell ensure BRE are being used cmd = 'grep -Gil "%(wwpns)s" %(path)s*/port_name' % {'wwpns': wwpns, 'path': path} try: # We need to run command in shell to expand the * glob out, _err = self._execute(cmd, shell=True) return [line.split('/')[4].split(':')[1:] for line in out.split('\n') if line.startswith(path)] except Exception as exc: LOG.debug('Could not get HBA channel and SCSI target ID, path: ' '%(path)s*, reason: %(reason)s', {'path': path, 'reason': exc}) return None def rescan_hosts(self, hbas, connection_properties): target_lun = connection_properties['target_lun'] for hba in hbas: # Try to get HBA channel and SCSI target to use as filters cts = self._get_hba_channel_scsi_target(hba, connection_properties) # If we couldn't get the channel and target use wildcards if not cts: cts = [('-', '-')] for hba_channel, target_id in cts: LOG.debug('Scanning host %(host)s (wwnn: %(wwnn)s, c: ' '%(channel)s, t: %(target)s, l: %(lun)s)', {'host': hba['host_device'], 'wwnn': hba['node_name'], 'channel': hba_channel, 'target': target_id, 'lun': target_lun}) self.echo_scsi_command( "/sys/class/scsi_host/%s/scan" % hba['host_device'], "%(c)s %(t)s %(l)s" % {'c': hba_channel, 't': target_id, 'l': target_lun}) def get_fc_hbas(self): """Get the Fibre Channel HBA information.""" if not self.has_fc_support(): # there is no FC support in the kernel loaded # so there is no need to even try to run systool LOG.debug("No Fibre Channel support detected on system.") return [] out = None try: out, _err = self._execute('systool', '-c', 'fc_host', '-v', run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: # This handles the case where rootwrap is used # and systool is not installed # 96 = nova.cmd.rootwrap.RC_NOEXECFOUND: if exc.exit_code == 96: LOG.warning("systool is not installed") return [] except OSError as exc: # This handles the case where rootwrap is NOT used # and systool is not installed if exc.errno == errno.ENOENT: LOG.warning("systool is not installed") return [] # No FC HBAs were found if out is None: return [] lines = out.split('\n') # ignore the first 2 lines lines = lines[2:] hbas = [] hba = {} lastline = None for line in lines: line = line.strip() # 2 newlines denotes a new hba port if line == '' and lastline == '': if len(hba) > 0: hbas.append(hba) hba = {} else: val = line.split('=') if len(val) == 2: key = val[0].strip().replace(" ", "") value = val[1].strip() hba[key] = value.replace('"', '') lastline = line return hbas def get_fc_hbas_info(self): """Get Fibre Channel WWNs and device paths from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() hbas_info = [] for hba in hbas: wwpn = hba['port_name'].replace('0x', '') wwnn = hba['node_name'].replace('0x', '') device_path = hba['ClassDevicepath'] device = hba['ClassDevice'] hbas_info.append({'port_name': wwpn, 'node_name': wwnn, 'host_device': device, 'device_path': device_path}) return hbas_info def get_fc_wwpns(self): """Get Fibre Channel WWPNs from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() wwpns = [] for hba in hbas: if hba['port_state'] == 'Online': wwpn = hba['port_name'].replace('0x', '') wwpns.append(wwpn) return wwpns def get_fc_wwnns(self): """Get Fibre Channel WWNNs from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() wwnns = [] for hba in hbas: if hba['port_state'] == 'Online': wwnn = hba['node_name'].replace('0x', '') wwnns.append(wwnn) return wwnns class LinuxFibreChannelS390X(LinuxFibreChannel): def get_fc_hbas_info(self): """Get Fibre Channel WWNs and device paths from the system, if any.""" hbas = self.get_fc_hbas() hbas_info = [] for hba in hbas: if hba['port_state'] == 'Online': wwpn = hba['port_name'].replace('0x', '') wwnn = hba['node_name'].replace('0x', '') device_path = hba['ClassDevicepath'] device = hba['ClassDevice'] hbas_info.append({'port_name': wwpn, 'node_name': wwnn, 'host_device': device, 'device_path': device_path}) return hbas_info def configure_scsi_device(self, device_number, target_wwn, lun): """Write the LUN to the port's unit_add attribute. If auto-discovery of Fibre-Channel target ports is disabled on s390 platforms, ports need to be added to the configuration. If auto-discovery of LUNs is disabled on s390 platforms luns need to be added to the configuration through the unit_add interface """ LOG.debug("Configure lun for s390: device_number=%(device_num)s " "target_wwn=%(target_wwn)s target_lun=%(target_lun)s", {'device_num': device_number, 'target_wwn': target_wwn, 'target_lun': lun}) filepath = ("/sys/bus/ccw/drivers/zfcp/%s/%s" % (device_number, target_wwn)) if not (os.path.exists(filepath)): zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/port_rescan" % (device_number)) LOG.debug("port_rescan call for s390: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, "1") except putils.ProcessExecutionError as exc: LOG.warning("port_rescan call for s390 failed exit" " %(code)s, stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/%s/unit_add" % (device_number, target_wwn)) LOG.debug("unit_add call for s390 execute: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, lun) except putils.ProcessExecutionError as exc: LOG.warning("unit_add call for s390 failed exit %(code)s, " "stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) def deconfigure_scsi_device(self, device_number, target_wwn, lun): """Write the LUN to the port's unit_remove attribute. If auto-discovery of LUNs is disabled on s390 platforms luns need to be removed from the configuration through the unit_remove interface """ LOG.debug("Deconfigure lun for s390: " "device_number=%(device_num)s " "target_wwn=%(target_wwn)s target_lun=%(target_lun)s", {'device_num': device_number, 'target_wwn': target_wwn, 'target_lun': lun}) zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/%s/unit_remove" % (device_number, target_wwn)) LOG.debug("unit_remove call for s390 execute: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, lun) except putils.ProcessExecutionError as exc: LOG.warning("unit_remove call for s390 failed exit %(code)s, " "stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) class LinuxFibreChannelPPC64(LinuxFibreChannel): def _get_hba_channel_scsi_target(self, hba, wwpn): """Try to get the HBA channel and SCSI target for an HBA. This method works for Fibre Channel targets iterating over all the target wwpn port and finding the c, t, l. so caller should expect us to return either None or an empty list. """ # Leave only the number from the host_device field (ie: host6) host_device = hba['host_device'] if host_device and len(host_device) > 4: host_device = host_device[4:] path = '/sys/class/fc_transport/target%s:' % host_device cmd = 'grep -il %(wwpn)s %(path)s*/port_name' % {'wwpn': wwpn, 'path': path} try: out, _err = self._execute(cmd, shell=True) return [line.split('/')[4].split(':')[1:] for line in out.split('\n') if line.startswith(path)] except Exception as exc: LOG.error("Could not get HBA channel and SCSI target ID, " "reason: %s", exc) return None def rescan_hosts(self, hbas, target_lun): for hba in hbas: # Try to get HBA channel and SCSI target to use as filters # Ignore HBA which does not have target wwn if 'target_wwn' not in hba.keys(): continue for wwpn in hba['target_wwn']: cts = self._get_hba_channel_scsi_target(hba, wwpn) # If we couldn't get the channel and target use wildcards if not cts: cts = [('-', '-')] for hba_channel, target_id in cts: LOG.debug('Scanning host %(host)s (wwpn: %(wwpn)s, c: ' '%(channel)s, t: %(target)s, l: %(lun)s)', {'host': hba['host_device'], 'wwpn': hba['target_wwn'], 'channel': hba_channel, 'target': target_id, 'lun': target_lun}) self.echo_scsi_command( "/sys/class/scsi_host/%s/scan" % hba['host_device'], "%(c)s %(t)s %(l)s" % {'c': hba_channel, 't': target_id, 'l': target_lun}) os-brick-2.3.0/os_brick/initiator/connector.py0000666000175100017510000002631313230233241021412 0ustar zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Brick Connector objects for each supported transport protocol. .. module: connector The connectors here are responsible for discovering and removing volumes for each of the supported transport protocols. """ import platform import socket import sys from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_utils import importutils from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') # List of connectors to call when getting # the connector properties for a host connector_list = [ 'os_brick.initiator.connectors.base.BaseLinuxConnector', 'os_brick.initiator.connectors.iscsi.ISCSIConnector', 'os_brick.initiator.connectors.fibre_channel.FibreChannelConnector', ('os_brick.initiator.connectors.fibre_channel_s390x.' 'FibreChannelConnectorS390X'), ('os_brick.initiator.connectors.fibre_channel_ppc64.' 'FibreChannelConnectorPPC64'), 'os_brick.initiator.connectors.aoe.AoEConnector', 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', 'os_brick.initiator.connectors.rbd.RBDConnector', 'os_brick.initiator.connectors.local.LocalConnector', 'os_brick.initiator.connectors.gpfs.GPFSConnector', 'os_brick.initiator.connectors.drbd.DRBDConnector', 'os_brick.initiator.connectors.huawei.HuaweiStorHyperConnector', 'os_brick.initiator.connectors.hgst.HGSTConnector', 'os_brick.initiator.connectors.scaleio.ScaleIOConnector', 'os_brick.initiator.connectors.disco.DISCOConnector', 'os_brick.initiator.connectors.vmware.VmdkConnector', 'os_brick.initiator.windows.base.BaseWindowsConnector', 'os_brick.initiator.windows.iscsi.WindowsISCSIConnector', 'os_brick.initiator.windows.fibre_channel.WindowsFCConnector', 'os_brick.initiator.windows.smbfs.WindowsSMBFSConnector', 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', 'os_brick.initiator.connectors.storpool.StorPoolConnector', 'os_brick.initiator.connectors.nvme.NVMeConnector', ] # Mappings used to determine who to contruct in the factory _connector_mapping_linux = { initiator.AOE: 'os_brick.initiator.connectors.aoe.AoEConnector', initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.GLUSTERFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.SCALITY: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.QUOBYTE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.VZSTORAGE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.ISER: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.FIBRE_CHANNEL: 'os_brick.initiator.connectors.fibre_channel.FibreChannelConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.HUAWEISDSHYPERVISOR: 'os_brick.initiator.connectors.huawei.HuaweiStorHyperConnector', initiator.HGST: 'os_brick.initiator.connectors.hgst.HGSTConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.SCALEIO: 'os_brick.initiator.connectors.scaleio.ScaleIOConnector', initiator.DISCO: 'os_brick.initiator.connectors.disco.DISCOConnector', initiator.SHEEPDOG: 'os_brick.initiator.connectors.sheepdog.SheepdogConnector', initiator.VMDK: 'os_brick.initiator.connectors.vmware.VmdkConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', initiator.VERITAS_HYPERSCALE: 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', initiator.STORPOOL: 'os_brick.initiator.connectors.storpool.StorPoolConnector', initiator.NVME: 'os_brick.initiator.connectors.nvme.NVMeConnector', } # Mapping for the S390X platform _connector_mapping_linux_s390x = { initiator.FIBRE_CHANNEL: 'os_brick.initiator.connectors.fibre_channel_s390x.' 'FibreChannelConnectorS390X', initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', } # Mapping for the PPC64 platform _connector_mapping_linux_ppc64 = { initiator.FIBRE_CHANNEL: ('os_brick.initiator.connectors.fibre_channel_ppc64.' 'FibreChannelConnectorPPC64'), initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', initiator.VZSTORAGE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.VERITAS_HYPERSCALE: 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', initiator.ISER: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', } # Mapping for the windows connectors _connector_mapping_windows = { initiator.ISCSI: 'os_brick.initiator.windows.iscsi.WindowsISCSIConnector', initiator.FIBRE_CHANNEL: 'os_brick.initiator.windows.fibre_channel.WindowsFCConnector', initiator.SMBFS: 'os_brick.initiator.windows.smbfs.WindowsSMBFSConnector', } # Create aliases to the old names until 2.0.0 # TODO(smcginnis) Remove this lookup once unit test code is updated to # point to the correct location for item in connector_list: _name = item.split('.')[-1] globals()[_name] = importutils.import_class(item) @utils.trace def get_connector_properties(root_helper, my_ip, multipath, enforce_multipath, host=None, execute=None): """Get the connection properties for all protocols. When the connector wants to use multipath, multipath=True should be specified. If enforce_multipath=True is specified too, an exception is thrown when multipathd is not running. Otherwise, it falls back to multipath=False and only the first path shown up is used. For the compatibility reason, even if multipath=False is specified, some cinder storage drivers may export the target for multipath, which can be found via sendtargets discovery. :param root_helper: The command prefix for executing as root. :type root_helper: str :param my_ip: The IP address of the local host. :type my_ip: str :param multipath: Enable multipath? :type multipath: bool :param enforce_multipath: Should we enforce that the multipath daemon is running? If the daemon isn't running then the return dict will have multipath as False. :type enforce_multipath: bool :param host: hostname. :param execute: execute helper. :returns: dict containing all of the collected initiator values. """ props = {} props['platform'] = platform.machine() props['os_type'] = sys.platform props['ip'] = my_ip props['host'] = host if host else socket.gethostname() for item in connector_list: connector = importutils.import_class(item) if (utils.platform_matches(props['platform'], connector.platform) and utils.os_matches(props['os_type'], connector.os_type)): props = utils.merge_dict(props, connector.get_connector_properties( root_helper, host=host, multipath=multipath, enforce_multipath=enforce_multipath, execute=execute)) return props # TODO(walter-boring) We have to keep this class defined here # so we don't break backwards compatibility class InitiatorConnector(object): @staticmethod def factory(protocol, root_helper, driver=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, arch=None, *args, **kwargs): """Build a Connector object based upon protocol and architecture.""" # We do this instead of assigning it in the definition # to help mocking for unit tests if arch is None: arch = platform.machine() # Set the correct mapping for imports if sys.platform == 'win32': _mapping = _connector_mapping_windows elif arch in (initiator.S390, initiator.S390X): _mapping = _connector_mapping_linux_s390x elif arch in (initiator.PPC64, initiator.PPC64LE): _mapping = _connector_mapping_linux_ppc64 else: _mapping = _connector_mapping_linux LOG.debug("Factory for %(protocol)s on %(arch)s", {'protocol': protocol, 'arch': arch}) protocol = protocol.upper() # set any special kwargs needed by connectors if protocol in (initiator.NFS, initiator.GLUSTERFS, initiator.SCALITY, initiator.QUOBYTE, initiator.VZSTORAGE): kwargs.update({'mount_type': protocol.lower()}) elif protocol == initiator.ISER: kwargs.update({'transport': 'iser'}) # now set all the default kwargs kwargs.update( {'root_helper': root_helper, 'driver': driver, 'use_multipath': use_multipath, 'device_scan_attempts': device_scan_attempts, }) connector = _mapping.get(protocol) if not connector: msg = (_("Invalid InitiatorConnector protocol " "specified %(protocol)s") % dict(protocol=protocol)) raise exception.InvalidConnectorProtocol(msg) conn_cls = importutils.import_class(connector) return conn_cls(*args, **kwargs) os-brick-2.3.0/os_brick/initiator/windows/0000775000175100017510000000000013230233405020533 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/initiator/windows/__init__.py0000666000175100017510000000000013230233223022632 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/initiator/windows/base.py0000666000175100017510000001053713230233223022025 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator import initiator_connector from os_brick import utils LOG = logging.getLogger(__name__) class BaseWindowsConnector(initiator_connector.InitiatorConnector): platform = initiator.PLATFORM_ALL os_type = initiator.OS_TYPE_WINDOWS DEFAULT_DEVICE_SCAN_INTERVAL = 2 def __init__(self, root_helper=None, *args, **kwargs): super(BaseWindowsConnector, self).__init__(root_helper, *args, **kwargs) self.device_scan_interval = kwargs.pop( 'device_scan_interval', self.DEFAULT_DEVICE_SCAN_INTERVAL) self._diskutils = utilsfactory.get_diskutils() @staticmethod def check_multipath_support(enforce_multipath): hostutils = utilsfactory.get_hostutils() mpio_enabled = hostutils.check_server_feature( hostutils.FEATURE_MPIO) if not mpio_enabled: err_msg = _("Using multipath connections for iSCSI and FC disks " "requires the Multipath IO Windows feature to be " "enabled. MPIO must be configured to claim such " "devices.") LOG.error(err_msg) if enforce_multipath: raise exception.BrickException(err_msg) return False return True @staticmethod def get_connector_properties(*args, **kwargs): multipath = kwargs['multipath'] enforce_multipath = kwargs['enforce_multipath'] props = {} props['multipath'] = ( multipath and BaseWindowsConnector.check_multipath_support(enforce_multipath)) return props def _get_scsi_wwn(self, device_number): # NOTE(lpetrut): The Linux connectors use scsi_id to retrieve the # disk unique id, which prepends the identifier type to the unique id # retrieved from the page 83 SCSI inquiry data. We'll do the same # to remain consistent. disk_uid, uid_type = self._diskutils.get_disk_uid_and_uid_type( device_number) scsi_wwn = '%s%s' % (uid_type, disk_uid) return scsi_wwn def check_valid_device(self, path, *args, **kwargs): try: with open(path, 'r') as dev: dev.read(1) except IOError: LOG.exception( "Failed to access the device on the path " "%(path)s", {"path": path}) return False return True def get_all_available_volumes(self): # TODO(lpetrut): query for disks based on the protocol used. return [] def _check_device_paths(self, device_paths): if len(device_paths) > 1: err_msg = _("Multiple volume paths were found: %s. This can " "occur if multipath is used and MPIO is not " "properly configured, thus not claiming the device " "paths. This issue must be addressed urgently as " "it can lead to data corruption.") raise exception.BrickException(err_msg % device_paths) @utils.trace def extend_volume(self, connection_properties): volume_paths = self.get_volume_paths(connection_properties) if not volume_paths: err_msg = _("Could not find the disk. Extend failed.") raise exception.NotFound(err_msg) device_path = volume_paths[0] device_number = self._diskutils.get_device_number_from_device_name( device_path) self._diskutils.refresh_disk(device_number) def get_search_path(self): return None os-brick-2.3.0/os_brick/initiator/windows/smbfs.py0000666000175100017510000000743313230233223022226 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from os_win import utilsfactory from os_brick.initiator.windows import base as win_conn_base from os_brick.remotefs import windows_remotefs as remotefs from os_brick import utils class WindowsSMBFSConnector(win_conn_base.BaseWindowsConnector): def __init__(self, *args, **kwargs): super(WindowsSMBFSConnector, self).__init__(*args, **kwargs) # If this flag is set, we use the local paths in case of local # shares. This is in fact mandatory in some cases, for example # for the Hyper-C scenario. self._local_path_for_loopback = kwargs.get('local_path_for_loopback', False) self._remotefsclient = remotefs.WindowsRemoteFsClient( mount_type='smbfs', *args, **kwargs) self._smbutils = utilsfactory.get_smbutils() @staticmethod def get_connector_properties(*args, **kwargs): # No connector properties updates in this case. return {} @utils.trace def connect_volume(self, connection_properties): self.ensure_share_mounted(connection_properties) disk_path = self._get_disk_path(connection_properties) device_info = {'type': 'file', 'path': disk_path} return device_info @utils.trace def disconnect_volume(self, connection_properties, force=False, ignore_errors=False): export_path = self._get_export_path(connection_properties) self._remotefsclient.unmount(export_path) def _get_export_path(self, connection_properties): return connection_properties['export'].replace('/', '\\') def _get_disk_path(self, connection_properties): # This is expected to be the share address, as an UNC path. export_path = self._get_export_path(connection_properties) mount_base = self._remotefsclient.get_mount_base() use_local_path = (self._local_path_for_loopback and self._smbutils.is_local_share(export_path)) disk_dir = export_path if mount_base: # This will be a symlink pointing to either the share # path directly or to the local share path, if requested # and available. disk_dir = self._remotefsclient.get_mount_point( export_path) elif use_local_path: share_name = self._remotefsclient.get_share_name(export_path) disk_dir = self._remotefsclient.get_local_share_path(share_name) disk_name = connection_properties['name'] disk_path = os.path.join(disk_dir, disk_name) return disk_path def get_search_path(self): return self._remotefsclient.get_mount_base() @utils.trace def get_volume_paths(self, connection_properties): return [self._get_disk_path(connection_properties)] def ensure_share_mounted(self, connection_properties): export_path = self._get_export_path(connection_properties) mount_options = connection_properties.get('options') self._remotefsclient.mount(export_path, mount_options) def extend_volume(self, connection_properties): raise NotImplementedError os-brick-2.3.0/os_brick/initiator/windows/fibre_channel.py0000666000175100017510000001150613230233223023667 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import time from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.initiator.windows import base as win_conn_base from os_brick import utils LOG = logging.getLogger(__name__) class WindowsFCConnector(win_conn_base.BaseWindowsConnector): def __init__(self, *args, **kwargs): super(WindowsFCConnector, self).__init__(*args, **kwargs) self._fc_utils = utilsfactory.get_fc_utils() @staticmethod def get_connector_properties(*args, **kwargs): props = {} fc_utils = utilsfactory.get_fc_utils() fc_utils.refresh_hba_configuration() fc_hba_ports = fc_utils.get_fc_hba_ports() if fc_hba_ports: wwnns = [] wwpns = [] for port in fc_hba_ports: wwnns.append(port['node_name']) wwpns.append(port['port_name']) props['wwpns'] = wwpns props['wwnns'] = list(set(wwnns)) return props @utils.trace def connect_volume(self, connection_properties): volume_paths = self.get_volume_paths(connection_properties) if not volume_paths: raise exception.NoFibreChannelVolumeDeviceFound() device_path = volume_paths[0] device_number = self._diskutils.get_device_number_from_device_name( device_path) scsi_wwn = self._get_scsi_wwn(device_number) device_info = {'type': 'block', 'path': device_path, 'number': device_number, 'scsi_wwn': scsi_wwn} return device_info @utils.trace def get_volume_paths(self, connection_properties): # Returns a list containing at most one disk path such as # \\.\PhysicalDrive4. # # If multipath is used and the MPIO service is properly configured # to claim the disks, we'll still get a single device path, having # the same format, which will be used for all the IO operations. disk_paths = set() for attempt in range(self.device_scan_attempts): self._diskutils.rescan_disks() volume_mappings = self._get_fc_volume_mappings( connection_properties) LOG.debug("Retrieved volume mappings %(vol_mappings)s " "for volume %(conn_props)s", dict(vol_mappings=volume_mappings, conn_props=connection_properties)) # Because of MPIO, we may not be able to get the device name # from a specific mapping if the disk was accessed through # an other HBA at that moment. In that case, the device name # will show up as an empty string. for mapping in volume_mappings: device_name = mapping['device_name'] if device_name: disk_paths.add(device_name) if disk_paths: break time.sleep(self.device_scan_interval) self._check_device_paths(disk_paths) return list(disk_paths) def _get_fc_volume_mappings(self, connection_properties): # Note(lpetrut): All the WWNs returned by os-win are upper case. target_wwpns = [wwpn.upper() for wwpn in connection_properties['target_wwn']] target_lun = connection_properties['target_lun'] volume_mappings = [] hba_mappings = self._get_fc_hba_mappings() for node_name in hba_mappings: target_mappings = self._fc_utils.get_fc_target_mappings(node_name) for mapping in target_mappings: if (mapping['port_name'] in target_wwpns and mapping['lun'] == target_lun): volume_mappings.append(mapping) return volume_mappings def _get_fc_hba_mappings(self): mappings = collections.defaultdict(list) fc_hba_ports = self._fc_utils.get_fc_hba_ports() for port in fc_hba_ports: mappings[port['node_name']].append(port['port_name']) return mappings @utils.trace def disconnect_volume(self, connection_properties, force=False, ignore_errors=False): pass os-brick-2.3.0/os_brick/initiator/windows/iscsi.py0000666000175100017510000001544513230233223022230 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_win import exceptions as os_win_exc from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base_iscsi from os_brick.initiator.windows import base as win_conn_base from os_brick import utils LOG = logging.getLogger(__name__) class WindowsISCSIConnector(win_conn_base.BaseWindowsConnector, base_iscsi.BaseISCSIConnector): def __init__(self, *args, **kwargs): super(WindowsISCSIConnector, self).__init__(*args, **kwargs) self.use_multipath = kwargs.pop('use_multipath', False) self.initiator_list = kwargs.pop('initiator_list', []) self._iscsi_utils = utilsfactory.get_iscsi_initiator_utils() self.validate_initiators() def validate_initiators(self): """Validates the list of requested initiator HBAs Validates the list of requested initiator HBAs to be used when establishing iSCSI sessions. """ valid_initiator_list = True if not self.initiator_list: LOG.info("No iSCSI initiator was explicitly requested. " "The Microsoft iSCSI initiator will choose the " "initiator when establishing sessions.") else: available_initiators = self._iscsi_utils.get_iscsi_initiators() for initiator in self.initiator_list: if initiator not in available_initiators: LOG.warning("The requested initiator %(req_initiator)s " "is not in the list of available initiators: " "%(avail_initiators)s.", dict(req_initiator=initiator, avail_initiators=available_initiators)) valid_initiator_list = False return valid_initiator_list def get_initiator(self): """Returns the iSCSI initiator node name.""" return self._iscsi_utils.get_iscsi_initiator() @staticmethod def get_connector_properties(*args, **kwargs): iscsi_utils = utilsfactory.get_iscsi_initiator_utils() initiator = iscsi_utils.get_iscsi_initiator() return dict(initiator=initiator) def _get_all_paths(self, connection_properties): initiator_list = self.initiator_list or [None] all_targets = self._get_all_targets(connection_properties) paths = [(initiator_name, target_portal, target_iqn, target_lun) for target_portal, target_iqn, target_lun in all_targets for initiator_name in initiator_list] return paths @utils.trace def connect_volume(self, connection_properties): volume_connected = False for (initiator_name, target_portal, target_iqn, target_lun) in self._get_all_paths(connection_properties): try: LOG.info("Attempting to establish an iSCSI session to " "target %(target_iqn)s on portal %(target_portal)s " "accessing LUN %(target_lun)s using initiator " "%(initiator_name)s.", dict(target_portal=target_portal, target_iqn=target_iqn, target_lun=target_lun, initiator_name=initiator_name)) self._iscsi_utils.login_storage_target( target_lun=target_lun, target_iqn=target_iqn, target_portal=target_portal, auth_username=connection_properties.get('auth_username'), auth_password=connection_properties.get('auth_password'), mpio_enabled=self.use_multipath, initiator_name=initiator_name, ensure_lun_available=False) self._iscsi_utils.ensure_lun_available( target_iqn=target_iqn, target_lun=target_lun, rescan_attempts=self.device_scan_attempts, retry_interval=self.device_scan_interval) if not volume_connected: (device_number, device_path) = ( self._iscsi_utils.get_device_number_and_path( target_iqn, target_lun)) volume_connected = True if not self.use_multipath: break except os_win_exc.OSWinException: LOG.exception("Could not establish the iSCSI session.") if not volume_connected: raise exception.BrickException( _("Could not connect volume %s.") % connection_properties) scsi_wwn = self._get_scsi_wwn(device_number) device_info = {'type': 'block', 'path': device_path, 'number': device_number, 'scsi_wwn': scsi_wwn} return device_info @utils.trace def disconnect_volume(self, connection_properties, force=False, ignore_errors=False): # We want to refresh the cached information first. self._diskutils.rescan_disks() for (target_portal, target_iqn, target_lun) in self._get_all_targets(connection_properties): luns = self._iscsi_utils.get_target_luns(target_iqn) # We disconnect the target only if it does not expose other # luns which may be in use. if not luns or luns == [target_lun]: self._iscsi_utils.logout_storage_target(target_iqn) @utils.trace def get_volume_paths(self, connection_properties): device_paths = set() for (target_portal, target_iqn, target_lun) in self._get_all_targets(connection_properties): (device_number, device_path) = self._iscsi_utils.get_device_number_and_path( target_iqn, target_lun) if device_path: device_paths.add(device_path) self._check_device_paths(device_paths) return list(device_paths) os-brick-2.3.0/os_brick/initiator/linuxsheepdog.py0000666000175100017510000001012513230233223022270 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Generic SheepDog Connection Utilities. """ import eventlet import io from oslo_concurrency import processutils from os_brick import exception from os_brick.i18n import _ class SheepdogVolumeIOWrapper(io.RawIOBase): """File-like object with Sheepdog backend.""" def __init__(self, addr, port, volume, snapshot_name=None): self._addr = addr self._port = port self._vdiname = volume self._snapshot_name = snapshot_name self._offset = 0 # SheepdogVolumeIOWrapper instance becomes invalid # if a write error occurs. self._valid = True def _execute(self, cmd, data=None): try: # NOTE(yamada-h): processutils.execute causes busy waiting # under eventlet. # To avoid wasting CPU resources, it should not be used for # the command which takes long time to execute. # For workaround, we replace a subprocess module with # the original one while only executing a read/write command. _processutils_subprocess = processutils.subprocess processutils.subprocess = eventlet.patcher.original('subprocess') return processutils.execute(*cmd, process_input=data)[0] except (processutils.ProcessExecutionError, OSError): self._valid = False raise exception.VolumeDriverException(name=self._vdiname) finally: processutils.subprocess = _processutils_subprocess def read(self, length=None): if not self._valid: raise exception.VolumeDriverException(name=self._vdiname) cmd = ['dog', 'vdi', 'read', '-a', self._addr, '-p', self._port] if self._snapshot_name: cmd.extend(('-s', self._snapshot_name)) cmd.extend((self._vdiname, self._offset)) if length: cmd.append(length) data = self._execute(cmd) self._offset += len(data) return data def write(self, data): if not self._valid: raise exception.VolumeDriverException(name=self._vdiname) length = len(data) cmd = ('dog', 'vdi', 'write', '-a', self._addr, '-p', self._port, self._vdiname, self._offset, length) self._execute(cmd, data) self._offset += length return length def seek(self, offset, whence=0): if not self._valid: raise exception.VolumeDriverException(name=self._vdiname) if whence == 0: # SEEK_SET or 0 - start of the stream (the default); # offset should be zero or positive new_offset = offset elif whence == 1: # SEEK_CUR or 1 - current stream position; offset may be negative new_offset = self._offset + offset else: # SEEK_END or 2 - end of the stream; offset is usually negative # TODO(yamada-h): Support SEEK_END raise IOError(_("Invalid argument - whence=%s not supported.") % whence) if new_offset < 0: raise IOError(_("Invalid argument - negative seek offset.")) self._offset = new_offset def tell(self): return self._offset def flush(self): pass def fileno(self): """Sheepdog does not have support for fileno so we raise IOError. Raising IOError is recommended way to notify caller that interface is not supported - see http://docs.python.org/2/library/io.html#io.IOBase """ raise IOError(_("fileno is not supported by SheepdogVolumeIOWrapper")) os-brick-2.3.0/os_brick/initiator/linuxscsi.py0000666000175100017510000006166713230233223021454 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic linux scsi subsystem and Multipath utilities. Note, this is not iSCSI. """ import glob import os import re import six from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick import executor from os_brick.privileged import rootwrap as priv_rootwrap from os_brick import utils LOG = logging.getLogger(__name__) MULTIPATH_ERROR_REGEX = re.compile("\w{3} \d+ \d\d:\d\d:\d\d \|.*$") MULTIPATH_WWID_REGEX = re.compile("\((?P.+)\)") MULTIPATH_DEVICE_ACTIONS = ['unchanged:', 'reject:', 'reload:', 'switchpg:', 'rename:', 'create:', 'resize:'] class LinuxSCSI(executor.Executor): # As found in drivers/scsi/scsi_lib.c WWN_TYPES = {'t10.': '1', 'eui.': '2', 'naa.': '3'} def echo_scsi_command(self, path, content): """Used to echo strings to scsi subsystem.""" args = ["-a", path] kwargs = dict(process_input=content, run_as_root=True, root_helper=self._root_helper) self._execute('tee', *args, **kwargs) def get_name_from_path(self, path): """Translates /dev/disk/by-path/ entry to /dev/sdX.""" name = os.path.realpath(path) if name.startswith("/dev/"): return name else: return None def remove_scsi_device(self, device, force=False, exc=None): """Removes a scsi device based upon /dev/sdX name.""" path = "/sys/block/%s/device/delete" % device.replace("/dev/", "") if os.path.exists(path): exc = exception.ExceptionChainer() if exc is None else exc # flush any outstanding IO first with exc.context(force, 'Flushing %s failed', device): self.flush_device_io(device) LOG.debug("Remove SCSI device %(device)s with %(path)s", {'device': device, 'path': path}) with exc.context(force, 'Removing %s failed', device): self.echo_scsi_command(path, "1") @utils.retry(exceptions=exception.VolumePathNotRemoved) def wait_for_volumes_removal(self, volumes_names): """Wait for device paths to be removed from the system.""" str_names = ', '.join(volumes_names) LOG.debug('Checking to see if SCSI volumes %s have been removed.', str_names) exist = [volume_name for volume_name in volumes_names if os.path.exists('/dev/' + volume_name)] if exist: LOG.debug('%s still exist.', ', '.join(exist)) raise exception.VolumePathNotRemoved(volume_path=exist) LOG.debug("SCSI volumes %s have been removed.", str_names) def get_device_info(self, device): (out, _err) = self._execute('sg_scan', device, run_as_root=True, root_helper=self._root_helper) dev_info = {'device': device, 'host': None, 'channel': None, 'id': None, 'lun': None} if out: line = out.strip() line = line.replace(device + ": ", "") info = line.split(" ") for item in info: if '=' in item: pair = item.split('=') dev_info[pair[0]] = pair[1] elif 'scsi' in item: dev_info['host'] = item.replace('scsi', '') return dev_info def get_sysfs_wwn(self, device_names): """Return the wwid from sysfs in any of devices in udev format.""" wwid = self.get_sysfs_wwid(device_names) glob_str = '/dev/disk/by-id/scsi-' wwn_paths = glob.glob(glob_str + '*') # If we don't have multiple designators on page 0x83 if wwid and glob_str + wwid in wwn_paths: return wwid # If we have multiple designators follow the symlinks for wwn_path in wwn_paths: try: if os.path.islink(wwn_path) and os.stat(wwn_path): path = os.path.realpath(wwn_path) if path.startswith('/dev/') and path[5:] in device_names: return wwn_path[len(glob_str):] except OSError: continue return '' def get_sysfs_wwid(self, device_names): """Return the wwid from sysfs in any of devices in udev format.""" for device_name in device_names: try: with open('/sys/block/%s/device/wwid' % device_name) as f: wwid = f.read().strip() except IOError: continue # The sysfs wwid has the wwn type in string format as a prefix, # but udev uses its numerical representation as returned by # scsi_id's page 0x83, so we need to map it udev_wwid = self.WWN_TYPES.get(wwid[:4], '8') + wwid[4:] return udev_wwid return '' def get_scsi_wwn(self, path): """Read the WWN from page 0x83 value for a SCSI device.""" (out, _err) = self._execute('/lib/udev/scsi_id', '--page', '0x83', '--whitelisted', path, run_as_root=True, root_helper=self._root_helper) return out.strip() @staticmethod def is_multipath_running(enforce_multipath, root_helper, execute=None): try: if execute is None: execute = priv_rootwrap.execute execute('multipathd', 'show', 'status', run_as_root=True, root_helper=root_helper) except putils.ProcessExecutionError as err: LOG.error('multipathd is not running: exit code %(err)s', {'err': err.exit_code}) if enforce_multipath: raise return False return True def get_dm_name(self, dm): """Get the Device map name given the device name of the dm on sysfs. :param dm: Device map name as seen in sysfs. ie: 'dm-0' :returns: String with the name, or empty string if not available. ie: '36e843b658476b7ed5bc1d4d10d9b1fde' """ try: with open('/sys/block/' + dm + '/dm/name') as f: return f.read().strip() except IOError: return '' def find_sysfs_multipath_dm(self, device_names): """Find the dm device name given a list of device names :param device_names: Iterable with device names, not paths. ie: ['sda'] :returns: String with the dm name or None if not found. ie: 'dm-0' """ glob_str = '/sys/block/%s/holders/dm-*' for dev_name in device_names: dms = glob.glob(glob_str % dev_name) if dms: __, device_name, __, dm = dms[0].rsplit('/', 3) return dm return None def remove_connection(self, devices_names, is_multipath, force=False, exc=None): """Remove LUNs and multipath associated with devices names. :param devices_names: Iterable with real device names ('sda', 'sdb') :param is_multipath: Whether this is a multipath connection or not :param force: Whether to forcefully disconnect even if flush fails. :param exc: ExceptionChainer where to add exceptions if forcing :returns: Multipath device map name if found and not flushed """ if not devices_names: return multipath_name = None exc = exception.ExceptionChainer() if exc is None else exc LOG.debug('Removing %(type)s devices %(devices)s', {'type': 'multipathed' if is_multipath else 'single pathed', 'devices': ', '.join(devices_names)}) if is_multipath: multipath_dm = self.find_sysfs_multipath_dm(devices_names) multipath_name = multipath_dm and self.get_dm_name(multipath_dm) if multipath_name: with exc.context(force, 'Flushing %s failed', multipath_name): self.flush_multipath_device(multipath_name) multipath_name = None for device_name in devices_names: self.remove_scsi_device('/dev/' + device_name, force, exc) # Wait until the symlinks are removed with exc.context(force, 'Some devices remain from %s', devices_names): try: self.wait_for_volumes_removal(devices_names) finally: # Since we use /dev/disk/by-id/scsi- links to get the wwn we # must ensure they are always removed. self._remove_scsi_symlinks(devices_names) return multipath_name def _remove_scsi_symlinks(self, devices_names): devices = ['/dev/' + dev for dev in devices_names] links = glob.glob('/dev/disk/by-id/scsi-*') unlink = [] for link in links: try: if os.path.realpath(link) in devices: unlink.append(link) except OSError: # A race condition in Python's posixpath:realpath just occurred # so we can ignore it because the file was just removed between # a check if file exists and a call to os.readlink continue if unlink: priv_rootwrap.unlink_root(no_errors=True, *unlink) def flush_device_io(self, device): """This is used to flush any remaining IO in the buffers.""" if os.path.exists(device): try: # NOTE(geguileo): With 30% connection error rates flush can get # stuck, set timeout to prevent it from hanging here forever. # Retry twice after 20 and 40 seconds. LOG.debug("Flushing IO for device %s", device) self._execute('blockdev', '--flushbufs', device, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: LOG.warning("Failed to flush IO buffers prior to removing " "device: %(code)s", {'code': exc.exit_code}) raise def flush_multipath_device(self, device_map_name): LOG.debug("Flush multipath device %s", device_map_name) # NOTE(geguileo): With 30% connection error rates flush can get stuck, # set timeout to prevent it from hanging here forever. Retry twice # after 20 and 40 seconds. self._execute('multipath', '-f', device_map_name, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self._root_helper) @utils.retry(exceptions=exception.VolumeDeviceNotFound) def wait_for_path(self, volume_path): """Wait for a path to show up.""" LOG.debug("Checking to see if %s exists yet.", volume_path) if not os.path.exists(volume_path): LOG.debug("%(path)s doesn't exists yet.", {'path': volume_path}) raise exception.VolumeDeviceNotFound( device=volume_path) else: LOG.debug("%s has shown up.", volume_path) @utils.retry(exceptions=exception.BlockDeviceReadOnly, retries=5) def wait_for_rw(self, wwn, device_path): """Wait for block device to be Read-Write.""" LOG.debug("Checking to see if %s is read-only.", device_path) out, info = self._execute('lsblk', '-o', 'NAME,RO', '-l', '-n') LOG.debug("lsblk output: %s", out) blkdevs = out.splitlines() for blkdev in blkdevs: # Entries might look like: # # "3624a93709a738ed78583fd120013902b (dm-1) 1" # # or # # "sdd 0" # # We are looking for the first and last part of them. For FC # multipath devices the name is in the format of ' (dm-)' blkdev_parts = blkdev.split(' ') ro = blkdev_parts[-1] name = blkdev_parts[0] # We must validate that all pieces of the dm-# device are rw, # if some are still ro it can cause problems. if wwn in name and int(ro) == 1: LOG.debug("Block device %s is read-only", device_path) self._execute('multipath', '-r', check_exit_code=[0, 1, 21], run_as_root=True, root_helper=self._root_helper) raise exception.BlockDeviceReadOnly( device=device_path) else: LOG.debug("Block device %s is not read-only.", device_path) def find_multipath_device_path(self, wwn): """Look for the multipath device file for a volume WWN. Multipath devices can show up in several places on a linux system. 1) When multipath friendly names are ON: a device file will show up in /dev/disk/by-id/dm-uuid-mpath- /dev/disk/by-id/dm-name-mpath /dev/disk/by-id/scsi-mpath /dev/mapper/mpath 2) When multipath friendly names are OFF: /dev/disk/by-id/dm-uuid-mpath- /dev/disk/by-id/scsi- /dev/mapper/ """ LOG.info("Find Multipath device file for volume WWN %(wwn)s", {'wwn': wwn}) # First look for the common path wwn_dict = {'wwn': wwn} path = "/dev/disk/by-id/dm-uuid-mpath-%(wwn)s" % wwn_dict try: self.wait_for_path(path) return path except exception.VolumeDeviceNotFound: pass # for some reason the common path wasn't found # lets try the dev mapper path path = "/dev/mapper/%(wwn)s" % wwn_dict try: self.wait_for_path(path) return path except exception.VolumeDeviceNotFound: pass # couldn't find a path LOG.warning("couldn't find a valid multipath device path for " "%(wwn)s", wwn_dict) return None def find_multipath_device(self, device): """Discover multipath devices for a mpath device. This uses the slow multipath -l command to find a multipath device description, then screen scrapes the output to discover the multipath device name and it's devices. """ mdev = None devices = [] out = None try: (out, _err) = self._execute('multipath', '-l', device, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: LOG.warning("multipath call failed exit %(code)s", {'code': exc.exit_code}) raise exception.CommandExecutionFailed( cmd='multipath -l %s' % device) if out: lines = out.strip() lines = lines.split("\n") lines = [line for line in lines if not re.match(MULTIPATH_ERROR_REGEX, line)] if lines: mdev_name = lines[0].split(" ")[0] if mdev_name in MULTIPATH_DEVICE_ACTIONS: mdev_name = lines[0].split(" ")[1] mdev = '/dev/mapper/%s' % mdev_name # Confirm that the device is present. try: os.stat(mdev) except OSError: LOG.warning("Couldn't find multipath device %s", mdev) return None wwid_search = MULTIPATH_WWID_REGEX.search(lines[0]) if wwid_search is not None: mdev_id = wwid_search.group('wwid') else: mdev_id = mdev_name LOG.debug("Found multipath device = %(mdev)s", {'mdev': mdev}) device_lines = lines[3:] for dev_line in device_lines: if dev_line.find("policy") != -1: continue dev_line = dev_line.lstrip(' |-`') dev_info = dev_line.split() address = dev_info[0].split(":") dev = {'device': '/dev/%s' % dev_info[1], 'host': address[0], 'channel': address[1], 'id': address[2], 'lun': address[3] } devices.append(dev) if mdev is not None: info = {"device": mdev, "id": mdev_id, "name": mdev_name, "devices": devices} return info return None def get_device_size(self, device): """Get the size in bytes of a volume.""" (out, _err) = self._execute('blockdev', '--getsize64', device, run_as_root=True, root_helper=self._root_helper) var = six.text_type(out.strip()) if var.isnumeric(): return int(var) else: return None def multipath_reconfigure(self): """Issue a multipathd reconfigure. When attachments come and go, the multipathd seems to get lost and not see the maps. This causes resize map to fail 100%. To overcome this we have to issue a reconfigure prior to resize map. """ (out, _err) = self._execute('multipathd', 'reconfigure', run_as_root=True, root_helper=self._root_helper) return out def multipath_resize_map(self, mpath_id): """Issue a multipath resize map on device. This forces the multipath daemon to update it's size information a particular multipath device. """ (out, _err) = self._execute('multipathd', 'resize', 'map', mpath_id, run_as_root=True, root_helper=self._root_helper) return out def extend_volume(self, volume_paths): """Signal the SCSI subsystem to test for volume resize. This function tries to signal the local system's kernel that an already attached volume might have been resized. """ LOG.debug("extend volume %s", volume_paths) for volume_path in volume_paths: device = self.get_device_info(volume_path) LOG.debug("Volume device info = %s", device) device_id = ("%(host)s:%(channel)s:%(id)s:%(lun)s" % {'host': device['host'], 'channel': device['channel'], 'id': device['id'], 'lun': device['lun']}) scsi_path = ("/sys/bus/scsi/drivers/sd/%(device_id)s" % {'device_id': device_id}) size = self.get_device_size(volume_path) LOG.debug("Starting size: %s", size) # now issue the device rescan rescan_path = "%(scsi_path)s/rescan" % {'scsi_path': scsi_path} self.echo_scsi_command(rescan_path, "1") new_size = self.get_device_size(volume_path) LOG.debug("volume size after scsi device rescan %s", new_size) scsi_wwn = self.get_scsi_wwn(volume_paths[0]) mpath_device = self.find_multipath_device_path(scsi_wwn) if mpath_device: # Force a reconfigure so that resize works self.multipath_reconfigure() size = self.get_device_size(mpath_device) LOG.info("mpath(%(device)s) current size %(size)s", {'device': mpath_device, 'size': size}) result = self.multipath_resize_map(scsi_wwn) if 'fail' in result: LOG.error("Multipathd failed to update the size mapping of " "multipath device %(scsi_wwn)s volume %(volume)s", {'scsi_wwn': scsi_wwn, 'volume': volume_paths}) return None new_size = self.get_device_size(mpath_device) LOG.info("mpath(%(device)s) new size %(size)s", {'device': mpath_device, 'size': new_size}) return new_size def process_lun_id(self, lun_ids): if isinstance(lun_ids, list): processed = [] for x in lun_ids: x = self._format_lun_id(x) processed.append(x) else: processed = self._format_lun_id(lun_ids) return processed def _format_lun_id(self, lun_id): # make sure lun_id is an int lun_id = int(lun_id) if lun_id < 256: return lun_id else: return ("0x%04x%04x00000000" % (lun_id & 0xffff, lun_id >> 16 & 0xffff)) def get_hctl(self, session, lun): """Given an iSCSI session return the host, channel, target, and lun.""" glob_str = '/sys/class/iscsi_host/host*/device/session' + session paths = glob.glob(glob_str + '/target*') if paths: __, channel, target = os.path.split(paths[0])[1].split(':') # Check if we can get the host else: target = channel = '-' paths = glob.glob(glob_str) if not paths: LOG.debug('No hctl found on session %s with lun %s', session, lun) return None # Extract the host number from the path host = paths[0][26:paths[0].index('/', 26)] res = (host, channel, target, lun) LOG.debug('HCTL %s found on session %s with lun %s', res, session, lun) return res def device_name_by_hctl(self, session, hctl): """Find the device name given a session and the hctl. :param session: A string with the session number "param hctl: An iterable with the host, channel, target, and lun as passed to scan. ie: ('5', '-', '-', '0') """ if '-' in hctl: hctl = ['*' if x == '-' else x for x in hctl] path = ('/sys/class/scsi_host/host%(h)s/device/session%(s)s/target' '%(h)s:%(c)s:%(t)s/%(h)s:%(c)s:%(t)s:%(l)s/block/*' % {'h': hctl[0], 'c': hctl[1], 't': hctl[2], 'l': hctl[3], 's': session}) # Sort devices and return the first so we don't return a partition devices = sorted(glob.glob(path)) device = os.path.split(devices[0])[1] if devices else None LOG.debug('Searching for a device in session %s and hctl %s yield: %s', session, hctl, device) return device def scan_iscsi(self, host, channel='-', target='-', lun='-'): """Send an iSCSI scan request given the host and optionally the ctl.""" LOG.debug('Scanning host %(host)s c: %(channel)s, ' 't: %(target)s, l: %(lun)s)', {'host': host, 'channel': channel, 'target': target, 'lun': lun}) self.echo_scsi_command('/sys/class/scsi_host/host%s/scan' % host, '%(c)s %(t)s %(l)s' % {'c': channel, 't': target, 'l': lun}) def multipath_add_wwid(self, wwid): """Add a wwid to the list of know multipath wwids. This has the effect of multipathd being willing to create a dm for a multipath even when there's only 1 device. """ out, err = self._execute('multipath', '-a', wwid, run_as_root=True, check_exit_code=False, root_helper=self._root_helper) return out.strip() == "wwid '" + wwid + "' added" def multipath_add_path(self, realpath): """Add a path to multipathd for monitoring. This has the effect of multipathd checking an already checked device for multipath. Together with `multipath_add_wwid` we can create a multipath when there's only 1 path. """ stdout, stderr = self._execute('multipathd', 'add', 'path', realpath, run_as_root=True, timeout=5, check_exit_code=False, root_helper=self._root_helper) return stdout.strip() == 'ok' os-brick-2.3.0/os_brick/initiator/host_driver.py0000666000175100017510000000215513230233223021746 0ustar zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import errno import os class HostDriver(object): def get_all_block_devices(self): """Get the list of all block devices seen in /dev/disk/by-path/.""" dir = "/dev/disk/by-path/" try: files = os.listdir(dir) except OSError as e: if e.errno == errno.ENOENT: files = [] else: raise devices = [] for file in files: devices.append(dir + file) return devices os-brick-2.3.0/os_brick/initiator/linuxrbd.py0000666000175100017510000001571513230233223021253 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may not # use this file except in compliance with the License. You may obtain a copy of # the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations under # the License. """Generic RBD connection utilities.""" import io from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import utils try: import rados import rbd except ImportError: rados = None rbd = None LOG = logging.getLogger(__name__) class RBDClient(object): def __init__(self, user, pool, *args, **kwargs): self.rbd_user = user self.rbd_pool = pool for attr in ['rbd_user', 'rbd_pool']: val = getattr(self, attr) if val is not None: setattr(self, attr, utils.convert_str(val)) # allow these to be overridden for testing self.rados = kwargs.get('rados', rados) self.rbd = kwargs.get('rbd', rbd) if self.rados is None: raise exception.InvalidParameterValue( err=_('rados module required')) if self.rbd is None: raise exception.InvalidParameterValue( err=_('rbd module required')) self.rbd_conf = kwargs.get('conffile', '/etc/ceph/ceph.conf') self.rbd_cluster_name = kwargs.get('rbd_cluster_name', 'ceph') self.rados_connect_timeout = kwargs.get('rados_connect_timeout', -1) self.client, self.ioctx = self.connect() def __enter__(self): return self def __exit__(self, type_, value, traceback): self.disconnect() def connect(self): LOG.debug("opening connection to ceph cluster (timeout=%s).", self.rados_connect_timeout) client = self.rados.Rados(rados_id=self.rbd_user, clustername=self.rbd_cluster_name, conffile=self.rbd_conf) try: if self.rados_connect_timeout >= 0: client.connect( timeout=self.rados_connect_timeout) else: client.connect() ioctx = client.open_ioctx(self.rbd_pool) return client, ioctx except self.rados.Error: msg = _("Error connecting to ceph cluster.") LOG.exception(msg) # shutdown cannot raise an exception client.shutdown() raise exception.BrickException(message=msg) def disconnect(self): # closing an ioctx cannot raise an exception self.ioctx.close() self.client.shutdown() class RBDVolume(object): """Context manager for dealing with an existing rbd volume.""" def __init__(self, client, name, snapshot=None, read_only=False): if snapshot is not None: snapshot = utils.convert_str(snapshot) try: self.image = client.rbd.Image(client.ioctx, utils.convert_str(name), snapshot=snapshot, read_only=read_only) except client.rbd.Error: LOG.exception("error opening rbd image %s", name) client.disconnect() raise # Ceph provides rbd.so to cinder, but we can't # get volume name from rbd.Image, so, we record # name here, so other modules can easily get # volume name. self.name = name self.client = client def close(self): try: self.image.close() finally: self.client.disconnect() def __enter__(self): return self def __exit__(self, type_, value, traceback): self.close() def __getattr__(self, attrib): return getattr(self.image, attrib) class RBDImageMetadata(object): """RBD image metadata to be used with RBDVolumeIOWrapper.""" def __init__(self, image, pool, user, conf): self.image = image self.pool = utils.convert_str(pool or '') self.user = utils.convert_str(user or '') self.conf = utils.convert_str(conf or '') class RBDVolumeIOWrapper(io.RawIOBase): """Enables LibRBD.Image objects to be treated as Python IO objects. Calling unimplemented interfaces will raise IOError. """ def __init__(self, rbd_volume): super(RBDVolumeIOWrapper, self).__init__() self._rbd_volume = rbd_volume self._offset = 0 def _inc_offset(self, length): self._offset += length @property def rbd_image(self): return self._rbd_volume.image @property def rbd_user(self): return self._rbd_volume.user @property def rbd_pool(self): return self._rbd_volume.pool @property def rbd_conf(self): return self._rbd_volume.conf def read(self, length=None): offset = self._offset total = self._rbd_volume.image.size() # NOTE(dosaboy): posix files do not barf if you read beyond their # length (they just return nothing) but rbd images do so we need to # return empty string if we have reached the end of the image. if (offset >= total): return b'' if length is None: length = total if (offset + length) > total: length = total - offset self._inc_offset(length) return self._rbd_volume.image.read(int(offset), int(length)) def write(self, data): self._rbd_volume.image.write(data, self._offset) self._inc_offset(len(data)) def seekable(self): return True def seek(self, offset, whence=0): if whence == 0: new_offset = offset elif whence == 1: new_offset = self._offset + offset elif whence == 2: new_offset = self._rbd_volume.image.size() new_offset += offset else: raise IOError(_("Invalid argument - whence=%s not supported") % (whence)) if (new_offset < 0): raise IOError(_("Invalid argument")) self._offset = new_offset def tell(self): return self._offset def flush(self): try: self._rbd_volume.image.flush() except AttributeError: LOG.warning("flush() not supported in this version of librbd") def fileno(self): """RBD does not have support for fileno() so we raise IOError. Raising IOError is recommended way to notify caller that interface is not supported - see http://docs.python.org/2/library/io.html#io.IOBase """ raise IOError(_("fileno() not supported by RBD()")) def close(self): self.rbd_image.close() os-brick-2.3.0/os_brick/initiator/connectors/0000775000175100017510000000000013230233405021216 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/initiator/connectors/remotefs.py0000666000175100017510000001120013230233223023406 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.remotefs import remotefs from os_brick import utils LOG = logging.getLogger(__name__) class RemoteFsConnector(base.BaseLinuxConnector): """Connector class to attach/detach NFS and GlusterFS volumes.""" def __init__(self, mount_type, root_helper, driver=None, execute=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): kwargs = kwargs or {} conn = kwargs.get('conn') mount_type_lower = mount_type.lower() if conn: mount_point_base = conn.get('mount_point_base') if mount_type_lower in ('nfs', 'glusterfs', 'scality', 'quobyte', 'vzstorage'): kwargs[mount_type_lower + '_mount_point_base'] = ( kwargs.get(mount_type_lower + '_mount_point_base') or mount_point_base) else: LOG.warning("Connection details not present." " RemoteFsClient may not initialize properly.") if mount_type_lower == 'scality': cls = remotefs.ScalityRemoteFsClient elif mount_type_lower == 'vzstorage': cls = remotefs.VZStorageRemoteFSClient else: cls = remotefs.RemoteFsClient self._remotefsclient = cls(mount_type, root_helper, execute=execute, *args, **kwargs) super(RemoteFsConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The RemoteFS connector properties.""" return {} def set_execute(self, execute): super(RemoteFsConnector, self).set_execute(execute) self._remotefsclient.set_execute(execute) def get_search_path(self): return self._remotefsclient.get_mount_base() def _get_volume_path(self, connection_properties): mnt_flags = [] if connection_properties.get('options'): mnt_flags = connection_properties['options'].split() nfs_share = connection_properties['export'] self._remotefsclient.mount(nfs_share, mnt_flags) mount_point = self._remotefsclient.get_mount_point(nfs_share) path = mount_point + '/' + connection_properties['name'] return path def get_volume_paths(self, connection_properties): path = self._get_volume_path(connection_properties) return [path] @utils.trace def connect_volume(self, connection_properties): """Ensure that the filesystem containing the volume is mounted. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: export - remote filesystem device (e.g. '172.18.194.100:/var/nfs') name - file name within the filesystem :type connection_properties: dict :returns: dict connection_properties may optionally include: options - options to pass to mount """ path = self._get_volume_path(connection_properties) return {'path': path} @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """No need to do anything to disconnect a volume in a filesystem. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/fibre_channel_s390x.py0000666000175100017510000000731313230233223025321 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import fibre_channel from os_brick.initiator import linuxfc LOG = logging.getLogger(__name__) class FibreChannelConnectorS390X(fibre_channel.FibreChannelConnector): """Connector class to attach/detach Fibre Channel volumes on S390X arch.""" platform = initiator.PLATFORM_S390 def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(FibreChannelConnectorS390X, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) LOG.debug("Initializing Fibre Channel connector for S390") self._linuxfc = linuxfc.LinuxFibreChannelS390X(root_helper, execute) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnectorS390X, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) def _get_host_devices(self, possible_devs, lun): host_devices = [] for pci_num, target_wwn in possible_devs: host_device = self._get_device_file_path( pci_num, target_wwn, lun) # NOTE(arne_r) # LUN driver path is the same on all distros, so no need to have # multiple calls here self._linuxfc.configure_scsi_device(pci_num, target_wwn, self._get_lun_string(lun)) host_devices.extend(host_device) return host_devices def _get_lun_string(self, lun): target_lun = 0 if lun <= 0xffff: target_lun = "0x%04x000000000000" % lun elif lun <= 0xffffffff: target_lun = "0x%08x00000000" % lun return target_lun def _get_device_file_path(self, pci_num, target_wwn, lun): # NOTE(arne_r) # Need to add two possible ways to resolve device paths, # depending on OS. Since it gets passed to '_get_possible_volume_paths' # having a mismatch is not a problem host_device = [ "/dev/disk/by-path/ccw-%s-zfcp-%s:%s" % ( pci_num, target_wwn, self._get_lun_string(lun)), "/dev/disk/by-path/ccw-%s-fc-%s-lun-%s" % ( pci_num, target_wwn, lun), ] return host_device def _remove_devices(self, connection_properties, devices): hbas = self._linuxfc.get_fc_hbas_info() ports = connection_properties['target_wwn'] possible_devs = self._get_possible_devices(hbas, ports) lun = connection_properties.get('target_lun', 0) target_lun = self._get_lun_string(lun) for pci_num, target_wwn in possible_devs: self._linuxfc.deconfigure_scsi_device(pci_num, target_wwn, target_lun) os-brick-2.3.0/os_brick/initiator/connectors/nvme.py0000666000175100017510000001547613230233223022552 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import re import time from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') class NVMeConnector(base.BaseLinuxConnector): """Connector class to attach/detach NVMe over fabric volumes.""" def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(NVMeConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The NVMe connector properties.""" return {} def get_search_path(self): return '/dev' def get_volume_paths(self, connection_properties): path = connection_properties['device_path'] LOG.debug("Path of volume to be extended is %(path)s", {'path': path}) return [path] def _get_nvme_devices(self): nvme_devices = [] pattern = r'/dev/nvme[0-9]n[0-9]' cmd = ['nvme', 'list'] try: (out, err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) for line in out.split('\n'): result = re.match(pattern, line) if result: nvme_devices.append(result.group(0)) LOG.debug("_get_nvme_devices returned %(nvme_devices)s", {'nvme_devices': nvme_devices}) return nvme_devices except putils.ProcessExecutionError: LOG.error( "Failed to list available NVMe connected controllers.") raise @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Discover and attach the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: nqn - NVMe subsystem name to the volume to be connected target_port - NVMe target port that hosts the nqn sybsystem target_portal - NVMe target ip that hosts the nqn sybsystem :type connection_properties: dict :returns: dict """ current_nvme_devices = self._get_nvme_devices() device_info = {'type': 'block'} conn_nqn = connection_properties['nqn'].split('.', 1)[1] target_portal = connection_properties['target_portal'] port = connection_properties['target_port'] nvme_transport_type = connection_properties['transport_type'] cmd = [ 'nvme', 'connect', '-t', nvme_transport_type, '-n', conn_nqn, '-a', target_portal, '-s', port] try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError: LOG.error( "Failed to connect to NVMe nqn " "%(conn_nqn)s", {'conn_nqn': conn_nqn}) raise for retry in range(1, self.device_scan_attempts + 1): all_nvme_devices = self._get_nvme_devices() path = set(all_nvme_devices) - set(current_nvme_devices) if path: break time.sleep(retry ** 2) else: LOG.error("Failed to retrieve available connected NVMe " "controllers when running nvme list") raise exception.TargetPortalNotFound(target_portal) path = list(path) LOG.debug("all_nvme_devices are %(all_nvme_devices)s", {'all_nvme_devices': all_nvme_devices}) device_info['path'] = path[0] LOG.debug("NVMe device to be connected to is %(path)s", {'path': path[0]}) return device_info @utils.trace @synchronized('disconnect_volume') def disconnect_volume(self, connection_properties, device_info): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: device_path - path to the volume to be connected :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ conn_nqn = connection_properties['nqn'] device_path = connection_properties['device_path'] LOG.debug( "Trying to disconnect from NVMe nqn " "%(conn_nqn)s with device_path %(device_path)s", {'conn_nqn': conn_nqn, 'device_path': device_path}) cmd = [ 'nvme', 'disconnect', '-d', device_path] try: self._execute( *cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError: LOG.error( "Failed to disconnect from NVMe nqn " "%(conn_nqn)s with device_path %(device_path)s", {'conn_nqn': conn_nqn, 'device_path': device_path}) raise @utils.trace @synchronized('extend_volume') def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an LVM volume. """ volume_paths = self.get_volume_paths(connection_properties) if volume_paths: return self._linuxscsi.extend_volume(volume_paths) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': connection_properties}) raise exception.VolumePathsNotFound() os-brick-2.3.0/os_brick/initiator/connectors/__init__.py0000666000175100017510000000000013230233223023315 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/initiator/connectors/rbd.py0000666000175100017510000002354613230233223022351 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import fileutils from oslo_utils import netutils from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator import linuxrbd from os_brick import utils LOG = logging.getLogger(__name__) class RBDConnector(base.BaseLinuxConnector): """"Connector class to attach/detach RBD volumes.""" def __init__(self, root_helper, driver=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(RBDConnector, self).__init__(root_helper, driver=driver, device_scan_attempts= device_scan_attempts, *args, **kwargs) self.do_local_attach = kwargs.get('do_local_attach', False) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The RBD connector properties.""" return {'do_local_attach': kwargs.get('do_local_attach', False)} def get_volume_paths(self, connection_properties): # TODO(e0ne): Implement this for local volume. return [] def get_search_path(self): # TODO(walter-boring): don't know where the connector # looks for RBD volumes. return None def get_all_available_volumes(self, connection_properties=None): # TODO(e0ne): Implement this for local volume. return [] def _sanitize_mon_hosts(self, hosts): def _sanitize_host(host): if netutils.is_valid_ipv6(host): host = '[%s]' % host return host return list(map(_sanitize_host, hosts)) def _check_or_get_keyring_contents(self, keyring, cluster_name, user): try: if keyring is None: keyring_path = ("/etc/ceph/%s.client.%s.keyring" % (cluster_name, user)) with open(keyring_path, 'r') as keyring_file: keyring = keyring_file.read() return keyring except IOError: msg = (_("Keyring path %s is not readable.") % (keyring_path)) raise exception.BrickException(msg=msg) def _create_ceph_conf(self, monitor_ips, monitor_ports, cluster_name, user, keyring): monitors = ["%s:%s" % (ip, port) for ip, port in zip(self._sanitize_mon_hosts(monitor_ips), monitor_ports)] mon_hosts = "mon_host = %s" % (','.join(monitors)) keyring = self._check_or_get_keyring_contents(keyring, cluster_name, user) try: fd, ceph_conf_path = tempfile.mkstemp(prefix="brickrbd_") with os.fdopen(fd, 'w') as conf_file: conf_file.writelines([mon_hosts, "\n", keyring, "\n"]) return ceph_conf_path except IOError: msg = (_("Failed to write data to %s.") % (ceph_conf_path)) raise exception.BrickException(msg=msg) def _get_rbd_handle(self, connection_properties): try: user = connection_properties['auth_username'] pool, volume = connection_properties['name'].split('/') cluster_name = connection_properties.get('cluster_name') monitor_ips = connection_properties.get('hosts') monitor_ports = connection_properties.get('ports') keyring = connection_properties.get('keyring') except IndexError: msg = _("Connect volume failed, malformed connection properties") raise exception.BrickException(msg=msg) conf = self._create_ceph_conf(monitor_ips, monitor_ports, str(cluster_name), user, keyring) try: rbd_client = linuxrbd.RBDClient(user, pool, conffile=conf, rbd_cluster_name=str(cluster_name)) rbd_volume = linuxrbd.RBDVolume(rbd_client, volume) rbd_handle = linuxrbd.RBDVolumeIOWrapper( linuxrbd.RBDImageMetadata(rbd_volume, pool, user, conf)) except Exception: fileutils.delete_if_exists(conf) raise return rbd_handle def _get_rbd_args(self, connection_properties): try: user = connection_properties['auth_username'] monitor_ips = connection_properties.get('hosts') monitor_ports = connection_properties.get('ports') except KeyError: msg = _("Connect volume failed, malformed connection properties") raise exception.BrickException(msg=msg) args = ['--id', user] if monitor_ips and monitor_ports: monitors = ["%s:%s" % (ip, port) for ip, port in zip( self._sanitize_mon_hosts(monitor_ips), monitor_ports)] for monitor in monitors: args += ['--mon_host', monitor] return args @staticmethod def get_rbd_device_name(pool, volume): """Return device name which will be generated by RBD kernel module. :param pool: RBD pool name. :type pool: string :param volume: RBD image name. :type volume: string """ return '/dev/rbd/{pool}/{volume}'.format(pool=pool, volume=volume) @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ do_local_attach = connection_properties.get('do_local_attach', self.do_local_attach) if do_local_attach: # NOTE(e0ne): sanity check if ceph-common is installed. cmd = ['which', 'rbd'] try: self._execute(*cmd) except putils.ProcessExecutionError: msg = _("ceph-common package is not installed.") LOG.error(msg) raise exception.BrickException(message=msg) # NOTE(e0ne): map volume to a block device # via the rbd kernel module. pool, volume = connection_properties['name'].split('/') rbd_dev_path = RBDConnector.get_rbd_device_name(pool, volume) if (not os.path.islink(rbd_dev_path) or not os.path.exists(os.path.realpath(rbd_dev_path))): cmd = ['rbd', 'map', volume, '--pool', pool] cmd += self._get_rbd_args(connection_properties) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) else: LOG.debug('volume %(vol)s is already mapped to local' ' device %(dev)s', {'vol': volume, 'dev': os.path.realpath(rbd_dev_path)}) return {'path': rbd_dev_path, 'type': 'block'} rbd_handle = self._get_rbd_handle(connection_properties) return {'path': rbd_handle} @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ do_local_attach = connection_properties.get('do_local_attach', self.do_local_attach) if do_local_attach: pool, volume = connection_properties['name'].split('/') dev_name = RBDConnector.get_rbd_device_name(pool, volume) cmd = ['rbd', 'unmap', dev_name] cmd += self._get_rbd_args(connection_properties) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) else: if device_info: rbd_handle = device_info.get('path', None) if rbd_handle is not None: fileutils.delete_if_exists(rbd_handle.rbd_conf) rbd_handle.close() def check_valid_device(self, path, run_as_root=True): """Verify an existing RBD handle is connected and valid.""" rbd_handle = path if rbd_handle is None: return False original_offset = rbd_handle.tell() try: rbd_handle.read(4096) except Exception as e: LOG.error("Failed to access RBD device handle: %(error)s", {"error": e}) return False finally: rbd_handle.seek(original_offset, 0) return True def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/aoe.py0000666000175100017510000001452013230233223022336 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_service import loopingcall from os_brick import exception from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 LOG = logging.getLogger(__name__) class AoEConnector(base.BaseLinuxConnector): """Connector class to attach/detach AoE volumes.""" def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(AoEConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The AoE connector properties.""" return {} def get_search_path(self): return '/dev/etherd' def get_volume_paths(self, connection_properties): aoe_device, aoe_path = self._get_aoe_info(connection_properties) volume_paths = [] if os.path.exists(aoe_path): volume_paths.append(aoe_path) return volume_paths def _get_aoe_info(self, connection_properties): shelf = connection_properties['target_shelf'] lun = connection_properties['target_lun'] aoe_device = 'e%(shelf)s.%(lun)s' % {'shelf': shelf, 'lun': lun} path = self.get_search_path() aoe_path = '%(path)s/%(device)s' % {'path': path, 'device': aoe_device} return aoe_device, aoe_path @utils.trace @lockutils.synchronized('aoe_control', 'aoe-') def connect_volume(self, connection_properties): """Discover and attach the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for AoE must include: target_shelf - shelf id of volume target_lun - lun id of volume """ aoe_device, aoe_path = self._get_aoe_info(connection_properties) device_info = { 'type': 'block', 'device': aoe_device, 'path': aoe_path, } if os.path.exists(aoe_path): self._aoe_revalidate(aoe_device) else: self._aoe_discover() waiting_status = {'tries': 0} # NOTE(jbr_): Device path is not always present immediately def _wait_for_discovery(aoe_path): if os.path.exists(aoe_path): raise loopingcall.LoopingCallDone if waiting_status['tries'] >= self.device_scan_attempts: raise exception.VolumeDeviceNotFound(device=aoe_path) LOG.info("AoE volume not yet found at: %(path)s. " "Try number: %(tries)s", {'path': aoe_device, 'tries': waiting_status['tries']}) self._aoe_discover() waiting_status['tries'] += 1 timer = loopingcall.FixedIntervalLoopingCall(_wait_for_discovery, aoe_path) timer.start(interval=2).wait() if waiting_status['tries']: LOG.debug("Found AoE device %(path)s " "(after %(tries)s rediscover)", {'path': aoe_path, 'tries': waiting_status['tries']}) return device_info @utils.trace @lockutils.synchronized('aoe_control', 'aoe-') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict connection_properties for AoE must include: target_shelf - shelf id of volume target_lun - lun id of volume """ aoe_device, aoe_path = self._get_aoe_info(connection_properties) if os.path.exists(aoe_path): self._aoe_flush(aoe_device) def _aoe_discover(self): (out, err) = self._execute('aoe-discover', run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-discover: stdout=%(out)s stderr%(err)s', {'out': out, 'err': err}) def _aoe_revalidate(self, aoe_device): (out, err) = self._execute('aoe-revalidate', aoe_device, run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-revalidate %(dev)s: stdout=%(out)s stderr%(err)s', {'dev': aoe_device, 'out': out, 'err': err}) def _aoe_flush(self, aoe_device): (out, err) = self._execute('aoe-flush', aoe_device, run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-flush %(dev)s: stdout=%(out)s stderr%(err)s', {'dev': aoe_device, 'out': out, 'err': err}) def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/gpfs.py0000666000175100017510000000307513230233223022534 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.i18n import _ from os_brick.initiator.connectors import local from os_brick import utils class GPFSConnector(local.LocalConnector): """"Connector class to attach/detach File System backed volumes.""" @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: device_path - path to the volume to be connected :type connection_properties: dict :returns: dict """ if 'device_path' not in connection_properties: msg = (_("Invalid connection_properties specified " "no device_path attribute.")) raise ValueError(msg) device_info = {'type': 'gpfs', 'path': connection_properties['device_path']} return device_info os-brick-2.3.0/os_brick/initiator/connectors/base.py0000666000175100017510000001152613230233223022507 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick import initiator from os_brick.initiator import host_driver from os_brick.initiator import initiator_connector from os_brick.initiator import linuxscsi LOG = logging.getLogger(__name__) class BaseLinuxConnector(initiator_connector.InitiatorConnector): os_type = initiator.OS_TYPE_LINUX def __init__(self, root_helper, driver=None, execute=None, *args, **kwargs): self._linuxscsi = linuxscsi.LinuxSCSI(root_helper, execute=execute) if not driver: driver = host_driver.HostDriver() self.set_driver(driver) super(BaseLinuxConnector, self).__init__(root_helper, execute=execute, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The generic connector properties.""" multipath = kwargs['multipath'] enforce_multipath = kwargs['enforce_multipath'] props = {} props['multipath'] = (multipath and linuxscsi.LinuxSCSI.is_multipath_running( enforce_multipath, root_helper, execute=kwargs.get('execute'))) return props def check_valid_device(self, path, run_as_root=True): cmd = ('dd', 'if=%(path)s' % {"path": path}, 'of=/dev/null', 'count=1') out, info = None, None try: out, info = self._execute(*cmd, run_as_root=run_as_root, root_helper=self._root_helper) except putils.ProcessExecutionError as e: LOG.error("Failed to access the device on the path " "%(path)s: %(error)s.", {"path": path, "error": e.stderr}) return False # If the info is none, the path does not exist. if info is None: return False return True def get_all_available_volumes(self, connection_properties=None): volumes = [] path = self.get_search_path() if path: # now find all entries in the search path if os.path.isdir(path): path_items = [path, '/*'] file_filter = ''.join(path_items) volumes = glob.glob(file_filter) return volumes def _discover_mpath_device(self, device_wwn, connection_properties, device_name): """This method discovers a multipath device. Discover a multipath device based on a defined connection_property and a device_wwn and return the multipath_id and path of the multipath enabled device if there is one. """ path = self._linuxscsi.find_multipath_device_path(device_wwn) device_path = None multipath_id = None if path is None: # find_multipath_device only accept realpath not symbolic path device_realpath = os.path.realpath(device_name) mpath_info = self._linuxscsi.find_multipath_device( device_realpath) if mpath_info: device_path = mpath_info['device'] multipath_id = device_wwn else: # we didn't find a multipath device. # so we assume the kernel only sees 1 device device_path = device_name LOG.debug("Unable to find multipath device name for " "volume. Using path %(device)s for volume.", {'device': device_path}) else: device_path = path multipath_id = device_wwn if connection_properties.get('access_mode', '') != 'ro': try: # Sometimes the multipath devices will show up as read only # initially and need additional time/rescans to get to RW. self._linuxscsi.wait_for_rw(device_wwn, device_path) except exception.BlockDeviceReadOnly: LOG.warning('Block device %s is still read-only. ' 'Continuing anyway.', device_path) return device_path, multipath_id os-brick-2.3.0/os_brick/initiator/connectors/fibre_channel_ppc64.py0000666000175100017510000000511113230233223025361 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import fibre_channel from os_brick.initiator import linuxfc LOG = logging.getLogger(__name__) class FibreChannelConnectorPPC64(fibre_channel.FibreChannelConnector): """Connector class to attach/detach Fibre Channel volumes on PPC64 arch.""" platform = initiator.PLATFORM_PPC64 def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(FibreChannelConnectorPPC64, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) self._linuxfc = linuxfc.LinuxFibreChannelPPC64(root_helper, execute) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnectorPPC64, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) def _get_host_devices(self, possible_devs, lun): host_devices = set() for pci_num, target_wwn in possible_devs: host_device = "/dev/disk/by-path/fc-%s-lun-%s" % ( target_wwn, self._linuxscsi.process_lun_id(lun)) host_devices.add(host_device) return list(host_devices) def _get_possible_volume_paths(self, connection_properties, hbas): ports = connection_properties['target_wwn'] it_map = connection_properties['initiator_target_map'] for hba in hbas: if hba['node_name'] in it_map.keys(): hba['target_wwn'] = it_map.get(hba['node_name']) possible_devs = self._get_possible_devices(hbas, ports) lun = connection_properties.get('target_lun', 0) host_paths = self._get_host_devices(possible_devs, lun) return host_paths os-brick-2.3.0/os_brick/initiator/connectors/disco.py0000666000175100017510000001652713230233223022704 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os import socket import struct from oslo_concurrency import lockutils from oslo_log import log as logging import six from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 synchronized = lockutils.synchronized_with_prefix('os-brick-') class DISCOConnector(base.BaseLinuxConnector): """Class implements the connector driver for DISCO.""" DISCO_PREFIX = 'dms' def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): """Init DISCO connector.""" super(DISCOConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs ) LOG.debug("Init DISCO connector") self.server_port = None self.server_ip = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The DISCO connector properties.""" return {} def get_search_path(self): """Get directory path where to get DISCO volumes.""" return "/dev" def get_volume_paths(self, connection_properties): """Get config for DISCO volume driver.""" self.get_config(connection_properties) volume_paths = [] disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) device_paths = [disco_dev] for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths def get_all_available_volumes(self, connection_properties=None): """Return all DISCO volumes that exist in the search directory.""" path = self.get_search_path() if os.path.isdir(path): path_items = [path, '/', self.DISCO_PREFIX, '*'] file_filter = ''.join(path_items) return glob.glob(file_filter) else: return [] def get_config(self, connection_properties): """Get config for DISCO volume driver.""" self.server_port = ( six.text_type(connection_properties['conf']['server_port'])) self.server_ip = ( six.text_type(connection_properties['conf']['server_ip'])) disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) device_info = {'type': 'block', 'path': disco_dev} return device_info @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Connect the volume. Returns xml for libvirt.""" LOG.debug("Enter in DISCO connect_volume") device_info = self.get_config(connection_properties) LOG.debug("Device info : %s.", device_info) disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) LOG.debug("Attaching %s", disco_dev) self._mount_disco_volume(disco_dev, disco_id) return device_info @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance.""" disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) LOG.debug("detaching %s", disco_dev) if os.path.exists(disco_dev): ret = self._send_disco_vol_cmd(self.server_ip, self.server_port, 2, disco_id) if ret is not None: msg = _("Detach volume failed") raise exception.BrickException(message=msg) else: LOG.info("Volume already detached from host") def _mount_disco_volume(self, path, volume_id): """Send request to mount volume on physical host.""" LOG.debug("Enter in mount disco volume %(port)s " "and %(ip)s.", {'port': self.server_port, 'ip': self.server_ip}) if not os.path.exists(path): ret = self._send_disco_vol_cmd(self.server_ip, self.server_port, 1, volume_id) if ret is not None: msg = _("Attach volume failed") raise exception.BrickException(message=msg) else: LOG.info("Volume already attached to host") def _connect_tcp_socket(self, client_ip, client_port): """Connect to TCP socket.""" sock = None for res in socket.getaddrinfo(client_ip, client_port, socket.AF_UNSPEC, socket.SOCK_STREAM): aff, socktype, proto, canonname, saa = res try: sock = socket.socket(aff, socktype, proto) except socket.error: sock = None continue try: sock.connect(saa) except socket.error: sock.close() sock = None continue break if sock is None: LOG.error("Cannot connect TCP socket") return sock def _send_disco_vol_cmd(self, client_ip, client_port, op_code, vol_id): """Send DISCO client socket command.""" s = self._connect_tcp_socket(client_ip, int(client_port)) if s is not None: inst_id = 'DEFAULT-INSTID' pktlen = 2 + 8 + len(inst_id) LOG.debug("pktlen=%(plen)s op=%(op)s " "vol_id=%(vol_id)s, inst_id=%(inst_id)s", {'plen': pktlen, 'op': op_code, 'vol_id': vol_id, 'inst_id': inst_id}) data = struct.pack("!HHQ14s", pktlen, op_code, int(vol_id), inst_id) s.sendall(data) ret = s.recv(4) s.close() LOG.debug("Received ret len=%(lenR)d, ret=%(ret)s", {'lenR': len(repr(ret)), 'ret': repr(ret)}) ret_val = "".join("%02x" % ord(c) for c in ret) if ret_val != '00000000': return 'ERROR' return None def extend_volume(self, connection_properties): raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/drbd.py0000666000175100017510000000721713230233223022512 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_concurrency import processutils as putils from os_brick.initiator.connectors import base from os_brick import utils class DRBDConnector(base.BaseLinuxConnector): """"Connector class to attach/detach DRBD resources.""" def __init__(self, root_helper, driver=None, execute=putils.execute, *args, **kwargs): super(DRBDConnector, self).__init__(root_helper, driver=driver, execute=execute, *args, **kwargs) self._execute = execute self._root_helper = root_helper @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The DRBD connector properties.""" return {} def check_valid_device(self, path, run_as_root=True): """Verify an existing volume.""" # TODO(linbit): check via drbdsetup first, to avoid blocking/hanging # in case of network problems? return super(DRBDConnector, self).check_valid_device(path, run_as_root) def get_all_available_volumes(self, connection_properties=None): base = "/dev/" blkdev_list = [] for e in os.listdir(base): path = base + e if os.path.isblk(path): blkdev_list.append(path) return blkdev_list def _drbdadm_command(self, cmd, data_dict, sh_secret): # TODO(linbit): Write that resource file to a permanent location? tmp = tempfile.NamedTemporaryFile(suffix="res", delete=False, mode="w") try: kv = {'shared-secret': sh_secret} tmp.write(data_dict['config'] % kv) tmp.close() (out, err) = self._execute('drbdadm', cmd, "-c", tmp.name, data_dict['name'], run_as_root=True, root_helper=self._root_helper) finally: os.unlink(tmp.name) return (out, err) @utils.trace def connect_volume(self, connection_properties): """Attach the volume.""" self._drbdadm_command("adjust", connection_properties, connection_properties['provider_auth']) device_info = { 'type': 'block', 'path': connection_properties['device'], } return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume.""" self._drbdadm_command("down", connection_properties, connection_properties['provider_auth']) def get_volume_paths(self, connection_properties): path = connection_properties['device'] return [path] def get_search_path(self): # TODO(linbit): is it allowed to return "/dev", or is that too broad? return None def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/scaleio.py0000666000175100017510000004361213230233223023215 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json import os import requests from six.moves import urllib from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 synchronized = lockutils.synchronized_with_prefix('os-brick-') class ScaleIOConnector(base.BaseLinuxConnector): """Class implements the connector driver for ScaleIO.""" OK_STATUS_CODE = 200 VOLUME_NOT_MAPPED_ERROR = 84 VOLUME_ALREADY_MAPPED_ERROR = 81 GET_GUID_CMD = ['/opt/emc/scaleio/sdc/bin/drv_cfg', '--query_guid'] def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(ScaleIOConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs ) self.local_sdc_ip = None self.server_ip = None self.server_port = None self.server_username = None self.server_password = None self.server_token = None self.volume_id = None self.volume_name = None self.volume_path = None self.iops_limit = None self.bandwidth_limit = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The ScaleIO connector properties.""" return {} def get_search_path(self): return "/dev/disk/by-id" def get_volume_paths(self, connection_properties): self.get_config(connection_properties) volume_paths = [] device_paths = [self._find_volume_path()] for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths def _find_volume_path(self): LOG.info( "Looking for volume %(volume_id)s, maximum tries: %(tries)s", {'volume_id': self.volume_id, 'tries': self.device_scan_attempts} ) # look for the volume in /dev/disk/by-id directory by_id_path = self.get_search_path() disk_filename = self._wait_for_volume_path(by_id_path) full_disk_name = ("%(path)s/%(filename)s" % {'path': by_id_path, 'filename': disk_filename}) LOG.info("Full disk name is %(full_path)s", {'full_path': full_disk_name}) return full_disk_name # NOTE: Usually 3 retries is enough to find the volume. # If there are network issues, it could take much longer. Set # the max retries to 15 to make sure we can find the volume. @utils.retry(exceptions=exception.BrickException, retries=15, backoff_rate=1) def _wait_for_volume_path(self, path): if not os.path.isdir(path): msg = ( _("ScaleIO volume %(volume_id)s not found at " "expected path.") % {'volume_id': self.volume_id} ) LOG.debug(msg) raise exception.BrickException(message=msg) disk_filename = None filenames = os.listdir(path) LOG.info( "Files found in %(path)s path: %(files)s ", {'path': path, 'files': filenames} ) for filename in filenames: if (filename.startswith("emc-vol") and filename.endswith(self.volume_id)): disk_filename = filename break if not disk_filename: msg = (_("ScaleIO volume %(volume_id)s not found.") % {'volume_id': self.volume_id}) LOG.debug(msg) raise exception.BrickException(message=msg) return disk_filename def _get_client_id(self): request = ( "https://%(server_ip)s:%(server_port)s/" "api/types/Client/instances/getByIp::%(sdc_ip)s/" % { 'server_ip': self.server_ip, 'server_port': self.server_port, 'sdc_ip': self.local_sdc_ip } ) LOG.info("ScaleIO get client id by ip request: %(request)s", {'request': request}) r = requests.get( request, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request) sdc_id = r.json() if not sdc_id: msg = (_("Client with ip %(sdc_ip)s was not found.") % {'sdc_ip': self.local_sdc_ip}) raise exception.BrickException(message=msg) if r.status_code != 200 and "errorCode" in sdc_id: msg = (_("Error getting sdc id from ip %(sdc_ip)s: %(err)s") % {'sdc_ip': self.local_sdc_ip, 'err': sdc_id['message']}) LOG.error(msg) raise exception.BrickException(message=msg) LOG.info("ScaleIO sdc id is %(sdc_id)s.", {'sdc_id': sdc_id}) return sdc_id def _get_volume_id(self): volname_encoded = urllib.parse.quote(self.volume_name, '') volname_double_encoded = urllib.parse.quote(volname_encoded, '') LOG.debug(_( "Volume name after double encoding is %(volume_name)s."), {'volume_name': volname_double_encoded} ) request = ( "https://%(server_ip)s:%(server_port)s/api/types/Volume/instances" "/getByName::%(encoded_volume_name)s" % { 'server_ip': self.server_ip, 'server_port': self.server_port, 'encoded_volume_name': volname_double_encoded } ) LOG.info( "ScaleIO get volume id by name request: %(request)s", {'request': request} ) r = requests.get(request, auth=(self.server_username, self.server_token), verify=False) r = self._check_response(r, request) volume_id = r.json() if not volume_id: msg = (_("Volume with name %(volume_name)s wasn't found.") % {'volume_name': self.volume_name}) LOG.error(msg) raise exception.BrickException(message=msg) if r.status_code != self.OK_STATUS_CODE and "errorCode" in volume_id: msg = ( _("Error getting volume id from name %(volume_name)s: " "%(err)s") % {'volume_name': self.volume_name, 'err': volume_id['message']} ) LOG.error(msg) raise exception.BrickException(message=msg) LOG.info("ScaleIO volume id is %(volume_id)s.", {'volume_id': volume_id}) return volume_id def _check_response(self, response, request, is_get_request=True, params=None): if response.status_code == 401 or response.status_code == 403: LOG.info("Token is invalid, " "going to re-login to get a new one") login_request = ( "https://%(server_ip)s:%(server_port)s/api/login" % {'server_ip': self.server_ip, 'server_port': self.server_port} ) r = requests.get( login_request, auth=(self.server_username, self.server_password), verify=False ) token = r.json() # repeat request with valid token LOG.debug(_("Going to perform request %(request)s again " "with valid token"), {'request': request}) if is_get_request: res = requests.get(request, auth=(self.server_username, token), verify=False) else: headers = {'content-type': 'application/json'} res = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, token), verify=False ) self.server_token = token return res return response def get_config(self, connection_properties): self.local_sdc_ip = connection_properties['hostIP'] self.volume_name = connection_properties['scaleIO_volname'] self.volume_id = connection_properties['scaleIO_volume_id'] self.server_ip = connection_properties['serverIP'] self.server_port = connection_properties['serverPort'] self.server_username = connection_properties['serverUsername'] self.server_password = connection_properties['serverPassword'] self.server_token = connection_properties['serverToken'] self.iops_limit = connection_properties['iopsLimit'] self.bandwidth_limit = connection_properties['bandwidthLimit'] device_info = {'type': 'block', 'path': self.volume_path} return device_info @utils.trace @lockutils.synchronized('scaleio', 'scaleio-') def connect_volume(self, connection_properties): """Connect the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ device_info = self.get_config(connection_properties) LOG.debug( _( "scaleIO Volume name: %(volume_name)s, SDC IP: %(sdc_ip)s, " "REST Server IP: %(server_ip)s, " "REST Server username: %(username)s, " "iops limit:%(iops_limit)s, " "bandwidth limit: %(bandwidth_limit)s." ), { 'volume_name': self.volume_name, 'volume_id': self.volume_id, 'sdc_ip': self.local_sdc_ip, 'server_ip': self.server_ip, 'username': self.server_username, 'iops_limit': self.iops_limit, 'bandwidth_limit': self.bandwidth_limit } ) LOG.info("ScaleIO sdc query guid command: %(cmd)s", {'cmd': self.GET_GUID_CMD}) try: (out, err) = self._execute(*self.GET_GUID_CMD, run_as_root=True, root_helper=self._root_helper) LOG.info("Map volume %(cmd)s: stdout=%(out)s " "stderr=%(err)s", {'cmd': self.GET_GUID_CMD, 'out': out, 'err': err}) except putils.ProcessExecutionError as e: msg = (_("Error querying sdc guid: %(err)s") % {'err': e.stderr}) LOG.error(msg) raise exception.BrickException(message=msg) guid = out LOG.info("Current sdc guid: %(guid)s", {'guid': guid}) params = {'guid': guid, 'allowMultipleMappings': 'TRUE'} self.volume_id = self.volume_id or self._get_volume_id() headers = {'content-type': 'application/json'} request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/addMappedSdc" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("map volume request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() error_code = response['errorCode'] if error_code == self.VOLUME_ALREADY_MAPPED_ERROR: LOG.warning( "Ignoring error mapping volume %(volume_name)s: " "volume already mapped.", {'volume_name': self.volume_name} ) else: msg = ( _("Error mapping volume %(volume_name)s: %(err)s") % {'volume_name': self.volume_name, 'err': response['message']} ) LOG.error(msg) raise exception.BrickException(message=msg) self.volume_path = self._find_volume_path() device_info['path'] = self.volume_path # Set QoS settings after map was performed if self.iops_limit is not None or self.bandwidth_limit is not None: params = {'guid': guid} if self.bandwidth_limit is not None: params['bandwidthLimitInKbps'] = self.bandwidth_limit if self.iops_limit is not None: params['iopsLimit'] = self.iops_limit request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/setMappedSdcLimits" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("Set client limit request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() LOG.info("Set client limit response: %(response)s", {'response': response}) msg = ( _("Error setting client limits for volume " "%(volume_name)s: %(err)s") % {'volume_name': self.volume_name, 'err': response['message']} ) LOG.error(msg) return device_info @utils.trace @lockutils.synchronized('scaleio', 'scaleio-') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect the ScaleIO volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ self.get_config(connection_properties) self.volume_id = self.volume_id or self._get_volume_id() LOG.info( "ScaleIO disconnect volume in ScaleIO brick volume driver." ) LOG.debug( _("ScaleIO Volume name: %(volume_name)s, SDC IP: %(sdc_ip)s, " "REST Server IP: %(server_ip)s"), {'volume_name': self.volume_name, 'sdc_ip': self.local_sdc_ip, 'server_ip': self.server_ip} ) LOG.info("ScaleIO sdc query guid command: %(cmd)s", {'cmd': self.GET_GUID_CMD}) try: (out, err) = self._execute(*self.GET_GUID_CMD, run_as_root=True, root_helper=self._root_helper) LOG.info( "Unmap volume %(cmd)s: stdout=%(out)s stderr=%(err)s", {'cmd': self.GET_GUID_CMD, 'out': out, 'err': err} ) except putils.ProcessExecutionError as e: msg = _("Error querying sdc guid: %(err)s") % {'err': e.stderr} LOG.error(msg) raise exception.BrickException(message=msg) guid = out LOG.info("Current sdc guid: %(guid)s", {'guid': guid}) params = {'guid': guid} headers = {'content-type': 'application/json'} request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/removeMappedSdc" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("Unmap volume request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() error_code = response['errorCode'] if error_code == self.VOLUME_NOT_MAPPED_ERROR: LOG.warning( "Ignoring error unmapping volume %(volume_id)s: " "volume not mapped.", {'volume_id': self.volume_name} ) else: msg = (_("Error unmapping volume %(volume_id)s: %(err)s") % {'volume_id': self.volume_name, 'err': response['message']}) LOG.error(msg) raise exception.BrickException(message=msg) def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/local.py0000666000175100017510000000554713230233223022675 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils class LocalConnector(base.BaseLinuxConnector): """"Connector class to attach/detach File System backed volumes.""" def __init__(self, root_helper, driver=None, *args, **kwargs): super(LocalConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The Local connector properties.""" return {} def get_volume_paths(self, connection_properties): path = connection_properties['device_path'] return [path] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): # TODO(walter-boring): not sure what to return here. return [] @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. ``connection_properties`` must include: - ``device_path`` - path to the volume to be connected :type connection_properties: dict :returns: dict """ if 'device_path' not in connection_properties: msg = (_("Invalid connection_properties specified " "no device_path attribute")) raise ValueError(msg) device_info = {'type': 'local', 'path': connection_properties['device_path']} return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ pass def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/huawei.py0000666000175100017510000001720413230233223023056 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') class HuaweiStorHyperConnector(base.BaseLinuxConnector): """"Connector class to attach/detach SDSHypervisor volumes.""" attached_success_code = 0 has_been_attached_code = 50151401 attach_mnid_done_code = 50151405 vbs_unnormal_code = 50151209 not_mount_node_code = 50155007 iscliexist = True def __init__(self, root_helper, driver=None, *args, **kwargs): self.cli_path = os.getenv('HUAWEISDSHYPERVISORCLI_PATH') if not self.cli_path: self.cli_path = '/usr/local/bin/sds/sds_cli' LOG.debug("CLI path is not configured, using default %s.", self.cli_path) if not os.path.isfile(self.cli_path): self.iscliexist = False LOG.error('SDS CLI file not found, ' 'HuaweiStorHyperConnector init failed.') super(HuaweiStorHyperConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HuaweiStor connector properties.""" return {} def get_search_path(self): # TODO(walter-boring): Where is the location on the filesystem to # look for Huawei volumes to show up? return None def get_all_available_volumes(self, connection_properties=None): # TODO(walter-boring): what to return here for all Huawei volumes ? return [] def get_volume_paths(self, connection_properties): volume_path = None try: volume_path = self._get_volume_path(connection_properties) except Exception: msg = _("Couldn't find a volume.") LOG.warning(msg) raise exception.BrickException(message=msg) return [volume_path] def _get_volume_path(self, connection_properties): out = self._query_attached_volume( connection_properties['volume_id']) if not out or int(out['ret_code']) != 0: msg = _("Couldn't find attached volume.") LOG.error(msg) raise exception.BrickException(message=msg) return out['dev_addr'] @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ LOG.debug("Connect_volume connection properties: %s.", connection_properties) out = self._attach_volume(connection_properties['volume_id']) if not out or int(out['ret_code']) not in (self.attached_success_code, self.has_been_attached_code, self.attach_mnid_done_code): msg = (_("Attach volume failed, " "error code is %s") % out['ret_code']) raise exception.BrickException(message=msg) try: volume_path = self._get_volume_path(connection_properties) except Exception: msg = _("query attached volume failed or volume not attached.") LOG.error(msg) raise exception.BrickException(message=msg) device_info = {'type': 'block', 'path': volume_path} return device_info @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ LOG.debug("Disconnect_volume: %s.", connection_properties) out = self._detach_volume(connection_properties['volume_id']) if not out or int(out['ret_code']) not in (self.attached_success_code, self.vbs_unnormal_code, self.not_mount_node_code): msg = (_("Disconnect_volume failed, " "error code is %s") % out['ret_code']) raise exception.BrickException(message=msg) def is_volume_connected(self, volume_name): """Check if volume already connected to host""" LOG.debug('Check if volume %s already connected to a host.', volume_name) out = self._query_attached_volume(volume_name) if out: return int(out['ret_code']) == 0 return False def _attach_volume(self, volume_name): return self._cli_cmd('attach', volume_name) def _detach_volume(self, volume_name): return self._cli_cmd('detach', volume_name) def _query_attached_volume(self, volume_name): return self._cli_cmd('querydev', volume_name) def _cli_cmd(self, method, volume_name): LOG.debug("Enter into _cli_cmd.") if not self.iscliexist: msg = _("SDS command line doesn't exist, " "can't execute SDS command.") raise exception.BrickException(message=msg) if not method or volume_name is None: return cmd = [self.cli_path, '-c', method, '-v', volume_name] out, clilog = self._execute(*cmd, run_as_root=False, root_helper=self._root_helper) analyse_result = self._analyze_output(out) LOG.debug('%(method)s volume returns %(analyse_result)s.', {'method': method, 'analyse_result': analyse_result}) if clilog: LOG.error("SDS CLI output some log: %s.", clilog) return analyse_result def _analyze_output(self, out): LOG.debug("Enter into _analyze_output.") if out: analyse_result = {} out_temp = out.split('\n') for line in out_temp: LOG.debug("Line is %s.", line) if line.find('=') != -1: key, val = line.split('=', 1) LOG.debug("%(key)s = %(val)s", {'key': key, 'val': val}) if key in ['ret_code', 'ret_desc', 'dev_addr']: analyse_result[key] = val return analyse_result else: return None def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/sheepdog.py0000666000175100017510000001105413230233223023367 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator import linuxsheepdog from os_brick import utils DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 LOG = logging.getLogger(__name__) class SheepdogConnector(base.BaseLinuxConnector): """"Connector class to attach/detach sheepdog volumes.""" def __init__(self, root_helper, driver=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(SheepdogConnector, self).__init__(root_helper, driver=driver, device_scan_attempts= device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The Sheepdog connector properties.""" return {} def get_volume_paths(self, connection_properties): # TODO(lixiaoy1): don't know where the connector # looks for sheepdog volumes. return [] def get_search_path(self): # TODO(lixiaoy1): don't know where the connector # looks for sheepdog volumes. return None def get_all_available_volumes(self, connection_properties=None): # TODO(lixiaoy1): not sure what to return here for sheepdog return [] def _get_sheepdog_handle(self, connection_properties): try: host = connection_properties['hosts'][0] name = connection_properties['name'] port = connection_properties['ports'][0] except IndexError: msg = _("Connect volume failed, malformed connection properties") raise exception.BrickException(msg=msg) sheepdog_handle = linuxsheepdog.SheepdogVolumeIOWrapper( host, port, name) return sheepdog_handle @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ sheepdog_handle = self._get_sheepdog_handle(connection_properties) return {'path': sheepdog_handle} @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ if device_info: sheepdog_handle = device_info.get('path', None) self.check_IO_handle_valid(sheepdog_handle, linuxsheepdog.SheepdogVolumeIOWrapper, 'Sheepdog') if sheepdog_handle is not None: sheepdog_handle.close() def check_valid_device(self, path, run_as_root=True): """Verify an existing sheepdog handle is connected and valid.""" sheepdog_handle = path if sheepdog_handle is None: return False original_offset = sheepdog_handle.tell() try: sheepdog_handle.read(4096) except Exception as e: LOG.error("Failed to access sheepdog device " "handle: %(error)s", {"error": e}) return False finally: sheepdog_handle.seek(original_offset, 0) return True def extend_volume(self, connection_properties): # TODO(lixiaoy1): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/hgst.py0000666000175100017510000001651713230233223022547 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import socket from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) class HGSTConnector(base.BaseLinuxConnector): """Connector class to attach/detach HGST volumes.""" VGCCLUSTER = 'vgc-cluster' def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(HGSTConnector, self).__init__(root_helper, driver=driver, device_scan_attempts= device_scan_attempts, *args, **kwargs) self._vgc_host = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HGST connector properties.""" return {} def _log_cli_err(self, err): """Dumps the full command output to a logfile in error cases.""" LOG.error("CLI fail: '%(cmd)s' = %(code)s\nout: %(stdout)s\n" "err: %(stderr)s", {'cmd': err.cmd, 'code': err.exit_code, 'stdout': err.stdout, 'stderr': err.stderr}) def _find_vgc_host(self): """Finds vgc-cluster hostname for this box.""" params = [self.VGCCLUSTER, "domain-list", "-1"] try: out, unused = self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = _("Unable to get list of domain members, check that " "the cluster is running.") raise exception.BrickException(message=msg) domain = out.splitlines() params = ["ip", "addr", "list"] try: out, unused = self._execute(*params, run_as_root=False) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = _("Unable to get list of IP addresses on this host, " "check permissions and networking.") raise exception.BrickException(message=msg) nets = out.splitlines() for host in domain: try: ip = socket.gethostbyname(host) for l in nets: x = l.strip() if x.startswith("inet %s/" % ip): return host except socket.error: pass msg = _("Current host isn't part of HGST domain.") raise exception.BrickException(message=msg) def _hostname(self): """Returns hostname to use for cluster operations on this box.""" if self._vgc_host is None: self._vgc_host = self._find_vgc_host() return self._vgc_host def get_search_path(self): return "/dev" def get_volume_paths(self, connection_properties): path = ("%(path)s/%(name)s" % {'path': self.get_search_path(), 'name': connection_properties['name']}) volume_path = None if os.path.exists(path): volume_path = path return [volume_path] @utils.trace def connect_volume(self, connection_properties): """Attach a Space volume to running host. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties for HGST must include: name - Name of space to attach :type connection_properties: dict :returns: dict """ if connection_properties is None: msg = _("Connection properties passed in as None.") raise exception.BrickException(message=msg) if 'name' not in connection_properties: msg = _("Connection properties missing 'name' field.") raise exception.BrickException(message=msg) device_info = { 'type': 'block', 'device': connection_properties['name'], 'path': '/dev/' + connection_properties['name'] } volname = device_info['device'] params = [self.VGCCLUSTER, 'space-set-apphosts'] params += ['-n', volname] params += ['-A', self._hostname()] params += ['--action', 'ADD'] try: self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = (_("Unable to set apphost for space %s") % volname) raise exception.BrickException(message=msg) return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. For HGST must include: name - Name of space to detach noremovehost - Host which should never be removed :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ if connection_properties is None: msg = _("Connection properties passed in as None.") raise exception.BrickException(message=msg) if 'name' not in connection_properties: msg = _("Connection properties missing 'name' field.") raise exception.BrickException(message=msg) if 'noremovehost' not in connection_properties: msg = _("Connection properties missing 'noremovehost' field.") raise exception.BrickException(message=msg) if connection_properties['noremovehost'] != self._hostname(): params = [self.VGCCLUSTER, 'space-set-apphosts'] params += ['-n', connection_properties['name']] params += ['-A', self._hostname()] params += ['--action', 'DELETE'] try: self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = (_("Unable to set apphost for space %s") % connection_properties['name']) raise exception.BrickException(message=msg) def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/base_iscsi.py0000666000175100017510000000343113230233223023675 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import copy from os_brick.initiator import initiator_connector class BaseISCSIConnector(initiator_connector.InitiatorConnector): def _iterate_all_targets(self, connection_properties): for portal, iqn, lun in self._get_all_targets(connection_properties): props = copy.deepcopy(connection_properties) props['target_portal'] = portal props['target_iqn'] = iqn props['target_lun'] = lun for key in ('target_portals', 'target_iqns', 'target_luns'): props.pop(key, None) yield props def _get_all_targets(self, connection_properties): if all([key in connection_properties for key in ('target_portals', 'target_iqns', 'target_luns')]): return zip(connection_properties['target_portals'], connection_properties['target_iqns'], connection_properties['target_luns']) return [(connection_properties['target_portal'], connection_properties['target_iqn'], connection_properties.get('target_lun', 0))] os-brick-2.3.0/os_brick/initiator/connectors/fibre_channel.py0000666000175100017510000002671713230233223024364 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_service import loopingcall import six from os_brick import exception from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator import linuxfc from os_brick import utils synchronized = lockutils.synchronized_with_prefix('os-brick-') LOG = logging.getLogger(__name__) class FibreChannelConnector(base.BaseLinuxConnector): """Connector class to attach/detach Fibre Channel volumes.""" def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): self._linuxfc = linuxfc.LinuxFibreChannel(root_helper, execute) super(FibreChannelConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnector, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The Fibre Channel connector properties.""" props = {} fc = linuxfc.LinuxFibreChannel(root_helper, execute=kwargs.get('execute')) wwpns = fc.get_fc_wwpns() if wwpns: props['wwpns'] = wwpns wwnns = fc.get_fc_wwnns() if wwnns: props['wwnns'] = wwnns return props def get_search_path(self): """Where do we look for FC based volumes.""" return '/dev/disk/by-path' def _get_possible_volume_paths(self, connection_properties, hbas): ports = connection_properties['target_wwn'] possible_devs = self._get_possible_devices(hbas, ports) lun = connection_properties.get('target_lun', 0) host_paths = self._get_host_devices(possible_devs, lun) return host_paths def get_volume_paths(self, connection_properties): volume_paths = [] # first fetch all of the potential paths that might exist # how the FC fabric is zoned may alter the actual list # that shows up on the system. So, we verify each path. hbas = self._linuxfc.get_fc_hbas_info() device_paths = self._get_possible_volume_paths( connection_properties, hbas) for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths @utils.trace @synchronized('extend_volume') def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an FC volume. """ volume_paths = self.get_volume_paths(connection_properties) if volume_paths: return self._linuxscsi.extend_volume(volume_paths) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': connection_properties}) raise exception.VolumePathsNotFound() @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Attach the volume to instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for Fibre Channel must include: target_wwn - World Wide Name target_lun - LUN id of the volume """ LOG.debug("execute = %s", self._execute) device_info = {'type': 'block'} hbas = self._linuxfc.get_fc_hbas_info() host_devices = self._get_possible_volume_paths( connection_properties, hbas) if len(host_devices) == 0: # this is empty because we don't have any FC HBAs LOG.warning("We are unable to locate any Fibre Channel devices") raise exception.NoFibreChannelHostsFound() # The /dev/disk/by-path/... node is not always present immediately # We only need to find the first device. Once we see the first device # multipath will have any others. def _wait_for_device_discovery(host_devices): for device in host_devices: LOG.debug("Looking for Fibre Channel dev %(device)s", {'device': device}) if os.path.exists(device) and self.check_valid_device(device): self.host_device = device # get the /dev/sdX device. This is used # to find the multipath device. self.device_name = os.path.realpath(device) raise loopingcall.LoopingCallDone() if self.tries >= self.device_scan_attempts: LOG.error("Fibre Channel volume device not found.") raise exception.NoFibreChannelVolumeDeviceFound() LOG.info("Fibre Channel volume device not yet found. " "Will rescan & retry. Try number: %(tries)s.", {'tries': self.tries}) self._linuxfc.rescan_hosts(hbas, connection_properties) self.tries = self.tries + 1 self.host_device = None self.device_name = None self.tries = 0 timer = loopingcall.FixedIntervalLoopingCall( _wait_for_device_discovery, host_devices) timer.start(interval=2).wait() if self.host_device is not None and self.device_name is not None: LOG.debug("Found Fibre Channel volume %(name)s " "(after %(tries)s rescans)", {'name': self.device_name, 'tries': self.tries}) # find out the WWN of the device device_wwn = self._linuxscsi.get_scsi_wwn(self.host_device) LOG.debug("Device WWN = '%(wwn)s'", {'wwn': device_wwn}) device_info['scsi_wwn'] = device_wwn # see if the new drive is part of a multipath # device. If so, we'll use the multipath device. if self.use_multipath: (device_path, multipath_id) = (super( FibreChannelConnector, self)._discover_mpath_device( device_wwn, connection_properties, self.device_name)) if multipath_id: # only set the multipath_id if we found one device_info['multipath_id'] = multipath_id else: device_path = self.host_device device_info['path'] = device_path LOG.debug("connect_volume returning %s", device_info) return device_info def _get_host_devices(self, possible_devs, lun): host_devices = [] for pci_num, target_wwn in possible_devs: host_device = "/dev/disk/by-path/pci-%s-fc-%s-lun-%s" % ( pci_num, target_wwn, self._linuxscsi.process_lun_id(lun)) host_devices.append(host_device) return host_devices def _get_possible_devices(self, hbas, wwnports): """Compute the possible fibre channel device options. :param hbas: available hba devices. :param wwnports: possible wwn addresses. Can either be string or list of strings. :returns: list of (pci_id, wwn) tuples Given one or more wwn (mac addresses for fibre channel) ports do the matrix math to figure out a set of pci device, wwn tuples that are potentially valid (they won't all be). This provides a search space for the device connection. """ # the wwn (think mac addresses for fiber channel devices) can # either be a single value or a list. Normalize it to a list # for further operations. wwns = [] if isinstance(wwnports, list): for wwn in wwnports: wwns.append(str(wwn)) elif isinstance(wwnports, six.string_types): wwns.append(str(wwnports)) raw_devices = [] for hba in hbas: pci_num = self._get_pci_num(hba) if pci_num is not None: for wwn in wwns: target_wwn = "0x%s" % wwn.lower() raw_devices.append((pci_num, target_wwn)) return raw_devices @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict connection_properties for Fibre Channel must include: target_wwn - World Wide Name target_lun - LUN id of the volume """ devices = [] wwn = None volume_paths = self.get_volume_paths(connection_properties) mpath_path = None for path in volume_paths: real_path = self._linuxscsi.get_name_from_path(path) if self.use_multipath and not mpath_path: wwn = self._linuxscsi.get_scsi_wwn(path) mpath_path = self._linuxscsi.find_multipath_device_path(wwn) if mpath_path: self._linuxscsi.flush_multipath_device(mpath_path) device_info = self._linuxscsi.get_device_info(real_path) devices.append(device_info) LOG.debug("devices to remove = %s", devices) self._remove_devices(connection_properties, devices) def _remove_devices(self, connection_properties, devices): # There may have been more than 1 device mounted # by the kernel for this volume. We have to remove # all of them for device in devices: self._linuxscsi.remove_scsi_device(device["device"]) def _get_pci_num(self, hba): # NOTE(walter-boring) # device path is in format of (FC and FCoE) : # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2 # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2 # /host3/fc_host/host3 # we always want the value prior to the host or net value if hba is not None: if "device_path" in hba: device_path = hba['device_path'].split('/') for index, value in enumerate(device_path): if value.startswith('net') or value.startswith('host'): return device_path[index - 1] return None os-brick-2.3.0/os_brick/initiator/connectors/fake.py0000666000175100017510000000313513230233223022500 0ustar zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import base from os_brick.initiator.connectors import base_iscsi class FakeConnector(base.BaseLinuxConnector): fake_path = '/dev/vdFAKE' def connect_volume(self, connection_properties): fake_device_info = {'type': 'fake', 'path': self.fake_path} return fake_device_info def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): pass def get_volume_paths(self, connection_properties): return [self.fake_path] def get_search_path(self): return '/dev/disk/by-path' def extend_volume(self, connection_properties): return None def get_all_available_volumes(self, connection_properties=None): return ['/dev/disk/by-path/fake-volume-1', '/dev/disk/by-path/fake-volume-X'] class FakeBaseISCSIConnector(FakeConnector, base_iscsi.BaseISCSIConnector): pass os-brick-2.3.0/os_brick/initiator/connectors/iscsi.py0000666000175100017510000015041513230233223022710 0ustar zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import copy import glob import os import re import time from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import excutils from oslo_utils import strutils from os_brick import exception from os_brick import executor from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator.connectors import base_iscsi from os_brick import utils synchronized = lockutils.synchronized_with_prefix('os-brick-') LOG = logging.getLogger(__name__) class ISCSIConnector(base.BaseLinuxConnector, base_iscsi.BaseISCSIConnector): """Connector class to attach/detach iSCSI volumes.""" supported_transports = ['be2iscsi', 'bnx2i', 'cxgb3i', 'default', 'cxgb4i', 'qla4xxx', 'ocs', 'iser', 'tcp'] VALID_SESSIONS_PREFIX = ('tcp:', 'iser:') def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, transport='default', *args, **kwargs): super(ISCSIConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, transport=transport, *args, **kwargs) self.use_multipath = use_multipath self.transport = self._validate_iface_transport(transport) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The iSCSI connector properties.""" props = {} iscsi = ISCSIConnector(root_helper=root_helper, execute=kwargs.get('execute')) initiator = iscsi.get_initiator() if initiator: props['initiator'] = initiator return props def get_search_path(self): """Where do we look for iSCSI based volumes.""" return '/dev/disk/by-path' def get_volume_paths(self, connection_properties): """Get the list of existing paths for a volume. This method's job is to simply report what might/should already exist for a volume. We aren't trying to attach/discover a new volume, but find any existing paths for a volume we think is already attached. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ volume_paths = [] # if there are no sessions, then target_portal won't exist if (('target_portal' not in connection_properties) and ('target_portals' not in connection_properties)): return volume_paths # Don't try and connect to the portals in the list as # this can create empty iSCSI sessions to hosts if they # didn't exist previously. # We are simply trying to find any existing volumes with # already connected sessions. host_devices = self._get_potential_volume_paths(connection_properties) for path in host_devices: if os.path.exists(path): volume_paths.append(path) return volume_paths def _get_iscsi_sessions_full(self): """Get iSCSI session information as a list of tuples. Uses iscsiadm -m session and from a command output like tcp: [1] 192.168.121.250:3260,1 iqn.2010-10.org.openstack:volume- This method will drop the node type and return a list like this: [('tcp:', '1', '192.168.121.250:3260', '1', 'iqn.2010-10.org.openstack:volume-')] """ out, err = self._run_iscsi_session() if err: LOG.warning("Couldn't find iscsi sessions because " "iscsiadm err: %s", err) return [] # Parse and clean the output from iscsiadm, which is in the form of: # transport_name: [session_id] ip_address:port,tpgt iqn node_type lines = [] for line in out.splitlines(): if line: info = line.split() sid = info[1][1:-1] portal, tpgt = info[2].split(',') lines.append((info[0], sid, portal, tpgt, info[3])) return lines def _get_iscsi_nodes(self): """Get iSCSI node information (portal, iqn) as a list of tuples. Uses iscsi_adm -m node and from a command output like 192.168.121.250:3260,1 iqn.2010-10.org.openstack:volume This method will drop the tpgt and return a list like this: [('192.168.121.250:3260', 'iqn.2010-10.org.openstack:volume')] """ out, err = self._execute('iscsiadm', '-m', 'node', run_as_root=True, root_helper=self._root_helper, check_exit_code=False) if err: LOG.warning("Couldn't find iSCSI nodes because iscsiadm err: %s", err) return [] # Parse and clean the output from iscsiadm which is in the form of: # ip_addresss:port,tpgt iqn lines = [] for line in out.splitlines(): if line: info = line.split() lines.append((info[0].split(',')[0], info[1])) return lines def _get_iscsi_sessions(self): """Return portals for all existing sessions.""" # entry: [tcp, [1], 192.168.121.250:3260,1 ...] return [entry[2] for entry in self._get_iscsi_sessions_full()] def _get_ips_iqns_luns(self, connection_properties, discover=True): """Build a list of ips, iqns, and luns. Used only when doing multipath, we have 3 cases: - All information is in the connection properties - We have to do an iSCSI discovery to get the information - We don't want to do another discovery and we query the discoverydb :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param discover: Wheter doing an iSCSI discovery is acceptable. :type discover: bool :returns: list of tuples of (ip, iqn, lun) """ try: if ('target_portals' in connection_properties and 'target_iqns' in connection_properties): # Use targets specified by connection_properties ips_iqns_luns = list( zip(connection_properties['target_portals'], connection_properties['target_iqns'], self._get_luns(connection_properties))) else: method = (self._discover_iscsi_portals if discover else self._get_discoverydb_portals) ips_iqns_luns = method(connection_properties) except exception.TargetPortalsNotFound: raise except Exception: if 'target_portals' in connection_properties: raise exception.TargetPortalsNotFound( target_portals=connection_properties['target_portals']) if 'target_portal' in connection_properties: raise exception.TargetPortalNotFound( target_portal=connection_properties['target_portal']) raise if not connection_properties.get('target_iqns'): # There are two types of iSCSI multipath devices. One which # shares the same iqn between multiple portals, and the other # which use different iqns on different portals. # Try to identify the type by checking the iscsiadm output # if the iqn is used by multiple portals. If it is, it's # the former, so use the supplied iqn. Otherwise, it's the # latter, so try the ip,iqn combinations to find the targets # which constitutes the multipath device. main_iqn = connection_properties['target_iqn'] all_portals = {(ip, lun) for ip, iqn, lun in ips_iqns_luns} match_portals = {(ip, lun) for ip, iqn, lun in ips_iqns_luns if iqn == main_iqn} if len(all_portals) == len(match_portals): ips_iqns_luns = [(p[0], main_iqn, p[1]) for p in all_portals] return ips_iqns_luns def _get_potential_volume_paths(self, connection_properties): """Build a list of potential volume paths that exist. Given a list of target_portals in the connection_properties, a list of paths might exist on the system during discovery. This method's job is to build that list of potential paths for a volume that might show up. This is only used during get_volume_paths time, we are looking to find a list of existing volume paths for the connection_properties. In this case, we don't want to connect to the portal. If we blindly try and connect to a portal, it could create a new iSCSI session that didn't exist previously, and then leave it stale. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: list """ if self.use_multipath: LOG.info("Multipath discovery for iSCSI enabled") # Multipath installed, discovering other targets if available host_devices = self._get_device_path(connection_properties) else: LOG.info("Multipath discovery for iSCSI not enabled.") iscsi_sessions = self._get_iscsi_sessions() host_devices = set() for props in self._iterate_all_targets(connection_properties): # If we aren't trying to connect to the portal, we # want to find ALL possible paths from all of the # alternate portals if props['target_portal'] in iscsi_sessions: paths = self._get_device_path(props) host_devices.update(paths) host_devices = list(host_devices) return host_devices def set_execute(self, execute): super(ISCSIConnector, self).set_execute(execute) self._linuxscsi.set_execute(execute) def _validate_iface_transport(self, transport_iface): """Check that given iscsi_iface uses only supported transports Accepted transport names for provided iface param are be2iscsi, bnx2i, cxgb3i, cxgb4i, default, qla4xxx, ocs, iser or tcp. Note the difference between transport and iface; unlike default(iscsi_tcp)/iser, this is not one and the same for offloaded transports, where the default format is transport_name.hwaddress :param transport_iface: The iscsi transport type. :type transport_iface: str :returns: str """ # Note that default(iscsi_tcp) and iser do not require a separate # iface file, just the transport is enough and do not need to be # validated. This is not the case for the other entries in # supported_transports array. if transport_iface in ['default', 'iser']: return transport_iface # Will return (6) if iscsi_iface file was not found, or (2) if iscsid # could not be contacted out = self._run_iscsiadm_bare(['-m', 'iface', '-I', transport_iface], check_exit_code=[0, 2, 6])[0] or "" LOG.debug("iscsiadm %(iface)s configuration: stdout=%(out)s.", {'iface': transport_iface, 'out': out}) for data in [line.split() for line in out.splitlines()]: if data[0] == 'iface.transport_name': if data[2] in self.supported_transports: return transport_iface LOG.warning("No useable transport found for iscsi iface %s. " "Falling back to default transport.", transport_iface) return 'default' def _get_transport(self): return self.transport @staticmethod def _get_luns(con_props, iqns=None): luns = con_props.get('target_luns') num_luns = len(con_props['target_iqns']) if iqns is None else len(iqns) return luns or [con_props.get('target_lun')] * num_luns def _get_discoverydb_portals(self, connection_properties): """Retrieve iscsi portals information from the discoverydb. Example of discoverydb command output: SENDTARGETS: DiscoveryAddress: 192.168.1.33,3260 DiscoveryAddress: 192.168.1.2,3260 Target: iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-20170531114245.9eff88 Portal: 192.168.1.3:3260,1 Iface Name: default Portal: 192.168.1.2:3260,1 Iface Name: default Target: iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-20170531114447.9eff88 Portal: 192.168.1.3:3260,1 Iface Name: default Portal: 192.168.1.2:3260,1 Iface Name: default DiscoveryAddress: 192.168.1.38,3260 iSNS: No targets found. STATIC: No targets found. FIRMWARE: No targets found. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: list of tuples of (ip, iqn, lun) """ ip, port = connection_properties['target_portal'].rsplit(':', 1) # NOTE(geguileo): I don't know if IPv6 will be reported with [] # or not, so we'll make them optional. ip = ip.replace('[', '\[?').replace(']', '\]?') out = self._run_iscsiadm_bare(['-m', 'discoverydb', '-o', 'show', '-P', 1])[0] or "" regex = ''.join(('^SENDTARGETS:\n.*?^DiscoveryAddress: ', ip, ',', port, '.*?\n(.*?)^(?:DiscoveryAddress|iSNS):.*')) LOG.debug('Regex to get portals from discoverydb: %s', regex) info = re.search(regex, out, re.DOTALL | re.MULTILINE) ips = [] iqns = [] if info: iscsi_transport = ('iser' if self._get_transport() == 'iser' else 'default') iface = 'Iface Name: ' + iscsi_transport current_iqn = '' current_ip = '' for line in info.group(1).splitlines(): line = line.strip() if line.startswith('Target:'): current_iqn = line[8:] elif line.startswith('Portal:'): current_ip = line[8:].split(',')[0] elif line.startswith(iface): if current_iqn and current_ip: iqns.append(current_iqn) ips.append(current_ip) current_ip = '' if not iqns: raise exception.TargetPortalsNotFound( _('Unable to find target portals information on discoverydb.')) luns = self._get_luns(connection_properties, iqns) return list(zip(ips, iqns, luns)) def _discover_iscsi_portals(self, connection_properties): out = None iscsi_transport = ('iser' if self._get_transport() == 'iser' else 'default') if connection_properties.get('discovery_auth_method'): try: self._run_iscsiadm_update_discoverydb(connection_properties, iscsi_transport) except putils.ProcessExecutionError as exception: # iscsiadm returns 6 for "db record not found" if exception.exit_code == 6: # Create a new record for this target and update the db self._run_iscsiadm_bare( ['-m', 'discoverydb', '-t', 'sendtargets', '-p', connection_properties['target_portal'], '-I', iscsi_transport, '--op', 'new'], check_exit_code=[0, 255]) self._run_iscsiadm_update_discoverydb( connection_properties ) else: LOG.error("Unable to find target portal: " "%(target_portal)s.", {'target_portal': connection_properties[ 'target_portal']}) raise old_node_startups = self._get_node_startup_values( connection_properties) out = self._run_iscsiadm_bare( ['-m', 'discoverydb', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal'], '--discover'], check_exit_code=[0, 255])[0] or "" self._recover_node_startup_values(connection_properties, old_node_startups) else: old_node_startups = self._get_node_startup_values( connection_properties) out = self._run_iscsiadm_bare( ['-m', 'discovery', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal']], check_exit_code=[0, 255])[0] or "" self._recover_node_startup_values(connection_properties, old_node_startups) ips, iqns = self._get_target_portals_from_iscsiadm_output(out) luns = self._get_luns(connection_properties, iqns) return list(zip(ips, iqns, luns)) def _run_iscsiadm_update_discoverydb(self, connection_properties, iscsi_transport='default'): return self._execute( 'iscsiadm', '-m', 'discoverydb', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal'], '--op', 'update', '-n', "discovery.sendtargets.auth.authmethod", '-v', connection_properties['discovery_auth_method'], '-n', "discovery.sendtargets.auth.username", '-v', connection_properties['discovery_auth_username'], '-n', "discovery.sendtargets.auth.password", '-v', connection_properties['discovery_auth_password'], run_as_root=True, root_helper=self._root_helper) @utils.trace @synchronized('extend_volume') def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an iSCSI volume. """ LOG.info("Extend volume for %s", strutils.mask_dict_password(connection_properties)) volume_paths = self.get_volume_paths(connection_properties) LOG.info("Found paths for volume %s", volume_paths) if volume_paths: return self._linuxscsi.extend_volume(volume_paths) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': strutils.mask_dict_password( connection_properties)}) raise exception.VolumePathsNotFound() @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Attach the volume to instance_name. NOTE: Will retry up to three times to handle the case where c-vol and n-cpu are both using os-brick to manage iSCSI sessions but they are on the same node and using different locking directories. In this case, even though this call is synchronized, they will be separate locks and can still overlap with connect and disconnect. Since a disconnect during an initial attach can't cause IO failure (the device has not been made available yet), we just try the connection again. :param connection_properties: The valid dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for iSCSI must include: target_portal(s) - ip and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume Note that plural keys may be used when use_multipath=True """ try: if self.use_multipath: return self._connect_multipath_volume(connection_properties) return self._connect_single_volume(connection_properties) except Exception: # NOTE(geguileo): By doing the cleanup here we ensure we only do # the logins once for multipath if they succeed, but retry if they # don't, which helps on bad network cases. with excutils.save_and_reraise_exception(): self._cleanup_connection(connection_properties, force=True) def _get_device_link(self, wwn, device, mpath): # These are the default symlinks that should always be there if mpath: symlink = '/dev/disk/by-id/dm-uuid-mpath-' + mpath else: symlink = '/dev/disk/by-id/scsi-' + wwn # If default symlinks are not there just search for anything that links # to our device. In my experience this will return the last added link # first, so if we are going to succeed this should be fast. if not os.path.realpath(symlink) == device: links_path = '/dev/disk/by-id/' for symlink in os.listdir(links_path): symlink = links_path + symlink if os.path.realpath(symlink) == device: break else: # Raising this will trigger the next retry raise exception.VolumeDeviceNotFound(device='/dev/disk/by-id') return symlink def _get_connect_result(self, con_props, wwn, devices_names, mpath=None): device = '/dev/' + (mpath or devices_names[0]) # NOTE(geguileo): This is only necessary because of the current # encryption flow that requires that connect_volume returns a symlink # because first we do the volume attach, then the libvirt config is # generated using the path returned by the atach, and then we do the # encryption attach, which is forced to preserve the path that was used # in the libvirt config. If we fix that flow in OS-brick, Nova, and # Cinder we can remove this and just return the real path. if con_props.get('encrypted'): device = self._get_device_link(wwn, device, mpath) result = {'type': 'block', 'scsi_wwn': wwn, 'path': device} if mpath: result['multipath_id'] = wwn return result @utils.retry(exceptions=(exception.VolumeDeviceNotFound)) def _connect_single_volume(self, connection_properties): """Connect to a volume using a single path.""" data = {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': [], 'just_added_devices': []} for props in self._iterate_all_targets(connection_properties): self._connect_vol(self.device_scan_attempts, props, data) found_devs = data['found_devices'] if found_devs: for __ in range(10): wwn = self._linuxscsi.get_sysfs_wwn(found_devs) if wwn: return self._get_connect_result(connection_properties, wwn, found_devs) time.sleep(1) LOG.debug('Could not find the WWN for %s.', found_devs[0]) # If we failed we must cleanup the connection, as we could be # leaving the node entry if it's not being used by another device. ips_iqns_luns = ((props['target_portal'], props['target_iqn'], props['target_lun']), ) self._cleanup_connection(props, ips_iqns_luns, force=True, ignore_errors=True) # Reset connection result values for next try data.update(num_logins=0, failed_logins=0, found_devices=[]) raise exception.VolumeDeviceNotFound(device='') def _connect_vol(self, rescans, props, data): """Make a connection to a volume, send scans and wait for the device. This method is specifically designed to support multithreading and share the results via a shared dictionary with fixed keys, which is thread safe. Since the heaviest operations are run via subprocesses we don't worry too much about the GIL or how the eventlets will handle the context switching. The method will only try to log in once, since iscsid's initiator already tries 8 times by default to do the login, or whatever value we have as node.session.initial_login_retry_max in our system. Shared dictionary has the following keys: - stop_connecting: When the caller wants us to stop the rescans - num_logins: Count of how many threads have successfully logged in - failed_logins: Count of how many threads have failed to log in - stopped_threads: How many threads have finished. This may be different than num_logins + failed_logins, since some threads may still be waiting for a device. - found_devices: List of devices the connections have found - just_added_devices: Devices that have been found and still have not been processed by the main thread that manages all the connecting threads. :param rescans: Number of rescans to perform before giving up. :param props: Properties of the connection. :param data: Shared data. """ device = hctl = None portal = props['target_portal'] session, manual_scan = self._connect_to_iscsi_portal(props) do_scans = rescans > 0 retry = 1 if session: data['num_logins'] += 1 LOG.debug('Connected to %s', portal) while do_scans: try: if not hctl: hctl = self._linuxscsi.get_hctl(session, props['target_lun']) # Scan is sent on connect by iscsid, so skip first rescan # but on manual scan mode we have to do it ourselves. if hctl: if retry > 1 or manual_scan: self._linuxscsi.scan_iscsi(*hctl) device = self._linuxscsi.device_name_by_hctl(session, hctl) if device: break except Exception: LOG.exception('Exception scanning %s', portal) pass retry += 1 do_scans = (retry <= rescans and not (device or data['stop_connecting'])) if do_scans: time.sleep(retry ** 2) if device: LOG.debug('Connected to %s using %s', device, strutils.mask_password(props)) else: LOG.warning('LUN %(lun)s on iSCSI portal %(portal)s not found ' 'on sysfs after logging in.', {'lun': props['target_lun'], 'portal': portal}) else: LOG.warning('Failed to connect to iSCSI portal %s.', portal) data['failed_logins'] += 1 if device: data['found_devices'].append(device) data['just_added_devices'].append(device) data['stopped_threads'] += 1 @utils.retry(exceptions=(exception.VolumeDeviceNotFound)) def _connect_multipath_volume(self, connection_properties): """Connect to a multipathed volume launching parallel login requests. We will be doing parallel login requests, which will considerably speed up the process when we have flaky connections. We'll always try to return a multipath device even if there's only one path discovered, that way we can return once we have logged in in all the portals, because the paths will come up later. To make this possible we tell multipathd that the wwid is a multipath as soon as we have one device, and then hint multipathd to reconsider that volume for a multipath asking to add the path, because even if it's already known by multipathd it would have been discarded if it was the first time this volume was seen here. """ wwn = mpath = None wwn_added = last_try_on = False found = [] just_added_devices = [] # Dict used to communicate with threads as detailed in _connect_vol data = {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': found, 'just_added_devices': just_added_devices} ips_iqns_luns = self._get_ips_iqns_luns(connection_properties) # Launch individual threads for each session with the own properties retries = self.device_scan_attempts threads = [] for ip, iqn, lun in ips_iqns_luns: props = connection_properties.copy() props.update(target_portal=ip, target_iqn=iqn, target_lun=lun) threads.append(executor.Thread(target=self._connect_vol, args=(retries, props, data))) for thread in threads: thread.start() # Continue until: # - All connection attempts have finished and none has logged in # - Multipath has been found and connection attempts have either # finished or have already logged in # - We have finished in all threads, logged in, found some device, and # 10 seconds have passed, which should be enough with up to 10% # network package drops. while not ((len(ips_iqns_luns) == data['stopped_threads'] and not found) or (mpath and len(ips_iqns_luns) == data['num_logins'] + data['failed_logins'])): # We have devices but we don't know the wwn yet if not wwn and found: wwn = self._linuxscsi.get_sysfs_wwn(found) # We have the wwn but not a multipath if wwn and not mpath: mpath = self._linuxscsi.find_sysfs_multipath_dm(found) if not (mpath or wwn_added): # Tell multipathd that this wwn is a multipath and hint # multipathd to recheck all the devices we have just # connected. We only do this once, since for any new # device multipathd will already know it is a multipath. # This is only useful if we have multipathd configured with # find_multipaths set to yes, and has no effect if it's set # to no. wwn_added = self._linuxscsi.multipath_add_wwid(wwn) while not mpath and just_added_devices: device_path = '/dev/' + just_added_devices.pop(0) self._linuxscsi.multipath_add_path(device_path) mpath = self._linuxscsi.find_sysfs_multipath_dm(found) # Give some extra time after all threads have finished. if (not last_try_on and found and len(ips_iqns_luns) == data['stopped_threads']): LOG.debug('All connection threads finished, giving 10 seconds ' 'for dm to appear.') last_try_on = time.time() + 10 elif last_try_on and last_try_on < time.time(): break time.sleep(1) data['stop_connecting'] = True for thread in threads: thread.join() # If we haven't found any devices let the caller do the cleanup if not found: raise exception.VolumeDeviceNotFound(device='') # NOTE(geguileo): If we cannot find the dm it's because all paths are # really bad, so we might as well raise a not found exception, but # in our best effort we'll return a device even if it's probably # useless. if not mpath: LOG.warning('No dm was created, connection to volume is probably ' 'bad and will perform poorly.') return self._get_connect_result(connection_properties, wwn, found, mpath) def _get_connection_devices(self, connection_properties, ips_iqns_luns=None): """Get map of devices by sessions from our connection. For each of the TCP sessions that correspond to our connection properties we generate a map of (ip, iqn) to (belong, other) where belong is a set of devices in that session that populated our system when we did a connection using connection properties, and other are any other devices that share that same session but are the result of connecting with different connection properties. We also include all nodes from our connection that don't have a session. If ips_iqns_luns parameter is provided connection_properties won't be used to get them. When doing multipath we may not have all the information on the connection properties (sendtargets was used on connect) so we may have to retrieve the info from the discoverydb. Call _get_ips_iqns_luns to do the right things. """ if not ips_iqns_luns: if self.use_multipath: # We are only called from disconnect, so don't discover ips_iqns_luns = self._get_ips_iqns_luns(connection_properties, discover=False) else: ips_iqns_luns = self._get_all_targets(connection_properties) LOG.debug('Getting connected devices for (ips,iqns,luns)=%s', ips_iqns_luns) nodes = self._get_iscsi_nodes() sessions = self._get_iscsi_sessions_full() # Use (portal, iqn) to map the session value sessions_map = {(s[2], s[4]): s[1] for s in sessions if s[0] in self.VALID_SESSIONS_PREFIX} # device_map will keep a tuple with devices from the connection and # others that don't belong to this connection" (belong, others) device_map = collections.defaultdict(lambda: (set(), set())) for ip, iqn, lun in ips_iqns_luns: session = sessions_map.get((ip, iqn)) # Our nodes that don't have a session will be returned as empty if not session: if (ip, iqn) in nodes: device_map[(ip, iqn)] = (set(), set()) continue # Get all devices for the session paths = glob.glob('/sys/class/scsi_host/host*/device/session' + session + '/target*/*:*:*:*/block/*') belong, others = device_map[(ip, iqn)] for path in paths: __, hctl, __, device = path.rsplit('/', 3) lun_path = int(hctl.rsplit(':', 1)[-1]) # For partitions turn them into the whole device: sde1 -> sde device = device.strip('0123456789') if lun_path == lun: belong.add(device) else: others.add(device) LOG.debug('Resulting device map %s', device_map) return device_map @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict that must include: target_portal(s) - IP and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :type ignore_errors: bool """ return self._cleanup_connection(connection_properties, force=force, ignore_errors=ignore_errors) def _cleanup_connection(self, connection_properties, ips_iqns_luns=None, force=False, ignore_errors=False): """Cleans up connection flushing and removing devices and multipath. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict that must include: target_portal(s) - IP and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume :param ips_iqns_luns: Use this list of tuples instead of information from the connection_properties. :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :type ignore_errors: bool """ exc = exception.ExceptionChainer() try: devices_map = self._get_connection_devices(connection_properties, ips_iqns_luns) except exception.TargetPortalsNotFound as exc: # When discovery sendtargets failed on connect there is no # information in the discoverydb, so there's nothing to clean. LOG.debug('Skipping cleanup %s', exc) return # Remove devices and multipath from this connection remove_devices = set() for remove, __ in devices_map.values(): remove_devices.update(remove) multipath_name = self._linuxscsi.remove_connection(remove_devices, self.use_multipath, force, exc) # Disconnect sessions and remove nodes that are left without devices disconnect = [conn for conn, (__, keep) in devices_map.items() if not keep] self._disconnect_connection(connection_properties, disconnect, force, exc) # If flushing the multipath failed before, try now after we have # removed the devices and we may have even logged off (only reaches # here with multipath_name if force=True). if multipath_name: LOG.debug('Flushing again multipath %s now that we removed the ' 'devices.', multipath_name) self._linuxscsi.flush_multipath_device(multipath_name) if exc: LOG.warning('There were errors removing %s, leftovers may remain ' 'in the system', remove_devices) if not ignore_errors: raise exc def _munge_portal(self, target): """Remove brackets from portal. In case IPv6 address was used the udev path should not contain any brackets. Udev code specifically forbids that. """ portal, iqn, lun = target return (portal.replace('[', '').replace(']', ''), iqn, self._linuxscsi.process_lun_id(lun)) def _get_device_path(self, connection_properties): if self._get_transport() == "default": return ["/dev/disk/by-path/ip-%s-iscsi-%s-lun-%s" % self._munge_portal(x) for x in self._get_all_targets(connection_properties)] else: # we are looking for paths in the format : # /dev/disk/by-path/ # pci-XXXX:XX:XX.X-ip-PORTAL:PORT-iscsi-IQN-lun-LUN_ID device_list = [] for x in self._get_all_targets(connection_properties): look_for_device = glob.glob( '/dev/disk/by-path/*ip-%s-iscsi-%s-lun-%s' % self._munge_portal(x)) if look_for_device: device_list.extend(look_for_device) return device_list def get_initiator(self): """Secure helper to read file as root.""" file_path = '/etc/iscsi/initiatorname.iscsi' try: lines, _err = self._execute('cat', file_path, run_as_root=True, root_helper=self._root_helper) for l in lines.split('\n'): if l.startswith('InitiatorName='): return l[l.index('=') + 1:].strip() except putils.ProcessExecutionError: LOG.warning("Could not find the iSCSI Initiator File %s", file_path) return None def _run_iscsiadm(self, connection_properties, iscsi_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) attempts = kwargs.pop('attempts', 1) delay_on_retry = kwargs.pop('delay_on_retry', True) (out, err) = self._execute('iscsiadm', '-m', 'node', '-T', connection_properties['target_iqn'], '-p', connection_properties['target_portal'], *iscsi_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code, attempts=attempts, delay_on_retry=delay_on_retry) msg = ("iscsiadm %(iscsi_command)s: stdout=%(out)s stderr=%(err)s" % {'iscsi_command': iscsi_command, 'out': out, 'err': err}) # don't let passwords be shown in log output LOG.debug(strutils.mask_password(msg)) return (out, err) def _iscsiadm_update(self, connection_properties, property_key, property_value, **kwargs): iscsi_command = ('--op', 'update', '-n', property_key, '-v', property_value) return self._run_iscsiadm(connection_properties, iscsi_command, **kwargs) def _get_target_portals_from_iscsiadm_output(self, output): # return both portals and iqns as 2 lists # # as we are parsing a command line utility, allow for the # possibility that additional debug data is spewed in the # stream, and only grab actual ip / iqn lines. ips = [] iqns = [] for data in [line.split() for line in output.splitlines()]: if len(data) == 2 and data[1].startswith('iqn.'): ips.append(data[0].split(',')[0]) iqns.append(data[1]) return ips, iqns def _connect_to_iscsi_portal(self, connection_properties): """Connect to an iSCSI portal-target an return the session id.""" portal = connection_properties['target_portal'].split(",")[0] target_iqn = connection_properties['target_iqn'] # NOTE(vish): If we are on the same host as nova volume, the # discovery makes the target so we don't need to # run --op new. Therefore, we check to see if the # target exists, and if we get 255 (Not Found), then # we run --op new. This will also happen if another # volume is using the same target. # iscsiadm returns 21 for "No records found" after version 2.0-871 LOG.info("Trying to connect to iSCSI portal %s", portal) out, err = self._run_iscsiadm(connection_properties, (), check_exit_code=(0, 21, 255)) if err: self._run_iscsiadm(connection_properties, ('--interface', self._get_transport(), '--op', 'new')) # Try to set the scan mode to manual res = self._iscsiadm_update(connection_properties, 'node.session.scan', 'manual', check_exit_code=False) manual_scan = not res[1] if connection_properties.get('auth_method'): self._iscsiadm_update(connection_properties, "node.session.auth.authmethod", connection_properties['auth_method']) self._iscsiadm_update(connection_properties, "node.session.auth.username", connection_properties['auth_username']) self._iscsiadm_update(connection_properties, "node.session.auth.password", connection_properties['auth_password']) # We exit once we are logged in or once we fail login while True: # Duplicate logins crash iscsiadm after load, so we scan active # sessions to see if the node is logged in. sessions = self._get_iscsi_sessions_full() for s in sessions: # Found our session, return session_id if (s[0] in self.VALID_SESSIONS_PREFIX and portal == s[2] and s[4] == target_iqn): return s[1], manual_scan try: # exit_code=15 means the session already exists, so it should # be regarded as successful login. self._run_iscsiadm(connection_properties, ("--login",), check_exit_code=(0, 15, 255)) except putils.ProcessExecutionError as err: LOG.warning('Failed to login iSCSI target %(iqn)s on portal ' '%(portal)s (exit code %(err)s).', {'iqn': target_iqn, 'portal': portal, 'err': err.exit_code}) return None, None self._iscsiadm_update(connection_properties, "node.startup", "automatic") def _disconnect_from_iscsi_portal(self, connection_properties): self._iscsiadm_update(connection_properties, "node.startup", "manual", check_exit_code=[0, 21, 255]) self._run_iscsiadm(connection_properties, ("--logout",), check_exit_code=[0, 21, 255]) self._run_iscsiadm(connection_properties, ('--op', 'delete'), check_exit_code=[0, 21, 255], attempts=5, delay_on_retry=True) def _disconnect_connection(self, connection_properties, connections, force, exc): LOG.debug('Disconnecting from: %s', connections) props = connection_properties.copy() for ip, iqn in connections: props['target_portal'] = ip props['target_iqn'] = iqn with exc.context(force, 'Disconnect from %s %s failed', ip, iqn): self._disconnect_from_iscsi_portal(props) def _run_iscsi_session(self): (out, err) = self._run_iscsiadm_bare(('-m', 'session'), check_exit_code=[0, 1, 21, 255]) LOG.debug("iscsi session list stdout=%(out)s stderr=%(err)s", {'out': out, 'err': err}) return (out, err) def _run_iscsiadm_bare(self, iscsi_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) (out, err) = self._execute('iscsiadm', *iscsi_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code) LOG.debug("iscsiadm %(iscsi_command)s: stdout=%(out)s stderr=%(err)s", {'iscsi_command': iscsi_command, 'out': out, 'err': err}) return (out, err) def _run_multipath(self, multipath_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) (out, err) = self._execute('multipath', *multipath_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code) LOG.debug("multipath %(multipath_command)s: " "stdout=%(out)s stderr=%(err)s", {'multipath_command': multipath_command, 'out': out, 'err': err}) return (out, err) def _get_node_startup_values(self, connection_properties): out, __ = self._run_iscsiadm_bare( ['-m', 'node', '--op', 'show', '-p', connection_properties['target_portal']]) or "" node_values = out.strip() node_values = node_values.split("\n") iqn = None startup = None startup_values = {} for node_value in node_values: node_keys = node_value.split() try: if node_keys[0] == "node.name": iqn = node_keys[2] elif node_keys[0] == "node.startup": startup = node_keys[2] if iqn and startup: startup_values[iqn] = startup iqn = None startup = None except IndexError: pass return startup_values def _recover_node_startup_values(self, connection_properties, old_node_startups): node_startups = self._get_node_startup_values(connection_properties) for iqn, node_startup in node_startups.items(): old_node_startup = old_node_startups.get(iqn, None) if old_node_startup and node_startup != old_node_startup: # _iscsiadm_update() only uses "target_portal" and "target_iqn" # of connection_properties. # And the recovering target belongs to the same target_portal # as discovering target. # So target_iqn is updated, and other values aren't updated. recover_connection = copy.deepcopy(connection_properties) recover_connection['target_iqn'] = [iqn] self._iscsiadm_update(recover_connection, "node.startup", old_node_startup) os-brick-2.3.0/os_brick/initiator/connectors/vrtshyperscale.py0000666000175100017510000001327513230233223024656 0ustar zuulzuul00000000000000# Copyright (c) 2017 Veritas Technologies LLC. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-vrts-hyperscale-') class HyperScaleConnector(base.BaseLinuxConnector): """Class implements the os-brick connector for HyperScale volumes.""" def __init__(self, root_helper, driver=None, execute=None, *args, **kwargs): super(HyperScaleConnector, self).__init__( root_helper, driver=driver, execute=execute, *args, **kwargs) def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def extend_volume(self, connection_properties): raise NotImplementedError @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HyperScale connector properties.""" return {} @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Connect a volume to an instance.""" out = None err = None device_info = {} volume_name = None if 'name' in connection_properties.keys(): volume_name = connection_properties['name'] if volume_name is None: msg = _("Failed to connect volume: invalid volume name.") raise exception.BrickException(message=msg) cmd_arg = {'operation': 'connect_volume'} cmd_arg['volume_guid'] = volume_name cmdarg_json = json.dumps(cmd_arg) LOG.debug("HyperScale command hscli: %(cmd_arg)s", {'cmd_arg': cmdarg_json}) try: (out, err) = self._execute('hscli', cmdarg_json, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as e: msg = (_("Error executing hscli: %(err)s") % {'err': e.stderr}) raise exception.BrickException(message=msg) LOG.debug("Result of hscli: stdout=%(out)s " "stderr=%(err)s", {'out': out, 'err': err}) if err or out is None or len(out) == 0: msg = (_("Failed to connect volume with stdout=%(out)s " "stderr=%(err)s") % {'out': out, 'err': err}) raise exception.BrickException(message=msg) output = json.loads(out) payload = output.get('payload') if payload is None: msg = _("Failed to connect volume: " "hscli returned invalid payload") raise exception.BrickException(message=msg) if ('vsa_ip' not in payload.keys() or 'refl_factor' not in payload.keys()): msg = _("Failed to connect volume: " "hscli returned invalid results") raise exception.BrickException(message=msg) device_info['vsa_ip'] = payload.get('vsa_ip') device_info['path'] = ( '/dev/' + connection_properties['name'][1:32]) refl_factor = int(payload.get('refl_factor')) device_info['refl_factor'] = str(refl_factor) if refl_factor > 0: if 'refl_targets' not in payload.keys(): msg = _("Failed to connect volume: " "hscli returned inconsistent results") raise exception.BrickException(message=msg) device_info['refl_targets'] = ( payload.get('refl_targets')) return device_info @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from an instance.""" volume_name = None if 'name' in connection_properties.keys(): volume_name = connection_properties['name'] if volume_name is None: msg = _("Failed to disconnect volume: invalid volume name") raise exception.BrickException(message=msg) cmd_arg = {'operation': 'disconnect_volume'} cmd_arg['volume_guid'] = volume_name cmdarg_json = json.dumps(cmd_arg) LOG.debug("HyperScale command hscli: %(cmd_arg)s", {'cmd_arg': cmdarg_json}) try: (out, err) = self._execute('hscli', cmdarg_json, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as e: msg = (_("Error executing hscli: %(err)s") % {'err': e.stderr}) raise exception.BrickException(message=msg) if err: msg = (_("Failed to connect volume: stdout=%(out)s " "stderr=%(err)s") % {'out': out, 'err': err}) raise exception.BrickException(message=msg) os-brick-2.3.0/os_brick/initiator/connectors/vmware.py0000666000175100017510000002737613230233223023110 0ustar zuulzuul00000000000000# Copyright (c) 2016 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_log import log as logging from oslo_utils import fileutils try: from oslo_vmware import api from oslo_vmware import exceptions as oslo_vmw_exceptions from oslo_vmware import image_transfer from oslo_vmware.objects import datastore from oslo_vmware import rw_handles from oslo_vmware import vim_util except ImportError: vim_util = None import six from os_brick import exception from os_brick.i18n import _ from os_brick.initiator import initiator_connector LOG = logging.getLogger(__name__) class VmdkConnector(initiator_connector.InitiatorConnector): """Connector for volumes created by the VMDK driver. This connector is only used for backup and restore of Cinder volumes. """ TMP_IMAGES_DATASTORE_FOLDER_PATH = "cinder_temp" def __init__(self, *args, **kwargs): # Check if oslo.vmware library is available. if vim_util is None: message = _("Missing oslo_vmware python module, ensure oslo.vmware" " library is installed and available.") raise exception.BrickException(message=message) super(VmdkConnector, self).__init__(*args, **kwargs) self._ip = None self._port = None self._username = None self._password = None self._api_retry_count = None self._task_poll_interval = None self._ca_file = None self._insecure = None self._tmp_dir = None self._timeout = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): return {} def check_valid_device(self, path, *args, **kwargs): try: with open(path, 'r') as dev: dev.read(1) except IOError: LOG.exception( "Failed to access the device on the path " "%(path)s", {"path": path}) return False return True def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): pass def _load_config(self, connection_properties): config = connection_properties['config'] self._ip = config['vmware_host_ip'] self._port = config['vmware_host_port'] self._username = config['vmware_host_username'] self._password = config['vmware_host_password'] self._api_retry_count = config['vmware_api_retry_count'] self._task_poll_interval = config['vmware_task_poll_interval'] self._ca_file = config['vmware_ca_file'] self._insecure = config['vmware_insecure'] self._tmp_dir = config['vmware_tmp_dir'] self._timeout = config['vmware_image_transfer_timeout_secs'] def _create_session(self): return api.VMwareAPISession(self._ip, self._username, self._password, self._api_retry_count, self._task_poll_interval, port=self._port, cacert=self._ca_file, insecure=self._insecure) def _create_temp_file(self, *args, **kwargs): fileutils.ensure_tree(self._tmp_dir) fd, tmp = tempfile.mkstemp(dir=self._tmp_dir, *args, **kwargs) os.close(fd) return tmp def _download_vmdk( self, tmp_file_path, session, backing, vmdk_path, vmdk_size): with open(tmp_file_path, "wb") as tmp_file: image_transfer.copy_stream_optimized_disk( None, self._timeout, tmp_file, session=session, host=self._ip, port=self._port, vm=backing, vmdk_file_path=vmdk_path, vmdk_size=vmdk_size) def connect_volume(self, connection_properties): # Download the volume vmdk from vCenter server to a temporary file # and return its path. self._load_config(connection_properties) session = self._create_session() tmp_file_path = self._create_temp_file( suffix=".vmdk", prefix=connection_properties['volume_id']) backing = vim_util.get_moref(connection_properties['volume'], "VirtualMachine") vmdk_path = connection_properties['vmdk_path'] vmdk_size = connection_properties['vmdk_size'] try: self._download_vmdk( tmp_file_path, session, backing, vmdk_path, vmdk_size) finally: session.logout() # Save the last modified time of the temporary so that we can decide # whether to upload the file back to vCenter server during disconnect. last_modified = os.path.getmtime(tmp_file_path) return {'path': tmp_file_path, 'last_modified': last_modified} def _snapshot_exists(self, session, backing): snapshot = session.invoke_api(vim_util, 'get_object_property', session.vim, backing, 'snapshot') if snapshot is None or snapshot.rootSnapshotList is None: return False return len(snapshot.rootSnapshotList) != 0 def _create_temp_ds_folder(self, session, ds_folder_path, dc_ref): fileManager = session.vim.service_content.fileManager try: session.invoke_api(session.vim, 'MakeDirectory', fileManager, name=ds_folder_path, datacenter=dc_ref) except oslo_vmw_exceptions.FileAlreadyExistsException: pass # Note(vbala) remove this method when we implement it in oslo.vmware def _upload_vmdk( self, read_handle, host, port, dc_name, ds_name, cookies, upload_file_path, file_size, cacerts, timeout_secs): write_handle = rw_handles.FileWriteHandle(host, port, dc_name, ds_name, cookies, upload_file_path, file_size, cacerts=cacerts) image_transfer._start_transfer(read_handle, write_handle, timeout_secs) def _disconnect(self, tmp_file_path, session, ds_ref, dc_ref, vmdk_path): # The restored volume is in compressed (streamOptimized) format. # So we upload it to a temporary location in vCenter datastore and copy # the compressed vmdk to the volume vmdk. The copy operation # decompresses the disk to a format suitable for attaching to Nova # instances in vCenter. dstore = datastore.get_datastore_by_ref(session, ds_ref) ds_path = dstore.build_path( VmdkConnector.TMP_IMAGES_DATASTORE_FOLDER_PATH, os.path.basename(tmp_file_path)) self._create_temp_ds_folder( session, six.text_type(ds_path.parent), dc_ref) with open(tmp_file_path, "rb") as tmp_file: dc_name = session.invoke_api( vim_util, 'get_object_property', session.vim, dc_ref, 'name') cookies = session.vim.client.options.transport.cookiejar cacerts = self._ca_file if self._ca_file else not self._insecure self._upload_vmdk( tmp_file, self._ip, self._port, dc_name, dstore.name, cookies, ds_path.rel_path, os.path.getsize(tmp_file_path), cacerts, self._timeout) # Delete the current volume vmdk because the copy operation does not # overwrite. LOG.debug("Deleting %s", vmdk_path) disk_mgr = session.vim.service_content.virtualDiskManager task = session.invoke_api(session.vim, 'DeleteVirtualDisk_Task', disk_mgr, name=vmdk_path, datacenter=dc_ref) session.wait_for_task(task) src = six.text_type(ds_path) LOG.debug("Copying %(src)s to %(dest)s", {'src': src, 'dest': vmdk_path}) task = session.invoke_api(session.vim, 'CopyVirtualDisk_Task', disk_mgr, sourceName=src, sourceDatacenter=dc_ref, destName=vmdk_path, destDatacenter=dc_ref) session.wait_for_task(task) # Delete the compressed vmdk at the temporary location. LOG.debug("Deleting %s", src) file_mgr = session.vim.service_content.fileManager task = session.invoke_api(session.vim, 'DeleteDatastoreFile_Task', file_mgr, name=src, datacenter=dc_ref) session.wait_for_task(task) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): tmp_file_path = device_info['path'] if not os.path.exists(tmp_file_path): msg = _("Vmdk: %s not found.") % tmp_file_path raise exception.NotFound(message=msg) session = None try: # We upload the temporary file to vCenter server only if it is # modified after connect_volume. if os.path.getmtime(tmp_file_path) > device_info['last_modified']: self._load_config(connection_properties) session = self._create_session() backing = vim_util.get_moref(connection_properties['volume'], "VirtualMachine") # Currently there is no way we can restore the volume if it # contains redo-log based snapshots (bug 1599026). if self._snapshot_exists(session, backing): msg = (_("Backing of volume: %s contains one or more " "snapshots; cannot disconnect.") % connection_properties['volume_id']) raise exception.BrickException(message=msg) ds_ref = vim_util.get_moref( connection_properties['datastore'], "Datastore") dc_ref = vim_util.get_moref( connection_properties['datacenter'], "Datacenter") vmdk_path = connection_properties['vmdk_path'] self._disconnect( tmp_file_path, session, ds_ref, dc_ref, vmdk_path) finally: os.remove(tmp_file_path) if session: session.logout() def extend_volume(self, connection_properties): raise NotImplementedError os-brick-2.3.0/os_brick/initiator/connectors/storpool.py0000666000175100017510000002254213230233223023456 0ustar zuulzuul00000000000000# Copyright (c) 2015 - 2017 StorPool # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from __future__ import absolute_import import os import six from oslo_log import log as logging from oslo_utils import importutils from os_brick import exception from os_brick.initiator.connectors import base LOG = logging.getLogger(__name__) spopenstack = importutils.try_import('storpool.spopenstack') class StorPoolConnector(base.BaseLinuxConnector): """"Connector class to attach/detach StorPool volumes.""" def __init__(self, root_helper, driver=None, *args, **kwargs): super(StorPoolConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) if spopenstack is not None: try: self._attach = spopenstack.AttachDB(log=LOG) except Exception as e: raise exception.BrickException( 'Could not initialize the StorPool API bindings: %s' % (e)) else: self._attach = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The StorPool connector properties.""" return {} def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain the StorPool 'client_id' and the common 'volume' and 'access_mode' values. :type connection_properties: dict :returns: dict """ client_id = connection_properties.get('client_id', None) if client_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no client ID specified.') volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) mode = connection_properties.get('access_mode', None) if mode is None or mode not in ('rw', 'ro'): raise exception.BrickException( 'Invalid access_mode specified in the connection data.') req_id = 'brick-%s-%s' % (client_id, volume_id) self._attach.add(req_id, { 'volume': volume, 'type': 'brick', 'id': req_id, 'rights': 1 if mode == 'ro' else 2, 'volsnap': False }) self._attach.sync(req_id, None) return {'type': 'block', 'path': '/dev/storpool/' + volume} def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. The connection_properties are the same as from connect_volume. The device_info is returned from connect_volume. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain the StorPool 'client_id' and the common 'volume' values. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. For StorPool, this parameter is ignored, the volume is always detached. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. For StorPool, this parameter is ignored, no exception is raised except on unexpected errors. :type ignore_errors: bool """ client_id = connection_properties.get('client_id', None) if client_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no client ID specified.') volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) req_id = 'brick-%s-%s' % (client_id, volume_id) self._attach.sync(req_id, volume) self._attach.remove(req_id) def get_search_path(self): return '/dev/storpool' def get_volume_paths(self, connection_properties): """Return the list of existing paths for a volume. The job of this method is to find out what paths in the system are associated with a volume as described by the connection_properties. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain 'volume' and 'device_path' values. :type connection_properties: dict """ volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) path = '/dev/storpool/' + volume dpath = connection_properties.get('device_path', None) if dpath is not None and dpath != path: raise exception.BrickException( 'Internal error: StorPool volume path {path} does not ' 'match device path {dpath}', {path: path, dpath: dpath}) return [path] def get_all_available_volumes(self, connection_properties=None): """Return all volumes that exist in the search directory. At connect_volume time, a Connector looks in a specific directory to discover a volume's paths showing up. This method's job is to return all paths in the directory that connect_volume uses to find a volume. This method is used in coordination with get_volume_paths() to verify that volumes have gone away after disconnect_volume has been called. :param connection_properties: The dictionary that describes all of the target volume attributes. Unused for the StorPool connector. :type connection_properties: dict """ names = [] prefix = self._attach.volumeName('') prefixlen = len(prefix) if os.path.isdir('/dev/storpool'): files = os.listdir('/dev/storpool') for entry in files: full = '/dev/storpool/' + entry if entry.startswith(prefix) and os.path.islink(full) and \ not os.path.isdir(full): names.append(entry[prefixlen:]) return names def _get_device_size(self, device): """Get the size in bytes of a volume.""" (out, _err) = self._execute('blockdev', '--getsize64', device, run_as_root=True, root_helper=self._root_helper) var = six.text_type(out) if var.isnumeric(): return int(var) else: return None def extend_volume(self, connection_properties): """Update the attached volume's size. This method will attempt to update the local hosts's volume after the volume has been extended on the remote system. The new volume size in bytes will be returned. If there is a failure to update, then None will be returned. :param connection_properties: The volume connection properties. :returns: new size of the volume. """ # The StorPool client (storpool_block service) running on this host # should have picked up the change already, so it is enough to query # the actual disk device to see if its size is correct. # # TODO(pp): query the API to see if this is really the case volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) path = '/dev/storpool/' + volume return self._get_device_size(path) os-brick-2.3.0/os_brick/i18n.py0000666000175100017510000000153113230233223016170 0ustar zuulzuul00000000000000# Copyright 2014 IBM Corp. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """oslo.i18n integration module. See https://docs.openstack.org/oslo.i18n/latest/ . """ import oslo_i18n as i18n DOMAIN = 'os-brick' _translators = i18n.TranslatorFactory(domain=DOMAIN) # The primary translation function using the well-known name "_" _ = _translators.primary os-brick-2.3.0/os_brick/tests/0000775000175100017510000000000013230233405016201 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/test_utils.py0000666000175100017510000002367713230233223020771 0ustar zuulzuul00000000000000# (c) Copyright 2015 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import functools import time import mock from os_brick import exception from os_brick.tests import base from os_brick import utils class WrongException(exception.BrickException): pass class TestRetryDecorator(base.TestCase): def test_no_retry_required(self): self.counter = 0 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exceptions=exception.VolumeDeviceNotFound, interval=2, retries=3, backoff_rate=2) def succeeds(): self.counter += 1 return 'success' ret = succeeds() self.assertFalse(mock_sleep.called) self.assertEqual(ret, 'success') self.assertEqual(self.counter, 1) def test_retries_once(self): self.counter = 0 interval = 2 backoff_rate = 2 retries = 3 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exception.VolumeDeviceNotFound, interval, retries, backoff_rate) def fails_once(): self.counter += 1 if self.counter < 2: raise exception.VolumeDeviceNotFound(device='fake') else: return 'success' ret = fails_once() self.assertEqual(ret, 'success') self.assertEqual(self.counter, 2) self.assertEqual(mock_sleep.call_count, 1) mock_sleep.assert_called_with(interval * backoff_rate) def test_limit_is_reached(self): self.counter = 0 retries = 3 interval = 2 backoff_rate = 4 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exception.VolumeDeviceNotFound, interval, retries, backoff_rate) def always_fails(): self.counter += 1 raise exception.VolumeDeviceNotFound(device='fake') self.assertRaises(exception.VolumeDeviceNotFound, always_fails) self.assertEqual(retries, self.counter) expected_sleep_arg = [] for i in range(retries): if i > 0: interval *= backoff_rate expected_sleep_arg.append(float(interval)) mock_sleep.assert_has_calls( list(map(mock.call, expected_sleep_arg))) def test_wrong_exception_no_retry(self): with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exceptions=exception.VolumeDeviceNotFound) def raise_unexpected_error(): raise WrongException("wrong exception") self.assertRaises(WrongException, raise_unexpected_error) self.assertFalse(mock_sleep.called) class LogTracingTestCase(base.TestCase): """Test out the log tracing.""" def test_utils_trace_method_default_logger(self): mock_log = self.mock_object(utils, 'LOG') @utils.trace def _trace_test_method_custom_logger(*args, **kwargs): return 'OK' result = _trace_test_method_custom_logger() self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) def test_utils_trace_method_inner_decorator(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) def _test_decorator(f): def blah(*args, **kwargs): return f(*args, **kwargs) return blah @_test_decorator @utils.trace def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the correct function name was logged for call in mock_log.debug.call_args_list: self.assertIn('_trace_test_method', str(call)) self.assertNotIn('blah', str(call)) def test_utils_trace_method_outer_decorator(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) def _test_decorator(f): def blah(*args, **kwargs): return f(*args, **kwargs) return blah @utils.trace @_test_decorator def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the incorrect function name was logged for call in mock_log.debug.call_args_list: self.assertNotIn('_trace_test_method', str(call)) self.assertIn('blah', str(call)) def test_utils_trace_method_outer_decorator_with_functools(self): mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True self.mock_object(utils.logging, 'getLogger', mock_log) mock_log = self.mock_object(utils, 'LOG') def _test_decorator(f): @functools.wraps(f) def wraps(*args, **kwargs): return f(*args, **kwargs) return wraps @utils.trace @_test_decorator def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method() self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the incorrect function name was logged for call in mock_log.debug.call_args_list: self.assertIn('_trace_test_method', str(call)) self.assertNotIn('wraps', str(call)) def test_utils_trace_method_with_exception(self): self.LOG = self.mock_object(utils, 'LOG') @utils.trace def _trace_test_method(*args, **kwargs): raise exception.VolumeDeviceNotFound('test message') self.assertRaises(exception.VolumeDeviceNotFound, _trace_test_method) exception_log = self.LOG.debug.call_args_list[1] self.assertIn('exception', str(exception_log)) self.assertIn('test message', str(exception_log)) def test_utils_trace_method_with_time(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) mock_time = mock.Mock(side_effect=[3.1, 6]) self.mock_object(time, 'time', mock_time) @utils.trace def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) return_log = mock_log.debug.call_args_list[1] self.assertIn('2900', str(return_log)) def test_utils_trace_method_with_password_dict(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): return {'something': 'test', 'password': 'Now you see me'} result = _trace_test_method(self) expected_unmasked_dict = {'something': 'test', 'password': 'Now you see me'} self.assertEqual(expected_unmasked_dict, result) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'password': '***'", str(mock_log.debug.call_args_list[1])) def test_utils_trace_method_with_password_str(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): return "'adminPass': 'Now you see me'" result = _trace_test_method(self) expected_unmasked_str = "'adminPass': 'Now you see me'" self.assertEqual(expected_unmasked_str, result) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'adminPass': '***'", str(mock_log.debug.call_args_list[1])) def test_utils_trace_method_with_password_in_formal_params(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): self.assertEqual('verybadpass', kwargs['connection']['data']['auth_password']) pass connector_properties = { 'data': { 'auth_password': 'verybadpass' } } _trace_test_method(self, connection=connector_properties) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'auth_password': '***'", str(mock_log.debug.call_args_list[0])) os-brick-2.3.0/os_brick/tests/test_executor.py0000666000175100017510000001527713230233223021464 0ustar zuulzuul00000000000000# encoding=utf8 # (c) Copyright 2015 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import threading import mock from oslo_concurrency import processutils as putils from oslo_context import context as context_utils import six import testtools from os_brick import executor as brick_executor from os_brick.privileged import rootwrap from os_brick.tests import base class TestExecutor(base.TestCase): def test_default_execute(self): executor = brick_executor.Executor(root_helper=None) self.assertEqual(rootwrap.execute, executor._Executor__execute) def test_none_execute(self): executor = brick_executor.Executor(root_helper=None, execute=None) self.assertEqual(rootwrap.execute, executor._Executor__execute) def test_fake_execute(self): mock_execute = mock.Mock() executor = brick_executor.Executor(root_helper=None, execute=mock_execute) self.assertEqual(mock_execute, executor._Executor__execute) @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_str_exception(self, execute_mock, stdin_mock): execute_mock.side_effect = putils.ProcessExecutionError( stdout='España', stderr='Zürich') executor = brick_executor.Executor(root_helper=None) exc = self.assertRaises(putils.ProcessExecutionError, executor._execute) self.assertEqual(u'Espa\xf1a', exc.stdout) self.assertEqual(u'Z\xfcrich', exc.stderr) @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_str(self, execute_mock, stdin_mock): execute_mock.return_value = ('España', 'Zürich') executor = brick_executor.Executor(root_helper=None) stdout, stderr = executor._execute() self.assertEqual(u'Espa\xf1a', stdout) self.assertEqual(u'Z\xfcrich', stderr) @testtools.skipUnless(six.PY3, 'Specific test for Python 3') @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_bytes_exception(self, execute_mock, stdin_mock): execute_mock.side_effect = putils.ProcessExecutionError( stdout=six.binary_type('España', 'utf-8'), stderr=six.binary_type('Zürich', 'utf-8')) executor = brick_executor.Executor(root_helper=None) exc = self.assertRaises(putils.ProcessExecutionError, executor._execute) self.assertEqual(u'Espa\xf1a', exc.stdout) self.assertEqual(u'Z\xfcrich', exc.stderr) @testtools.skipUnless(six.PY3, 'Specific test for Python 3') @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_bytes(self, execute_mock, stdin_mock): execute_mock.return_value = (six.binary_type('España', 'utf-8'), six.binary_type('Zürich', 'utf-8')) executor = brick_executor.Executor(root_helper=None) stdout, stderr = executor._execute() self.assertEqual(u'Espa\xf1a', stdout) self.assertEqual(u'Z\xfcrich', stderr) class TestThread(base.TestCase): def _store_context(self, result): """Stores current thread's context in result list.""" result.append(context_utils.get_current()) def _run_threads(self, threads): for thread in threads: thread.start() for thread in threads: thread.join() def _do_test(self, thread_class, expected, result=None): if result is None: result = [] threads = [thread_class(target=self._store_context, args=[result]) for i in range(3)] self._run_threads(threads) self.assertEqual([expected] * len(threads), result) def test_normal_thread(self): """Test normal threads don't inherit parent's context.""" context = context_utils.RequestContext() context.update_store() self._do_test(threading.Thread, None) def test_no_context(self, result=None): """Test when parent has no context.""" context_utils._request_store.context = None self._do_test(brick_executor.Thread, None, result) def test_with_context(self, result=None): """Test that our class actually inherits the context.""" context = context_utils.RequestContext() context.update_store() self._do_test(brick_executor.Thread, context, result) def _run_test(self, test_method, test_args, result): """Run one of the normal tests and store the result. Meant to be run in a different thread, thus the need to store the result, because by the time the join call completes the test's stack is no longer available and the exception will have been lost. """ try: test_method(test_args) result.append(True) except Exception: result.append(False) raise def test_no_cross_mix(self): """Test there's no shared global context between threads.""" result = [] contexts = [[], [], []] threads = [threading.Thread(target=self._run_test, args=[self.test_with_context, contexts[0], result]), threading.Thread(target=self._run_test, args=[self.test_no_context, contexts[1], result]), threading.Thread(target=self._run_test, args=[self.test_with_context, contexts[2], result])] self._run_threads(threads) # Check that all tests run without raising an exception self.assertEqual([True, True, True], result) # Check that the context were not shared self.assertNotEqual(contexts[0], contexts[2]) os-brick-2.3.0/os_brick/tests/__init__.py0000666000175100017510000000000013230233223020300 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/remotefs/0000775000175100017510000000000013230233405020025 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/remotefs/__init__.py0000666000175100017510000000000013230233223022124 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/remotefs/test_windows_remotefs.py0000666000175100017510000001317013230233223025036 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import exception from os_brick.remotefs import windows_remotefs from os_brick.tests import base @ddt.ddt class WindowsRemotefsClientTestCase(base.TestCase): _FAKE_SHARE_NAME = 'fake_share' _FAKE_SHARE_SERVER = 'fake_share_server' _FAKE_SHARE = '\\\\%s\\%s' % (_FAKE_SHARE_SERVER, _FAKE_SHARE_NAME) @mock.patch.object(windows_remotefs, 'utilsfactory') def setUp(self, mock_utilsfactory): super(WindowsRemotefsClientTestCase, self).setUp() self._remotefs = windows_remotefs.WindowsRemoteFsClient( mount_type='smbfs') self._remotefs._mount_base = mock.sentinel.mount_base self._smbutils = self._remotefs._smbutils self._pathutils = self._remotefs._pathutils @ddt.data({}, {'expect_existing': False}, {'local_path': mock.sentinel.local_path}) @ddt.unpack def test_get_local_share_path(self, expect_existing=True, local_path=None): self._smbutils.get_smb_share_path.return_value = local_path if not local_path and expect_existing: self.assertRaises( exception.VolumePathsNotFound, self._remotefs.get_local_share_path, mock.sentinel.share_name, expect_existing=expect_existing) else: share_path = self._remotefs.get_local_share_path( mock.sentinel.share_name, expect_existing=expect_existing) self.assertEqual(local_path, share_path) def test_get_share_name(self): resulted_name = self._remotefs.get_share_name(self._FAKE_SHARE) self.assertEqual(self._FAKE_SHARE_NAME, resulted_name) @ddt.data(True, False) @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, '_create_mount_point') def test_mount(self, is_local_share, mock_create_mount_point): flags = '-o pass=password' self._remotefs._mount_options = '-o user=username,randomopt' self._remotefs._local_path_for_loopback = True self._smbutils.check_smb_mapping.return_value = False self._smbutils.is_local_share.return_value = is_local_share self._remotefs.mount(self._FAKE_SHARE, flags) if is_local_share: self.assertFalse(self._smbutils.check_smb_mapping.called) self.assertFalse(self._smbutils.mount_smb_share.called) else: self._smbutils.check_smb_mapping.assert_called_once_with( self._FAKE_SHARE) self._smbutils.mount_smb_share.assert_called_once_with( self._FAKE_SHARE, username='username', password='password') mock_create_mount_point.assert_called_once_with(self._FAKE_SHARE, is_local_share) def test_unmount(self): self._remotefs.unmount(self._FAKE_SHARE) self._smbutils.unmount_smb_share.assert_called_once_with( self._FAKE_SHARE) @ddt.data({'use_local_path': True}, {'path_exists': True, 'is_symlink': True}, {'path_exists': True}) @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, 'get_local_share_path') @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, 'get_mount_point') @mock.patch.object(windows_remotefs, 'os') @ddt.unpack def test_create_mount_point(self, mock_os, mock_get_mount_point, mock_get_local_share_path, path_exists=False, is_symlink=False, use_local_path=False): mock_os.path.exists.return_value = path_exists mock_os.isdir.return_value = False self._pathutils.is_symlink.return_value = is_symlink if path_exists and not is_symlink: self.assertRaises(exception.BrickException, self._remotefs._create_mount_point, self._FAKE_SHARE, use_local_path) else: self._remotefs._create_mount_point(self._FAKE_SHARE, use_local_path) mock_get_mount_point.assert_called_once_with(self._FAKE_SHARE) mock_os.path.isdir.assert_called_once_with(mock.sentinel.mount_base) if use_local_path: mock_get_local_share_path.assert_called_once_with( self._FAKE_SHARE_NAME) expected_symlink_target = mock_get_local_share_path.return_value else: expected_symlink_target = self._FAKE_SHARE.replace('/', '\\') if path_exists: self._pathutils.is_symlink.assert_called_once_with( mock_get_mount_point.return_value) else: self._pathutils.create_sym_link.assert_called_once_with( mock_get_mount_point.return_value, expected_symlink_target) os-brick-2.3.0/os_brick/tests/remotefs/test_remotefs.py0000666000175100017510000002457713230233223023301 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os import tempfile from oslo_concurrency import processutils as putils import six from os_brick import exception from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.remotefs import remotefs from os_brick.tests import base class RemoteFsClientTestCase(base.TestCase): def setUp(self): super(RemoteFsClientTestCase, self).setUp() self.mock_execute = self.mock_object(priv_rootwrap, 'execute', return_value=None) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_cifs(self, mock_read_mounts): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') share = '10.0.0.1:/qwe' mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('mount', '-t', 'cifs', share, mount_point, run_as_root=True, root_helper='true', check_exit_code=0)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_nfs(self, mock_read_mounts): client = remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/mnt') share = '10.0.0.1:/qwe' mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('mount', '-t', 'nfs', '-o', 'vers=4,minorversion=1', share, mount_point, check_exit_code=0, run_as_root=True, root_helper='true')] self.mock_execute.assert_has_calls(calls) def test_read_mounts(self): mounts = """device1 on mnt_point1 device2 on mnt_point2 type ext4 opts""" with mock.patch.object(priv_rootwrap, 'execute', return_value=[mounts, '']): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') ret = client._read_mounts() self.assertEqual(ret, {'mnt_point1': 'device1', 'mnt_point2': 'device2'}) @mock.patch.object(priv_rootwrap, 'execute') @mock.patch.object(remotefs.RemoteFsClient, '_do_mount') def test_mount_already_mounted(self, mock_do_mount, mock_execute): share = "10.0.0.1:/share" client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') mounts = {client.get_mount_point(share): 'some_dev'} with mock.patch.object(client, '_read_mounts', return_value=mounts): client.mount(share) self.assertEqual(mock_do_mount.call_count, 0) self.assertEqual(mock_execute.call_count, 0) def _test_no_mount_point(self, fs_type): self.assertRaises(exception.InvalidParameterValue, remotefs.RemoteFsClient, fs_type, root_helper='true') def test_no_mount_point_nfs(self): self._test_no_mount_point('nfs') def test_no_mount_point_cifs(self): self._test_no_mount_point('cifs') def test_no_mount_point_glusterfs(self): self._test_no_mount_point('glusterfs') def test_no_mount_point_vzstorage(self): self._test_no_mount_point('vzstorage') def test_no_mount_point_quobyte(self): self._test_no_mount_point('quobyte') def test_invalid_fs(self): self.assertRaises(exception.ProtocolNotSupported, remotefs.RemoteFsClient, 'my_fs', root_helper='true') def test_init_sets_mount_base(self): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/fake', cifs_mount_point_base='/fake2') # Tests that although the FS type is "cifs", the config option # starts with "smbfs_" self.assertEqual('/fake', client._mount_base) @mock.patch('os_brick.remotefs.remotefs.RemoteFsClient._check_nfs_options') def test_init_nfs_calls_check_nfs_options(self, mock_check_nfs_options): remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/fake') mock_check_nfs_options.assert_called_once_with() class VZStorageRemoteFSClientTestVase(RemoteFsClientTestCase): @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_by_cluster_name(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') share = 'qwe' cluster_name = share mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_with_auth(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') cluster_name = 'qwe' password = '123456' share = '%s:%s' % (cluster_name, password) mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('pstorage', '-c', cluster_name, 'auth-node', '-P', process_input=password, root_helper='true', run_as_root=True), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch('os.path.exists', return_value=False) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_with_mds_list(self, mock_read_mounts, mock_exists): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') cluster_name = 'qwe' mds_list = ['10.0.0.1', '10.0.0.2'] share = '%s:/%s' % (','.join(mds_list), cluster_name) mount_point = client.get_mount_point(share) vz_conf_dir = os.path.join('/etc/pstorage/clusters/', cluster_name) tmp_dir = '/tmp/fake_dir/' with mock.patch.object(tempfile, 'mkdtemp', return_value=tmp_dir): mock_open = mock.mock_open() with mock.patch.object(six.moves.builtins, "open", mock_open, create=True): client.mount(share) write_calls = [mock.call(tmp_dir + 'bs_list', 'w'), mock.call().__enter__(), mock.call().write('10.0.0.1\n'), mock.call().write('10.0.0.2\n'), mock.call().__exit__(None, None, None)] mock_open.assert_has_calls(write_calls) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('cp', '-rf', tmp_dir, vz_conf_dir, run_as_root=True, root_helper='true'), mock.call('chown', '-R', 'root:root', vz_conf_dir, run_as_root=True, root_helper='true'), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_invalid_share(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') self.assertRaises(exception.BrickException, client.mount, ':') class ScalityRemoteFsClientTestCase(base.TestCase): def test_no_mount_point_scality(self): self.assertRaises(exception.InvalidParameterValue, remotefs.ScalityRemoteFsClient, 'scality', root_helper='true') def test_get_mount_point(self): fsclient = remotefs.ScalityRemoteFsClient( 'scality', root_helper='true', scality_mount_point_base='/fake') self.assertEqual('/fake/path/00', fsclient.get_mount_point('path')) @mock.patch('oslo_concurrency.processutils.execute', return_value=None) @mock.patch('os_brick.remotefs.remotefs.RemoteFsClient._do_mount') def test_mount(self, mock_do_mount, mock_execute): fsclient = remotefs.ScalityRemoteFsClient( 'scality', root_helper='true', scality_mount_point_base='/fake', execute=putils.execute) with mock.patch.object(fsclient, '_read_mounts', return_value={}): fsclient.mount('fake') mock_execute.assert_called_once_with( 'mkdir', '-p', '/fake', check_exit_code=0) mock_do_mount.assert_called_once_with( 'sofs', '/etc/sfused.conf', '/fake') os-brick-2.3.0/os_brick/tests/base.py0000666000175100017510000000746413230233223017500 0ustar zuulzuul00000000000000# Copyright 2010-2011 OpenStack Foundation # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import logging import os import testtools import fixtures import mock from oslo_utils import strutils class TestCase(testtools.TestCase): """Test case base class for all unit tests.""" SENTINEL = object() def setUp(self): """Run before each test method to initialize test environment.""" super(TestCase, self).setUp() test_timeout = os.environ.get('OS_TEST_TIMEOUT', 0) try: test_timeout = int(test_timeout) except ValueError: # If timeout value is invalid do not set a timeout. test_timeout = 0 if test_timeout > 0: self.useFixture(fixtures.Timeout(test_timeout, gentle=True)) self.useFixture(fixtures.NestedTempfile()) self.useFixture(fixtures.TempHomeDir()) environ_enabled = (lambda var_name: strutils.bool_from_string(os.environ.get(var_name))) if environ_enabled('OS_STDOUT_CAPTURE'): stdout = self.useFixture(fixtures.StringStream('stdout')).stream self.useFixture(fixtures.MonkeyPatch('sys.stdout', stdout)) if environ_enabled('OS_STDERR_CAPTURE'): stderr = self.useFixture(fixtures.StringStream('stderr')).stream self.useFixture(fixtures.MonkeyPatch('sys.stderr', stderr)) if environ_enabled('OS_LOG_CAPTURE'): log_format = '%(levelname)s [%(name)s] %(message)s' if environ_enabled('OS_DEBUG'): level = logging.DEBUG else: level = logging.INFO self.useFixture(fixtures.LoggerFixture(nuke_handlers=False, format=log_format, level=level)) def _common_cleanup(self): """Runs after each test method to tear down test environment.""" # Stop any timers for x in self.injected: try: x.stop() except AssertionError: pass # Delete attributes that don't start with _ so they don't pin # memory around unnecessarily for the duration of the test # suite for key in [k for k in self.__dict__.keys() if k[0] != '_']: del self.__dict__[key] def log_level(self, level): """Set logging level to the specified value.""" log_root = logging.getLogger(None).logger log_root.setLevel(level) def mock_object(self, obj, attr_name, new_attr=SENTINEL, **kwargs): """Use python mock to mock an object attribute Mocks the specified objects attribute with the given value. Automatically performs 'addCleanup' for the mock. """ args = [obj, attr_name] if new_attr is not self.SENTINEL: args.append(new_attr) patcher = mock.patch.object(*args, **kwargs) mocked = patcher.start() self.addCleanup(patcher.stop) return mocked def patch(self, path, *args, **kwargs): """Use python mock to mock a path with automatic cleanup.""" patcher = mock.patch(path, *args, **kwargs) result = patcher.start() self.addCleanup(patcher.stop) return result os-brick-2.3.0/os_brick/tests/test_exception.py0000666000175100017510000000432413230233223021613 0ustar zuulzuul00000000000000 # Copyright 2010 United States Government as represented by the # Administrator of the National Aeronautics and Space Administration. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import six from os_brick import exception from os_brick.tests import base class BrickExceptionTestCase(base.TestCase): def test_default_error_msg(self): class FakeBrickException(exception.BrickException): message = "default message" exc = FakeBrickException() self.assertEqual(six.text_type(exc), 'default message') def test_error_msg(self): self.assertEqual(six.text_type(exception.BrickException('test')), 'test') def test_default_error_msg_with_kwargs(self): class FakeBrickException(exception.BrickException): message = "default message: %(code)s" exc = FakeBrickException(code=500) self.assertEqual(six.text_type(exc), 'default message: 500') def test_error_msg_exception_with_kwargs(self): class FakeBrickException(exception.BrickException): message = "default message: %(mispelled_code)s" exc = FakeBrickException(code=500) self.assertEqual(six.text_type(exc), 'default message: %(mispelled_code)s') def test_default_error_code(self): class FakeBrickException(exception.BrickException): code = 404 exc = FakeBrickException() self.assertEqual(exc.kwargs['code'], 404) def test_error_code_from_kwarg(self): class FakeBrickException(exception.BrickException): code = 500 exc = FakeBrickException(code=404) self.assertEqual(exc.kwargs['code'], 404) os-brick-2.3.0/os_brick/tests/windows/0000775000175100017510000000000013230233405017673 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/windows/test_iscsi.py0000666000175100017510000002203413230233223022417 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_win import exceptions as os_win_exc from os_brick import exception from os_brick.initiator.windows import iscsi from os_brick.tests.windows import test_base @ddt.ddt class WindowsISCSIConnectorTestCase(test_base.WindowsConnectorTestBase): @mock.patch.object(iscsi.WindowsISCSIConnector, 'validate_initiators') def setUp(self, mock_validate_connectors): super(WindowsISCSIConnectorTestCase, self).setUp() self._diskutils = mock.Mock() self._iscsi_utils = mock.Mock() self._connector = iscsi.WindowsISCSIConnector( device_scan_interval=mock.sentinel.rescan_interval) self._connector._diskutils = self._diskutils self._connector._iscsi_utils = self._iscsi_utils @ddt.data({'requested_initiators': [mock.sentinel.initiator_0], 'available_initiators': [mock.sentinel.initiator_0, mock.sentinel.initiator_1]}, {'requested_initiators': [mock.sentinel.initiator_0], 'available_initiators': [mock.sentinel.initiator_1]}, {'requested_initiators': [], 'available_initiators': [mock.sentinel.software_initiator]}) @ddt.unpack def test_validate_initiators(self, requested_initiators, available_initiators): self._iscsi_utils.get_iscsi_initiators.return_value = ( available_initiators) self._connector.initiator_list = requested_initiators expected_valid_initiator = not ( set(requested_initiators).difference(set(available_initiators))) valid_initiator = self._connector.validate_initiators() self.assertEqual(expected_valid_initiator, valid_initiator) def test_get_initiator(self): initiator = self._connector.get_initiator() self.assertEqual(self._iscsi_utils.get_iscsi_initiator.return_value, initiator) @mock.patch.object(iscsi, 'utilsfactory') def test_get_connector_properties(self, mock_utilsfactory): mock_iscsi_utils = ( mock_utilsfactory.get_iscsi_initiator_utils.return_value) props = self._connector.get_connector_properties() expected_props = dict( initiator=mock_iscsi_utils.get_iscsi_initiator.return_value) self.assertEqual(expected_props, props) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') def test_get_all_paths(self, mock_get_all_targets): initiators = [mock.sentinel.initiator_0, mock.sentinel.initiator_1] all_targets = [(mock.sentinel.portal_0, mock.sentinel.target_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.target_1, mock.sentinel.lun_1)] self._connector.initiator_list = initiators mock_get_all_targets.return_value = all_targets expected_paths = [ (initiator_name, target_portal, target_iqn, target_lun) for target_portal, target_iqn, target_lun in all_targets for initiator_name in initiators] all_paths = self._connector._get_all_paths(mock.sentinel.conn_props) self.assertEqual(expected_paths, all_paths) mock_get_all_targets.assert_called_once_with(mock.sentinel.conn_props) @ddt.data(True, False) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_scsi_wwn') @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_paths') def test_connect_volume(self, use_multipath, mock_get_all_paths, mock_get_scsi_wwn): fake_paths = [(mock.sentinel.initiator_name, mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun)] * 3 fake_conn_props = dict(auth_username=mock.sentinel.auth_username, auth_password=mock.sentinel.auth_password) mock_get_all_paths.return_value = fake_paths self._iscsi_utils.login_storage_target.side_effect = [ os_win_exc.OSWinException, None, None] self._iscsi_utils.get_device_number_and_path.return_value = ( mock.sentinel.device_number, mock.sentinel.device_path) self._connector.use_multipath = use_multipath device_info = self._connector.connect_volume(fake_conn_props) expected_device_info = dict(type='block', path=mock.sentinel.device_path, number=mock.sentinel.device_number, scsi_wwn=mock_get_scsi_wwn.return_value) self.assertEqual(expected_device_info, device_info) mock_get_all_paths.assert_called_once_with(fake_conn_props) expected_login_attempts = 3 if use_multipath else 2 self._iscsi_utils.login_storage_target.assert_has_calls( [mock.call(target_lun=mock.sentinel.target_lun, target_iqn=mock.sentinel.target_iqn, target_portal=mock.sentinel.target_portal, auth_username=mock.sentinel.auth_username, auth_password=mock.sentinel.auth_password, mpio_enabled=use_multipath, initiator_name=mock.sentinel.initiator_name, ensure_lun_available=False)] * expected_login_attempts) self._iscsi_utils.ensure_lun_available.assert_has_calls( [mock.call(target_iqn=mock.sentinel.target_iqn, target_lun=mock.sentinel.target_lun, rescan_attempts=( self._connector.device_scan_attempts), retry_interval=mock.sentinel.rescan_interval)] * (expected_login_attempts - 1)) self._iscsi_utils.get_device_number_and_path.assert_called_once_with( mock.sentinel.target_iqn, mock.sentinel.target_lun) mock_get_scsi_wwn.assert_called_once_with(mock.sentinel.device_number) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_paths') def test_connect_volume_exc(self, mock_get_all_paths): fake_paths = [(mock.sentinel.initiator_name, mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun)] * 3 mock_get_all_paths.return_value = fake_paths self._iscsi_utils.login_storage_target.side_effect = ( os_win_exc.OSWinException) self._connector.use_multipath = True self.assertRaises(exception.BrickException, self._connector.connect_volume, connection_properties={}) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') def test_disconnect_volume(self, mock_get_all_targets): targets = [ (mock.sentinel.portal_0, mock.sentinel.tg_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.tg_1, mock.sentinel.lun_1)] mock_get_all_targets.return_value = targets self._iscsi_utils.get_target_luns.return_value = [mock.sentinel.lun_0] self._connector.disconnect_volume(mock.sentinel.conn_props) self._diskutils.rescan_disks.assert_called_once_with() mock_get_all_targets.assert_called_once_with(mock.sentinel.conn_props) self._iscsi_utils.logout_storage_target.assert_called_once_with( mock.sentinel.tg_0) self._iscsi_utils.get_target_luns.assert_has_calls( [mock.call(mock.sentinel.tg_0), mock.call(mock.sentinel.tg_1)]) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') @mock.patch.object(iscsi.WindowsISCSIConnector, '_check_device_paths') def test_get_volume_paths(self, mock_check_dev_paths, mock_get_all_targets): targets = [ (mock.sentinel.portal_0, mock.sentinel.tg_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.tg_1, mock.sentinel.lun_1)] mock_get_all_targets.return_value = targets self._iscsi_utils.get_device_number_and_path.return_value = [ mock.sentinel.dev_num, mock.sentinel.dev_path] volume_paths = self._connector.get_volume_paths( mock.sentinel.conn_props) expected_paths = [mock.sentinel.dev_path] self.assertEqual(expected_paths, volume_paths) mock_check_dev_paths.assert_called_once_with(set(expected_paths)) os-brick-2.3.0/os_brick/tests/windows/__init__.py0000666000175100017510000000000013230233223021772 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/windows/test_fibre_channel.py0000666000175100017510000001650313230233223024070 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import exception from os_brick.initiator.windows import fibre_channel as fc from os_brick.tests.windows import test_base @ddt.ddt class WindowsFCConnectorTestCase(test_base.WindowsConnectorTestBase): def setUp(self): super(WindowsFCConnectorTestCase, self).setUp() self._connector = fc.WindowsFCConnector( device_scan_interval=mock.sentinel.rescan_interval) self._diskutils = self._connector._diskutils self._fc_utils = self._connector._fc_utils @ddt.data(True, False) @mock.patch.object(fc.utilsfactory, 'get_fc_utils') def test_get_volume_connector_props(self, valid_fc_hba_ports, mock_get_fc_utils): fake_fc_hba_ports = [{'node_name': mock.sentinel.node_name, 'port_name': mock.sentinel.port_name}, {'node_name': mock.sentinel.second_node_name, 'port_name': mock.sentinel.second_port_name}] self._fc_utils = mock_get_fc_utils.return_value self._fc_utils.get_fc_hba_ports.return_value = ( fake_fc_hba_ports if valid_fc_hba_ports else []) props = self._connector.get_connector_properties() self._fc_utils.refresh_hba_configuration.assert_called_once_with() self._fc_utils.get_fc_hba_ports.assert_called_once_with() if valid_fc_hba_ports: expected_props = { 'wwpns': [mock.sentinel.port_name, mock.sentinel.second_port_name], 'wwnns': [mock.sentinel.node_name, mock.sentinel.second_node_name] } else: expected_props = {} self.assertItemsEqual(expected_props, props) @mock.patch.object(fc.WindowsFCConnector, '_get_scsi_wwn') @mock.patch.object(fc.WindowsFCConnector, 'get_volume_paths') def test_connect_volume(self, mock_get_vol_paths, mock_get_scsi_wwn): mock_get_vol_paths.return_value = [mock.sentinel.dev_name] mock_get_dev_num = self._diskutils.get_device_number_from_device_name mock_get_dev_num.return_value = mock.sentinel.dev_num expected_device_info = dict(type='block', path=mock.sentinel.dev_name, number=mock.sentinel.dev_num, scsi_wwn=mock_get_scsi_wwn.return_value) device_info = self._connector.connect_volume(mock.sentinel.conn_props) self.assertEqual(expected_device_info, device_info) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) mock_get_dev_num.assert_called_once_with(mock.sentinel.dev_name) mock_get_scsi_wwn.assert_called_once_with(mock.sentinel.dev_num) @mock.patch.object(fc.WindowsFCConnector, 'get_volume_paths') def test_connect_volume_not_found(self, mock_get_vol_paths): mock_get_vol_paths.return_value = [] self.assertRaises(exception.NoFibreChannelVolumeDeviceFound, self._connector.connect_volume, mock.sentinel.conn_props) @ddt.data({'volume_mappings': [], 'expected_paths': []}, {'volume_mappings': [dict(device_name='')] * 3, 'expected_paths': []}, {'volume_mappings': [dict(device_name=''), dict(device_name=mock.sentinel.disk_path)], 'expected_paths': [mock.sentinel.disk_path]}) @ddt.unpack @mock.patch('time.sleep') @mock.patch.object(fc.WindowsFCConnector, '_get_fc_volume_mappings') @mock.patch.object(fc.WindowsFCConnector, '_check_device_paths') def test_get_volume_paths(self, mock_check_device_paths, mock_get_fc_mappings, mock_sleep, volume_mappings, expected_paths): mock_get_fc_mappings.return_value = volume_mappings vol_paths = self._connector.get_volume_paths(mock.sentinel.conn_props) self.assertEqual(expected_paths, vol_paths) # In this test case, either the volume is found after the first # attempt, either it's not found at all, in which case we'd expect # the number of retries to be the requested maximum number of rescans. expected_try_count = (1 if expected_paths else self._connector.device_scan_attempts) self._diskutils.rescan_disks.assert_has_calls( [mock.call()] * expected_try_count) mock_get_fc_mappings.assert_has_calls( [mock.call(mock.sentinel.conn_props)] * expected_try_count) mock_check_device_paths.assert_called_once_with( set(vol_paths)) mock_sleep.assert_has_calls( [mock.call(mock.sentinel.rescan_interval)] * (expected_try_count - 1)) @mock.patch.object(fc.WindowsFCConnector, '_get_fc_hba_mappings') def test_get_fc_volume_mappings(self, mock_get_fc_hba_mappings): fake_target_wwpn = 'FAKE_TARGET_WWPN' fake_conn_props = dict(target_lun=mock.sentinel.target_lun, target_wwn=[fake_target_wwpn]) mock_hba_mappings = {mock.sentinel.node_name: mock.sentinel.hba_ports} mock_get_fc_hba_mappings.return_value = mock_hba_mappings all_target_mappings = [{'device_name': mock.sentinel.dev_name, 'port_name': fake_target_wwpn, 'lun': mock.sentinel.target_lun}, {'device_name': mock.sentinel.dev_name_1, 'port_name': mock.sentinel.target_port_name_1, 'lun': mock.sentinel.target_lun}, {'device_name': mock.sentinel.dev_name, 'port_name': mock.sentinel.target_port_name, 'lun': mock.sentinel.target_lun_1}] expected_mappings = [all_target_mappings[0]] self._fc_utils.get_fc_target_mappings.return_value = ( all_target_mappings) volume_mappings = self._connector._get_fc_volume_mappings( fake_conn_props) self.assertEqual(expected_mappings, volume_mappings) def test_get_fc_hba_mappings(self): fake_fc_hba_ports = [{'node_name': mock.sentinel.node_name, 'port_name': mock.sentinel.port_name}] self._fc_utils.get_fc_hba_ports.return_value = fake_fc_hba_ports resulted_mappings = self._connector._get_fc_hba_mappings() expected_mappings = { mock.sentinel.node_name: [mock.sentinel.port_name]} self.assertEqual(expected_mappings, resulted_mappings) os-brick-2.3.0/os_brick/tests/windows/test_smbfs.py0000666000175100017510000001332513230233223022422 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import ddt import mock from os_brick.initiator.windows import smbfs from os_brick.remotefs import windows_remotefs from os_brick.tests.windows import test_base @ddt.ddt class WindowsSMBFSConnectorTestCase(test_base.WindowsConnectorTestBase): @mock.patch.object(windows_remotefs, 'WindowsRemoteFsClient') def setUp(self, mock_remotefs_cls): super(WindowsSMBFSConnectorTestCase, self).setUp() self._connector = smbfs.WindowsSMBFSConnector() self._remotefs = mock_remotefs_cls.return_value @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') @mock.patch.object(smbfs.WindowsSMBFSConnector, 'ensure_share_mounted') def test_connect_volume(self, mock_ensure_mounted, mock_get_disk_path): device_info = self._connector.connect_volume(mock.sentinel.conn_props) expected_info = dict(type='file', path=mock_get_disk_path.return_value) self.assertEqual(expected_info, device_info) mock_ensure_mounted.assert_called_once_with(mock.sentinel.conn_props) mock_get_disk_path.assert_called_once_with(mock.sentinel.conn_props) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_export_path') def test_disconnect_volume(self, mock_get_export_path): self._connector.disconnect_volume(mock.sentinel.conn_props) self._remotefs.unmount.assert_called_once_with( mock_get_export_path.return_value) mock_get_export_path.assert_called_once_with(mock.sentinel.conn_props) def test_get_export_path(self): fake_export = '//ip/share' fake_conn_props = dict(export=fake_export) expected_export = fake_export.replace('/', '\\') export_path = self._connector._get_export_path(fake_conn_props) self.assertEqual(expected_export, export_path) @ddt.data({}, {'mount_base': mock.sentinel.mount_base}, {'is_local_share': True}, {'is_local_share': True, 'local_path_for_loopbk': True}) @ddt.unpack def test_get_disk_path(self, mount_base=None, local_path_for_loopbk=False, is_local_share=False): fake_mount_point = r'C:\\fake_mount_point' fake_share_name = 'fake_share' fake_local_share_path = 'C:\\%s' % fake_share_name fake_export_path = '\\\\host\\%s' % fake_share_name fake_disk_name = 'fake_disk.vhdx' fake_conn_props = dict(name=fake_disk_name, export=fake_export_path) self._remotefs.get_mount_base.return_value = mount_base self._remotefs.get_mount_point.return_value = fake_mount_point self._remotefs.get_local_share_path.return_value = ( fake_local_share_path) self._remotefs.get_share_name.return_value = fake_share_name self._connector._local_path_for_loopback = local_path_for_loopbk self._connector._smbutils.is_local_share.return_value = is_local_share expecting_local = local_path_for_loopbk and is_local_share if mount_base: expected_export_path = fake_mount_point elif expecting_local: # In this case, we expect the local share export path to be # used directly. expected_export_path = fake_local_share_path else: expected_export_path = fake_export_path expected_disk_path = os.path.join(expected_export_path, fake_disk_name) disk_path = self._connector._get_disk_path(fake_conn_props) self.assertEqual(expected_disk_path, disk_path) if mount_base: self._remotefs.get_mount_point.assert_called_once_with( fake_export_path) elif expecting_local: self._connector._smbutils.is_local_share.assert_called_once_with( fake_export_path) self._remotefs.get_share_name.assert_called_once_with( fake_export_path) self._remotefs.get_local_share_path.assert_called_once_with( fake_share_name) def test_get_search_path(self): search_path = self._connector.get_search_path() self.assertEqual(search_path, self._remotefs.get_mount_base.return_value) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') def test_volume_paths(self, mock_get_disk_path): expected_paths = [mock_get_disk_path.return_value] volume_paths = self._connector.get_volume_paths( mock.sentinel.conn_props) self.assertEqual(expected_paths, volume_paths) mock_get_disk_path.assert_called_once_with( mock.sentinel.conn_props) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_export_path') def test_ensure_share_mounted(self, mock_get_export_path): fake_conn_props = dict(options=mock.sentinel.mount_opts) self._connector.ensure_share_mounted(fake_conn_props) self._remotefs.mount.assert_called_once_with( mock_get_export_path.return_value, mock.sentinel.mount_opts) os-brick-2.3.0/os_brick/tests/windows/test_base_connector.py0000666000175100017510000001315513230233223024275 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock import six from os_brick import exception from os_brick.initiator.windows import base as base_win_conn from os_brick.tests.windows import fake_win_conn from os_brick.tests.windows import test_base @ddt.ddt class BaseWindowsConnectorTestCase(test_base.WindowsConnectorTestBase): def setUp(self): super(BaseWindowsConnectorTestCase, self).setUp() self._diskutils = mock.Mock() self._connector = fake_win_conn.FakeWindowsConnector() self._connector._diskutils = self._diskutils @ddt.data({}, {'feature_available': True}, {'feature_available': False, 'enforce_multipath': True}) @ddt.unpack @mock.patch.object(base_win_conn.utilsfactory, 'get_hostutils') def test_check_multipath_support(self, mock_get_hostutils, feature_available=True, enforce_multipath=False): mock_hostutils = mock_get_hostutils.return_value mock_hostutils.check_server_feature.return_value = feature_available check_mpio = base_win_conn.BaseWindowsConnector.check_multipath_support if feature_available or not enforce_multipath: multipath_support = check_mpio( enforce_multipath=enforce_multipath) self.assertEqual(feature_available, multipath_support) else: self.assertRaises(exception.BrickException, check_mpio, enforce_multipath=enforce_multipath) mock_hostutils.check_server_feature.assert_called_once_with( mock_hostutils.FEATURE_MPIO) @ddt.data({}, {'mpio_requested': False}, {'mpio_available': True}) @mock.patch.object(base_win_conn.BaseWindowsConnector, 'check_multipath_support') @ddt.unpack def test_get_connector_properties(self, mock_check_mpio, mpio_requested=True, mpio_available=True): mock_check_mpio.return_value = mpio_available enforce_multipath = False props = base_win_conn.BaseWindowsConnector.get_connector_properties( multipath=mpio_requested, enforce_multipath=enforce_multipath) self.assertEqual(mpio_requested and mpio_available, props['multipath']) if mpio_requested: mock_check_mpio.assert_called_once_with(enforce_multipath) def test_get_scsi_wwn(self): mock_get_uid_and_type = self._diskutils.get_disk_uid_and_uid_type mock_get_uid_and_type.return_value = (mock.sentinel.disk_uid, mock.sentinel.uid_type) scsi_wwn = self._connector._get_scsi_wwn(mock.sentinel.dev_num) expected_wwn = '%s%s' % (mock.sentinel.uid_type, mock.sentinel.disk_uid) self.assertEqual(expected_wwn, scsi_wwn) mock_get_uid_and_type.assert_called_once_with(mock.sentinel.dev_num) @ddt.data(None, IOError) @mock.patch.object(six.moves.builtins, 'open') def test_check_valid_device(self, exc, mock_open): mock_open.side_effect = exc valid_device = self._connector.check_valid_device( mock.sentinel.dev_path) self.assertEqual(not exc, valid_device) mock_open.assert_any_call(mock.sentinel.dev_path, 'r') mock_read = mock_open.return_value.__enter__.return_value.read if not exc: mock_read.assert_called_once_with(1) def test_check_device_paths(self): # We expect an exception to be raised if the same volume # can be accessed through multiple paths. device_paths = [mock.sentinel.dev_path_0, mock.sentinel.dev_path_1] self.assertRaises(exception.BrickException, self._connector._check_device_paths, device_paths) @mock.patch.object(fake_win_conn.FakeWindowsConnector, 'get_volume_paths') def test_extend_volume(self, mock_get_vol_paths): mock_vol_paths = [mock.sentinel.dev_path] mock_get_vol_paths.return_value = mock_vol_paths self._connector.extend_volume(mock.sentinel.conn_props) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) mock_get_dev_num = self._diskutils.get_device_number_from_device_name mock_get_dev_num.assert_called_once_with(mock.sentinel.dev_path) self._diskutils.refresh_disk.assert_called_once_with( mock_get_dev_num.return_value) @mock.patch.object(fake_win_conn.FakeWindowsConnector, 'get_volume_paths') def test_extend_volume_missing_path(self, mock_get_vol_paths): mock_get_vol_paths.return_value = [] self.assertRaises(exception.NotFound, self._connector.extend_volume, mock.sentinel.conn_props) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) os-brick-2.3.0/os_brick/tests/windows/test_base.py0000666000175100017510000000252313230233223022220 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_win import utilsfactory from os_brick.tests import base class WindowsConnectorTestBase(base.TestCase): @mock.patch('sys.platform', 'win32') def setUp(self): super(WindowsConnectorTestBase, self).setUp() # All the Windows connectors use os_win.utilsfactory to fetch Windows # specific utils. During init, those will run methods that will fail # on other platforms. To make testing easier and avoid checking the # platform in the code, we can simply mock this factory method. utilsfactory_patcher = mock.patch.object( utilsfactory, '_get_class') utilsfactory_patcher.start() self.addCleanup(utilsfactory_patcher.stop) os-brick-2.3.0/os_brick/tests/windows/fake_win_conn.py0000666000175100017510000000225213230233223023046 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.windows import base as win_conn_base class FakeWindowsConnector(win_conn_base.BaseWindowsConnector): def connect_volume(self, connection_properties): return {} def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): pass def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): return [] os-brick-2.3.0/os_brick/tests/windows/test_factory.py0000666000175100017510000000301313230233223022750 0ustar zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import initiator from os_brick.initiator import connector from os_brick.initiator.windows import fibre_channel from os_brick.initiator.windows import iscsi from os_brick.initiator.windows import smbfs from os_brick.tests.windows import test_base @ddt.ddt class WindowsConnectorFactoryTestCase(test_base.WindowsConnectorTestBase): @ddt.data({'proto': initiator.ISCSI, 'expected_cls': iscsi.WindowsISCSIConnector}, {'proto': initiator.FIBRE_CHANNEL, 'expected_cls': fibre_channel.WindowsFCConnector}, {'proto': initiator.SMBFS, 'expected_cls': smbfs.WindowsSMBFSConnector}) @ddt.unpack @mock.patch('sys.platform', 'win32') def test_factory(self, proto, expected_cls): obj = connector.InitiatorConnector.factory(proto, None) self.assertIsInstance(obj, expected_cls) os-brick-2.3.0/os_brick/tests/test_brick.py0000666000175100017510000000135013230233223020703 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ test_os_brick ---------------------------------- Tests for `os_brick` module. """ from os_brick.tests import base class TestBrick(base.TestCase): def test_something(self): pass os-brick-2.3.0/os_brick/tests/local_dev/0000775000175100017510000000000013230233405020131 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/local_dev/__init__.py0000666000175100017510000000000013230233223022230 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/local_dev/test_brick_lvm.py0000666000175100017510000004017213230233223023516 0ustar zuulzuul00000000000000# Copyright 2012 OpenStack Foundation # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from oslo_concurrency import processutils from os_brick import exception from os_brick.local_dev import lvm as brick from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base class BrickLvmTestCase(base.TestCase): def setUp(self): super(BrickLvmTestCase, self).setUp() self.volume_group_name = 'fake-vg' # Stub processutils.execute for static methods self.mock_object(priv_rootwrap, 'execute', self.fake_execute) self.vg = brick.LVM(self.volume_group_name, 'sudo', create_vg=False, physical_volumes=None, lvm_type='default', executor=self.fake_execute) def failed_fake_execute(obj, *cmd, **kwargs): return ("\n", "fake-error") def fake_pretend_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.03.00 (2012-03-06)\n", "") def fake_old_lvm_version(obj, *cmd, **kwargs): # Does not support thin prov or snap activation return (" LVM version: 2.02.65(2) (2012-03-06)\n", "") def fake_customised_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.02.100(2)-RHEL6 (2013-09-12)\n", "") def fake_f23_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.02.132(2) (2015-09-22)\n", "") def fake_execute(obj, *cmd, **kwargs): # TODO(eharney): remove this and move to per-test mocked execute calls cmd_string = ', '.join(cmd) data = "\n" if ('env, LC_ALL=C, vgs, --noheadings, --unit=g, -o, name' == cmd_string): data = " fake-vg\n" data += " some-other-vg\n" elif ('env, LC_ALL=C, vgs, --noheadings, -o, name, fake-vg' == cmd_string): data = " fake-vg\n" elif 'env, LC_ALL=C, vgs, --version' in cmd_string: data = " LVM version: 2.02.95(2) (2012-03-06)\n" elif ('env, LC_ALL=C, vgs, --noheadings, -o, uuid, fake-vg' in cmd_string): data = " kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1\n" elif 'env, LC_ALL=C, vgs, --noheadings, --unit=g, ' \ '-o, name,size,free,lv_count,uuid, ' \ '--separator, :, --nosuffix' in cmd_string: data = (" test-prov-cap-vg-unit:10.00:10.00:0:" "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z4\n") if 'test-prov-cap-vg-unit' in cmd_string: return (data, "") data = (" test-prov-cap-vg-no-unit:10.00:10.00:0:" "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z4\n") if 'test-prov-cap-vg-no-unit' in cmd_string: return (data, "") data = " fake-vg:10.00:10.00:0:"\ "kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1\n" if 'fake-vg' in cmd_string: return (data, "") data += " fake-vg-2:10.00:10.00:0:"\ "lWyauW-dKpG-Rz7E-xtKY-jeju-QsYU-SLG7Z2\n" data += " fake-vg-3:10.00:10.00:0:"\ "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z3\n" elif ('env, LC_ALL=C, lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size, --nosuffix, ' 'fake-vg/lv-nothere' in cmd_string): raise processutils.ProcessExecutionError( stderr="One or more specified logical volume(s) not found.") elif ('env, LC_ALL=C, lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size, --nosuffix, ' 'fake-vg/lv-newerror' in cmd_string): raise processutils.ProcessExecutionError( stderr="Failed to find logical volume \"fake-vg/lv-newerror\"") elif ('env, LC_ALL=C, lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size' in cmd_string): if 'fake-unknown' in cmd_string: raise processutils.ProcessExecutionError( stderr="One or more volume(s) not found." ) if 'test-prov-cap-vg-unit' in cmd_string: data = " fake-vg test-prov-cap-pool-unit 9.50g\n" data += " fake-vg fake-volume-1 1.00g\n" data += " fake-vg fake-volume-2 2.00g\n" elif 'test-prov-cap-vg-no-unit' in cmd_string: data = " fake-vg test-prov-cap-pool-no-unit 9.50\n" data += " fake-vg fake-volume-1 1.00\n" data += " fake-vg fake-volume-2 2.00\n" elif 'test-found-lv-name' in cmd_string: data = " fake-vg test-found-lv-name 9.50\n" else: data = " fake-vg fake-1 1.00g\n" data += " fake-vg fake-2 1.00g\n" elif ('env, LC_ALL=C, lvdisplay, --noheading, -C, -o, Attr' in cmd_string): if 'test-volumes' in cmd_string: data = ' wi-a-' else: data = ' owi-a-' elif 'env, LC_ALL=C, pvs, --noheadings' in cmd_string: data = " fake-vg|/dev/sda|10.00|1.00\n" data += " fake-vg|/dev/sdb|10.00|1.00\n" data += " fake-vg|/dev/sdc|10.00|8.99\n" data += " fake-vg-2|/dev/sdd|10.00|9.99\n" elif 'env, LC_ALL=C, lvs, --noheadings, --unit=g' \ ', -o, size,data_percent, --separator, :' in cmd_string: if 'test-prov-cap-pool' in cmd_string: data = " 9.5:20\n" else: data = " 9:12\n" elif 'lvcreate, -T, -L, ' in cmd_string: pass elif 'lvcreate, -T, -l, 100%FREE' in cmd_string: pass elif 'lvcreate, -T, -V, ' in cmd_string: pass elif 'lvcreate, -n, ' in cmd_string: pass elif 'lvcreate, --name, ' in cmd_string: pass elif 'lvextend, -L, ' in cmd_string: pass else: raise AssertionError('unexpected command called: %s' % cmd_string) return (data, "") def test_create_lv_snapshot(self): self.assertIsNone(self.vg.create_lv_snapshot('snapshot-1', 'fake-1')) with mock.patch.object(self.vg, 'get_volume', return_value=None): try: self.vg.create_lv_snapshot('snapshot-1', 'fake-non-existent') except exception.VolumeDeviceNotFound as e: self.assertEqual('fake-non-existent', e.kwargs['device']) else: self.fail("Exception not raised") def test_vg_exists(self): self.assertTrue(self.vg._vg_exists()) def test_get_vg_uuid(self): self.assertEqual('kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1', self.vg._get_vg_uuid()[0]) def test_get_all_volumes(self): out = self.vg.get_volumes() self.assertEqual('fake-1', out[0]['name']) self.assertEqual('1.00g', out[0]['size']) self.assertEqual('fake-vg', out[0]['vg']) def test_get_volume(self): self.assertEqual('fake-1', self.vg.get_volume('fake-1')['name']) def test_get_volume_none(self): self.assertIsNone(self.vg.get_volume('fake-unknown')) def test_get_lv_info_notfound(self): # lv-nothere will raise lvm < 2.102.112 exception self.assertEqual( [], self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='lv-nothere') ) # lv-newerror will raise lvm > 2.102.112 exception self.assertEqual( [], self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='lv-newerror') ) def test_get_lv_info_found(self): lv_info = [{'size': '9.50', 'name': 'test-found-lv-name', 'vg': 'fake-vg'}] self.assertEqual( lv_info, self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='test-found-lv-name') ) def test_get_lv_info_no_lv_name(self): lv_info = [{'name': 'fake-1', 'size': '1.00g', 'vg': 'fake-vg'}, {'name': 'fake-2', 'size': '1.00g', 'vg': 'fake-vg'}] self.assertEqual( lv_info, self.vg.get_lv_info( 'sudo', vg_name='fake-vg') ) def test_get_all_physical_volumes(self): # Filtered VG version pvs = self.vg.get_all_physical_volumes('sudo', 'fake-vg') self.assertEqual(3, len(pvs)) # Non-Filtered, all VG's pvs = self.vg.get_all_physical_volumes('sudo') self.assertEqual(4, len(pvs)) def test_get_physical_volumes(self): pvs = self.vg.get_physical_volumes() self.assertEqual(3, len(pvs)) def test_get_volume_groups(self): self.assertEqual(3, len(self.vg.get_all_volume_groups('sudo'))) self.assertEqual(1, len(self.vg.get_all_volume_groups('sudo', 'fake-vg'))) def test_thin_support(self): # lvm.supports_thin() is a static method and doesn't # use the self._executor fake we pass in on init # so we need to stub processutils.execute appropriately with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_execute): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_pretend_lvm_version): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_customised_lvm_version): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) def test_snapshot_lv_activate_support(self): self.vg._supports_snapshot_lv_activation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_execute): self.assertTrue(self.vg.supports_snapshot_lv_activation) self.vg._supports_snapshot_lv_activation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_snapshot_lv_activation) self.vg._supports_snapshot_lv_activation = None def test_lvchange_ignskipact_support_yes(self): """Tests if lvchange -K is available via a lvm2 version check.""" self.vg._supports_lvchange_ignoreskipactivation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_pretend_lvm_version): self.assertTrue(self.vg.supports_lvchange_ignoreskipactivation) self.vg._supports_lvchange_ignoreskipactivation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_lvchange_ignoreskipactivation) self.vg._supports_lvchange_ignoreskipactivation = None def test_thin_pool_creation_manual(self): # The size of fake-vg volume group is 10g, so the calculated thin # pool size should be 9.5g (95% of 10g). self.vg.create_thin_pool() def test_thin_pool_provisioned_capacity(self): self.vg.vg_thin_pool = "test-prov-cap-pool-unit" self.vg.vg_name = 'test-prov-cap-vg-unit' self.assertIsNone(self.vg.create_thin_pool(name=self.vg.vg_thin_pool)) self.assertEqual(9.50, self.vg.vg_thin_pool_size) self.assertEqual(7.6, self.vg.vg_thin_pool_free_space) self.assertEqual(3.0, self.vg.vg_provisioned_capacity) self.vg.vg_thin_pool = "test-prov-cap-pool-no-unit" self.vg.vg_name = 'test-prov-cap-vg-no-unit' self.assertIsNone(self.vg.create_thin_pool(name=self.vg.vg_thin_pool)) self.assertEqual(9.50, self.vg.vg_thin_pool_size) self.assertEqual(7.6, self.vg.vg_thin_pool_free_space) self.assertEqual(3.0, self.vg.vg_provisioned_capacity) def test_thin_pool_free_space(self): # The size of fake-vg-pool is 9g and the allocated data sums up to # 12% so the calculated free space should be 7.92 self.assertEqual(float("7.92"), self.vg._get_thin_pool_free_space("fake-vg", "fake-vg-pool")) def test_volume_create_after_thin_creation(self): """Test self.vg.vg_thin_pool is set to pool_name See bug #1220286 for more info. """ vg_name = "vg-name" pool_name = vg_name + "-pool" pool_path = "%s/%s" % (vg_name, pool_name) def executor(obj, *cmd, **kwargs): self.assertEqual(pool_path, cmd[-1]) self.vg._executor = executor self.vg.create_thin_pool(pool_name) self.vg.create_volume("test", "1G", lv_type='thin') self.assertEqual(pool_name, self.vg.vg_thin_pool) def test_lv_has_snapshot(self): self.assertTrue(self.vg.lv_has_snapshot('fake-vg')) self.assertFalse(self.vg.lv_has_snapshot('test-volumes')) def test_activate_lv(self): self.vg._supports_lvchange_ignoreskipactivation = True with mock.patch.object(self.vg, '_execute') as mock_exec: self.vg.activate_lv('my-lv') expected = [mock.call('lvchange', '-a', 'y', '--yes', '-K', 'fake-vg/my-lv', root_helper='sudo', run_as_root=True)] self.assertEqual(expected, mock_exec.call_args_list) def test_get_mirrored_available_capacity(self): self.assertEqual(2.0, self.vg.vg_mirror_free_space(1)) def test_lv_extend(self): self.vg.deactivate_lv = mock.MagicMock() # Extend lv with snapshot and make sure deactivate called self.vg.create_volume("test", "1G") self.vg.extend_volume("test", "2G") self.vg.deactivate_lv.assert_called_once_with('test') self.vg.deactivate_lv.reset_mock() # Extend lv without snapshot so deactivate should not be called self.vg.create_volume("test", "1G") self.vg.vg_name = "test-volumes" self.vg.extend_volume("test", "2G") self.assertFalse(self.vg.deactivate_lv.called) def test_lv_deactivate(self): with mock.patch.object(self.vg, '_execute'): is_active_mock = mock.Mock() is_active_mock.return_value = False self.vg._lv_is_active = is_active_mock self.vg.create_volume('test', '1G') self.vg.deactivate_lv('test') @mock.patch('time.sleep') def test_lv_deactivate_timeout(self, mock_sleep): with mock.patch.object(self.vg, '_execute'): is_active_mock = mock.Mock() is_active_mock.return_value = True self.vg._lv_is_active = is_active_mock self.vg.create_volume('test', '1G') self.assertRaises(exception.VolumeNotDeactivated, self.vg.deactivate_lv, 'test') def test_lv_is_active(self): self.vg.create_volume('test', '1G') with mock.patch.object(self.vg, '_execute', return_value=['owi-a---', '']): self.assertTrue(self.vg._lv_is_active('test')) with mock.patch.object(self.vg, '_execute', return_value=['owi-----', '']): self.assertFalse(self.vg._lv_is_active('test')) os-brick-2.3.0/os_brick/tests/local_dev/fake_lvm.py0000666000175100017510000000334513230233223022274 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. class FakeBrickLVM(object): """Logs and records calls, for unit tests.""" def __init__(self, vg_name, create, pv_list, vtype, execute=None): super(FakeBrickLVM, self).__init__() self.vg_size = '5.00' self.vg_free_space = '5.00' self.vg_name = vg_name def supports_thin_provisioning(): return False def get_volumes(self): return ['fake-volume'] def get_volume(self, name): return ['name'] def get_all_physical_volumes(vg_name=None): return [] def get_physical_volumes(self): return [] def update_volume_group_info(self): pass def create_thin_pool(self, name=None, size_str=0): pass def create_volume(self, name, size_str, lv_type='default', mirror_count=0): pass def create_lv_snapshot(self, name, source_lv_name, lv_type='default'): pass def delete(self, name): pass def revert(self, snapshot_name): pass def lv_has_snapshot(self, name): return False def activate_lv(self, lv, is_snapshot=False, permanent=False): pass def rename_volume(self, lv_name, new_name): pass os-brick-2.3.0/os_brick/tests/encryptors/0000775000175100017510000000000013230233405020411 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/encryptors/__init__.py0000666000175100017510000000000013230233223022510 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/encryptors/test_luks.py0000666000175100017510000002676713230233223023022 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import binascii import mock from castellan.common.objects import symmetric_key as key from os_brick.encryptors import luks from os_brick.tests.encryptors import test_cryptsetup from oslo_concurrency import processutils as putils class LuksEncryptorTestCase(test_cryptsetup.CryptsetupEncryptorTestCase): def _create(self): return luks.LuksEncryptor(root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) @mock.patch('os_brick.executor.Executor._execute') def test_is_luks(self, mock_execute): luks.is_luks(self.root_helper, self.dev_path, execute=mock_execute) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os_brick.encryptors.luks.LOG') def test_is_luks_with_error(self, mock_log, mock_execute): error_msg = "Device %s is not a valid LUKS device." % self.dev_path mock_execute.side_effect = putils.ProcessExecutionError( exit_code=1, stderr=error_msg) luks.is_luks(self.root_helper, self.dev_path, execute=mock_execute) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ]) self.assertEqual(1, mock_log.warning.call_count) # warning logged @mock.patch('os_brick.executor.Executor._execute') def test__format_volume(self, mock_execute): self.encryptor._format_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--key-file=-', self.dev_path, process_input='passphrase', root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), ]) @mock.patch('os_brick.executor.Executor._execute') def test__open_volume(self, mock_execute): self.encryptor._open_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input='passphrase', root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume(self, mock_execute): fake_key = '0c84146034e747639b698368807286df' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_not_formatted(self, mock_execute): fake_key = 'bc37c5eccebe403f9cc2d0dd20dac2bc' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=1), # luksOpen putils.ProcessExecutionError(exit_code=1), # isLuks mock.DEFAULT, # luksFormat mock.DEFAULT, # luksOpen mock.DEFAULT, # ln ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--key-file=-', self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_fail(self, mock_execute): fake_key = 'ea6c2e1b8f7f4f84ae3560116d659ba2' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=1), # luksOpen mock.DEFAULT, # isLuks ] self.assertRaises(putils.ProcessExecutionError, self.encryptor.attach_volume, None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') def test__close_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, attempts=3, run_as_root=True, check_exit_code=[0, 4]), ]) @mock.patch('os_brick.executor.Executor._execute') def test_detach_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, attempts=3, run_as_root=True, check_exit_code=[0, 4]), ]) def test_get_mangled_passphrase(self): # Confirm that a mangled passphrase is provided as per bug#1633518 unmangled_raw_key = bytes(binascii.unhexlify('0725230b')) symmetric_key = key.SymmetricKey('AES', len(unmangled_raw_key) * 8, unmangled_raw_key) unmangled_encoded_key = symmetric_key.get_encoded() self.assertEqual(self.encryptor._get_mangled_passphrase( unmangled_encoded_key), '72523b') @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_unmangle_passphrase(self, mock_execute): fake_key = '0725230b' fake_key_mangled = '72523b' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = \ test_cryptsetup.fake__get_key(None, fake_key) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=2), # luksOpen mock.DEFAULT, # luksOpen mock.DEFAULT, # luksClose mock.DEFAULT, # luksAddKey mock.DEFAULT, # luksOpen mock.DEFAULT, # luksClose mock.DEFAULT, # luksRemoveKey mock.DEFAULT, # luksOpen mock.DEFAULT, # ln ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4], attempts=3), mock.call('cryptsetup', 'luksAddKey', self.dev_path, process_input=''.join([fake_key_mangled, '\n', fake_key, '\n', fake_key]), root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4], attempts=3), mock.call('cryptsetup', 'luksRemoveKey', self.dev_path, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) self.assertEqual(9, mock_execute.call_count) os-brick-2.3.0/os_brick/tests/encryptors/test_base.py0000666000175100017510000001754713230233223022752 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from castellan.tests.unit.key_manager import fake import mock from os_brick import encryptors from os_brick.tests import base class VolumeEncryptorTestCase(base.TestCase): def _create(self): pass def setUp(self): super(VolumeEncryptorTestCase, self).setUp() self.connection_info = { "data": { "device_path": "/dev/disk/by-path/" "ip-192.0.2.0:3260-iscsi-iqn.2010-10.org.openstack" ":volume-fake_uuid-lun-1", }, } self.root_helper = None self.keymgr = fake.fake_api() self.encryptor = self._create() class BaseEncryptorTestCase(VolumeEncryptorTestCase): def _test_get_encryptor(self, provider, expected_provider_class): encryption = {'control_location': 'front-end', 'provider': provider} encryptor = encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) self.assertIsInstance(encryptor, expected_provider_class) def test_get_encryptors(self): self._test_get_encryptor('luks', encryptors.luks.LuksEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('os_brick.encryptors.luks.LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('nova.volume.encryptors.luks.LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('plain', encryptors.cryptsetup.CryptsetupEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor( 'os_brick.encryptors.cryptsetup.CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor( 'nova.volume.encryptors.cryptsetup.CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor(None, encryptors.nop.NoOpEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('NoOpEncryptor', encryptors.nop.NoOpEncryptor) self._test_get_encryptor('os_brick.encryptors.nop.NoOpEncryptor', encryptors.nop.NoOpEncryptor) self._test_get_encryptor('nova.volume.encryptors.nop.NoopEncryptor', encryptors.nop.NoOpEncryptor) def test_get_error_encryptors(self): encryption = {'control_location': 'front-end', 'provider': 'ErrorEncryptor'} self.assertRaises(ValueError, encryptors.get_volume_encryptor, root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) @mock.patch('os_brick.encryptors.LOG') def test_error_log(self, log): encryption = {'control_location': 'front-end', 'provider': 'TestEncryptor'} provider = 'TestEncryptor' try: encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) except Exception as e: log.error.assert_called_once_with("Error instantiating " "%(provider)s: " "%(exception)s", {'provider': provider, 'exception': e}) @mock.patch('os_brick.encryptors.LOG') def test_get_missing_out_of_tree_encryptor_log(self, log): provider = 'TestEncryptor' encryption = {'control_location': 'front-end', 'provider': provider} try: encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) except Exception as e: log.error.assert_called_once_with("Error instantiating " "%(provider)s: " "%(exception)s", {'provider': provider, 'exception': e}) log.warning.assert_called_once_with("Use of the out of tree " "encryptor class %(provider)s " "will be blocked with the " "Queens release of os-brick.", {'provider': provider}) @mock.patch('os_brick.encryptors.LOG') def test_get_direct_encryptor_log(self, log): encryption = {'control_location': 'front-end', 'provider': 'LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) encryption = {'control_location': 'front-end', 'provider': 'os_brick.encryptors.luks.LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) encryption = {'control_location': 'front-end', 'provider': 'nova.volume.encryptors.luks.LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) log.warning.assert_has_calls([ mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'LuksEncryptor'}), mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'os_brick.encryptors.luks.LuksEncryptor'}), mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'nova.volume.encryptors.luks.LuksEncryptor'})]) os-brick-2.3.0/os_brick/tests/encryptors/test_nop.py0000666000175100017510000000224313230233223022617 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import nop from os_brick.tests.encryptors import test_base class NoOpEncryptorTestCase(test_base.VolumeEncryptorTestCase): def _create(self): return nop.NoOpEncryptor(root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) def test_attach_volume(self): self.encryptor.attach_volume(None) def test_detach_volume(self): self.encryptor.detach_volume() os-brick-2.3.0/os_brick/tests/encryptors/test_cryptsetup.py0000666000175100017510000001770613230233223024257 0ustar zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import binascii import copy import mock import six from castellan.common.objects import symmetric_key as key from castellan.tests.unit.key_manager import fake from os_brick.encryptors import cryptsetup from os_brick import exception from os_brick.tests.encryptors import test_base from oslo_concurrency import processutils as putils def fake__get_key(context, passphrase): raw = bytes(binascii.unhexlify(passphrase)) symmetric_key = key.SymmetricKey('AES', len(raw) * 8, raw) return symmetric_key class CryptsetupEncryptorTestCase(test_base.VolumeEncryptorTestCase): @mock.patch('os.path.exists', return_value=False) def _create(self, mock_exists): return cryptsetup.CryptsetupEncryptor( connection_info=self.connection_info, root_helper=self.root_helper, keymgr=self.keymgr) def setUp(self): super(CryptsetupEncryptorTestCase, self).setUp() self.dev_path = self.connection_info['data']['device_path'] self.dev_name = 'crypt-%s' % self.dev_path.split('/')[-1] self.symlink_path = self.dev_path @mock.patch('os_brick.executor.Executor._execute') def test__open_volume(self, mock_execute): self.encryptor._open_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input='passphrase', run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume(self, mock_execute): fake_key = 'e8b76872e3b04c18b3b6656bbf6f5089' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = fake__get_key(None, fake_key) self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test__close_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'remove', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4]), ]) @mock.patch('os_brick.executor.Executor._execute') def test_detach_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'remove', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4]), ]) def test_init_volume_encryption_not_supported(self): # Tests that creating a CryptsetupEncryptor fails if there is no # device_path key. type = 'unencryptable' data = dict(volume_id='a194699b-aa07-4433-a945-a5d23802043e') connection_info = dict(driver_volume_type=type, data=data) exc = self.assertRaises(exception.VolumeEncryptionNotSupported, cryptsetup.CryptsetupEncryptor, root_helper=self.root_helper, connection_info=connection_info, keymgr=fake.fake_api()) self.assertIn(type, six.text_type(exc)) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os.path.exists', return_value=True) def test_init_volume_encryption_with_old_name(self, mock_exists, mock_execute): # If an old name crypt device exists, dev_path should be the old name. old_dev_name = self.dev_path.split('/')[-1] encryptor = cryptsetup.CryptsetupEncryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) self.assertFalse(encryptor.dev_name.startswith('crypt-')) self.assertEqual(old_dev_name, encryptor.dev_name) self.assertEqual(self.dev_path, encryptor.dev_path) self.assertEqual(self.symlink_path, encryptor.symlink_path) mock_exists.assert_called_once_with('/dev/mapper/%s' % old_dev_name) mock_execute.assert_called_once_with( 'cryptsetup', 'status', old_dev_name, run_as_root=True) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os.path.exists', side_effect=[False, True]) def test_init_volume_encryption_with_wwn(self, mock_exists, mock_execute): # If an wwn name crypt device exists, dev_path should be based on wwn. old_dev_name = self.dev_path.split('/')[-1] wwn = 'fake_wwn' connection_info = copy.deepcopy(self.connection_info) connection_info['data']['multipath_id'] = wwn encryptor = cryptsetup.CryptsetupEncryptor( root_helper=self.root_helper, connection_info=connection_info, keymgr=fake.fake_api(), execute=mock_execute) self.assertFalse(encryptor.dev_name.startswith('crypt-')) self.assertEqual(wwn, encryptor.dev_name) self.assertEqual(self.dev_path, encryptor.dev_path) self.assertEqual(self.symlink_path, encryptor.symlink_path) mock_exists.assert_has_calls([ mock.call('/dev/mapper/%s' % old_dev_name), mock.call('/dev/mapper/%s' % wwn)]) mock_execute.assert_called_once_with( 'cryptsetup', 'status', wwn, run_as_root=True) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_unmangle_passphrase(self, mock_execute): fake_key = '0725230b' fake_key_mangled = '72523b' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = fake__get_key(None, fake_key) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=2), # luksOpen mock.DEFAULT, mock.DEFAULT, ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) self.assertEqual(3, mock_execute.call_count) os-brick-2.3.0/os_brick/tests/initiator/0000775000175100017510000000000013230233405020203 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/initiator/test_linuxfc.py0000666000175100017510000004043313230233223023270 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os.path import mock from os_brick.initiator import linuxfc from os_brick.tests import base class LinuxFCTestCase(base.TestCase): def setUp(self): super(LinuxFCTestCase, self).setUp() self.cmds = [] self.mock_object(os.path, 'exists', return_value=True) self.mock_object(os.path, 'isdir', return_value=True) self.lfc = linuxfc.LinuxFibreChannel(None, execute=self.fake_execute) def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def test_has_fc_support(self): self.mock_object(os.path, 'isdir', return_value=False) has_fc = self.lfc.has_fc_support() self.assertFalse(has_fc) self.mock_object(os.path, 'isdir', return_value=True) has_fc = self.lfc.has_fc_support() self.assertTrue(has_fc) @staticmethod def __get_rescan_info(zone_manager=False): connection_properties = { 'initiator_target_map': {'50014380186af83c': ['514f0c50023f6c00'], '50014380186af83e': ['514f0c50023f6c01']}, 'target_discovered': False, 'target_lun': 1, 'target_wwn': ['514f0c50023f6c00', '514f0c50023f6c01'] } hbas = [ {'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.0/host6/fc_host/host6'), 'host_device': 'host6', 'node_name': '50014380186af83d', 'port_name': '50014380186af83c'}, {'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.1/host7/fc_host/host7'), 'host_device': 'host7', 'node_name': '50014380186af83f', 'port_name': '50014380186af83e'}, ] if not zone_manager: del connection_properties['initiator_target_map'] return hbas, connection_properties def test__get_hba_channel_scsi_target_single_wwpn(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info() con_props['target_wwn'] = con_props['target_wwn'][0] with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = [['0', '1']] self.assertListEqual(expected, res) def test__get_hba_channel_scsi_target_multiple_wwpn(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n' '/sys/class/fc_transport/target6:0:2/port_name\n', '') hbas, con_props = self.__get_rescan_info() with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00\|514f0c50023f6c01" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = [['0', '1'], ['0', '2']] self.assertListEqual(expected, res) def test__get_hba_channel_scsi_target_zone_manager(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = [['0', '1']] self.assertListEqual(expected, res) def test__get_hba_channel_scsi_target_not_found(self): hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=('', '')) as execute_mock: res = self.lfc._get_hba_channel_scsi_target(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) self.assertListEqual([], res) def test__get_hba_channel_scsi_target_exception(self): hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', side_effect=Exception) as execute_mock: res = self.lfc._get_hba_channel_scsi_target(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) self.assertIsNone(res) def test_rescan_hosts(self): get_chan_results = [[['2', '3'], ['4', '5']], [['6', '7']]] hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=None) as execute_mock, \ mock.patch.object(self.lfc, '_get_hba_channel_scsi_target', side_effect=get_chan_results) as mock_get_chan: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='2 3 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='4 5 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host7/scan', process_input='6 7 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) expected_calls = [mock.call(hbas[0], con_props), mock.call(hbas[1], con_props)] mock_get_chan.assert_has_calls(expected_calls) def test_rescan_hosts_wildcard(self): hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_get_hba_channel_scsi_target', side_effect=(None, [])), \ mock.patch.object(self.lfc, '_execute', side_effect=None) as execute_mock: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='- - 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host7/scan', process_input='- - 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) def test_get_fc_hbas_fail(self): def fake_exec1(a, b, c, d, run_as_root=True, root_helper='sudo'): raise OSError def fake_exec2(a, b, c, d, run_as_root=True, root_helper='sudo'): return None, 'None found' self.lfc._execute = fake_exec1 hbas = self.lfc.get_fc_hbas() self.assertEqual(0, len(hbas)) self.lfc._execute = fake_exec2 hbas = self.lfc.get_fc_hbas() self.assertEqual(0, len(hbas)) def test_get_fc_hbas(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec hbas = self.lfc.get_fc_hbas() self.assertEqual(2, len(hbas)) hba1 = hbas[0] self.assertEqual("host0", hba1["ClassDevice"]) hba2 = hbas[1] self.assertEqual("host2", hba2["ClassDevice"]) def test_get_fc_hbas_info(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec hbas_info = self.lfc.get_fc_hbas_info() expected_info = [{'device_path': '/sys/devices/pci0000:20/' '0000:20:03.0/0000:21:00.0/' 'host0/fc_host/host0', 'host_device': 'host0', 'node_name': '50014380242b9751', 'port_name': '50014380242b9750'}, {'device_path': '/sys/devices/pci0000:20/' '0000:20:03.0/0000:21:00.1/' 'host2/fc_host/host2', 'host_device': 'host2', 'node_name': '50014380242b9753', 'port_name': '50014380242b9752'}, ] self.assertEqual(expected_info, hbas_info) def test_get_fc_wwpns(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec wwpns = self.lfc.get_fc_wwpns() expected_wwpns = ['50014380242b9750', '50014380242b9752'] self.assertEqual(expected_wwpns, wwpns) def test_get_fc_wwnns(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec wwnns = self.lfc.get_fc_wwpns() expected_wwnns = ['50014380242b9750', '50014380242b9752'] self.assertEqual(expected_wwnns, wwnns) SYSTOOL_FC = """ Class = "fc_host" Class Device = "host0" Class Device path = "/sys/devices/pci0000:20/0000:20:03.0/\ 0000:21:00.0/host0/fc_host/host0" dev_loss_tmo = "16" fabric_name = "0x100000051ea338b9" issue_lip = max_npiv_vports = "0" node_name = "0x50014380242b9751" npiv_vports_inuse = "0" port_id = "0x960d0d" port_name = "0x50014380242b9750" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QMH2572 FW:v4.04.04 DVR:v8.03.07.12-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = vport_create = vport_delete = Device = "host0" Device path = "/sys/devices/pci0000:20/0000:20:03.0/0000:21:00.0/host0" edc = optrom_ctl = reset = uevent = "DEVTYPE=scsi_host" Class Device = "host2" Class Device path = "/sys/devices/pci0000:20/0000:20:03.0/\ 0000:21:00.1/host2/fc_host/host2" dev_loss_tmo = "16" fabric_name = "0x100000051ea33b79" issue_lip = max_npiv_vports = "0" node_name = "0x50014380242b9753" npiv_vports_inuse = "0" port_id = "0x970e09" port_name = "0x50014380242b9752" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QMH2572 FW:v4.04.04 DVR:v8.03.07.12-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = vport_create = vport_delete = Device = "host2" Device path = "/sys/devices/pci0000:20/0000:20:03.0/0000:21:00.1/host2" edc = optrom_ctl = reset = uevent = "DEVTYPE=scsi_host" """ class LinuxFCS390XTestCase(LinuxFCTestCase): def setUp(self): super(LinuxFCS390XTestCase, self).setUp() self.cmds = [] self.lfc = linuxfc.LinuxFibreChannelS390X(None, execute=self.fake_execute) def test_get_fc_hbas_info(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC_S390X, None self.lfc._execute = fake_exec hbas_info = self.lfc.get_fc_hbas_info() expected = [{'device_path': '/sys/devices/css0/0.0.02ea/' '0.0.3080/host0/fc_host/host0', 'host_device': 'host0', 'node_name': '1234567898765432', 'port_name': 'c05076ffe680a960'}] self.assertEqual(expected, hbas_info) @mock.patch.object(os.path, 'exists', return_value=False) def test_configure_scsi_device(self, mock_execute): device_number = "0.0.2319" target_wwn = "0x50014380242b9751" lun = 1 self.lfc.configure_scsi_device(device_number, target_wwn, lun) expected_commands = [('tee -a /sys/bus/ccw/drivers/zfcp/0.0.2319/' 'port_rescan'), ('tee -a /sys/bus/ccw/drivers/zfcp/0.0.2319/' '0x50014380242b9751/unit_add')] self.assertEqual(expected_commands, self.cmds) def test_deconfigure_scsi_device(self): device_number = "0.0.2319" target_wwn = "0x50014380242b9751" lun = 1 self.lfc.deconfigure_scsi_device(device_number, target_wwn, lun) expected_commands = [('tee -a /sys/bus/ccw/drivers/zfcp/' '0.0.2319/0x50014380242b9751/unit_remove')] self.assertEqual(expected_commands, self.cmds) SYSTOOL_FC_S390X = """ Class = "fc_host" Class Device = "host0" Class Device path = "/sys/devices/css0/0.0.02ea/0.0.3080/host0/fc_host/host0" active_fc4s = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 " dev_loss_tmo = "60" maxframe_size = "2112 bytes" node_name = "0x1234567898765432" permanent_port_name = "0xc05076ffe6803081" port_id = "0x010014" port_name = "0xc05076ffe680a960" port_state = "Online" port_type = "NPIV VPORT" serial_number = "IBM00000000000P30" speed = "8 Gbit" supported_classes = "Class 2, Class 3" supported_fc4s = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 " supported_speeds = "2 Gbit, 4 Gbit" symbolic_name = "IBM 2827 00000000000P30 \ PCHID: 0308 NPIV UlpId: 01EA0A00 DEVNO: 0.0.1234 NAME: dummy" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = Device = "host0" Device path = "/sys/devices/css0/0.0.02ea/0.0.3080/host0" uevent = "DEVTYPE=scsi_host" """ os-brick-2.3.0/os_brick/tests/initiator/test_linuxrbd.py0000666000175100017510000001720213230233223023445 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may not # use this file except in compliance with the License. You may obtain a copy of # the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations under # the License. import mock from os_brick import exception from os_brick.initiator import linuxrbd from os_brick.tests import base from os_brick import utils class MockRados(object): class Error(Exception): pass class ioctx(object): def __init__(self, *args, **kwargs): pass def __enter__(self, *args, **kwargs): return self def __exit__(self, *args, **kwargs): return False def close(self, *args, **kwargs): pass class Rados(object): def __init__(self, *args, **kwargs): pass def __enter__(self, *args, **kwargs): return self def __exit__(self, *args, **kwargs): return False def connect(self, *args, **kwargs): pass def open_ioctx(self, *args, **kwargs): return MockRados.ioctx() def shutdown(self, *args, **kwargs): pass class RBDClientTestCase(base.TestCase): def setUp(self): super(RBDClientTestCase, self).setUp() @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') def test_with_client(self, mock_rados, mock_rbd): with linuxrbd.RBDClient('test_user', 'test_pool') as client: # Verify object attributes are assigned as expected self.assertEqual('/etc/ceph/ceph.conf', client.rbd_conf) self.assertEqual(utils.convert_str('test_user'), client.rbd_user) self.assertEqual(utils.convert_str('test_pool'), client.rbd_pool) # Assert connect is called with correct paramaters mock_rados.Rados.assert_called_once_with( clustername='ceph', rados_id=utils.convert_str('test_user'), conffile='/etc/ceph/ceph.conf') # Ensure correct calls to connect to cluster self.assertEqual( 1, mock_rados.Rados.return_value.connect.call_count) mock_rados.Rados.return_value.open_ioctx.assert_called_once_with( utils.convert_str('test_pool')) self.assertEqual(1, mock_rados.Rados.return_value.shutdown.call_count) @mock.patch.object(MockRados.Rados, 'connect', side_effect=MockRados.Error) def test_with_client_error(self, _): linuxrbd.rados = MockRados linuxrbd.rados.Error = MockRados.Error def test(): with linuxrbd.RBDClient('test_user', 'test_pool'): pass self.assertRaises(exception.BrickException, test) class RBDVolumeIOWrapperTestCase(base.TestCase): def setUp(self): super(RBDVolumeIOWrapperTestCase, self).setUp() self.mock_volume = mock.Mock() self.mock_volume_wrapper = \ linuxrbd.RBDVolumeIOWrapper(self.mock_volume) self.data_length = 1024 self.full_data = 'abcd' * 256 def test_init(self): self.assertEqual(self.mock_volume, self.mock_volume_wrapper._rbd_volume) self.assertEqual(0, self.mock_volume_wrapper._offset) def test_inc_offset(self): self.mock_volume_wrapper._inc_offset(10) self.mock_volume_wrapper._inc_offset(10) self.assertEqual(20, self.mock_volume_wrapper._offset) def test_read(self): def mock_read(offset, length): return self.full_data[offset:length] self.mock_volume.image.read.side_effect = mock_read self.mock_volume.image.size.return_value = self.data_length data = self.mock_volume_wrapper.read() self.assertEqual(self.full_data, data) data = self.mock_volume_wrapper.read() self.assertEqual(b'', data) self.mock_volume_wrapper.seek(0) data = self.mock_volume_wrapper.read() self.assertEqual(self.full_data, data) self.mock_volume_wrapper.seek(0) data = self.mock_volume_wrapper.read(10) self.assertEqual(self.full_data[:10], data) def test_write(self): self.mock_volume_wrapper.write(self.full_data) self.assertEqual(1024, self.mock_volume_wrapper._offset) def test_seekable(self): self.assertTrue(self.mock_volume_wrapper.seekable) def test_seek(self): self.assertEqual(0, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10) self.assertEqual(10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10) self.assertEqual(10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10, 1) self.assertEqual(20, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(0) self.mock_volume_wrapper.write(self.full_data) self.mock_volume.image.size.return_value = self.data_length self.mock_volume_wrapper.seek(0) self.assertEqual(0, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10, 2) self.assertEqual(self.data_length + 10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(-10, 2) self.assertEqual(self.data_length - 10, self.mock_volume_wrapper._offset) # test exceptions. self.assertRaises(IOError, self.mock_volume_wrapper.seek, 0, 3) self.assertRaises(IOError, self.mock_volume_wrapper.seek, -1) # offset should not have been changed by any of the previous # operations. self.assertEqual(self.data_length - 10, self.mock_volume_wrapper._offset) def test_tell(self): self.assertEqual(0, self.mock_volume_wrapper.tell()) self.mock_volume_wrapper._inc_offset(10) self.assertEqual(10, self.mock_volume_wrapper.tell()) def test_flush(self): with mock.patch.object(linuxrbd, 'LOG') as mock_logger: self.mock_volume.image.flush = mock.Mock() self.mock_volume_wrapper.flush() self.assertEqual(1, self.mock_volume.image.flush.call_count) self.mock_volume.image.flush.reset_mock() # this should be caught and logged silently. self.mock_volume.image.flush.side_effect = AttributeError self.mock_volume_wrapper.flush() self.assertEqual(1, self.mock_volume.image.flush.call_count) self.assertEqual(1, mock_logger.warning.call_count) def test_fileno(self): self.assertRaises(IOError, self.mock_volume_wrapper.fileno) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(linuxrbd.RBDClient, 'disconnect') def test_close(self, rbd_disconnect, mock_rados, mock_rbd): rbd_client = linuxrbd.RBDClient('user', 'pool') rbd_volume = linuxrbd.RBDVolume(rbd_client, 'volume') rbd_handle = linuxrbd.RBDVolumeIOWrapper( linuxrbd.RBDImageMetadata(rbd_volume, 'pool', 'user', None)) rbd_handle.close() self.assertEqual(1, rbd_disconnect.call_count) class RBDVolumeTestCase(base.TestCase): def test_name_attribute(self): mock_client = mock.Mock() rbd_volume = linuxrbd.RBDVolume(mock_client, 'volume') self.assertEqual('volume', rbd_volume.name) os-brick-2.3.0/os_brick/tests/initiator/__init__.py0000666000175100017510000000000013230233223022302 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/initiator/test_host_driver.py0000666000175100017510000000325013230233223024144 0ustar zuulzuul00000000000000# Copyright (c) 2015 Scality # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import errno import mock from os_brick.initiator import host_driver from os_brick.tests import base class HostDriverTestCase(base.TestCase): def test_get_all_block_devices(self): fake_dev = ['device1', 'device2'] expected = ['/dev/disk/by-path/' + dev for dev in fake_dev] driver = host_driver.HostDriver() with mock.patch('os.listdir', return_value=fake_dev): actual = driver.get_all_block_devices() self.assertEqual(expected, actual) def test_get_all_block_devices_when_oserror_is_enoent(self): driver = host_driver.HostDriver() oserror = OSError(errno.ENOENT, "") with mock.patch('os.listdir', side_effect=oserror): block_devices = driver.get_all_block_devices() self.assertEqual([], block_devices) def test_get_all_block_devices_when_oserror_is_not_enoent(self): driver = host_driver.HostDriver() oserror = OSError(errno.ENOMEM, "") with mock.patch('os.listdir', side_effect=oserror): self.assertRaises(OSError, driver.get_all_block_devices) os-brick-2.3.0/os_brick/tests/initiator/test_linuxscsi.py0000666000175100017510000012733413230233223023647 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import os.path import textwrap import time import ddt import mock from oslo_log import log as logging from os_brick import exception from os_brick.initiator import linuxscsi from os_brick.tests import base LOG = logging.getLogger(__name__) @ddt.ddt class LinuxSCSITestCase(base.TestCase): def setUp(self): super(LinuxSCSITestCase, self).setUp() self.cmds = [] self.mock_object(os.path, 'realpath', return_value='/dev/sdc') self.mock_object(os, 'stat', returns=os.stat(__file__)) self.linuxscsi = linuxscsi.LinuxSCSI(None, execute=self.fake_execute) def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def test_echo_scsi_command(self): self.linuxscsi.echo_scsi_command("/some/path", "1") expected_commands = ['tee -a /some/path'] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(os.path, 'realpath') def test_get_name_from_path(self, realpath_mock): device_name = "/dev/sdc" realpath_mock.return_value = device_name disk_path = ("/dev/disk/by-path/ip-10.10.220.253:3260-" "iscsi-iqn.2000-05.com.3pardata:21810002ac00383d-lun-0") name = self.linuxscsi.get_name_from_path(disk_path) self.assertEqual(device_name, name) disk_path = ("/dev/disk/by-path/pci-0000:00:00.0-ip-10.9.8.7:3260-" "iscsi-iqn.2000-05.com.openstack:2180002ac00383d-lun-0") name = self.linuxscsi.get_name_from_path(disk_path) self.assertEqual(device_name, name) realpath_mock.return_value = "bogus" name = self.linuxscsi.get_name_from_path(disk_path) self.assertIsNone(name) @mock.patch.object(os.path, 'exists', return_value=False) def test_remove_scsi_device(self, exists_mock): self.linuxscsi.remove_scsi_device("/dev/sdc") expected_commands = [] self.assertEqual(expected_commands, self.cmds) exists_mock.return_value = True self.linuxscsi.remove_scsi_device("/dev/sdc") expected_commands = [ ('blockdev --flushbufs /dev/sdc'), ('tee -a /sys/block/sdc/device/delete')] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'echo_scsi_command') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_device_io') @mock.patch.object(os.path, 'exists', return_value=True) def test_remove_scsi_device_force(self, exists_mock, flush_mock, echo_mock): """With force we'll always call delete even if flush fails.""" exc = exception.ExceptionChainer() flush_mock.side_effect = Exception() echo_mock.side_effect = Exception() device = '/dev/sdc' self.linuxscsi.remove_scsi_device(device, force=True, exc=exc) # The context manager has caught the exceptions self.assertTrue(exc) flush_mock.assert_called_once_with(device) echo_mock.assert_called_once_with('/sys/block/sdc/device/delete', '1') @mock.patch('time.sleep') @mock.patch('os.path.exists', return_value=True) def test_wait_for_volumes_removal_failure(self, exists_mock, sleep_mock): retries = 3 names = ('sda', 'sdb') self.assertRaises(exception.VolumePathNotRemoved, self.linuxscsi.wait_for_volumes_removal, names) exists_mock.assert_has_calls([mock.call('/dev/' + name) for name in names] * retries) self.assertEqual(retries - 1, sleep_mock.call_count) @mock.patch('time.sleep') @mock.patch('os.path.exists', side_effect=(True, True, False, False)) def test_wait_for_volumes_removal_retry(self, exists_mock, sleep_mock): names = ('sda', 'sdb') self.linuxscsi.wait_for_volumes_removal(names) exists_mock.assert_has_calls([mock.call('/dev/' + name) for name in names] * 2) self.assertEqual(1, sleep_mock.call_count) def test_flush_multipath_device(self): dm_map_name = '3600d0230000000000e13955cc3757800' with mock.patch.object(self.linuxscsi, '_execute') as exec_mock: self.linuxscsi.flush_multipath_device(dm_map_name) exec_mock.assert_called_once_with( 'multipath', '-f', dm_map_name, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self.linuxscsi._root_helper) def test_get_scsi_wwn(self): fake_path = '/dev/disk/by-id/somepath' fake_wwn = '1234567890' def fake_execute(*cmd, **kwargs): return fake_wwn, None self.linuxscsi._execute = fake_execute wwn = self.linuxscsi.get_scsi_wwn(fake_path) self.assertEqual(fake_wwn, wwn) @mock.patch('six.moves.builtins.open') def test_get_dm_name(self, open_mock): dm_map_name = '3600d0230000000000e13955cc3757800' cm_open = open_mock.return_value.__enter__.return_value cm_open.read.return_value = dm_map_name res = self.linuxscsi.get_dm_name('dm-0') self.assertEqual(dm_map_name, res) open_mock.assert_called_once_with('/sys/block/dm-0/dm/name') @mock.patch('six.moves.builtins.open', side_effect=IOError) def test_get_dm_name_failure(self, open_mock): self.assertEqual('', self.linuxscsi.get_dm_name('dm-0')) @mock.patch('glob.glob', side_effect=[[], ['/sys/block/sda/holders/dm-9']]) def test_find_sysfs_multipath_dm(self, glob_mock): device_names = ('sda', 'sdb') res = self.linuxscsi.find_sysfs_multipath_dm(device_names) self.assertEqual('dm-9', res) glob_mock.assert_has_calls([mock.call('/sys/block/sda/holders/dm-*'), mock.call('/sys/block/sdb/holders/dm-*')]) @mock.patch('glob.glob', return_value=[]) def test_find_sysfs_multipath_dm_not_found(self, glob_mock): device_names = ('sda', 'sdb') res = self.linuxscsi.find_sysfs_multipath_dm(device_names) self.assertIsNone(res) glob_mock.assert_has_calls([mock.call('/sys/block/sda/holders/dm-*'), mock.call('/sys/block/sdb/holders/dm-*')]) @mock.patch.object(linuxscsi.LinuxSCSI, '_execute') @mock.patch('os.path.exists', return_value=True) def test_flush_device_io(self, exists_mock, exec_mock): device = '/dev/sda' self.linuxscsi.flush_device_io(device) exists_mock.assert_called_once_with(device) exec_mock.assert_called_once_with( 'blockdev', '--flushbufs', device, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self.linuxscsi._root_helper) @mock.patch('os.path.exists', return_value=False) def test_flush_device_io_non_existent(self, exists_mock): device = '/dev/sda' self.linuxscsi.flush_device_io(device) exists_mock.assert_called_once_with(device) @mock.patch.object(os.path, 'exists', return_value=True) def test_find_multipath_device_path(self, exists_mock): fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) expected_path = '/dev/disk/by-id/dm-uuid-mpath-%s' % fake_wwn self.assertEqual(expected_path, found_path) @mock.patch('time.sleep') @mock.patch.object(os.path, 'exists') def test_find_multipath_device_path_mapper(self, exists_mock, sleep_mock): # the wait loop tries 3 times before it gives up # we want to test failing to find the # /dev/disk/by-id/dm-uuid-mpath- path # but finding the # /dev/mapper/ path exists_mock.side_effect = [False, False, False, True] fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) expected_path = '/dev/mapper/%s' % fake_wwn self.assertEqual(expected_path, found_path) self.assertTrue(sleep_mock.called) @mock.patch.object(os.path, 'exists', return_value=False) @mock.patch.object(time, 'sleep') def test_find_multipath_device_path_fail(self, exists_mock, sleep_mock): fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) self.assertIsNone(found_path) @mock.patch.object(os.path, 'exists', return_value=False) @mock.patch.object(time, 'sleep') def test_wait_for_path_not_found(self, exists_mock, sleep_mock): path = "/dev/disk/by-id/dm-uuid-mpath-%s" % '1234567890' self.assertRaisesRegexp(exception.VolumeDeviceNotFound, r'Volume device not found at %s' % path, self.linuxscsi.wait_for_path, path) @ddt.data({'do_raise': False, 'force': False}, {'do_raise': True, 'force': True}) @ddt.unpack @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dm_name') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch('os_brick.initiator.linuxscsi.LinuxSCSI.remove_scsi_device') def test_remove_connection_multipath_complete(self, remove_mock, wait_mock, find_dm_mock, get_dm_name_mock, flush_mp_mock, remove_link_mock, do_raise, force): if do_raise: flush_mp_mock.side_effect = Exception devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() mp_name = self.linuxscsi.remove_connection(devices_names, is_multipath=True, force=mock.sentinel.Force, exc=exc) find_dm_mock.assert_called_once_with(devices_names) get_dm_name_mock.assert_called_once_with(find_dm_mock.return_value) flush_mp_mock.assert_called_once_with(get_dm_name_mock.return_value) self.assertEqual(get_dm_name_mock.return_value if do_raise else None, mp_name) remove_mock.assert_has_calls([ mock.call('/dev/sda', mock.sentinel.Force, exc), mock.call('/dev/sdb', mock.sentinel.Force, exc)]) wait_mock.assert_called_once_with(devices_names) self.assertEqual(do_raise, bool(exc)) remove_link_mock.assert_called_once_with(devices_names) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device', side_effect=Exception) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dm_name') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_multipath_fail(self, remove_mock, wait_mock, find_dm_mock, get_dm_name_mock, flush_mp_mock, remove_link_mock): flush_mp_mock.side_effect = exception.ExceptionChainer devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() self.assertRaises(exception.ExceptionChainer, self.linuxscsi.remove_connection, devices_names, is_multipath=True, force=False, exc=exc) find_dm_mock.assert_called_once_with(devices_names) get_dm_name_mock.assert_called_once_with(find_dm_mock.return_value) flush_mp_mock.assert_called_once_with(get_dm_name_mock.return_value) remove_mock.assert_not_called() wait_mock.assert_not_called() remove_link_mock.assert_not_called() self.assertTrue(bool(exc)) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_singlepath(self, remove_mock, wait_mock, remove_link_mock): devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() self.linuxscsi.remove_connection(devices_names, is_multipath=False, force=mock.sentinel.Force, exc=exc) remove_mock.assert_has_calls( [mock.call('/dev/sda', mock.sentinel.Force, exc), mock.call('/dev/sdb', mock.sentinel.Force, exc)]) wait_mock.assert_called_once_with(devices_names) remove_link_mock.assert_called_once_with(devices_names) def test_find_multipath_device_3par_ufn(self): def fake_execute(*cmd, **kwargs): out = ("mpath6 (350002ac20398383d) dm-3 3PARdata,VV\n" "size=2.0G features='0' hwhandler='0' wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 0:0:0:1 sde 8:64 active undef running\n" " `- 2:0:0:1 sdf 8:80 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sde') self.assertEqual("350002ac20398383d", info["id"]) self.assertEqual("mpath6", info["name"]) self.assertEqual("/dev/mapper/mpath6", info["device"]) self.assertEqual("/dev/sde", info['devices'][0]['device']) self.assertEqual("0", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("1", info['devices'][0]['lun']) self.assertEqual("/dev/sdf", info['devices'][1]['device']) self.assertEqual("2", info['devices'][1]['host']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("0", info['devices'][1]['channel']) self.assertEqual("1", info['devices'][1]['lun']) def test_find_multipath_device_svc(self): def fake_execute(*cmd, **kwargs): out = ("36005076da00638089c000000000004d5 dm-2 IBM,2145\n" "size=954M features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "|-+- policy='round-robin 0' prio=-1 status=active\n" "| |- 6:0:2:0 sde 8:64 active undef running\n" "| `- 6:0:4:0 sdg 8:96 active undef running\n" "`-+- policy='round-robin 0' prio=-1 status=enabled\n" " |- 6:0:3:0 sdf 8:80 active undef running\n" " `- 6:0:5:0 sdh 8:112 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sde') self.assertEqual("36005076da00638089c000000000004d5", info["id"]) self.assertEqual("36005076da00638089c000000000004d5", info["name"]) self.assertEqual("/dev/mapper/36005076da00638089c000000000004d5", info["device"]) self.assertEqual("/dev/sde", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdf", info['devices'][2]['device']) self.assertEqual("6", info['devices'][2]['host']) self.assertEqual("0", info['devices'][2]['channel']) self.assertEqual("3", info['devices'][2]['id']) self.assertEqual("0", info['devices'][2]['lun']) def test_find_multipath_device_ds8000(self): def fake_execute(*cmd, **kwargs): out = ("36005076303ffc48e0000000000000101 dm-2 IBM,2107900\n" "size=1.0G features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 6:0:2:0 sdd 8:64 active undef running\n" " `- 6:1:0:3 sdc 8:32 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') self.assertEqual("36005076303ffc48e0000000000000101", info["id"]) self.assertEqual("36005076303ffc48e0000000000000101", info["name"]) self.assertEqual("/dev/mapper/36005076303ffc48e0000000000000101", info["device"]) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) def test_find_multipath_device_with_error(self): def fake_execute(*cmd, **kwargs): out = ("Oct 13 10:24:01 | /lib/udev/scsi_id exited with 1\n" "36005076303ffc48e0000000000000101 dm-2 IBM,2107900\n" "size=1.0G features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 6:0:2:0 sdd 8:64 active undef running\n" " `- 6:1:0:3 sdc 8:32 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') self.assertEqual("36005076303ffc48e0000000000000101", info["id"]) self.assertEqual("36005076303ffc48e0000000000000101", info["name"]) self.assertEqual("/dev/mapper/36005076303ffc48e0000000000000101", info["device"]) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) @mock.patch.object(time, 'sleep') def test_wait_for_rw(self, mock_sleep): lsblk_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.return_value = (lsblk_output, None) self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd120014a2bb' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn # Ensure no exception is raised and no sleep is called self.linuxscsi.wait_for_rw(wwn, path) self.assertFalse(mock_sleep.called) @mock.patch.object(time, 'sleep') def test_wait_for_rw_needs_retry(self, mock_sleep): lsblk_ro_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" lsblk_rw_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.side_effect = [(lsblk_ro_output, None), ('', None), # multipath -r output (lsblk_rw_output, None)] self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd1200143029' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn self.linuxscsi.wait_for_rw(wwn, path) self.assertEqual(1, mock_sleep.call_count) @mock.patch.object(time, 'sleep') def test_wait_for_rw_always_readonly(self, mock_sleep): lsblk_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.return_value = (lsblk_output, None) self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd120014a2bb' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn self.assertRaises(exception.BlockDeviceReadOnly, self.linuxscsi.wait_for_rw, wwn, path) self.assertEqual(4, mock_sleep.call_count) def test_find_multipath_device_with_action(self): def fake_execute(*cmd, **kwargs): out = textwrap.dedent(""" create: 36005076303ffc48e0000000000000101 dm-2 IBM,2107900 size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=-1 status=active |- 6:0:2:0 sdd 8:64 active undef running `- 6:1:0:3 sdc 8:32 active undef running """) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') LOG.error("Device info: %s", info) self.assertEqual('36005076303ffc48e0000000000000101', info['id']) self.assertEqual('36005076303ffc48e0000000000000101', info['name']) self.assertEqual('/dev/mapper/36005076303ffc48e0000000000000101', info['device']) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) def test_get_device_size(self): mock_execute = mock.Mock() self.linuxscsi._execute = mock_execute size = '1024' mock_execute.return_value = (size, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertEqual(int(size), ret_size) size = 'junk' mock_execute.return_value = (size, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertIsNone(ret_size) size_bad = '1024\n' size_good = 1024 mock_execute.return_value = (size_bad, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertEqual(size_good, ret_size) def test_multipath_reconfigure(self): self.linuxscsi.multipath_reconfigure() expected_commands = ['multipathd reconfigure'] self.assertEqual(expected_commands, self.cmds) def test_multipath_resize_map(self): wwn = '1234567890123456' self.linuxscsi.multipath_resize_map(wwn) expected_commands = ['multipathd resize map %s' % wwn] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_no_mpath(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path): """Test extending a volume where there is no multipath device.""" fake_device = {'host': '0', 'channel': '0', 'id': '0', 'lun': '1'} mock_device_info.return_value = fake_device first_size = 1024 second_size = 2048 mock_device_size.side_effect = [first_size, second_size] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = None ret_size = self.linuxscsi.extend_volume(['/dev/fake']) self.assertEqual(second_size, ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan'] self.assertEqual(expected_cmds, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_with_mpath(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path): """Test extending a volume where there is a multipath device.""" mock_device_info.side_effect = [{'host': host, 'channel': '0', 'id': '0', 'lun': '1'} for host in ['0', '1']] mock_device_size.side_effect = [1024, 2048, 1024, 2048, 1024, 2048] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = ('/dev/mapper/dm-uuid-mpath-%s' % wwn) ret_size = self.linuxscsi.extend_volume(['/dev/fake1', '/dev/fake2']) self.assertEqual(2048, ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan', 'tee -a /sys/bus/scsi/drivers/sd/1:0:0:1/rescan', 'multipathd reconfigure', 'multipathd resize map %s' % wwn] self.assertEqual(expected_cmds, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_resize_map') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_with_mpath_fail(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path, mock_mpath_resize_map): """Test extending a volume where there is a multipath device fail.""" mock_device_info.side_effect = [{'host': host, 'channel': '0', 'id': '0', 'lun': '1'} for host in ['0', '1']] mock_device_size.side_effect = [1024, 2048, 1024, 2048, 1024, 2048] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = ('/dev/mapper/dm-uuid-mpath-%s' % wwn) mock_mpath_resize_map.return_value = 'fail' ret_size = self.linuxscsi.extend_volume(['/dev/fake1', '/dev/fake2']) self.assertIsNone(ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan', 'tee -a /sys/bus/scsi/drivers/sd/1:0:0:1/rescan', 'multipathd reconfigure'] self.assertEqual(expected_cmds, self.cmds) def test_process_lun_id_list(self): lun_list = [2, 255, 88, 370, 5, 256] result = self.linuxscsi.process_lun_id(lun_list) expected = [2, 255, 88, '0x0172000000000000', 5, '0x0100000000000000'] self.assertEqual(expected, result) def test_process_lun_id_single_val_make_hex(self): lun_id = 499 result = self.linuxscsi.process_lun_id(lun_id) expected = '0x01f3000000000000' self.assertEqual(expected, result) def test_process_lun_id_single_val_make_hex_border_case(self): lun_id = 256 result = self.linuxscsi.process_lun_id(lun_id) expected = '0x0100000000000000' self.assertEqual(expected, result) def test_process_lun_id_single_var_return(self): lun_id = 13 result = self.linuxscsi.process_lun_id(lun_id) expected = 13 self.assertEqual(expected, result) @mock.patch('os_brick.privileged.rootwrap') def test_is_multipath_running_default_executor(self, mock_rootwrap): self.assertTrue( linuxscsi.LinuxSCSI.is_multipath_running( False, None, mock_rootwrap.execute)) mock_rootwrap.execute.assert_called_once_with( 'multipathd', 'show', 'status', run_as_root=True, root_helper=None) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_single_designator(self, get_wwid_mock, glob_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'wwid1' res = self.linuxscsi.get_sysfs_wwn(mock.sentinel.device_names) self.assertEqual('wwid1', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(mock.sentinel.device_names) @mock.patch('os.path.realpath', side_effect=('/other/path', '/dev/sda', '/dev/sdb')) @mock.patch('os.path.islink', side_effect=(False, True, True, True, True)) @mock.patch('os.stat', side_effect=(False, True, True, True)) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_multiple_designators(self, get_wwid_mock, glob_mock, stat_mock, islink_mock, realpath_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-fail-link', '/dev/disk/by-id/scsi-fail-stat', '/dev/disk/by-id/scsi-non-dev', '/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'pre-wwid' devices = ['sdb', 'sdc'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('wwid2', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(devices) @mock.patch('os.path.realpath', side_effect=('/dev/sda', '/dev/sdb')) @mock.patch('os.path.islink', return_value=True) @mock.patch('os.stat', return_value=True) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_not_found(self, get_wwid_mock, glob_mock, stat_mock, islink_mock, realpath_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'pre-wwid' devices = ['sdc'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(devices) @ddt.data({'wwn_type': 't10.', 'num_val': '1'}, {'wwn_type': 'eui.', 'num_val': '2'}, {'wwn_type': 'naa.', 'num_val': '3'}) @ddt.unpack @mock.patch('six.moves.builtins.open') def test_get_sysfs_wwid(self, open_mock, wwn_type, num_val): read_fail = mock.MagicMock() read_fail.__enter__.return_value.read.side_effect = IOError read_data = mock.MagicMock() read_data.__enter__.return_value.read.return_value = (wwn_type + 'wwid1\n') open_mock.side_effect = (IOError, read_fail, read_data) res = self.linuxscsi.get_sysfs_wwid(['sda', 'sdb', 'sdc']) self.assertEqual(num_val + 'wwid1', res) open_mock.assert_has_calls([mock.call('/sys/block/sda/device/wwid'), mock.call('/sys/block/sdb/device/wwid'), mock.call('/sys/block/sdc/device/wwid')]) @mock.patch('six.moves.builtins.open', side_effect=IOError) def test_get_sysfs_wwid_not_found(self, open_mock): res = self.linuxscsi.get_sysfs_wwid(['sda', 'sdb']) self.assertEqual('', res) open_mock.assert_has_calls([mock.call('/sys/block/sda/device/wwid'), mock.call('/sys/block/sdb/device/wwid')]) @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=['/dev/sda', '/dev/sdb', '/dev/sdc']) def test_remove_scsi_symlinks(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2', '/dev/disk/by-id/scsi-wwid3'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sdb', 'sdc', 'sdd']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_called_once_with(no_errors=True, *paths[1:]) @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=['/dev/sda', '/dev/sdb']) def test_remove_scsi_symlinks_no_links(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sdd', 'sde']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_not_called() @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=[OSError, '/dev/sda']) def test_remove_scsi_symlinks_race_condition(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sda']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_called_once_with(paths[1], no_errors=True) @mock.patch('glob.glob') def test_get_hctl_with_target(self, glob_mock): glob_mock.return_value = [ '/sys/class/iscsi_host/host3/device/session1/target3:4:5', '/sys/class/iscsi_host/host3/device/session1/target3:4:6'] res = self.linuxscsi.get_hctl('1', '2') self.assertEqual(('3', '4', '5', '2'), res) glob_mock.assert_called_once_with( '/sys/class/iscsi_host/host*/device/session1/target*') @mock.patch('glob.glob') def test_get_hctl_no_target(self, glob_mock): glob_mock.side_effect = [ [], ['/sys/class/iscsi_host/host3/device/session1', '/sys/class/iscsi_host/host3/device/session1']] res = self.linuxscsi.get_hctl('1', '2') self.assertEqual(('3', '-', '-', '2'), res) glob_mock.assert_has_calls( [mock.call('/sys/class/iscsi_host/host*/device/session1/target*'), mock.call('/sys/class/iscsi_host/host*/device/session1')]) @mock.patch('glob.glob', return_value=[]) def test_get_hctl_no_paths(self, glob_mock): res = self.linuxscsi.get_hctl('1', '2') self.assertIsNone(res) glob_mock.assert_has_calls( [mock.call('/sys/class/iscsi_host/host*/device/session1/target*'), mock.call('/sys/class/iscsi_host/host*/device/session1')]) @mock.patch('glob.glob') def test_device_name_by_hctl(self, glob_mock): glob_mock.return_value = [ '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda2', '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda'] res = self.linuxscsi.device_name_by_hctl('1', ('3', '4', '5', '2')) self.assertEqual('sda', res) glob_mock.assert_called_once_with( '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/*') @mock.patch('glob.glob') def test_device_name_by_hctl_wildcards(self, glob_mock): glob_mock.return_value = [ '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda2', '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda'] res = self.linuxscsi.device_name_by_hctl('1', ('3', '-', '-', '2')) self.assertEqual('sda', res) glob_mock.assert_called_once_with( '/sys/class/scsi_host/host3/device/session1/target3:*:*/3:*:*:2/' 'block/*') @mock.patch('glob.glob', mock.Mock(return_value=[])) def test_device_name_by_hctl_no_devices(self): res = self.linuxscsi.device_name_by_hctl('1', ('4', '5', '6', '2')) self.assertIsNone(res) @mock.patch.object(linuxscsi.LinuxSCSI, 'echo_scsi_command') def test_scsi_iscsi(self, echo_mock): self.linuxscsi.scan_iscsi('host', 'channel', 'target', 'lun') echo_mock.assert_called_once_with('/sys/class/scsi_host/hosthost/scan', 'channel target lun') def test_multipath_add_wwid(self): self.linuxscsi.multipath_add_wwid('wwid1') self.assertEqual(['multipath -a wwid1'], self.cmds) def test_multipath_add_path(self): self.linuxscsi.multipath_add_path('/dev/sda') self.assertEqual(['multipathd add path /dev/sda'], self.cmds) os-brick-2.3.0/os_brick/tests/initiator/test_linuxsheepdog.py0000666000175100017510000001035013230233223024471 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick import exception from os_brick.initiator import linuxsheepdog from os_brick.tests import base from oslo_concurrency import processutils SHEEP_ADDR = '127.0.0.1' SHEEP_PORT = 7000 class SheepdogVolumeIOWrapperTestCase(base.TestCase): def setUp(self): super(SheepdogVolumeIOWrapperTestCase, self).setUp() self.volume = 'volume-2f9b2ff5-987b-4412-a91c-23caaf0d5aff' self.snapshot_name = 'snapshot-bf452d80-068a-43d7-ba9f-196cf47bd0be' self.vdi_wrapper = linuxsheepdog.SheepdogVolumeIOWrapper( SHEEP_ADDR, SHEEP_PORT, self.volume) self.snapshot_wrapper = linuxsheepdog.SheepdogVolumeIOWrapper( SHEEP_ADDR, SHEEP_PORT, self.volume, self.snapshot_name) self.execute = mock.MagicMock() self.mock_object(processutils, 'execute', self.execute) def test_init(self): self.assertEqual(self.volume, self.vdi_wrapper._vdiname) self.assertIsNone(self.vdi_wrapper._snapshot_name) self.assertEqual(0, self.vdi_wrapper._offset) self.assertEqual(self.snapshot_name, self.snapshot_wrapper._snapshot_name) def test_execute(self): cmd = ('cmd1', 'arg1') data = 'data1' self.vdi_wrapper._execute(cmd, data) self.execute.assert_called_once_with(*cmd, process_input=data) def test_execute_error(self): cmd = ('cmd1', 'arg1') data = 'data1' self.mock_object(processutils, 'execute', mock.MagicMock(side_effect=OSError)) args = (cmd, data) self.assertRaises(exception.VolumeDriverException, self.vdi_wrapper._execute, *args) def test_read_vdi(self): self.vdi_wrapper.read() self.execute.assert_called_once_with( 'dog', 'vdi', 'read', '-a', SHEEP_ADDR, '-p', SHEEP_PORT, self.volume, 0, process_input=None) def test_read_vdi_invalid(self): self.vdi_wrapper._valid = False self.assertRaises(exception.VolumeDriverException, self.vdi_wrapper.read) def test_write_vdi(self): data = 'data1' self.vdi_wrapper.write(data) self.execute.assert_called_once_with( 'dog', 'vdi', 'write', '-a', SHEEP_ADDR, '-p', SHEEP_PORT, self.volume, 0, len(data), process_input=data) self.assertEqual(len(data), self.vdi_wrapper.tell()) def test_write_vdi_invalid(self): self.vdi_wrapper._valid = False self.assertRaises(exception.VolumeDriverException, self.vdi_wrapper.write, 'dummy_data') def test_read_snapshot(self): self.snapshot_wrapper.read() self.execute.assert_called_once_with( 'dog', 'vdi', 'read', '-a', SHEEP_ADDR, '-p', SHEEP_PORT, '-s', self.snapshot_name, self.volume, 0, process_input=None) def test_seek(self): self.vdi_wrapper.seek(12345) self.assertEqual(12345, self.vdi_wrapper.tell()) self.vdi_wrapper.seek(-2345, whence=1) self.assertEqual(10000, self.vdi_wrapper.tell()) # This results in negative offset. self.assertRaises(IOError, self.vdi_wrapper.seek, -20000, whence=1) def test_seek_invalid(self): seek_num = 12345 self.vdi_wrapper._valid = False self.assertRaises(exception.VolumeDriverException, self.vdi_wrapper.seek, seek_num) def test_flush(self): # flush does nothing. self.vdi_wrapper.flush() self.assertFalse(self.execute.called) def test_fileno(self): self.assertRaises(IOError, self.vdi_wrapper.fileno) os-brick-2.3.0/os_brick/tests/initiator/test_connector.py0000666000175100017510000002535413230233223023617 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import platform import sys import mock from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator import connector from os_brick.initiator.connectors import base from os_brick.initiator.connectors import fake from os_brick.initiator.connectors import iscsi from os_brick.initiator import linuxfc from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base as test_base MY_IP = '10.0.0.1' FAKE_SCSI_WWN = '1234567890' class ConnectorUtilsTestCase(test_base.TestCase): @mock.patch.object(iscsi.ISCSIConnector, 'get_initiator', return_value='fakeinitiator') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_wwpns', return_value=None) @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_wwnns', return_value=None) @mock.patch.object(platform, 'machine', mock.Mock(return_value='s390x')) @mock.patch('sys.platform', 'linux2') def _test_brick_get_connector_properties(self, multipath, enforce_multipath, multipath_result, mock_wwnns, mock_wwpns, mock_initiator, host='fakehost'): props_actual = connector.get_connector_properties('sudo', MY_IP, multipath, enforce_multipath, host=host) os_type = 'linux2' platform = 's390x' props = {'initiator': 'fakeinitiator', 'host': host, 'ip': MY_IP, 'multipath': multipath_result, 'os_type': os_type, 'platform': platform, 'do_local_attach': False} self.assertEqual(props, props_actual) def test_brick_get_connector_properties_connectors_called(self): """Make sure every connector is called.""" mock_list = [] # Make sure every connector is called for item in connector.connector_list: patched = mock.MagicMock() patched.platform = platform.machine() patched.os_type = sys.platform patched.__name__ = item patched.get_connector_properties.return_value = {} patcher = mock.patch(item, new=patched) patcher.start() self.addCleanup(patcher.stop) mock_list.append(patched) connector.get_connector_properties('sudo', MY_IP, True, True) for item in mock_list: assert item.get_connector_properties.called def test_brick_get_connector_properties(self): self._test_brick_get_connector_properties(False, False, False) @mock.patch.object(priv_rootwrap, 'execute') def test_brick_get_connector_properties_multipath(self, mock_execute): self._test_brick_get_connector_properties(True, True, True) mock_execute.assert_called_once_with('multipathd', 'show', 'status', run_as_root=True, root_helper='sudo') @mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError) def test_brick_get_connector_properties_fallback(self, mock_execute): self._test_brick_get_connector_properties(True, False, False) mock_execute.assert_called_once_with('multipathd', 'show', 'status', run_as_root=True, root_helper='sudo') @mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError) def test_brick_get_connector_properties_raise(self, mock_execute): self.assertRaises(putils.ProcessExecutionError, self._test_brick_get_connector_properties, True, True, None) def test_brick_connector_properties_override_hostname(self): override_host = 'myhostname' self._test_brick_get_connector_properties(False, False, False, host=override_host) class ConnectorTestCase(test_base.TestCase): def setUp(self): super(ConnectorTestCase, self).setUp() self.cmds = [] def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def fake_connection(self): return { 'driver_volume_type': 'fake', 'data': { 'volume_id': 'fake_volume_id', 'target_portal': 'fake_location', 'target_iqn': 'fake_iqn', 'target_lun': 1, } } def test_connect_volume(self): self.connector = fake.FakeConnector(None) device_info = self.connector.connect_volume(self.fake_connection()) self.assertIn('type', device_info) self.assertIn('path', device_info) def test_disconnect_volume(self): self.connector = fake.FakeConnector(None) def test_get_connector_properties(self): with mock.patch.object(priv_rootwrap, 'execute') as mock_exec: mock_exec.return_value = True multipath = True enforce_multipath = True props = base.BaseLinuxConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'multipath': True} self.assertEqual(expected_props, props) multipath = False enforce_multipath = True props = base.BaseLinuxConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'multipath': False} self.assertEqual(expected_props, props) with mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError): multipath = True enforce_multipath = True self.assertRaises( putils.ProcessExecutionError, base.BaseLinuxConnector.get_connector_properties, 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) def test_factory(self): obj = connector.InitiatorConnector.factory('iscsi', None) self.assertEqual("ISCSIConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('iscsi', None, arch='ppc64le') self.assertEqual("ISCSIConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('fibre_channel', None, arch='x86_64') self.assertEqual("FibreChannelConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('fibre_channel', None, arch='s390x') self.assertEqual("FibreChannelConnectorS390X", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('aoe', None, arch='x86_64') self.assertEqual("AoEConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'nfs', None, nfs_mount_point_base='/mnt/test') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'glusterfs', None, glusterfs_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'scality', None, scality_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('local', None) self.assertEqual("LocalConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('gpfs', None) self.assertEqual("GPFSConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'huaweisdshypervisor', None, arch='x86_64') self.assertEqual("HuaweiStorHyperConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( "scaleio", None, arch='x86_64') self.assertEqual("ScaleIOConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'quobyte', None, quobyte_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( "disco", None, arch='x86_64') self.assertEqual("DISCOConnector", obj.__class__.__name__) self.assertRaises(exception.InvalidConnectorProtocol, connector.InitiatorConnector.factory, "bogus", None) def test_check_valid_device_with_wrong_path(self): self.connector = fake.FakeConnector(None) self.connector._execute = \ lambda *args, **kwargs: ("", None) self.assertFalse(self.connector.check_valid_device('/d0v')) def test_check_valid_device(self): self.connector = fake.FakeConnector(None) self.connector._execute = \ lambda *args, **kwargs: ("", "") self.assertTrue(self.connector.check_valid_device('/dev')) def test_check_valid_device_with_cmd_error(self): def raise_except(*args, **kwargs): raise putils.ProcessExecutionError self.connector = fake.FakeConnector(None) with mock.patch.object(self.connector, '_execute', side_effect=putils.ProcessExecutionError): self.assertFalse(self.connector.check_valid_device('/dev')) os-brick-2.3.0/os_brick/tests/initiator/connectors/0000775000175100017510000000000013230233405022360 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/initiator/connectors/test_scaleio.py0000666000175100017510000002506013230233223025413 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json import mock import os import requests import six from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator.connectors import scaleio from os_brick.tests.initiator import test_connector class ScaleIOConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for ScaleIO connector.""" # Fake volume information vol = { 'id': 'vol1', 'name': 'test_volume', 'provider_id': 'vol1' } # Fake SDC GUID fake_guid = 'FAKE_GUID' def setUp(self): super(ScaleIOConnectorTestCase, self).setUp() self.fake_connection_properties = { 'hostIP': test_connector.MY_IP, 'serverIP': test_connector.MY_IP, 'scaleIO_volname': self.vol['name'], 'scaleIO_volume_id': self.vol['provider_id'], 'serverPort': 443, 'serverUsername': 'test', 'serverPassword': 'fake', 'serverToken': 'fake_token', 'iopsLimit': None, 'bandwidthLimit': None } # Formatting string for REST API calls self.action_format = "instances/Volume::{}/action/{{}}".format( self.vol['id']) self.get_volume_api = 'types/Volume/instances/getByName::{}'.format( self.vol['name']) # Map of REST API calls to responses self.mock_calls = { self.get_volume_api: self.MockHTTPSResponse(json.dumps(self.vol['id'])), self.action_format.format('addMappedSdc'): self.MockHTTPSResponse(''), self.action_format.format('setMappedSdcLimits'): self.MockHTTPSResponse(''), self.action_format.format('removeMappedSdc'): self.MockHTTPSResponse(''), } # Default error REST response self.error_404 = self.MockHTTPSResponse(content=dict( errorCode=0, message='HTTP 404', ), status_code=404) # Patch the request and os calls to fake versions self.mock_object(requests, 'get', self.handle_scaleio_request) self.mock_object(requests, 'post', self.handle_scaleio_request) self.mock_object(os.path, 'isdir', return_value=True) self.mock_object(os, 'listdir', return_value=["emc-vol-{}".format(self.vol['id'])]) # The actual ScaleIO connector self.connector = scaleio.ScaleIOConnector( 'sudo', execute=self.fake_execute) class MockHTTPSResponse(requests.Response): """Mock HTTP Response Defines the https replies from the mocked calls to do_request() """ def __init__(self, content, status_code=200): super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).__init__() self._content = content self.encoding = 'UTF-8' self.status_code = status_code def json(self, **kwargs): if isinstance(self._content, six.string_types): return super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).json(**kwargs) return self._content @property def text(self): if not isinstance(self._content, six.string_types): return json.dumps(self._content) self._content = self._content.encode('utf-8') return super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).text def fake_execute(self, *cmd, **kwargs): """Fakes the rootwrap call""" return self.fake_guid, None def fake_missing_execute(self, *cmd, **kwargs): """Error when trying to call rootwrap drv_cfg""" raise putils.ProcessExecutionError("Test missing drv_cfg.") def handle_scaleio_request(self, url, *args, **kwargs): """Fake REST server""" api_call = url.split(':', 2)[2].split('/', 1)[1].replace('api/', '') if 'setMappedSdcLimits' in api_call: self.assertNotIn("iops_limit", kwargs['data']) if "iopsLimit" not in kwargs['data']: self.assertIn("bandwidthLimitInKbps", kwargs['data']) elif "bandwidthLimitInKbps" not in kwargs['data']: self.assertIn("iopsLimit", kwargs['data']) else: self.assertIn("bandwidthLimitInKbps", kwargs['data']) self.assertIn("iopsLimit", kwargs['data']) try: return self.mock_calls[api_call] except KeyError: return self.error_404 def test_get_search_path(self): expected = "/dev/disk/by-id" actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(scaleio.ScaleIOConnector, '_wait_for_volume_path') def test_get_volume_paths(self, mock_wait_for_path, mock_exists): mock_wait_for_path.return_value = "emc-vol-vol1" expected = ['/dev/disk/by-id/emc-vol-vol1'] actual = self.connector.get_volume_paths( self.fake_connection_properties) self.assertEqual(expected, actual) def test_get_connector_properties(self): props = scaleio.ScaleIOConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume(self): """Successful connect to volume""" self.connector.connect_volume(self.fake_connection_properties) def test_connect_with_bandwidth_limit(self): """Successful connect to volume with bandwidth limit""" self.fake_connection_properties['bandwidthLimit'] = '500' self.test_connect_volume() def test_connect_with_iops_limit(self): """Successful connect to volume with iops limit""" self.fake_connection_properties['iopsLimit'] = '80' self.test_connect_volume() def test_connect_with_iops_and_bandwidth_limits(self): """Successful connect with iops and bandwidth limits""" self.fake_connection_properties['bandwidthLimit'] = '500' self.fake_connection_properties['iopsLimit'] = '80' self.test_connect_volume() def test_disconnect_volume(self): """Successful disconnect from volume""" self.connector.disconnect_volume(self.fake_connection_properties, None) def test_error_id(self): """Fail to connect with bad volume name""" self.fake_connection_properties['scaleIO_volume_id'] = 'bad_id' self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( dict(errorCode='404', message='Test volume not found'), 404) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_no_volume_id(self): """Faile to connect with no volume id""" self.fake_connection_properties['scaleIO_volume_id'] = None self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( 'null', 200) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_bad_login(self): """Fail to connect with bad authentication""" self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( 'null', 401) self.mock_calls['login'] = self.MockHTTPSResponse('null', 401) self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=401, message='bad login'), 401) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_bad_drv_cfg(self): """Fail to connect with missing rootwrap executable""" self.connector.set_execute(self.fake_missing_execute) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_map_volume(self): """Fail to connect with REST API failure""" self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_NOT_MAPPED_ERROR, message='Test error map volume'), 500) self.assertRaises(exception.BrickException, self.test_connect_volume) @mock.patch('time.sleep') def test_error_path_not_found(self, sleep_mock): """Timeout waiting for volume to map to local file system""" self.mock_object(os, 'listdir', return_value=["emc-vol-no-volume"]) self.assertRaises(exception.BrickException, self.test_connect_volume) self.assertTrue(sleep_mock.called) def test_map_volume_already_mapped(self): """Ignore REST API failure for volume already mapped""" self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_ALREADY_MAPPED_ERROR, message='Test error map volume'), 500) self.test_connect_volume() def test_error_disconnect_volume(self): """Fail to disconnect with REST API failure""" self.mock_calls[self.action_format.format( 'removeMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_ALREADY_MAPPED_ERROR, message='Test error map volume'), 500) self.assertRaises(exception.BrickException, self.test_disconnect_volume) def test_disconnect_volume_not_mapped(self): """Ignore REST API failure for volume not mapped""" self.mock_calls[self.action_format.format( 'removeMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_NOT_MAPPED_ERROR, message='Test error map volume'), 500) self.test_disconnect_volume() def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.fake_connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_base_iscsi.py0000666000175100017510000000646113230233223026104 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import base_iscsi from os_brick.initiator.connectors import fake from os_brick.tests import base as test_base class BaseISCSIConnectorTestCase(test_base.TestCase): def setUp(self): super(BaseISCSIConnectorTestCase, self).setUp() self.connector = fake.FakeBaseISCSIConnector(None) @mock.patch.object(base_iscsi.BaseISCSIConnector, '_get_all_targets') def test_iterate_all_targets(self, mock_get_all_targets): # extra_property cannot be a sentinel, a copied sentinel will not # identical to the original one. connection_properties = { 'target_portals': mock.sentinel.target_portals, 'target_iqns': mock.sentinel.target_iqns, 'target_luns': mock.sentinel.target_luns, 'extra_property': 'extra_property'} mock_get_all_targets.return_value = [( mock.sentinel.portal, mock.sentinel.iqn, mock.sentinel.lun)] # method is a generator, and it yields dictionaries. list() will # iterate over all of the method's items. list_props = list( self.connector._iterate_all_targets(connection_properties)) mock_get_all_targets.assert_called_once_with(connection_properties) self.assertEqual(1, len(list_props)) expected_props = {'target_portal': mock.sentinel.portal, 'target_iqn': mock.sentinel.iqn, 'target_lun': mock.sentinel.lun, 'extra_property': 'extra_property'} self.assertEqual(expected_props, list_props[0]) def test_get_all_targets(self): connection_properties = { 'target_portals': [mock.sentinel.target_portals], 'target_iqns': [mock.sentinel.target_iqns], 'target_luns': [mock.sentinel.target_luns]} all_targets = self.connector._get_all_targets(connection_properties) expected_targets = zip([mock.sentinel.target_portals], [mock.sentinel.target_iqns], [mock.sentinel.target_luns]) self.assertEqual(list(expected_targets), list(all_targets)) def test_get_all_targets_single_target(self): connection_properties = { 'target_portal': mock.sentinel.target_portal, 'target_iqn': mock.sentinel.target_iqn, 'target_lun': mock.sentinel.target_lun} all_targets = self.connector._get_all_targets(connection_properties) expected_target = (mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun) self.assertEqual([expected_target], all_targets) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_vmware.py0000666000175100017510000003502613230233223025300 0ustar zuulzuul00000000000000# Copyright (c) 2016 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from oslo_utils import units from oslo_vmware.objects import datastore from oslo_vmware import vim_util from os_brick import exception from os_brick.initiator.connectors import vmware from os_brick.tests.initiator import test_connector @ddt.ddt class VmdkConnectorTestCase(test_connector.ConnectorTestCase): IP = '127.0.0.1' PORT = 443 USERNAME = 'username' PASSWORD = 'password' API_RETRY_COUNT = 3 TASK_POLL_INTERVAL = 5.0 CA_FILE = "/etc/ssl/rui-ca-cert.pem" TMP_DIR = "/vmware-tmp" IMG_TX_TIMEOUT = 10 VMDK_CONNECTOR = vmware.VmdkConnector def setUp(self): super(VmdkConnectorTestCase, self).setUp() self._connector = vmware.VmdkConnector(None) self._connector._ip = self.IP self._connector._port = self.PORT self._connector._username = self.USERNAME self._connector._password = self.PASSWORD self._connector._api_retry_count = self.API_RETRY_COUNT self._connector._task_poll_interval = self.TASK_POLL_INTERVAL self._connector._ca_file = self.CA_FILE self._connector._insecure = True self._connector._tmp_dir = self.TMP_DIR self._connector._timeout = self.IMG_TX_TIMEOUT def test_load_config(self): config = { 'vmware_host_ip': 'localhost', 'vmware_host_port': 1234, 'vmware_host_username': 'root', 'vmware_host_password': 'pswd', 'vmware_api_retry_count': 1, 'vmware_task_poll_interval': 1.0, 'vmware_ca_file': None, 'vmware_insecure': False, 'vmware_tmp_dir': '/tmp', 'vmware_image_transfer_timeout_secs': 5, } self._connector._load_config({'config': config}) self.assertEqual('localhost', self._connector._ip) self.assertEqual(1234, self._connector._port) self.assertEqual('root', self._connector._username) self.assertEqual('pswd', self._connector._password) self.assertEqual(1, self._connector._api_retry_count) self.assertEqual(1.0, self._connector._task_poll_interval) self.assertIsNone(self._connector._ca_file) self.assertFalse(self._connector._insecure) self.assertEqual('/tmp', self._connector._tmp_dir) self.assertEqual(5, self._connector._timeout) @mock.patch('oslo_vmware.api.VMwareAPISession') def test_create_session(self, session): session.return_value = mock.sentinel.session ret = self._connector._create_session() self.assertEqual(mock.sentinel.session, ret) session.assert_called_once_with( self._connector._ip, self._connector._username, self._connector._password, self._connector._api_retry_count, self._connector._task_poll_interval, port=self._connector._port, cacert=self._connector._ca_file, insecure=self._connector._insecure) @mock.patch('oslo_utils.fileutils.ensure_tree') @mock.patch('tempfile.mkstemp') @mock.patch('os.close') def test_create_temp_file( self, close, mkstemp, ensure_tree): fd = mock.sentinel.fd tmp = mock.sentinel.tmp mkstemp.return_value = (fd, tmp) prefix = ".vmdk" suffix = "test" ret = self._connector._create_temp_file(prefix=prefix, suffix=suffix) self.assertEqual(tmp, ret) ensure_tree.assert_called_once_with(self._connector._tmp_dir) mkstemp.assert_called_once_with(dir=self._connector._tmp_dir, prefix=prefix, suffix=suffix) close.assert_called_once_with(fd) @mock.patch('os_brick.initiator.connectors.vmware.open', create=True) @mock.patch('oslo_vmware.image_transfer.copy_stream_optimized_disk') def test_download_vmdk(self, copy_disk, file_open): file_open_ret = mock.Mock() tmp_file = mock.sentinel.tmp_file file_open_ret.__enter__ = mock.Mock(return_value=tmp_file) file_open_ret.__exit__ = mock.Mock(return_value=None) file_open.return_value = file_open_ret tmp_file_path = mock.sentinel.tmp_file_path session = mock.sentinel.session backing = mock.sentinel.backing vmdk_path = mock.sentinel.vmdk_path vmdk_size = mock.sentinel.vmdk_size self._connector._download_vmdk( tmp_file_path, session, backing, vmdk_path, vmdk_size) file_open.assert_called_once_with(tmp_file_path, 'wb') copy_disk.assert_called_once_with(None, self._connector._timeout, tmp_file, session=session, host=self._connector._ip, port=self._connector._port, vm=backing, vmdk_file_path=vmdk_path, vmdk_size=vmdk_size) def _create_connection_properties(self): return {'volume_id': 'ed083474-d325-4a99-b301-269111654f0d', 'volume': 'ref-1', 'vmdk_path': '[ds] foo/bar.vmdk', 'vmdk_size': units.Gi, 'datastore': 'ds-1', 'datacenter': 'dc-1', } @mock.patch.object(VMDK_CONNECTOR, '_load_config') @mock.patch.object(VMDK_CONNECTOR, '_create_session') @mock.patch.object(VMDK_CONNECTOR, '_create_temp_file') @mock.patch('oslo_vmware.vim_util.get_moref') @mock.patch.object(VMDK_CONNECTOR, '_download_vmdk') @mock.patch('os.path.getmtime') def test_connect_volume( self, getmtime, download_vmdk, get_moref, create_temp_file, create_session, load_config): session = mock.Mock() create_session.return_value = session tmp_file_path = mock.sentinel.tmp_file_path create_temp_file.return_value = tmp_file_path backing = mock.sentinel.backing get_moref.return_value = backing last_modified = mock.sentinel.last_modified getmtime.return_value = last_modified props = self._create_connection_properties() ret = self._connector.connect_volume(props) self.assertEqual(tmp_file_path, ret['path']) self.assertEqual(last_modified, ret['last_modified']) load_config.assert_called_once_with(props) create_session.assert_called_once_with() create_temp_file.assert_called_once_with( suffix=".vmdk", prefix=props['volume_id']) download_vmdk.assert_called_once_with( tmp_file_path, session, backing, props['vmdk_path'], props['vmdk_size']) session.logout.assert_called_once_with() @ddt.data((None, False), ([mock.sentinel.snap], True)) @ddt.unpack def test_snapshot_exists(self, snap_list, exp_return_value): snapshot = mock.Mock(rootSnapshotList=snap_list) session = mock.Mock() session.invoke_api.return_value = snapshot backing = mock.sentinel.backing ret = self._connector._snapshot_exists(session, backing) self.assertEqual(exp_return_value, ret) session.invoke_api.assert_called_once_with( vim_util, 'get_object_property', session.vim, backing, 'snapshot') def test_create_temp_ds_folder(self): session = mock.Mock() ds_folder_path = mock.sentinel.ds_folder_path dc_ref = mock.sentinel.dc_ref self._connector._create_temp_ds_folder(session, ds_folder_path, dc_ref) session.invoke_api.assert_called_once_with( session.vim, 'MakeDirectory', session.vim.service_content.fileManager, name=ds_folder_path, datacenter=dc_ref) @mock.patch('oslo_vmware.objects.datastore.get_datastore_by_ref') @mock.patch.object(VMDK_CONNECTOR, '_create_temp_ds_folder') @mock.patch('os_brick.initiator.connectors.vmware.open', create=True) @mock.patch.object(VMDK_CONNECTOR, '_upload_vmdk') @mock.patch('os.path.getsize') def test_disconnect( self, getsize, upload_vmdk, file_open, create_temp_ds_folder, get_ds_by_ref): ds_ref = mock.sentinel.ds_ref ds_name = 'datastore-1' dstore = datastore.Datastore(ds_ref, ds_name) get_ds_by_ref.return_value = dstore file_open_ret = mock.Mock() tmp_file = mock.sentinel.tmp_file file_open_ret.__enter__ = mock.Mock(return_value=tmp_file) file_open_ret.__exit__ = mock.Mock(return_value=None) file_open.return_value = file_open_ret dc_name = mock.sentinel.dc_name delete_task = mock.sentinel.delete_vdisk_task copy_task = mock.sentinel.copy_vdisk_task delete_file_task = mock.sentinel.delete_file_task session = mock.Mock() session.invoke_api.side_effect = [ dc_name, delete_task, copy_task, delete_file_task] getsize.return_value = units.Gi tmp_file_path = '/tmp/foo.vmdk' dc_ref = mock.sentinel.dc_ref vmdk_path = mock.sentinel.vmdk_path self._connector._disconnect( tmp_file_path, session, ds_ref, dc_ref, vmdk_path) tmp_folder_path = self._connector.TMP_IMAGES_DATASTORE_FOLDER_PATH ds_folder_path = '[%s] %s' % (ds_name, tmp_folder_path) create_temp_ds_folder.assert_called_once_with( session, ds_folder_path, dc_ref) file_open.assert_called_once_with(tmp_file_path, "rb") self.assertEqual( mock.call(vim_util, 'get_object_property', session.vim, dc_ref, 'name'), session.invoke_api.call_args_list[0]) exp_rel_path = '%s/foo.vmdk' % tmp_folder_path upload_vmdk.assert_called_once_with( tmp_file, self._connector._ip, self._connector._port, dc_name, ds_name, session.vim.client.options.transport.cookiejar, exp_rel_path, units.Gi, self._connector._ca_file, self._connector._timeout) disk_mgr = session.vim.service_content.virtualDiskManager self.assertEqual( mock.call(session.vim, 'DeleteVirtualDisk_Task', disk_mgr, name=vmdk_path, datacenter=dc_ref), session.invoke_api.call_args_list[1]) self.assertEqual(mock.call(delete_task), session.wait_for_task.call_args_list[0]) src = '[%s] %s' % (ds_name, exp_rel_path) self.assertEqual( mock.call(session.vim, 'CopyVirtualDisk_Task', disk_mgr, sourceName=src, sourceDatacenter=dc_ref, destName=vmdk_path, destDatacenter=dc_ref), session.invoke_api.call_args_list[2]) self.assertEqual(mock.call(copy_task), session.wait_for_task.call_args_list[1]) file_mgr = session.vim.service_content.fileManager self.assertEqual( mock.call(session.vim, 'DeleteDatastoreFile_Task', file_mgr, name=src, datacenter=dc_ref), session.invoke_api.call_args_list[3]) self.assertEqual(mock.call(delete_file_task), session.wait_for_task.call_args_list[2]) @mock.patch('os.path.exists') def test_disconnect_volume_with_missing_temp_file(self, path_exists): path_exists.return_value = False path = mock.sentinel.path self.assertRaises(exception.NotFound, self._connector.disconnect_volume, mock.ANY, {'path': path}) path_exists.assert_called_once_with(path) @mock.patch('os.path.exists') @mock.patch('os.path.getmtime') @mock.patch.object(VMDK_CONNECTOR, '_disconnect') @mock.patch('os.remove') def test_disconnect_volume_with_unmodified_file( self, remove, disconnect, getmtime, path_exists): path_exists.return_value = True mtime = 1467802060 getmtime.return_value = mtime path = mock.sentinel.path self._connector.disconnect_volume(mock.ANY, {'path': path, 'last_modified': mtime}) path_exists.assert_called_once_with(path) getmtime.assert_called_once_with(path) disconnect.assert_not_called() remove.assert_called_once_with(path) @mock.patch('os.path.exists') @mock.patch('os.path.getmtime') @mock.patch.object(VMDK_CONNECTOR, '_load_config') @mock.patch.object(VMDK_CONNECTOR, '_create_session') @mock.patch('oslo_vmware.vim_util.get_moref') @mock.patch.object(VMDK_CONNECTOR, '_snapshot_exists') @mock.patch.object(VMDK_CONNECTOR, '_disconnect') @mock.patch('os.remove') def test_disconnect_volume( self, remove, disconnect, snapshot_exists, get_moref, create_session, load_config, getmtime, path_exists): path_exists.return_value = True mtime = 1467802060 getmtime.return_value = mtime session = mock.Mock() create_session.return_value = session snapshot_exists.return_value = False backing = mock.sentinel.backing ds_ref = mock.sentinel.ds_ref dc_ref = mock.sentinel.dc_ref get_moref.side_effect = [backing, ds_ref, dc_ref] props = self._create_connection_properties() path = mock.sentinel.path self._connector.disconnect_volume(props, {'path': path, 'last_modified': mtime - 1}) path_exists.assert_called_once_with(path) getmtime.assert_called_once_with(path) load_config.assert_called_once_with(props) create_session.assert_called_once_with() snapshot_exists.assert_called_once_with(session, backing) disconnect.assert_called_once_with( path, session, ds_ref, dc_ref, props['vmdk_path']) remove.assert_called_once_with(path) session.logout.assert_called_once_with() os-brick-2.3.0/os_brick/tests/initiator/connectors/test_fibre_channel_ppc64.py0000666000175100017510000000441213230233223027565 0ustar zuulzuul00000000000000# (c) Copyright 2013 IBM Company # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import fibre_channel_ppc64 from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector class FibreChannelConnectorPPC64TestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorPPC64TestCase, self).setUp() self.connector = fibre_channel_ppc64.FibreChannelConnectorPPC64( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertEqual(self.connector._linuxfc.__class__.__name__, "LinuxFibreChannelPPC64") self.assertIsNotNone(self.connector._linuxscsi) @mock.patch.object(linuxscsi.LinuxSCSI, 'process_lun_id', return_value='2') def test_get_host_devices(self, mock_process_lun_id): lun = 2 possible_devs = [(3, "0x5005076802232ade"), (3, "0x5005076802332ade"), ] devices = self.connector._get_host_devices(possible_devs, lun) self.assertEqual(2, len(devices)) device_path = "/dev/disk/by-path/fc-0x5005076802332ade-lun-2" self.assertIn(device_path, devices) device_path = "/dev/disk/by-path/fc-0x5005076802232ade-lun-2" self.assertIn(device_path, devices) # test duplicates possible_devs = [(3, "0x5005076802232ade"), (3, "0x5005076802232ade"), ] devices = self.connector._get_host_devices(possible_devs, lun) self.assertEqual(1, len(devices)) device_path = "/dev/disk/by-path/fc-0x5005076802232ade-lun-2" self.assertIn(device_path, devices) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_hgst.py0000666000175100017510000002077013230233223024744 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator import connector from os_brick.initiator.connectors import hgst from os_brick.tests.initiator import test_connector class HGSTConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for HGST initiator class.""" IP_OUTPUT = """ 1: lo: mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 169.254.169.254/32 scope link lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: em1: mtu 1500 qdisc mq master link/ether 00:25:90:d9:18:08 brd ff:ff:ff:ff:ff:ff inet6 fe80::225:90ff:fed9:1808/64 scope link valid_lft forever preferred_lft forever 3: em2: mtu 1500 qdisc mq state link/ether 00:25:90:d9:18:09 brd ff:ff:ff:ff:ff:ff inet 192.168.0.23/24 brd 192.168.0.255 scope global em2 valid_lft forever preferred_lft forever inet6 fe80::225:90ff:fed9:1809/64 scope link valid_lft forever preferred_lft forever """ DOMAIN_OUTPUT = """localhost""" DOMAIN_FAILED = """this.better.not.resolve.to.a.name.or.else""" SET_APPHOST_OUTPUT = """ VLVM_SET_APPHOSTS0000000395 Request Succeeded """ def setUp(self): super(HGSTConnectorTestCase, self).setUp() self.connector = hgst.HGSTConnector( None, execute=self._fake_exec) self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False def _fake_exec_set_apphosts(self, *cmd): if self._fail_set_apphosts: raise putils.ProcessExecutionError(None, None, 1) else: return self.SET_APPHOST_OUTPUT, '' def _fake_exec_ip(self, *cmd): if self._fail_ip: # Remove localhost so there is no IP match return self.IP_OUTPUT.replace("127.0.0.1", "x.x.x.x"), '' else: return self.IP_OUTPUT, '' def _fake_exec_domain_list(self, *cmd): if self._fail_domain_list: return self.DOMAIN_FAILED, '' else: return self.DOMAIN_OUTPUT, '' def _fake_exec(self, *cmd, **kwargs): self.cmdline = " ".join(cmd) if cmd[0] == "ip": return self._fake_exec_ip(*cmd) elif cmd[0] == "vgc-cluster": if cmd[1] == "domain-list": return self._fake_exec_domain_list(*cmd) elif cmd[1] == "space-set-apphosts": return self._fake_exec_set_apphosts(*cmd) else: return '', '' def test_factory(self): """Can we instantiate a HGSTConnector of the right kind?""" obj = connector.InitiatorConnector.factory('HGST', None, arch='x86_64') self.assertEqual("HGSTConnector", obj.__class__.__name__) def test_get_search_path(self): expected = "/dev" actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(os.path, 'exists', return_value=True) def test_get_volume_paths(self, mock_exists): cprops = {'name': 'space', 'noremovehost': 'stor1'} path = "/dev/%s" % cprops['name'] expected = [path] actual = self.connector.get_volume_paths(cprops) self.assertEqual(expected, actual) def test_connect_volume(self): """Tests that a simple connection succeeds""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} dev_info = self.connector.connect_volume(cprops) self.assertEqual('block', dev_info['type']) self.assertEqual('space', dev_info['device']) self.assertEqual('/dev/space', dev_info['path']) def test_get_connector_properties(self): props = hgst.HGSTConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume_nohost_fail(self): """This host should not be found, connect should fail.""" self._fail_set_apphosts = False self._fail_ip = True self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) def test_connect_volume_nospace_fail(self): """The space command will fail, exception to be thrown""" self._fail_set_apphosts = True self._fail_ip = False self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) def test_disconnect_volume(self): """Simple disconnection should pass and disconnect me""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'stor1'} self.connector.disconnect_volume(cprops, None) exp_cli = ("vgc-cluster space-set-apphosts -n space " "-A localhost --action DELETE") self.assertEqual(exp_cli, self.cmdline) def test_disconnect_volume_nohost(self): """Should not run a setapphosts because localhost will""" """be the noremotehost""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'localhost'} self.connector.disconnect_volume(cprops, None) # The last command should be the IP listing, not set apphosts exp_cli = ("ip addr list") self.assertEqual(exp_cli, self.cmdline) def test_disconnect_volume_fails(self): """The set-apphosts should fail, exception to be thrown""" self._fail_set_apphosts = True self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) def test_bad_connection_properties(self): """Send in connection_properties missing required fields""" # Invalid connection_properties self.assertRaises(exception.BrickException, self.connector.connect_volume, None) # Name required for connect_volume cprops = {'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) # Invalid connection_properties self.assertRaises(exception.BrickException, self.connector.disconnect_volume, None, None) # Name and noremovehost needed for disconnect_volume cprops = {'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) cprops = {'name': 'space'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) def test_extend_volume(self): cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(NotImplementedError, self.connector.extend_volume, cprops) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_iscsi.py0000666000175100017510000021775713230233223025126 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import mock import os import ddt from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator.connectors import iscsi from os_brick.initiator import linuxscsi from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests.initiator import test_connector @ddt.ddt class ISCSIConnectorTestCase(test_connector.ConnectorTestCase): SINGLE_CON_PROPS = {'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1', 'encryption': False, 'target_lun': '1'} CON_PROPS = { 'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1', 'target_lun': 4, 'target_portals': ['ip1:port1', 'ip2:port2', 'ip3:port3', 'ip4:port4'], 'target_iqns': ['tgt1', 'tgt2', 'tgt3', 'tgt4'], 'target_luns': [4, 5, 6, 7], } def setUp(self): super(ISCSIConnectorTestCase, self).setUp() self.connector = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=False) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) self.mock_object(self.connector._linuxscsi, 'get_name_from_path', return_value="/dev/sdb") self._fake_iqn = 'iqn.1234-56.foo.bar:01:23456789abc' self._name = 'volume-00000001' self._iqn = 'iqn.2010-10.org.openstack:%s' % self._name self._location = '10.0.2.15:3260' self._lun = 1 @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsi_session') def test_get_iscsi_sessions_full(self, sessions_mock): iscsiadm_result = ('tcp: [session1] ip1:port1,1 tgt1 (non-flash)\n' 'tcp: [session2] ip2:port2,-1 tgt2 (non-flash)\n' 'tcp: [session3] ip3:port3,1 tgt3\n') sessions_mock.return_value = (iscsiadm_result, '') res = self.connector._get_iscsi_sessions_full() expected = [('tcp:', 'session1', 'ip1:port1', '1', 'tgt1'), ('tcp:', 'session2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', 'session3', 'ip3:port3', '1', 'tgt3')] self.assertListEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsi_session', return_value=(None, 'error')) def test_get_iscsi_sessions_full_error(self, sessions_mock): res = self.connector._get_iscsi_sessions_full() self.assertEqual([], res) sessions_mock.assert_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_get_iscsi_sessions(self, sessions_mock): sessions_mock.return_value = [ ('tcp:', 'session1', 'ip1:port1', '1', 'tgt1'), ('tcp:', 'session2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', 'session3', 'ip3:port3', '1', 'tgt3')] res = self.connector._get_iscsi_sessions() expected = ['ip1:port1', 'ip2:port2', 'ip3:port3'] self.assertListEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full', return_value=[]) def test_get_iscsi_sessions_no_sessions(self, sessions_mock): res = self.connector._get_iscsi_sessions() self.assertListEqual([], res) sessions_mock.assert_called() @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_iscsi_nodes(self, exec_mock): iscsiadm_result = ('ip1:port1,1 tgt1\nip2:port2,-1 tgt2\n' 'ip3:port3,1 tgt3\n') exec_mock.return_value = (iscsiadm_result, '') res = self.connector._get_iscsi_nodes() expected = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] self.assertListEqual(expected, res) exec_mock.assert_called_once_with( 'iscsiadm', '-m', 'node', run_as_root=True, root_helper=self.connector._root_helper, check_exit_code=False) @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_iscsi_nodes_error(self, exec_mock): exec_mock.return_value = (None, 'error') res = self.connector._get_iscsi_nodes() self.assertEqual([], res) @mock.patch.object(iscsi.ISCSIConnector, '_get_ips_iqns_luns') @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices(self, nodes_mock, sessions_mock, glob_mock, iql_mock): self.connector.use_multipath = True iql_mock.return_value = self.connector._get_all_targets(self.CON_PROPS) # List sessions from other targets and non tcp sessions sessions_mock.return_value = [ ('non-tcp:', '0', 'ip1:port1', '1', 'tgt1'), ('tcp:', '1', 'ip1:port1', '1', 'tgt1'), ('tcp:', '2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', '3', 'ip1:port1', '1', 'tgt4'), ('tcp:', '4', 'ip2:port2', '-1', 'tgt5')] # List 1 node without sessions nodes_mock.return_value = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session1/target6/1:2:6:4/block/sda', sys_cls + '1/device/session1/target6/1:2:6:4/block/sda1'], [sys_cls + '2/device/session2/target7/2:2:7:5/block/sdb', sys_cls + '2/device/session2/target7/2:2:7:4/block/sdc'], ] res = self.connector._get_connection_devices(self.CON_PROPS) expected = {('ip1:port1', 'tgt1'): ({'sda'}, set()), ('ip2:port2', 'tgt2'): ({'sdb'}, {'sdc'}), ('ip3:port3', 'tgt3'): (set(), set())} self.assertDictEqual(expected, res) iql_mock.assert_called_once_with(self.CON_PROPS, discover=False) @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices_with_iqns(self, nodes_mock, sessions_mock, glob_mock): ips_iqns_luns = self.connector._get_all_targets(self.CON_PROPS) # List sessions from other targets and non tcp sessions sessions_mock.return_value = [ ('non-tcp:', '0', 'ip1:port1', '1', 'tgt1'), ('tcp:', '1', 'ip1:port1', '1', 'tgt1'), ('tcp:', '2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', '3', 'ip1:port1', '1', 'tgt4'), ('tcp:', '4', 'ip2:port2', '-1', 'tgt5')] # List 1 node without sessions nodes_mock.return_value = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session1/target6/1:2:6:4/block/sda', sys_cls + '1/device/session1/target6/1:2:6:4/block/sda1'], [sys_cls + '2/device/session2/target7/2:2:7:5/block/sdb', sys_cls + '2/device/session2/target7/2:2:7:4/block/sdc'], ] with mock.patch.object(iscsi.ISCSIConnector, '_get_all_targets') as get_targets_mock: res = self.connector._get_connection_devices(mock.sentinel.props, ips_iqns_luns) expected = {('ip1:port1', 'tgt1'): ({'sda'}, set()), ('ip2:port2', 'tgt2'): ({'sdb'}, {'sdc'}), ('ip3:port3', 'tgt3'): (set(), set())} self.assertDictEqual(expected, res) get_targets_mock.assert_not_called() def generate_device(self, location, iqn, transport=None, lun=1): dev_format = "ip-%s-iscsi-%s-lun-%s" % (location, iqn, lun) if transport: dev_format = "pci-0000:00:00.0-" + dev_format fake_dev_path = "/dev/disk/by-path/" + dev_format return fake_dev_path def iscsi_connection(self, volume, location, iqn): return { 'driver_volume_type': 'iscsi', 'data': { 'volume_id': volume['id'], 'target_portal': location, 'target_iqn': iqn, 'target_lun': 1, } } def iscsi_connection_multipath(self, volume, locations, iqns, luns): return { 'driver_volume_type': 'iscsi', 'data': { 'volume_id': volume['id'], 'target_portals': locations, 'target_iqns': iqns, 'target_luns': luns, } } def iscsi_connection_chap(self, volume, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password): return { 'driver_volume_type': 'iscsi', 'data': { 'auth_method': auth_method, 'auth_username': auth_username, 'auth_password': auth_password, 'discovery_auth_method': discovery_auth_method, 'discovery_auth_username': discovery_auth_username, 'discovery_auth_password': discovery_auth_password, 'target_lun': 1, 'volume_id': volume['id'], 'target_iqn': iqn, 'target_portal': location, } } def _initiator_get_text(self, *arg, **kwargs): text = ('## DO NOT EDIT OR REMOVE THIS FILE!\n' '## If you remove this file, the iSCSI daemon ' 'will not start.\n' '## If you change the InitiatorName, existing ' 'access control lists\n' '## may reject this initiator. The InitiatorName must ' 'be unique\n' '## for each iSCSI initiator. Do NOT duplicate iSCSI ' 'InitiatorNames.\n' 'InitiatorName=%s' % self._fake_iqn) return text, None def test_get_initiator(self): def initiator_no_file(*args, **kwargs): raise putils.ProcessExecutionError('No file') self.connector._execute = initiator_no_file initiator = self.connector.get_initiator() self.assertIsNone(initiator) self.connector._execute = self._initiator_get_text initiator = self.connector.get_initiator() self.assertEqual(initiator, self._fake_iqn) def test_get_connector_properties(self): with mock.patch.object(priv_rootwrap, 'execute') as mock_exec: mock_exec.return_value = self._initiator_get_text() multipath = True enforce_multipath = True props = iscsi.ISCSIConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'initiator': self._fake_iqn} self.assertEqual(expected_props, props) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_brick_iscsi_validate_transport(self, mock_iscsiadm): sample_output = ('# BEGIN RECORD 2.0-872\n' 'iface.iscsi_ifacename = %s.fake_suffix\n' 'iface.net_ifacename = \n' 'iface.ipaddress = \n' 'iface.hwaddress = 00:53:00:00:53:00\n' 'iface.transport_name = %s\n' 'iface.initiatorname = \n' '# END RECORD') for tport in self.connector.supported_transports: mock_iscsiadm.return_value = (sample_output % (tport, tport), '') self.assertEqual(tport + '.fake_suffix', self.connector._validate_iface_transport( tport + '.fake_suffix')) mock_iscsiadm.return_value = ("", 'iscsiadm: Could not ' 'read iface fake_transport (6)') self.assertEqual('default', self.connector._validate_iface_transport( 'fake_transport')) def test_get_search_path(self): search_path = self.connector.get_search_path() expected = "/dev/disk/by-path" self.assertEqual(expected, search_path) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(iscsi.ISCSIConnector, '_get_potential_volume_paths') def test_get_volume_paths(self, mock_potential_paths, mock_exists): name1 = 'volume-00000001-1' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn = 'iqn.2010-10.org.openstack:%s' % name1 fake_path = ("/dev/disk/by-path/ip-%(ip)s-iscsi-%(iqn)s-lun-%(lun)s" % {'ip': '10.0.2.15', 'iqn': iqn, 'lun': 1}) fake_devices = [fake_path] expected = fake_devices mock_potential_paths.return_value = fake_devices connection_properties = self.iscsi_connection(vol, [location], [iqn]) volume_paths = self.connector.get_volume_paths( connection_properties['data']) self.assertEqual(expected, volume_paths) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') def test_discover_mpath_device(self, mock_multipath_device, mock_multipath_device_path): location1 = '10.0.2.15:3260' location2 = '[2001:db8::1]:3260' name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 fake_multipath_dev = '/dev/mapper/fake-multipath-dev' fake_raw_dev = '/dev/disk/by-path/fake-raw-lun' vol = {'id': 1, 'name': name1} connection_properties = self.iscsi_connection_multipath( vol, [location1, location2], [iqn1, iqn2], [1, 2]) mock_multipath_device_path.return_value = fake_multipath_dev mock_multipath_device.return_value = test_connector.FAKE_SCSI_WWN (result_path, result_mpath_id) = ( self.connector_with_multipath._discover_mpath_device( test_connector.FAKE_SCSI_WWN, connection_properties['data'], fake_raw_dev)) result = {'path': result_path, 'multipath_id': result_mpath_id} expected_result = {'path': fake_multipath_dev, 'multipath_id': test_connector.FAKE_SCSI_WWN} self.assertEqual(expected_result, result) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(os.path, 'realpath') def test_discover_mpath_device_by_realpath(self, mock_realpath, mock_multipath_device, mock_multipath_device_path): FAKE_SCSI_WWN = '1234567890' location1 = '10.0.2.15:3260' location2 = '[2001:db8::1]:3260' name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 fake_multipath_dev = None fake_raw_dev = '/dev/disk/by-path/fake-raw-lun' vol = {'id': 1, 'name': name1} connection_properties = self.iscsi_connection_multipath( vol, [location1, location2], [iqn1, iqn2], [1, 2]) mock_multipath_device_path.return_value = fake_multipath_dev mock_multipath_device.return_value = { 'device': '/dev/mapper/%s' % FAKE_SCSI_WWN} mock_realpath.return_value = '/dev/sdvc' (result_path, result_mpath_id) = ( self.connector_with_multipath._discover_mpath_device( FAKE_SCSI_WWN, connection_properties['data'], fake_raw_dev)) mock_multipath_device.assert_called_with('/dev/sdvc') result = {'path': result_path, 'multipath_id': result_mpath_id} expected_result = {'path': '/dev/mapper/%s' % FAKE_SCSI_WWN, 'multipath_id': FAKE_SCSI_WWN} self.assertEqual(expected_result, result) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_mp(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = True res = self.connector.connect_volume(self.CON_PROPS) self.assertEqual(con_mp_mock.return_value, res) con_single_mock.assert_not_called() con_mp_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_mp_failure(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = True con_mp_mock.side_effect = exception.BrickException self.assertRaises(exception.BrickException, self.connector.connect_volume, self.CON_PROPS) con_single_mock.assert_not_called() con_mp_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_called_once_with(self.CON_PROPS, force=True) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_sp(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = False res = self.connector.connect_volume(self.CON_PROPS) self.assertEqual(con_single_mock.return_value, res) con_mp_mock.assert_not_called() con_single_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_sp_failure(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = False con_single_mock.side_effect = exception.BrickException self.assertRaises(exception.BrickException, self.connector.connect_volume, self.CON_PROPS) con_mp_mock.assert_not_called() con_single_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_called_once_with(self.CON_PROPS, force=True) def test_discover_iscsi_portals(self): location = '10.0.2.15:3260' name = 'volume-00000001' iqn = 'iqn.2010-10.org.openstack:%s' % name vol = {'id': 1, 'name': name} auth_method = 'CHAP' auth_username = 'fake_chap_username' auth_password = 'fake_chap_password' discovery_auth_method = 'CHAP' discovery_auth_username = 'fake_chap_username' discovery_auth_password = 'fake_chap_password' connection_properties = self.iscsi_connection_chap( vol, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) for transport in ['default', 'iser', 'badTransport']: interface = 'iser' if transport == 'iser' else 'default' self.mock_object(self.connector_with_multipath, '_get_transport', mock.Mock(return_value=interface)) self.connector_with_multipath._discover_iscsi_portals( connection_properties['data']) expected_cmds = [ 'iscsiadm -m discoverydb -t sendtargets -I %(iface)s ' '-p %(location)s --op update ' '-n discovery.sendtargets.auth.authmethod -v %(auth_method)s ' '-n discovery.sendtargets.auth.username -v %(username)s ' '-n discovery.sendtargets.auth.password -v %(password)s' % {'iface': interface, 'location': location, 'auth_method': discovery_auth_method, 'username': discovery_auth_username, 'password': discovery_auth_password}, 'iscsiadm -m node --op show -p %s' % location, 'iscsiadm -m discoverydb -t sendtargets -I %(iface)s' ' -p %(location)s --discover' % {'iface': interface, 'location': location}, 'iscsiadm -m node --op show -p %s' % location] self.assertEqual(expected_cmds, self.cmds) # Reset to run with a different transport type self.cmds = list() @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_update_discoverydb') @mock.patch.object(os.path, 'exists', return_value=True) def test_iscsi_portals_with_chap_discovery( self, exists, update_discoverydb): location = '10.0.2.15:3260' name = 'volume-00000001' iqn = 'iqn.2010-10.org.openstack:%s' % name vol = {'id': 1, 'name': name} auth_method = 'CHAP' auth_username = 'fake_chap_username' auth_password = 'fake_chap_password' discovery_auth_method = 'CHAP' discovery_auth_username = 'fake_chap_username' discovery_auth_password = 'fake_chap_password' connection_properties = self.iscsi_connection_chap( vol, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) self.cmds = [] # The first call returns an error code = 6, mocking an empty # discovery db. The second one mocks a successful return and the # third one a dummy exit code, which will trigger the # TargetPortalNotFound exception in connect_volume update_discoverydb.side_effect = [ putils.ProcessExecutionError(None, None, 6), ("", ""), putils.ProcessExecutionError(None, None, 9)] self.connector_with_multipath._discover_iscsi_portals( connection_properties['data']) update_discoverydb.assert_called_with(connection_properties['data']) expected_cmds = [ 'iscsiadm -m discoverydb -t sendtargets -p %s -I default' ' --op new' % location, 'iscsiadm -m node --op show -p %s' % location, 'iscsiadm -m discoverydb -t sendtargets -I default -p %s' ' --discover' % location, 'iscsiadm -m node --op show -p %s' % location] self.assertEqual(expected_cmds, self.cmds) self.assertRaises(exception.TargetPortalNotFound, self.connector_with_multipath.connect_volume, connection_properties['data']) def test_get_target_portals_from_iscsiadm_output(self): connector = self.connector test_output = '''10.15.84.19:3260,1 iqn.1992-08.com.netapp:sn.33615311 10.15.85.19:3260,2 iqn.1992-08.com.netapp:sn.33615311 ''' res = connector._get_target_portals_from_iscsiadm_output(test_output) ips = ['10.15.84.19:3260', '10.15.85.19:3260'] iqns = ['iqn.1992-08.com.netapp:sn.33615311', 'iqn.1992-08.com.netapp:sn.33615311'] expected = (ips, iqns) self.assertEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') def test_disconnect_volume(self, cleanup_mock): res = self.connector.disconnect_volume(mock.sentinel.con_props, mock.sentinel.dev_info, mock.sentinel.Force, mock.sentinel.ignore_errors) self.assertEqual(cleanup_mock.return_value, res) cleanup_mock.assert_called_once_with( mock.sentinel.con_props, force=mock.sentinel.Force, ignore_errors=mock.sentinel.ignore_errors) @ddt.data(True, False) @mock.patch.object(iscsi.ISCSIConnector, '_get_transport') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals(self, is_iser, iscsiadm_mock, transport_mock): params = { 'iqn1': self.SINGLE_CON_PROPS['target_iqn'], 'iqn2': 'iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-2017.9ef', 'addr': self.SINGLE_CON_PROPS['target_portal'].replace(':', ','), 'ip1': self.SINGLE_CON_PROPS['target_portal'], 'ip2': '192.168.1.3:3260', 'transport': 'iser' if is_iser else 'default', 'other_transport': 'default' if is_iser else 'iser', } iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: %(addr)s\n' 'Target: %(iqn1)s\n' ' Portal: %(ip2)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(other_transport)s\n' 'Target: %(iqn2)s\n' ' Portal: %(ip2)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(transport)s\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n' % params, None) transport_mock.return_value = 'iser' if is_iser else 'non-iser' res = self.connector._get_discoverydb_portals(self.SINGLE_CON_PROPS) expected = [(params['ip2'], params['iqn1'], self.SINGLE_CON_PROPS['target_lun']), (params['ip1'], params['iqn1'], self.SINGLE_CON_PROPS['target_lun']), (params['ip2'], params['iqn2'], self.SINGLE_CON_PROPS['target_lun']), (params['ip1'], params['iqn2'], self.SINGLE_CON_PROPS['target_lun'])] self.assertListEqual(expected, res) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_get_transport', return_value='') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals_error(self, iscsiadm_mock, transport_mock): """DiscoveryAddress is not present.""" iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n', None) self.assertRaises(exception.TargetPortalsNotFound, self.connector._get_discoverydb_portals, self.SINGLE_CON_PROPS) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_transport', return_value='') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals_error_is_present(self, iscsiadm_mock, transport_mock): """DiscoveryAddress is present but wrong iterface.""" params = { 'iqn': self.SINGLE_CON_PROPS['target_iqn'], 'addr': self.SINGLE_CON_PROPS['target_portal'].replace(':', ','), 'ip': self.SINGLE_CON_PROPS['target_portal'], } iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: %(addr)s\n' 'Target: %(iqn)s\n' ' Portal: %(ip)s,1\n' ' Iface Name: iser\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n' % params, None) self.assertRaises(exception.TargetPortalsNotFound, self.connector._get_discoverydb_portals, self.SINGLE_CON_PROPS) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_connection') @mock.patch.object(iscsi.ISCSIConnector, '_get_connection_devices') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_connection', return_value=None) def test_cleanup_connection(self, remove_mock, flush_mock, con_devs_mock, discon_mock): # Return an ordered dicts instead of normal dict for discon_mock.assert con_devs_mock.return_value = collections.OrderedDict(( (('ip1:port1', 'tgt1'), ({'sda'}, set())), (('ip2:port2', 'tgt2'), ({'sdb'}, {'sdc'})), (('ip3:port3', 'tgt3'), (set(), set())))) with mock.patch.object(self.connector, 'use_multipath') as use_mp_mock: self.connector._cleanup_connection( self.CON_PROPS, ips_iqns_luns=mock.sentinel.ips_iqns_luns, force=False, ignore_errors=False) con_devs_mock.assert_called_once_with(self.CON_PROPS, mock.sentinel.ips_iqns_luns) remove_mock.assert_called_once_with({'sda', 'sdb'}, use_mp_mock, False, mock.ANY) discon_mock.assert_called_once_with( self.CON_PROPS, [('ip1:port1', 'tgt1'), ('ip3:port3', 'tgt3')], False, mock.ANY) flush_mock.assert_not_called() @mock.patch('os_brick.exception.ExceptionChainer.__nonzero__', mock.Mock(return_value=True)) @mock.patch('os_brick.exception.ExceptionChainer.__bool__', mock.Mock(return_value=True)) @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_connection') @mock.patch.object(iscsi.ISCSIConnector, '_get_connection_devices') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_connection', return_value=mock.sentinel.mp_name) def test_cleanup_connection_force_failure(self, remove_mock, flush_mock, con_devs_mock, discon_mock): # Return an ordered dicts instead of normal dict for discon_mock.assert con_devs_mock.return_value = collections.OrderedDict(( (('ip1:port1', 'tgt1'), ({'sda'}, set())), (('ip2:port2', 'tgt2'), ({'sdb'}, {'sdc'})), (('ip3:port3', 'tgt3'), (set(), set())))) with mock.patch.object(self.connector, 'use_multipath', wraps=True) as use_mp_mock: self.assertRaises(exception.ExceptionChainer, self.connector._cleanup_connection, self.CON_PROPS, ips_iqns_luns=mock.sentinel.ips_iqns_luns, force=mock.sentinel.force, ignore_errors=False) con_devs_mock.assert_called_once_with(self.CON_PROPS, mock.sentinel.ips_iqns_luns) remove_mock.assert_called_once_with({'sda', 'sdb'}, use_mp_mock, mock.sentinel.force, mock.ANY) discon_mock.assert_called_once_with( self.CON_PROPS, [('ip1:port1', 'tgt1'), ('ip3:port3', 'tgt3')], mock.sentinel.force, mock.ANY) flush_mock.assert_called_once_with(mock.sentinel.mp_name) def test_cleanup_connection_no_data_discoverydb(self): self.connector.use_multipath = True with mock.patch.object(self.connector, '_get_discoverydb_portals', side_effect=exception.TargetPortalsNotFound), \ mock.patch.object(self.connector._linuxscsi, 'remove_connection') as mock_remove: # This will not raise and exception self.connector._cleanup_connection(self.SINGLE_CON_PROPS) mock_remove.assert_not_called() @ddt.data({'do_raise': False, 'force': False}, {'do_raise': True, 'force': True}, {'do_raise': True, 'force': False}) @ddt.unpack @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_from_iscsi_portal') def test_disconnect_connection(self, disconnect_mock, do_raise, force): will_raise = do_raise and not force actual_call_args = [] # Since we reuse the copied dictionary on _disconnect_connection # changing its values we cannot use mock's assert_has_calls def my_disconnect(con_props): actual_call_args.append(con_props.copy()) if do_raise: raise exception.ExceptionChainer() disconnect_mock.side_effect = my_disconnect connections = (('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2')) original_props = self.CON_PROPS.copy() exc = exception.ExceptionChainer() if will_raise: self.assertRaises(exception.ExceptionChainer, self.connector._disconnect_connection, self.CON_PROPS, connections, force=force, exc=exc) else: self.connector._disconnect_connection(self.CON_PROPS, connections, force=force, exc=exc) # Passed properties should not be altered by the method call self.assertDictEqual(original_props, self.CON_PROPS) expected = [original_props.copy(), original_props.copy()] for i, (ip, iqn) in enumerate(connections): expected[i].update(target_portal=ip, target_iqn=iqn) # If we are failing and not forcing we won't make all the alls if will_raise: expected = expected[:1] self.assertListEqual(expected, actual_call_args) # No exceptions have been caught by ExceptionChainer context manager self.assertEqual(do_raise, bool(exc)) def test_disconnect_from_iscsi_portal(self): self.connector._disconnect_from_iscsi_portal(self.CON_PROPS) expected_prefix = ('iscsiadm -m node -T %s -p %s ' % (self.CON_PROPS['target_iqn'], self.CON_PROPS['target_portal'])) expected = [ expected_prefix + '--op update -n node.startup -v manual', expected_prefix + '--logout', expected_prefix + '--op delete', ] self.assertListEqual(expected, self.cmds) def test_iscsiadm_discover_parsing(self): # Ensure that parsing iscsiadm discover ignores cruft. ips = ["192.168.204.82:3260", "192.168.204.82:3261"] iqns = ["iqn.2010-10.org.openstack:volume-" "f9b12623-6ce3-4dac-a71f-09ad4249bdd3", "iqn.2010-10.org.openstack:volume-" "f9b12623-6ce3-4dac-a71f-09ad4249bdd4"] # This slight wonkiness brought to you by pep8, as the actual # example output runs about 97 chars wide. sample_input = """Loading iscsi modules: done Starting iSCSI initiator service: done Setting up iSCSI targets: unused %s %s %s %s """ % (ips[0] + ',1', iqns[0], ips[1] + ',1', iqns[1]) out = self.connector.\ _get_target_portals_from_iscsiadm_output(sample_input) self.assertEqual((ips, iqns), out) def test_sanitize_log_run_iscsiadm(self): # Tests that the parameters to the _run_iscsiadm function # are sanitized for when passwords are logged. def fake_debug(*args, **kwargs): self.assertIn('node.session.auth.password', args[0]) self.assertNotIn('scrubme', args[0]) volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") iscsi_properties = connection_info['data'] with mock.patch.object(iscsi.LOG, 'debug', side_effect=fake_debug) as debug_mock: self.connector._iscsiadm_update(iscsi_properties, 'node.session.auth.password', 'scrubme') # we don't care what the log message is, we just want to make sure # our stub method is called which asserts the password is scrubbed self.assertTrue(debug_mock.called) @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume(self, mock_volume_paths, mock_scsi_extend): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") new_size = self.connector.extend_volume(connection_info['data']) self.assertEqual(fake_new_size, new_size) @mock.patch.object(iscsi.LOG, 'info') @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_mask_password(self, mock_volume_paths, mock_scsi_extend, mock_log_info): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection_chap( volume, "10.0.2.15:3260", "fake_iqn", 'CHAP', 'fake_user', 'fake_password', 'CHAP1', 'fake_user1', 'fake_password1') self.connector.extend_volume(connection_info['data']) self.assertEqual(2, mock_log_info.call_count) self.assertIn("'auth_password': '***'", str(mock_log_info.call_args_list[0])) self.assertIn("'discovery_auth_password': '***'", str(mock_log_info.call_args_list[0])) @mock.patch.object(iscsi.LOG, 'warning') @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_mask_password_no_paths(self, mock_volume_paths, mock_scsi_extend, mock_log_warning): fake_new_size = 1024 mock_volume_paths.return_value = [] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection_chap( volume, "10.0.2.15:3260", "fake_iqn", 'CHAP', 'fake_user', 'fake_password', 'CHAP1', 'fake_user1', 'fake_password1') self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) self.assertEqual(1, mock_log_warning.call_count) self.assertIn("'auth_password': '***'", str(mock_log_warning.call_args_list[0])) self.assertIn("'discovery_auth_password': '***'", str(mock_log_warning.call_args_list[0])) @mock.patch.object(os.path, 'isdir') def test_get_all_available_volumes_path_not_dir(self, mock_isdir): mock_isdir.return_value = False expected = [] actual = self.connector.get_all_available_volumes() self.assertItemsEqual(expected, actual) @mock.patch.object(iscsi.ISCSIConnector, '_get_device_path') def test_get_potential_paths_mpath(self, get_path_mock): self.connector.use_multipath = True res = self.connector._get_potential_volume_paths(self.CON_PROPS) get_path_mock.assert_called_once_with(self.CON_PROPS) self.assertEqual(get_path_mock.return_value, res) self.assertEqual([], self.cmds) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions') @mock.patch.object(iscsi.ISCSIConnector, '_get_device_path') def test_get_potential_paths_single_path(self, get_path_mock, get_sessions_mock): get_path_mock.side_effect = [['path1'], ['path2'], ['path3', 'path4']] get_sessions_mock.return_value = [ 'ip1:port1', 'ip2:port2', 'ip3:port3'] self.connector.use_multipath = False res = self.connector._get_potential_volume_paths(self.CON_PROPS) self.assertEqual({'path1', 'path2', 'path3', 'path4'}, set(res)) get_sessions_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_with_target_iqns(self, discover_mock): res = self.connector._get_ips_iqns_luns(self.CON_PROPS) expected = list(self.connector._get_all_targets(self.CON_PROPS)) self.assertListEqual(expected, res) discover_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_discoverydb_portals') @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_discoverydb(self, discover_mock, db_portals_mock): db_portals_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS, discover=False) self.assertListEqual(db_portals_mock.return_value, res) db_portals_mock.assert_called_once_with(self.SINGLE_CON_PROPS) discover_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_no_target_iqns_share_iqn(self, discover_mock): discover_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip1:port1', 'tgt2', '1'), ('ip2:port2', 'tgt1', '2'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS) expected = {('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt1', '2')} self.assertEqual(expected, set(res)) @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_no_target_iqns_diff_iqn(self, discover_mock): discover_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS) self.assertEqual(discover_mock.return_value, res) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_all_new(self, get_sessions_mock): """Connect creating node and session.""" session = 'session2' get_sessions_mock.side_effect = [ [('tcp:', 'session1', 'ip1:port1', '1', 'tgt')], [('tcp:', 'session1', 'ip1:port1', '1', 'tgt'), ('tcp:', session, 'ip1:port1', '-1', 'tgt1')] ] with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', 'error'), ('', None), ('', None), ('', None), ('', None)] res = self.connector._connect_to_iscsi_portal(self.CON_PROPS) # True refers to "manual scans", since the call to update # node.session.scan didn't fail they are set to manual self.assertEqual((session, True), res) prefix = 'iscsiadm -m node -T tgt1 -p ip1:port1' expected_cmds = [ prefix, prefix + ' --interface default --op new', prefix + ' --op update -n node.session.scan -v manual', prefix + ' --login', prefix + ' --op update -n node.startup -v automatic' ] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) self.assertEqual(2, get_sessions_mock.call_count) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_all_exists_chap(self, get_sessions_mock): """Node and session already exists and we use chap authentication.""" session = 'session2' get_sessions_mock.return_value = [('tcp:', session, 'ip1:port1', '-1', 'tgt1')] con_props = self.CON_PROPS.copy() con_props.update(auth_method='CHAP', auth_username='user', auth_password='pwd') res = self.connector._connect_to_iscsi_portal(con_props) # False refers to "manual scans", so we have manual iscsi scans self.assertEqual((session, True), res) prefix = 'iscsiadm -m node -T tgt1 -p ip1:port1' expected_cmds = [ prefix, prefix + ' --op update -n node.session.scan -v manual', prefix + ' --op update -n node.session.auth.authmethod -v CHAP', prefix + ' --op update -n node.session.auth.username -v user', prefix + ' --op update -n node.session.auth.password -v pwd', ] self.assertListEqual(expected_cmds, self.cmds) get_sessions_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_fail_login(self, get_sessions_mock): get_sessions_mock.return_value = [] with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', None), ('', None), putils.ProcessExecutionError] res = self.connector._connect_to_iscsi_portal(self.CON_PROPS) self.assertEqual((None, None), res) expected_cmds = ['iscsiadm -m node -T tgt1 -p ip1:port1', 'iscsiadm -m node -T tgt1 -p ip1:port1 ' '--op update -n node.session.scan -v manual', 'iscsiadm -m node -T tgt1 -p ip1:port1 --login'] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) get_sessions_mock.assert_called_once_with() @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', side_effect=(None, 'tgt2')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch('time.sleep') def test_connect_single_volume(self, sleep_mock, cleanup_mock, connect_mock, get_wwn_mock): def my_connect(rescans, props, data): if props['target_iqn'] == 'tgt2': # Succeed on second call data['found_devices'].append('sdz') connect_mock.side_effect = my_connect res = self.connector._connect_single_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'tgt2', 'path': '/dev/sdz'} self.assertEqual(expected, res) get_wwn_mock.assert_has_calls([mock.call(['sdz']), mock.call(['sdz'])]) sleep_mock.assert_called_once_with(1) cleanup_mock.assert_called_once_with( {'target_lun': 4, 'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1'}, (('ip1:port1', 'tgt1', 4),), force=True, ignore_errors=True) @staticmethod def _get_connect_vol_data(): return {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': [], 'just_added_devices': []} @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', side_effect=(None, 'tgt2')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch('time.sleep') def test_connect_single_volume_not_found(self, sleep_mock, cleanup_mock, connect_mock, get_wwn_mock): self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_single_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() # Called twice by the retry mechanism self.assertEqual(2, sleep_mock.call_count) props = list(self.connector._get_all_targets(self.CON_PROPS)) calls_per_try = [ mock.call({'target_portal': prop[0], 'target_iqn': prop[1], 'target_lun': prop[2], 'volume_id': 'vol_id'}, (prop,), force=True, ignore_errors=True) for prop in props ] cleanup_mock.assert_has_calls(calls_per_try * 3) data = self._get_connect_vol_data() calls_per_try = [mock.call(self.connector.device_scan_attempts, {'target_portal': prop[0], 'target_iqn': prop[1], 'target_lun': prop[2], 'volume_id': 'vol_id'}, data) for prop in props] connect_mock.assert_has_calls(calls_per_try * 3) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_all_succeed(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': 'sda', 'tgt2': 'sdb', 'tgt3': 'sdc', 'tgt4': 'sdd'} data['stopped_threads'] += 1 data['num_logins'] += 1 dev = devs[props['target_iqn']] data['found_devices'].append(dev) data['just_added_devices'].append(dev) connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'multipath_id': 'wwn', 'path': '/dev/dm-0'} self.assertEqual(expected, res) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sda', 'sdb', 'sdc', 'sdd'], result) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 2) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_all_fail(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): data['stopped_threads'] += 1 data['failed_logins'] += 1 connect_mock.side_effect = my_connect self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_multipath_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() add_wwid_mock.assert_not_called() add_path_mock.assert_not_called() find_dm_mock.assert_not_called() self.assertEqual(4 * 3, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_some_fail_mp_found(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': '', 'tgt2': 'sdb', 'tgt3': '', 'tgt4': 'sdd'} data['stopped_threads'] += 1 dev = devs[props['target_iqn']] if dev: data['num_logins'] += 1 data['found_devices'].append(dev) data['just_added_devices'].append(dev) else: data['failed_logins'] += 1 connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'multipath_id': 'wwn', 'path': '/dev/dm-0'} self.assertEqual(expected, res) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sdb', 'sdd'], result) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 2) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', return_value=None) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.time, 'time', side_effect=(0, 0, 11, 0)) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_some_fail_mp_not_found(self, sleep_mock, connect_mock, time_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': '', 'tgt2': 'sdb', 'tgt3': '', 'tgt4': 'sdd'} data['stopped_threads'] += 1 dev = devs[props['target_iqn']] if dev: data['num_logins'] += 1 data['found_devices'].append(dev) data['just_added_devices'].append(dev) else: data['failed_logins'] += 1 connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = [{'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sdb'}, {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sdd'}] # It can only be one of the 2 self.assertIn(res, expected) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sdb', 'sdd'], result) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 4) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', return_value=None) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.time, 'time', side_effect=(0, 0, 11, 0)) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep', mock.Mock()) def test_connect_multipath_volume_all_loging_not_found(self, connect_mock, time_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): data['stopped_threads'] += 1 data['num_logins'] += 1 connect_mock.side_effect = my_connect self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_multipath_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() add_wwid_mock.assert_not_called() add_path_mock.assert_not_called() find_dm_mock.assert_not_called() self.assertEqual(12, connect_mock.call_count) @mock.patch('time.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', return_value='sda') @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol(self, connect_mock, dev_name_mock, scan_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] connect_mock.return_value = (mock.sentinel.session, False) with mock.patch.object(lscsi, 'get_hctl', side_effect=(None, hctl)) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, found_devices=['sda'], just_added_devices=['sda']) self.assertDictEqual(expected, data) connect_mock.assert_called_once_with(self.CON_PROPS) hctl_mock.assert_has_calls([mock.call(mock.sentinel.session, self.CON_PROPS['target_lun']), mock.call(mock.sentinel.session, self.CON_PROPS['target_lun'])]) scan_mock.assert_called_once_with(*hctl) dev_name_mock.assert_called_once_with(mock.sentinel.session, hctl) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal', return_value=(None, False)) def test_connect_vol_no_session(self, connect_mock): data = self._get_connect_vol_data() self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(failed_logins=1, stopped_threads=1) self.assertDictEqual(expected, data) @mock.patch('time.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', return_value=None) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_not_found(self, connect_mock, dev_name_mock, scan_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] # True because we are simulating we have manual scans connect_mock.return_value = (mock.sentinel.session, True) with mock.patch.object(lscsi, 'get_hctl', side_effect=(hctl,)) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1) self.assertDictEqual(expected, data) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) # We have 3 scans because on manual mode we also scan on connect scan_mock.assert_has_calls([mock.call(*hctl)] * 3) dev_name_mock.assert_has_calls( [mock.call(mock.sentinel.session, hctl), mock.call(mock.sentinel.session, hctl)]) @mock.patch('time.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_stop_connecting(self, connect_mock, scan_mock): data = self._get_connect_vol_data() def device_name_by_hctl(session, hctl): data['stop_connecting'] = True return None lscsi = self.connector._linuxscsi hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] connect_mock.return_value = (mock.sentinel.session, False) with mock.patch.object(lscsi, 'get_hctl', return_value=hctl) as hctl_mock, \ mock.patch.object( lscsi, 'device_name_by_hctl', side_effect=device_name_by_hctl) as dev_name_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, stop_connecting=True) self.assertDictEqual(expected, data) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) scan_mock.assert_not_called() dev_name_mock.assert_called_once_with(mock.sentinel.session, hctl) @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link') def test__get_connect_result(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = False res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb']) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sda'} self.assertDictEqual(expected, res) get_link_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link') def test__get_connect_result_mpath(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = False res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb'], 'mpath') expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/mpath', 'multipath_id': 'wwn'} self.assertDictEqual(expected, res) get_link_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link', return_value='/dev/disk/by-id/scsi-wwn') def test__get_connect_result_encrypted(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = True res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb']) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': get_link_mock.return_value} self.assertDictEqual(expected, res) get_link_mock.assert_called_once_with('wwn', '/dev/sda', None) @mock.patch('os.path.realpath', return_value='/dev/sda') def test__get_device_link(self, realpath_mock): symlink = '/dev/disk/by-id/scsi-wwn' res = self.connector._get_device_link('wwn', '/dev/sda', None) self.assertEqual(symlink, res) realpath_mock.assert_called_once_with(symlink) @mock.patch('os.path.realpath', return_value='/dev/dm-0') def test__get_device_link_multipath(self, realpath_mock): symlink = '/dev/disk/by-id/dm-uuid-mpath-wwn' res = self.connector._get_device_link('wwn', '/dev/dm-0', 'wwn') self.assertEqual(symlink, res) realpath_mock.assert_called_once_with(symlink) @mock.patch('os.path.realpath', side_effect=('/dev/sdz', '/dev/sdy', '/dev/sda', '/dev/sdx')) @mock.patch('os.listdir', return_value=['dm-...', 'scsi-wwn', 'scsi-...']) def test__get_device_link_check_links(self, listdir_mock, realpath_mock): res = self.connector._get_device_link('wwn', '/dev/sda', None) self.assertEqual(res, '/dev/disk/by-id/scsi-wwn') listdir_mock.assert_called_once_with('/dev/disk/by-id/') realpath_mock.assert_has_calls([ mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-wwn')]) @mock.patch('os.path.realpath', return_value='/dev/sdz') @mock.patch('os.listdir', return_value=['dm-...', 'scsi-...']) def test__get_device_link_not_found(self, listdir_mock, realpath_mock): self.assertRaises(exception.VolumeDeviceNotFound, self.connector._get_device_link, 'wwn', '/dev/sda', None) listdir_mock.assert_called_once_with('/dev/disk/by-id/') realpath_mock.assert_has_calls([ mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-...')]) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_node_startup_values(self, run_iscsiadm_bare_mock): name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' name3 = 'volume-00000001-3' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 iqn3 = 'iqn.2010-10.org.openstack:%s' % name3 connection_properties = self.iscsi_connection(vol, [location], [iqn1]) node_startup1 = "manual" node_startup2 = "automatic" node_startup3 = "manual" node_values = ( '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n' '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n' '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n') % (iqn1, node_startup1, iqn2, node_startup2, iqn3, node_startup3) run_iscsiadm_bare_mock.return_value = (node_values, None) node_startups =\ self.connector._get_node_startup_values( connection_properties['data']) expected_node_startups = {iqn1: node_startup1, iqn2: node_startup2, iqn3: node_startup3} self.assertEqual(node_startups, expected_node_startups) @mock.patch.object(iscsi.ISCSIConnector, '_get_node_startup_values') @mock.patch.object(iscsi.ISCSIConnector, '_iscsiadm_update') def test_recover_node_startup_values(self, iscsiadm_update_mock, get_node_startup_values_mock): name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' name3 = 'volume-00000001-3' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 iqn3 = 'iqn.2010-10.org.openstack:%s' % name3 connection_properties = self.iscsi_connection(vol, [location], [iqn1]) recover_connection = self.iscsi_connection(vol, [location], [iqn2]) node_startup1 = "manual" node_startup2 = "automatic" node_startup3 = "manual" get_node_startup_values_mock.return_value = {iqn1: node_startup1, iqn2: node_startup2, iqn3: node_startup3} old_node_startup_values = {iqn1: node_startup1, iqn2: "manual", iqn3: node_startup3} self.connector._recover_node_startup_values( connection_properties['data'], old_node_startup_values) iscsiadm_update_mock.assert_called_once_with( recover_connection['data'], "node.startup", "manual") os-brick-2.3.0/os_brick/tests/initiator/connectors/__init__.py0000666000175100017510000000000013230233223024457 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/initiator/connectors/test_fibre_channel_s390x.py0000666000175100017510000000673013230233223027524 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import fibre_channel_s390x from os_brick.initiator import linuxfc from os_brick.tests.initiator import test_connector class FibreChannelConnectorS390XTestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorS390XTestCase, self).setUp() self.connector = fibre_channel_s390x.FibreChannelConnectorS390X( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertEqual(self.connector._linuxfc.__class__.__name__, "LinuxFibreChannelS390X") self.assertIsNotNone(self.connector._linuxscsi) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'configure_scsi_device') def test_get_host_devices(self, mock_configure_scsi_device): lun = 2 possible_devs = [(3, 5), ] devices = self.connector._get_host_devices(possible_devs, lun) mock_configure_scsi_device.assert_called_with(3, 5, "0x0002000000000000") self.assertEqual(2, len(devices)) device_path = "/dev/disk/by-path/ccw-3-zfcp-5:0x0002000000000000" self.assertEqual(devices[0], device_path) device_path = "/dev/disk/by-path/ccw-3-fc-5-lun-2" self.assertEqual(devices[1], device_path) def test_get_lun_string(self): lun = 1 lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x0001000000000000") lun = 0xff lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x00ff000000000000") lun = 0x101 lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x0101000000000000") lun = 0x4020400a lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x4020400a00000000") @mock.patch.object(fibre_channel_s390x.FibreChannelConnectorS390X, '_get_possible_devices', return_value=[(3, 5), ]) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'get_fc_hbas_info', return_value=[]) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'deconfigure_scsi_device') def test_remove_devices(self, mock_deconfigure_scsi_device, mock_get_fc_hbas_info, mock_get_possible_devices): connection_properties = {'target_wwn': 5, 'target_lun': 2} self.connector._remove_devices(connection_properties, devices=None) mock_deconfigure_scsi_device.assert_called_with(3, 5, "0x0002000000000000") mock_get_fc_hbas_info.assert_called_once_with() mock_get_possible_devices.assert_called_once_with([], 5) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_iser.py0000666000175100017510000000642513230233223024742 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import iscsi from os_brick.tests.initiator import test_connector class ISERConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(ISERConnectorTestCase, self).setUp() self.connector = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=False) self.connection_data = { 'volume_id': 'volume_id', 'target_portal': 'ip:port', 'target_iqn': 'target_1', 'target_lun': 1, 'target_portals': ['ip:port'], 'target_iqns': ['target_1'], 'target_luns': [1] } @mock.patch.object(iscsi.ISCSIConnector, '_get_ips_iqns_luns') @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices( self, nodes_mock, sessions_mock, glob_mock, iql_mock): self.connector.use_multipath = True iql_mock.return_value = \ self.connector._get_all_targets(self.connection_data) # mocked iSCSI sessions sessions_mock.return_value = \ [('iser:', '0', 'ip:port', '1', 'target_1')] # mocked iSCSI nodes nodes_mock.return_value = [('ip:port', 'target_1')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session/target/1:1:1:1/block/sda'] ] res = self.connector._get_connection_devices(self.connection_data) expected = {('ip:port', 'target_1'): ({'sda'}, set())} self.assertDictEqual(expected, res) iql_mock.assert_called_once_with(self.connection_data, discover=False) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_connect_to_iscsi_portal(self, exec_mock, sessions_mock): """Connect to portal while session already established""" # connected sessions sessions_mock.side_effect = [ [('iser:', 'session_iser', 'ip:port', '1', 'target_1')] ] exec_mock.side_effect = [('', None), ('', None), ('', None)] res = self.connector._connect_to_iscsi_portal(self.connection_data) # session name is expected to be in the result. self.assertEqual(("session_iser", True), res) prefix = 'iscsiadm -m node -T target_1 -p ip:port' expected_cmds = [ prefix, prefix + ' --op update -n node.session.scan -v manual' ] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_disco.py0000666000175100017510000001266613230233223025105 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os from os_brick import exception from os_brick.initiator.connectors import disco from os_brick.tests.initiator import test_connector class DISCOConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for DISCO connector.""" # Fake volume information volume = { 'name': 'a-disco-volume', 'disco_id': '1234567' } # Conf for test conf = { 'ip': test_connector.MY_IP, 'port': 9898 } def setUp(self): super(DISCOConnectorTestCase, self).setUp() self.fake_connection_properties = { 'name': self.volume['name'], 'disco_id': self.volume['disco_id'], 'conf': { 'server_ip': self.conf['ip'], 'server_port': self.conf['port']} } self.fake_volume_status = {'attached': True, 'detached': False} self.fake_request_status = {'success': None, 'fail': 'ERROR'} self.volume_status = 'detached' self.request_status = 'success' # Patch the request and os calls to fake versions self.mock_object(disco.DISCOConnector, '_send_disco_vol_cmd', self.perform_disco_request) self.mock_object(os.path, 'exists', self.is_volume_attached) self.mock_object(glob, 'glob', self.list_disco_volume) # The actual DISCO connector self.connector = disco.DISCOConnector( 'sudo', execute=self.fake_execute) def perform_disco_request(self, *cmd, **kwargs): """Fake the socket call.""" return self.fake_request_status[self.request_status] def is_volume_attached(self, *cmd, **kwargs): """Fake volume detection check.""" return self.fake_volume_status[self.volume_status] def list_disco_volume(self, *cmd, **kwargs): """Fake the glob call.""" path_dir = self.connector.get_search_path() volume_id = self.volume['disco_id'] volume_items = [path_dir, '/', self.connector.DISCO_PREFIX, volume_id] volume_path = ''.join(volume_items) return [volume_path] def test_get_connector_properties(self): props = disco.DISCOConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): """DISCO volumes should be under /dev.""" expected = "/dev" actual = self.connector.get_search_path() self.assertEqual(expected, actual) def test_get_volume_paths(self): """Test to get all the path for a specific volume.""" expected = ['/dev/dms1234567'] self.volume_status = 'attached' actual = self.connector.get_volume_paths( self.fake_connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): """Attach a volume.""" self.connector.connect_volume(self.fake_connection_properties) def test_connect_volume_already_attached(self): """Make sure that we don't issue the request.""" self.request_status = 'fail' self.volume_status = 'attached' self.test_connect_volume() def test_connect_volume_request_fail(self): """Fail the attach request.""" self.volume_status = 'detached' self.request_status = 'fail' self.assertRaises(exception.BrickException, self.test_connect_volume) def test_disconnect_volume(self): """Detach a volume.""" self.connector.disconnect_volume(self.fake_connection_properties, None) def test_disconnect_volume_attached(self): """Detach a volume attached.""" self.request_status = 'success' self.volume_status = 'attached' self.test_disconnect_volume() def test_disconnect_volume_already_detached(self): """Ensure that we don't issue the request.""" self.request_status = 'fail' self.volume_status = 'detached' self.test_disconnect_volume() def test_disconnect_volume_request_fail(self): """Fail the detach request.""" self.volume_status = 'attached' self.request_status = 'fail' self.assertRaises(exception.BrickException, self.test_disconnect_volume) def test_get_all_available_volumes(self): """Test to get all the available DISCO volumes.""" expected = ['/dev/dms1234567'] actual = self.connector.get_all_available_volumes(None) self.assertItemsEqual(expected, actual) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.fake_connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_local.py0000666000175100017510000000435413230233223025071 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import local from os_brick.tests.initiator import test_connector class LocalConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(LocalConnectorTestCase, self).setUp() self.connection_properties = {'name': 'foo', 'device_path': '/tmp/bar'} self.connector = local.LocalConnector(None) def test_get_connector_properties(self): props = local.LocalConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): actual = self.connector.get_search_path() self.assertIsNone(actual) def test_get_volume_paths(self): expected = [self.connection_properties['device_path']] actual = self.connector.get_volume_paths( self.connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): cprops = self.connection_properties dev_info = self.connector.connect_volume(cprops) self.assertEqual(dev_info['type'], 'local') self.assertEqual(dev_info['path'], cprops['device_path']) def test_connect_volume_with_invalid_connection_data(self): cprops = {} self.assertRaises(ValueError, self.connector.connect_volume, cprops) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_remotefs.py0000666000175100017510000000635513230233223025626 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import remotefs from os_brick.remotefs import remotefs as remotefs_client from os_brick.tests.initiator import test_connector class RemoteFsConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for Remote FS initiator class.""" TEST_DEV = '172.18.194.100:/var/nfs' TEST_PATH = '/mnt/test/df0808229363aad55c27da50c38d6328' TEST_BASE = '/mnt/test' TEST_NAME = '9c592d52-ce47-4263-8c21-4ecf3c029cdb' def setUp(self): super(RemoteFsConnectorTestCase, self).setUp() self.connection_properties = { 'export': self.TEST_DEV, 'name': self.TEST_NAME} self.connector = remotefs.RemoteFsConnector( 'nfs', root_helper='sudo', nfs_mount_point_base=self.TEST_BASE, nfs_mount_options='vers=3') @mock.patch('os_brick.remotefs.remotefs.ScalityRemoteFsClient') def test_init_with_scality(self, mock_scality_remotefs_client): remotefs.RemoteFsConnector('scality', root_helper='sudo') self.assertEqual(1, mock_scality_remotefs_client.call_count) def test_get_connector_properties(self): props = remotefs.RemoteFsConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): expected = self.TEST_BASE actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(remotefs_client.RemoteFsClient, 'mount') def test_get_volume_paths(self, mock_mount): path = ("%(path)s/%(name)s" % {'path': self.TEST_PATH, 'name': self.TEST_NAME}) expected = [path] actual = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) @mock.patch.object(remotefs_client.RemoteFsClient, 'mount') @mock.patch.object(remotefs_client.RemoteFsClient, 'get_mount_point', return_value="something") def test_connect_volume(self, mount_point_mock, mount_mock): """Test the basic connect volume case.""" self.connector.connect_volume(self.connection_properties) def test_disconnect_volume(self): """Nothing should happen here -- make sure it doesn't blow up.""" self.connector.disconnect_volume(self.connection_properties, {}) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_nvme.py0000666000175100017510000001632613230233223024746 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator.connectors import nvme from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector FAKE_NVME_LIST_OUTPUT = """ Node SN Model \ Namespace Usage Format FW Rev\n ---------------- -------------------- ---------------------------------------\ - --------- -------------------------- ---------------- --------\n /dev/nvme0n1 67ff9467da6e5567 Linux \ 10 1.07 GB / 1.07 GB 512 B + 0 B 4.8.0-58\n """ class NVMeConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for NVMe initiator class.""" def setUp(self): super(NVMeConnectorTestCase, self).setUp() self.connector = nvme.NVMeConnector(None, execute=self.fake_execute) def _nvme_list_cmd(self, *args, **kwargs): return FAKE_NVME_LIST_OUTPUT, None def test__get_nvme_devices(self): expected = ['/dev/nvme0n1'] self.connector._execute = self._nvme_list_cmd actual = self.connector._get_nvme_devices() self.assertEqual(expected, actual) @mock.patch.object(nvme.NVMeConnector, '_execute') def test_get_nvme_devices_raise(self, mock_execute): mock_execute.side_effect = putils.ProcessExecutionError self.assertRaises(putils.ProcessExecutionError, self.connector._get_nvme_devices) @mock.patch.object(nvme.NVMeConnector, '_get_nvme_devices') @mock.patch.object(nvme.NVMeConnector, '_execute') @mock.patch('time.sleep') def test_connect_volume(self, mock_sleep, mock_execute, mock_devices): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.side_effect = [ ['/dev/nvme0n1'], ['/dev/nvme0n2']] device_info = self.connector.connect_volume( connection_properties) self.assertEqual('/dev/nvme0n2', device_info['path']) self.assertEqual('block', device_info['type']) self.assertEqual(2, mock_devices.call_count) mock_execute.assert_called_once_with( 'nvme', 'connect', '-t', connection_properties['transport_type'], '-n', 'volume_123', '-a', connection_properties['target_portal'], '-s', connection_properties['target_port'], root_helper=None, run_as_root=True) @mock.patch.object(nvme.NVMeConnector, '_execute') def test_connect_volume_raise(self, mock_execute): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_execute.side_effect = putils.ProcessExecutionError self.assertRaises(putils.ProcessExecutionError, self.connector.connect_volume, connection_properties) @mock.patch.object(nvme.NVMeConnector, '_get_nvme_devices') @mock.patch.object(nvme.NVMeConnector, '_execute') @mock.patch('time.sleep') def test_connect_volume_max_retry( self, mock_sleep, mock_execute, mock_devices): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.return_value = '/dev/nvme0n1' self.assertRaises(exception.TargetPortalNotFound, self.connector.connect_volume, connection_properties) @mock.patch.object(nvme.NVMeConnector, '_execute') def test_disconnect_volume(self, mock_execute): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} self.connector.disconnect_volume(connection_properties, None) mock_execute.asert_called_once_with( 'nvme', 'disconnect', '-n', 'volume_123', root_helper=None, run_as_root=True) @mock.patch.object(nvme.NVMeConnector, '_execute') def test_disconnect_volume_raise(self, mock_execute): mock_execute.side_effect = putils.ProcessExecutionError connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} self.assertRaises(putils.ProcessExecutionError, self.connector.disconnect_volume, connection_properties, None) @mock.patch.object(nvme.NVMeConnector, 'get_volume_paths') def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_properties) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(nvme.NVMeConnector, 'get_volume_paths') def test_extend_volume(self, mock_volume_paths, mock_scsi_extend): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} new_size = self.connector.extend_volume(connection_properties) self.assertEqual(fake_new_size, new_size) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_drbd.py0000666000175100017510000000520013230233223024701 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import drbd from os_brick.tests.initiator import test_connector class DRBDConnectorTestCase(test_connector.ConnectorTestCase): RESOURCE_TEMPLATE = ''' resource r0 { on host1 { } net { shared-secret "%(shared-secret)s"; } } ''' def setUp(self): super(DRBDConnectorTestCase, self).setUp() self.connector = drbd.DRBDConnector( None, execute=self._fake_exec) self.execs = [] def _fake_exec(self, *cmd, **kwargs): self.execs.append(cmd) # out, err return ('', '') def test_get_connector_properties(self): props = drbd.DRBDConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume(self): """Test connect_volume.""" cprop = { 'provider_auth': 'my-secret', 'config': self.RESOURCE_TEMPLATE, 'name': 'my-precious', 'device': '/dev/drbd951722', 'data': {}, } res = self.connector.connect_volume(cprop) self.assertEqual(cprop['device'], res['path']) self.assertEqual('adjust', self.execs[0][1]) self.assertEqual(cprop['name'], self.execs[0][4]) def test_disconnect_volume(self): """Test the disconnect volume case.""" cprop = { 'provider_auth': 'my-secret', 'config': self.RESOURCE_TEMPLATE, 'name': 'my-precious', 'device': '/dev/drbd951722', 'data': {}, } dev_info = {} self.connector.disconnect_volume(cprop, dev_info) self.assertEqual('down', self.execs[0][1]) def test_extend_volume(self): cprop = {'name': 'something'} self.assertRaises(NotImplementedError, self.connector.extend_volume, cprop) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_rbd.py0000666000175100017510000002745013230233223024550 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import exception from os_brick.initiator.connectors import rbd from os_brick.initiator import linuxrbd from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests.initiator import test_connector from os_brick import utils @ddt.ddt class RBDConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(RBDConnectorTestCase, self).setUp() self.user = 'fake_user' self.pool = 'fake_pool' self.volume = 'fake_volume' self.clustername = 'fake_ceph' self.hosts = ['192.168.10.2'] self.ports = ['6789'] self.keyring = "[client.cinder]\n key = test\n" self.connection_properties = { 'auth_username': self.user, 'name': '%s/%s' % (self.pool, self.volume), 'cluster_name': self.clustername, 'hosts': self.hosts, 'ports': self.ports, 'keyring': self.keyring, } def test_get_search_path(self): rbd_connector = rbd.RBDConnector(None) path = rbd_connector.get_search_path() self.assertIsNone(path) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') def test_get_volume_paths(self, mock_rados, mock_rbd): rbd_connector = rbd.RBDConnector(None) expected = [] actual = rbd_connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) def test_get_connector_properties(self): props = rbd.RBDConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {'do_local_attach': False} self.assertEqual(expected_props, props) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(rbd.RBDConnector, '_create_ceph_conf') @mock.patch('os.path.exists') def test_connect_volume(self, mock_path, mock_conf, mock_rados, mock_rbd): """Test the connect volume case.""" rbd_connector = rbd.RBDConnector(None) mock_path.return_value = False mock_conf.return_value = "/tmp/fake_dir/fake_ceph.conf" device_info = rbd_connector.connect_volume(self.connection_properties) # Ensure rados is instantiated correctly mock_rados.Rados.assert_called_once_with( clustername=self.clustername, rados_id=utils.convert_str(self.user), conffile='/tmp/fake_dir/fake_ceph.conf') # Ensure correct calls to connect to cluster self.assertEqual(1, mock_rados.Rados.return_value.connect.call_count) mock_rados.Rados.return_value.open_ioctx.assert_called_once_with( utils.convert_str(self.pool)) # Ensure rbd image is instantiated correctly mock_rbd.Image.assert_called_once_with( mock_rados.Rados.return_value.open_ioctx.return_value, utils.convert_str(self.volume), read_only=False, snapshot=None) # Ensure expected object is returned correctly self.assertIsInstance(device_info['path'], linuxrbd.RBDVolumeIOWrapper) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(rbd.RBDConnector, '_create_ceph_conf') @mock.patch('os.path.exists') def test_provided_keyring(self, mock_path, mock_conf, mock_rados, mock_rbd): conn = rbd.RBDConnector(None) mock_path.return_value = False mock_conf.return_value = "/tmp/fake_dir/fake_ceph.conf" self.connection_properties['keyring'] = self.keyring conn.connect_volume(self.connection_properties) mock_conf.assert_called_once_with(self.hosts, self.ports, self.clustername, self.user, self.keyring) def test_keyring_is_none(self): conn = rbd.RBDConnector(None) keyring = None keyring_data = "[client.cinder]\n key = test\n" mockopen = mock.mock_open(read_data=keyring_data) mockopen.return_value.__exit__ = mock.Mock() with mock.patch('os_brick.initiator.connectors.rbd.open', mockopen, create=True): self.assertEqual( conn._check_or_get_keyring_contents(keyring, 'cluster', 'user'), keyring_data) def test_keyring_raise_error(self): conn = rbd.RBDConnector(None) keyring = None mockopen = mock.mock_open() mockopen.return_value = "" with mock.patch('os_brick.initiator.connectors.rbd.open', mockopen, create=True) as mock_keyring_file: mock_keyring_file.side_effect = IOError self.assertRaises(exception.BrickException, conn._check_or_get_keyring_contents, keyring, 'cluster', 'user') @ddt.data((['192.168.1.1', '192.168.1.2'], ['192.168.1.1', '192.168.1.2']), (['3ffe:1900:4545:3:200:f8ff:fe21:67cf', 'fe80:0:0:0:200:f8ff:fe21:67cf'], ['[3ffe:1900:4545:3:200:f8ff:fe21:67cf]', '[fe80:0:0:0:200:f8ff:fe21:67cf]']), (['foobar', 'fizzbuzz'], ['foobar', 'fizzbuzz']), (['192.168.1.1', '3ffe:1900:4545:3:200:f8ff:fe21:67cf', 'hello, world!'], ['192.168.1.1', '[3ffe:1900:4545:3:200:f8ff:fe21:67cf]', 'hello, world!'])) @ddt.unpack def test_sanitize_mon_host(self, hosts_in, hosts_out): conn = rbd.RBDConnector(None) self.assertEqual(hosts_out, conn._sanitize_mon_hosts(hosts_in)) @mock.patch('os_brick.initiator.connectors.rbd.tempfile.mkstemp') def test_create_ceph_conf(self, mock_mkstemp): mockopen = mock.mock_open() fd = mock.sentinel.fd tmpfile = mock.sentinel.tmpfile mock_mkstemp.return_value = (fd, tmpfile) with mock.patch('os.fdopen', mockopen, create=True): rbd_connector = rbd.RBDConnector(None) conf_path = rbd_connector._create_ceph_conf( self.hosts, self.ports, self.clustername, self.user, self.keyring) self.assertEqual(conf_path, tmpfile) mock_mkstemp.assert_called_once_with(prefix='brickrbd_') @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1, execute_call2]) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) @mock.patch('os.path.exists') @mock.patch('os.path.islink') @mock.patch('os.path.realpath') def test_connect_local_volume_dev_exist(self, mock_realpath, mock_islink, mock_exists, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} mock_realpath.return_value = '/dev/rbd0' mock_islink.return_value = True mock_exists.return_value = True device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1]) self.assertFalse(execute_call2 in mock_execute.mock_calls) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume_without_mons(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user'} device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1, execute_call2]) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume_without_auth(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'hosts': ['192.168.10.2'], 'ports': ['6789']} self.assertRaises(exception.BrickException, rbd_connector.connect_volume, conn) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(linuxrbd.RBDVolumeIOWrapper, 'close') def test_disconnect_volume(self, volume_close, mock_rados, mock_rbd): """Test the disconnect volume case.""" rbd_connector = rbd.RBDConnector(None) device_info = rbd_connector.connect_volume(self.connection_properties) rbd_connector.disconnect_volume( self.connection_properties, device_info) self.assertEqual(1, volume_close.call_count) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_disconnect_local_volume(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} rbd_connector.disconnect_volume(conn, None) dev_name = '/dev/rbd/pool/image' cmd = ['rbd', 'unmap', dev_name, '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] mock_execute.assert_called_once_with(*cmd, root_helper=None, run_as_root=True) def test_extend_volume(self): rbd_connector = rbd.RBDConnector(None) self.assertRaises(NotImplementedError, rbd_connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_fibre_channel.py0000666000175100017510000005434513230233223026563 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os import six from os_brick import exception from os_brick.initiator.connectors import base from os_brick.initiator.connectors import fibre_channel from os_brick.initiator import linuxfc from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector class FibreChannelConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorTestCase, self).setUp() self.connector = fibre_channel.FibreChannelConnector( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertIsNotNone(self.connector._linuxscsi) def fake_get_fc_hbas(self): return [{'ClassDevice': 'host1', 'ClassDevicePath': '/sys/devices/pci0000:00/0000:00:03.0' '/0000:05:00.2/host1/fc_host/host1', 'dev_loss_tmo': '30', 'fabric_name': '0x1000000533f55566', 'issue_lip': '', 'max_npiv_vports': '255', 'maxframe_size': '2048 bytes', 'node_name': '0x200010604b019419', 'npiv_vports_inuse': '0', 'port_id': '0x680409', 'port_name': '0x100010604b019419', 'port_state': 'Online', 'port_type': 'NPort (fabric via point-to-point)', 'speed': '10 Gbit', 'supported_classes': 'Class 3', 'supported_speeds': '10 Gbit', 'symbolic_name': 'Emulex 554M FV4.0.493.0 DV8.3.27', 'tgtid_bind_type': 'wwpn (World Wide Port Name)', 'uevent': None, 'vport_create': '', 'vport_delete': ''}] def fake_get_fc_hbas_info(self): hbas = self.fake_get_fc_hbas() info = [{'port_name': hbas[0]['port_name'].replace('0x', ''), 'node_name': hbas[0]['node_name'].replace('0x', ''), 'host_device': hbas[0]['ClassDevice'], 'device_path': hbas[0]['ClassDevicePath']}] return info def fibrechan_connection(self, volume, location, wwn): return {'driver_volume_type': 'fibrechan', 'data': { 'volume_id': volume['id'], 'target_portal': location, 'target_wwn': wwn, 'target_lun': 1, }} @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') def test_get_connector_properties(self, mock_hbas): mock_hbas.return_value = self.fake_get_fc_hbas() multipath = True enforce_multipath = True props = fibre_channel.FibreChannelConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) hbas = self.fake_get_fc_hbas() expected_props = {'wwpns': [hbas[0]['port_name'].replace('0x', '')], 'wwnns': [hbas[0]['node_name'].replace('0x', '')]} self.assertEqual(expected_props, props) def test_get_search_path(self): search_path = self.connector.get_search_path() expected = "/dev/disk/by-path" self.assertEqual(expected, search_path) def test_get_pci_num(self): hba = {'device_path': "/sys/devices/pci0000:00/0000:00:03.0" "/0000:05:00.3/host2/fc_host/host2"} pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:05:00.3", pci_num) hba = {'device_path': "/sys/devices/pci0000:00/0000:00:03.0" "/0000:05:00.3/0000:06:00.6/host2/fc_host/host2"} pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:06:00.6", pci_num) hba = {'device_path': "/sys/devices/pci0000:20/0000:20:03.0" "/0000:21:00.2/net/ens2f2/ctlr_2/host3" "/fc_host/host3"} pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:21:00.2", pci_num) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') def test_get_volume_paths(self, fake_fc_hbas_info, fake_fc_hbas, fake_exists): fake_fc_hbas.side_effect = self.fake_get_fc_hbas fake_fc_hbas_info.side_effect = self.fake_get_fc_hbas_info name = 'volume-00000001' vol = {'id': 1, 'name': name} location = '10.0.2.15:3260' wwn = '1234567890123456' connection_info = self.fibrechan_connection(vol, location, wwn) volume_paths = self.connector.get_volume_paths( connection_info['data']) expected = ['/dev/disk/by-path/pci-0000:05:00.2' '-fc-0x1234567890123456-lun-1'] self.assertEqual(expected, volume_paths) @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, remove_device_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock): check_valid_device_mock.return_value = True get_fc_hbas_mock.side_effect = self.fake_get_fc_hbas get_fc_hbas_info_mock.side_effect = self.fake_get_fc_hbas_info wwn = '1234567890' multipath_devname = '/dev/md-1' devices = {"device": multipath_devname, "id": wwn, "devices": [{'device': '/dev/sdb', 'address': '1:0:0:1', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}]} get_device_info_mock.return_value = devices['devices'][0] get_scsi_wwn_mock.return_value = wwn location = '10.0.2.15:3260' name = 'volume-00000001' vol = {'id': 1, 'name': name} # Should work for string, unicode, and list wwns = ['1234567890123456', six.text_type('1234567890123456'), ['1234567890123456', '1234567890123457']] for wwn in wwns: connection_info = self.fibrechan_connection(vol, location, wwn) dev_info = self.connector.connect_volume(connection_info['data']) exp_wwn = wwn[0] if isinstance(wwn, list) else wwn dev_str = ('/dev/disk/by-path/pci-0000:05:00.2-fc-0x%s-lun-1' % exp_wwn) self.assertEqual(dev_info['type'], 'block') self.assertEqual(dev_info['path'], dev_str) self.assertNotIn('multipath_id', dev_info) self.assertNotIn('devices', dev_info) self.connector.disconnect_volume(connection_info['data'], dev_info) expected_commands = [] self.assertEqual(expected_commands, self.cmds) # Should not work for anything other than string, unicode, and list connection_info = self.fibrechan_connection(vol, location, 123) self.assertRaises(exception.NoFibreChannelHostsFound, self.connector.connect_volume, connection_info['data']) get_fc_hbas_mock.side_effect = [[]] get_fc_hbas_info_mock.side_effect = [[]] self.assertRaises(exception.NoFibreChannelHostsFound, self.connector.connect_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') def _test_connect_volume_multipath(self, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, access_mode, should_wait_for_rw, find_mp_device_path_mock): self.connector.use_multipath = True get_fc_hbas_mock.side_effect = self.fake_get_fc_hbas get_fc_hbas_info_mock.side_effect = self.fake_get_fc_hbas_info wwn = '1234567890' multipath_devname = '/dev/md-1' devices = {"device": multipath_devname, "id": wwn, "devices": [{'device': '/dev/sdb', 'address': '1:0:0:1', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}, {'device': '/dev/sdc', 'address': '1:0:0:2', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}]} get_device_info_mock.side_effect = devices['devices'] get_scsi_wwn_mock.return_value = wwn location = '10.0.2.15:3260' name = 'volume-00000001' vol = {'id': 1, 'name': name} initiator_wwn = ['1234567890123456', '1234567890123457'] find_mp_device_path_mock.return_value = '/dev/mapper/mpatha' find_mp_dev_mock.return_value = {"device": "dm-3", "id": wwn, "name": "mpatha"} connection_info = self.fibrechan_connection(vol, location, initiator_wwn) connection_info['data']['access_mode'] = access_mode self.connector.connect_volume(connection_info['data']) self.assertEqual(should_wait_for_rw, wait_for_rw_mock.called) self.connector.disconnect_volume(connection_info['data'], devices['devices'][0]) expected_commands = [ 'multipath -f ' + find_mp_device_path_mock.return_value, 'blockdev --flushbufs /dev/sdb', 'tee -a /sys/block/sdb/device/delete', 'blockdev --flushbufs /dev/sdc', 'tee -a /sys/block/sdc/device/delete', ] self.assertEqual(expected_commands, self.cmds) return connection_info @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_rw(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', True) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_no_access_mode(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, None, True) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_ro(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'ro', False) @mock.patch.object(base.BaseLinuxConnector, '_discover_mpath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_not_found(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, discover_mp_dev_mock): check_valid_device_mock.return_value = True discover_mp_dev_mock.return_value = ("/dev/disk/by-path/something", None) connection_info = self._test_connect_volume_multipath( get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', False) self.assertNotIn('multipathd_id', connection_info['data']) @mock.patch.object(fibre_channel.FibreChannelConnector, 'get_volume_paths') def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] volume = {'id': 'fake_uuid'} wwn = '1234567890123456' connection_info = self.fibrechan_connection(volume, "10.0.2.15:3260", wwn) self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(fibre_channel.FibreChannelConnector, 'get_volume_paths') def test_extend_volume(self, mock_volume_paths, mock_scsi_extend): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} wwn = '1234567890123456' connection_info = self.fibrechan_connection(volume, "10.0.2.15:3260", wwn) new_size = self.connector.extend_volume(connection_info['data']) self.assertEqual(fake_new_size, new_size) @mock.patch.object(os.path, 'isdir') def test_get_all_available_volumes_path_not_dir(self, mock_isdir): mock_isdir.return_value = False expected = [] actual = self.connector.get_all_available_volumes() self.assertItemsEqual(expected, actual) @mock.patch('eventlet.greenthread.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_device_not_valid(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = False self.assertRaises(exception.NoFibreChannelVolumeDeviceFound, self._test_connect_volume_multipath, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', True) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_aoe.py0000666000175100017510000001150613230233223024540 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os from oslo_service import loopingcall from os_brick import exception from os_brick.initiator.connectors import aoe from os_brick.tests.initiator import test_connector class FakeFixedIntervalLoopingCall(object): def __init__(self, f=None, *args, **kw): self.args = args self.kw = kw self.f = f self._stop = False def stop(self): self._stop = True def wait(self): return self def start(self, interval, initial_delay=None): while not self._stop: try: self.f(*self.args, **self.kw) except loopingcall.LoopingCallDone: return self except Exception: raise class AoEConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for AoE initiator class.""" def setUp(self): super(AoEConnectorTestCase, self).setUp() self.connector = aoe.AoEConnector('sudo') self.connection_properties = {'target_shelf': 'fake_shelf', 'target_lun': 'fake_lun'} self.mock_object(loopingcall, 'FixedIntervalLoopingCall', FakeFixedIntervalLoopingCall) def test_get_search_path(self): expected = "/dev/etherd" actual_path = self.connector.get_search_path() self.assertEqual(expected, actual_path) @mock.patch.object(os.path, 'exists', return_value=True) def test_get_volume_paths(self, mock_exists): expected = ["/dev/etherd/efake_shelf.fake_lun"] paths = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, paths) def test_get_connector_properties(self): props = aoe.AoEConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) @mock.patch.object(os.path, 'exists', side_effect=[True, True]) def test_connect_volume(self, exists_mock): """Ensure that if path exist aoe-revalidate was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.connector.connect_volume(self.connection_properties) @mock.patch.object(os.path, 'exists', side_effect=[False, True]) def test_connect_volume_without_path(self, exists_mock): """Ensure that if path doesn't exist aoe-discovery was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) expected_info = { 'type': 'block', 'device': aoe_device, 'path': aoe_path, } with mock.patch.object(self.connector, '_execute', return_value=["", ""]): volume_info = self.connector.connect_volume( self.connection_properties) self.assertDictEqual(volume_info, expected_info) @mock.patch.object(os.path, 'exists', return_value=False) def test_connect_volume_could_not_discover_path(self, exists_mock): _aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.assertRaises(exception.VolumeDeviceNotFound, self.connector.connect_volume, self.connection_properties) @mock.patch.object(os.path, 'exists', return_value=True) def test_disconnect_volume(self, mock_exists): """Ensure that if path exist aoe-revaliadte was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.connector.disconnect_volume(self.connection_properties, {}) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_vrtshyperscale.py0000666000175100017510000001315513230233223027054 0ustar zuulzuul00000000000000# Copyright (c) 2017 Veritas Technologies LLC # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json from oslo_concurrency import processutils from os_brick import exception from os_brick.initiator.connectors import vrtshyperscale from os_brick.tests.initiator import test_connector DEVICE_NAME = '{8ee71c33-dcd0-4267-8f2b-e0742ecabe9f}' DEVICE_PATH = '/dev/8ee71c33-dcd0-4267-8f2b-e0742ec' class HyperScaleConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for Veritas HyperScale os-brick connector.""" def _fake_execute_success(self, *cmd, **kwargs): """Mock successful execution of hscli""" result_json = "" err = 0 args = json.loads(cmd[1]) if args['operation'] == 'connect_volume': result = {} payload = {} payload['vsa_ip'] = '192.0.2.2' payload['refl_factor'] = '2' payload['refl_targets'] = '192.0.2.3,192.0.2.4' result['payload'] = payload result_json = json.dumps(result) return (result_json, err) def _fake_execute_hscli_missing(self, *cmd, **kwargs): """Mock attempt to execute missing hscli""" raise processutils.ProcessExecutionError() return ("", 0) def _fake_execute_hscli_err(self, *cmd, **kwargs): """Mock hscli returning error""" result_json = "" err = 'fake_hscli_error_msg' return (result_json, err) def _fake_execute_hscli_res_inval(self, *cmd, **kwargs): """Mock hscli returning unexpected values""" result_json = "" err = 0 result = {} payload = {} payload['unexpected'] = 'junk' result['payload'] = payload result_json = json.dumps(result) return (result_json, err) def test_connect_volume_normal(self): """Test results of successful connect_volume()""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = { 'name': DEVICE_NAME } device_info = connector.connect_volume(fake_connection_properties) self.assertEqual('192.0.2.2', device_info['vsa_ip']) self.assertEqual('2', device_info['refl_factor']) self.assertEqual('192.0.2.3,192.0.2.4', device_info['refl_targets']) self.assertEqual(DEVICE_PATH, device_info['path']) def test_connect_volume_arg_missing(self): """Test connect_volume with missing missing arguments""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = {} self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_missing(self): """Test connect_volume that can't call hscli""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_missing) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_err(self): """Test connect_volume when hscli returns an error""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_err) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_res_inval(self): """Test connect_volume if hscli returns an invalid result""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_res_inval) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_disconnect_volume_normal(self): """Test successful disconnect_volume call""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = { 'name': DEVICE_NAME } fake_device_info = {} connector.disconnect_volume(fake_connection_properties, fake_device_info) def test_disconnect_volume_arg_missing(self): """Test disconnect_volume with missing arguments""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = {} fake_device_info = {} self.assertRaises(exception.BrickException, connector.disconnect_volume, fake_connection_properties, fake_device_info) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_sheepdog.py0000666000175100017510000000651413230233223025575 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick import exception from os_brick.initiator.connectors import sheepdog from os_brick.initiator import linuxsheepdog from os_brick.tests.initiator import test_connector class SheepdogConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(SheepdogConnectorTestCase, self).setUp() self.hosts = ['fake_hosts'] self.ports = ['fake_ports'] self.volume = 'fake_volume' self.connection_properties = { 'hosts': self.hosts, 'name': self.volume, 'ports': self.ports, } def test_get_connector_properties(self): props = sheepdog.SheepdogConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): sd_connector = sheepdog.SheepdogConnector(None) path = sd_connector.get_search_path() self.assertIsNone(path) def test_get_volume_paths(self): sd_connector = sheepdog.SheepdogConnector(None) expected = [] actual = sd_connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): """Test the connect volume case.""" sd_connector = sheepdog.SheepdogConnector(None) device_info = sd_connector.connect_volume(self.connection_properties) # Ensure expected object is returned correctly self.assertIsInstance(device_info['path'], linuxsheepdog.SheepdogVolumeIOWrapper) @mock.patch.object(linuxsheepdog.SheepdogVolumeIOWrapper, 'close') def test_disconnect_volume(self, volume_close): """Test the disconnect volume case.""" sd_connector = sheepdog.SheepdogConnector(None) device_info = sd_connector.connect_volume(self.connection_properties) sd_connector.disconnect_volume(self.connection_properties, device_info) self.assertEqual(1, volume_close.call_count) def test_disconnect_volume_with_invalid_handle(self): """Test the disconnect volume case with invalid handle.""" sd_connector = sheepdog.SheepdogConnector(None) device_info = {'path': 'fake_handle'} self.assertRaises(exception.InvalidIOHandleObject, sd_connector.disconnect_volume, self.connection_properties, device_info) def test_extend_volume(self): sd_connector = sheepdog.SheepdogConnector(None) self.assertRaises(NotImplementedError, sd_connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_huawei.py0000666000175100017510000002443113230233223025257 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os import tempfile from os_brick import exception from os_brick.initiator.connectors import huawei from os_brick.tests.initiator import test_connector class HuaweiStorHyperConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for StorHyper initiator class.""" attached = False def setUp(self): super(HuaweiStorHyperConnectorTestCase, self).setUp() self.fake_sdscli_file = tempfile.mktemp() self.addCleanup(os.remove, self.fake_sdscli_file) newefile = open(self.fake_sdscli_file, 'w') newefile.write('test') newefile.close() self.connector = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute) self.connector.cli_path = self.fake_sdscli_file self.connector.iscliexist = True self.connector_fail = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute_fail) self.connector_fail.cli_path = self.fake_sdscli_file self.connector_fail.iscliexist = True self.connector_nocli = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute_fail) self.connector_nocli.cli_path = self.fake_sdscli_file self.connector_nocli.iscliexist = False self.connection_properties = { 'access_mode': 'rw', 'qos_specs': None, 'volume_id': 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f' } self.device_info = {'type': 'block', 'path': '/dev/vdxxx'} HuaweiStorHyperConnectorTestCase.attached = False def fake_execute(self, *cmd, **kwargs): method = cmd[2] self.cmds.append(" ".join(cmd)) if 'attach' == method: HuaweiStorHyperConnectorTestCase.attached = True return 'ret_code=0', None if 'querydev' == method: if HuaweiStorHyperConnectorTestCase.attached: return 'ret_code=0\ndev_addr=/dev/vdxxx', None else: return 'ret_code=1\ndev_addr=/dev/vdxxx', None if 'detach' == method: HuaweiStorHyperConnectorTestCase.attached = False return 'ret_code=0', None def fake_execute_fail(self, *cmd, **kwargs): method = cmd[2] self.cmds.append(" ".join(cmd)) if 'attach' == method: HuaweiStorHyperConnectorTestCase.attached = False return 'ret_code=330151401', None if 'querydev' == method: if HuaweiStorHyperConnectorTestCase.attached: return 'ret_code=0\ndev_addr=/dev/vdxxx', None else: return 'ret_code=1\ndev_addr=/dev/vdxxx', None if 'detach' == method: HuaweiStorHyperConnectorTestCase.attached = True return 'ret_code=330155007', None def test_get_connector_properties(self): props = huawei.HuaweiStorHyperConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): actual = self.connector.get_search_path() self.assertIsNone(actual) @mock.patch.object(huawei.HuaweiStorHyperConnector, '_query_attached_volume') def test_get_volume_paths(self, mock_query_attached): path = self.device_info['path'] mock_query_attached.return_value = {'ret_code': 0, 'dev_addr': path} expected = [path] actual = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): """Test the basic connect volume case.""" retval = self.connector.connect_volume(self.connection_properties) self.assertEqual(self.device_info, retval) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_disconnect_volume(self): """Test the basic disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.connector.disconnect_volume(self.connection_properties, self.device_info) self.assertEqual(False, HuaweiStorHyperConnectorTestCase.attached) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_is_volume_connected(self): """Test if volume connected to host case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) is_connected = self.connector.is_volume_connected( 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f') self.assertEqual(HuaweiStorHyperConnectorTestCase.attached, is_connected) self.connector.disconnect_volume(self.connection_properties, self.device_info) self.assertEqual(False, HuaweiStorHyperConnectorTestCase.attached) is_connected = self.connector.is_volume_connected( 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f') self.assertEqual(HuaweiStorHyperConnectorTestCase.attached, is_connected) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test__analyze_output(self): cliout = 'ret_code=0\ndev_addr=/dev/vdxxx\nret_desc="success"' analyze_result = {'dev_addr': '/dev/vdxxx', 'ret_desc': '"success"', 'ret_code': '0'} result = self.connector._analyze_output(cliout) self.assertEqual(analyze_result, result) def test_connect_volume_fail(self): """Test the fail connect volume case.""" self.assertRaises(exception.BrickException, self.connector_fail.connect_volume, self.connection_properties) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_disconnect_volume_fail(self): """Test the fail disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.assertRaises(exception.BrickException, self.connector_fail.disconnect_volume, self.connection_properties, self.device_info) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_connect_volume_nocli(self): """Test the fail connect volume case.""" self.assertRaises(exception.BrickException, self.connector_nocli.connect_volume, self.connection_properties) def test_disconnect_volume_nocli(self): """Test the fail disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.assertRaises(exception.BrickException, self.connector_nocli.disconnect_volume, self.connection_properties, self.device_info) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_gpfs.py0000666000175100017510000000272513230233223024736 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import gpfs from os_brick.tests.initiator.connectors import test_local class GPFSConnectorTestCase(test_local.LocalConnectorTestCase): def setUp(self): super(GPFSConnectorTestCase, self).setUp() self.connection_properties = {'name': 'foo', 'device_path': '/tmp/bar'} self.connector = gpfs.GPFSConnector(None) def test_connect_volume(self): cprops = self.connection_properties dev_info = self.connector.connect_volume(cprops) self.assertEqual(dev_info['type'], 'gpfs') self.assertEqual(dev_info['path'], cprops['device_path']) def test_connect_volume_with_invalid_connection_data(self): cprops = {} self.assertRaises(ValueError, self.connector.connect_volume, cprops) os-brick-2.3.0/os_brick/tests/initiator/connectors/test_storpool.py0000666000175100017510000001260713230233223025660 0ustar zuulzuul00000000000000# Copyright (c) 2015 - 2017 StorPool # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick import exception from os_brick.initiator.connectors import storpool as connector from os_brick.tests.initiator import test_connector def volumeNameExt(vid): return 'os--volume--{id}'.format(id=vid) class MockStorPoolADB(object): def __init__(self, log): self.requests = {} self.attached = {} def api(self): pass def add(self, req_id, req): if req_id in self.requests: raise Exception('Duplicate MockStorPool request added') self.requests[req_id] = req def remove(self, req_id): req = self.requests.get(req_id, None) if req is None: raise Exception('Unknown MockStorPool request removed') elif req['volume'] in self.attached: raise Exception('Removing attached MockStorPool volume') del self.requests[req_id] def sync(self, req_id, detached): req = self.requests.get(req_id, None) if req is None: raise Exception('Unknown MockStorPool request synced') volume = req.get('volume', None) if volume is None: raise Exception('MockStorPool request without volume') if detached is None: if volume in self.attached: raise Exception('Duplicate MockStorPool request synced') self.attached[volume] = req else: if volume != detached: raise Exception( 'Mismatched volumes on a MockStorPool request removal') elif detached not in self.attached: raise Exception('MockStorPool request not attached yet') del self.attached[detached] def volumeName(self, vid): return volumeNameExt(vid) spopenstack = mock.Mock() spopenstack.AttachDB = MockStorPoolADB connector.spopenstack = spopenstack class StorPoolConnectorTestCase(test_connector.ConnectorTestCase): def volumeName(self, vid): return volumeNameExt(vid) def execute(self, *cmd, **kwargs): if cmd[0] == 'blockdev': self.assertEqual(len(cmd), 3) self.assertEqual(cmd[1], '--getsize64') self.assertEqual(cmd[2], '/dev/storpool/' + self.volumeName(self.fakeProp['volume'])) return (str(self.fakeSize), None) raise Exception("Unrecognized command passed to " + type(self).__name__ + ".execute(): " + str.join(", ", map(lambda s: "'" + s + "'", cmd))) def setUp(self): super(StorPoolConnectorTestCase, self).setUp() self.fakeProp = { 'volume': 'sp-vol-1', 'client_id': 1, 'access_mode': 'rw', } self.fakeConnection = None self.fakeSize = 1024 * 1024 * 1024 self.connector = connector.StorPoolConnector( None, execute=self.execute) self.adb = self.connector._attach def test_connect_volume(self): self.assertNotIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) conn = self.connector.connect_volume(self.fakeProp) self.assertIn('type', conn) self.assertIn('path', conn) self.assertIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) self.assertEqual(self.connector.get_search_path(), '/dev/storpool') paths = self.connector.get_volume_paths(self.fakeProp) self.assertEqual(len(paths), 1) self.assertEqual(paths[0], "/dev/storpool/" + self.volumeName(self.fakeProp['volume'])) self.fakeConnection = conn def test_disconnect_volume(self): if self.fakeConnection is None: self.test_connect_volume() self.assertIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) self.connector.disconnect_volume(self.fakeProp, None) self.assertNotIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) def test_connect_exceptions(self): """Raise exceptions on missing connection information""" fake = self.fakeProp for key in fake.keys(): c = dict(fake) del c[key] self.assertRaises(exception.BrickException, self.connector.connect_volume, c) if key != 'access_mode': self.assertRaises(exception.BrickException, self.connector.disconnect_volume, c, None) def test_extend_volume(self): if self.fakeConnection is None: self.test_connect_volume() self.fakeSize += 1024 * 1024 * 1024 newSize = self.connector.extend_volume(self.fakeProp) self.assertEqual(newSize, self.fakeSize) os-brick-2.3.0/os_brick/tests/privileged/0000775000175100017510000000000013230233405020333 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/privileged/__init__.py0000666000175100017510000000000013230233223022432 0ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/tests/privileged/test_rootwrap.py0000666000175100017510000001613713230233223023631 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import time import mock from oslo_concurrency import processutils as putils import six from os_brick import exception from os_brick import privileged from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base class PrivRootwrapTestCase(base.TestCase): def setUp(self): super(PrivRootwrapTestCase, self).setUp() # Bypass privsep and run these simple functions in-process # (allows reading back the modified state of mocks) privileged.default.set_client_mode(False) self.addCleanup(privileged.default.set_client_mode, True) @mock.patch('os_brick.privileged.rootwrap.execute_root') @mock.patch('oslo_concurrency.processutils.execute') def test_execute(self, mock_putils_exec, mock_exec_root): priv_rootwrap.execute('echo', 'foo', run_as_root=False) self.assertFalse(mock_exec_root.called) priv_rootwrap.execute('echo', 'foo', run_as_root=True, root_helper='baz', check_exit_code=0) mock_exec_root.assert_called_once_with( 'echo', 'foo', check_exit_code=0) @mock.patch('oslo_concurrency.processutils.execute') def test_execute_root(self, mock_putils_exec): priv_rootwrap.execute_root('echo', 'foo', check_exit_code=0) mock_putils_exec.assert_called_once_with( 'echo', 'foo', check_exit_code=0, shell=False, run_as_root=False, delay_on_retry=False, on_completion=mock.ANY, on_execute=mock.ANY) # Exact exception isn't particularly important, but these # should be errors: self.assertRaises(TypeError, priv_rootwrap.execute_root, 'foo', shell=True) self.assertRaises(TypeError, priv_rootwrap.execute_root, 'foo', run_as_root=True) @mock.patch('oslo_concurrency.processutils.execute', side_effect=OSError(42, 'mock error')) def test_oserror_raise(self, mock_putils_exec): self.assertRaises(putils.ProcessExecutionError, priv_rootwrap.execute, 'foo') @mock.patch.object(priv_rootwrap.execute_root.privsep_entrypoint, 'client_mode', False) @mock.patch.object(priv_rootwrap, 'custom_execute') def test_execute_as_root(self, exec_mock): res = priv_rootwrap.execute(mock.sentinel.cmds, run_as_root=True, root_helper=mock.sentinel.root_helper, keyword_arg=mock.sentinel.kwarg) self.assertEqual(exec_mock.return_value, res) exec_mock.assert_called_once_with(mock.sentinel.cmds, shell=False, run_as_root=False, keyword_arg=mock.sentinel.kwarg) def test_custom_execute(self): on_execute = mock.Mock() on_completion = mock.Mock() msg = 'hola' out, err = priv_rootwrap.custom_execute('echo', msg, on_execute=on_execute, on_completion=on_completion) self.assertEqual(msg + '\n', out) self.assertEqual('', err) on_execute.assert_called_once_with(mock.ANY) proc = on_execute.call_args[0][0] on_completion.assert_called_once_with(proc) @mock.patch('time.sleep') def test_custom_execute_timeout_raises_with_retries(self, sleep_mock): on_execute = mock.Mock() on_completion = mock.Mock() t0 = time.time() self.assertRaises(exception.ExecutionTimeout, priv_rootwrap.custom_execute, 'sleep', '2', timeout=0.05, raise_timeout=True, interval=2, backoff_rate=3, attempts=3, on_execute=on_execute, on_completion=on_completion) t1 = time.time() self.assertLess(t1 - t0, 0.3) sleep_mock.assert_has_calls([mock.call(0), mock.call(6), mock.call(0), mock.call(18), mock.call(0)]) expected_calls = [mock.call(args[0][0]) for args in on_execute.call_args_list] on_execute.assert_has_calls(expected_calls) on_completion.assert_has_calls(expected_calls) def test_custom_execute_timeout_no_raise(self): t0 = time.time() out, err = priv_rootwrap.custom_execute('sleep', '2', timeout=0.05, raise_timeout=False) t1 = time.time() self.assertEqual('', out) self.assertIsInstance(err, six.string_types) self.assertLess(t1 - t0, 0.3) def test_custom_execute_check_exit_code(self): self.assertRaises(putils.ProcessExecutionError, priv_rootwrap.custom_execute, 'ls', '-y', check_exit_code=True) def test_custom_execute_no_check_exit_code(self): out, err = priv_rootwrap.custom_execute('ls', '-y', check_exit_code=False) self.assertEqual('', out) self.assertIsInstance(err, six.string_types) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] priv_rootwrap.unlink_root(*links, no_errors=True) unlink_mock.assert_has_calls([mock.call(links[0]), mock.call(links[1])]) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root_raise(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] self.assertRaises(IOError, priv_rootwrap.unlink_root, *links, no_errors=False) unlink_mock.assert_called_once_with(links[0]) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root_raise_at_end(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] self.assertRaises(exception.ExceptionChainer, priv_rootwrap.unlink_root, *links, raise_at_end=True) unlink_mock.assert_has_calls([mock.call(links[0]), mock.call(links[1])]) os-brick-2.3.0/os_brick/privileged/0000775000175100017510000000000013230233405017171 5ustar zuulzuul00000000000000os-brick-2.3.0/os_brick/privileged/__init__.py0000666000175100017510000000162113230233223021302 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_privsep import capabilities as c from oslo_privsep import priv_context # It is expected that most (if not all) os-brick operations can be # executed with these privileges. default = priv_context.PrivContext( __name__, cfg_section='privsep_osbrick', pypath=__name__ + '.default', capabilities=[c.CAP_SYS_ADMIN], ) os-brick-2.3.0/os_brick/privileged/rootwrap.py0000666000175100017510000002070413230233223021423 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Just in case it wasn't clear, this is a massive security back-door. `execute_root()` (or the same via `execute(run_as_root=True)`) allows any command to be run as the privileged user (default "root"). This is intended only as an expedient transition and should be removed ASAP. This is not completely unreasonable because: 1. We have no tool/workflow for merging changes to rootwrap filter configs from os-brick into nova/cinder, which makes it difficult to evolve these loosely coupled projects. 2. Let's not pretend the earlier situation was any better. The rootwrap filters config contained several entries like "allow cp as root with any arguments", etc, and would have posed only a mild inconvenience to an attacker. At least with privsep we can (in principle) run the "root" commands as a non-root uid, with restricted Linux capabilities. The plan is to switch os-brick to privsep using this module (removing the urgency of (1)), then work on the larger refactor that addresses (2) in followup changes. """ import os import signal import six import threading import time from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import strutils from os_brick import exception from os_brick import privileged LOG = logging.getLogger(__name__) def custom_execute(*cmd, **kwargs): """Custom execute with additional functionality on top of Oslo's. Additional features are timeouts and exponential backoff retries. The exponential backoff retries replaces standard Oslo random sleep times that range from 200ms to 2seconds when attempts is greater than 1, but it is disabled if delay_on_retry is passed as a parameter. Exponential backoff is controlled via interval and backoff_rate parameters, just like the os_brick.utils.retry decorator. To use the timeout mechanism to stop the subprocess with a specific signal after a number of seconds we must pass a non-zero timeout value in the call. When using multiple attempts and timeout at the same time the method will only raise the timeout exception to the caller if the last try timeouts. Timeout mechanism is controlled with timeout, signal, and raise_timeout parameters. :param interval: The multiplier :param backoff_rate: Base used for the exponential backoff :param timeout: Timeout defined in seconds :param signal: Signal to use to stop the process on timeout :param raise_timeout: Raise and exception on timeout or return error as stderr. Defaults to raising if check_exit_code is not False. :returns: Tuple with stdout and stderr """ # Since python 2 doesn't have nonlocal we use a mutable variable to store # the previous attempt number, the timeout handler, and the process that # timed out shared_data = [0, None, None] def on_timeout(proc): sanitized_cmd = strutils.mask_password(' '.join(cmd)) LOG.warning('Stopping %(cmd)s with signal %(signal)s after %(time)ss.', {'signal': sig_end, 'cmd': sanitized_cmd, 'time': timeout}) shared_data[2] = proc proc.send_signal(sig_end) def on_execute(proc): # Call user's on_execute method if on_execute_call: on_execute_call(proc) # Sleep if this is not the first try and we have a timeout interval if shared_data[0] and interval: exp = backoff_rate ** shared_data[0] wait_for = max(0, interval * exp) LOG.debug('Sleeping for %s seconds', wait_for) time.sleep(wait_for) # Increase the number of tries and start the timeout timer shared_data[0] += 1 if timeout: shared_data[2] = None shared_data[1] = threading.Timer(timeout, on_timeout, (proc,)) shared_data[1].start() def on_completion(proc): # This is always called regardless of success or failure # Cancel the timeout timer if shared_data[1]: shared_data[1].cancel() # Call user's on_completion method if on_completion_call: on_completion_call(proc) # We will be doing the wait ourselves in on_execute if 'delay_on_retry' in kwargs: interval = None else: kwargs['delay_on_retry'] = False interval = kwargs.pop('interval', 1) backoff_rate = kwargs.pop('backoff_rate', 2) timeout = kwargs.pop('timeout', None) sig_end = kwargs.pop('signal', signal.SIGTERM) default_raise_timeout = kwargs.get('check_exit_code', True) raise_timeout = kwargs.pop('raise_timeout', default_raise_timeout) on_execute_call = kwargs.pop('on_execute', None) on_completion_call = kwargs.pop('on_completion', None) try: return putils.execute(on_execute=on_execute, on_completion=on_completion, *cmd, **kwargs) except putils.ProcessExecutionError: # proc is only stored if a timeout happened proc = shared_data[2] if proc: sanitized_cmd = strutils.mask_password(' '.join(cmd)) msg = ('Time out on proc %(pid)s after waiting %(time)s seconds ' 'when running %(cmd)s' % {'pid': proc.pid, 'time': timeout, 'cmd': sanitized_cmd}) LOG.debug(msg) if raise_timeout: raise exception.ExecutionTimeout(stdout='', stderr=msg, cmd=sanitized_cmd) return '', msg raise # Entrypoint used for rootwrap.py transition code. Don't use this for # other purposes, since it will be removed when we think the # transition is finished. def execute(*cmd, **kwargs): """NB: Raises processutils.ProcessExecutionError on failure.""" run_as_root = kwargs.pop('run_as_root', False) kwargs.pop('root_helper', None) try: if run_as_root: return execute_root(*cmd, **kwargs) else: return custom_execute(*cmd, **kwargs) except OSError as e: # Note: # putils.execute('bogus', run_as_root=True) # raises ProcessExecutionError(exit_code=1) (because there's a # "sh -c bogus" involved in there somewhere, but: # putils.execute('bogus', run_as_root=False) # raises OSError(not found). # # Lots of code in os-brick catches only ProcessExecutionError # and never encountered the latter when using rootwrap. # Rather than fix all the callers, we just always raise # ProcessExecutionError here :( sanitized_cmd = strutils.mask_password(' '.join(cmd)) raise putils.ProcessExecutionError( cmd=sanitized_cmd, description=six.text_type(e)) # See comment on `execute` @privileged.default.entrypoint def execute_root(*cmd, **kwargs): """NB: Raises processutils.ProcessExecutionError/OSError on failure.""" return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) @privileged.default.entrypoint def unlink_root(*links, **kwargs): """Unlink system links with sys admin privileges. By default it will raise an exception if a link does not exist and stop unlinking remaining links. This behavior can be modified passing optional parameters `no_errors` and `raise_at_end`. :param no_errors: Don't raise an exception on error "param raise_at_end: Don't raise an exception on first error, try to unlink all links and then raise a ChainedException with all the errors that where found. """ no_errors = kwargs.get('no_errors', False) raise_at_end = kwargs.get('raise_at_end', False) exc = exception.ExceptionChainer() catch_exception = no_errors or raise_at_end for link in links: with exc.context(catch_exception, 'Unlink failed for %s', link): os.unlink(link) if not no_errors and raise_at_end and exc: raise exc os-brick-2.3.0/os_brick/exception.py0000666000175100017510000001607713230233223017422 0ustar zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Exceptions for the Brick library.""" from oslo_concurrency import processutils as putils import six import traceback from os_brick.i18n import _ from oslo_log import log as logging LOG = logging.getLogger(__name__) class BrickException(Exception): """Base Brick Exception To correctly use this class, inherit from it and define a 'msg_fmt' property. That msg_fmt will get printf'd with the keyword arguments provided to the constructor. """ message = _("An unknown exception occurred.") code = 500 headers = {} safe = False def __init__(self, message=None, **kwargs): self.kwargs = kwargs if 'code' not in self.kwargs: try: self.kwargs['code'] = self.code except AttributeError: pass if not message: try: message = self.message % kwargs except Exception: # kwargs doesn't match a variable in the message # log the issue and the kwargs LOG.exception("Exception in string format operation. " "msg='%s'", self.message) for name, value in kwargs.items(): LOG.error("%(name)s: %(value)s", {'name': name, 'value': value}) # at least get the core message out if something happened message = self.message # Put the message in 'msg' so that we can access it. If we have it in # message it will be overshadowed by the class' message attribute self.msg = message super(BrickException, self).__init__(message) def __unicode__(self): return six.text_type(self.msg) class NotFound(BrickException): message = _("Resource could not be found.") code = 404 safe = True class Invalid(BrickException): message = _("Unacceptable parameters.") code = 400 # Cannot be templated as the error syntax varies. # msg needs to be constructed when raised. class InvalidParameterValue(Invalid): message = _("%(err)s") class NoFibreChannelHostsFound(BrickException): message = _("We are unable to locate any Fibre Channel devices.") class NoFibreChannelVolumeDeviceFound(BrickException): message = _("Unable to find a Fibre Channel volume device.") class VolumeNotDeactivated(BrickException): message = _('Volume %(name)s was not deactivated in time.') class VolumeDeviceNotFound(BrickException): message = _("Volume device not found at %(device)s.") class VolumePathsNotFound(BrickException): message = _("Could not find any paths for the volume.") class VolumePathNotRemoved(BrickException): message = _("Volume path %(volume_path)s was not removed in time.") class ProtocolNotSupported(BrickException): message = _("Connect to volume via protocol %(protocol)s not supported.") class TargetPortalNotFound(BrickException): message = _("Unable to find target portal %(target_portal)s.") class TargetPortalsNotFound(BrickException): message = _("Unable to find target portal in %(target_portals)s.") class FailedISCSITargetPortalLogin(BrickException): message = _("Unable to login to iSCSI Target Portal") class BlockDeviceReadOnly(BrickException): message = _("Block device %(device)s is Read-Only.") class VolumeGroupNotFound(BrickException): message = _("Unable to find Volume Group: %(vg_name)s") class VolumeGroupCreationFailed(BrickException): message = _("Failed to create Volume Group: %(vg_name)s") class CommandExecutionFailed(BrickException): message = _("Failed to execute command %(cmd)s") class VolumeDriverException(BrickException): message = _('An error occurred while IO to volume %(name)s.') class InvalidIOHandleObject(BrickException): message = _('IO handle of %(protocol)s has wrong object ' 'type %(actual_type)s.') class VolumeEncryptionNotSupported(Invalid): message = _("Volume encryption is not supported for %(volume_type)s " "volume %(volume_id)s.") # NOTE(mriedem): This extends ValueError to maintain backward compatibility. class InvalidConnectorProtocol(ValueError): pass class ExceptionChainer(BrickException): """A Exception that can contain a group of exceptions. This exception serves as a container for exceptions, useful when we want to store all exceptions that happened during a series of steps and then raise them all together as one. The representation of the exception will include all exceptions and their tracebacks. This class also includes a context manager for convenience, one that will support both swallowing the exception as if nothing had happened and raising the exception. In both cases the exception will be stored. If a message is provided to the context manager it will be formatted and logged with warning level. """ def __init__(self, *args, **kwargs): self._exceptions = [] self._repr = None super(ExceptionChainer, self).__init__(*args, **kwargs) def __repr__(self): # Since generating the representation can be slow we cache it if not self._repr: tracebacks = ( ''.join(traceback.format_exception(*e)).replace('\n', '\n\t') for e in self._exceptions) self._repr = '\n'.join('\nChained Exception #%s\n\t%s' % (i + 1, t) for i, t in enumerate(tracebacks)) return self._repr __str__ = __unicode__ = __repr__ def __nonzero__(self): # We want to be able to do boolean checks on the exception return bool(self._exceptions) __bool__ = __nonzero__ # For Python 3 def add_exception(self, exc_type, exc_val, exc_tb): # Clear the representation cache self._repr = None self._exceptions.append((exc_type, exc_val, exc_tb)) def context(self, catch_exception, msg='', *msg_args): self._catch_exception = catch_exception self._exc_msg = msg self._exc_msg_args = msg_args return self def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type: self.add_exception(exc_type, exc_val, exc_tb) if self._exc_msg: LOG.warning(self._exc_msg, *self._exc_msg_args) if self._catch_exception: return True class ExecutionTimeout(putils.ProcessExecutionError): pass os-brick-2.3.0/os_brick/utils.py0000666000175100017510000001356113230233223016557 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # """Utilities and helper functions.""" import functools import inspect import logging as py_logging import retrying import six import time from oslo_log import log as logging from oslo_utils import encodeutils from oslo_utils import strutils from os_brick.i18n import _ LOG = logging.getLogger(__name__) def retry(exceptions, interval=1, retries=3, backoff_rate=2): def _retry_on_exception(e): return isinstance(e, exceptions) def _backoff_sleep(previous_attempt_number, delay_since_first_attempt_ms): exp = backoff_rate ** previous_attempt_number wait_for = max(0, interval * exp) LOG.debug("Sleeping for %s seconds", wait_for) return wait_for * 1000.0 def _print_stop(previous_attempt_number, delay_since_first_attempt_ms): delay_since_first_attempt = delay_since_first_attempt_ms / 1000.0 LOG.debug("Failed attempt %s", previous_attempt_number) LOG.debug("Have been at this for %s seconds", delay_since_first_attempt) return previous_attempt_number == retries if retries < 1: raise ValueError(_('Retries must be greater than or ' 'equal to 1 (received: %s). ') % retries) def _decorator(f): @six.wraps(f) def _wrapper(*args, **kwargs): r = retrying.Retrying(retry_on_exception=_retry_on_exception, wait_func=_backoff_sleep, stop_func=_print_stop) return r.call(f, *args, **kwargs) return _wrapper return _decorator def platform_matches(current_platform, connector_platform): curr_p = current_platform.upper() conn_p = connector_platform.upper() if conn_p == 'ALL': return True # Add tests against families of platforms if curr_p == conn_p: return True return False def os_matches(current_os, connector_os): curr_os = current_os.upper() conn_os = connector_os.upper() if conn_os == 'ALL': return True # add tests against OSs if (conn_os == curr_os or conn_os in curr_os): return True return False def merge_dict(dict1, dict2): """Try to safely merge 2 dictionaries.""" if type(dict1) is not dict: raise Exception("dict1 is not a dictionary") if type(dict2) is not dict: raise Exception("dict2 is not a dictionary") dict3 = dict1.copy() dict3.update(dict2) return dict3 def trace(f): """Trace calls to the decorated function. This decorator should always be defined as the outermost decorator so it is defined last. This is important so it does not interfere with other decorators. Using this decorator on a function will cause its execution to be logged at `DEBUG` level with arguments, return values, and exceptions. :returns: a function decorator """ func_name = f.__name__ @functools.wraps(f) def trace_logging_wrapper(*args, **kwargs): if len(args) > 0: maybe_self = args[0] else: maybe_self = kwargs.get('self', None) if maybe_self and hasattr(maybe_self, '__module__'): logger = logging.getLogger(maybe_self.__module__) else: logger = LOG # NOTE(ameade): Don't bother going any further if DEBUG log level # is not enabled for the logger. if not logger.isEnabledFor(py_logging.DEBUG): return f(*args, **kwargs) all_args = inspect.getcallargs(f, *args, **kwargs) logger.debug('==> %(func)s: call %(all_args)r', {'func': func_name, # NOTE(mriedem): We have to stringify the dict first # and don't use mask_dict_password because it results in # an infinite recursion failure. 'all_args': strutils.mask_password( six.text_type(all_args))}) start_time = time.time() * 1000 try: result = f(*args, **kwargs) except Exception as exc: total_time = int(round(time.time() * 1000)) - start_time logger.debug('<== %(func)s: exception (%(time)dms) %(exc)r', {'func': func_name, 'time': total_time, 'exc': exc}) raise total_time = int(round(time.time() * 1000)) - start_time if isinstance(result, dict): mask_result = strutils.mask_dict_password(result) elif isinstance(result, six.string_types): mask_result = strutils.mask_password(result) else: mask_result = result logger.debug('<== %(func)s: return (%(time)dms) %(result)r', {'func': func_name, 'time': total_time, 'result': mask_result}) return result return trace_logging_wrapper def convert_str(text): """Convert to native string. Convert bytes and Unicode strings to native strings: * convert to bytes on Python 2: encode Unicode using encodeutils.safe_encode() * convert to Unicode on Python 3: decode bytes from UTF-8 """ if six.PY2: return encodeutils.to_utf8(text) else: if isinstance(text, bytes): return text.decode('utf-8') else: return text os-brick-2.3.0/.mailmap0000666000175100017510000000013113230233223014660 0ustar zuulzuul00000000000000# Format is: # # os-brick-2.3.0/AUTHORS0000664000175100017510000001022613230233404014314 0ustar zuulzuul00000000000000Alfredo Moralejo Andreas Jaeger Andreas Scheuring Angus Lees Anish Bhatt Anthony Lee Arne Recknagel Arnon Yaari Aviram Bar-Haim Avishay Traeger Bertrand Lallau Cao Xuan Hoang ChangBo Guo(gcb) Chhavi Agarwal Chris MacNaughton Christopher Uhler Daniel Pawlik Dirk Mueller Dmitry Guryanov Dmitry Guryanov Doug Hellmann Earle F. Philhower, III Eric Harney Flavio Percoco Gorka Eguileor Hahyun Ivan Kolodyazhny Jack Lu Jay S. Bryant Ji-Wei John Griffith Jon Bernard Jordan Pittier Jose Porrua Keiichi KII Kendall Nelson Kendall Nelson Lee Yarwood LisaLi Liu Qing Lucian Petrut Lukas Bezdicka Maciej Kucia Matan Sabag Mathieu Gagné Matt Riedemann Matthew Booth Michael Price Michal Dulko Michał Dulko Mike Perez Monty Taylor Naga Venkata Nate Potter Ondřej Nový OpenStack Release Bot Patricia Domingues Patrick East Peter Penchev Peter Wang Philipp Reisner Rafael Folco Rawan Herzallah Rikimaru Honjo Ryan Rossiter Sean McGinnis Sean McGinnis Sergey Vilgelm Shilpa Jagannath Silvan Kaiser Stefan Amann Stephen Finucane Swapnil Kulkarni (coolsvap) Thelo Gaultier Thomas Bechtold Tomoki Sekiyama Tony Breeds Tony Xu Van Hung Pham Victor Stinner Vipin Balachandran Walter A. Boring IV Walter A. Boring IV Walter A. Boring IV Xiaojun Liao Xing Yang Yusuke Hayashi Zhao Liqiang Zuul ankitagrawal cheng cheng li digvijay2016 felix23ma haobing1 howardlee jeremy.zhang lihaijing lisali liuyamin melissaml wang yong wanghongxu wangxiyuan weiweigu xianming mao yuyafei zhangdaolong zhanghongtao zhangsong zhangyanxian zhangyanxian