././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/0000775000175000017500000000000000000000000013256 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/.coveragerc0000664000175000017500000000013500000000000015376 0ustar00zuulzuul00000000000000[run] branch = True source = os_brick omit = os_brick/tests/* [report] ignore_errors = True ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/.mailmap0000664000175000017500000000013100000000000014672 0ustar00zuulzuul00000000000000# Format is: # # ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/.stestr.conf0000664000175000017500000000010100000000000015517 0ustar00zuulzuul00000000000000[DEFAULT] test_path=${OS_TEST_PATH:-./os_brick/tests} top_dir=./ ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/.zuul.yaml0000664000175000017500000000222000000000000015213 0ustar00zuulzuul00000000000000- project: templates: - check-requirements - lib-forward-testing-python3 - openstack-python3-ussuri-jobs - publish-openstack-docs-pti - release-notes-jobs-python3 check: jobs: - os-brick-src-devstack-plugin-ceph: voting: false - os-brick-src-tempest-lvm-lio-barbican gate: jobs: - os-brick-src-tempest-lvm-lio-barbican experimental: jobs: - openstack-tox-pylint - job: name: os-brick-src-devstack-plugin-ceph description: | Tempest job which tests os-brick from source. Former names for this job were: * legacy-tempest-dsvm-full-ceph-plugin-src-os-brick parent: cinder-plugin-ceph-tempest required-projects: - opendev.org/openstack/os-brick - job: name: os-brick-src-tempest-lvm-lio-barbican parent: cinder-tempest-plugin-lvm-lio-barbican description: | Specialized cinder-tempest-lvm-lio-barbican which runs against os-brick from sources. Former names for this job were: * legacy-tempest-dsvm-full-lio-src-os-brick required-projects: - opendev.org/openstack/os-brick ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/AUTHORS0000664000175000017500000001332500000000000014332 0ustar00zuulzuul00000000000000Alan Bishop Alex Kavanagh Alfredo Moralejo Andreas Jaeger Andreas Scheuring Angus Lees Anish Bhatt Anthony Lee Arne Recknagel Arnon Yaari Aviram Bar-Haim Avishay Traeger Bertrand Lallau Brian Rosmaita Cao Xuan Hoang ChangBo Guo(gcb) Charles Short Charles Short Chhavi Agarwal Chris M Chris MacNaughton Christopher Uhler Chuck Short Corey Bryant Daniel Pawlik David Vallee Delisle Dirk Mueller Dmitry Guryanov Dmitry Guryanov Doug Hellmann Earle F. Philhower, III Eric Harney Eric Young Erik Olof Gunnar Andersson Flavio Percoco Ghanshyam Mann Gorka Eguileor Hahyun Hamdy Khader Ivan Kolodyazhny Ivan Pchelintsev Jack Lu Jay S. Bryant Ji-Wei John Griffith Jon Bernard Jordan Pittier Jose Porrua Keiichi KII Kendall Nelson Kendall Nelson Lee Yarwood LisaLi Liu Qing Lucian Petrut Luigi Toscano Lukas Bezdicka Luong Anh Tuan Maciej Kucia Matan Sabag Mathieu Gagné Matt Riedemann Matthew Booth Michael Price Michal Dulko Michał Dulko Mike Perez Monty Taylor Naga Venkata Nate Potter Ondřej Nový OpenStack Release Bot Patricia Domingues Patrick East Pawel Kaminski Peter Penchev Peter Wang Philipp Reisner Rafael Folco Rawan Herzallah Rikimaru Honjo Rui Yuan Dou Ryan Rossiter Sahid Orentino Ferdjaoui Sam Wan Sean McGinnis Sean McGinnis Sean McGinnis Sergey Vilgelm Shilpa Jagannath Silvan Kaiser Sophie Huang Stefan Amann Stephen Finucane Swapnil Kulkarni (coolsvap) Szczerbik, Przemyslaw Takashi Kajinami Tejdeep Kautharam Thelo Gaultier Thomas Bechtold Tomoki Sekiyama Tony Breeds Tony Xu Van Hung Pham Victor Stinner Vipin Balachandran Vu Cong Tuan Walter A. Boring IV Walter A. Boring IV Walter A. Boring IV Xiaojun Liao Xing Yang Yingxin Yong Huang Yury Kulazhenkov Yusuke Hayashi Zhao Liqiang ZhijunWei ankitagrawal caixiaoyu caoyuan cheng cheng li digvijay2016 felix23ma haobing1 howardlee iain MacDonnell imacdonn jacky06 jeremy.zhang jichenjc kangyufei lihaijing lisali liuyamin melissaml pengyuesheng qingszhao shenjiatong wang yong wanghongxu wangxiyuan weiweigu whoami-rajat xianming mao yenai yuyafei zengjia zhangdaolong zhanghongtao zhangsong zhangyanxian zhangyanxian ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/CONTRIBUTING.rst0000664000175000017500000000113300000000000015715 0ustar00zuulzuul00000000000000The source repository for this project can be found at: https://opendev.org/openstack/os-brick Pull requests submitted through GitHub are not monitored. To start contributing to OpenStack, follow the steps in the contribution guide to set up and use Gerrit: https://docs.openstack.org/contributors/code-and-documentation/quick-start.html Bugs should be filed on Launchpad: https://bugs.launchpad.net/os-brick For more specific information about contributing to this repository, see the os-brick contributor guide: https://docs.openstack.org/os-brick/latest/contributor/contributing.html ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982234.0 os-brick-3.0.8/ChangeLog0000664000175000017500000005605400000000000015042 0ustar00zuulzuul00000000000000CHANGES ======= 3.0.8 ----- * Use file locks in connectors * multipath/iscsi: iSCSI connections are not reinitiated after reboot 3.0.7 ----- * Avoid unhandled exceptions during connecting to iSCSI portals * multipath/iscsi: remove devices from multipath monitoring * Drop lower-constraints job 3.0.6 ----- * iSCSI: Fix flushing after multipath cfg change 3.0.5 ----- * ScaleIO: More connection info backward compatibility * FC: Fix not flushing on detach * Improve error handling on target query * Add oslo.context dependency * Adjust lower-constraints 3.0.4 ----- * New fix for rbd connector to work with ceph octopus 3.0.3 ----- * rbd: Warn if ceph udev rules are not configured * Leverage the iSCSI mpath to get the WWN * iSCSI detect multipath DM with no WWN * Add release note for scaleio connector upgrade * ScaleIO: Connection info backward compatibility * Improve WWN detection * prepend platform info to by-path string * rbd: Support 'rbd showmapped' output from ceph 13.2.0+ 3.0.2 ----- * Remove VxFlex OS credentials from connection\_properties * rbd: Correct local\_attach disconnect test and showmapped arguments * Fix hacking min version to 3.0.1 * Update TOX/UPPER\_CONSTRAINTS\_FILE for stable/ussuri * Update .gitreview for stable/ussuri 3.0.1 ----- * Add release note for ussuri cycle release * rbd: Use showmapped to find the root RBD device during disconnect\_volume * doc/source/conf.py is not executable * Ussuri contrib docs community goal 3.0.0 ----- * connectors/nvme: Wait until nvme device shows up in kernel * Skip cryptsetup password quality checking * Drop requirements for unsupported python versions * Raise hacking version to 2.0.0 * Read mounts from /proc/mounts instead of running mount * Remove Sheepdog connector * Port the os-bricks jobs to Zuul v3 2.11.0 ------ * StorPool: wait for the device to be resized * Remove Python 2.7 support from testing and gates * iscsi: Add \_get\_device\_link retry when waiting for /dev/disk/by-id/ to populate * StorPool: parse the output of \`blockdev\` correctly * Fix tox 'bindep' environment * Split connector list by platform * FC improve logging * Update FC connection\_properties examples * Fix FC scan too broad * nvmeof: Fix broken UTs * Remove VxFlexOS connector external dependencies * Switch to Ussuri jobs * Add linuxscsi get\_device\_info unit test * nvmeof: Use subnqn to disconnect a volume * Update the constraints url * Require oslo.privsep 1.32.0 * Update master for stable/train * Blacklist eventlet 0.25.0 * Change PDF file name 2.10.0 ------ * encryptors: Introduce support for LUKS2 * Rename nvme to nvmeof * Add pdf documentation build in tox * Fix param in s390x platform * encryptors: Deprecate the CryptsetupEncryptor * Blacklist sphinx 2.1.0 (autodoc bug) * Fix bad argument to iscsiadm in iSCSI discovery * Bump the openstackdocstheme extension to 1.20 * Delete redundant code * Sync Sphinx requirement 2.9.1 ----- * luks: Explicitly use the luks1 type to ensure LUKS v1 is used * iSCSI single path: Don't fail if there's no WWN * Add Python 3 Train unit tests * Make NFS already mounted message debug level * Check path alive before get scsi wwn * linuxscsi: Stop waiting for multipath devices during extend\_volume * luks: Default to LUKS v1 when formatting volumes * FC: Ignore some HBAs from map for single WWNN 2.9.0 ----- * Provide setting to ignore lvm descriptor leak warnings * Ignore pep8 W503/W504 * Replace git.openstack.org URLs with opendev.org URLs * OpenDev Migration Patch * Add generate\_connector\_list * Fix invalid escape sequence warnings * Update master for stable/stein 2.8.1 ----- * Fix ScaleIO KeyError after upgrade * Revert "rename ScaleIO connector to VxFlex OS" * Revert "Fix VxFlexOs KeyError after upgrade" * Revert "Verify WWN of connected iSCSI devices if passed" * Remove trailing newline character in UUID * Fix VxFlexOs KeyError after upgrade * Remove py35 from setup.cfg 2.8.0 ----- * Drop py35 jobs * add python 3.7 unit test job * Fix get keyring content failed when ceph auth disabled * rename ScaleIO connector to VxFlex OS * Py3: Fix invalid escape sequencees * Fix FC case sensitive scanning * Make sure looping calls are properly mocked * Add slowest test output to end of test run * Handle None value 'inititator\_target\_map' * Don't warn on missing dmidecode * iSCSI: log exception if portals not found * VMware: Detach backing vmdk during disconnect * Verify WWN of connected iSCSI devices if passed * Add missing params in NoOpEncryptor * Update hacking version * Add retry to \`nvme connect\` in nvme connector 2.7.0 ----- * Support RSD scenario of nvme connector * Remove time checks from test\_custom\_execute\_timeout\_\* tests * Fix create ceph conf failed when cephx disable * Tests: Fix PrivRootwrapTestCase failure * Change openstack-dev to openstack-discuss * Fix NFS "already mounted" detection * Windows SMBFS: fix using share subdirs 2.6.2 ----- * removing older python version 3.4 from setup.cfg * Context manager to handle shared\_targets * Fix a spelling mistake * Fix a spelling mistake * Retry executing command "nvme list" when fail * Remove unused connection properties * Improve VolumePathsNotFound message details 2.6.1 ----- * Add LIO barbican tests to .zuul.yaml * Succeed on iSCSI detach when path just went down * Remove meanless debug log 2.6.0 ----- * 'iscsiadm -m session' failure handling * The validation of iscsi session should be case insensitive * Improve docstrings * Optimize FC device checking * Ignore volume disconnect if it is not connected * Fix spelling mistakes * Cleanup Zuul config file * add lib-forward-testing-python3 test job * add python 3.6 unit test job * switch documentation job to new PTI * import zuul job settings from project-config * Improve detection of multipathd running * Improve iSCSI device detection speed * Replace assertRaisesRegexp with assertRaisesRegex * Add staticmethod decorator in InitiatorConnector * Modify the verification in RBDConnector * Fix multipath disconnect with path failure * Tests: Add unit tests for nfs mount race * Update reno for stable/rocky 2.5.3 ----- * FC Allow for multipath volumes with different LUNs * Windows SMBFS: avoid mounting local shares by default * Remove testrepository * RemoteFS: don't fail in do\_mount if already mounted * Add release note link in README 2.5.2 ----- * Handle multiple errors in multipath -l parsing 2.5.1 ----- * fix tox python3 overrides * Switch to using stestr * FC fix for scanning only connected HBA's 2.5.0 ----- * Trivial: Update pypi url to new url * adding sheepdog connector for PPC64 * Fix FC: Only scan connected HBAs * add lower-constraints job * Include "nqn." in subsystem name * add a getter for connector mapping * uncap eventlet * Fix bindep for multipath * Updated from global requirements 2.4.0 ----- * Accept ISCSI\_ERR\_NO\_OBJS\_FOUND from iscsiadm * Incorporate the connection\_properties input for PPC64 * Adding support to extend attached ScaleIO volumes * Windows iSCSI: ensure disks are claimed by MPIO * Updated from global requirements * Updated from global requirements * Updated from global requirements * Windows FC: fix disk scan issues * Enable hacking-extensions H204, H205 * Updated from global requirements * Update reno for stable/queens * s390x fc: Fix device path for Ubuntu with ds8k 2.3.0 ----- * adding iSER connector for PPC64 * adding VERITAS\_HYPERSCALE connector for PPC64 * Updated from global requirements * adding VZSTORAGE connector for PPC64 * Updated from global requirements * Update supported transports for iscsi connector * Remove the unnecessary pv\_list assign during LVM object init 2.2.0 ----- * Remove requirement on oslo.serialization * Cleanup test-requirements * Updated from global requirements * Windows SMBFS: allow mounting vhd/x images * Windows: fix connectors 'disconnect\_volume' signature * Avoid tox\_install.sh for constraints support * Recover node.startup values after discovering * Add the StorPool brick connector 2.1.1 ----- * set vg\_thin\_pool\_size to float type * Fix a typographical error in a release notes entry 2.1.0 ----- * Make close on cryptsetup volumes idempotent * Make close on luks volumes idempotent * Remove setting of version/release from releasenotes * Adding NVMEoF for initiator CLI * Updated from global requirements * Fixing FC scanning * Updated from global requirements * Always set ignoreskipactivation on snapshot creation 2.0.0 ----- * Enable Python hash randomization for tests * Remove legacy connector constants * Add .stestr.conf configuration * Fix \_remove\_scsi\_symlinks\_no\_links test * Protect against race within os.path.realpath * rescan fails for hba missing target wwn * Updated from global requirements * Updated from global requirements * Fix vmware migrate available volume bug * FC PPC64 device discovery issue * Add attribute 'name' to class RBDVolume * Updated from global requirements * Updated from global requirements * Fix iSCSI volume attachment over RDMA transport * doc: Restructure docs for doc-migration * doc: Remove cruft from conf.py * doc: Switch from oslosphinx to openstackdocstheme * Update reno for stable/pike * Fix ISCSIConnector.\_get\_potential\_volume\_paths logic * FC connector logs number of attempts incorrectly * Enable some off-by-default checks * Update and replace http with https for doc links * Get the right portal from output of iscsiadm command * Update and optimize documentation links * Updated from global requirements * Add client connect exception unit test for rbd 1.15.1 ------ * Don't obscure logs on iSCSI sendtargets failure * Return WWN in multipath\_id * Return symlinks for encrypted volumes 1.15.0 ------ * Revert "Don't use ignoreskipactivation for thin LVM" * Don't use ignoreskipactivation for thin LVM * Fix iSCSI cleanup fix on discovery backends * Fix manual scan for discovery type backends * Fix ceph incremental backup fail * Updated from global requirements 1.14.0 ------ * iSCSI multipath: improve logging on connect * Fix iSCSI cleanup issue when using discovery * Updated from global requirements * Add open-iscsi manual scan support * Refactor iSCSI connect * Fix slow test\_connect\_volume\_device\_not\_valid test * Add libssl to bindep * Refactor iSCSI disconnect * Updated from global requirements * Force LUN\_ID to an int 1.13.1 ------ * Fix supported connectors for Power platform * Removed invalid comments in tox.ini [flake8] * Stop ignoring H904 hacking rule in tox * Stop ignoring H405 hacking rule in tox * Stop ignoring E265 pycodestyle rule in tox * Stop ignoring E123 and E125 pycodestyle rules * Update hacking version to align with Cinder * Fixed the veritas connector path * Change code to be more Pythonic 1.13.0 ------ * Return correct device path from Veritas connector * Prevent rbd map again if it's already mapped * Check host device alive before multipath id discovery * Updated from global requirements * Changed way of providing RBD keyring from keyring\_path to client token * encryptors: Delay removal of legacy provider names * Change log level on \_get\_hba\_channel\_scsi\_target * Veritas os-brick connector should use privsep * Adding support for FibreChannelConnector for PPC64 * Updated from global requirements 1.12.0 ------ * Fixed generated temp file problem for RBD backend * Replace random uuid with fake uuid in unit tests * Updated from global requirements * Mask logging of connection info for iSCSI connector * Include identity information in rbd commands * os-brick connector for Veritas HyperScale * Move vzstorage related code out of RemoteFsClient * Updated from global requirements * RBD: consider a custom keyring in connection info * Add Ocata release notes page * Using assertIsNone(x) instead of assertEqual(None, x) * Remove log translations * Updated from global requirements * Fix iSCSI multipath rescan * Retry multipath flush when map is in use * Fix multipath flush when using friendly names * Fix unittest run on s390x host 1.11.0 ------ * Updated from global requirements * Encryptors: Fix compat with Nova encryptors for Ocata * Add Python 3.5 classifier and venv 1.10.0 ------ * Fix a wrong indentation * s390 FC device path fix for Ubuntu 1.9.0 ----- * Updated from global requirements * encryptors: Introduce encryption provider constants * Add debug to tox environment * Windows connectors: add device\_scan\_interval arg * Add curl to bindep * Removes unnecessary utf-8 encoding * Replace assertDictMatch with assertDictEqual * Add Constraints support and missing bindep.txt dependencies * Move castellan to test-reqs * Fix import method to follow community guideline * Remove the duplicate calls to rescan * Code cleanup in initiator/linuxfc.py * Updated from global requirements * linuxfc: log path when HBA not found * RBD: ensure temporary config gets deleted * os-brick: Add bindep support * Show team and repo badges on README * Updated from global requirements * Add developer docs url in README.rst(trivial) * encryptors: Workaround mangled passphrases * encryptors: Mock os\_brick.executor correctly * RBD: enclose ipv6 addresses in square brackets * Updated from global requirements * Mask passwords in utils.trace for func params 1.8.0 ----- * Updated from global requirements * Raise specific exception for an invalid protocol connector * Updated from global requirements * Multipath device keeps old size when extending volume * Updated from global requirements * Delete deprecated Hacking in tox.ini 1.7.0 ----- * Delete MANIFEST.in in os-brick * Drop py33 support * Windows remotefs: create mountpoints at the expected location * linuxrbd: remove obsolete comment on close() * Enable release notes translation * Detect if Fibre Channel support exists * Close connection to ceph after cinder bakcup * Updated from global requirements * Updated from global requirements * Replace 'assertTrue(a not in b)' with 'assertNotIn(a, b)' * s390x iscsi support enablement * Docstrings should not start with a space * Use assertEqual() instead of assertDictEqual() * Stop calling multipath -r when attaching/detaching iSCSI volumes * DISCO: Log init message as debug * Change warning to info logging for connected volume rescans * standardize release note page ordering * Mock time.sleep for test\_lv\_deactivate\_timeout test * Update reno for stable/newton * Change assertTrue(isinstance()) with optimal assert * Remove self.\_\_dict\_\_ for formatting strings * Create connector aliases to the new connectors refactor * TrivialFix: Remove logging import unused 1.6.0 ----- * Fix cmd execution stderr, stdout unicode errors * Mask out passwords when tracing * RBD: Fix typo in rados timeout assignment * Fixes with customized ceph cluster name * Add retries to iSCSI connect\_volume * Add connector for GPFS volumes * Add missing %s in print message * Fix linuxrbd to work with Python 3 * Add tracing unit tests * Wrong param makes exception message throws inaccurate * Fix the typo in the file * Add connector for vmdk volumes * Fix iSCSI discovery with ISER transport * RemoteFsClient extend Executor * Add Windows Fibre Channel connector * Add Windows SMBFS connector * Fix FC multipath cleanup * Fix weak test\_vzstorage\_with\_mds\_list * Fix the mocking mess * Fix FC multipath rescan * Update the home-page info with the developer documentation * Splitting Out Connectors from connector.py * Remove race condition from lvextend 1.5.0 ----- * Updated from global requirements * Mock write and read operations to filesystem * Local attach feature in RBD connector * Remove useless info logging in check\_valid\_device * ScaleIO to get volume name from connection properties * Add ignore for . directories * Upgrade tox to 2.0 * Add trace facility * Fix string interpolation to delayed to be handled by the logging code * Replace assertEqual(None, \*) with assertIsNone in tests * Fix wrong path used in iscsi "multipath -l" * Updated from global requirements * Remove unused LOG to keep code clean * Fix multipath iSCSI encrypted volume attach failure * Updated from global requirements * release note for windows iSCSI * Add Windows iSCSI connector * Make code line length less than 79 characters * Updated from global requirements * Replace ip with portal to express more accurately * Fix argument order for assertEqual to (expected, observed) * Add fast8 to quickly test pep8 changes * Make RBDImageMetadata and RBDVolumeIOWrapper re-usable 1.4.0 ----- * Copy encryptors from Nova to os-brick * Disconnect multipath iscsi may logout session * Add support for processutils.execute * Updated from global requirements * Mock time.sleep in ISCSIConnectorTestCase * Updated from global requirements * Updated from global requirements * Updated from global requirements * Ensure that the base connector is platform independent * Updated from global requirements * os-brick refactor get\_connector\_properties * Handle exception case with only target\_portals * Retire ISERConnector from documentation * LVM: Create thin pool with 100%FREE * Fix coverage generation * Trivial rootwrap -> privsep replacement * Updated from global requirements * Updated from global requirements 1.3.0 ----- * LVM: Call supports\_thin\_provisioning as static * Add pylint tox env * Don't use oslo-incubator stuff * Update reno for stable/mitaka * Replace \_get\_multipath\_device\_name with \_discover\_mpath\_device * Fixes get\_all\_available\_volumes return value * Updated from global requirements * Fix Scality SOFS support * Actually run the RemoteFSClient unit tests * Mock time.sleep() in 3 unit tests 1.1.0 ----- * Fix setting the multipath\_id * Updated from global requirements * Add sheepdog support * Include multipath -ll output in failed to parse warning 1.0.0 ----- * Fix iSCSI Multipath * Add missing release notes * Lun id's > 255 should be converted to hex * Updated from global requirements * Fix output returned from get\_all\_available\_volumes * Raise exception in find\_multipath\_device * Updated from global requirements * Remove multipath -l logic from ISCSI connector * Add vzstorage protocol for remotefs connections * Add reno for release notes management * Fix get\_device\_size with newlines * Updated from global requirements 0.8.0 ----- * Add connector for ITRI DISCO cinder driver * os-brick add extend\_volume API * os-brick add cinder local\_dev lvm code * Revert "Use assertTrue/False instead of assertEqual(T/F)" * Fix another unit test failure * Use assertTrue/False instead of assertEqual(T/F) * Actually log the command used in \_run\_iscsiadm * Updated from global requirements * remove python 2.6 trove classifier 0.7.0 ----- * DRBD connector class * Updated from global requirements * Deprecated tox -downloadcache option removed * ScaleIO could connect wrong volume to VM * Allow RBDClient to be used from a with-statement * Updated from global requirements * Remove brackets from portal * Minor documentation fixes for the method parameters 0.6.0 ----- * Add requests to project requirements * Add quobyte protocol for remotefs connections * Correct a log message * Brick add param documentation to connectors * Updated from global requirements * Multipath Device Action Being Parsed as Name * Fix iopsLimit parameter in ScaleIO connector * Parse FCoE sysfs device paths * Add new Connector APIs for path validation * Updated from global requirements * Fix test\_connect\_volume when skip is bypassed * Fetch and return SCSI WWN * Update minimum tox version to 1.8 * Updated from global requirements * Wait for FC multipath devices to become writable * Check RBDConnector.disconnect\_volume device\_info argument * Updated from global requirements * Fix silent iSCSI login failures * Change os-brick to use ostestr * Updated from global requirements * Fix iSCSI multipath cleanup * Removed use of deprecated LOG.warn * Fix typo in vgc-cluster command in rootwrap file 0.5.0 ----- * Change ignore-errors to ignore\_errors * Updated from global requirements * Add fancy pypi version and download images * iSCSI fix misleading Log warning on connect fail * Fix missing value types for log message * Log a message when can’t find multipath device * Removed unused dependency: discover * Use 'device' instead of 'volume\_path' 0.4.0 ----- * Add support for --interface option in iscsiadm * FC Stop calling multipath command line * Updated from global requirements * Add rootwrap filters * Handle FC LUN IDs greater 255 correctly on s390x architectures * Fix incorrect comments in FibreChannelConnector * Adding CHAP discovery logic to os-brick * Updated from global requirements * Remove the iSCSI rescan during disconnect * Remotefs: add ScalityFS support * Updated from global requirements * Updated from global requirements * Change SCSI device removal backoff rate * Changed connector protocols to use constants * Updated from global requirements * Fix race in check and access of /dev/disk/by-path/ * Updated from global requirements 0.3.2 ----- * remotefs: add virtuozzo storage support * Perform port\_rescan on s390x platforms * FC discover existing devices for removal 0.3.1 ----- * Use pbr's automatically generated changelog 0.3.0 ----- * Updated from global requirements * Updated from global requirements * Update changelog to 0.3.0 being latest * Fix mock==1.1.0 break unit tests * Cleanup Python 3 changes * Prep for 0.2.1 release * Add connector driver for the ScaleIO cinder driver * Added ABCMeta class to the InitiatorConnector * Remove unused oslo incubator files * update os-brick to pass python3 tests * Updated from global requirements * FC Eliminate the need to return devices list * Switch to oslo.service * Add RBD connector * Add HGST Solutions connector * Support host type specific block volume attachment * Updated from global requirements * optimize multipath call to identify IQN * Updated from global requirements * Trivial exception parameter name fix for Huawei * Fix connecting unnecessary iSCSI sessions issue * Fix disconnecting necessary iSCSI sessions issue * Add retry to iSCSI delete * Updated from global requirements * Add missing connectors to factory test * Fix local connector test case inheritance 0.2.0 ----- * Allow overriding the host field * Assign the platform after declaration * Added a unit test for masking iscsiadm passwords * Preparing for the 0.1.1 release * ISCSI be careful parsing iscsiadm output * Updated from global requirements * Drop use of 'oslo' namespace package 0.1.0 ----- * Update README to work with release tools * Brick: Fix race in removing iSCSI device * Update os-brick requirements * Mask passwords with iscsiadm commands * Sync latest \_i18n module for os\_brick * Use oslo\_log instead of openstack.common.log * Sync loopingcall from oslo-incubator for os-brick * Fix wrong command for \_rescan\_multipath * Fix multipath device discovery when UFN is enabled * Use six.text\_type instead of unicode * Fix missing translations for log messages * Remove error messages from multipath command output before parsing * Remove mocks after each unit test finished * Correct project name in .gitreview * Adjust os-brick to support FCP on System z systems * Use target\_portals/iqns/luns for alternative target information * Fix comments style according to Hacking rules * Update the documentation for os-brick * Failover to alternative iSCSI portals on login failure * Remove some unused exceptions from Cinder * Brick os-brick up to par with cinder brick * renamed the project to os-brick * Created the Brick library from Cinder ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/HACKING.rst0000664000175000017500000000020400000000000015050 0ustar00zuulzuul00000000000000brick Style Commandments ======================== Read the OpenStack Style Commandments https://docs.openstack.org/hacking/latest/ ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/LICENSE0000664000175000017500000002363700000000000014276 0ustar00zuulzuul00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/PKG-INFO0000664000175000017500000000510700000000000014356 0ustar00zuulzuul00000000000000Metadata-Version: 1.1 Name: os-brick Version: 3.0.8 Summary: OpenStack Cinder brick library for managing local volume attaches Home-page: https://docs.openstack.org/os-brick/ Author: OpenStack Author-email: openstack-discuss@lists.openstack.org License: UNKNOWN Description: ======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on ===== brick ===== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ Release notes for the project can be found at: https://docs.openstack.org/releasenotes/os-brick * License: Apache License, Version 2.0 * Source: https://opendev.org/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/README.rst0000664000175000017500000000270700000000000014753 0ustar00zuulzuul00000000000000======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on ===== brick ===== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ Release notes for the project can be found at: https://docs.openstack.org/releasenotes/os-brick * License: Apache License, Version 2.0 * Source: https://opendev.org/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/babel.cfg0000664000175000017500000000002100000000000014775 0ustar00zuulzuul00000000000000[python: **.py] ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/bindep.txt0000664000175000017500000000263400000000000015265 0ustar00zuulzuul00000000000000# This is a cross-platform list tracking distribution packages needed for # install and tests # see https://docs.openstack.org/infra/bindep/ for additional information. curl device-mapper-multipath [platform:rpm] multipath-tools [platform:dpkg] sg3-utils [platform:dpkg] sg3_utils [platform:rpm] libxml2-devel [platform:rpm] libxml2-dev [platform:dpkg] libxslt-devel [platform:rpm] libxslt1-dev [platform:dpkg] libssl-dev [platform:dpkg] openssl-devel [platform:rpm !platform:suse] libopenssl-devel [platform:suse !platform:rpm] # Binary dependencies for PDF doc generation fonts-liberation [doc platform:dpkg] texlive-latex-base [doc platform:dpkg] texlive-latex-extra [doc platform:dpkg] texlive-xetex [doc platform:dpkg] texlive-fonts-recommended [doc platform:dpkg] xindy [doc platform:dpkg] latexmk [doc platform:dpkg] texlive [doc platform:rpm] texlive-fncychap [doc platform:rpm] texlive-titlesec [doc platform:rpm] texlive-tabulary [doc platform:rpm] texlive-framed [doc platform:rpm] texlive-wrapfig [doc platform:rpm] texlive-upquote [doc platform:rpm] texlive-capt-of [doc platform:rpm] texlive-needspace [doc platform:rpm] texlive-polyglossia [doc platform:rpm] texlive-xetex [doc platform:rpm] texlive-xindy [doc platform:rpm] texlive-courier [doc platform:rpm] latexmk [doc platform:rpm] python3-sphinxcontrib-svg2pdfconverter-common [doc platform:rpm] librsvg2-tools [doc platform:rpm] librsvg2-bin [doc platform:dpkg] ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/0000775000175000017500000000000000000000000014023 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/requirements.txt0000664000175000017500000000070500000000000017311 0ustar00zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. openstackdocstheme>=1.20.0 # Apache-2.0 reno>=2.5.0 # Apache-2.0 sphinx!=1.6.6,!=1.6.7,!=2.1.0,>=1.6.3 # BSD os-api-ref>=1.4.0 # Apache-2.0 sphinxcontrib-apidoc>=0.2.0 # BSD sphinx-feature-classification>=0.1.0 # Apache 2.0 mock>=2.0.0 # BSD ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/0000775000175000017500000000000000000000000015323 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/conf.py0000664000175000017500000000561300000000000016627 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', 'reno.sphinxext', 'openstackdocstheme', ] # The master toctree document. master_doc = 'index' # General information about the project. copyright = u'2015, OpenStack Foundation' # If true, '()' will be appended to :func: etc. cross-reference text. add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). add_module_names = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. html_theme = 'openstackdocs' # -- Options for openstackdocstheme ------------------------------------------- repository_name = 'openstack/os-brick' bug_project = 'os-brick' bug_tag = '' # -- Options for LaTeX output ------------------------------------------------- # The paper size ('letter' or 'a4'). # latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). # latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', 'doc-os-brick.tex', u'OS Brick Documentation', 'Cinder Contributors', 'manual') ] # The name of an image file (relative to this directory) to place at the top of # the title page. # latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. # latex_use_parts = False # Additional stuff for the LaTeX preamble. # latex_preamble = '' # Documents to append as an appendix to all manuals. # latex_appendices = [] # If false, no module index is generated. # latex_use_modindex = True # Disable usage of xindy https://bugzilla.redhat.com/show_bug.cgi?id=1643664 latex_use_xindy = False latex_domain_indices = False latex_elements = { 'makeindex': '', 'printindex': '', 'preamble': r'\setcounter{tocdepth}{3}', } latex_additional_files = ['os_brick.sty'] ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/contributor/0000775000175000017500000000000000000000000017675 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/contributor/contributing.rst0000664000175000017500000000130200000000000023132 0ustar00zuulzuul00000000000000============================ So You Want to Contribute... ============================ For general information on contributing to OpenStack, please check out the `contributor guide `_ to get started. It covers all the basics that are common to all OpenStack projects: the accounts you need, the basics of interacting with our Gerrit review system, how we communicate as a community, etc. The os-brick library is maintained by the OpenStack Cinder project. To understand our development process and how you can contribute to it, please look at the Cinder project's general contributor's page: http://docs.openstack.org/cinder/latest/contributor/contributing.html ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/index.rst0000664000175000017500000000070400000000000017165 0ustar00zuulzuul00000000000000======== os-brick ======== `os-brick` is a Python package containing classes that help with volume discovery and removal from a host. Installation Guide ------------------ .. toctree:: :maxdepth: 2 install/index Usage Guide ----------- .. toctree:: :maxdepth: 2 user/tutorial Reference --------- .. toctree:: :maxdepth: 2 reference/index Contributing ------------ .. toctree:: :maxdepth: 2 contributor/contributing ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/install/0000775000175000017500000000000000000000000016771 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/install/index.rst0000664000175000017500000000057000000000000020634 0ustar00zuulzuul00000000000000============ Installation ============ At the command line: .. code-block:: shell $ pip install os-brick Or, if you have virtualenvwrapper installed: .. code-block:: shell $ mkvirtualenv os-brick $ pip install os-brick Or, from source: .. code-block:: shell $ git clone https://opendev.org/openstack/os-brick $ cd os-brick $ python setup.py install ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/reference/0000775000175000017500000000000000000000000017261 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/reference/index.rst0000664000175000017500000000035300000000000021123 0ustar00zuulzuul00000000000000API Documentation ================= The **os-brick** package provides the ability to collect host initiator information as well as discovery volumes and removal of volumes from a host. .. toctree:: :maxdepth: 2 os_brick/index ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/reference/os_brick/0000775000175000017500000000000000000000000021054 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/reference/os_brick/exception.rst0000664000175000017500000000112300000000000023601 0ustar00zuulzuul00000000000000:mod:`exception` -- Exceptions ============================== .. automodule:: os_brick.exception :synopsis: Exceptions generated by os-brick .. autoclass:: os_brick.exception.BrickException .. autoclass:: os_brick.exception.NotFound .. autoclass:: os_brick.exception.Invalid .. autoclass:: os_brick.exception.InvalidParameterValue .. autoclass:: os_brick.exception.NoFibreChannelHostsFound .. autoclass:: os_brick.exception.NoFibreChannelVolumeDeviceFound .. autoclass:: os_brick.exception.VolumeDeviceNotFound .. autoclass:: os_brick.exception.ProtocolNotSupported ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/reference/os_brick/index.rst0000664000175000017500000000034400000000000022716 0ustar00zuulzuul00000000000000:mod:`os_brick` -- OpenStack Brick library ========================================== .. automodule:: os_brick :synopsis: OpenStack Brick library Sub-modules: .. toctree:: :maxdepth: 2 initiator/index exception ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/reference/os_brick/initiator/0000775000175000017500000000000000000000000023056 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/reference/os_brick/initiator/connector.rst0000664000175000017500000000171000000000000025601 0ustar00zuulzuul00000000000000:mod:`connector` -- Connector ============================= .. automodule:: os_brick.initiator.connector :synopsis: Connector module for os-brick .. autoclass:: os_brick.initiator.connector.InitiatorConnector .. automethod:: factory .. autoclass:: os_brick.initiator.connector.ISCSIConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.FibreChannelConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.AoEConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.LocalConnector .. automethod:: connect_volume .. automethod:: disconnect_volume .. autoclass:: os_brick.initiator.connector.HuaweiStorHyperConnector .. automethod:: connect_volume .. automethod:: disconnect_volume ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/reference/os_brick/initiator/index.rst0000664000175000017500000000027200000000000024720 0ustar00zuulzuul00000000000000:mod:`initiator` -- Initiator ============================= .. automodule:: os_brick.initiator :synopsis: Initiator module Sub-modules: .. toctree:: :maxdepth: 2 connector ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/doc/source/user/0000775000175000017500000000000000000000000016301 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/doc/source/user/tutorial.rst0000664000175000017500000000221000000000000020671 0ustar00zuulzuul00000000000000======== Tutorial ======== This tutorial is intended as an introduction to working with **os-brick**. Prerequisites ------------- Before we start, make sure that you have the **os-brick** distribution :doc:`installed `. In the Python shell, the following should run without raising an exception: .. code-block:: bash >>> import os_brick Fetch all of the initiator information from the host ---------------------------------------------------- An example of how to collect the initiator information that is needed to export a volume to this host. .. code-block:: python from os_brick.initiator import connector # what helper do you want to use to get root access? root_helper = "sudo" # The ip address of the host you are running on my_ip = "192.168.1.1" # Do you want to support multipath connections? multipath = True # Do you want to enforce that multipath daemon is running? enforce_multipath = False initiator = connector.get_connector_properties(root_helper, my_ip, multipath, enforce_multipath) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/etc/0000775000175000017500000000000000000000000014031 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/etc/os-brick/0000775000175000017500000000000000000000000015542 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1446776 os-brick-3.0.8/etc/os-brick/rootwrap.d/0000775000175000017500000000000000000000000017641 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/etc/os-brick/rootwrap.d/os-brick.filters0000664000175000017500000000072700000000000022752 0ustar00zuulzuul00000000000000# os-brick command filters # This file should be owned by (and only-writeable by) the root user [Filters] # privileged/__init__.py: priv_context.PrivContext(default) # This line ties the superuser privs with the config files, context name, # and (implicitly) the actual python code invoked. privsep-rootwrap: RegExpFilter, privsep-helper, root, privsep-helper, --config-file, /etc/(?!\.\.).*, --privsep_context, os_brick.privileged.default, --privsep_sock_path, /tmp/.* ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/lower-constraints.txt0000664000175000017500000000270600000000000017521 0ustar00zuulzuul00000000000000alabaster==0.7.10 appdirs==1.3.0 asn1crypto==0.23.0 Babel==2.3.4 castellan==0.16.0 cffi==1.7.0 cliff==2.8.0 cmd2==0.8.0 coverage==4.0 cryptography==2.1 ddt==1.0.1 debtcollector==1.2.0 docutils==0.11 dulwich==0.15.0 eventlet==0.18.2 extras==1.0.0 fasteners==0.7.0 fixtures==3.0.0 future==0.16.0 greenlet==0.4.10 idna==2.6 imagesize==0.7.1 iso8601==0.1.11 Jinja2==2.10 keystoneauth1==3.4.0 linecache2==1.0.0 lxml==3.4.1 MarkupSafe==1.0 mccabe==0.6.0 mock==2.0.0 monotonic==0.6 mox3==0.20.0 msgpack-python==0.4.0 netaddr==0.7.18 netifaces==0.10.4 openstackdocstheme==1.20.0 os-client-config==1.28.0 os-testr==1.0.0 os-win==3.0.0 oslo.concurrency==3.26.0 oslo.config==5.2.0 oslo.context==2.19.2 oslo.i18n==3.15.3 oslo.log==3.36.0 oslo.privsep==1.32.0 oslo.serialization==2.18.0 oslo.service==1.24.0 oslo.utils==3.33.0 oslo.vmware==2.17.0 oslotest==3.2.0 Paste==2.0.2 PasteDeploy==1.5.0 pbr==2.0.0 prettytable==0.7.2 pycparser==2.18 Pygments==2.2.0 pyinotify==0.9.6 pyparsing==2.1.0 pyperclip==1.5.27 python-barbicanclient==4.5.2 python-dateutil==2.5.3 python-mimeparse==1.6.0 python-subunit==1.0.0 pytz==2013.6 PyYAML==3.12 reno==2.5.0 repoze.lru==0.7 requests==2.14.2 requestsexceptions==1.2.0 retrying==1.2.3 rfc3986==0.3.1 Routes==2.3.1 six==1.10.0 snowballstemmer==1.2.1 Sphinx==1.6.2 sphinxcontrib-websupport==1.0.1 stestr==1.0.0 stevedore==1.20.0 suds-jurko==0.6 testscenarios==0.4 testtools==2.2.0 traceback2==1.4.0 unittest2==1.1.0 urllib3==1.21.1 WebOb==1.7.1 wrapt==1.7.0 ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1446776 os-brick-3.0.8/os_brick/0000775000175000017500000000000000000000000015051 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/__init__.py0000664000175000017500000000000000000000000017150 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1446776 os-brick-3.0.8/os_brick/encryptors/0000775000175000017500000000000000000000000017261 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/encryptors/__init__.py0000664000175000017500000001214600000000000021376 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import nop from oslo_log import log as logging from oslo_utils import importutils from oslo_utils import strutils LOG = logging.getLogger(__name__) LUKS = "luks" LUKS2 = "luks2" PLAIN = "plain" FORMAT_TO_FRONTEND_ENCRYPTOR_MAP = { LUKS: 'os_brick.encryptors.luks.LuksEncryptor', LUKS2: 'os_brick.encryptors.luks.Luks2Encryptor', PLAIN: 'os_brick.encryptors.cryptsetup.CryptsetupEncryptor' } LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP = { "nova.volume.encryptors.luks.LuksEncryptor": LUKS, "nova.volume.encryptors.cryptsetup.CryptsetupEncryptor": PLAIN, "nova.volume.encryptors.nop.NoopEncryptor": None, "os_brick.encryptors.luks.LuksEncryptor": LUKS, "os_brick.encryptors.cryptsetup.CryptsetupEncryptor": PLAIN, "os_brick.encryptors.nop.NoopEncryptor": None, "LuksEncryptor": LUKS, "CryptsetupEncryptor": PLAIN, "NoOpEncryptor": None, } def get_volume_encryptor(root_helper, connection_info, keymgr, execute=None, *args, **kwargs): """Creates a VolumeEncryptor used to encrypt the specified volume. :param: the connection information used to attach the volume :returns VolumeEncryptor: the VolumeEncryptor for the volume """ encryptor = nop.NoOpEncryptor(root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) location = kwargs.get('control_location', None) if location and location.lower() == 'front-end': # case insensitive provider = kwargs.get('provider') # TODO(lyarwood): Remove the following in Queens and raise an # ERROR if provider is not a key in SUPPORTED_ENCRYPTION_PROVIDERS. # Until then continue to allow both the class name and path to be used. if provider in LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP: LOG.warning("Use of the in tree encryptor class %(provider)s" " by directly referencing the implementation class" " will be blocked in the Queens release of" " os-brick.", {'provider': provider}) provider = LEGACY_PROVIDER_CLASS_TO_FORMAT_MAP[provider] if provider in FORMAT_TO_FRONTEND_ENCRYPTOR_MAP: provider = FORMAT_TO_FRONTEND_ENCRYPTOR_MAP[provider] elif provider is None: provider = "os_brick.encryptors.nop.NoOpEncryptor" else: LOG.warning("Use of the out of tree encryptor class " "%(provider)s will be blocked with the Queens " "release of os-brick.", {'provider': provider}) try: encryptor = importutils.import_object( provider, root_helper, connection_info, keymgr, execute, **kwargs) except Exception as e: LOG.error("Error instantiating %(provider)s: %(exception)s", {'provider': provider, 'exception': e}) raise msg = ("Using volume encryptor '%(encryptor)s' for connection: " "%(connection_info)s" % {'encryptor': encryptor, 'connection_info': connection_info}) LOG.debug(strutils.mask_password(msg)) return encryptor def get_encryption_metadata(context, volume_api, volume_id, connection_info): metadata = {} if ('data' in connection_info and connection_info['data'].get('encrypted', False)): try: metadata = volume_api.get_volume_encryption_metadata(context, volume_id) if not metadata: LOG.warning('Volume %s should be encrypted but there is no ' 'encryption metadata.', volume_id) except Exception as e: LOG.error("Failed to retrieve encryption metadata for " "volume %(volume_id)s: %(exception)s", {'volume_id': volume_id, 'exception': e}) raise if metadata: msg = ("Using volume encryption metadata '%(metadata)s' for " "connection: %(connection_info)s" % {'metadata': metadata, 'connection_info': connection_info}) LOG.debug(strutils.mask_password(msg)) return metadata ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/encryptors/base.py0000664000175000017500000000420500000000000020546 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import abc from os_brick import executor import six @six.add_metaclass(abc.ABCMeta) class VolumeEncryptor(executor.Executor): """Base class to support encrypted volumes. A VolumeEncryptor provides hooks for attaching and detaching volumes, which are called immediately prior to attaching the volume to an instance and immediately following detaching the volume from an instance. This class performs no actions for either hook. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(VolumeEncryptor, self).__init__(root_helper, execute=execute, *args, **kwargs) self._key_manager = keymgr self.encryption_key_id = kwargs.get('encryption_key_id') def _get_key(self, context): """Retrieves the encryption key for the specified volume. :param: the connection information used to attach the volume """ return self._key_manager.get(context, self.encryption_key_id) @abc.abstractmethod def attach_volume(self, context, **kwargs): """Hook called immediately prior to attaching a volume to an instance. """ pass @abc.abstractmethod def detach_volume(self, **kwargs): """Hook called immediately after detaching a volume from an instance. """ pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/encryptors/cryptsetup.py0000664000175000017500000002060200000000000022055 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import array import binascii import os from os_brick.encryptors import base from os_brick import exception from oslo_concurrency import processutils from oslo_log import log as logging from oslo_log import versionutils LOG = logging.getLogger(__name__) class CryptsetupEncryptor(base.VolumeEncryptor): """A VolumeEncryptor based on dm-crypt. This VolumeEncryptor uses dm-crypt to encrypt the specified volume. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(CryptsetupEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) # Fail if no device_path was set when connecting the volume, e.g. in # the case of libvirt network volume drivers. data = connection_info['data'] if not data.get('device_path'): volume_id = data.get('volume_id') or connection_info.get('serial') raise exception.VolumeEncryptionNotSupported( volume_id=volume_id, volume_type=connection_info['driver_volume_type']) # the device's path as given to libvirt -- e.g., /dev/disk/by-path/... self.symlink_path = connection_info['data']['device_path'] # a unique name for the volume -- e.g., the iSCSI participant name self.dev_name = 'crypt-%s' % os.path.basename(self.symlink_path) # NOTE(lixiaoy1): This is to import fix for 1439869 from Nova. # NOTE(tsekiyama): In older version of nova, dev_name was the same # as the symlink name. Now it has 'crypt-' prefix to avoid conflict # with multipath device symlink. To enable rolling update, we use the # old name when the encrypted volume already exists. old_dev_name = os.path.basename(self.symlink_path) wwn = data.get('multipath_id') if self._is_crypt_device_available(old_dev_name): self.dev_name = old_dev_name LOG.debug("Using old encrypted volume name: %s", self.dev_name) elif wwn and wwn != old_dev_name: # FibreChannel device could be named '/dev/mapper/'. if self._is_crypt_device_available(wwn): self.dev_name = wwn LOG.debug("Using encrypted volume name from wwn: %s", self.dev_name) # the device's actual path on the compute host -- e.g., /dev/sd_ self.dev_path = os.path.realpath(self.symlink_path) def _is_crypt_device_available(self, dev_name): if not os.path.exists('/dev/mapper/%s' % dev_name): return False try: self._execute('cryptsetup', 'status', dev_name, run_as_root=True) except processutils.ProcessExecutionError as e: # If /dev/mapper/ is a non-crypt block device (such as a # normal disk or multipath device), exit_code will be 1. In the # case, we will omit the warning message. if e.exit_code != 1: LOG.warning('cryptsetup status %(dev_name)s exited ' 'abnormally (status %(exit_code)s): %(err)s', {"dev_name": dev_name, "exit_code": e.exit_code, "err": e.stderr}) return False return True def _get_passphrase(self, key): """Convert raw key to string.""" return binascii.hexlify(key).decode('utf-8') def _open_volume(self, passphrase, **kwargs): """Open the LUKS partition on the volume using passphrase. :param passphrase: the passphrase used to access the volume """ LOG.debug("opening encrypted volume %s", self.dev_path) # NOTE(joel-coffman): cryptsetup will strip trailing newlines from # input specified on stdin unless --key-file=- is specified. cmd = ["cryptsetup", "create", "--key-file=-"] cipher = kwargs.get("cipher", None) if cipher is not None: cmd.extend(["--cipher", cipher]) key_size = kwargs.get("key_size", None) if key_size is not None: cmd.extend(["--key-size", key_size]) cmd.extend([self.dev_name, self.dev_path]) self._execute(*cmd, process_input=passphrase, check_exit_code=True, run_as_root=True, root_helper=self._root_helper) def _get_mangled_passphrase(self, key): """Convert the raw key into a list of unsigned int's and then a string """ # NOTE(lyarwood): This replicates the methods used prior to Newton to # first encode the passphrase as a list of unsigned int's before # decoding back into a string. This method strips any leading 0's # of the resulting hex digit pairs, resulting in a different # passphrase being returned. encoded_key = array.array('B', key).tolist() return ''.join(hex(x).replace('0x', '') for x in encoded_key) def attach_volume(self, context, **kwargs): """Shadow the device and pass an unencrypted version to the instance. Transparent disk encryption is achieved by mounting the volume via dm-crypt and passing the resulting device to the instance. The instance is unaware of the underlying encryption due to modifying the original symbolic link to refer to the device mounted by dm-crypt. """ # TODO(lyarwood): Remove this encryptor and refactor the LUKS based # encryptors in the U release. versionutils.report_deprecated_feature( LOG, "The plain CryptsetupEncryptor is deprecated and will be removed " "in a future release. Existing users are encouraged to retype " "any existing volumes using this encryptor to the 'luks' " "LuksEncryptor or 'luks2' Luks2Encryptor encryptors as soon as " "possible.") key = self._get_key(context).get_encoded() passphrase = self._get_passphrase(key) try: self._open_volume(passphrase, **kwargs) except processutils.ProcessExecutionError as e: if e.exit_code == 2: # NOTE(lyarwood): Workaround bug#1633518 by attempting to use # a mangled passphrase to open the device.. LOG.info("Unable to open %s with the current passphrase, " "attempting to use a mangled passphrase to open " "the volume.", self.dev_path) self._open_volume(self._get_mangled_passphrase(key), **kwargs) # modify the original symbolic link to refer to the decrypted device self._execute('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self._root_helper, run_as_root=True, check_exit_code=True) def _close_volume(self, **kwargs): """Closes the device (effectively removes the dm-crypt mapping).""" LOG.debug("closing encrypted volume %s", self.dev_path) # NOTE(mdbooth): remove will return 4 (wrong device specified) if # the device doesn't exist. We assume here that the caller hasn't # specified the wrong device, and that it doesn't exist because it # isn't open. We don't fail in this case in order to make this # operation idempotent. self._execute('cryptsetup', 'remove', self.dev_name, run_as_root=True, check_exit_code=[0, 4], root_helper=self._root_helper) def detach_volume(self, **kwargs): """Removes the dm-crypt mapping for the device.""" self._close_volume(**kwargs) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/encryptors/luks.py0000664000175000017500000002367700000000000020630 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import cryptsetup from os_brick.privileged import rootwrap as priv_rootwrap from oslo_concurrency import processutils as putils from oslo_log import log as logging LOG = logging.getLogger(__name__) def is_luks(root_helper, device, execute=None): """Checks if the specified device uses LUKS for encryption. :param device: the device to check :returns: true if the specified device uses LUKS; false otherwise """ try: # check to see if the device uses LUKS: exit status is 0 # if the device is a LUKS partition and non-zero if not if execute is None: execute = priv_rootwrap.execute execute('cryptsetup', 'isLuks', '--verbose', device, run_as_root=True, root_helper=root_helper, check_exit_code=True) return True except putils.ProcessExecutionError as e: LOG.warning("isLuks exited abnormally (status %(exit_code)s): " "%(stderr)s", {"exit_code": e.exit_code, "stderr": e.stderr}) return False class LuksEncryptor(cryptsetup.CryptsetupEncryptor): """A VolumeEncryptor based on LUKS. This VolumeEncryptor uses dm-crypt to encrypt the specified volume. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(LuksEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) def _format_volume(self, passphrase, **kwargs): """Creates a LUKS v1 header on the volume. :param passphrase: the passphrase used to access the volume """ self._format_luks_volume(passphrase, 'luks1', **kwargs) def _format_luks_volume(self, passphrase, version, **kwargs): """Creates a LUKS header of a given version or type on the volume. :param passphrase: the passphrase used to access the volume :param version: the LUKS version or type to use: one of `luks`, `luks1`, or `luks2`. Be aware that `luks` gives you the default LUKS format preferred by the particular cryptsetup being used (depends on version and compile time parameters), which could be either LUKS1 or LUKS2, so it's better to be specific about what you want here """ LOG.debug("formatting encrypted volume %s", self.dev_path) # NOTE(joel-coffman): cryptsetup will strip trailing newlines from # input specified on stdin unless --key-file=- is specified. cmd = ["cryptsetup", "--batch-mode", "luksFormat", "--type", version, "--key-file=-"] cipher = kwargs.get("cipher", None) if cipher is not None: cmd.extend(["--cipher", cipher]) key_size = kwargs.get("key_size", None) if key_size is not None: cmd.extend(["--key-size", key_size]) cmd.extend([self.dev_path]) self._execute(*cmd, process_input=passphrase, check_exit_code=True, run_as_root=True, root_helper=self._root_helper, attempts=3) def _open_volume(self, passphrase, **kwargs): """Opens the LUKS partition on the volume using passphrase. :param passphrase: the passphrase used to access the volume """ LOG.debug("opening encrypted volume %s", self.dev_path) self._execute('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=passphrase, run_as_root=True, check_exit_code=True, root_helper=self._root_helper) def _unmangle_volume(self, key, passphrase, **kwargs): """Workaround for bug#1633518 First identify if a mangled passphrase is used and if found then replace with the correct unmangled version of the passphrase. """ mangled_passphrase = self._get_mangled_passphrase(key) self._open_volume(mangled_passphrase, **kwargs) self._close_volume(**kwargs) LOG.debug("%s correctly opened with a mangled passphrase, replacing " "this with the original passphrase", self.dev_path) # NOTE(lyarwood): Now that we are sure that the mangled passphrase is # used attempt to add the correct passphrase before removing the # mangled version from the volume. # luksAddKey currently prompts for the following input : # Enter any existing passphrase: # Enter new passphrase for key slot: # Verify passphrase: self._execute('cryptsetup', 'luksAddKey', self.dev_path, '--force-password', process_input=''.join([mangled_passphrase, '\n', passphrase, '\n', passphrase]), run_as_root=True, check_exit_code=True, root_helper=self._root_helper) # Verify that we can open the volume with the current passphrase # before removing the mangled passphrase. self._open_volume(passphrase, **kwargs) self._close_volume(**kwargs) # luksRemoveKey only prompts for the key to remove. self._execute('cryptsetup', 'luksRemoveKey', self.dev_path, process_input=mangled_passphrase, run_as_root=True, check_exit_code=True, root_helper=self._root_helper) LOG.debug("%s mangled passphrase successfully replaced", self.dev_path) def attach_volume(self, context, **kwargs): """Shadow the device and pass an unencrypted version to the instance. Transparent disk encryption is achieved by mounting the volume via dm-crypt and passing the resulting device to the instance. The instance is unaware of the underlying encryption due to modifying the original symbolic link to refer to the device mounted by dm-crypt. """ key = self._get_key(context).get_encoded() passphrase = self._get_passphrase(key) try: self._open_volume(passphrase, **kwargs) except putils.ProcessExecutionError as e: if e.exit_code == 1 and not is_luks(self._root_helper, self.dev_path, execute=self._execute): # the device has never been formatted; format it and try again LOG.info("%s is not a valid LUKS device;" " formatting device for first use", self.dev_path) self._format_volume(passphrase, **kwargs) self._open_volume(passphrase, **kwargs) elif e.exit_code == 2: # NOTE(lyarwood): Workaround bug#1633518 by replacing any # mangled passphrases that are found on the volume. # TODO(lyarwood): Remove workaround during R. LOG.warning("%s is not usable with the current " "passphrase, attempting to use a mangled " "passphrase to open the volume.", self.dev_path) self._unmangle_volume(key, passphrase, **kwargs) self._open_volume(passphrase, **kwargs) else: raise # modify the original symbolic link to refer to the decrypted device self._execute('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self._root_helper, run_as_root=True, check_exit_code=True) def _close_volume(self, **kwargs): """Closes the device (effectively removes the dm-crypt mapping).""" LOG.debug("closing encrypted volume %s", self.dev_path) # NOTE(mdbooth): luksClose will return 4 (wrong device specified) if # the device doesn't exist. We assume here that the caller hasn't # specified the wrong device, and that it doesn't exist because it # isn't open. We don't fail in this case in order to make this # operation idempotent. self._execute('cryptsetup', 'luksClose', self.dev_name, run_as_root=True, check_exit_code=[0, 4], root_helper=self._root_helper, attempts=3) class Luks2Encryptor(LuksEncryptor): """A VolumeEncryptor based on LUKS v2. This VolumeEncryptor uses dm-crypt to encrypt the specified volume. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(Luks2Encryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) def _format_volume(self, passphrase, **kwargs): """Creates a LUKS v2 header on the volume. :param passphrase: the passphrase used to access the volume """ self._format_luks_volume(passphrase, 'luks2', **kwargs) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/encryptors/nop.py0000664000175000017500000000300700000000000020427 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import base class NoOpEncryptor(base.VolumeEncryptor): """A VolumeEncryptor that does nothing. This class exists solely to wrap regular (i.e., unencrypted) volumes so that they do not require special handling with respect to an encrypted volume. This implementation performs no action when a volume is attached or detached. """ def __init__(self, root_helper, connection_info, keymgr, execute=None, *args, **kwargs): super(NoOpEncryptor, self).__init__( root_helper=root_helper, connection_info=connection_info, keymgr=keymgr, execute=execute, *args, **kwargs) def attach_volume(self, context, **kwargs): pass def detach_volume(self, **kwargs): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/exception.py0000664000175000017500000001610500000000000017424 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Exceptions for the Brick library.""" from oslo_concurrency import processutils as putils import six import traceback from os_brick.i18n import _ from oslo_log import log as logging LOG = logging.getLogger(__name__) class BrickException(Exception): """Base Brick Exception To correctly use this class, inherit from it and define a 'msg_fmt' property. That msg_fmt will get printf'd with the keyword arguments provided to the constructor. """ message = _("An unknown exception occurred.") code = 500 headers = {} safe = False def __init__(self, message=None, **kwargs): self.kwargs = kwargs if 'code' not in self.kwargs: try: self.kwargs['code'] = self.code except AttributeError: pass if not message: try: message = self.message % kwargs except Exception: # kwargs doesn't match a variable in the message # log the issue and the kwargs LOG.exception("Exception in string format operation. " "msg='%s'", self.message) for name, value in kwargs.items(): LOG.error("%(name)s: %(value)s", {'name': name, 'value': value}) # at least get the core message out if something happened message = self.message # Put the message in 'msg' so that we can access it. If we have it in # message it will be overshadowed by the class' message attribute self.msg = message super(BrickException, self).__init__(message) def __unicode__(self): return six.text_type(self.msg) class NotFound(BrickException): message = _("Resource could not be found.") code = 404 safe = True class Invalid(BrickException): message = _("Unacceptable parameters.") code = 400 # Cannot be templated as the error syntax varies. # msg needs to be constructed when raised. class InvalidParameterValue(Invalid): message = _("%(err)s") class NoFibreChannelHostsFound(BrickException): message = _("We are unable to locate any Fibre Channel devices.") class NoFibreChannelVolumeDeviceFound(BrickException): message = _("Unable to find a Fibre Channel volume device.") class VolumeNotDeactivated(BrickException): message = _('Volume %(name)s was not deactivated in time.') class VolumeDeviceNotFound(BrickException): message = _("Volume device not found at %(device)s.") class VolumePathsNotFound(BrickException): message = _("Could not find any paths for the volume.") class VolumePathNotRemoved(BrickException): message = _("Volume path %(volume_path)s was not removed in time.") class ProtocolNotSupported(BrickException): message = _("Connect to volume via protocol %(protocol)s not supported.") class TargetPortalNotFound(BrickException): message = _("Unable to find target portal %(target_portal)s.") class TargetPortalsNotFound(TargetPortalNotFound): message = _("Unable to find target portal in %(target_portals)s.") class FailedISCSITargetPortalLogin(BrickException): message = _("Unable to login to iSCSI Target Portal") class BlockDeviceReadOnly(BrickException): message = _("Block device %(device)s is Read-Only.") class VolumeGroupNotFound(BrickException): message = _("Unable to find Volume Group: %(vg_name)s") class VolumeGroupCreationFailed(BrickException): message = _("Failed to create Volume Group: %(vg_name)s") class CommandExecutionFailed(BrickException): message = _("Failed to execute command %(cmd)s") class VolumeDriverException(BrickException): message = _('An error occurred while IO to volume %(name)s.') class InvalidIOHandleObject(BrickException): message = _('IO handle of %(protocol)s has wrong object ' 'type %(actual_type)s.') class VolumeEncryptionNotSupported(Invalid): message = _("Volume encryption is not supported for %(volume_type)s " "volume %(volume_id)s.") # NOTE(mriedem): This extends ValueError to maintain backward compatibility. class InvalidConnectorProtocol(ValueError): pass class ExceptionChainer(BrickException): """A Exception that can contain a group of exceptions. This exception serves as a container for exceptions, useful when we want to store all exceptions that happened during a series of steps and then raise them all together as one. The representation of the exception will include all exceptions and their tracebacks. This class also includes a context manager for convenience, one that will support both swallowing the exception as if nothing had happened and raising the exception. In both cases the exception will be stored. If a message is provided to the context manager it will be formatted and logged with warning level. """ def __init__(self, *args, **kwargs): self._exceptions = [] self._repr = None super(ExceptionChainer, self).__init__(*args, **kwargs) def __repr__(self): # Since generating the representation can be slow we cache it if not self._repr: tracebacks = ( ''.join(traceback.format_exception(*e)).replace('\n', '\n\t') for e in self._exceptions) self._repr = '\n'.join('\nChained Exception #%s\n\t%s' % (i + 1, t) for i, t in enumerate(tracebacks)) return self._repr __str__ = __unicode__ = __repr__ def __nonzero__(self): # We want to be able to do boolean checks on the exception return bool(self._exceptions) __bool__ = __nonzero__ # For Python 3 def add_exception(self, exc_type, exc_val, exc_tb): # Clear the representation cache self._repr = None self._exceptions.append((exc_type, exc_val, exc_tb)) def context(self, catch_exception, msg='', *msg_args): self._catch_exception = catch_exception self._exc_msg = msg self._exc_msg_args = msg_args return self def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type: self.add_exception(exc_type, exc_val, exc_tb) if self._exc_msg: LOG.warning(self._exc_msg, *self._exc_msg_args) if self._catch_exception: return True class ExecutionTimeout(putils.ProcessExecutionError): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/executor.py0000664000175000017500000000561400000000000017267 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic exec utility that allows us to set the execute and root_helper attributes for putils. Some projects need their own execute wrapper and root_helper settings, so this provides that hook. """ import threading from oslo_concurrency import processutils as putils from oslo_context import context as context_utils from oslo_utils import encodeutils from os_brick.privileged import rootwrap as priv_rootwrap class Executor(object): def __init__(self, root_helper, execute=None, *args, **kwargs): if execute is None: execute = priv_rootwrap.execute self.set_execute(execute) self.set_root_helper(root_helper) @staticmethod def safe_decode(string): return string and encodeutils.safe_decode(string, errors='ignore') @classmethod def make_putils_error_safe(cls, exc): """Converts ProcessExecutionError string attributes to unicode.""" for field in ('stderr', 'stdout', 'cmd', 'description'): value = getattr(exc, field, None) if value: setattr(exc, field, cls.safe_decode(value)) def _execute(self, *args, **kwargs): try: result = self.__execute(*args, **kwargs) if result: result = (self.safe_decode(result[0]), self.safe_decode(result[1])) return result except putils.ProcessExecutionError as e: self.make_putils_error_safe(e) raise def set_execute(self, execute): self.__execute = execute def set_root_helper(self, helper): self._root_helper = helper class Thread(threading.Thread): """Thread class that inherits the parent's context. This is useful when you are spawning a thread and want LOG entries to display the right context information, such as the request. """ def __init__(self, *args, **kwargs): # Store the caller's context as a private variable shared among threads self.__context__ = context_utils.get_current() super(Thread, self).__init__(*args, **kwargs) def run(self): # Store the context in the current thread's request store if self.__context__: self.__context__.update_store() super(Thread, self).run() ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/i18n.py0000664000175000017500000000153100000000000016202 0ustar00zuulzuul00000000000000# Copyright 2014 IBM Corp. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """oslo.i18n integration module. See https://docs.openstack.org/oslo.i18n/latest/ . """ import oslo_i18n as i18n DOMAIN = 'os-brick' _translators = i18n.TranslatorFactory(domain=DOMAIN) # The primary translation function using the well-known name "_" _ = _translators.primary ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1446776 os-brick-3.0.8/os_brick/initiator/0000775000175000017500000000000000000000000017053 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/__init__.py0000664000175000017500000000321300000000000021163 0ustar00zuulzuul00000000000000# Copyright 2015 OpenStack Foundation # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Brick's Initiator module. The initator module contains the capabilities for discovering the initiator information as well as discovering and removing volumes from a host. """ import re DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 MULTIPATH_ERROR_REGEX = re.compile(r"\w{3} \d+ \d\d:\d\d:\d\d \|.*$") MULTIPATH_PATH_CHECK_REGEX = re.compile(r"\s+\d+:\d+:\d+:\d+\s+") PLATFORM_ALL = 'ALL' PLATFORM_x86 = 'X86' PLATFORM_S390 = 'S390' PLATFORM_PPC64 = 'PPC64' OS_TYPE_ALL = 'ALL' OS_TYPE_LINUX = 'LINUX' OS_TYPE_WINDOWS = 'WIN' S390X = "s390x" S390 = "s390" PPC64 = "ppc64" PPC64LE = "ppc64le" ISCSI = "ISCSI" ISER = "ISER" FIBRE_CHANNEL = "FIBRE_CHANNEL" AOE = "AOE" DRBD = "DRBD" NFS = "NFS" SMBFS = 'SMBFS' GLUSTERFS = "GLUSTERFS" LOCAL = "LOCAL" HUAWEISDSHYPERVISOR = "HUAWEISDSHYPERVISOR" HGST = "HGST" RBD = "RBD" SCALEIO = "SCALEIO" SCALITY = "SCALITY" QUOBYTE = "QUOBYTE" DISCO = "DISCO" VZSTORAGE = "VZSTORAGE" VMDK = "VMDK" GPFS = "GPFS" VERITAS_HYPERSCALE = "VERITAS_HYPERSCALE" STORPOOL = "STORPOOL" NVME = "NVME" NVMEOF = "NVMEOF" ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connector.py0000664000175000017500000003024100000000000021417 0ustar00zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Brick Connector objects for each supported transport protocol. .. module: connector The connectors here are responsible for discovering and removing volumes for each of the supported transport protocols. """ import platform import socket import sys from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_utils import importutils from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') # List of connectors to call when getting # the connector properties for a host windows_connector_list = [ 'os_brick.initiator.windows.base.BaseWindowsConnector', 'os_brick.initiator.windows.iscsi.WindowsISCSIConnector', 'os_brick.initiator.windows.fibre_channel.WindowsFCConnector', 'os_brick.initiator.windows.smbfs.WindowsSMBFSConnector' ] unix_connector_list = [ 'os_brick.initiator.connectors.base.BaseLinuxConnector', 'os_brick.initiator.connectors.iscsi.ISCSIConnector', 'os_brick.initiator.connectors.fibre_channel.FibreChannelConnector', ('os_brick.initiator.connectors.fibre_channel_s390x.' 'FibreChannelConnectorS390X'), ('os_brick.initiator.connectors.fibre_channel_ppc64.' 'FibreChannelConnectorPPC64'), 'os_brick.initiator.connectors.aoe.AoEConnector', 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', 'os_brick.initiator.connectors.rbd.RBDConnector', 'os_brick.initiator.connectors.local.LocalConnector', 'os_brick.initiator.connectors.gpfs.GPFSConnector', 'os_brick.initiator.connectors.drbd.DRBDConnector', 'os_brick.initiator.connectors.huawei.HuaweiStorHyperConnector', 'os_brick.initiator.connectors.hgst.HGSTConnector', 'os_brick.initiator.connectors.scaleio.ScaleIOConnector', 'os_brick.initiator.connectors.disco.DISCOConnector', 'os_brick.initiator.connectors.vmware.VmdkConnector', 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', 'os_brick.initiator.connectors.storpool.StorPoolConnector', 'os_brick.initiator.connectors.nvmeof.NVMeOFConnector', ] def _get_connector_list(): if sys.platform != 'win32': return unix_connector_list else: return windows_connector_list # Mappings used to determine who to construct in the factory _connector_mapping_linux = { initiator.AOE: 'os_brick.initiator.connectors.aoe.AoEConnector', initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.GLUSTERFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.SCALITY: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.QUOBYTE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.VZSTORAGE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.ISER: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.FIBRE_CHANNEL: 'os_brick.initiator.connectors.fibre_channel.FibreChannelConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.HUAWEISDSHYPERVISOR: 'os_brick.initiator.connectors.huawei.HuaweiStorHyperConnector', initiator.HGST: 'os_brick.initiator.connectors.hgst.HGSTConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.SCALEIO: 'os_brick.initiator.connectors.scaleio.ScaleIOConnector', initiator.DISCO: 'os_brick.initiator.connectors.disco.DISCOConnector', initiator.VMDK: 'os_brick.initiator.connectors.vmware.VmdkConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', initiator.VERITAS_HYPERSCALE: 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', initiator.STORPOOL: 'os_brick.initiator.connectors.storpool.StorPoolConnector', # Leave this in for backwards compatibility # This isn't an NVME connector, but NVME Over Fabrics initiator.NVME: 'os_brick.initiator.connectors.nvmeof.NVMeOFConnector', initiator.NVMEOF: 'os_brick.initiator.connectors.nvmeof.NVMeOFConnector', } # Mapping for the S390X platform _connector_mapping_linux_s390x = { initiator.FIBRE_CHANNEL: 'os_brick.initiator.connectors.fibre_channel_s390x.' 'FibreChannelConnectorS390X', initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', } # Mapping for the PPC64 platform _connector_mapping_linux_ppc64 = { initiator.FIBRE_CHANNEL: ('os_brick.initiator.connectors.fibre_channel_ppc64.' 'FibreChannelConnectorPPC64'), initiator.DRBD: 'os_brick.initiator.connectors.drbd.DRBDConnector', initiator.NFS: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.ISCSI: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', initiator.LOCAL: 'os_brick.initiator.connectors.local.LocalConnector', initiator.RBD: 'os_brick.initiator.connectors.rbd.RBDConnector', initiator.GPFS: 'os_brick.initiator.connectors.gpfs.GPFSConnector', initiator.VZSTORAGE: 'os_brick.initiator.connectors.remotefs.RemoteFsConnector', initiator.VERITAS_HYPERSCALE: 'os_brick.initiator.connectors.vrtshyperscale.HyperScaleConnector', initiator.ISER: 'os_brick.initiator.connectors.iscsi.ISCSIConnector', } # Mapping for the windows connectors _connector_mapping_windows = { initiator.ISCSI: 'os_brick.initiator.windows.iscsi.WindowsISCSIConnector', initiator.FIBRE_CHANNEL: 'os_brick.initiator.windows.fibre_channel.WindowsFCConnector', initiator.SMBFS: 'os_brick.initiator.windows.smbfs.WindowsSMBFSConnector', } # Create aliases to the old names until 2.0.0 # TODO(smcginnis) Remove this lookup once unit test code is updated to # point to the correct location def _set_aliases(): conn_list = _get_connector_list() # TODO(lpetrut): Cinder is explicitly trying to use those two # connectors. We should drop this once we fix Cinder and # get passed the backwards compatibility period. if sys.platform == 'win32': conn_list += [ 'os_brick.initiator.connectors.iscsi.ISCSIConnector', ('os_brick.initiator.connectors.fibre_channel.' 'FibreChannelConnector'), ] for item in conn_list: _name = item.split('.')[-1] globals()[_name] = importutils.import_class(item) _set_aliases() @utils.trace def get_connector_properties(root_helper, my_ip, multipath, enforce_multipath, host=None, execute=None): """Get the connection properties for all protocols. When the connector wants to use multipath, multipath=True should be specified. If enforce_multipath=True is specified too, an exception is thrown when multipathd is not running. Otherwise, it falls back to multipath=False and only the first path shown up is used. For the compatibility reason, even if multipath=False is specified, some cinder storage drivers may export the target for multipath, which can be found via sendtargets discovery. :param root_helper: The command prefix for executing as root. :type root_helper: str :param my_ip: The IP address of the local host. :type my_ip: str :param multipath: Enable multipath? :type multipath: bool :param enforce_multipath: Should we enforce that the multipath daemon is running? If the daemon isn't running then the return dict will have multipath as False. :type enforce_multipath: bool :param host: hostname. :param execute: execute helper. :returns: dict containing all of the collected initiator values. """ props = {} props['platform'] = platform.machine() props['os_type'] = sys.platform props['ip'] = my_ip props['host'] = host if host else socket.gethostname() for item in _get_connector_list(): connector = importutils.import_class(item) if (utils.platform_matches(props['platform'], connector.platform) and utils.os_matches(props['os_type'], connector.os_type)): props = utils.merge_dict(props, connector.get_connector_properties( root_helper, host=host, multipath=multipath, enforce_multipath=enforce_multipath, execute=execute)) return props def get_connector_mapping(arch=None): """Get connector mapping based on platform. This is used by Nova to get the right connector information. :param arch: The architecture being requested. """ # We do this instead of assigning it in the definition # to help mocking for unit tests if arch is None: arch = platform.machine() # Set the correct mapping for imports if sys.platform == 'win32': return _connector_mapping_windows elif arch in (initiator.S390, initiator.S390X): return _connector_mapping_linux_s390x elif arch in (initiator.PPC64, initiator.PPC64LE): return _connector_mapping_linux_ppc64 else: return _connector_mapping_linux # TODO(walter-boring) We have to keep this class defined here # so we don't break backwards compatibility class InitiatorConnector(object): @staticmethod def factory(protocol, root_helper, driver=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, arch=None, *args, **kwargs): """Build a Connector object based upon protocol and architecture.""" _mapping = get_connector_mapping(arch) LOG.debug("Factory for %(protocol)s on %(arch)s", {'protocol': protocol, 'arch': arch}) protocol = protocol.upper() # set any special kwargs needed by connectors if protocol in (initiator.NFS, initiator.GLUSTERFS, initiator.SCALITY, initiator.QUOBYTE, initiator.VZSTORAGE): kwargs.update({'mount_type': protocol.lower()}) elif protocol == initiator.ISER: kwargs.update({'transport': 'iser'}) # now set all the default kwargs kwargs.update( {'root_helper': root_helper, 'driver': driver, 'use_multipath': use_multipath, 'device_scan_attempts': device_scan_attempts, }) connector = _mapping.get(protocol) if not connector: msg = (_("Invalid InitiatorConnector protocol " "specified %(protocol)s") % dict(protocol=protocol)) raise exception.InvalidConnectorProtocol(msg) conn_cls = importutils.import_class(connector) return conn_cls(*args, **kwargs) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/initiator/connectors/0000775000175000017500000000000000000000000021230 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/__init__.py0000664000175000017500000000000000000000000023327 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/aoe.py0000664000175000017500000001452000000000000022350 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_service import loopingcall from os_brick import exception from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 LOG = logging.getLogger(__name__) class AoEConnector(base.BaseLinuxConnector): """Connector class to attach/detach AoE volumes.""" def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(AoEConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The AoE connector properties.""" return {} def get_search_path(self): return '/dev/etherd' def get_volume_paths(self, connection_properties): aoe_device, aoe_path = self._get_aoe_info(connection_properties) volume_paths = [] if os.path.exists(aoe_path): volume_paths.append(aoe_path) return volume_paths def _get_aoe_info(self, connection_properties): shelf = connection_properties['target_shelf'] lun = connection_properties['target_lun'] aoe_device = 'e%(shelf)s.%(lun)s' % {'shelf': shelf, 'lun': lun} path = self.get_search_path() aoe_path = '%(path)s/%(device)s' % {'path': path, 'device': aoe_device} return aoe_device, aoe_path @utils.trace @lockutils.synchronized('aoe_control', 'aoe-') def connect_volume(self, connection_properties): """Discover and attach the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for AoE must include: target_shelf - shelf id of volume target_lun - lun id of volume """ aoe_device, aoe_path = self._get_aoe_info(connection_properties) device_info = { 'type': 'block', 'device': aoe_device, 'path': aoe_path, } if os.path.exists(aoe_path): self._aoe_revalidate(aoe_device) else: self._aoe_discover() waiting_status = {'tries': 0} # NOTE(jbr_): Device path is not always present immediately def _wait_for_discovery(aoe_path): if os.path.exists(aoe_path): raise loopingcall.LoopingCallDone if waiting_status['tries'] >= self.device_scan_attempts: raise exception.VolumeDeviceNotFound(device=aoe_path) LOG.info("AoE volume not yet found at: %(path)s. " "Try number: %(tries)s", {'path': aoe_device, 'tries': waiting_status['tries']}) self._aoe_discover() waiting_status['tries'] += 1 timer = loopingcall.FixedIntervalLoopingCall(_wait_for_discovery, aoe_path) timer.start(interval=2).wait() if waiting_status['tries']: LOG.debug("Found AoE device %(path)s " "(after %(tries)s rediscover)", {'path': aoe_path, 'tries': waiting_status['tries']}) return device_info @utils.trace @lockutils.synchronized('aoe_control', 'aoe-') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict connection_properties for AoE must include: target_shelf - shelf id of volume target_lun - lun id of volume """ aoe_device, aoe_path = self._get_aoe_info(connection_properties) if os.path.exists(aoe_path): self._aoe_flush(aoe_device) def _aoe_discover(self): (out, err) = self._execute('aoe-discover', run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-discover: stdout=%(out)s stderr%(err)s', {'out': out, 'err': err}) def _aoe_revalidate(self, aoe_device): (out, err) = self._execute('aoe-revalidate', aoe_device, run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-revalidate %(dev)s: stdout=%(out)s stderr%(err)s', {'dev': aoe_device, 'out': out, 'err': err}) def _aoe_flush(self, aoe_device): (out, err) = self._execute('aoe-flush', aoe_device, run_as_root=True, root_helper=self._root_helper, check_exit_code=0) LOG.debug('aoe-flush %(dev)s: stdout=%(out)s stderr%(err)s', {'dev': aoe_device, 'out': out, 'err': err}) def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/base.py0000664000175000017500000001152600000000000022521 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick import initiator from os_brick.initiator import host_driver from os_brick.initiator import initiator_connector from os_brick.initiator import linuxscsi LOG = logging.getLogger(__name__) class BaseLinuxConnector(initiator_connector.InitiatorConnector): os_type = initiator.OS_TYPE_LINUX def __init__(self, root_helper, driver=None, execute=None, *args, **kwargs): self._linuxscsi = linuxscsi.LinuxSCSI(root_helper, execute=execute) if not driver: driver = host_driver.HostDriver() self.set_driver(driver) super(BaseLinuxConnector, self).__init__(root_helper, execute=execute, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The generic connector properties.""" multipath = kwargs['multipath'] enforce_multipath = kwargs['enforce_multipath'] props = {} props['multipath'] = (multipath and linuxscsi.LinuxSCSI.is_multipath_running( enforce_multipath, root_helper, execute=kwargs.get('execute'))) return props def check_valid_device(self, path, run_as_root=True): cmd = ('dd', 'if=%(path)s' % {"path": path}, 'of=/dev/null', 'count=1') out, info = None, None try: out, info = self._execute(*cmd, run_as_root=run_as_root, root_helper=self._root_helper) except putils.ProcessExecutionError as e: LOG.error("Failed to access the device on the path " "%(path)s: %(error)s.", {"path": path, "error": e.stderr}) return False # If the info is none, the path does not exist. if info is None: return False return True def get_all_available_volumes(self, connection_properties=None): volumes = [] path = self.get_search_path() if path: # now find all entries in the search path if os.path.isdir(path): path_items = [path, '/*'] file_filter = ''.join(path_items) volumes = glob.glob(file_filter) return volumes def _discover_mpath_device(self, device_wwn, connection_properties, device_name): """This method discovers a multipath device. Discover a multipath device based on a defined connection_property and a device_wwn and return the multipath_id and path of the multipath enabled device if there is one. """ path = self._linuxscsi.find_multipath_device_path(device_wwn) device_path = None multipath_id = None if path is None: # find_multipath_device only accept realpath not symbolic path device_realpath = os.path.realpath(device_name) mpath_info = self._linuxscsi.find_multipath_device( device_realpath) if mpath_info: device_path = mpath_info['device'] multipath_id = device_wwn else: # we didn't find a multipath device. # so we assume the kernel only sees 1 device device_path = device_name LOG.debug("Unable to find multipath device name for " "volume. Using path %(device)s for volume.", {'device': device_path}) else: device_path = path multipath_id = device_wwn if connection_properties.get('access_mode', '') != 'ro': try: # Sometimes the multipath devices will show up as read only # initially and need additional time/rescans to get to RW. self._linuxscsi.wait_for_rw(device_wwn, device_path) except exception.BlockDeviceReadOnly: LOG.warning('Block device %s is still read-only. ' 'Continuing anyway.', device_path) return device_path, multipath_id ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/base_iscsi.py0000664000175000017500000000372300000000000023713 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import copy from os_brick.initiator import initiator_connector class BaseISCSIConnector(initiator_connector.InitiatorConnector): def _iterate_all_targets(self, connection_properties): for portal, iqn, lun in self._get_all_targets(connection_properties): props = copy.deepcopy(connection_properties) props['target_portal'] = portal props['target_iqn'] = iqn props['target_lun'] = lun for key in ('target_portals', 'target_iqns', 'target_luns'): props.pop(key, None) yield props @staticmethod def _get_luns(con_props, iqns=None): luns = con_props.get('target_luns') num_luns = len(con_props['target_iqns']) if iqns is None else len(iqns) return luns or [con_props['target_lun']] * num_luns def _get_all_targets(self, connection_properties): if all(key in connection_properties for key in ('target_portals', 'target_iqns')): return list(zip(connection_properties['target_portals'], connection_properties['target_iqns'], self._get_luns(connection_properties))) return [(connection_properties['target_portal'], connection_properties['target_iqn'], connection_properties.get('target_lun', 0))] ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/disco.py0000664000175000017500000001644300000000000022713 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os import socket import struct from oslo_concurrency import lockutils from oslo_log import log as logging import six from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 synchronized = lockutils.synchronized_with_prefix('os-brick-') class DISCOConnector(base.BaseLinuxConnector): """Class implements the connector driver for DISCO.""" DISCO_PREFIX = 'dms' def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): """Init DISCO connector.""" super(DISCOConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs ) LOG.debug("Init DISCO connector") self.server_port = None self.server_ip = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The DISCO connector properties.""" return {} def get_search_path(self): """Get directory path where to get DISCO volumes.""" return "/dev" def get_volume_paths(self, connection_properties): """Get config for DISCO volume driver.""" self.get_config(connection_properties) volume_paths = [] disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) device_paths = [disco_dev] for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths def get_all_available_volumes(self, connection_properties=None): """Return all DISCO volumes that exist in the search directory.""" path = self.get_search_path() if os.path.isdir(path): path_items = [path, '/', self.DISCO_PREFIX, '*'] file_filter = ''.join(path_items) return glob.glob(file_filter) else: return [] def get_config(self, connection_properties): """Get config for DISCO volume driver.""" self.server_port = ( six.text_type(connection_properties['conf']['server_port'])) self.server_ip = ( six.text_type(connection_properties['conf']['server_ip'])) disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) device_info = {'type': 'block', 'path': disco_dev} return device_info @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Connect the volume. Returns xml for libvirt.""" LOG.debug("Enter in DISCO connect_volume") device_info = self.get_config(connection_properties) LOG.debug("Device info : %s.", device_info) disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) LOG.debug("Attaching %s", disco_dev) self._mount_disco_volume(disco_dev, disco_id) return device_info @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance.""" disco_id = connection_properties['disco_id'] disco_dev = '/dev/dms%s' % (disco_id) LOG.debug("detaching %s", disco_dev) if os.path.exists(disco_dev): ret = self._send_disco_vol_cmd(self.server_ip, self.server_port, 2, disco_id) if ret is not None: msg = _("Detach volume failed") raise exception.BrickException(message=msg) else: LOG.info("Volume already detached from host") def _mount_disco_volume(self, path, volume_id): """Send request to mount volume on physical host.""" LOG.debug("Enter in mount disco volume %(port)s " "and %(ip)s.", {'port': self.server_port, 'ip': self.server_ip}) if not os.path.exists(path): ret = self._send_disco_vol_cmd(self.server_ip, self.server_port, 1, volume_id) if ret is not None: msg = _("Attach volume failed") raise exception.BrickException(message=msg) else: LOG.info("Volume already attached to host") def _connect_tcp_socket(self, client_ip, client_port): """Connect to TCP socket.""" sock = None for res in socket.getaddrinfo(client_ip, client_port, socket.AF_UNSPEC, socket.SOCK_STREAM): aff, socktype, proto, canonname, saa = res try: sock = socket.socket(aff, socktype, proto) except socket.error: sock = None continue try: sock.connect(saa) except socket.error: sock.close() sock = None continue break if sock is None: LOG.error("Cannot connect TCP socket") return sock def _send_disco_vol_cmd(self, client_ip, client_port, op_code, vol_id): """Send DISCO client socket command.""" s = self._connect_tcp_socket(client_ip, int(client_port)) if s is not None: inst_id = 'DEFAULT-INSTID' pktlen = 2 + 8 + len(inst_id) LOG.debug("pktlen=%(plen)s op=%(op)s " "vol_id=%(vol_id)s, inst_id=%(inst_id)s", {'plen': pktlen, 'op': op_code, 'vol_id': vol_id, 'inst_id': inst_id}) data = struct.pack("!HHQ14s", pktlen, op_code, int(vol_id), inst_id) s.sendall(data) ret = s.recv(4) s.close() LOG.debug("Received ret len=%(lenR)d, ret=%(ret)s", {'lenR': len(repr(ret)), 'ret': repr(ret)}) ret_val = "".join("%02x" % ord(c) for c in ret) if ret_val != '00000000': return 'ERROR' return None def extend_volume(self, connection_properties): raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/drbd.py0000664000175000017500000000721700000000000022524 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_concurrency import processutils as putils from os_brick.initiator.connectors import base from os_brick import utils class DRBDConnector(base.BaseLinuxConnector): """"Connector class to attach/detach DRBD resources.""" def __init__(self, root_helper, driver=None, execute=putils.execute, *args, **kwargs): super(DRBDConnector, self).__init__(root_helper, driver=driver, execute=execute, *args, **kwargs) self._execute = execute self._root_helper = root_helper @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The DRBD connector properties.""" return {} def check_valid_device(self, path, run_as_root=True): """Verify an existing volume.""" # TODO(linbit): check via drbdsetup first, to avoid blocking/hanging # in case of network problems? return super(DRBDConnector, self).check_valid_device(path, run_as_root) def get_all_available_volumes(self, connection_properties=None): base = "/dev/" blkdev_list = [] for e in os.listdir(base): path = base + e if os.path.isblk(path): blkdev_list.append(path) return blkdev_list def _drbdadm_command(self, cmd, data_dict, sh_secret): # TODO(linbit): Write that resource file to a permanent location? tmp = tempfile.NamedTemporaryFile(suffix="res", delete=False, mode="w") try: kv = {'shared-secret': sh_secret} tmp.write(data_dict['config'] % kv) tmp.close() (out, err) = self._execute('drbdadm', cmd, "-c", tmp.name, data_dict['name'], run_as_root=True, root_helper=self._root_helper) finally: os.unlink(tmp.name) return (out, err) @utils.trace def connect_volume(self, connection_properties): """Attach the volume.""" self._drbdadm_command("adjust", connection_properties, connection_properties['provider_auth']) device_info = { 'type': 'block', 'path': connection_properties['device'], } return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume.""" self._drbdadm_command("down", connection_properties, connection_properties['provider_auth']) def get_volume_paths(self, connection_properties): path = connection_properties['device'] return [path] def get_search_path(self): # TODO(linbit): is it allowed to return "/dev", or is that too broad? return None def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/fake.py0000664000175000017500000000313500000000000022512 0ustar00zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import base from os_brick.initiator.connectors import base_iscsi class FakeConnector(base.BaseLinuxConnector): fake_path = '/dev/vdFAKE' def connect_volume(self, connection_properties): fake_device_info = {'type': 'fake', 'path': self.fake_path} return fake_device_info def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): pass def get_volume_paths(self, connection_properties): return [self.fake_path] def get_search_path(self): return '/dev/disk/by-path' def extend_volume(self, connection_properties): return None def get_all_available_volumes(self, connection_properties=None): return ['/dev/disk/by-path/fake-volume-1', '/dev/disk/by-path/fake-volume-X'] class FakeBaseISCSIConnector(FakeConnector, base_iscsi.BaseISCSIConnector): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/fibre_channel.py0000664000175000017500000004137300000000000024371 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from oslo_service import loopingcall import six from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator import linuxfc from os_brick import utils synchronized = lockutils.synchronized_with_prefix('os-brick-') LOG = logging.getLogger(__name__) class FibreChannelConnector(base.BaseLinuxConnector): """Connector class to attach/detach Fibre Channel volumes.""" def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): self._linuxfc = linuxfc.LinuxFibreChannel(root_helper, execute) super(FibreChannelConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnector, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The Fibre Channel connector properties.""" props = {} fc = linuxfc.LinuxFibreChannel(root_helper, execute=kwargs.get('execute')) wwpns = fc.get_fc_wwpns() if wwpns: props['wwpns'] = wwpns wwnns = fc.get_fc_wwnns() if wwnns: props['wwnns'] = wwnns return props def get_search_path(self): """Where do we look for FC based volumes.""" return '/dev/disk/by-path' def _add_targets_to_connection_properties(self, connection_properties): LOG.debug('Adding targets to connection properties receives: %s', connection_properties) target_wwn = connection_properties.get('target_wwn') target_wwns = connection_properties.get('target_wwns') if target_wwns: wwns = target_wwns elif isinstance(target_wwn, list): wwns = target_wwn elif isinstance(target_wwn, six.string_types): wwns = [target_wwn] else: wwns = [] # Convert wwns to lower case wwns = [wwn.lower() for wwn in wwns] if target_wwns: connection_properties['target_wwns'] = wwns elif target_wwn: connection_properties['target_wwn'] = wwns target_lun = connection_properties.get('target_lun', 0) target_luns = connection_properties.get('target_luns') if target_luns: luns = target_luns elif isinstance(target_lun, int): luns = [target_lun] else: luns = [] if len(luns) == len(wwns): # Handles single wwwn + lun or multiple, potentially # different wwns or luns targets = list(zip(wwns, luns)) elif len(luns) == 1 and len(wwns) > 1: # For the case of multiple wwns, but a single lun (old path) targets = [(wwn, luns[0]) for wwn in wwns] else: # Something is wrong, this shouldn't happen. msg = _("Unable to find potential volume paths for FC device " "with luns: %(luns)s and wwns: %(wwns)s.") % { "luns": luns, "wwns": wwns} LOG.error(msg) raise exception.VolumePathsNotFound(msg) connection_properties['targets'] = targets wwpn_lun_map = {wwpn: lun for wwpn, lun in targets} # If there is an initiator_target_map we can update it too and generate # the initiator_target_lun_map from it if connection_properties.get('initiator_target_map') is not None: # Convert it to lower case itmap = connection_properties['initiator_target_map'] itmap = {k.lower(): [port.lower() for port in v] for k, v in itmap.items()} connection_properties['initiator_target_map'] = itmap itmaplun = dict() for init_wwpn, target_wwpns in itmap.items(): itmaplun[init_wwpn] = [(target_wwpn, wwpn_lun_map[target_wwpn]) for target_wwpn in target_wwpns if target_wwpn in wwpn_lun_map] # We added the if in the previous list comprehension in case # drivers return targets in the map that are not reported in # target_wwn or target_wwns, but we warn about it. if len(itmaplun[init_wwpn]) != len(itmap[init_wwpn]): unknown = set(itmap[init_wwpn]) unknown.difference_update(itmaplun[init_wwpn]) LOG.warning('Driver returned an unknown targets in the ' 'initiator mapping %s', ', '.join(unknown)) connection_properties['initiator_target_lun_map'] = itmaplun LOG.debug('Adding targets to connection properties returns: %s', connection_properties) return connection_properties def _get_possible_volume_paths(self, connection_properties, hbas): targets = connection_properties['targets'] possible_devs = self._get_possible_devices(hbas, targets) host_paths = self._get_host_devices(possible_devs) return host_paths def get_volume_paths(self, connection_properties): volume_paths = [] # first fetch all of the potential paths that might exist # how the FC fabric is zoned may alter the actual list # that shows up on the system. So, we verify each path. hbas = self._linuxfc.get_fc_hbas_info() device_paths = self._get_possible_volume_paths( connection_properties, hbas) for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths @utils.trace @synchronized('extend_volume', external=True) def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an FC volume. """ connection_properties = self._add_targets_to_connection_properties( connection_properties) volume_paths = self.get_volume_paths(connection_properties) if volume_paths: return self._linuxscsi.extend_volume( volume_paths, use_multipath=self.use_multipath) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': connection_properties}) raise exception.VolumePathsNotFound() @utils.trace @synchronized('connect_volume', external=True) def connect_volume(self, connection_properties): """Attach the volume to instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for Fibre Channel must include: target_wwn - World Wide Name target_lun - LUN id of the volume """ device_info = {'type': 'block'} connection_properties = self._add_targets_to_connection_properties( connection_properties) hbas = self._linuxfc.get_fc_hbas_info() if not hbas: LOG.warning("We are unable to locate any Fibre Channel devices.") raise exception.NoFibreChannelHostsFound() host_devices = self._get_possible_volume_paths( connection_properties, hbas) # The /dev/disk/by-path/... node is not always present immediately # We only need to find the first device. Once we see the first device # multipath will have any others. def _wait_for_device_discovery(host_devices): for device in host_devices: LOG.debug("Looking for Fibre Channel dev %(device)s", {'device': device}) if os.path.exists(device) and self.check_valid_device(device): self.host_device = device # get the /dev/sdX device. This variable is maintained to # keep the same log output. self.device_name = os.path.realpath(device) raise loopingcall.LoopingCallDone() if self.tries >= self.device_scan_attempts: LOG.error("Fibre Channel volume device not found.") raise exception.NoFibreChannelVolumeDeviceFound() LOG.info("Fibre Channel volume device not yet found. " "Will rescan & retry. Try number: %(tries)s.", {'tries': self.tries}) self._linuxfc.rescan_hosts(hbas, connection_properties) self.tries = self.tries + 1 self.host_device = None self.device_name = None self.tries = 0 timer = loopingcall.FixedIntervalLoopingCall( _wait_for_device_discovery, host_devices) timer.start(interval=2).wait() LOG.debug("Found Fibre Channel volume %(name)s " "(after %(tries)s rescans.)", {'name': self.device_name, 'tries': self.tries}) # find out the WWN of the device device_wwn = self._linuxscsi.get_scsi_wwn(self.host_device) LOG.debug("Device WWN = '%(wwn)s'", {'wwn': device_wwn}) device_info['scsi_wwn'] = device_wwn # see if the new drive is part of a multipath # device. If so, we'll use the multipath device. if self.use_multipath: # Pass a symlink, not a real path, otherwise we'll get a real path # back if we don't find a multipath and we'll return that to the # caller, breaking Nova's encryption which requires a symlink. (device_path, multipath_id) = self._discover_mpath_device( device_wwn, connection_properties, self.host_device) if multipath_id: # only set the multipath_id if we found one device_info['multipath_id'] = multipath_id else: device_path = self.host_device device_info['path'] = device_path return device_info def _get_host_devices(self, possible_devs): """Compute the device paths on the system with an id, wwn, and lun :param possible_devs: list of (platform, pci_id, wwn, lun) tuples :return: list of device paths on the system based on the possible_devs """ host_devices = [] for platform, pci_num, target_wwn, lun in possible_devs: host_device = "/dev/disk/by-path/%spci-%s-fc-%s-lun-%s" % ( platform + '-' if platform else '', pci_num, target_wwn, self._linuxscsi.process_lun_id(lun)) host_devices.append(host_device) return host_devices def _get_possible_devices(self, hbas, targets): """Compute the possible fibre channel device options. :param hbas: available hba devices. :param targets: tuple of possible wwn addresses and lun combinations. :returns: list of (platform, pci_id, wwn, lun) tuples Given one or more wwn (mac addresses for fibre channel) ports do the matrix math to figure out a set of pci device, wwn tuples that are potentially valid (they won't all be). This provides a search space for the device connection. """ raw_devices = [] for hba in hbas: platform, pci_num = self._get_pci_num(hba) if pci_num is not None: for wwn, lun in targets: target_wwn = "0x%s" % wwn.lower() raw_devices.append((platform, pci_num, target_wwn, lun)) return raw_devices @utils.trace @synchronized('connect_volume', external=True) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict connection_properties for Fibre Channel must include: target_wwn - World Wide Name target_lun - LUN id of the volume """ devices = [] wwn = None connection_properties = self._add_targets_to_connection_properties( connection_properties) volume_paths = self.get_volume_paths(connection_properties) mpath_path = None for path in volume_paths: real_path = self._linuxscsi.get_name_from_path(path) if (self.use_multipath and not mpath_path and self.check_valid_device(path)): wwn = self._linuxscsi.get_scsi_wwn(path) mpath_path = self._linuxscsi.find_multipath_device_path(wwn) if mpath_path: self._linuxscsi.flush_multipath_device(mpath_path) dev_info = self._linuxscsi.get_device_info(real_path) devices.append(dev_info) LOG.debug("devices to remove = %s", devices) self._remove_devices(connection_properties, devices, device_info) def _remove_devices(self, connection_properties, devices, device_info): # There may have been more than 1 device mounted # by the kernel for this volume. We have to remove # all of them path_used = self._linuxscsi.get_dev_path(connection_properties, device_info) # NOTE: Due to bug #1897787 device_info may have a real path for some # single paths instead of a symlink as it should have, so it'll only # be a multipath if it was a symlink (not real path) and it wasn't a # single path symlink (those have filenames starting with pci-) # We don't use os.path.islink in case the file is no longer there. was_symlink = path_used.count(os.sep) > 2 # We check for /pci because that's the value we return for single # paths, whereas for multipaths we have multiple link formats. was_multipath = '/pci-' not in path_used and was_symlink for device in devices: device_path = device['device'] flush = self._linuxscsi.requires_flush(device_path, path_used, was_multipath) self._linuxscsi.remove_scsi_device(device_path, flush=flush) def _get_pci_num(self, hba): # NOTE(walter-boring) # device path is in format of (FC and FCoE) : # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2 # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2 # /host3/fc_host/host3 # we always want the value prior to the host or net value # on non x86_64 device, pci devices may be appended on platform device, # /sys/devices/platform/smb/smb:motherboard/80040000000.peu0-c0/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2 # noqa # so also return a platform id if it exists platform = None if hba is not None: if "device_path" in hba: device_path = hba['device_path'].split('/') has_platform = (len(device_path) > 3 and device_path[3] == 'platform') for index, value in enumerate(device_path): if has_platform and value.startswith('pci'): platform = "platform-%s" % device_path[index - 1] if value.startswith('net') or value.startswith('host'): return platform, device_path[index - 1] return None, None ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/fibre_channel_ppc64.py0000664000175000017500000000367300000000000025406 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import fibre_channel LOG = logging.getLogger(__name__) class FibreChannelConnectorPPC64(fibre_channel.FibreChannelConnector): """Connector class to attach/detach Fibre Channel volumes on PPC64 arch.""" platform = initiator.PLATFORM_PPC64 def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(FibreChannelConnectorPPC64, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnectorPPC64, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) def _get_host_devices(self, possible_devs, lun): host_devices = set() for pci_num, target_wwn in possible_devs: host_device = "/dev/disk/by-path/fc-%s-lun-%s" % ( target_wwn, self._linuxscsi.process_lun_id(lun)) host_devices.add(host_device) return list(host_devices) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/fibre_channel_s390x.py0000664000175000017500000000764400000000000025342 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import fibre_channel from os_brick.initiator import linuxfc LOG = logging.getLogger(__name__) class FibreChannelConnectorS390X(fibre_channel.FibreChannelConnector): """Connector class to attach/detach Fibre Channel volumes on S390X arch.""" platform = initiator.PLATFORM_S390 def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(FibreChannelConnectorS390X, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) LOG.debug("Initializing Fibre Channel connector for S390") self._linuxfc = linuxfc.LinuxFibreChannelS390X(root_helper, execute) self.use_multipath = use_multipath def set_execute(self, execute): super(FibreChannelConnectorS390X, self).set_execute(execute) self._linuxscsi.set_execute(execute) self._linuxfc.set_execute(execute) def _get_host_devices(self, possible_devs): host_devices = [] for pci_num, target_wwn, lun in possible_devs: host_device = self._get_device_file_path( pci_num, target_wwn, lun) # NOTE(arne_r) # LUN driver path is the same on all distros, so no need to have # multiple calls here self._linuxfc.configure_scsi_device(pci_num, target_wwn, self._get_lun_string(lun)) host_devices.extend(host_device) return host_devices def _get_lun_string(self, lun): target_lun = 0 if lun <= 0xffff: target_lun = "0x%04x000000000000" % lun elif lun <= 0xffffffff: target_lun = "0x%08x00000000" % lun return target_lun def _get_device_file_path(self, pci_num, target_wwn, lun): # NOTE(arne_r) # Need to add multiple possible ways to resolve device paths, # depending on OS. Since it gets passed to '_get_possible_volume_paths' # having a mismatch is not a problem host_device = [ # RHEL based "/dev/disk/by-path/ccw-%s-zfcp-%s:%s" % ( pci_num, target_wwn, self._get_lun_string(lun)), # Debian based (e.g. for storwize) "/dev/disk/by-path/ccw-%s-fc-%s-lun-%s" % ( pci_num, target_wwn, lun), # Debian based (e.g. for ds8k) "/dev/disk/by-path/ccw-%s-fc-%s-lun-%s" % ( pci_num, target_wwn, self._get_lun_string(lun)), ] return host_device def _remove_devices(self, connection_properties, devices, device_info): hbas = self._linuxfc.get_fc_hbas_info() targets = connection_properties['targets'] possible_devs = self._get_possible_devices(hbas, targets) for platform, pci_num, target_wwn, lun in possible_devs: target_lun = self._get_lun_string(lun) self._linuxfc.deconfigure_scsi_device(pci_num, target_wwn, target_lun) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/gpfs.py0000664000175000017500000000307500000000000022546 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.i18n import _ from os_brick.initiator.connectors import local from os_brick import utils class GPFSConnector(local.LocalConnector): """"Connector class to attach/detach File System backed volumes.""" @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: device_path - path to the volume to be connected :type connection_properties: dict :returns: dict """ if 'device_path' not in connection_properties: msg = (_("Invalid connection_properties specified " "no device_path attribute.")) raise ValueError(msg) device_info = {'type': 'gpfs', 'path': connection_properties['device_path']} return device_info ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/hgst.py0000664000175000017500000001651700000000000022561 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import socket from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) class HGSTConnector(base.BaseLinuxConnector): """Connector class to attach/detach HGST volumes.""" VGCCLUSTER = 'vgc-cluster' def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(HGSTConnector, self).__init__(root_helper, driver=driver, device_scan_attempts= device_scan_attempts, *args, **kwargs) self._vgc_host = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HGST connector properties.""" return {} def _log_cli_err(self, err): """Dumps the full command output to a logfile in error cases.""" LOG.error("CLI fail: '%(cmd)s' = %(code)s\nout: %(stdout)s\n" "err: %(stderr)s", {'cmd': err.cmd, 'code': err.exit_code, 'stdout': err.stdout, 'stderr': err.stderr}) def _find_vgc_host(self): """Finds vgc-cluster hostname for this box.""" params = [self.VGCCLUSTER, "domain-list", "-1"] try: out, unused = self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = _("Unable to get list of domain members, check that " "the cluster is running.") raise exception.BrickException(message=msg) domain = out.splitlines() params = ["ip", "addr", "list"] try: out, unused = self._execute(*params, run_as_root=False) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = _("Unable to get list of IP addresses on this host, " "check permissions and networking.") raise exception.BrickException(message=msg) nets = out.splitlines() for host in domain: try: ip = socket.gethostbyname(host) for l in nets: x = l.strip() if x.startswith("inet %s/" % ip): return host except socket.error: pass msg = _("Current host isn't part of HGST domain.") raise exception.BrickException(message=msg) def _hostname(self): """Returns hostname to use for cluster operations on this box.""" if self._vgc_host is None: self._vgc_host = self._find_vgc_host() return self._vgc_host def get_search_path(self): return "/dev" def get_volume_paths(self, connection_properties): path = ("%(path)s/%(name)s" % {'path': self.get_search_path(), 'name': connection_properties['name']}) volume_path = None if os.path.exists(path): volume_path = path return [volume_path] @utils.trace def connect_volume(self, connection_properties): """Attach a Space volume to running host. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties for HGST must include: name - Name of space to attach :type connection_properties: dict :returns: dict """ if connection_properties is None: msg = _("Connection properties passed in as None.") raise exception.BrickException(message=msg) if 'name' not in connection_properties: msg = _("Connection properties missing 'name' field.") raise exception.BrickException(message=msg) device_info = { 'type': 'block', 'device': connection_properties['name'], 'path': '/dev/' + connection_properties['name'] } volname = device_info['device'] params = [self.VGCCLUSTER, 'space-set-apphosts'] params += ['-n', volname] params += ['-A', self._hostname()] params += ['--action', 'ADD'] try: self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = (_("Unable to set apphost for space %s") % volname) raise exception.BrickException(message=msg) return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. For HGST must include: name - Name of space to detach noremovehost - Host which should never be removed :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ if connection_properties is None: msg = _("Connection properties passed in as None.") raise exception.BrickException(message=msg) if 'name' not in connection_properties: msg = _("Connection properties missing 'name' field.") raise exception.BrickException(message=msg) if 'noremovehost' not in connection_properties: msg = _("Connection properties missing 'noremovehost' field.") raise exception.BrickException(message=msg) if connection_properties['noremovehost'] != self._hostname(): params = [self.VGCCLUSTER, 'space-set-apphosts'] params += ['-n', connection_properties['name']] params += ['-A', self._hostname()] params += ['--action', 'DELETE'] try: self._execute(*params, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as err: self._log_cli_err(err) msg = (_("Unable to set apphost for space %s") % connection_properties['name']) raise exception.BrickException(message=msg) def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/huawei.py0000664000175000017500000001724200000000000023072 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from oslo_concurrency import lockutils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') class HuaweiStorHyperConnector(base.BaseLinuxConnector): """"Connector class to attach/detach SDSHypervisor volumes.""" attached_success_code = 0 has_been_attached_code = 50151401 attach_mnid_done_code = 50151405 vbs_unnormal_code = 50151209 not_mount_node_code = 50155007 iscliexist = True def __init__(self, root_helper, driver=None, *args, **kwargs): self.cli_path = os.getenv('HUAWEISDSHYPERVISORCLI_PATH') if not self.cli_path: self.cli_path = '/usr/local/bin/sds/sds_cli' LOG.debug("CLI path is not configured, using default %s.", self.cli_path) if not os.path.isfile(self.cli_path): self.iscliexist = False LOG.error('SDS CLI file not found, ' 'HuaweiStorHyperConnector init failed.') super(HuaweiStorHyperConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HuaweiStor connector properties.""" return {} def get_search_path(self): # TODO(walter-boring): Where is the location on the filesystem to # look for Huawei volumes to show up? return None def get_all_available_volumes(self, connection_properties=None): # TODO(walter-boring): what to return here for all Huawei volumes ? return [] def get_volume_paths(self, connection_properties): volume_path = None try: volume_path = self._get_volume_path(connection_properties) except Exception: msg = _("Couldn't find a volume.") LOG.warning(msg) raise exception.BrickException(message=msg) return [volume_path] def _get_volume_path(self, connection_properties): out = self._query_attached_volume( connection_properties['volume_id']) if not out or int(out['ret_code']) != 0: msg = _("Couldn't find attached volume.") LOG.error(msg) raise exception.BrickException(message=msg) return out['dev_addr'] @utils.trace @synchronized('connect_volume', external=True) def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ LOG.debug("Connect_volume connection properties: %s.", connection_properties) out = self._attach_volume(connection_properties['volume_id']) if not out or int(out['ret_code']) not in (self.attached_success_code, self.has_been_attached_code, self.attach_mnid_done_code): msg = (_("Attach volume failed, " "error code is %s") % out['ret_code']) raise exception.BrickException(message=msg) try: volume_path = self._get_volume_path(connection_properties) except Exception: msg = _("query attached volume failed or volume not attached.") LOG.error(msg) raise exception.BrickException(message=msg) device_info = {'type': 'block', 'path': volume_path} return device_info @utils.trace @synchronized('connect_volume', external=True) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ LOG.debug("Disconnect_volume: %s.", connection_properties) out = self._detach_volume(connection_properties['volume_id']) if not out or int(out['ret_code']) not in (self.attached_success_code, self.vbs_unnormal_code, self.not_mount_node_code): msg = (_("Disconnect_volume failed, " "error code is %s") % out['ret_code']) raise exception.BrickException(message=msg) def is_volume_connected(self, volume_name): """Check if volume already connected to host""" LOG.debug('Check if volume %s already connected to a host.', volume_name) out = self._query_attached_volume(volume_name) if out: return int(out['ret_code']) == 0 return False def _attach_volume(self, volume_name): return self._cli_cmd('attach', volume_name) def _detach_volume(self, volume_name): return self._cli_cmd('detach', volume_name) def _query_attached_volume(self, volume_name): return self._cli_cmd('querydev', volume_name) def _cli_cmd(self, method, volume_name): LOG.debug("Enter into _cli_cmd.") if not self.iscliexist: msg = _("SDS command line doesn't exist, " "can't execute SDS command.") raise exception.BrickException(message=msg) if not method or volume_name is None: return cmd = [self.cli_path, '-c', method, '-v', volume_name] out, clilog = self._execute(*cmd, run_as_root=False, root_helper=self._root_helper) analyse_result = self._analyze_output(out) LOG.debug('%(method)s volume returns %(analyse_result)s.', {'method': method, 'analyse_result': analyse_result}) if clilog: LOG.error("SDS CLI output some log: %s.", clilog) return analyse_result def _analyze_output(self, out): LOG.debug("Enter into _analyze_output.") if out: analyse_result = {} out_temp = out.split('\n') for line in out_temp: LOG.debug("Line is %s.", line) if line.find('=') != -1: key, val = line.split('=', 1) LOG.debug("%(key)s = %(val)s", {'key': key, 'val': val}) if key in ['ret_code', 'ret_desc', 'dev_addr']: analyse_result[key] = val return analyse_result else: return None def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/iscsi.py0000664000175000017500000015734400000000000022732 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import copy import glob import os import re import time from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import excutils from oslo_utils import strutils from os_brick import exception from os_brick import executor from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator.connectors import base_iscsi from os_brick.initiator import utils as initiator_utils from os_brick import utils synchronized = lockutils.synchronized_with_prefix('os-brick-') LOG = logging.getLogger(__name__) class ISCSIConnector(base.BaseLinuxConnector, base_iscsi.BaseISCSIConnector): """Connector class to attach/detach iSCSI volumes.""" supported_transports = ['be2iscsi', 'bnx2i', 'cxgb3i', 'default', 'cxgb4i', 'qla4xxx', 'ocs', 'iser', 'tcp'] VALID_SESSIONS_PREFIX = ('tcp:', 'iser:') def __init__(self, root_helper, driver=None, execute=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, transport='default', *args, **kwargs): super(ISCSIConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, transport=transport, *args, **kwargs) self.use_multipath = use_multipath self.transport = self._validate_iface_transport(transport) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The iSCSI connector properties.""" props = {} iscsi = ISCSIConnector(root_helper=root_helper, execute=kwargs.get('execute')) initiator = iscsi.get_initiator() if initiator: props['initiator'] = initiator return props def get_search_path(self): """Where do we look for iSCSI based volumes.""" return '/dev/disk/by-path' def get_volume_paths(self, connection_properties): """Get the list of existing paths for a volume. This method's job is to simply report what might/should already exist for a volume. We aren't trying to attach/discover a new volume, but find any existing paths for a volume we think is already attached. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ volume_paths = [] # if there are no sessions, then target_portal won't exist if (('target_portal' not in connection_properties) and ('target_portals' not in connection_properties)): return volume_paths # Don't try and connect to the portals in the list as # this can create empty iSCSI sessions to hosts if they # didn't exist previously. # We are simply trying to find any existing volumes with # already connected sessions. host_devices = self._get_potential_volume_paths(connection_properties) for path in host_devices: if os.path.exists(path): volume_paths.append(path) return volume_paths def _get_iscsi_sessions_full(self): """Get iSCSI session information as a list of tuples. Uses iscsiadm -m session and from a command output like tcp: [1] 192.168.121.250:3260,1 iqn.2010-10.org.openstack: volume- (non-flash) This method will drop the node type and return a list like this: [('tcp:', '1', '192.168.121.250:3260', '1', 'iqn.2010-10.org.openstack:volume-')] """ out, err = self._run_iscsi_session() if err: LOG.warning("iscsiadm stderr output when getting sessions: %s", err) # Parse and clean the output from iscsiadm, which is in the form of: # transport_name: [session_id] ip_address:port,tpgt iqn node_type lines = [] for line in out.splitlines(): if line: info = line.split() sid = info[1][1:-1] portal, tpgt = info[2].split(',') lines.append((info[0], sid, portal, tpgt, info[3])) return lines def _get_iscsi_nodes(self): """Get iSCSI node information (portal, iqn) as a list of tuples. Uses iscsiadm -m node and from a command output like 192.168.121.250:3260,1 iqn.2010-10.org.openstack:volume This method will drop the tpgt and return a list like this: [('192.168.121.250:3260', 'iqn.2010-10.org.openstack:volume')] """ out, err = self._execute('iscsiadm', '-m', 'node', run_as_root=True, root_helper=self._root_helper, check_exit_code=False) if err: LOG.warning("Couldn't find iSCSI nodes because iscsiadm err: %s", err) return [] # Parse and clean the output from iscsiadm which is in the form of: # ip_addresss:port,tpgt iqn lines = [] for line in out.splitlines(): if line: info = line.split() try: lines.append((info[0].split(',')[0], info[1])) except IndexError: pass return lines def _get_iscsi_sessions(self): """Return portals for all existing sessions.""" # entry: [tcp, [1], 192.168.121.250:3260,1 ...] return [entry[2] for entry in self._get_iscsi_sessions_full()] def _get_ips_iqns_luns(self, connection_properties, discover=True, is_disconnect_call=False): """Build a list of ips, iqns, and luns. Used when doing singlepath and multipath, and we have 4 cases: - All information is in the connection properties - We have to do an iSCSI discovery to get the information - We don't want to do another discovery and we query the discoverydb - Discovery failed because it was actually a single pathed attachment :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param discover: Whether doing an iSCSI discovery is acceptable. :type discover: bool :param is_disconnect_call: Whether this is a call coming from a user disconnect_volume call or a call from some other operation's cleanup. :type is_disconnect_call: bool :returns: list of tuples of (ip, iqn, lun) """ # There are cases where we don't know if the local attach was done # using multipathing or single pathing, so assume multipathing. try: if ('target_portals' in connection_properties and 'target_iqns' in connection_properties): # Use targets specified by connection_properties ips_iqns_luns = self._get_all_targets(connection_properties) else: method = (self._discover_iscsi_portals if discover else self._get_discoverydb_portals) ips_iqns_luns = method(connection_properties) except exception.TargetPortalNotFound: # Discovery failed, on disconnect this will happen if we # are detaching a single pathed connection, so we use the # connection properties to return the tuple. if is_disconnect_call: return self._get_all_targets(connection_properties) raise except Exception: LOG.exception('Exception encountered during portal discovery') if 'target_portals' in connection_properties: raise exception.TargetPortalsNotFound( target_portals=connection_properties['target_portals']) if 'target_portal' in connection_properties: raise exception.TargetPortalNotFound( target_portal=connection_properties['target_portal']) raise if not connection_properties.get('target_iqns'): # There are two types of iSCSI multipath devices. One which # shares the same iqn between multiple portals, and the other # which use different iqns on different portals. # Try to identify the type by checking the iscsiadm output # if the iqn is used by multiple portals. If it is, it's # the former, so use the supplied iqn. Otherwise, it's the # latter, so try the ip,iqn combinations to find the targets # which constitutes the multipath device. main_iqn = connection_properties['target_iqn'] all_portals = {(ip, lun) for ip, iqn, lun in ips_iqns_luns} match_portals = {(ip, lun) for ip, iqn, lun in ips_iqns_luns if iqn == main_iqn} if len(all_portals) == len(match_portals): ips_iqns_luns = [(p[0], main_iqn, p[1]) for p in all_portals] return ips_iqns_luns def _get_potential_volume_paths(self, connection_properties): """Build a list of potential volume paths that exist. Given a list of target_portals in the connection_properties, a list of paths might exist on the system during discovery. This method's job is to build that list of potential paths for a volume that might show up. This is only used during get_volume_paths time, we are looking to find a list of existing volume paths for the connection_properties. In this case, we don't want to connect to the portal. If we blindly try and connect to a portal, it could create a new iSCSI session that didn't exist previously, and then leave it stale. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: list """ if self.use_multipath: LOG.info("Multipath discovery for iSCSI enabled") # Multipath installed, discovering other targets if available host_devices = self._get_device_path(connection_properties) else: LOG.info("Multipath discovery for iSCSI not enabled.") iscsi_sessions = self._get_iscsi_sessions() host_devices = set() for props in self._iterate_all_targets(connection_properties): # If we aren't trying to connect to the portal, we # want to find ALL possible paths from all of the # alternate portals if props['target_portal'] in iscsi_sessions: paths = self._get_device_path(props) host_devices.update(paths) host_devices = list(host_devices) return host_devices def set_execute(self, execute): super(ISCSIConnector, self).set_execute(execute) self._linuxscsi.set_execute(execute) def _validate_iface_transport(self, transport_iface): """Check that given iscsi_iface uses only supported transports Accepted transport names for provided iface param are be2iscsi, bnx2i, cxgb3i, cxgb4i, default, qla4xxx, ocs, iser or tcp. Note the difference between transport and iface; unlike default(iscsi_tcp)/iser, this is not one and the same for offloaded transports, where the default format is transport_name.hwaddress :param transport_iface: The iscsi transport type. :type transport_iface: str :returns: str """ # Note that default(iscsi_tcp) and iser do not require a separate # iface file, just the transport is enough and do not need to be # validated. This is not the case for the other entries in # supported_transports array. if transport_iface in ['default', 'iser']: return transport_iface # Will return (6) if iscsi_iface file was not found, or (2) if iscsid # could not be contacted out = self._run_iscsiadm_bare(['-m', 'iface', '-I', transport_iface], check_exit_code=[0, 2, 6])[0] or "" LOG.debug("iscsiadm %(iface)s configuration: stdout=%(out)s.", {'iface': transport_iface, 'out': out}) for data in [line.split() for line in out.splitlines()]: if data[0] == 'iface.transport_name': if data[2] in self.supported_transports: return transport_iface LOG.warning("No useable transport found for iscsi iface %s. " "Falling back to default transport.", transport_iface) return 'default' def _get_transport(self): return self.transport def _get_discoverydb_portals(self, connection_properties): """Retrieve iscsi portals information from the discoverydb. Example of discoverydb command output: SENDTARGETS: DiscoveryAddress: 192.168.1.33,3260 DiscoveryAddress: 192.168.1.2,3260 Target: iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-20170531114245.9eff88 Portal: 192.168.1.3:3260,1 Iface Name: default Portal: 192.168.1.2:3260,1 Iface Name: default Target: iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-20170531114447.9eff88 Portal: 192.168.1.3:3260,1 Iface Name: default Portal: 192.168.1.2:3260,1 Iface Name: default DiscoveryAddress: 192.168.1.38,3260 iSNS: No targets found. STATIC: No targets found. FIRMWARE: No targets found. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: list of tuples of (ip, iqn, lun) """ ip, port = connection_properties['target_portal'].rsplit(':', 1) # NOTE(geguileo): I don't know if IPv6 will be reported with [] # or not, so we'll make them optional. ip = ip.replace('[', r'\[?').replace(']', r'\]?') out = self._run_iscsiadm_bare(['-m', 'discoverydb', '-o', 'show', '-P', 1])[0] or "" regex = ''.join(('^SENDTARGETS:\n.*?^DiscoveryAddress: ', ip, ',', port, '.*?\n(.*?)^(?:DiscoveryAddress|iSNS):.*')) LOG.debug('Regex to get portals from discoverydb: %s', regex) info = re.search(regex, out, re.DOTALL | re.MULTILINE) ips = [] iqns = [] if info: iscsi_transport = ('iser' if self._get_transport() == 'iser' else 'default') iface = 'Iface Name: ' + iscsi_transport current_iqn = '' current_ip = '' for line in info.group(1).splitlines(): line = line.strip() if line.startswith('Target:'): current_iqn = line[8:] elif line.startswith('Portal:'): current_ip = line[8:].split(',')[0] elif line.startswith(iface): if current_iqn and current_ip: iqns.append(current_iqn) ips.append(current_ip) current_ip = '' if not iqns: raise exception.TargetPortalsNotFound( _('Unable to find target portals information on discoverydb.')) luns = self._get_luns(connection_properties, iqns) return list(zip(ips, iqns, luns)) def _discover_iscsi_portals(self, connection_properties): out = None iscsi_transport = ('iser' if self._get_transport() == 'iser' else 'default') if connection_properties.get('discovery_auth_method'): try: self._run_iscsiadm_update_discoverydb(connection_properties, iscsi_transport) except putils.ProcessExecutionError as exception: # iscsiadm returns 6 for "db record not found" if exception.exit_code == 6: # Create a new record for this target and update the db self._run_iscsiadm_bare( ['-m', 'discoverydb', '-t', 'sendtargets', '-p', connection_properties['target_portal'], '-I', iscsi_transport, '--op', 'new'], check_exit_code=[0, 255]) self._run_iscsiadm_update_discoverydb( connection_properties ) else: LOG.error("Unable to find target portal: " "%(target_portal)s.", {'target_portal': connection_properties[ 'target_portal']}) raise old_node_startups = self._get_node_startup_values( connection_properties) out = self._run_iscsiadm_bare( ['-m', 'discoverydb', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal'], '--discover'], check_exit_code=[0, 255])[0] or "" self._recover_node_startup_values(connection_properties, old_node_startups) else: old_node_startups = self._get_node_startup_values( connection_properties) out = self._run_iscsiadm_bare( ['-m', 'discovery', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal']], check_exit_code=[0, 255])[0] or "" self._recover_node_startup_values(connection_properties, old_node_startups) ips, iqns = self._get_target_portals_from_iscsiadm_output(out) luns = self._get_luns(connection_properties, iqns) return list(zip(ips, iqns, luns)) def _run_iscsiadm_update_discoverydb(self, connection_properties, iscsi_transport='default'): return self._execute( 'iscsiadm', '-m', 'discoverydb', '-t', 'sendtargets', '-I', iscsi_transport, '-p', connection_properties['target_portal'], '--op', 'update', '-n', "discovery.sendtargets.auth.authmethod", '-v', connection_properties['discovery_auth_method'], '-n', "discovery.sendtargets.auth.username", '-v', connection_properties['discovery_auth_username'], '-n', "discovery.sendtargets.auth.password", '-v', connection_properties['discovery_auth_password'], run_as_root=True, root_helper=self._root_helper) @utils.trace @synchronized('extend_volume', external=True) def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an iSCSI volume. """ LOG.info("Extend volume for %s", strutils.mask_dict_password(connection_properties)) volume_paths = self.get_volume_paths(connection_properties) LOG.info("Found paths for volume %s", volume_paths) if volume_paths: return self._linuxscsi.extend_volume( volume_paths, use_multipath=self.use_multipath) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': strutils.mask_dict_password( connection_properties)}) raise exception.VolumePathsNotFound() @utils.trace @synchronized('connect_volume', external=True) def connect_volume(self, connection_properties): """Attach the volume to instance_name. :param connection_properties: The valid dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict connection_properties for iSCSI must include: target_portal(s) - ip and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume Note that plural keys may be used when use_multipath=True """ try: if self.use_multipath: return self._connect_multipath_volume(connection_properties) return self._connect_single_volume(connection_properties) except Exception: # NOTE(geguileo): By doing the cleanup here we ensure we only do # the logins once for multipath if they succeed, but retry if they # don't, which helps on bad network cases. with excutils.save_and_reraise_exception(): self._cleanup_connection(connection_properties, force=True) @utils.retry(exceptions=(exception.VolumeDeviceNotFound)) def _get_device_link(self, wwn, device, mpath): # These are the default symlinks that should always be there if mpath: symlink = '/dev/disk/by-id/dm-uuid-mpath-' + mpath else: symlink = '/dev/disk/by-id/scsi-' + wwn # If default symlinks are not there just search for anything that links # to our device. In my experience this will return the last added link # first, so if we are going to succeed this should be fast. if not os.path.realpath(symlink) == device: links_path = '/dev/disk/by-id/' for symlink in os.listdir(links_path): symlink = links_path + symlink if os.path.realpath(symlink) == device: break else: # Raising this will trigger the next retry raise exception.VolumeDeviceNotFound(device='/dev/disk/by-id') return symlink def _get_connect_result(self, con_props, wwn, devices_names, mpath=None): device = '/dev/' + (mpath or devices_names[0]) # NOTE(geguileo): This is only necessary because of the current # encryption flow that requires that connect_volume returns a symlink # because first we do the volume attach, then the libvirt config is # generated using the path returned by the atach, and then we do the # encryption attach, which is forced to preserve the path that was used # in the libvirt config. If we fix that flow in OS-brick, Nova, and # Cinder we can remove this and just return the real path. if con_props.get('encrypted'): device = self._get_device_link(wwn, device, mpath) result = {'type': 'block', 'scsi_wwn': wwn, 'path': device} if mpath: result['multipath_id'] = wwn return result @utils.retry(exceptions=(exception.VolumeDeviceNotFound)) def _connect_single_volume(self, connection_properties): """Connect to a volume using a single path.""" data = {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': [], 'just_added_devices': []} for props in self._iterate_all_targets(connection_properties): self._connect_vol(self.device_scan_attempts, props, data) found_devs = data['found_devices'] if found_devs: for __ in range(10): wwn = self._linuxscsi.get_sysfs_wwn(found_devs) if wwn: break time.sleep(1) else: LOG.debug('Could not find the WWN for %s.', found_devs[0]) return self._get_connect_result(connection_properties, wwn, found_devs) # If we failed we must cleanup the connection, as we could be # leaving the node entry if it's not being used by another device. ips_iqns_luns = ((props['target_portal'], props['target_iqn'], props['target_lun']), ) self._cleanup_connection(props, ips_iqns_luns, force=True, ignore_errors=True) # Reset connection result values for next try data.update(num_logins=0, failed_logins=0, found_devices=[]) raise exception.VolumeDeviceNotFound(device='') def _connect_vol(self, rescans, props, data): """Make a connection to a volume, send scans and wait for the device. This method is specifically designed to support multithreading and share the results via a shared dictionary with fixed keys, which is thread safe. Since the heaviest operations are run via subprocesses we don't worry too much about the GIL or how the eventlets will handle the context switching. The method will only try to log in once, since iscsid's initiator already tries 8 times by default to do the login, or whatever value we have as node.session.initial_login_retry_max in our system. Shared dictionary has the following keys: - stop_connecting: When the caller wants us to stop the rescans - num_logins: Count of how many threads have successfully logged in - failed_logins: Count of how many threads have failed to log in - stopped_threads: How many threads have finished. This may be different than num_logins + failed_logins, since some threads may still be waiting for a device. - found_devices: List of devices the connections have found - just_added_devices: Devices that have been found and still have not been processed by the main thread that manages all the connecting threads. :param rescans: Number of rescans to perform before giving up. :param props: Properties of the connection. :param data: Shared data. """ device = hctl = None portal = props['target_portal'] try: session, manual_scan = self._connect_to_iscsi_portal(props) except Exception: LOG.exception('Exception connecting to %s', portal) session = None if session: do_scans = rescans > 0 or manual_scan # Scan is sent on connect by iscsid, but we must do it manually on # manual scan mode. This scan cannot count towards total rescans. if manual_scan: num_rescans = -1 seconds_next_scan = 0 else: num_rescans = 0 seconds_next_scan = 4 data['num_logins'] += 1 LOG.debug('Connected to %s', portal) while do_scans: try: if not hctl: hctl = self._linuxscsi.get_hctl(session, props['target_lun']) if hctl: if seconds_next_scan <= 0: num_rescans += 1 self._linuxscsi.scan_iscsi(*hctl) # 4 seconds on 1st rescan, 9s on 2nd, 16s on 3rd seconds_next_scan = (num_rescans + 2) ** 2 device = self._linuxscsi.device_name_by_hctl(session, hctl) if device: break except Exception: LOG.exception('Exception scanning %s', portal) pass do_scans = (num_rescans <= rescans and not (device or data['stop_connecting'])) if do_scans: time.sleep(1) seconds_next_scan -= 1 if device: LOG.debug('Connected to %s using %s', device, strutils.mask_password(props)) else: LOG.warning('LUN %(lun)s on iSCSI portal %(portal)s not found ' 'on sysfs after logging in.', {'lun': props['target_lun'], 'portal': portal}) else: LOG.warning('Failed to connect to iSCSI portal %s.', portal) data['failed_logins'] += 1 if device: data['found_devices'].append(device) data['just_added_devices'].append(device) data['stopped_threads'] += 1 @utils.retry(exceptions=(exception.VolumeDeviceNotFound)) def _connect_multipath_volume(self, connection_properties): """Connect to a multipathed volume launching parallel login requests. We will be doing parallel login requests, which will considerably speed up the process when we have flaky connections. We'll always try to return a multipath device even if there's only one path discovered, that way we can return once we have logged in in all the portals, because the paths will come up later. To make this possible we tell multipathd that the wwid is a multipath as soon as we have one device, and then hint multipathd to reconsider that volume for a multipath asking to add the path, because even if it's already known by multipathd it would have been discarded if it was the first time this volume was seen here. """ wwn = mpath = None wwn_added = last_try_on = False found = [] just_added_devices = [] # Dict used to communicate with threads as detailed in _connect_vol data = {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': found, 'just_added_devices': just_added_devices} ips_iqns_luns = self._get_ips_iqns_luns(connection_properties) # Launch individual threads for each session with the own properties retries = self.device_scan_attempts threads = [] for ip, iqn, lun in ips_iqns_luns: props = connection_properties.copy() props.update(target_portal=ip, target_iqn=iqn, target_lun=lun) # NOTE(yenai): The method _connect_vol is used for parallelize # logins, we shouldn't give these arguments; and it will make a # mess in the debug message in _connect_vol. So, kick them out: for key in ('target_portals', 'target_iqns', 'target_luns'): props.pop(key, None) threads.append(executor.Thread(target=self._connect_vol, args=(retries, props, data))) for thread in threads: thread.start() # Continue until: # - All connection attempts have finished and none has logged in # - Multipath has been found and connection attempts have either # finished or have already logged in # - We have finished in all threads, logged in, found some device, and # 10 seconds have passed, which should be enough with up to 10% # network package drops. while not ((len(ips_iqns_luns) == data['stopped_threads'] and not found) or (mpath and len(ips_iqns_luns) == data['num_logins'] + data['failed_logins'])): # We have devices but we don't know the wwn yet if not wwn and found: wwn = self._linuxscsi.get_sysfs_wwn(found, mpath) if not mpath and found: mpath = self._linuxscsi.find_sysfs_multipath_dm(found) # We have the wwn but not a multipath if wwn and not(mpath or wwn_added): # Tell multipathd that this wwn is a multipath and hint # multipathd to recheck all the devices we have just # connected. We only do this once, since for any new # device multipathd will already know it is a multipath. # This is only useful if we have multipathd configured with # find_multipaths set to yes, and has no effect if it's set # to no. wwn_added = self._linuxscsi.multipath_add_wwid(wwn) while not mpath and just_added_devices: device_path = '/dev/' + just_added_devices.pop(0) self._linuxscsi.multipath_add_path(device_path) mpath = self._linuxscsi.find_sysfs_multipath_dm(found) # Give some extra time after all threads have finished. if (not last_try_on and found and len(ips_iqns_luns) == data['stopped_threads']): LOG.debug('All connection threads finished, giving 10 seconds ' 'for dm to appear.') last_try_on = time.time() + 10 elif last_try_on and last_try_on < time.time(): break time.sleep(1) data['stop_connecting'] = True for thread in threads: thread.join() # If we haven't found any devices let the caller do the cleanup if not found: raise exception.VolumeDeviceNotFound(device='') # NOTE(geguileo): If we cannot find the dm it's because all paths are # really bad, so we might as well raise a not found exception, but # in our best effort we'll return a device even if it's probably # useless. if not mpath: LOG.warning('No dm was created, connection to volume is probably ' 'bad and will perform poorly.') elif not wwn: wwn = self._linuxscsi.get_sysfs_wwn(found, mpath) return self._get_connect_result(connection_properties, wwn, found, mpath) def _get_connection_devices(self, connection_properties, ips_iqns_luns=None, is_disconnect_call=False): """Get map of devices by sessions from our connection. For each of the TCP sessions that correspond to our connection properties we generate a map of (ip, iqn) to (belong, other) where belong is a set of devices in that session that populated our system when we did a connection using connection properties, and other are any other devices that share that same session but are the result of connecting with different connection properties. We also include all nodes from our connection that don't have a session. If ips_iqns_luns parameter is provided connection_properties won't be used to get them. When doing multipath we may not have all the information on the connection properties (sendtargets was used on connect) so we may have to retrieve the info from the discoverydb. Call _get_ips_iqns_luns to do the right things. This method currently assumes that it's only called by the _cleanup_conection method. """ if not ips_iqns_luns: # This is a cleanup, don't do discovery ips_iqns_luns = self._get_ips_iqns_luns( connection_properties, discover=False, is_disconnect_call=is_disconnect_call) LOG.debug('Getting connected devices for (ips,iqns,luns)=%s', ips_iqns_luns) nodes = self._get_iscsi_nodes() sessions = self._get_iscsi_sessions_full() # Use (portal, iqn) to map the session value sessions_map = {(s[2], s[4]): s[1] for s in sessions if s[0] in self.VALID_SESSIONS_PREFIX} # device_map will keep a tuple with devices from the connection and # others that don't belong to this connection" (belong, others) device_map = collections.defaultdict(lambda: (set(), set())) for ip, iqn, lun in ips_iqns_luns: session = sessions_map.get((ip, iqn)) # Our nodes that don't have a session will be returned as empty if not session: if (ip, iqn) in nodes: device_map[(ip, iqn)] = (set(), set()) continue # Get all devices for the session paths = glob.glob('/sys/class/scsi_host/host*/device/session' + session + '/target*/*:*:*:*/block/*') belong, others = device_map[(ip, iqn)] for path in paths: __, hctl, __, device = path.rsplit('/', 3) lun_path = int(hctl.rsplit(':', 1)[-1]) # For partitions turn them into the whole device: sde1 -> sde device = device.strip('0123456789') if lun_path == lun: belong.add(device) else: others.add(device) LOG.debug('Resulting device map %s', device_map) return device_map @utils.trace @synchronized('connect_volume', external=True) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach the volume from instance_name. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict that must include: target_portal(s) - IP and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :type ignore_errors: bool """ return self._cleanup_connection(connection_properties, force=force, ignore_errors=ignore_errors, device_info=device_info, is_disconnect_call=True) def _cleanup_connection(self, connection_properties, ips_iqns_luns=None, force=False, ignore_errors=False, device_info=None, is_disconnect_call=False): """Cleans up connection flushing and removing devices and multipath. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict that must include: target_portal(s) - IP and optional port target_iqn(s) - iSCSI Qualified Name target_lun(s) - LUN id of the volume :param ips_iqns_luns: Use this list of tuples instead of information from the connection_properties. :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :param device_info: Attached device information. :param is_disconnect_call: Whether this is a call coming from a user disconnect_volume call or a call from some other operation's cleanup. :type is_disconnect_call: bool :type ignore_errors: bool """ exc = exception.ExceptionChainer() try: devices_map = self._get_connection_devices(connection_properties, ips_iqns_luns, is_disconnect_call) except exception.TargetPortalNotFound as exc: # When discovery sendtargets failed on connect there is no # information in the discoverydb, so there's nothing to clean. LOG.debug('Skipping cleanup %s', exc) return # Remove devices and multipath from this connection remove_devices = set() for remove, __ in devices_map.values(): remove_devices.update(remove) path_used = self._linuxscsi.get_dev_path(connection_properties, device_info) was_multipath = (path_used.startswith('/dev/dm-') or 'mpath' in path_used) multipath_name = self._linuxscsi.remove_connection( remove_devices, force, exc, path_used, was_multipath) # Disconnect sessions and remove nodes that are left without devices disconnect = [conn for conn, (__, keep) in devices_map.items() if not keep] self._disconnect_connection(connection_properties, disconnect, force, exc) # If flushing the multipath failed before, try now after we have # removed the devices and we may have even logged off (only reaches # here with multipath_name if force=True). if multipath_name: LOG.debug('Flushing again multipath %s now that we removed the ' 'devices.', multipath_name) self._linuxscsi.flush_multipath_device(multipath_name) if exc: LOG.warning('There were errors removing %s, leftovers may remain ' 'in the system', remove_devices) if not ignore_errors: raise exc def _munge_portal(self, target): """Remove brackets from portal. In case IPv6 address was used the udev path should not contain any brackets. Udev code specifically forbids that. """ portal, iqn, lun = target return (portal.replace('[', '').replace(']', ''), iqn, self._linuxscsi.process_lun_id(lun)) def _get_device_path(self, connection_properties): if self._get_transport() == "default": return ["/dev/disk/by-path/ip-%s-iscsi-%s-lun-%s" % self._munge_portal(x) for x in self._get_all_targets(connection_properties)] else: # we are looking for paths in the format : # /dev/disk/by-path/ # pci-XXXX:XX:XX.X-ip-PORTAL:PORT-iscsi-IQN-lun-LUN_ID device_list = [] for x in self._get_all_targets(connection_properties): look_for_device = glob.glob( '/dev/disk/by-path/*ip-%s-iscsi-%s-lun-%s' % self._munge_portal(x)) if look_for_device: device_list.extend(look_for_device) return device_list def get_initiator(self): """Secure helper to read file as root.""" file_path = '/etc/iscsi/initiatorname.iscsi' try: lines, _err = self._execute('cat', file_path, run_as_root=True, root_helper=self._root_helper) for l in lines.split('\n'): if l.startswith('InitiatorName='): return l[l.index('=') + 1:].strip() except putils.ProcessExecutionError: LOG.warning("Could not find the iSCSI Initiator File %s", file_path) return None def _run_iscsiadm(self, connection_properties, iscsi_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) attempts = kwargs.pop('attempts', 1) delay_on_retry = kwargs.pop('delay_on_retry', True) (out, err) = self._execute('iscsiadm', '-m', 'node', '-T', connection_properties['target_iqn'], '-p', connection_properties['target_portal'], *iscsi_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code, attempts=attempts, delay_on_retry=delay_on_retry) msg = ("iscsiadm %(iscsi_command)s: stdout=%(out)s stderr=%(err)s" % {'iscsi_command': iscsi_command, 'out': out, 'err': err}) # don't let passwords be shown in log output LOG.debug(strutils.mask_password(msg)) return (out, err) def _iscsiadm_update(self, connection_properties, property_key, property_value, **kwargs): iscsi_command = ('--op', 'update', '-n', property_key, '-v', property_value) return self._run_iscsiadm(connection_properties, iscsi_command, **kwargs) def _get_target_portals_from_iscsiadm_output(self, output): # return both portals and iqns as 2 lists # # as we are parsing a command line utility, allow for the # possibility that additional debug data is spewed in the # stream, and only grab actual ip / iqn lines. ips = [] iqns = [] for data in [line.split() for line in output.splitlines()]: if len(data) == 2 and data[1].startswith('iqn.'): ips.append(data[0].split(',')[0]) iqns.append(data[1]) return ips, iqns def _connect_to_iscsi_portal(self, connection_properties): """Safely connect to iSCSI portal-target and return the session id.""" portal = connection_properties['target_portal'].split(",")[0] target_iqn = connection_properties['target_iqn'] lock_name = f'connect_to_iscsi_portal-{portal}-{target_iqn}' method = synchronized( lock_name, external=True)(self._connect_to_iscsi_portal_unsafe) return method(connection_properties) @utils.retry((exception.BrickException)) def _connect_to_iscsi_portal_unsafe(self, connection_properties): """Connect to an iSCSI portal-target an return the session id.""" portal = connection_properties['target_portal'].split(",")[0] target_iqn = connection_properties['target_iqn'] # NOTE(vish): If we are on the same host as nova volume, the # discovery makes the target so we don't need to # run --op new. Therefore, we check to see if the # target exists, and if we get 255 (Not Found), then # we run --op new. This will also happen if another # volume is using the same target. # iscsiadm returns 21 for "No records found" after version 2.0-871 LOG.info("Trying to connect to iSCSI portal %s", portal) out, err = self._run_iscsiadm(connection_properties, (), check_exit_code=(0, 21, 255)) if err: out_new, err_new = self._run_iscsiadm(connection_properties, ('--interface', self._get_transport(), '--op', 'new'), check_exit_code=(0, 6)) if err_new: # retry if iscsiadm returns 6 for "database failure" LOG.debug("Retrying to connect to iSCSI portal %s", portal) msg = (_("Encountered database failure for %s.") % (portal)) raise exception.BrickException(msg=msg) # Try to set the scan mode to manual res = self._iscsiadm_update(connection_properties, 'node.session.scan', 'manual', check_exit_code=False) manual_scan = not res[1] # Update global indicator of manual scan support used for # shared_targets locking so we support upgrading open iscsi to a # version supporting the manual scan feature without restarting Nova # or Cinder. initiator_utils.ISCSI_SUPPORTS_MANUAL_SCAN = manual_scan if connection_properties.get('auth_method'): self._iscsiadm_update(connection_properties, "node.session.auth.authmethod", connection_properties['auth_method']) self._iscsiadm_update(connection_properties, "node.session.auth.username", connection_properties['auth_username']) self._iscsiadm_update(connection_properties, "node.session.auth.password", connection_properties['auth_password']) # We exit once we are logged in or once we fail login while True: # Duplicate logins crash iscsiadm after load, so we scan active # sessions to see if the node is logged in. sessions = self._get_iscsi_sessions_full() for s in sessions: # Found our session, return session_id if (s[0] in self.VALID_SESSIONS_PREFIX and portal.lower() == s[2].lower() and s[4] == target_iqn): return s[1], manual_scan try: # exit_code=15 means the session already exists, so it should # be regarded as successful login. self._run_iscsiadm(connection_properties, ("--login",), check_exit_code=(0, 15, 255)) except putils.ProcessExecutionError as err: LOG.warning('Failed to login iSCSI target %(iqn)s on portal ' '%(portal)s (exit code %(err)s).', {'iqn': target_iqn, 'portal': portal, 'err': err.exit_code}) return None, None self._iscsiadm_update(connection_properties, "node.startup", "automatic") def _disconnect_from_iscsi_portal(self, connection_properties): self._iscsiadm_update(connection_properties, "node.startup", "manual", check_exit_code=[0, 21, 255]) self._run_iscsiadm(connection_properties, ("--logout",), check_exit_code=[0, 21, 255]) self._run_iscsiadm(connection_properties, ('--op', 'delete'), check_exit_code=[0, 21, 255], attempts=5, delay_on_retry=True) def _disconnect_connection(self, connection_properties, connections, force, exc): LOG.debug('Disconnecting from: %s', connections) props = connection_properties.copy() for ip, iqn in connections: props['target_portal'] = ip props['target_iqn'] = iqn with exc.context(force, 'Disconnect from %s %s failed', ip, iqn): self._disconnect_from_iscsi_portal(props) def _run_iscsi_session(self): (out, err) = self._run_iscsiadm_bare(('-m', 'session'), check_exit_code=[0, 21, 255]) LOG.debug("iscsi session list stdout=%(out)s stderr=%(err)s", {'out': out, 'err': err}) return (out, err) def _run_iscsiadm_bare(self, iscsi_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) (out, err) = self._execute('iscsiadm', *iscsi_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code) LOG.debug("iscsiadm %(iscsi_command)s: stdout=%(out)s stderr=%(err)s", {'iscsi_command': iscsi_command, 'out': out, 'err': err}) return (out, err) def _run_multipath(self, multipath_command, **kwargs): check_exit_code = kwargs.pop('check_exit_code', 0) (out, err) = self._execute('multipath', *multipath_command, run_as_root=True, root_helper=self._root_helper, check_exit_code=check_exit_code) LOG.debug("multipath %(multipath_command)s: " "stdout=%(out)s stderr=%(err)s", {'multipath_command': multipath_command, 'out': out, 'err': err}) return (out, err) def _get_node_startup_values(self, connection_properties): # Exit code 21 (ISCSI_ERR_NO_OBJS_FOUND) occurs when no nodes # exist - must consider this an empty (successful) result. out, __ = self._run_iscsiadm_bare( ['-m', 'node', '--op', 'show', '-p', connection_properties['target_portal']], check_exit_code=(0, 21)) or "" node_values = out.strip() node_values = node_values.split("\n") iqn = None startup = None startup_values = {} for node_value in node_values: node_keys = node_value.split() try: if node_keys[0] == "node.name": iqn = node_keys[2] elif node_keys[0] == "node.startup": startup = node_keys[2] if iqn and startup: startup_values[iqn] = startup iqn = None startup = None except IndexError: pass return startup_values def _recover_node_startup_values(self, connection_properties, old_node_startups): node_startups = self._get_node_startup_values(connection_properties) for iqn, node_startup in node_startups.items(): old_node_startup = old_node_startups.get(iqn, None) if old_node_startup and node_startup != old_node_startup: # _iscsiadm_update() only uses "target_portal" and "target_iqn" # of connection_properties. # And the recovering target belongs to the same target_portal # as discovering target. # So target_iqn is updated, and other values aren't updated. recover_connection = copy.deepcopy(connection_properties) recover_connection['target_iqn'] = iqn self._iscsiadm_update(recover_connection, "node.startup", old_node_startup) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/local.py0000664000175000017500000000554700000000000022707 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils class LocalConnector(base.BaseLinuxConnector): """"Connector class to attach/detach File System backed volumes.""" def __init__(self, root_helper, driver=None, *args, **kwargs): super(LocalConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The Local connector properties.""" return {} def get_volume_paths(self, connection_properties): path = connection_properties['device_path'] return [path] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): # TODO(walter-boring): not sure what to return here. return [] @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. ``connection_properties`` must include: - ``device_path`` - path to the volume to be connected :type connection_properties: dict :returns: dict """ if 'device_path' not in connection_properties: msg = (_("Invalid connection_properties specified " "no device_path attribute")) raise ValueError(msg) device_info = {'type': 'local', 'path': connection_properties['device_path']} return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ pass def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/nvmeof.py0000664000175000017500000003077300000000000023106 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json import re import time from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils DEVICE_SCAN_ATTEMPTS_DEFAULT = 5 LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-') class NVMeOFConnector(base.BaseLinuxConnector): """Connector class to attach/detach NVMe over fabric volumes.""" def __init__(self, root_helper, driver=None, use_multipath=False, device_scan_attempts=DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(NVMeOFConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs) self.use_multipath = use_multipath def _get_system_uuid(self): # RSD requires system_uuid to let Cinder RSD Driver identify # Nova node for later RSD volume attachment. try: out, err = self._execute('cat', '/sys/class/dmi/id/product_uuid', root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError: try: out, err = self._execute('dmidecode', '-ssystem-uuid', root_helper=self._root_helper, run_as_root=True) if not out: LOG.warning('dmidecode returned empty system-uuid') except putils.ProcessExecutionError as e: LOG.debug("Unable to locate dmidecode. For Cinder RSD Backend," " please make sure it is installed: %s", e) out = "" return out.strip() @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The NVMeOF connector properties.""" nvme = NVMeOFConnector(root_helper=root_helper, execute=kwargs.get('execute')) uuid = nvme._get_system_uuid() if uuid: return {"system uuid": uuid} else: return {} def get_search_path(self): return '/dev' def get_volume_paths(self, connection_properties): path = connection_properties['device_path'] LOG.debug("Path of volume to be extended is %(path)s", {'path': path}) return [path] def _get_nvme_devices(self): nvme_devices = [] # match nvme devices like /dev/nvme10n10 pattern = r'/dev/nvme[0-9]+n[0-9]+' cmd = ['nvme', 'list'] for retry in range(1, self.device_scan_attempts + 1): try: (out, err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) for line in out.split('\n'): result = re.match(pattern, line) if result: nvme_devices.append(result.group(0)) LOG.debug("_get_nvme_devices returned %(nvme_devices)s", {'nvme_devices': nvme_devices}) return nvme_devices except putils.ProcessExecutionError: LOG.warning( "Failed to list available NVMe connected controllers, " "retrying.") time.sleep(retry ** 2) else: msg = _("Failed to retrieve available connected NVMe controllers " "when running nvme list.") raise exception.CommandExecutionFailed(message=msg) @utils.retry(exceptions=exception.VolumePathsNotFound) def _get_device_path(self, current_nvme_devices): all_nvme_devices = self._get_nvme_devices() LOG.debug("all_nvme_devices are %(all_nvme_devices)s", {'all_nvme_devices': all_nvme_devices}) path = set(all_nvme_devices) - set(current_nvme_devices) if not path: raise exception.VolumePathsNotFound() return list(path) @utils.retry(exceptions=putils.ProcessExecutionError) def _try_connect_nvme(self, cmd): self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) def _get_nvme_subsys(self): # Example output: # { # 'Subsystems' : [ # { # 'Name' : 'nvme-subsys0', # 'NQN' : 'nqn.2016-06.io.spdk:cnode1' # }, # { # 'Paths' : [ # { # 'Name' : 'nvme0', # 'Transport' : 'rdma', # 'Address' : 'traddr=10.0.2.15 trsvcid=4420' # } # ] # } # ] # } # cmd = ['nvme', 'list-subsys', '-o', 'json'] ret_val = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) return ret_val @utils.retry(exceptions=exception.NotFound, retries=5) def _is_nvme_available(self, nvme_name): nvme_name_pattern = "/dev/%sn[0-9]+" % nvme_name for nvme_dev_name in self._get_nvme_devices(): if re.match(nvme_name_pattern, nvme_dev_name): return True else: LOG.error("Failed to find nvme device") raise exception.NotFound() def _wait_for_blk(self, nvme_transport_type, conn_nqn, target_portal, port): # Find nvme name in subsystem list and wait max 15 seconds # until new volume will be available in kernel nvme_name = "" nvme_address = "traddr=%s trsvcid=%s" % (target_portal, port) # Get nvme subsystems in order to find # nvme name for connected nvme try: (out, err) = self._get_nvme_subsys() except putils.ProcessExecutionError: LOG.error("Failed to get nvme subsystems") raise # Get subsystem list. Throw exception if out is currupt or empty try: subsystems = json.loads(out)['Subsystems'] except Exception: return False # Find nvme name among subsystems for i in range(0, int(len(subsystems) / 2)): subsystem = subsystems[i * 2] if 'NQN' in subsystem and subsystem['NQN'] == conn_nqn: for path in subsystems[i * 2 + 1]['Paths']: if (path['Transport'] == nvme_transport_type and path['Address'] == nvme_address): nvme_name = path['Name'] break if not nvme_name: return False # Wait until nvme will be available in kernel return self._is_nvme_available(nvme_name) def _try_disconnect_volume(self, conn_nqn, ignore_errors=False): cmd = [ 'nvme', 'disconnect', '-n', conn_nqn] try: self._execute( *cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError: LOG.error( "Failed to disconnect from NVMe nqn " "%(conn_nqn)s", {'conn_nqn': conn_nqn}) if not ignore_errors: raise @utils.trace @synchronized('connect_volume', external=True) def connect_volume(self, connection_properties): """Discover and attach the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: nqn - NVMe subsystem name to the volume to be connected target_port - NVMe target port that hosts the nqn sybsystem target_portal - NVMe target ip that hosts the nqn sybsystem :type connection_properties: dict :returns: dict """ current_nvme_devices = self._get_nvme_devices() device_info = {'type': 'block'} conn_nqn = connection_properties['nqn'] target_portal = connection_properties['target_portal'] port = connection_properties['target_port'] nvme_transport_type = connection_properties['transport_type'] host_nqn = connection_properties.get('host_nqn') cmd = [ 'nvme', 'connect', '-t', nvme_transport_type, '-n', conn_nqn, '-a', target_portal, '-s', port] if host_nqn: cmd.extend(['-q', host_nqn]) self._try_connect_nvme(cmd) try: self._wait_for_blk(nvme_transport_type, conn_nqn, target_portal, port) except exception.NotFound: LOG.error("Waiting for nvme failed") self._try_disconnect_volume(conn_nqn, True) raise exception.NotFound(message="nvme connect: NVMe device " "not found") path = self._get_device_path(current_nvme_devices) device_info['path'] = path[0] LOG.debug("NVMe device to be connected to is %(path)s", {'path': path[0]}) return device_info @utils.trace @synchronized('disconnect_volume', external=True) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Detach and flush the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: device_path - path to the volume to be connected :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ conn_nqn = connection_properties['nqn'] if device_info and device_info.get('path'): device_path = device_info.get('path') else: device_path = connection_properties['device_path'] or '' current_nvme_devices = self._get_nvme_devices() if device_path not in current_nvme_devices: LOG.warning("Trying to disconnect device %(device_path)s with " "subnqn %(conn_nqn)s that is not connected.", {'device_path': device_path, 'conn_nqn': conn_nqn}) return LOG.debug( "Trying to disconnect from device %(device_path)s with " "subnqn %(conn_nqn)s", {'device_path': device_path, 'conn_nqn': conn_nqn}) cmd = [ 'nvme', 'disconnect', '-n', conn_nqn] try: self._execute( *cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError: LOG.error( "Failed to disconnect from NVMe nqn " "%(conn_nqn)s with device_path %(device_path)s", {'conn_nqn': conn_nqn, 'device_path': device_path}) if not ignore_errors: raise @utils.trace @synchronized('extend_volume', external=True) def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for an LVM volume. """ volume_paths = self.get_volume_paths(connection_properties) if volume_paths: return self._linuxscsi.extend_volume( volume_paths, use_multipath=self.use_multipath) else: LOG.warning("Couldn't find any volume paths on the host to " "extend volume for %(props)s", {'props': connection_properties}) raise exception.VolumePathsNotFound() ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/rbd.py0000664000175000017500000003144500000000000022360 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_serialization import jsonutils from oslo_utils import fileutils from oslo_utils import netutils from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.initiator import linuxrbd from os_brick import utils LOG = logging.getLogger(__name__) class RBDConnector(base.BaseLinuxConnector): """"Connector class to attach/detach RBD volumes.""" def __init__(self, root_helper, driver=None, use_multipath=False, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(RBDConnector, self).__init__(root_helper, driver=driver, device_scan_attempts= device_scan_attempts, *args, **kwargs) self.do_local_attach = kwargs.get('do_local_attach', False) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The RBD connector properties.""" return {'do_local_attach': kwargs.get('do_local_attach', False)} def get_volume_paths(self, connection_properties): # TODO(e0ne): Implement this for local volume. return [] def get_search_path(self): # TODO(walter-boring): don't know where the connector # looks for RBD volumes. return None def get_all_available_volumes(self, connection_properties=None): # TODO(e0ne): Implement this for local volume. return [] def _sanitize_mon_hosts(self, hosts): def _sanitize_host(host): if netutils.is_valid_ipv6(host): host = '[%s]' % host return host return list(map(_sanitize_host, hosts)) def _check_or_get_keyring_contents(self, keyring, cluster_name, user): try: if keyring is None: if user: keyring_path = ("/etc/ceph/%s.client.%s.keyring" % (cluster_name, user)) with open(keyring_path, 'r') as keyring_file: keyring = keyring_file.read() else: keyring = '' return keyring except IOError: msg = (_("Keyring path %s is not readable.") % (keyring_path)) raise exception.BrickException(msg=msg) def _create_ceph_conf(self, monitor_ips, monitor_ports, cluster_name, user, keyring): monitors = ["%s:%s" % (ip, port) for ip, port in zip(self._sanitize_mon_hosts(monitor_ips), monitor_ports)] mon_hosts = "mon_host = %s" % (','.join(monitors)) keyring = self._check_or_get_keyring_contents(keyring, cluster_name, user) try: fd, ceph_conf_path = tempfile.mkstemp(prefix="brickrbd_") with os.fdopen(fd, 'w') as conf_file: # Bug #1865754 - '[global]' has been the appropriate # place for this stuff since at least Hammer, but in # Octopus (15.2.0+), Ceph began enforcing this. conf_file.writelines(["[global]", "\n", mon_hosts, "\n", keyring, "\n"]) return ceph_conf_path except IOError: msg = (_("Failed to write data to %s.") % (ceph_conf_path)) raise exception.BrickException(msg=msg) def _get_rbd_handle(self, connection_properties): try: user = connection_properties['auth_username'] pool, volume = connection_properties['name'].split('/') cluster_name = connection_properties['cluster_name'] monitor_ips = connection_properties['hosts'] monitor_ports = connection_properties['ports'] keyring = connection_properties.get('keyring') except (KeyError, ValueError): msg = _("Connect volume failed, malformed connection properties.") raise exception.BrickException(msg=msg) conf = self._create_ceph_conf(monitor_ips, monitor_ports, str(cluster_name), user, keyring) try: rbd_client = linuxrbd.RBDClient(user, pool, conffile=conf, rbd_cluster_name=str(cluster_name)) rbd_volume = linuxrbd.RBDVolume(rbd_client, volume) rbd_handle = linuxrbd.RBDVolumeIOWrapper( linuxrbd.RBDImageMetadata(rbd_volume, pool, user, conf)) except Exception: fileutils.delete_if_exists(conf) raise return rbd_handle def _get_rbd_args(self, connection_properties): try: user = connection_properties['auth_username'] monitor_ips = connection_properties.get('hosts') monitor_ports = connection_properties.get('ports') except KeyError: msg = _("Connect volume failed, malformed connection properties") raise exception.BrickException(msg=msg) args = ['--id', user] if monitor_ips and monitor_ports: monitors = ["%s:%s" % (ip, port) for ip, port in zip( self._sanitize_mon_hosts(monitor_ips), monitor_ports)] for monitor in monitors: args += ['--mon_host', monitor] return args @staticmethod def get_rbd_device_name(pool, volume): """Return device name which will be generated by RBD kernel module. :param pool: RBD pool name. :type pool: string :param volume: RBD image name. :type volume: string """ return '/dev/rbd/{pool}/{volume}'.format(pool=pool, volume=volume) @utils.trace def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ do_local_attach = connection_properties.get('do_local_attach', self.do_local_attach) if do_local_attach: # NOTE(e0ne): sanity check if ceph-common is installed. cmd = ['which', 'rbd'] try: self._execute(*cmd) except putils.ProcessExecutionError: msg = _("ceph-common package is not installed.") LOG.error(msg) raise exception.BrickException(message=msg) # NOTE(e0ne): map volume to a block device # via the rbd kernel module. pool, volume = connection_properties['name'].split('/') rbd_dev_path = RBDConnector.get_rbd_device_name(pool, volume) if ( not os.path.islink(rbd_dev_path) or not os.path.exists(os.path.realpath(rbd_dev_path)) ): cmd = ['rbd', 'map', volume, '--pool', pool] cmd += self._get_rbd_args(connection_properties) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) else: LOG.debug( 'Volume %(vol)s is already mapped to local device %(dev)s', {'vol': volume, 'dev': os.path.realpath(rbd_dev_path)} ) if ( not os.path.islink(rbd_dev_path) or not os.path.exists(os.path.realpath(rbd_dev_path)) ): LOG.warning( 'Volume %(vol)s has not been mapped to local device ' '%(dev)s; is the udev daemon running and are the ' 'ceph-renamer udev rules configured? See bug #1884114 for ' 'more information.', {'vol': volume, 'dev': rbd_dev_path}, ) return {'path': rbd_dev_path, 'type': 'block'} rbd_handle = self._get_rbd_handle(connection_properties) return {'path': rbd_handle} def _find_root_device(self, connection_properties): """Find the underlying /dev/rbd* device for a mapping. Use the showmapped command to list all acive mappings and find the underlying /dev/rbd* device that corresponds to our pool and volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: '/dev/rbd*' or None if no active mapping is found. """ __, volume = connection_properties['name'].split('/') cmd = ['rbd', 'showmapped', '--format=json'] cmd += self._get_rbd_args(connection_properties) (out, err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) # ceph v13.2.0 (Mimic) changed the output format of 'rbd showmapped' # from a dict of mappings keyed by ID to a simple list of mappings # https://docs.ceph.com/docs/master/releases/mimic/ # # before: # # { # "0": { # "pool":"volumes", # "namespace":"", # "name":"volume-6d54cb90-a5d1-40d8-9cb2-c6adf43a02af", # "snap":"-", # "device":"/dev/rbd0" # } # } # # after: # # [ # { # "id":"0", # "pool":"volumes", # "namespace":"", # "name":"volume-6d54cb90-a5d1-40d8-9cb2-c6adf43a02af", # "snap":"-", # "device":"/dev/rbd0" # } # ] # # TODO(stephenfin): Drop when we drop support for ceph 13.2.0 mappings = jsonutils.loads(out) if isinstance(mappings, dict): # yes, we're losing the ID field but we don't need it here mappings = mappings.values() for mapping in mappings: if mapping['name'] == volume: return mapping['device'] return None @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ do_local_attach = connection_properties.get('do_local_attach', self.do_local_attach) if do_local_attach: root_device = self._find_root_device(connection_properties) if root_device: cmd = ['rbd', 'unmap', root_device] cmd += self._get_rbd_args(connection_properties) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) else: if device_info: rbd_handle = device_info.get('path', None) if rbd_handle is not None: fileutils.delete_if_exists(rbd_handle.rbd_conf) rbd_handle.close() def check_valid_device(self, path, run_as_root=True): """Verify an existing RBD handle is connected and valid.""" rbd_handle = path if rbd_handle is None: return False original_offset = rbd_handle.tell() try: rbd_handle.read(4096) except Exception as e: LOG.error("Failed to access RBD device handle: %(error)s", {"error": e}) return False finally: rbd_handle.seek(original_offset, 0) return True def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/remotefs.py0000664000175000017500000001120000000000000023420 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_log import log as logging from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.remotefs import remotefs from os_brick import utils LOG = logging.getLogger(__name__) class RemoteFsConnector(base.BaseLinuxConnector): """Connector class to attach/detach NFS and GlusterFS volumes.""" def __init__(self, mount_type, root_helper, driver=None, execute=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): kwargs = kwargs or {} conn = kwargs.get('conn') mount_type_lower = mount_type.lower() if conn: mount_point_base = conn.get('mount_point_base') if mount_type_lower in ('nfs', 'glusterfs', 'scality', 'quobyte', 'vzstorage'): kwargs[mount_type_lower + '_mount_point_base'] = ( kwargs.get(mount_type_lower + '_mount_point_base') or mount_point_base) else: LOG.warning("Connection details not present." " RemoteFsClient may not initialize properly.") if mount_type_lower == 'scality': cls = remotefs.ScalityRemoteFsClient elif mount_type_lower == 'vzstorage': cls = remotefs.VZStorageRemoteFSClient else: cls = remotefs.RemoteFsClient self._remotefsclient = cls(mount_type, root_helper, execute=execute, *args, **kwargs) super(RemoteFsConnector, self).__init__( root_helper, driver=driver, execute=execute, device_scan_attempts=device_scan_attempts, *args, **kwargs) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The RemoteFS connector properties.""" return {} def set_execute(self, execute): super(RemoteFsConnector, self).set_execute(execute) self._remotefsclient.set_execute(execute) def get_search_path(self): return self._remotefsclient.get_mount_base() def _get_volume_path(self, connection_properties): mnt_flags = [] if connection_properties.get('options'): mnt_flags = connection_properties['options'].split() nfs_share = connection_properties['export'] self._remotefsclient.mount(nfs_share, mnt_flags) mount_point = self._remotefsclient.get_mount_point(nfs_share) path = mount_point + '/' + connection_properties['name'] return path def get_volume_paths(self, connection_properties): path = self._get_volume_path(connection_properties) return [path] @utils.trace def connect_volume(self, connection_properties): """Ensure that the filesystem containing the volume is mounted. :param connection_properties: The dictionary that describes all of the target volume attributes. connection_properties must include: export - remote filesystem device (e.g. '172.18.194.100:/var/nfs') name - file name within the filesystem :type connection_properties: dict :returns: dict connection_properties may optionally include: options - options to pass to mount """ path = self._get_volume_path(connection_properties) return {'path': path} @utils.trace def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """No need to do anything to disconnect a volume in a filesystem. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict """ def extend_volume(self, connection_properties): # TODO(walter-boring): is this possible? raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/scaleio.py0000664000175000017500000004754600000000000023241 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json import os import requests import six from six.moves import urllib from oslo_concurrency import lockutils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator.connectors import base from os_brick.privileged import scaleio as priv_scaleio from os_brick import utils LOG = logging.getLogger(__name__) DEVICE_SCAN_ATTEMPTS_DEFAULT = 3 CONNECTOR_CONF_PATH = '/opt/emc/scaleio/openstack/connector.conf' synchronized = lockutils.synchronized_with_prefix('os-brick-') def io(_type, nr): """Implementation of _IO macro from .""" return ioc(0x0, _type, nr, 0) def ioc(direction, _type, nr, size): """Implementation of _IOC macro from .""" return direction | (size & 0x1fff) << 16 | ord(_type) << 8 | nr class ScaleIOConnector(base.BaseLinuxConnector): """Class implements the connector driver for ScaleIO.""" OK_STATUS_CODE = 200 VOLUME_NOT_MAPPED_ERROR = 84 VOLUME_ALREADY_MAPPED_ERROR = 81 GET_GUID_OP_CODE = io('a', 14) RESCAN_VOLS_OP_CODE = io('a', 10) def __init__(self, root_helper, driver=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(ScaleIOConnector, self).__init__( root_helper, driver=driver, device_scan_attempts=device_scan_attempts, *args, **kwargs ) self.local_sdc_ip = None self.server_ip = None self.server_port = None self.server_username = None self.server_password = None self.server_token = None self.volume_id = None self.volume_name = None self.volume_path = None self.iops_limit = None self.bandwidth_limit = None def _get_guid(self): try: guid = priv_scaleio.get_guid(self.GET_GUID_OP_CODE) LOG.info("Current sdc guid: %s", guid) return guid except (IOError, OSError, ValueError) as e: msg = _("Error querying sdc guid: %s") % e LOG.error(msg) raise exception.BrickException(message=msg) @staticmethod def _get_password_token(connection_properties): # In old connection format we had the password and token in properties if 'serverPassword' in connection_properties: return (connection_properties['serverPassword'], connection_properties['serverToken']) # The new format reads password from file and doesn't have the token LOG.info("Get ScaleIO connector password from configuration file") try: password = priv_scaleio.get_connector_password( CONNECTOR_CONF_PATH, connection_properties['config_group'], connection_properties.get('failed_over', False)) return password, None except Exception as e: msg = _("Error getting ScaleIO connector password from " "configuration file: %s") % e LOG.error(msg) raise exception.BrickException(message=msg) def _rescan_vols(self): LOG.info("ScaleIO rescan volumes") try: priv_scaleio.rescan_vols(self.RESCAN_VOLS_OP_CODE) except (IOError, OSError) as e: msg = _("Error querying volumes: %s") % e LOG.error(msg) raise exception.BrickException(message=msg) @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The ScaleIO connector properties.""" return {} def get_search_path(self): return "/dev/disk/by-id" def get_volume_paths(self, connection_properties): self.get_config(connection_properties) volume_paths = [] device_paths = [self._find_volume_path()] for path in device_paths: if os.path.exists(path): volume_paths.append(path) return volume_paths def _find_volume_path(self): LOG.info( "Looking for volume %(volume_id)s, maximum tries: %(tries)s", {'volume_id': self.volume_id, 'tries': self.device_scan_attempts} ) # look for the volume in /dev/disk/by-id directory by_id_path = self.get_search_path() disk_filename = self._wait_for_volume_path(by_id_path) full_disk_name = ("%(path)s/%(filename)s" % {'path': by_id_path, 'filename': disk_filename}) LOG.info("Full disk name is %(full_path)s", {'full_path': full_disk_name}) return full_disk_name # NOTE: Usually 3 retries is enough to find the volume. # If there are network issues, it could take much longer. Set # the max retries to 15 to make sure we can find the volume. @utils.retry(exceptions=exception.BrickException, retries=15, backoff_rate=1) def _wait_for_volume_path(self, path): if not os.path.isdir(path): msg = ( _("ScaleIO volume %(volume_id)s not found at " "expected path.") % {'volume_id': self.volume_id} ) LOG.debug(msg) raise exception.BrickException(message=msg) disk_filename = None filenames = os.listdir(path) LOG.info( "Files found in %(path)s path: %(files)s ", {'path': path, 'files': filenames} ) for filename in filenames: if (filename.startswith("emc-vol") and filename.endswith(self.volume_id)): disk_filename = filename break if not disk_filename: msg = (_("ScaleIO volume %(volume_id)s not found.") % {'volume_id': self.volume_id}) LOG.debug(msg) raise exception.BrickException(message=msg) return disk_filename def _get_client_id(self): request = ( "https://%(server_ip)s:%(server_port)s/" "api/types/Client/instances/getByIp::%(sdc_ip)s/" % { 'server_ip': self.server_ip, 'server_port': self.server_port, 'sdc_ip': self.local_sdc_ip } ) LOG.info("ScaleIO get client id by ip request: %(request)s", {'request': request}) r = requests.get( request, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request) sdc_id = r.json() if not sdc_id: msg = (_("Client with ip %(sdc_ip)s was not found.") % {'sdc_ip': self.local_sdc_ip}) raise exception.BrickException(message=msg) if r.status_code != 200 and "errorCode" in sdc_id: msg = (_("Error getting sdc id from ip %(sdc_ip)s: %(err)s") % {'sdc_ip': self.local_sdc_ip, 'err': sdc_id['message']}) LOG.error(msg) raise exception.BrickException(message=msg) LOG.info("ScaleIO sdc id is %(sdc_id)s.", {'sdc_id': sdc_id}) return sdc_id def _get_volume_id(self): volname_encoded = urllib.parse.quote(self.volume_name, '') volname_double_encoded = urllib.parse.quote(volname_encoded, '') LOG.debug(_( "Volume name after double encoding is %(volume_name)s."), {'volume_name': volname_double_encoded} ) request = ( "https://%(server_ip)s:%(server_port)s/api/types/Volume/instances" "/getByName::%(encoded_volume_name)s" % { 'server_ip': self.server_ip, 'server_port': self.server_port, 'encoded_volume_name': volname_double_encoded } ) LOG.info( "ScaleIO get volume id by name request: %(request)s", {'request': request} ) r = requests.get(request, auth=(self.server_username, self.server_token), verify=False) r = self._check_response(r, request) volume_id = r.json() if not volume_id: msg = (_("Volume with name %(volume_name)s wasn't found.") % {'volume_name': self.volume_name}) LOG.error(msg) raise exception.BrickException(message=msg) if r.status_code != self.OK_STATUS_CODE and "errorCode" in volume_id: msg = ( _("Error getting volume id from name %(volume_name)s: " "%(err)s") % {'volume_name': self.volume_name, 'err': volume_id['message']} ) LOG.error(msg) raise exception.BrickException(message=msg) LOG.info("ScaleIO volume id is %(volume_id)s.", {'volume_id': volume_id}) return volume_id def _check_response(self, response, request, is_get_request=True, params=None): if response.status_code == 401 or response.status_code == 403: LOG.info("Token is invalid, " "going to re-login to get a new one") login_request = ( "https://%(server_ip)s:%(server_port)s/api/login" % {'server_ip': self.server_ip, 'server_port': self.server_port} ) r = requests.get( login_request, auth=(self.server_username, self.server_password), verify=False ) token = r.json() # repeat request with valid token LOG.debug(_("Going to perform request %(request)s again " "with valid token"), {'request': request}) if is_get_request: res = requests.get(request, auth=(self.server_username, token), verify=False) else: headers = {'content-type': 'application/json'} res = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, token), verify=False ) self.server_token = token return res return response def get_config(self, connection_properties): self.local_sdc_ip = connection_properties['hostIP'] self.volume_name = connection_properties['scaleIO_volname'] # instances which were created before Newton release don't have # 'scaleIO_volume_id' property, in such cases connector will resolve # volume_id from volname self.volume_id = connection_properties.get('scaleIO_volume_id') self.server_ip = connection_properties['serverIP'] self.server_port = connection_properties['serverPort'] self.server_username = connection_properties['serverUsername'] self.server_password, self.server_token = self._get_password_token( connection_properties) self.iops_limit = connection_properties['iopsLimit'] self.bandwidth_limit = connection_properties['bandwidthLimit'] device_info = {'type': 'block', 'path': self.volume_path} return device_info @utils.trace @lockutils.synchronized('scaleio', 'scaleio-', external=True) def connect_volume(self, connection_properties): """Connect the volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ device_info = self.get_config(connection_properties) LOG.debug( _( "scaleIO Volume name: %(volume_name)s, SDC IP: %(sdc_ip)s, " "REST Server IP: %(server_ip)s, " "REST Server username: %(username)s, " "iops limit: %(iops_limit)s, " "bandwidth limit: %(bandwidth_limit)s." ), { 'volume_name': self.volume_name, 'volume_id': self.volume_id, 'sdc_ip': self.local_sdc_ip, 'server_ip': self.server_ip, 'username': self.server_username, 'iops_limit': self.iops_limit, 'bandwidth_limit': self.bandwidth_limit } ) guid = self._get_guid() params = {'guid': guid, 'allowMultipleMappings': 'TRUE'} self.volume_id = self.volume_id or self._get_volume_id() headers = {'content-type': 'application/json'} request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/addMappedSdc" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("map volume request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() error_code = response['errorCode'] if error_code == self.VOLUME_ALREADY_MAPPED_ERROR: LOG.warning( "Ignoring error mapping volume %(volume_name)s: " "volume already mapped.", {'volume_name': self.volume_name} ) else: msg = ( _("Error mapping volume %(volume_name)s: %(err)s") % {'volume_name': self.volume_name, 'err': response['message']} ) LOG.error(msg) raise exception.BrickException(message=msg) self.volume_path = self._find_volume_path() device_info['path'] = self.volume_path # Set QoS settings after map was performed if self.iops_limit is not None or self.bandwidth_limit is not None: params = {'guid': guid} if self.bandwidth_limit is not None: params['bandwidthLimitInKbps'] = self.bandwidth_limit if self.iops_limit is not None: params['iopsLimit'] = self.iops_limit request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/setMappedSdcLimits" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("Set client limit request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() LOG.info("Set client limit response: %(response)s", {'response': response}) msg = ( _("Error setting client limits for volume " "%(volume_name)s: %(err)s") % {'volume_name': self.volume_name, 'err': response['message']} ) LOG.error(msg) return device_info @utils.trace @lockutils.synchronized('scaleio', 'scaleio-', external=True) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect the ScaleIO volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. """ self.get_config(connection_properties) self.volume_id = self.volume_id or self._get_volume_id() LOG.info( "ScaleIO disconnect volume in ScaleIO brick volume driver." ) LOG.debug( _("ScaleIO Volume name: %(volume_name)s, SDC IP: %(sdc_ip)s, " "REST Server IP: %(server_ip)s"), {'volume_name': self.volume_name, 'sdc_ip': self.local_sdc_ip, 'server_ip': self.server_ip} ) guid = self._get_guid() params = {'guid': guid} headers = {'content-type': 'application/json'} request = ( "https://%(server_ip)s:%(server_port)s/api/instances/" "Volume::%(volume_id)s/action/removeMappedSdc" % {'server_ip': self.server_ip, 'server_port': self.server_port, 'volume_id': self.volume_id} ) LOG.info("Unmap volume request: %(request)s", {'request': request}) r = requests.post( request, data=json.dumps(params), headers=headers, auth=(self.server_username, self.server_token), verify=False ) r = self._check_response(r, request, False, params) if r.status_code != self.OK_STATUS_CODE: response = r.json() error_code = response['errorCode'] if error_code == self.VOLUME_NOT_MAPPED_ERROR: LOG.warning( "Ignoring error unmapping volume %(volume_id)s: " "volume not mapped.", {'volume_id': self.volume_name} ) else: msg = (_("Error unmapping volume %(volume_id)s: %(err)s") % {'volume_id': self.volume_name, 'err': response['message']}) LOG.error(msg) raise exception.BrickException(message=msg) def extend_volume(self, connection_properties): """Update the local kernel's size information. Try and update the local kernel's size information for a ScaleIO volume. """ self._rescan_vols() volume_paths = self.get_volume_paths(connection_properties) if volume_paths: return self.get_device_size(volume_paths[0]) # if we got here, the volume is not mapped msg = (_("Error extending ScaleIO volume")) LOG.error(msg) raise exception.BrickException(message=msg) def get_device_size(self, device): """Get the size in bytes of a volume.""" (out, _err) = self._execute('blockdev', '--getsize64', device, run_as_root=True, root_helper=self._root_helper) var = six.text_type(out.strip()) LOG.debug("Device %(dev)s size: %(var)s", {'dev': device, 'var': var}) if var.isnumeric(): return int(var) else: return None ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/storpool.py0000664000175000017500000002377300000000000023477 0ustar00zuulzuul00000000000000# Copyright (c) 2015 - 2017 StorPool # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from __future__ import absolute_import import os import time import six from oslo_log import log as logging from oslo_utils import importutils from os_brick import exception from os_brick.initiator.connectors import base LOG = logging.getLogger(__name__) spopenstack = importutils.try_import('storpool.spopenstack') class StorPoolConnector(base.BaseLinuxConnector): """"Connector class to attach/detach StorPool volumes.""" def __init__(self, root_helper, driver=None, *args, **kwargs): super(StorPoolConnector, self).__init__(root_helper, driver=driver, *args, **kwargs) if spopenstack is not None: try: self._attach = spopenstack.AttachDB(log=LOG) except Exception as e: raise exception.BrickException( 'Could not initialize the StorPool API bindings: %s' % (e)) else: self._attach = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The StorPool connector properties.""" return {} def connect_volume(self, connection_properties): """Connect to a volume. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain the StorPool 'client_id' and the common 'volume' and 'access_mode' values. :type connection_properties: dict :returns: dict """ client_id = connection_properties.get('client_id', None) if client_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no client ID specified.') volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) mode = connection_properties.get('access_mode', None) if mode is None or mode not in ('rw', 'ro'): raise exception.BrickException( 'Invalid access_mode specified in the connection data.') req_id = 'brick-%s-%s' % (client_id, volume_id) self._attach.add(req_id, { 'volume': volume, 'type': 'brick', 'id': req_id, 'rights': 1 if mode == 'ro' else 2, 'volsnap': False }) self._attach.sync(req_id, None) return {'type': 'block', 'path': '/dev/storpool/' + volume} def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. The connection_properties are the same as from connect_volume. The device_info is returned from connect_volume. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain the StorPool 'client_id' and the common 'volume' values. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. For StorPool, this parameter is ignored, the volume is always detached. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. For StorPool, this parameter is ignored, no exception is raised except on unexpected errors. :type ignore_errors: bool """ client_id = connection_properties.get('client_id', None) if client_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no client ID specified.') volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) req_id = 'brick-%s-%s' % (client_id, volume_id) self._attach.sync(req_id, volume) self._attach.remove(req_id) def get_search_path(self): return '/dev/storpool' def get_volume_paths(self, connection_properties): """Return the list of existing paths for a volume. The job of this method is to find out what paths in the system are associated with a volume as described by the connection_properties. :param connection_properties: The dictionary that describes all of the target volume attributes; it needs to contain 'volume' and 'device_path' values. :type connection_properties: dict """ volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') volume = self._attach.volumeName(volume_id) path = '/dev/storpool/' + volume dpath = connection_properties.get('device_path', None) if dpath is not None and dpath != path: raise exception.BrickException( 'Internal error: StorPool volume path {path} does not ' 'match device path {dpath}', {path: path, dpath: dpath}) return [path] def get_all_available_volumes(self, connection_properties=None): """Return all volumes that exist in the search directory. At connect_volume time, a Connector looks in a specific directory to discover a volume's paths showing up. This method's job is to return all paths in the directory that connect_volume uses to find a volume. This method is used in coordination with get_volume_paths() to verify that volumes have gone away after disconnect_volume has been called. :param connection_properties: The dictionary that describes all of the target volume attributes. Unused for the StorPool connector. :type connection_properties: dict """ names = [] prefix = self._attach.volumeName('') prefixlen = len(prefix) if os.path.isdir('/dev/storpool'): files = os.listdir('/dev/storpool') for entry in files: full = '/dev/storpool/' + entry if entry.startswith(prefix) and os.path.islink(full) and \ not os.path.isdir(full): names.append(entry[prefixlen:]) return names def _get_device_size(self, device): """Get the size in bytes of a volume.""" (out, _err) = self._execute('blockdev', '--getsize64', device, run_as_root=True, root_helper=self._root_helper) var = six.text_type(out).strip() if var.isnumeric(): return int(var) else: return None def extend_volume(self, connection_properties): """Update the attached volume's size. This method will attempt to update the local hosts's volume after the volume has been extended on the remote system. The new volume size in bytes will be returned. If there is a failure to update, then None will be returned. :param connection_properties: The volume connection properties. :returns: new size of the volume. """ # The StorPool client (storpool_block service) running on this host # should have picked up the change already, so it is enough to query # the actual disk device to see if its size is correct. # volume_id = connection_properties.get('volume', None) if volume_id is None: raise exception.BrickException( 'Invalid StorPool connection data, no volume ID specified.') # Get the expected (new) size from the StorPool API volume = self._attach.volumeName(volume_id) LOG.debug('Querying the StorPool API for the size of %(vol)s', {'vol': volume}) vdata = self._attach.api().volumeList(volume)[0] LOG.debug('Got size %(size)d', {'size': vdata.size}) # Wait for the StorPool client to update the size of the local device path = '/dev/storpool/' + volume for _ in range(10): size = self._get_device_size(path) LOG.debug('Got local size %(size)d', {'size': size}) if size == vdata.size: return size time.sleep(0.1) else: size = self._get_device_size(path) LOG.debug('Last attempt: local size %(size)d', {'size': size}) return size ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/vmware.py0000664000175000017500000003430600000000000023111 0ustar00zuulzuul00000000000000# Copyright (c) 2016 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import tempfile from oslo_log import log as logging from oslo_utils import fileutils try: from oslo_vmware import api from oslo_vmware import exceptions as oslo_vmw_exceptions from oslo_vmware import image_transfer from oslo_vmware.objects import datastore from oslo_vmware import rw_handles from oslo_vmware import vim_util except ImportError: vim_util = None import six from os_brick import exception from os_brick.i18n import _ from os_brick.initiator import initiator_connector LOG = logging.getLogger(__name__) class VmdkConnector(initiator_connector.InitiatorConnector): """Connector for volumes created by the VMDK driver. This connector is only used for backup and restore of Cinder volumes. """ TMP_IMAGES_DATASTORE_FOLDER_PATH = "cinder_temp" def __init__(self, *args, **kwargs): # Check if oslo.vmware library is available. if vim_util is None: message = _("Missing oslo_vmware python module, ensure oslo.vmware" " library is installed and available.") raise exception.BrickException(message=message) super(VmdkConnector, self).__init__(*args, **kwargs) self._ip = None self._port = None self._username = None self._password = None self._api_retry_count = None self._task_poll_interval = None self._ca_file = None self._insecure = None self._tmp_dir = None self._timeout = None @staticmethod def get_connector_properties(root_helper, *args, **kwargs): return {} def check_valid_device(self, path, *args, **kwargs): try: with open(path, 'r') as dev: dev.read(1) except IOError: LOG.exception( "Failed to access the device on the path " "%(path)s", {"path": path}) return False return True def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): pass def _load_config(self, connection_properties): config = connection_properties['config'] self._ip = config['vmware_host_ip'] self._port = config['vmware_host_port'] self._username = config['vmware_host_username'] self._password = config['vmware_host_password'] self._api_retry_count = config['vmware_api_retry_count'] self._task_poll_interval = config['vmware_task_poll_interval'] self._ca_file = config['vmware_ca_file'] self._insecure = config['vmware_insecure'] self._tmp_dir = config['vmware_tmp_dir'] self._timeout = config['vmware_image_transfer_timeout_secs'] def _create_session(self): return api.VMwareAPISession(self._ip, self._username, self._password, self._api_retry_count, self._task_poll_interval, port=self._port, cacert=self._ca_file, insecure=self._insecure) def _create_temp_file(self, *args, **kwargs): fileutils.ensure_tree(self._tmp_dir) fd, tmp = tempfile.mkstemp(dir=self._tmp_dir, *args, **kwargs) os.close(fd) return tmp def _download_vmdk( self, tmp_file_path, session, backing, vmdk_path, vmdk_size): with open(tmp_file_path, "wb") as tmp_file: image_transfer.copy_stream_optimized_disk( None, self._timeout, tmp_file, session=session, host=self._ip, port=self._port, vm=backing, vmdk_file_path=vmdk_path, vmdk_size=vmdk_size) def connect_volume(self, connection_properties): # Download the volume vmdk from vCenter server to a temporary file # and return its path. self._load_config(connection_properties) session = self._create_session() tmp_file_path = self._create_temp_file( suffix=".vmdk", prefix=connection_properties['volume_id']) backing = vim_util.get_moref(connection_properties['volume'], "VirtualMachine") vmdk_path = connection_properties['vmdk_path'] vmdk_size = connection_properties['vmdk_size'] try: self._download_vmdk( tmp_file_path, session, backing, vmdk_path, vmdk_size) finally: session.logout() # Save the last modified time of the temporary so that we can decide # whether to upload the file back to vCenter server during disconnect. last_modified = os.path.getmtime(tmp_file_path) return {'path': tmp_file_path, 'last_modified': last_modified} def _snapshot_exists(self, session, backing): snapshot = session.invoke_api(vim_util, 'get_object_property', session.vim, backing, 'snapshot') if snapshot is None or snapshot.rootSnapshotList is None: return False return len(snapshot.rootSnapshotList) != 0 def _create_temp_ds_folder(self, session, ds_folder_path, dc_ref): fileManager = session.vim.service_content.fileManager try: session.invoke_api(session.vim, 'MakeDirectory', fileManager, name=ds_folder_path, datacenter=dc_ref) except oslo_vmw_exceptions.FileAlreadyExistsException: pass # Note(vbala) remove this method when we implement it in oslo.vmware def _upload_vmdk( self, read_handle, host, port, dc_name, ds_name, cookies, upload_file_path, file_size, cacerts, timeout_secs): write_handle = rw_handles.FileWriteHandle(host, port, dc_name, ds_name, cookies, upload_file_path, file_size, cacerts=cacerts) image_transfer._start_transfer(read_handle, write_handle, timeout_secs) def _get_disk_device(self, session, backing): hardware_devices = session.invoke_api(vim_util, 'get_object_property', session.vim, backing, 'config.hardware.device') if hardware_devices.__class__.__name__ == "ArrayOfVirtualDevice": hardware_devices = hardware_devices.VirtualDevice for device in hardware_devices: if device.__class__.__name__ == "VirtualDisk": return device def _create_spec_for_disk_remove(self, session, disk_device): cf = session.vim.client.factory disk_spec = cf.create('ns0:VirtualDeviceConfigSpec') disk_spec.operation = 'remove' disk_spec.fileOperation = 'destroy' disk_spec.device = disk_device return disk_spec def _reconfigure_backing(self, session, backing, reconfig_spec): LOG.debug("Reconfiguring backing VM: %(backing)s with spec: %(spec)s.", {'backing': backing, 'spec': reconfig_spec}) reconfig_task = session.invoke_api(session.vim, "ReconfigVM_Task", backing, spec=reconfig_spec) LOG.debug("Task: %s created for reconfiguring backing VM.", reconfig_task) session.wait_for_task(reconfig_task) def _detach_disk_from_backing(self, session, backing, disk_device): LOG.debug("Reconfiguring backing VM: %(backing)s to remove disk: " "%(disk_device)s.", {'backing': backing, 'disk_device': disk_device}) cf = session.vim.client.factory reconfig_spec = cf.create('ns0:VirtualMachineConfigSpec') spec = self._create_spec_for_disk_remove(session, disk_device) reconfig_spec.deviceChange = [spec] self._reconfigure_backing(session, backing, reconfig_spec) def _attach_disk_to_backing(self, session, backing, disk_device): LOG.debug("Reconfiguring backing VM: %(backing)s to add disk: " "%(disk_device)s.", {'backing': backing, 'disk_device': disk_device}) cf = session.vim.client.factory reconfig_spec = cf.create('ns0:VirtualMachineConfigSpec') disk_spec = cf.create('ns0:VirtualDeviceConfigSpec') disk_spec.operation = 'add' disk_spec.device = disk_device reconfig_spec.deviceChange = [disk_spec] self._reconfigure_backing(session, backing, reconfig_spec) def _disconnect( self, backing, tmp_file_path, session, ds_ref, dc_ref, vmdk_path): # The restored volume is in compressed (streamOptimized) format. # So we upload it to a temporary location in vCenter datastore and copy # the compressed vmdk to the volume vmdk. The copy operation # decompresses the disk to a format suitable for attaching to Nova # instances in vCenter. dstore = datastore.get_datastore_by_ref(session, ds_ref) ds_path = dstore.build_path( VmdkConnector.TMP_IMAGES_DATASTORE_FOLDER_PATH, os.path.basename(tmp_file_path)) self._create_temp_ds_folder( session, six.text_type(ds_path.parent), dc_ref) with open(tmp_file_path, "rb") as tmp_file: dc_name = session.invoke_api( vim_util, 'get_object_property', session.vim, dc_ref, 'name') cookies = session.vim.client.options.transport.cookiejar cacerts = self._ca_file if self._ca_file else not self._insecure self._upload_vmdk( tmp_file, self._ip, self._port, dc_name, dstore.name, cookies, ds_path.rel_path, os.path.getsize(tmp_file_path), cacerts, self._timeout) disk_device = self._get_disk_device(session, backing) self._detach_disk_from_backing(session, backing, disk_device) src = six.text_type(ds_path) LOG.debug("Copying %(src)s to %(dest)s", {'src': src, 'dest': vmdk_path}) disk_mgr = session.vim.service_content.virtualDiskManager task = session.invoke_api(session.vim, 'CopyVirtualDisk_Task', disk_mgr, sourceName=src, sourceDatacenter=dc_ref, destName=vmdk_path, destDatacenter=dc_ref) session.wait_for_task(task) self._attach_disk_to_backing(session, backing, disk_device) # Delete the compressed vmdk at the temporary location. LOG.debug("Deleting %s", src) file_mgr = session.vim.service_content.fileManager task = session.invoke_api(session.vim, 'DeleteDatastoreFile_Task', file_mgr, name=src, datacenter=dc_ref) session.wait_for_task(task) def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): tmp_file_path = device_info['path'] if not os.path.exists(tmp_file_path): msg = _("Vmdk: %s not found.") % tmp_file_path raise exception.NotFound(message=msg) session = None try: # We upload the temporary file to vCenter server only if it is # modified after connect_volume. if os.path.getmtime(tmp_file_path) > device_info['last_modified']: self._load_config(connection_properties) session = self._create_session() backing = vim_util.get_moref(connection_properties['volume'], "VirtualMachine") # Currently there is no way we can restore the volume if it # contains redo-log based snapshots (bug 1599026). if self._snapshot_exists(session, backing): msg = (_("Backing of volume: %s contains one or more " "snapshots; cannot disconnect.") % connection_properties['volume_id']) raise exception.BrickException(message=msg) ds_ref = vim_util.get_moref( connection_properties['datastore'], "Datastore") dc_ref = vim_util.get_moref( connection_properties['datacenter'], "Datacenter") vmdk_path = connection_properties['vmdk_path'] self._disconnect( backing, tmp_file_path, session, ds_ref, dc_ref, vmdk_path) finally: os.remove(tmp_file_path) if session: session.logout() def extend_volume(self, connection_properties): raise NotImplementedError ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/connectors/vrtshyperscale.py0000664000175000017500000001327500000000000024670 0ustar00zuulzuul00000000000000# Copyright (c) 2017 Veritas Technologies LLC. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base from os_brick import utils LOG = logging.getLogger(__name__) synchronized = lockutils.synchronized_with_prefix('os-brick-vrts-hyperscale-') class HyperScaleConnector(base.BaseLinuxConnector): """Class implements the os-brick connector for HyperScale volumes.""" def __init__(self, root_helper, driver=None, execute=None, *args, **kwargs): super(HyperScaleConnector, self).__init__( root_helper, driver=driver, execute=execute, *args, **kwargs) def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def extend_volume(self, connection_properties): raise NotImplementedError @staticmethod def get_connector_properties(root_helper, *args, **kwargs): """The HyperScale connector properties.""" return {} @utils.trace @synchronized('connect_volume') def connect_volume(self, connection_properties): """Connect a volume to an instance.""" out = None err = None device_info = {} volume_name = None if 'name' in connection_properties.keys(): volume_name = connection_properties['name'] if volume_name is None: msg = _("Failed to connect volume: invalid volume name.") raise exception.BrickException(message=msg) cmd_arg = {'operation': 'connect_volume'} cmd_arg['volume_guid'] = volume_name cmdarg_json = json.dumps(cmd_arg) LOG.debug("HyperScale command hscli: %(cmd_arg)s", {'cmd_arg': cmdarg_json}) try: (out, err) = self._execute('hscli', cmdarg_json, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as e: msg = (_("Error executing hscli: %(err)s") % {'err': e.stderr}) raise exception.BrickException(message=msg) LOG.debug("Result of hscli: stdout=%(out)s " "stderr=%(err)s", {'out': out, 'err': err}) if err or out is None or len(out) == 0: msg = (_("Failed to connect volume with stdout=%(out)s " "stderr=%(err)s") % {'out': out, 'err': err}) raise exception.BrickException(message=msg) output = json.loads(out) payload = output.get('payload') if payload is None: msg = _("Failed to connect volume: " "hscli returned invalid payload") raise exception.BrickException(message=msg) if ('vsa_ip' not in payload.keys() or 'refl_factor' not in payload.keys()): msg = _("Failed to connect volume: " "hscli returned invalid results") raise exception.BrickException(message=msg) device_info['vsa_ip'] = payload.get('vsa_ip') device_info['path'] = ( '/dev/' + connection_properties['name'][1:32]) refl_factor = int(payload.get('refl_factor')) device_info['refl_factor'] = str(refl_factor) if refl_factor > 0: if 'refl_targets' not in payload.keys(): msg = _("Failed to connect volume: " "hscli returned inconsistent results") raise exception.BrickException(message=msg) device_info['refl_targets'] = ( payload.get('refl_targets')) return device_info @utils.trace @synchronized('connect_volume') def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from an instance.""" volume_name = None if 'name' in connection_properties.keys(): volume_name = connection_properties['name'] if volume_name is None: msg = _("Failed to disconnect volume: invalid volume name") raise exception.BrickException(message=msg) cmd_arg = {'operation': 'disconnect_volume'} cmd_arg['volume_guid'] = volume_name cmdarg_json = json.dumps(cmd_arg) LOG.debug("HyperScale command hscli: %(cmd_arg)s", {'cmd_arg': cmdarg_json}) try: (out, err) = self._execute('hscli', cmdarg_json, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as e: msg = (_("Error executing hscli: %(err)s") % {'err': e.stderr}) raise exception.BrickException(message=msg) if err: msg = (_("Failed to connect volume: stdout=%(out)s " "stderr=%(err)s") % {'out': out, 'err': err}) raise exception.BrickException(message=msg) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/host_driver.py0000664000175000017500000000215500000000000021760 0ustar00zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import errno import os class HostDriver(object): def get_all_block_devices(self): """Get the list of all block devices seen in /dev/disk/by-path/.""" dir = "/dev/disk/by-path/" try: files = os.listdir(dir) except OSError as e: if e.errno == errno.ENOENT: files = [] else: raise devices = [] for file in files: devices.append(dir + file) return devices ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/initiator_connector.py0000664000175000017500000002013300000000000023500 0ustar00zuulzuul00000000000000# All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import abc import six from os_brick import exception from os_brick import executor from os_brick import initiator @six.add_metaclass(abc.ABCMeta) class InitiatorConnector(executor.Executor): # This object can be used on any platform (x86, S390) platform = initiator.PLATFORM_ALL # This object can be used on any os type (linux, windows) os_type = initiator.OS_TYPE_ALL def __init__(self, root_helper, driver=None, execute=None, device_scan_attempts=initiator.DEVICE_SCAN_ATTEMPTS_DEFAULT, *args, **kwargs): super(InitiatorConnector, self).__init__(root_helper, execute=execute, *args, **kwargs) self.device_scan_attempts = device_scan_attempts def set_driver(self, driver): """The driver is used to find used LUNs.""" self.driver = driver @staticmethod @abc.abstractmethod def get_connector_properties(root_helper, *args, **kwargs): """The generic connector properties.""" pass @abc.abstractmethod def check_valid_device(self, path, run_as_root=True): """Test to see if the device path is a real device. :param path: The file system path for the device. :type path: str :param run_as_root: run the tests as root user? :type run_as_root: bool :returns: bool """ pass @abc.abstractmethod def connect_volume(self, connection_properties): """Connect to a volume. The connection_properties describes the information needed by the specific protocol to use to make the connection. The connection_properties is a dictionary that describes the target volume. It varies slightly by protocol type (iscsi, fibre_channel), but the structure is usually the same. An example for iSCSI: {'driver_volume_type': 'iscsi', 'data': { 'target_luns': [0, 2], 'target_iqns': ['iqn.2000-05.com.3pardata:20810002ac00383d', 'iqn.2000-05.com.3pardata:21810002ac00383d'], 'target_discovered': True, 'encrypted': False, 'qos_specs': None, 'target_portals': ['10.52.1.11:3260', '10.52.2.11:3260'], 'access_mode': 'rw', }} An example for fibre_channel with single lun: {'driver_volume_type': 'fibre_channel', 'data': { 'initiator_target_map': {'100010604b010459': ['20210002AC00383D'], '100010604b01045d': ['20220002AC00383D']}, 'target_discovered': True, 'encrypted': False, 'qos_specs': None, 'target_lun': 1, 'access_mode': 'rw', 'target_wwn': [ '20210002AC00383D', '20220002AC00383D', ], }} An example for fibre_channel target_wwns and with different LUNs and all host ports mapped to target ports: {'driver_volume_type': 'fibre_channel', 'data': { 'initiator_target_map': { '100010604b010459': ['20210002AC00383D', '20220002AC00383D'], '100010604b01045d': ['20210002AC00383D', '20220002AC00383D'] }, 'target_discovered': True, 'encrypted': False, 'qos_specs': None, 'target_luns': [1, 2], 'access_mode': 'rw', 'target_wwns': ['20210002AC00383D', '20220002AC00383D'], }} For FC the dictionary could also present the enable_wildcard_scan key with a boolean value (defaults to True) in case a driver doesn't want OS-Brick to use a SCSI scan with wildcards when the FC initiator on the host doesn't find any target port. This is useful for drivers that know that sysfs gets populated whenever there's a connection between the host's HBA and the storage array's target ports. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :returns: dict """ pass @abc.abstractmethod def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): """Disconnect a volume from the local host. The connection_properties are the same as from connect_volume. The device_info is returned from connect_volume. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict :param device_info: historical difference, but same as connection_props :type device_info: dict :param force: Whether to forcefully disconnect even if flush fails. :type force: bool :param ignore_errors: When force is True, this will decide whether to ignore errors or raise an exception once finished the operation. Default is False. :type ignore_errors: bool """ pass @abc.abstractmethod def get_volume_paths(self, connection_properties): """Return the list of existing paths for a volume. The job of this method is to find out what paths in the system are associated with a volume as described by the connection_properties. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ pass @abc.abstractmethod def get_search_path(self): """Return the directory where a Connector looks for volumes. Some Connectors need the information in the connection_properties to determine the search path. """ pass @abc.abstractmethod def extend_volume(self, connection_properties): """Update the attached volume's size. This method will attempt to update the local hosts's volume after the volume has been extended on the remote system. The new volume size in bytes will be returned. If there is a failure to update, then None will be returned. :param connection_properties: The volume connection properties. :returns: new size of the volume. """ pass @abc.abstractmethod def get_all_available_volumes(self, connection_properties=None): """Return all volumes that exist in the search directory. At connect_volume time, a Connector looks in a specific directory to discover a volume's paths showing up. This method's job is to return all paths in the directory that connect_volume uses to find a volume. This method is used in coordination with get_volume_paths() to verify that volumes have gone away after disconnect_volume has been called. :param connection_properties: The dictionary that describes all of the target volume attributes. :type connection_properties: dict """ pass def check_IO_handle_valid(self, handle, data_type, protocol): """Check IO handle has correct data type.""" if (handle and not isinstance(handle, data_type)): raise exception.InvalidIOHandleObject( protocol=protocol, actual_type=type(handle)) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/linuxfc.py0000664000175000017500000003417500000000000021107 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic linux Fibre Channel utilities.""" import errno import os from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick.initiator import linuxscsi LOG = logging.getLogger(__name__) class LinuxFibreChannel(linuxscsi.LinuxSCSI): def has_fc_support(self): FC_HOST_SYSFS_PATH = '/sys/class/fc_host' if os.path.isdir(FC_HOST_SYSFS_PATH): return True else: return False def _get_hba_channel_scsi_target_lun(self, hba, conn_props): """Get HBA channels, SCSI targets, LUNs to FC targets for given HBA. Given an HBA and the connection properties we look for the HBA channels and SCSI targets for each of the FC targets that this HBA has been granted permission to connect. For drivers that don't return an initiator to target map we try to find the info for all the target ports. For drivers that return an initiator_target_map we use the initiator_target_lun_map entry that was generated by the FC connector based on the contents of the connection information data to know which target ports to look for. :returns: 2-Tuple with the first entry being a list of [c, t, l] entries where the target port was found, and the second entry of the tuple being a set of luns for ports that were not found. """ # We want the targets' WWPNs, so we use the initiator_target_map if # present for this hba or default to targets if not present. targets = conn_props['targets'] if conn_props.get('initiator_target_map') is not None: # This map we try to use was generated by the FC connector targets = conn_props['initiator_target_lun_map'].get( hba['port_name'], targets) # Leave only the number from the host_device field (ie: host6) host_device = hba['host_device'] if host_device and len(host_device) > 4: host_device = host_device[4:] path = '/sys/class/fc_transport/target%s:' % host_device ctls = [] luns_not_found = set() for wwpn, lun in targets: cmd = 'grep -Gil "%(wwpns)s" %(path)s*/port_name' % {'wwpns': wwpn, 'path': path} try: # We need to run command in shell to expand the * glob out, _err = self._execute(cmd, shell=True) ctls += [line.split('/')[4].split(':')[1:] + [lun] for line in out.split('\n') if line.startswith(path)] except Exception as exc: LOG.debug('Could not get HBA channel and SCSI target ID, path:' ' %(path)s*, reason: %(reason)s', {'path': path, 'reason': exc}) # If we didn't find any paths add it to the not found list luns_not_found.add(lun) return ctls, luns_not_found def rescan_hosts(self, hbas, connection_properties): LOG.debug('Rescaning HBAs %(hbas)s with connection properties ' '%(conn_props)s', {'hbas': hbas, 'conn_props': connection_properties}) # Use initiator_target_lun_map (generated from initiator_target_map by # the FC connector) as HBA exclusion map ports = connection_properties.get('initiator_target_lun_map') if ports: hbas = [hba for hba in hbas if hba['port_name'] in ports] LOG.debug('Using initiator target map to exclude HBAs: %s', hbas) # Most storage arrays get their target ports automatically detected # by the Linux FC initiator and sysfs gets populated with that # information, but there are some that don't. We'll do a narrow scan # using the channel, target, and LUN for the former and a wider scan # for the latter. If all paths to a former type of array were down on # the system boot the array could look like it's of the latter type # and make us bring us unwanted volumes into the system by doing a # broad scan. To prevent this from happening Cinder drivers can use # the "enable_wildcard_scan" key in the connection_info to let us know # they don't want us to do broad scans even in those cases. broad_scan = connection_properties.get('enable_wildcard_scan', True) if not broad_scan: LOG.debug('Connection info disallows broad SCSI scanning') process = [] skipped = [] get_ctls = self._get_hba_channel_scsi_target_lun for hba in hbas: ctls, luns_wildcards = get_ctls(hba, connection_properties) # If we found the target ports, ignore HBAs that din't find them if ctls: process.append((hba, ctls)) # If target ports not found and should have, then the HBA is not # connected to our storage elif not broad_scan: LOG.debug('Skipping HBA %s, nothing to scan, target port ' 'not connected to initiator', hba['node_name']) # If we haven't found any target ports we may need to do broad # SCSI scans elif not process: skipped.append((hba, [('-', '-', lun) for lun in luns_wildcards])) # If we didn't find any target ports use wildcards if they are enabled process = process or skipped for hba, ctls in process: for hba_channel, target_id, target_lun in ctls: LOG.debug('Scanning %(host)s (wwnn: %(wwnn)s, c: ' '%(channel)s, t: %(target)s, l: %(lun)s)', {'host': hba['host_device'], 'wwnn': hba['node_name'], 'channel': hba_channel, 'target': target_id, 'lun': target_lun}) self.echo_scsi_command( "/sys/class/scsi_host/%s/scan" % hba['host_device'], "%(c)s %(t)s %(l)s" % {'c': hba_channel, 't': target_id, 'l': target_lun}) def get_fc_hbas(self): """Get the Fibre Channel HBA information.""" if not self.has_fc_support(): # there is no FC support in the kernel loaded # so there is no need to even try to run systool LOG.debug("No Fibre Channel support detected on system.") return [] out = None try: out, _err = self._execute('systool', '-c', 'fc_host', '-v', run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: # This handles the case where rootwrap is used # and systool is not installed # 96 = nova.cmd.rootwrap.RC_NOEXECFOUND: if exc.exit_code == 96: LOG.warning("systool is not installed") return [] except OSError as exc: # This handles the case where rootwrap is NOT used # and systool is not installed if exc.errno == errno.ENOENT: LOG.warning("systool is not installed") return [] # No FC HBAs were found if out is None: return [] lines = out.split('\n') # ignore the first 2 lines lines = lines[2:] hbas = [] hba = {} lastline = None for line in lines: line = line.strip() # 2 newlines denotes a new hba port if line == '' and lastline == '': if len(hba) > 0: hbas.append(hba) hba = {} else: val = line.split('=') if len(val) == 2: key = val[0].strip().replace(" ", "") value = val[1].strip() hba[key] = value.replace('"', '') lastline = line return hbas def get_fc_hbas_info(self): """Get Fibre Channel WWNs and device paths from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() hbas_info = [] for hba in hbas: wwpn = hba['port_name'].replace('0x', '') wwnn = hba['node_name'].replace('0x', '') device_path = hba['ClassDevicepath'] device = hba['ClassDevice'] hbas_info.append({'port_name': wwpn, 'node_name': wwnn, 'host_device': device, 'device_path': device_path}) return hbas_info def get_fc_wwpns(self): """Get Fibre Channel WWPNs from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() wwpns = [] for hba in hbas: if hba['port_state'] == 'Online': wwpn = hba['port_name'].replace('0x', '') wwpns.append(wwpn) return wwpns def get_fc_wwnns(self): """Get Fibre Channel WWNNs from the system, if any.""" # Note(walter-boring) modern Linux kernels contain the FC HBA's in /sys # and are obtainable via the systool app hbas = self.get_fc_hbas() wwnns = [] for hba in hbas: if hba['port_state'] == 'Online': wwnn = hba['node_name'].replace('0x', '') wwnns.append(wwnn) return wwnns class LinuxFibreChannelS390X(LinuxFibreChannel): def get_fc_hbas_info(self): """Get Fibre Channel WWNs and device paths from the system, if any.""" hbas = self.get_fc_hbas() hbas_info = [] for hba in hbas: if hba['port_state'] == 'Online': wwpn = hba['port_name'].replace('0x', '') wwnn = hba['node_name'].replace('0x', '') device_path = hba['ClassDevicepath'] device = hba['ClassDevice'] hbas_info.append({'port_name': wwpn, 'node_name': wwnn, 'host_device': device, 'device_path': device_path}) return hbas_info def configure_scsi_device(self, device_number, target_wwn, lun): """Write the LUN to the port's unit_add attribute. If auto-discovery of Fibre-Channel target ports is disabled on s390 platforms, ports need to be added to the configuration. If auto-discovery of LUNs is disabled on s390 platforms luns need to be added to the configuration through the unit_add interface """ LOG.debug("Configure lun for s390: device_number=%(device_num)s " "target_wwn=%(target_wwn)s target_lun=%(target_lun)s", {'device_num': device_number, 'target_wwn': target_wwn, 'target_lun': lun}) filepath = ("/sys/bus/ccw/drivers/zfcp/%s/%s" % (device_number, target_wwn)) if not (os.path.exists(filepath)): zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/port_rescan" % (device_number)) LOG.debug("port_rescan call for s390: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, "1") except putils.ProcessExecutionError as exc: LOG.warning("port_rescan call for s390 failed exit" " %(code)s, stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/%s/unit_add" % (device_number, target_wwn)) LOG.debug("unit_add call for s390 execute: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, lun) except putils.ProcessExecutionError as exc: LOG.warning("unit_add call for s390 failed exit %(code)s, " "stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) def deconfigure_scsi_device(self, device_number, target_wwn, lun): """Write the LUN to the port's unit_remove attribute. If auto-discovery of LUNs is disabled on s390 platforms luns need to be removed from the configuration through the unit_remove interface """ LOG.debug("Deconfigure lun for s390: " "device_number=%(device_num)s " "target_wwn=%(target_wwn)s target_lun=%(target_lun)s", {'device_num': device_number, 'target_wwn': target_wwn, 'target_lun': lun}) zfcp_device_command = ("/sys/bus/ccw/drivers/zfcp/%s/%s/unit_remove" % (device_number, target_wwn)) LOG.debug("unit_remove call for s390 execute: %s", zfcp_device_command) try: self.echo_scsi_command(zfcp_device_command, lun) except putils.ProcessExecutionError as exc: LOG.warning("unit_remove call for s390 failed exit %(code)s, " "stderr %(stderr)s", {'code': exc.exit_code, 'stderr': exc.stderr}) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/linuxrbd.py0000664000175000017500000001571500000000000021265 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may not # use this file except in compliance with the License. You may obtain a copy of # the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations under # the License. """Generic RBD connection utilities.""" import io from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import utils try: import rados import rbd except ImportError: rados = None rbd = None LOG = logging.getLogger(__name__) class RBDClient(object): def __init__(self, user, pool, *args, **kwargs): self.rbd_user = user self.rbd_pool = pool for attr in ['rbd_user', 'rbd_pool']: val = getattr(self, attr) if val is not None: setattr(self, attr, utils.convert_str(val)) # allow these to be overridden for testing self.rados = kwargs.get('rados', rados) self.rbd = kwargs.get('rbd', rbd) if self.rados is None: raise exception.InvalidParameterValue( err=_('rados module required')) if self.rbd is None: raise exception.InvalidParameterValue( err=_('rbd module required')) self.rbd_conf = kwargs.get('conffile', '/etc/ceph/ceph.conf') self.rbd_cluster_name = kwargs.get('rbd_cluster_name', 'ceph') self.rados_connect_timeout = kwargs.get('rados_connect_timeout', -1) self.client, self.ioctx = self.connect() def __enter__(self): return self def __exit__(self, type_, value, traceback): self.disconnect() def connect(self): LOG.debug("opening connection to ceph cluster (timeout=%s).", self.rados_connect_timeout) client = self.rados.Rados(rados_id=self.rbd_user, clustername=self.rbd_cluster_name, conffile=self.rbd_conf) try: if self.rados_connect_timeout >= 0: client.connect( timeout=self.rados_connect_timeout) else: client.connect() ioctx = client.open_ioctx(self.rbd_pool) return client, ioctx except self.rados.Error: msg = _("Error connecting to ceph cluster.") LOG.exception(msg) # shutdown cannot raise an exception client.shutdown() raise exception.BrickException(message=msg) def disconnect(self): # closing an ioctx cannot raise an exception self.ioctx.close() self.client.shutdown() class RBDVolume(object): """Context manager for dealing with an existing rbd volume.""" def __init__(self, client, name, snapshot=None, read_only=False): if snapshot is not None: snapshot = utils.convert_str(snapshot) try: self.image = client.rbd.Image(client.ioctx, utils.convert_str(name), snapshot=snapshot, read_only=read_only) except client.rbd.Error: LOG.exception("error opening rbd image %s", name) client.disconnect() raise # Ceph provides rbd.so to cinder, but we can't # get volume name from rbd.Image, so, we record # name here, so other modules can easily get # volume name. self.name = name self.client = client def close(self): try: self.image.close() finally: self.client.disconnect() def __enter__(self): return self def __exit__(self, type_, value, traceback): self.close() def __getattr__(self, attrib): return getattr(self.image, attrib) class RBDImageMetadata(object): """RBD image metadata to be used with RBDVolumeIOWrapper.""" def __init__(self, image, pool, user, conf): self.image = image self.pool = utils.convert_str(pool or '') self.user = utils.convert_str(user or '') self.conf = utils.convert_str(conf or '') class RBDVolumeIOWrapper(io.RawIOBase): """Enables LibRBD.Image objects to be treated as Python IO objects. Calling unimplemented interfaces will raise IOError. """ def __init__(self, rbd_volume): super(RBDVolumeIOWrapper, self).__init__() self._rbd_volume = rbd_volume self._offset = 0 def _inc_offset(self, length): self._offset += length @property def rbd_image(self): return self._rbd_volume.image @property def rbd_user(self): return self._rbd_volume.user @property def rbd_pool(self): return self._rbd_volume.pool @property def rbd_conf(self): return self._rbd_volume.conf def read(self, length=None): offset = self._offset total = self._rbd_volume.image.size() # NOTE(dosaboy): posix files do not barf if you read beyond their # length (they just return nothing) but rbd images do so we need to # return empty string if we have reached the end of the image. if (offset >= total): return b'' if length is None: length = total if (offset + length) > total: length = total - offset self._inc_offset(length) return self._rbd_volume.image.read(int(offset), int(length)) def write(self, data): self._rbd_volume.image.write(data, self._offset) self._inc_offset(len(data)) def seekable(self): return True def seek(self, offset, whence=0): if whence == 0: new_offset = offset elif whence == 1: new_offset = self._offset + offset elif whence == 2: new_offset = self._rbd_volume.image.size() new_offset += offset else: raise IOError(_("Invalid argument - whence=%s not supported") % (whence)) if (new_offset < 0): raise IOError(_("Invalid argument")) self._offset = new_offset def tell(self): return self._offset def flush(self): try: self._rbd_volume.image.flush() except AttributeError: LOG.warning("flush() not supported in this version of librbd") def fileno(self): """RBD does not have support for fileno() so we raise IOError. Raising IOError is recommended way to notify caller that interface is not supported - see http://docs.python.org/2/library/io.html#io.IOBase """ raise IOError(_("fileno() not supported by RBD()")) def close(self): self.rbd_image.close() ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/linuxscsi.py0000664000175000017500000007445600000000000021466 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generic linux scsi subsystem and Multipath utilities. Note, this is not iSCSI. """ import glob import os import re import time from oslo_concurrency import processutils as putils from oslo_log import log as logging import six from os_brick import exception from os_brick import executor from os_brick.privileged import rootwrap as priv_rootwrap from os_brick import utils LOG = logging.getLogger(__name__) MULTIPATH_ERROR_REGEX = re.compile(r"\w{3} \d+ \d\d:\d\d:\d\d \|.*$") MULTIPATH_WWID_REGEX = re.compile(r"\((?P.+)\)") MULTIPATH_DEVICE_ACTIONS = ['unchanged:', 'reject:', 'reload:', 'switchpg:', 'rename:', 'create:', 'resize:'] class LinuxSCSI(executor.Executor): # As found in drivers/scsi/scsi_lib.c WWN_TYPES = {'t10.': '1', 'eui.': '2', 'naa.': '3'} def echo_scsi_command(self, path, content): """Used to echo strings to scsi subsystem.""" args = ["-a", path] kwargs = dict(process_input=content, run_as_root=True, root_helper=self._root_helper) self._execute('tee', *args, **kwargs) def get_name_from_path(self, path): """Translates /dev/disk/by-path/ entry to /dev/sdX.""" name = os.path.realpath(path) if name.startswith("/dev/"): return name else: return None def remove_scsi_device(self, device, force=False, exc=None, flush=True): """Removes a scsi device based upon /dev/sdX name.""" path = "/sys/block/%s/device/delete" % device.replace("/dev/", "") if os.path.exists(path): exc = exception.ExceptionChainer() if exc is None else exc if flush: # flush any outstanding IO first with exc.context(force, 'Flushing %s failed', device): self.flush_device_io(device) LOG.debug("Remove SCSI device %(device)s with %(path)s", {'device': device, 'path': path}) with exc.context(force, 'Removing %s failed', device): self.echo_scsi_command(path, "1") def wait_for_volumes_removal(self, volumes_names): """Wait for device paths to be removed from the system.""" str_names = ', '.join(volumes_names) LOG.debug('Checking to see if SCSI volumes %s have been removed.', str_names) exist = ['/dev/' + volume_name for volume_name in volumes_names] # It can take up to 30 seconds to remove a SCSI device if the path # failed right before we start detaching, which is unlikely, but we # still shouldn't fail in that case. for i in range(61): exist = [path for path in exist if os.path.exists(path)] if not exist: LOG.debug("SCSI volumes %s have been removed.", str_names) return # Don't sleep on the last try since we are quitting if i < 60: time.sleep(0.5) # Log every 5 seconds if i % 10 == 0: LOG.debug('%s still exist.', ', '.join(exist)) raise exception.VolumePathNotRemoved(volume_path=exist) def get_device_info(self, device): (out, _err) = self._execute('sg_scan', device, run_as_root=True, root_helper=self._root_helper) dev_info = {'device': device, 'host': None, 'channel': None, 'id': None, 'lun': None} if out: line = out.strip() line = line.replace(device + ": ", "") info = line.split(" ") for item in info: if '=' in item: pair = item.split('=') dev_info[pair[0]] = pair[1] elif 'scsi' in item: dev_info['host'] = item.replace('scsi', '') return dev_info def get_sysfs_wwn(self, device_names, mpath=None): """Return the wwid from sysfs in any of devices in udev format.""" # If we have a multipath DM we know that it has found the WWN if mpath: # We have the WWN in /uuid even with friendly names, unline /name try: with open('/sys/block/%s/dm/uuid' % mpath) as f: # Contents are matph-WWN, so get the part we want wwid = f.read().strip()[6:] if wwid: # Check should not be needed, but just in case return wwid except Exception as exc: LOG.warning('Failed to read the DM uuid: %s', exc) wwid = self.get_sysfs_wwid(device_names) glob_str = '/dev/disk/by-id/scsi-' wwn_paths = glob.glob(glob_str + '*') # If we don't have multiple designators on page 0x83 if wwid and glob_str + wwid in wwn_paths: return wwid # If we have multiple designators use symlinks to find out the wwn device_names = set(device_names) for wwn_path in wwn_paths: try: if os.path.islink(wwn_path) and os.stat(wwn_path): path = os.path.realpath(wwn_path) if path.startswith('/dev/'): name = path[5:] # Symlink may point to the multipath dm if the attach # was too fast or we took long to check it. Check # devices belonging to the multipath DM. if name.startswith('dm-'): # Get the devices that belong to the DM slaves_path = '/sys/class/block/%s/slaves' % name dm_devs = os.listdir(slaves_path) # This is the right wwn_path if the devices we have # attached belong to the dm we followed if device_names.intersection(dm_devs): break # This is the right wwn_path if devices we have elif name in device_names: break except OSError: continue else: return '' return wwn_path[len(glob_str):] def get_sysfs_wwid(self, device_names): """Return the wwid from sysfs in any of devices in udev format.""" for device_name in device_names: try: with open('/sys/block/%s/device/wwid' % device_name) as f: wwid = f.read().strip() except IOError: continue # The sysfs wwid has the wwn type in string format as a prefix, # but udev uses its numerical representation as returned by # scsi_id's page 0x83, so we need to map it udev_wwid = self.WWN_TYPES.get(wwid[:4], '8') + wwid[4:] return udev_wwid return '' def get_scsi_wwn(self, path): """Read the WWN from page 0x83 value for a SCSI device.""" (out, _err) = self._execute('/lib/udev/scsi_id', '--page', '0x83', '--whitelisted', path, run_as_root=True, root_helper=self._root_helper) return out.strip() @staticmethod def is_multipath_running(enforce_multipath, root_helper, execute=None): try: if execute is None: execute = priv_rootwrap.execute cmd = ('multipathd', 'show', 'status') out, _err = execute(*cmd, run_as_root=True, root_helper=root_helper) # There was a bug in multipathd where it didn't return an error # code and just printed the error message in stdout. if out and out.startswith('error receiving packet'): raise putils.ProcessExecutionError('', out, 1, cmd, None) except putils.ProcessExecutionError as err: LOG.error('multipathd is not running: exit code %(err)s', {'err': err.exit_code}) if enforce_multipath: raise return False return True def get_dm_name(self, dm): """Get the Device map name given the device name of the dm on sysfs. :param dm: Device map name as seen in sysfs. ie: 'dm-0' :returns: String with the name, or empty string if not available. ie: '36e843b658476b7ed5bc1d4d10d9b1fde' """ try: with open('/sys/block/' + dm + '/dm/name') as f: return f.read().strip() except IOError: return '' def find_sysfs_multipath_dm(self, device_names): """Find the dm device name given a list of device names :param device_names: Iterable with device names, not paths. ie: ['sda'] :returns: String with the dm name or None if not found. ie: 'dm-0' """ glob_str = '/sys/block/%s/holders/dm-*' for dev_name in device_names: dms = glob.glob(glob_str % dev_name) if dms: __, device_name, __, dm = dms[0].rsplit('/', 3) return dm return None @staticmethod def get_dev_path(connection_properties, device_info): """Determine what path was used by Nova/Cinder to access volume.""" if device_info and device_info.get('path'): return device_info.get('path') return connection_properties.get('device_path') or '' @staticmethod def requires_flush(path, path_used, was_multipath): """Check if a device needs to be flushed when detaching. A device representing a single path connection to a volume must only be flushed if it has been used directly by Nova or Cinder to write data. If the path has been used via a multipath DM or if the device was part of a multipath but a different single path was used for I/O (instead of the multipath) then we don't need to flush. """ # No used path happens on failed attachs, when we don't care about # individual flushes. if not path_used: return False path = os.path.realpath(path) path_used = os.path.realpath(path_used) # Need to flush this device if we used this specific path. We check # this before checking if it's multipath in case we don't detect it # being multipath correctly (as in bug #1897787). if path_used == path: return True # We flush individual path if Nova didn't use a multipath and we # replaced the symlink to a real device with a link to the decrypted # DM. We know we replaced it because it doesn't link to /dev/XYZ, # instead it maps to /dev/mapped/crypt-XYZ return not was_multipath and '/dev' != os.path.split(path_used)[0] def remove_connection(self, devices_names, force=False, exc=None, path_used=None, was_multipath=False): """Remove LUNs and multipath associated with devices names. :param devices_names: Iterable with real device names ('sda', 'sdb') :param force: Whether to forcefully disconnect even if flush fails. :param exc: ExceptionChainer where to add exceptions if forcing :param path_used: What path was used by Nova/Cinder for I/O :param was_multipath: If the path used for I/O was a multipath :returns: Multipath device map name if found and not flushed """ if not devices_names: return exc = exception.ExceptionChainer() if exc is None else exc multipath_dm = self.find_sysfs_multipath_dm(devices_names) LOG.debug('Removing %(type)s devices %(devices)s', {'type': 'multipathed' if multipath_dm else 'single pathed', 'devices': ', '.join(devices_names)}) multipath_name = multipath_dm and self.get_dm_name(multipath_dm) if multipath_name: with exc.context(force, 'Flushing %s failed', multipath_name): self.flush_multipath_device(multipath_name) multipath_name = None multipath_running = True else: multipath_running = self.is_multipath_running( enforce_multipath=False, root_helper=self._root_helper) for device_name in devices_names: dev_path = '/dev/' + device_name if multipath_running: # Recent multipathd doesn't remove path devices in time when # it receives mutiple udev events in a short span, so here we # tell multipathd to remove the path device immediately. # Even if this step fails, later removing an iscsi device # triggers a udev event and multipathd can remove the path # device based on the udev event self.multipath_del_path(dev_path) flush = self.requires_flush(dev_path, path_used, was_multipath) self.remove_scsi_device(dev_path, force, exc, flush) # Wait until the symlinks are removed with exc.context(force, 'Some devices remain from %s', devices_names): try: self.wait_for_volumes_removal(devices_names) finally: # Since we use /dev/disk/by-id/scsi- links to get the wwn we # must ensure they are always removed. self._remove_scsi_symlinks(devices_names) return multipath_name def _remove_scsi_symlinks(self, devices_names): devices = ['/dev/' + dev for dev in devices_names] links = glob.glob('/dev/disk/by-id/scsi-*') unlink = [] for link in links: try: if os.path.realpath(link) in devices: unlink.append(link) except OSError: # A race condition in Python's posixpath:realpath just occurred # so we can ignore it because the file was just removed between # a check if file exists and a call to os.readlink continue if unlink: priv_rootwrap.unlink_root(no_errors=True, *unlink) def flush_device_io(self, device): """This is used to flush any remaining IO in the buffers.""" if os.path.exists(device): try: # NOTE(geguileo): With 30% connection error rates flush can get # stuck, set timeout to prevent it from hanging here forever. # Retry twice after 20 and 40 seconds. LOG.debug("Flushing IO for device %s", device) self._execute('blockdev', '--flushbufs', device, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: LOG.warning("Failed to flush IO buffers prior to removing " "device: %(code)s", {'code': exc.exit_code}) raise def flush_multipath_device(self, device_map_name): LOG.debug("Flush multipath device %s", device_map_name) # NOTE(geguileo): With 30% connection error rates flush can get stuck, # set timeout to prevent it from hanging here forever. Retry twice # after 20 and 40 seconds. self._execute('multipath', '-f', device_map_name, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self._root_helper) @utils.retry(exceptions=exception.VolumeDeviceNotFound) def wait_for_path(self, volume_path): """Wait for a path to show up.""" LOG.debug("Checking to see if %s exists yet.", volume_path) if not os.path.exists(volume_path): LOG.debug("%(path)s doesn't exists yet.", {'path': volume_path}) raise exception.VolumeDeviceNotFound( device=volume_path) else: LOG.debug("%s has shown up.", volume_path) @utils.retry(exceptions=exception.BlockDeviceReadOnly, retries=5) def wait_for_rw(self, wwn, device_path): """Wait for block device to be Read-Write.""" LOG.debug("Checking to see if %s is read-only.", device_path) out, info = self._execute('lsblk', '-o', 'NAME,RO', '-l', '-n') LOG.debug("lsblk output: %s", out) blkdevs = out.splitlines() for blkdev in blkdevs: # Entries might look like: # # "3624a93709a738ed78583fd120013902b (dm-1) 1" # # or # # "sdd 0" # # We are looking for the first and last part of them. For FC # multipath devices the name is in the format of ' (dm-)' blkdev_parts = blkdev.split(' ') ro = blkdev_parts[-1] name = blkdev_parts[0] # We must validate that all pieces of the dm-# device are rw, # if some are still ro it can cause problems. if wwn in name and int(ro) == 1: LOG.debug("Block device %s is read-only", device_path) self._execute('multipath', '-r', check_exit_code=[0, 1, 21], run_as_root=True, root_helper=self._root_helper) raise exception.BlockDeviceReadOnly( device=device_path) else: LOG.debug("Block device %s is not read-only.", device_path) def find_multipath_device_path(self, wwn): """Look for the multipath device file for a volume WWN. Multipath devices can show up in several places on a linux system. 1) When multipath friendly names are ON: a device file will show up in /dev/disk/by-id/dm-uuid-mpath- /dev/disk/by-id/dm-name-mpath /dev/disk/by-id/scsi-mpath /dev/mapper/mpath 2) When multipath friendly names are OFF: /dev/disk/by-id/dm-uuid-mpath- /dev/disk/by-id/scsi- /dev/mapper/ """ LOG.info("Find Multipath device file for volume WWN %(wwn)s", {'wwn': wwn}) # First look for the common path wwn_dict = {'wwn': wwn} path = "/dev/disk/by-id/dm-uuid-mpath-%(wwn)s" % wwn_dict try: self.wait_for_path(path) return path except exception.VolumeDeviceNotFound: pass # for some reason the common path wasn't found # lets try the dev mapper path path = "/dev/mapper/%(wwn)s" % wwn_dict try: self.wait_for_path(path) return path except exception.VolumeDeviceNotFound: pass # couldn't find a path LOG.warning("couldn't find a valid multipath device path for " "%(wwn)s", wwn_dict) return None def find_multipath_device(self, device): """Discover multipath devices for a mpath device. This uses the slow multipath -l command to find a multipath device description, then screen scrapes the output to discover the multipath device name and it's devices. """ mdev = None devices = [] out = None try: (out, _err) = self._execute('multipath', '-l', device, run_as_root=True, root_helper=self._root_helper) except putils.ProcessExecutionError as exc: LOG.warning("multipath call failed exit %(code)s", {'code': exc.exit_code}) raise exception.CommandExecutionFailed( cmd='multipath -l %s' % device) if out: lines = out.strip() lines = lines.split("\n") lines = [line for line in lines if not re.match(MULTIPATH_ERROR_REGEX, line) and len(line)] if lines: mdev_name = lines[0].split(" ")[0] if mdev_name in MULTIPATH_DEVICE_ACTIONS: mdev_name = lines[0].split(" ")[1] mdev = '/dev/mapper/%s' % mdev_name # Confirm that the device is present. try: os.stat(mdev) except OSError: LOG.warning("Couldn't find multipath device %s", mdev) return None wwid_search = MULTIPATH_WWID_REGEX.search(lines[0]) if wwid_search is not None: mdev_id = wwid_search.group('wwid') else: mdev_id = mdev_name LOG.debug("Found multipath device = %(mdev)s", {'mdev': mdev}) device_lines = lines[3:] for dev_line in device_lines: if dev_line.find("policy") != -1: continue dev_line = dev_line.lstrip(' |-`') dev_info = dev_line.split() address = dev_info[0].split(":") dev = {'device': '/dev/%s' % dev_info[1], 'host': address[0], 'channel': address[1], 'id': address[2], 'lun': address[3] } devices.append(dev) if mdev is not None: info = {"device": mdev, "id": mdev_id, "name": mdev_name, "devices": devices} return info return None def get_device_size(self, device): """Get the size in bytes of a volume.""" (out, _err) = self._execute('blockdev', '--getsize64', device, run_as_root=True, root_helper=self._root_helper) var = six.text_type(out.strip()) if var.isnumeric(): return int(var) else: return None def multipath_reconfigure(self): """Issue a multipathd reconfigure. When attachments come and go, the multipathd seems to get lost and not see the maps. This causes resize map to fail 100%. To overcome this we have to issue a reconfigure prior to resize map. """ (out, _err) = self._execute('multipathd', 'reconfigure', run_as_root=True, root_helper=self._root_helper) return out def multipath_resize_map(self, mpath_id): """Issue a multipath resize map on device. This forces the multipath daemon to update it's size information a particular multipath device. """ (out, _err) = self._execute('multipathd', 'resize', 'map', mpath_id, run_as_root=True, root_helper=self._root_helper) return out def extend_volume(self, volume_paths, use_multipath=False): """Signal the SCSI subsystem to test for volume resize. This function tries to signal the local system's kernel that an already attached volume might have been resized. """ LOG.debug("extend volume %s", volume_paths) for volume_path in volume_paths: device = self.get_device_info(volume_path) LOG.debug("Volume device info = %s", device) device_id = ("%(host)s:%(channel)s:%(id)s:%(lun)s" % {'host': device['host'], 'channel': device['channel'], 'id': device['id'], 'lun': device['lun']}) scsi_path = ("/sys/bus/scsi/drivers/sd/%(device_id)s" % {'device_id': device_id}) size = self.get_device_size(volume_path) LOG.debug("Starting size: %s", size) # now issue the device rescan rescan_path = "%(scsi_path)s/rescan" % {'scsi_path': scsi_path} self.echo_scsi_command(rescan_path, "1") new_size = self.get_device_size(volume_path) LOG.debug("volume size after scsi device rescan %s", new_size) scsi_wwn = self.get_scsi_wwn(volume_paths[0]) if use_multipath: mpath_device = self.find_multipath_device_path(scsi_wwn) if mpath_device: # Force a reconfigure so that resize works self.multipath_reconfigure() size = self.get_device_size(mpath_device) LOG.info("mpath(%(device)s) current size %(size)s", {'device': mpath_device, 'size': size}) result = self.multipath_resize_map(scsi_wwn) if 'fail' in result: LOG.error("Multipathd failed to update the size mapping " "of multipath device %(scsi_wwn)s volume " "%(volume)s", {'scsi_wwn': scsi_wwn, 'volume': volume_paths}) return None new_size = self.get_device_size(mpath_device) LOG.info("mpath(%(device)s) new size %(size)s", {'device': mpath_device, 'size': new_size}) return new_size def process_lun_id(self, lun_ids): if isinstance(lun_ids, list): processed = [] for x in lun_ids: x = self._format_lun_id(x) processed.append(x) else: processed = self._format_lun_id(lun_ids) return processed def _format_lun_id(self, lun_id): # make sure lun_id is an int lun_id = int(lun_id) if lun_id < 256: return lun_id else: return ("0x%04x%04x00000000" % (lun_id & 0xffff, lun_id >> 16 & 0xffff)) def get_hctl(self, session, lun): """Given an iSCSI session return the host, channel, target, and lun.""" glob_str = '/sys/class/iscsi_host/host*/device/session' + session paths = glob.glob(glob_str + '/target*') if paths: __, channel, target = os.path.split(paths[0])[1].split(':') # Check if we can get the host else: target = channel = '-' paths = glob.glob(glob_str) if not paths: LOG.debug('No hctl found on session %s with lun %s', session, lun) return None # Extract the host number from the path host = paths[0][26:paths[0].index('/', 26)] res = (host, channel, target, lun) LOG.debug('HCTL %s found on session %s with lun %s', res, session, lun) return res def device_name_by_hctl(self, session, hctl): """Find the device name given a session and the hctl. :param session: A string with the session number :param hctl: An iterable with the host, channel, target, and lun as passed to scan. ie: ('5', '-', '-', '0') """ if '-' in hctl: hctl = ['*' if x == '-' else x for x in hctl] path = ('/sys/class/scsi_host/host%(h)s/device/session%(s)s/target' '%(h)s:%(c)s:%(t)s/%(h)s:%(c)s:%(t)s:%(l)s/block/*' % {'h': hctl[0], 'c': hctl[1], 't': hctl[2], 'l': hctl[3], 's': session}) # Sort devices and return the first so we don't return a partition devices = sorted(glob.glob(path)) device = os.path.split(devices[0])[1] if devices else None LOG.debug('Searching for a device in session %s and hctl %s yield: %s', session, hctl, device) return device def scan_iscsi(self, host, channel='-', target='-', lun='-'): """Send an iSCSI scan request given the host and optionally the ctl.""" LOG.debug('Scanning host %(host)s c: %(channel)s, ' 't: %(target)s, l: %(lun)s)', {'host': host, 'channel': channel, 'target': target, 'lun': lun}) self.echo_scsi_command('/sys/class/scsi_host/host%s/scan' % host, '%(c)s %(t)s %(l)s' % {'c': channel, 't': target, 'l': lun}) def multipath_add_wwid(self, wwid): """Add a wwid to the list of know multipath wwids. This has the effect of multipathd being willing to create a dm for a multipath even when there's only 1 device. """ out, err = self._execute('multipath', '-a', wwid, run_as_root=True, check_exit_code=False, root_helper=self._root_helper) return out.strip() == "wwid '" + wwid + "' added" def multipath_add_path(self, realpath): """Add a path to multipathd for monitoring. This has the effect of multipathd checking an already checked device for multipath. Together with `multipath_add_wwid` we can create a multipath when there's only 1 path. """ stdout, stderr = self._execute('multipathd', 'add', 'path', realpath, run_as_root=True, timeout=5, check_exit_code=False, root_helper=self._root_helper) return stdout.strip() == 'ok' def multipath_del_path(self, realpath): """Remove a path from multipathd for monitoring.""" stdout, stderr = self._execute('multipathd', 'del', 'path', realpath, run_as_root=True, timeout=5, check_exit_code=False, root_helper=self._root_helper) return stdout.strip() == 'ok' ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/utils.py0000664000175000017500000000274400000000000020574 0ustar00zuulzuul00000000000000# Copyright 2018 Red Hat, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import contextlib import os from oslo_concurrency import lockutils from oslo_concurrency import processutils as putils def check_manual_scan(): if os.name == 'nt': return False try: putils.execute('grep', '-F', 'node.session.scan', '/sbin/iscsiadm') except putils.ProcessExecutionError: return False return True ISCSI_SUPPORTS_MANUAL_SCAN = check_manual_scan() @contextlib.contextmanager def guard_connection(device): """Context Manager handling locks for attach/detach operations.""" if ISCSI_SUPPORTS_MANUAL_SCAN or not device.get('shared_targets'): yield else: # Cinder passes an OVO, but Nova passes a dictionary, so we use dict # key access that works with both. with lockutils.lock(device['service_uuid'], 'os-brick-', external=True): yield ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/initiator/windows/0000775000175000017500000000000000000000000020545 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/windows/__init__.py0000664000175000017500000000000000000000000022644 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/windows/base.py0000664000175000017500000001053700000000000022037 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick import initiator from os_brick.initiator import initiator_connector from os_brick import utils LOG = logging.getLogger(__name__) class BaseWindowsConnector(initiator_connector.InitiatorConnector): platform = initiator.PLATFORM_ALL os_type = initiator.OS_TYPE_WINDOWS DEFAULT_DEVICE_SCAN_INTERVAL = 2 def __init__(self, root_helper=None, *args, **kwargs): super(BaseWindowsConnector, self).__init__(root_helper, *args, **kwargs) self.device_scan_interval = kwargs.pop( 'device_scan_interval', self.DEFAULT_DEVICE_SCAN_INTERVAL) self._diskutils = utilsfactory.get_diskutils() @staticmethod def check_multipath_support(enforce_multipath): hostutils = utilsfactory.get_hostutils() mpio_enabled = hostutils.check_server_feature( hostutils.FEATURE_MPIO) if not mpio_enabled: err_msg = _("Using multipath connections for iSCSI and FC disks " "requires the Multipath IO Windows feature to be " "enabled. MPIO must be configured to claim such " "devices.") LOG.error(err_msg) if enforce_multipath: raise exception.BrickException(err_msg) return False return True @staticmethod def get_connector_properties(*args, **kwargs): multipath = kwargs['multipath'] enforce_multipath = kwargs['enforce_multipath'] props = {} props['multipath'] = ( multipath and BaseWindowsConnector.check_multipath_support(enforce_multipath)) return props def _get_scsi_wwn(self, device_number): # NOTE(lpetrut): The Linux connectors use scsi_id to retrieve the # disk unique id, which prepends the identifier type to the unique id # retrieved from the page 83 SCSI inquiry data. We'll do the same # to remain consistent. disk_uid, uid_type = self._diskutils.get_disk_uid_and_uid_type( device_number) scsi_wwn = '%s%s' % (uid_type, disk_uid) return scsi_wwn def check_valid_device(self, path, *args, **kwargs): try: with open(path, 'r') as dev: dev.read(1) except IOError: LOG.exception( "Failed to access the device on the path " "%(path)s", {"path": path}) return False return True def get_all_available_volumes(self): # TODO(lpetrut): query for disks based on the protocol used. return [] def _check_device_paths(self, device_paths): if len(device_paths) > 1: err_msg = _("Multiple volume paths were found: %s. This can " "occur if multipath is used and MPIO is not " "properly configured, thus not claiming the device " "paths. This issue must be addressed urgently as " "it can lead to data corruption.") raise exception.BrickException(err_msg % device_paths) @utils.trace def extend_volume(self, connection_properties): volume_paths = self.get_volume_paths(connection_properties) if not volume_paths: err_msg = _("Could not find the disk. Extend failed.") raise exception.NotFound(err_msg) device_path = volume_paths[0] device_number = self._diskutils.get_device_number_from_device_name( device_path) self._diskutils.refresh_disk(device_number) def get_search_path(self): return None ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/windows/fibre_channel.py0000664000175000017500000002035300000000000023701 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import time from os_win import exceptions as os_win_exc from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.windows import base as win_conn_base from os_brick import utils LOG = logging.getLogger(__name__) class WindowsFCConnector(win_conn_base.BaseWindowsConnector): def __init__(self, *args, **kwargs): super(WindowsFCConnector, self).__init__(*args, **kwargs) self.use_multipath = kwargs.get('use_multipath', False) self._fc_utils = utilsfactory.get_fc_utils() @staticmethod def get_connector_properties(*args, **kwargs): props = {} fc_utils = utilsfactory.get_fc_utils() fc_utils.refresh_hba_configuration() fc_hba_ports = fc_utils.get_fc_hba_ports() if fc_hba_ports: wwnns = [] wwpns = [] for port in fc_hba_ports: wwnns.append(port['node_name']) wwpns.append(port['port_name']) props['wwpns'] = wwpns props['wwnns'] = list(set(wwnns)) return props @utils.trace def connect_volume(self, connection_properties): volume_paths = self.get_volume_paths(connection_properties) if not volume_paths: raise exception.NoFibreChannelVolumeDeviceFound() device_path = volume_paths[0] device_number = self._diskutils.get_device_number_from_device_name( device_path) scsi_wwn = self._get_scsi_wwn(device_number) device_info = {'type': 'block', 'path': device_path, 'number': device_number, 'scsi_wwn': scsi_wwn} return device_info @utils.trace def get_volume_paths(self, connection_properties): # Returns a list containing at most one disk path such as # \\.\PhysicalDrive4. # # If multipath is used and the MPIO service is properly configured # to claim the disks, we'll still get a single device path, having # the same format, which will be used for all the IO operations. for attempt_num in range(self.device_scan_attempts): disk_paths = set() if attempt_num: time.sleep(self.device_scan_interval) self._diskutils.rescan_disks() volume_mappings = self._get_fc_volume_mappings( connection_properties) LOG.debug("Retrieved volume mappings %(vol_mappings)s " "for volume %(conn_props)s", dict(vol_mappings=volume_mappings, conn_props=connection_properties)) for mapping in volume_mappings: device_name = mapping['device_name'] if device_name: disk_paths.add(device_name) if not disk_paths and volume_mappings: fcp_lun = volume_mappings[0]['fcp_lun'] try: disk_paths = self._get_disk_paths_by_scsi_id( connection_properties, fcp_lun) disk_paths = set(disk_paths or []) except os_win_exc.OSWinException as ex: LOG.debug("Failed to retrieve disk paths by SCSI ID. " "Exception: %s", ex) if not disk_paths: LOG.debug("No disk path retrieved yet.") continue if len(disk_paths) > 1: LOG.debug("Multiple disk paths retrieved: %s This may happen " "if MPIO did not claim them yet.", disk_paths) continue dev_num = self._diskutils.get_device_number_from_device_name( list(disk_paths)[0]) if self.use_multipath and not self._diskutils.is_mpio_disk( dev_num): LOG.debug("Multipath was requested but the disk %s was not " "claimed yet by the MPIO service.", dev_num) continue return list(disk_paths) return [] def _get_fc_volume_mappings(self, connection_properties): # Note(lpetrut): All the WWNs returned by os-win are upper case. target_wwpns = [wwpn.upper() for wwpn in connection_properties['target_wwn']] target_lun = connection_properties['target_lun'] volume_mappings = [] hba_mappings = self._get_fc_hba_mappings() for node_name in hba_mappings: target_mappings = self._fc_utils.get_fc_target_mappings(node_name) for mapping in target_mappings: if (mapping['port_name'] in target_wwpns and mapping['lun'] == target_lun): volume_mappings.append(mapping) return volume_mappings def _get_fc_hba_mappings(self): mappings = collections.defaultdict(list) fc_hba_ports = self._fc_utils.get_fc_hba_ports() for port in fc_hba_ports: mappings[port['node_name']].append(port['port_name']) return mappings def _get_disk_paths_by_scsi_id(self, connection_properties, fcp_lun): for local_port_wwn, remote_port_wwns in connection_properties[ 'initiator_target_map'].items(): for remote_port_wwn in remote_port_wwns: try: dev_nums = self._get_dev_nums_by_scsi_id( local_port_wwn, remote_port_wwn, fcp_lun) # This may raise a DiskNotFound exception if the disks # are meanwhile claimed by the MPIO service. disk_paths = [ self._diskutils.get_device_name_by_device_number( dev_num) for dev_num in dev_nums] return disk_paths except os_win_exc.FCException as ex: LOG.debug("Failed to retrieve volume paths by SCSI id. " "Exception: %s", ex) continue return [] def _get_dev_nums_by_scsi_id(self, local_port_wwn, remote_port_wwn, fcp_lun): LOG.debug("Fetching SCSI Unique ID for FCP lun %(fcp_lun)s. " "Port WWN: %(local_port_wwn)s. " "Remote port WWN: %(remote_port_wwn)s.", dict(fcp_lun=fcp_lun, local_port_wwn=local_port_wwn, remote_port_wwn=remote_port_wwn)) local_hba_wwn = self._get_fc_hba_wwn_for_port(local_port_wwn) # This will return the SCSI identifiers in the order of precedence # used by Windows. identifiers = self._fc_utils.get_scsi_device_identifiers( local_hba_wwn, local_port_wwn, remote_port_wwn, fcp_lun) if identifiers: identifier = identifiers[0] dev_nums = self._diskutils.get_disk_numbers_by_unique_id( unique_id=identifier['id'], unique_id_format=identifier['type']) return dev_nums return [] def _get_fc_hba_wwn_for_port(self, port_wwn): fc_hba_ports = self._fc_utils.get_fc_hba_ports() for port in fc_hba_ports: if port_wwn.upper() == port['port_name']: return port['node_name'] err_msg = _("Could not find any FC HBA port " "having WWN '%s'.") % port_wwn raise exception.NotFound(err_msg) @utils.trace def disconnect_volume(self, connection_properties, device_info=None, force=False, ignore_errors=False): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/windows/iscsi.py0000664000175000017500000001634700000000000022244 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_win import exceptions as os_win_exc from os_win import utilsfactory from oslo_log import log as logging from os_brick import exception from os_brick.i18n import _ from os_brick.initiator.connectors import base_iscsi from os_brick.initiator.windows import base as win_conn_base from os_brick import utils LOG = logging.getLogger(__name__) class WindowsISCSIConnector(win_conn_base.BaseWindowsConnector, base_iscsi.BaseISCSIConnector): def __init__(self, *args, **kwargs): super(WindowsISCSIConnector, self).__init__(*args, **kwargs) self.use_multipath = kwargs.pop('use_multipath', False) self.initiator_list = kwargs.pop('initiator_list', []) self._iscsi_utils = utilsfactory.get_iscsi_initiator_utils() self.validate_initiators() def validate_initiators(self): """Validates the list of requested initiator HBAs Validates the list of requested initiator HBAs to be used when establishing iSCSI sessions. """ valid_initiator_list = True if not self.initiator_list: LOG.info("No iSCSI initiator was explicitly requested. " "The Microsoft iSCSI initiator will choose the " "initiator when establishing sessions.") else: available_initiators = self._iscsi_utils.get_iscsi_initiators() for initiator in self.initiator_list: if initiator not in available_initiators: LOG.warning("The requested initiator %(req_initiator)s " "is not in the list of available initiators: " "%(avail_initiators)s.", dict(req_initiator=initiator, avail_initiators=available_initiators)) valid_initiator_list = False return valid_initiator_list def get_initiator(self): """Returns the iSCSI initiator node name.""" return self._iscsi_utils.get_iscsi_initiator() @staticmethod def get_connector_properties(*args, **kwargs): iscsi_utils = utilsfactory.get_iscsi_initiator_utils() initiator = iscsi_utils.get_iscsi_initiator() return dict(initiator=initiator) def _get_all_paths(self, connection_properties): initiator_list = self.initiator_list or [None] all_targets = self._get_all_targets(connection_properties) paths = [(initiator_name, target_portal, target_iqn, target_lun) for target_portal, target_iqn, target_lun in all_targets for initiator_name in initiator_list] return paths @utils.trace def connect_volume(self, connection_properties): connected_target_mappings = set() volume_connected = False for (initiator_name, target_portal, target_iqn, target_lun) in self._get_all_paths(connection_properties): try: LOG.info("Attempting to establish an iSCSI session to " "target %(target_iqn)s on portal %(target_portal)s " "accessing LUN %(target_lun)s using initiator " "%(initiator_name)s.", dict(target_portal=target_portal, target_iqn=target_iqn, target_lun=target_lun, initiator_name=initiator_name)) self._iscsi_utils.login_storage_target( target_lun=target_lun, target_iqn=target_iqn, target_portal=target_portal, auth_username=connection_properties.get('auth_username'), auth_password=connection_properties.get('auth_password'), mpio_enabled=self.use_multipath, initiator_name=initiator_name, ensure_lun_available=False) connected_target_mappings.add((target_iqn, target_lun)) if not self.use_multipath: break except os_win_exc.OSWinException: LOG.exception("Could not establish the iSCSI session.") for target_iqn, target_lun in connected_target_mappings: try: (device_number, device_path) = self._iscsi_utils.get_device_number_and_path( target_iqn, target_lun, retry_attempts=self.device_scan_attempts, retry_interval=self.device_scan_interval, rescan_disks=True, ensure_mpio_claimed=self.use_multipath) volume_connected = True except os_win_exc.OSWinException: LOG.exception("Could not retrieve device path for target " "%(target_iqn)s and lun %(target_lun)s.", dict(target_iqn=target_iqn, target_lun=target_lun)) if not volume_connected: raise exception.BrickException( _("Could not connect volume %s.") % connection_properties) scsi_wwn = self._get_scsi_wwn(device_number) device_info = {'type': 'block', 'path': device_path, 'number': device_number, 'scsi_wwn': scsi_wwn} return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info=None, force=False, ignore_errors=False): # We want to refresh the cached information first. self._diskutils.rescan_disks() for (target_portal, target_iqn, target_lun) in self._get_all_targets(connection_properties): luns = self._iscsi_utils.get_target_luns(target_iqn) # We disconnect the target only if it does not expose other # luns which may be in use. if not luns or luns == [target_lun]: self._iscsi_utils.logout_storage_target(target_iqn) @utils.trace def get_volume_paths(self, connection_properties): device_paths = set() for (target_portal, target_iqn, target_lun) in self._get_all_targets(connection_properties): (device_number, device_path) = self._iscsi_utils.get_device_number_and_path( target_iqn, target_lun, ensure_mpio_claimed=self.use_multipath) if device_path: device_paths.add(device_path) self._check_device_paths(device_paths) return list(device_paths) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/initiator/windows/smbfs.py0000664000175000017500000001211300000000000022227 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os from os_win import utilsfactory from os_brick.initiator.windows import base as win_conn_base from os_brick.remotefs import windows_remotefs as remotefs from os_brick import utils # The Windows SMBFS connector expects to receive VHD/x images stored on SMB # shares, exposed by the Cinder SMBFS driver. class WindowsSMBFSConnector(win_conn_base.BaseWindowsConnector): def __init__(self, *args, **kwargs): super(WindowsSMBFSConnector, self).__init__(*args, **kwargs) # If this flag is set, we use the local paths in case of local # shares. This is in fact mandatory in some cases, for example # for the Hyper-C scenario. self._local_path_for_loopback = kwargs.get('local_path_for_loopback', True) self._expect_raw_disk = kwargs.get('expect_raw_disk', False) self._remotefsclient = remotefs.WindowsRemoteFsClient( mount_type='smbfs', *args, **kwargs) self._smbutils = utilsfactory.get_smbutils() self._vhdutils = utilsfactory.get_vhdutils() self._diskutils = utilsfactory.get_diskutils() @staticmethod def get_connector_properties(*args, **kwargs): # No connector properties updates in this case. return {} @utils.trace def connect_volume(self, connection_properties): self.ensure_share_mounted(connection_properties) # This will be a virtual disk image path. disk_path = self._get_disk_path(connection_properties) if self._expect_raw_disk: # The caller expects a direct accessible raw disk. We'll # mount the image and bring the new disk offline, which will # allow direct IO, while ensuring that any partiton residing # on it will be unmounted. read_only = connection_properties.get('access_mode') == 'ro' self._vhdutils.attach_virtual_disk(disk_path, read_only=read_only) raw_disk_path = self._vhdutils.get_virtual_disk_physical_path( disk_path) dev_num = self._diskutils.get_device_number_from_device_name( raw_disk_path) self._diskutils.set_disk_offline(dev_num) else: raw_disk_path = None device_info = {'type': 'file', 'path': raw_disk_path if self._expect_raw_disk else disk_path} return device_info @utils.trace def disconnect_volume(self, connection_properties, device_info=None, force=False, ignore_errors=False): export_path = self._get_export_path(connection_properties) disk_path = self._get_disk_path(connection_properties) # The detach method will silently continue if the disk is # not attached. self._vhdutils.detach_virtual_disk(disk_path) self._remotefsclient.unmount(export_path) def _get_export_path(self, connection_properties): return connection_properties['export'].replace('/', '\\') def _get_disk_path(self, connection_properties): # This is expected to be the share address, as an UNC path. export_path = self._get_export_path(connection_properties) mount_base = self._remotefsclient.get_mount_base() use_local_path = (self._local_path_for_loopback and self._smbutils.is_local_share(export_path)) disk_dir = export_path if mount_base: # This will be a symlink pointing to either the share # path directly or to the local share path, if requested # and available. disk_dir = self._remotefsclient.get_mount_point( export_path) elif use_local_path: disk_dir = self._remotefsclient.get_local_share_path(export_path) disk_name = connection_properties['name'] disk_path = os.path.join(disk_dir, disk_name) return disk_path def get_search_path(self): return self._remotefsclient.get_mount_base() @utils.trace def get_volume_paths(self, connection_properties): return [self._get_disk_path(connection_properties)] def ensure_share_mounted(self, connection_properties): export_path = self._get_export_path(connection_properties) mount_options = connection_properties.get('options') self._remotefsclient.mount(export_path, mount_options) def extend_volume(self, connection_properties): raise NotImplementedError ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/local_dev/0000775000175000017500000000000000000000000017001 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/local_dev/__init__.py0000664000175000017500000000000000000000000021100 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/local_dev/lvm.py0000664000175000017500000010042000000000000020146 0ustar00zuulzuul00000000000000# Copyright 2013 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """LVM class for performing LVM operations.""" import math import os import re from os_brick import exception from os_brick import executor from os_brick.privileged import rootwrap as priv_rootwrap from os_brick import utils from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import excutils from six import moves LOG = logging.getLogger(__name__) class LVM(executor.Executor): """LVM object to enable various LVM related operations.""" LVM_CMD_PREFIX = ['env', 'LC_ALL=C'] def __init__(self, vg_name, root_helper, create_vg=False, physical_volumes=None, lvm_type='default', executor=None, lvm_conf=None, suppress_fd_warn=False): """Initialize the LVM object. The LVM object is based on an LVM VolumeGroup, one instantiation for each VolumeGroup you have/use. :param vg_name: Name of existing VG or VG to create :param root_helper: Execution root_helper method to use :param create_vg: Indicates the VG doesn't exist and we want to create it :param physical_volumes: List of PVs to build VG on :param lvm_type: VG and Volume type (default, or thin) :param executor: Execute method to use, None uses oslo_concurrency.processutils :param suppress_fd_warn: Add suppress FD Warn to LVM env """ super(LVM, self).__init__(execute=executor, root_helper=root_helper) self.vg_name = vg_name self.pv_list = [] self.vg_size = 0.0 self.vg_free_space = 0.0 self.vg_lv_count = 0 self.vg_uuid = None self.vg_thin_pool = None self.vg_thin_pool_size = 0.0 self.vg_thin_pool_free_space = 0.0 self._supports_snapshot_lv_activation = None self._supports_lvchange_ignoreskipactivation = None self.vg_provisioned_capacity = 0.0 # Ensure LVM_SYSTEM_DIR has been added to LVM.LVM_CMD_PREFIX # before the first LVM command is executed, and use the directory # where the specified lvm_conf file is located as the value. # NOTE(jdg): We use the temp var here becuase LVM_CMD_PREFIX is a # class global and if you use append here, you'll literally just keep # appending values to the global. _lvm_cmd_prefix = ['env', 'LC_ALL=C'] if lvm_conf and os.path.isfile(lvm_conf): lvm_sys_dir = os.path.dirname(lvm_conf) _lvm_cmd_prefix.append('LVM_SYSTEM_DIR=' + lvm_sys_dir) if suppress_fd_warn: _lvm_cmd_prefix.append('LVM_SUPPRESS_FD_WARNINGS=1') LVM.LVM_CMD_PREFIX = _lvm_cmd_prefix if create_vg and physical_volumes is not None: try: self._create_vg(physical_volumes) except putils.ProcessExecutionError as err: LOG.exception('Error creating Volume Group') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise exception.VolumeGroupCreationFailed(vg_name=self.vg_name) if self._vg_exists() is False: LOG.error('Unable to locate Volume Group %s', vg_name) raise exception.VolumeGroupNotFound(vg_name=vg_name) # NOTE: we assume that the VG has been activated outside of Cinder if lvm_type == 'thin': pool_name = "%s-pool" % self.vg_name if self.get_volume(pool_name) is None: try: self.create_thin_pool(pool_name) except putils.ProcessExecutionError: # Maybe we just lost the race against another copy of # this driver being in init in parallel - e.g. # cinder-volume and cinder-backup starting in parallel if self.get_volume(pool_name) is None: raise self.vg_thin_pool = pool_name self.activate_lv(self.vg_thin_pool) self.pv_list = self.get_all_physical_volumes(root_helper, vg_name) def _vg_exists(self): """Simple check to see if VG exists. :returns: True if vg specified in object exists, else False """ exists = False cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '-o', 'name', self.vg_name] (out, _err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: volume_groups = out.split() if self.vg_name in volume_groups: exists = True return exists def _create_vg(self, pv_list): cmd = ['vgcreate', self.vg_name, ','.join(pv_list)] self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) def _get_vg_uuid(self): cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '-o', 'uuid', self.vg_name] (out, _err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: return out.split() else: return [] def _get_thin_pool_free_space(self, vg_name, thin_pool_name): """Returns available thin pool free space. :param vg_name: the vg where the pool is placed :param thin_pool_name: the thin pool to gather info for :returns: Free space in GB (float), calculated using data_percent """ cmd = LVM.LVM_CMD_PREFIX + ['lvs', '--noheadings', '--unit=g', '-o', 'size,data_percent', '--separator', ':', '--nosuffix'] # NOTE(gfidente): data_percent only applies to some types of LV so we # make sure to append the actual thin pool name cmd.append("/dev/%s/%s" % (vg_name, thin_pool_name)) free_space = 0.0 try: (out, err) = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out is not None: out = out.strip() data = out.split(':') pool_size = float(data[0]) data_percent = float(data[1]) consumed_space = pool_size / 100 * data_percent free_space = pool_size - consumed_space free_space = round(free_space, 2) # Need noqa due to a false error about the 'err' variable being unused # even though it is used in the logging. Possibly related to # https://github.com/PyCQA/pyflakes/issues/378. except putils.ProcessExecutionError as err: # noqa LOG.exception('Error querying thin pool about data_percent') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) return free_space @staticmethod def get_lvm_version(root_helper): """Static method to get LVM version from system. :param root_helper: root_helper to use for execute :returns: version 3-tuple """ cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--version'] (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) lines = out.split('\n') for line in lines: if 'LVM version' in line: version_list = line.split() # NOTE(gfidente): version is formatted as follows: # major.minor.patchlevel(library API version)[-customisation] version = version_list[2] version_filter = r"(\d+)\.(\d+)\.(\d+).*" r = re.search(version_filter, version) version_tuple = tuple(map(int, r.group(1, 2, 3))) return version_tuple @staticmethod def supports_thin_provisioning(root_helper): """Static method to check for thin LVM support on a system. :param root_helper: root_helper to use for execute :returns: True if supported, False otherwise """ return LVM.get_lvm_version(root_helper) >= (2, 2, 95) @property def supports_snapshot_lv_activation(self): """Property indicating whether snap activation changes are supported. Check for LVM version >= 2.02.91. (LVM2 git: e8a40f6 Allow to activate snapshot) :returns: True/False indicating support """ if self._supports_snapshot_lv_activation is not None: return self._supports_snapshot_lv_activation self._supports_snapshot_lv_activation = ( self.get_lvm_version(self._root_helper) >= (2, 2, 91)) return self._supports_snapshot_lv_activation @property def supports_lvchange_ignoreskipactivation(self): """Property indicating whether lvchange can ignore skip activation. Check for LVM version >= 2.02.99. (LVM2 git: ab789c1bc add --ignoreactivationskip to lvchange) """ if self._supports_lvchange_ignoreskipactivation is not None: return self._supports_lvchange_ignoreskipactivation self._supports_lvchange_ignoreskipactivation = ( self.get_lvm_version(self._root_helper) >= (2, 2, 99)) return self._supports_lvchange_ignoreskipactivation @property def supports_full_pool_create(self): """Property indicating whether 100% pool creation is supported. Check for LVM version >= 2.02.115. Ref: https://bugzilla.redhat.com/show_bug.cgi?id=998347 """ if self.get_lvm_version(self._root_helper) >= (2, 2, 115): return True else: return False @staticmethod def get_lv_info(root_helper, vg_name=None, lv_name=None): """Retrieve info about LVs (all, in a VG, or a single LV). :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :param lv_name: optional, gathers info for only the specified LV :returns: List of Dictionaries with LV info """ cmd = LVM.LVM_CMD_PREFIX + ['lvs', '--noheadings', '--unit=g', '-o', 'vg_name,name,size', '--nosuffix'] if lv_name is not None and vg_name is not None: cmd.append("%s/%s" % (vg_name, lv_name)) elif vg_name is not None: cmd.append(vg_name) try: (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) except putils.ProcessExecutionError as err: with excutils.save_and_reraise_exception(reraise=True) as ctx: if "not found" in err.stderr or "Failed to find" in err.stderr: ctx.reraise = False LOG.info("Logical Volume not found when querying " "LVM info. (vg_name=%(vg)s, lv_name=%(lv)s", {'vg': vg_name, 'lv': lv_name}) out = None lv_list = [] if out is not None: volumes = out.split() iterator = moves.zip(*[iter(volumes)] * 3) # pylint: disable=E1101 for vg, name, size in iterator: lv_list.append({"vg": vg, "name": name, "size": size}) return lv_list def get_volumes(self, lv_name=None): """Get all LV's associated with this instantiation (VG). :returns: List of Dictionaries with LV info """ return self.get_lv_info(self._root_helper, self.vg_name, lv_name) def get_volume(self, name): """Get reference object of volume specified by name. :returns: dict representation of Logical Volume if exists """ ref_list = self.get_volumes(name) for r in ref_list: if r['name'] == name: return r return None @staticmethod def get_all_physical_volumes(root_helper, vg_name=None): """Static method to get all PVs on a system. :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :returns: List of Dictionaries with PV info """ field_sep = '|' cmd = LVM.LVM_CMD_PREFIX + ['pvs', '--noheadings', '--unit=g', '-o', 'vg_name,name,size,free', '--separator', field_sep, '--nosuffix'] (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) pvs = out.split() if vg_name is not None: pvs = [pv for pv in pvs if vg_name == pv.split(field_sep)[0]] pv_list = [] for pv in pvs: fields = pv.split(field_sep) pv_list.append({'vg': fields[0], 'name': fields[1], 'size': float(fields[2]), 'available': float(fields[3])}) return pv_list def get_physical_volumes(self): """Get all PVs associated with this instantiation (VG). :returns: List of Dictionaries with PV info """ self.pv_list = self.get_all_physical_volumes(self._root_helper, self.vg_name) return self.pv_list @staticmethod def get_all_volume_groups(root_helper, vg_name=None): """Static method to get all VGs on a system. :param root_helper: root_helper to use for execute :param vg_name: optional, gathers info for only the specified VG :returns: List of Dictionaries with VG info """ cmd = LVM.LVM_CMD_PREFIX + ['vgs', '--noheadings', '--unit=g', '-o', 'name,size,free,lv_count,uuid', '--separator', ':', '--nosuffix'] if vg_name is not None: cmd.append(vg_name) (out, _err) = priv_rootwrap.execute(*cmd, root_helper=root_helper, run_as_root=True) vg_list = [] if out is not None: vgs = out.split() for vg in vgs: fields = vg.split(':') vg_list.append({'name': fields[0], 'size': float(fields[1]), 'available': float(fields[2]), 'lv_count': int(fields[3]), 'uuid': fields[4]}) return vg_list def update_volume_group_info(self): """Update VG info for this instantiation. Used to update member fields of object and provide a dict of info for caller. :returns: Dictionaries of VG info """ vg_list = self.get_all_volume_groups(self._root_helper, self.vg_name) if len(vg_list) != 1: LOG.error('Unable to find VG: %s', self.vg_name) raise exception.VolumeGroupNotFound(vg_name=self.vg_name) self.vg_size = float(vg_list[0]['size']) self.vg_free_space = float(vg_list[0]['available']) self.vg_lv_count = int(vg_list[0]['lv_count']) self.vg_uuid = vg_list[0]['uuid'] total_vols_size = 0.0 if self.vg_thin_pool is not None: # NOTE(xyang): If providing only self.vg_name, # get_lv_info will output info on the thin pool and all # individual volumes. # get_lv_info(self._root_helper, 'stack-vg') # sudo lvs --noheadings --unit=g -o vg_name,name,size # --nosuffix stack-vg # stack-vg stack-pool 9.51 # stack-vg volume-13380d16-54c3-4979-9d22-172082dbc1a1 1.00 # stack-vg volume-629e13ab-7759-46a5-b155-ee1eb20ca892 1.00 # stack-vg volume-e3e6281c-51ee-464c-b1a7-db6c0854622c 1.00 # # If providing both self.vg_name and self.vg_thin_pool, # get_lv_info will output only info on the thin pool, but not # individual volumes. # get_lv_info(self._root_helper, 'stack-vg', 'stack-pool') # sudo lvs --noheadings --unit=g -o vg_name,name,size # --nosuffix stack-vg/stack-pool # stack-vg stack-pool 9.51 # # We need info on both the thin pool and the volumes, # therefore we should provide only self.vg_name, but not # self.vg_thin_pool here. for lv in self.get_lv_info(self._root_helper, self.vg_name): lvsize = lv['size'] # get_lv_info runs "lvs" command with "--nosuffix". # This removes "g" from "1.00g" and only outputs "1.00". # Running "lvs" command without "--nosuffix" will output # "1.00g" if "g" is the unit. # Remove the unit if it is in lv['size']. if not lv['size'][-1].isdigit(): lvsize = lvsize[:-1] if lv['name'] == self.vg_thin_pool: self.vg_thin_pool_size = float(lvsize) tpfs = self._get_thin_pool_free_space(self.vg_name, self.vg_thin_pool) self.vg_thin_pool_free_space = tpfs else: total_vols_size = total_vols_size + float(lvsize) total_vols_size = round(total_vols_size, 2) self.vg_provisioned_capacity = total_vols_size def _calculate_thin_pool_size(self): """Calculates the correct size for a thin pool. Ideally we would use 100% of the containing volume group and be done. But the 100%VG notation to lvcreate is not implemented and thus cannot be used. See https://bugzilla.redhat.com/show_bug.cgi?id=998347 Further, some amount of free space must remain in the volume group for metadata for the contained logical volumes. The exact amount depends on how much volume sharing you expect. :returns: An lvcreate-ready string for the number of calculated bytes. """ # make sure volume group information is current self.update_volume_group_info() if LVM.supports_full_pool_create: return ["-l", "100%FREE"] # leave 5% free for metadata return ["-L", "%sg" % (self.vg_free_space * 0.95)] def create_thin_pool(self, name=None): """Creates a thin provisioning pool for this VG. The syntax here is slightly different than the default lvcreate -T, so we'll just write a custom cmd here and do it. :param name: Name to use for pool, default is "-pool" :returns: The size string passed to the lvcreate command """ if not LVM.supports_thin_provisioning(self._root_helper): LOG.error('Requested to setup thin provisioning, ' 'however current LVM version does not ' 'support it.') return None if name is None: name = '%s-pool' % self.vg_name vg_pool_name = '%s/%s' % (self.vg_name, name) size_args = self._calculate_thin_pool_size() cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-T'] cmd.extend(size_args) cmd.append(vg_pool_name) LOG.debug("Creating thin pool '%(pool)s' with size %(size)s of " "total %(free)sg", {'pool': vg_pool_name, 'size': size_args, 'free': self.vg_free_space}) self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) self.vg_thin_pool = name return def create_volume(self, name, size_str, lv_type='default', mirror_count=0): """Creates a logical volume on the object's VG. :param name: Name to use when creating Logical Volume :param size_str: Size to use when creating Logical Volume :param lv_type: Type of Volume (default or thin) :param mirror_count: Use LVM mirroring with specified count """ if lv_type == 'thin': pool_path = '%s/%s' % (self.vg_name, self.vg_thin_pool) cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-T', '-V', size_str, '-n', name, pool_path] else: cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '-n', name, self.vg_name, '-L', size_str] if mirror_count > 0: cmd.extend(['-m', mirror_count, '--nosync', '--mirrorlog', 'mirrored']) terras = int(size_str[:-1]) / 1024.0 if terras >= 1.5: rsize = int(2 ** math.ceil(math.log(terras) / math.log(2))) # NOTE(vish): Next power of two for region size. See: # http://red.ht/U2BPOD cmd.extend(['-R', str(rsize)]) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error creating Volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise @utils.retry(putils.ProcessExecutionError) def create_lv_snapshot(self, name, source_lv_name, lv_type='default'): """Creates a snapshot of a logical volume. :param name: Name to assign to new snapshot :param source_lv_name: Name of Logical Volume to snapshot :param lv_type: Type of LV (default or thin) """ source_lvref = self.get_volume(source_lv_name) if source_lvref is None: LOG.error("Trying to create snapshot by non-existent LV: %s", source_lv_name) raise exception.VolumeDeviceNotFound(device=source_lv_name) cmd = LVM.LVM_CMD_PREFIX + ['lvcreate', '--name', name, '-k', 'y', '--snapshot', '%s/%s' % (self.vg_name, source_lv_name)] if lv_type != 'thin': size = source_lvref['size'] cmd.extend(['-L', '%sg' % (size)]) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error creating snapshot') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise def _mangle_lv_name(self, name): # Linux LVM reserves name that starts with snapshot, so that # such volume name can't be created. Mangle it. if not name.startswith('snapshot'): return name return '_' + name def _lv_is_active(self, name): cmd = LVM.LVM_CMD_PREFIX + ['lvdisplay', '--noheading', '-C', '-o', 'Attr', '%s/%s' % (self.vg_name, name)] out, _err = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out: out = out.strip() # An example output might be '-wi-a----'; the 4th index specifies # the status of the volume. 'a' for active, '-' for inactive. if (out[4] == 'a'): return True return False def deactivate_lv(self, name): lv_path = self.vg_name + '/' + self._mangle_lv_name(name) cmd = ['lvchange', '-a', 'n'] cmd.append(lv_path) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error deactivating LV') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise # Wait until lv is deactivated to return in # order to prevent a race condition. self._wait_for_volume_deactivation(name) @utils.retry(exceptions=exception.VolumeNotDeactivated, retries=3, backoff_rate=1) def _wait_for_volume_deactivation(self, name): LOG.debug("Checking to see if volume %s has been deactivated.", name) if self._lv_is_active(name): LOG.debug("Volume %s is still active.", name) raise exception.VolumeNotDeactivated(name=name) else: LOG.debug("Volume %s has been deactivated.", name) def activate_lv(self, name, is_snapshot=False, permanent=False): """Ensure that logical volume/snapshot logical volume is activated. :param name: Name of LV to activate :param is_snapshot: whether LV is a snapshot :param permanent: whether we should drop skipactivation flag :raises: putils.ProcessExecutionError """ # This is a no-op if requested for a snapshot on a version # of LVM that doesn't support snapshot activation. # (Assume snapshot LV is always active.) if is_snapshot and not self.supports_snapshot_lv_activation: return lv_path = self.vg_name + '/' + self._mangle_lv_name(name) # Must pass --yes to activate both the snap LV and its origin LV. # Otherwise lvchange asks if you would like to do this interactively, # and fails. cmd = ['lvchange', '-a', 'y', '--yes'] if self.supports_lvchange_ignoreskipactivation: cmd.append('-K') # If permanent=True is specified, drop the skipactivation flag in # order to make this LV automatically activated after next reboot. if permanent: cmd += ['-k', 'n'] cmd.append(lv_path) try: self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error activating LV') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise @utils.retry(putils.ProcessExecutionError) def delete(self, name): """Delete logical volume or snapshot. :param name: Name of LV to delete """ def run_udevadm_settle(): self._execute('udevadm', 'settle', root_helper=self._root_helper, run_as_root=True, check_exit_code=False) # LV removal seems to be a race with other writers or udev in # some cases (see LP #1270192), so we enable retry deactivation LVM_CONFIG = 'activation { retry_deactivation = 1} ' try: self._execute( 'lvremove', '--config', LVM_CONFIG, '-f', '%s/%s' % (self.vg_name, name), root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.debug('Error reported running lvremove: CMD: %(command)s, ' 'RESPONSE: %(response)s', {'command': err.cmd, 'response': err.stderr}) LOG.debug('Attempting udev settle and retry of lvremove...') run_udevadm_settle() # The previous failing lvremove -f might leave behind # suspended devices; when lvmetad is not available, any # further lvm command will block forever. # Therefore we need to skip suspended devices on retry. LVM_CONFIG += 'devices { ignore_suspended_devices = 1}' self._execute( 'lvremove', '--config', LVM_CONFIG, '-f', '%s/%s' % (self.vg_name, name), root_helper=self._root_helper, run_as_root=True) LOG.debug('Successfully deleted volume: %s after ' 'udev settle.', name) def revert(self, snapshot_name): """Revert an LV from snapshot. :param snapshot_name: Name of snapshot to revert """ self._execute('lvconvert', '--merge', snapshot_name, root_helper=self._root_helper, run_as_root=True) def lv_has_snapshot(self, name): cmd = LVM.LVM_CMD_PREFIX + ['lvdisplay', '--noheading', '-C', '-o', 'Attr', '%s/%s' % (self.vg_name, name)] out, _err = self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) if out: out = out.strip() if (out[0] == 'o') or (out[0] == 'O'): return True return False def extend_volume(self, lv_name, new_size): """Extend the size of an existing volume.""" # Volumes with snaps have attributes 'o' or 'O' and will be # deactivated, but Thin Volumes with snaps have attribute 'V' # and won't be deactivated because the lv_has_snapshot method looks # for 'o' or 'O' if self.lv_has_snapshot(lv_name): self.deactivate_lv(lv_name) try: cmd = LVM.LVM_CMD_PREFIX + ['lvextend', '-L', new_size, '%s/%s' % (self.vg_name, lv_name)] self._execute(*cmd, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error extending Volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise def vg_mirror_free_space(self, mirror_count): free_capacity = 0.0 disks = [] for pv in self.pv_list: disks.append(float(pv['available'])) while True: disks = sorted([a for a in disks if a > 0.0], reverse=True) if len(disks) <= mirror_count: break # consume the smallest disk disk = disks[-1] disks = disks[:-1] # match extents for each mirror on the largest disks for index in list(range(mirror_count)): disks[index] -= disk free_capacity += disk return free_capacity def vg_mirror_size(self, mirror_count): return (self.vg_free_space / (mirror_count + 1)) def rename_volume(self, lv_name, new_name): """Change the name of an existing volume.""" try: self._execute('lvrename', self.vg_name, lv_name, new_name, root_helper=self._root_helper, run_as_root=True) except putils.ProcessExecutionError as err: LOG.exception('Error renaming logical volume') LOG.error('Cmd :%s', err.cmd) LOG.error('StdOut :%s', err.stdout) LOG.error('StdErr :%s', err.stderr) raise ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/privileged/0000775000175000017500000000000000000000000017203 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/privileged/__init__.py0000664000175000017500000000162100000000000021314 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslo_privsep import capabilities as c from oslo_privsep import priv_context # It is expected that most (if not all) os-brick operations can be # executed with these privileges. default = priv_context.PrivContext( __name__, cfg_section='privsep_osbrick', pypath=__name__ + '.default', capabilities=[c.CAP_SYS_ADMIN], ) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/privileged/rootwrap.py0000664000175000017500000002070400000000000021435 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Just in case it wasn't clear, this is a massive security back-door. `execute_root()` (or the same via `execute(run_as_root=True)`) allows any command to be run as the privileged user (default "root"). This is intended only as an expedient transition and should be removed ASAP. This is not completely unreasonable because: 1. We have no tool/workflow for merging changes to rootwrap filter configs from os-brick into nova/cinder, which makes it difficult to evolve these loosely coupled projects. 2. Let's not pretend the earlier situation was any better. The rootwrap filters config contained several entries like "allow cp as root with any arguments", etc, and would have posed only a mild inconvenience to an attacker. At least with privsep we can (in principle) run the "root" commands as a non-root uid, with restricted Linux capabilities. The plan is to switch os-brick to privsep using this module (removing the urgency of (1)), then work on the larger refactor that addresses (2) in followup changes. """ import os import signal import six import threading import time from oslo_concurrency import processutils as putils from oslo_log import log as logging from oslo_utils import strutils from os_brick import exception from os_brick import privileged LOG = logging.getLogger(__name__) def custom_execute(*cmd, **kwargs): """Custom execute with additional functionality on top of Oslo's. Additional features are timeouts and exponential backoff retries. The exponential backoff retries replaces standard Oslo random sleep times that range from 200ms to 2seconds when attempts is greater than 1, but it is disabled if delay_on_retry is passed as a parameter. Exponential backoff is controlled via interval and backoff_rate parameters, just like the os_brick.utils.retry decorator. To use the timeout mechanism to stop the subprocess with a specific signal after a number of seconds we must pass a non-zero timeout value in the call. When using multiple attempts and timeout at the same time the method will only raise the timeout exception to the caller if the last try timeouts. Timeout mechanism is controlled with timeout, signal, and raise_timeout parameters. :param interval: The multiplier :param backoff_rate: Base used for the exponential backoff :param timeout: Timeout defined in seconds :param signal: Signal to use to stop the process on timeout :param raise_timeout: Raise and exception on timeout or return error as stderr. Defaults to raising if check_exit_code is not False. :returns: Tuple with stdout and stderr """ # Since python 2 doesn't have nonlocal we use a mutable variable to store # the previous attempt number, the timeout handler, and the process that # timed out shared_data = [0, None, None] def on_timeout(proc): sanitized_cmd = strutils.mask_password(' '.join(cmd)) LOG.warning('Stopping %(cmd)s with signal %(signal)s after %(time)ss.', {'signal': sig_end, 'cmd': sanitized_cmd, 'time': timeout}) shared_data[2] = proc proc.send_signal(sig_end) def on_execute(proc): # Call user's on_execute method if on_execute_call: on_execute_call(proc) # Sleep if this is not the first try and we have a timeout interval if shared_data[0] and interval: exp = backoff_rate ** shared_data[0] wait_for = max(0, interval * exp) LOG.debug('Sleeping for %s seconds', wait_for) time.sleep(wait_for) # Increase the number of tries and start the timeout timer shared_data[0] += 1 if timeout: shared_data[2] = None shared_data[1] = threading.Timer(timeout, on_timeout, (proc,)) shared_data[1].start() def on_completion(proc): # This is always called regardless of success or failure # Cancel the timeout timer if shared_data[1]: shared_data[1].cancel() # Call user's on_completion method if on_completion_call: on_completion_call(proc) # We will be doing the wait ourselves in on_execute if 'delay_on_retry' in kwargs: interval = None else: kwargs['delay_on_retry'] = False interval = kwargs.pop('interval', 1) backoff_rate = kwargs.pop('backoff_rate', 2) timeout = kwargs.pop('timeout', None) sig_end = kwargs.pop('signal', signal.SIGTERM) default_raise_timeout = kwargs.get('check_exit_code', True) raise_timeout = kwargs.pop('raise_timeout', default_raise_timeout) on_execute_call = kwargs.pop('on_execute', None) on_completion_call = kwargs.pop('on_completion', None) try: return putils.execute(on_execute=on_execute, on_completion=on_completion, *cmd, **kwargs) except putils.ProcessExecutionError: # proc is only stored if a timeout happened proc = shared_data[2] if proc: sanitized_cmd = strutils.mask_password(' '.join(cmd)) msg = ('Time out on proc %(pid)s after waiting %(time)s seconds ' 'when running %(cmd)s' % {'pid': proc.pid, 'time': timeout, 'cmd': sanitized_cmd}) LOG.debug(msg) if raise_timeout: raise exception.ExecutionTimeout(stdout='', stderr=msg, cmd=sanitized_cmd) return '', msg raise # Entrypoint used for rootwrap.py transition code. Don't use this for # other purposes, since it will be removed when we think the # transition is finished. def execute(*cmd, **kwargs): """NB: Raises processutils.ProcessExecutionError on failure.""" run_as_root = kwargs.pop('run_as_root', False) kwargs.pop('root_helper', None) try: if run_as_root: return execute_root(*cmd, **kwargs) else: return custom_execute(*cmd, **kwargs) except OSError as e: # Note: # putils.execute('bogus', run_as_root=True) # raises ProcessExecutionError(exit_code=1) (because there's a # "sh -c bogus" involved in there somewhere, but: # putils.execute('bogus', run_as_root=False) # raises OSError(not found). # # Lots of code in os-brick catches only ProcessExecutionError # and never encountered the latter when using rootwrap. # Rather than fix all the callers, we just always raise # ProcessExecutionError here :( sanitized_cmd = strutils.mask_password(' '.join(cmd)) raise putils.ProcessExecutionError( cmd=sanitized_cmd, description=six.text_type(e)) # See comment on `execute` @privileged.default.entrypoint def execute_root(*cmd, **kwargs): """NB: Raises processutils.ProcessExecutionError/OSError on failure.""" return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) @privileged.default.entrypoint def unlink_root(*links, **kwargs): """Unlink system links with sys admin privileges. By default it will raise an exception if a link does not exist and stop unlinking remaining links. This behavior can be modified passing optional parameters `no_errors` and `raise_at_end`. :param no_errors: Don't raise an exception on error "param raise_at_end: Don't raise an exception on first error, try to unlink all links and then raise a ChainedException with all the errors that where found. """ no_errors = kwargs.get('no_errors', False) raise_at_end = kwargs.get('raise_at_end', False) exc = exception.ExceptionChainer() catch_exception = no_errors or raise_at_end for link in links: with exc.context(catch_exception, 'Unlink failed for %s', link): os.unlink(link) if not no_errors and raise_at_end and exc: raise exc ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/privileged/scaleio.py0000664000175000017500000000560400000000000021201 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from binascii import hexlify import configparser from contextlib import contextmanager from fcntl import ioctl import os import struct import uuid from os_brick import exception from os_brick import privileged SCINI_DEVICE_PATH = '/dev/scini' @contextmanager def open_scini_device(): """Open scini device for low-level I/O using contextmanager. File descriptor will be closed after all operations performed if it was opened successfully. :return: scini device file descriptor :rtype: int """ fd = None try: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) yield fd finally: if fd: os.close(fd) @privileged.default.entrypoint def get_guid(op_code): """Query ScaleIO sdc GUID via ioctl request. :param op_code: operational code :type op_code: int :return: ScaleIO sdc GUID :rtype: str """ with open_scini_device() as fd: out = ioctl(fd, op_code, struct.pack('QQQ', 0, 0, 0)) # The first 8 bytes contain a return code that is not used # so they can be discarded. out_to_hex = hexlify(out[8:]).decode() return str(uuid.UUID(out_to_hex)) @privileged.default.entrypoint def rescan_vols(op_code): """Rescan ScaleIO volumes via ioctl request. :param op_code: operational code :type op_code: int """ with open_scini_device() as fd: ioctl(fd, op_code, struct.pack('Q', 0)) @privileged.default.entrypoint def get_connector_password(filename, config_group, failed_over): """Read ScaleIO connector configuration file and get appropriate password. :param filename: path to connector configuration file :type filename: str :param config_group: name of section in configuration file :type config_group: str :param failed_over: flag representing if storage is in failed over state :type failed_over: bool :return: connector password :rtype: str """ if not os.path.isfile(filename): msg = ( "ScaleIO connector configuration file " "is not found in path %s." % filename ) raise exception.BrickException(message=msg) conf = configparser.ConfigParser() conf.read(filename) password_key = ( "replicating_san_password" if failed_over else "san_password" ) return conf[config_group][password_key] ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/remotefs/0000775000175000017500000000000000000000000016675 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/remotefs/__init__.py0000664000175000017500000000000000000000000020774 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/remotefs/remotefs.py0000664000175000017500000002555100000000000021103 0ustar00zuulzuul00000000000000# Copyright (c) 2013 OpenStack Foundation # All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Remote filesystem client utilities.""" import hashlib import os import re import tempfile from oslo_concurrency import processutils from oslo_log import log as logging import six from os_brick import exception from os_brick import executor from os_brick.i18n import _ LOG = logging.getLogger(__name__) class RemoteFsClient(executor.Executor): def __init__(self, mount_type, root_helper, execute=None, *args, **kwargs): super(RemoteFsClient, self).__init__(root_helper, execute=execute, *args, **kwargs) mount_type_to_option_prefix = { 'nfs': 'nfs', 'cifs': 'smbfs', 'glusterfs': 'glusterfs', 'vzstorage': 'vzstorage', 'quobyte': 'quobyte', 'scality': 'scality' } if mount_type not in mount_type_to_option_prefix: raise exception.ProtocolNotSupported(protocol=mount_type) self._mount_type = mount_type option_prefix = mount_type_to_option_prefix[mount_type] self._mount_base = kwargs.get(option_prefix + '_mount_point_base') if not self._mount_base: raise exception.InvalidParameterValue( err=_('%s_mount_point_base required') % option_prefix) self._mount_options = kwargs.get(option_prefix + '_mount_options') if mount_type == "nfs": self._check_nfs_options() def get_mount_base(self): return self._mount_base def _get_hash_str(self, base_str): """Return a string that represents hash of base_str (hex format).""" if isinstance(base_str, six.text_type): base_str = base_str.encode('utf-8') return hashlib.md5(base_str).hexdigest() def get_mount_point(self, device_name): """Get Mount Point. :param device_name: example 172.18.194.100:/var/nfs """ return os.path.join(self._mount_base, self._get_hash_str(device_name)) def _read_mounts(self): """Returns a dict of mounts and their mountpoint Format reference: http://man7.org/linux/man-pages/man5/fstab.5.html """ with open("/proc/mounts", "r") as mounts: # Remove empty lines and split lines by whitespace lines = [l.split() for l in mounts.read().splitlines() if l.strip()] # Return {mountpoint: mountdevice}. Fields 2nd and 1st as per # http://man7.org/linux/man-pages/man5/fstab.5.html return {line[1]: line[0] for line in lines if line[0] != '#'} def mount(self, share, flags=None): """Mount given share.""" mount_path = self.get_mount_point(share) if mount_path in self._read_mounts(): LOG.debug('Already mounted: %s', mount_path) return self._execute('mkdir', '-p', mount_path, check_exit_code=0) if self._mount_type == 'nfs': self._mount_nfs(share, mount_path, flags) else: self._do_mount(self._mount_type, share, mount_path, self._mount_options, flags) def _do_mount(self, mount_type, share, mount_path, mount_options=None, flags=None): """Mounts share based on the specified params.""" mnt_cmd = ['mount', '-t', mount_type] if mount_options is not None: mnt_cmd.extend(['-o', mount_options]) if flags is not None: mnt_cmd.extend(flags) mnt_cmd.extend([share, mount_path]) try: self._execute(*mnt_cmd, root_helper=self._root_helper, run_as_root=True, check_exit_code=0) except processutils.ProcessExecutionError as exc: if 'already mounted' in exc.stderr: LOG.debug("Already mounted: %s", share) # The error message can say "busy or already mounted" when the # share didn't actually mount, so look for it. if share in self._read_mounts(): return LOG.error("Failed to mount %(share)s, reason: %(reason)s", {'share': share, 'reason': exc.stderr}) raise def _mount_nfs(self, nfs_share, mount_path, flags=None): """Mount nfs share using present mount types.""" mnt_errors = {} # This loop allows us to first try to mount with NFS 4.1 for pNFS # support but falls back to mount NFS 4 or NFS 3 if either the client # or server do not support it. for mnt_type in sorted(self._nfs_mount_type_opts.keys(), reverse=True): options = self._nfs_mount_type_opts[mnt_type] try: self._do_mount('nfs', nfs_share, mount_path, options, flags) LOG.debug('Mounted %(sh)s using %(mnt_type)s.', {'sh': nfs_share, 'mnt_type': mnt_type}) return except Exception as e: mnt_errors[mnt_type] = six.text_type(e) LOG.debug('Failed to do %s mount.', mnt_type) raise exception.BrickException(_("NFS mount failed for share %(sh)s. " "Error - %(error)s") % {'sh': nfs_share, 'error': mnt_errors}) def _check_nfs_options(self): """Checks and prepares nfs mount type options.""" self._nfs_mount_type_opts = {'nfs': self._mount_options} nfs_vers_opt_patterns = ['^nfsvers', '^vers', r'^v[\d]'] for opt in nfs_vers_opt_patterns: if self._option_exists(self._mount_options, opt): return # pNFS requires NFS 4.1. The mount.nfs4 utility does not automatically # negotiate 4.1 support, we have to ask for it by specifying two # options: vers=4 and minorversion=1. pnfs_opts = self._update_option(self._mount_options, 'vers', '4') pnfs_opts = self._update_option(pnfs_opts, 'minorversion', '1') self._nfs_mount_type_opts['pnfs'] = pnfs_opts def _option_exists(self, options, opt_pattern): """Checks if the option exists in nfs options and returns position.""" options = [x.strip() for x in options.split(',')] if options else [] pos = 0 for opt in options: pos = pos + 1 if re.match(opt_pattern, opt, flags=0): return pos return 0 def _update_option(self, options, option, value=None): """Update option if exists else adds it and returns new options.""" opts = [x.strip() for x in options.split(',')] if options else [] pos = self._option_exists(options, option) if pos: opts.pop(pos - 1) opt = '%s=%s' % (option, value) if value else option opts.append(opt) return ",".join(opts) if len(opts) > 1 else opts[0] class ScalityRemoteFsClient(RemoteFsClient): def __init__(self, mount_type, root_helper, execute=None, *args, **kwargs): super(ScalityRemoteFsClient, self).__init__(mount_type, root_helper, execute=execute, *args, **kwargs) self._mount_type = mount_type self._mount_base = kwargs.get( 'scality_mount_point_base', "").rstrip('/') if not self._mount_base: raise exception.InvalidParameterValue( err=_('scality_mount_point_base required')) self._mount_options = None def get_mount_point(self, device_name): return os.path.join(self._mount_base, device_name, "00") def mount(self, share, flags=None): """Mount the Scality ScaleOut FS. The `share` argument is ignored because you can't mount several SOFS at the same type on a single server. But we want to keep the same method signature for class inheritance purpose. """ if self._mount_base in self._read_mounts(): LOG.debug('Already mounted: %s', self._mount_base) return self._execute('mkdir', '-p', self._mount_base, check_exit_code=0) super(ScalityRemoteFsClient, self)._do_mount( 'sofs', '/etc/sfused.conf', self._mount_base) class VZStorageRemoteFSClient(RemoteFsClient): def _vzstorage_write_mds_list(self, cluster_name, mdss): tmp_dir = tempfile.mkdtemp(prefix='vzstorage-') tmp_bs_path = os.path.join(tmp_dir, 'bs_list') with open(tmp_bs_path, 'w') as f: for mds in mdss: f.write(mds + "\n") conf_dir = os.path.join('/etc/pstorage/clusters', cluster_name) if os.path.exists(conf_dir): bs_path = os.path.join(conf_dir, 'bs_list') self._execute('cp', '-f', tmp_bs_path, bs_path, root_helper=self._root_helper, run_as_root=True) else: self._execute('cp', '-rf', tmp_dir, conf_dir, root_helper=self._root_helper, run_as_root=True) self._execute('chown', '-R', 'root:root', conf_dir, root_helper=self._root_helper, run_as_root=True) def _do_mount(self, mount_type, vz_share, mount_path, mount_options=None, flags=None): m = re.search(r"(?:(\S+):\/)?([a-zA-Z0-9_-]+)(?::(\S+))?", vz_share) if not m: msg = (_("Invalid Virtuozzo Storage share specification: %r." "Must be: [MDS1[,MDS2],...:/][:PASSWORD].") % vz_share) raise exception.BrickException(msg) mdss = m.group(1) cluster_name = m.group(2) passwd = m.group(3) if mdss: mdss = mdss.split(',') self._vzstorage_write_mds_list(cluster_name, mdss) if passwd: self._execute('pstorage', '-c', cluster_name, 'auth-node', '-P', process_input=passwd, root_helper=self._root_helper, run_as_root=True) mnt_cmd = ['pstorage-mount', '-c', cluster_name] if flags: mnt_cmd.extend(flags) mnt_cmd.extend([mount_path]) self._execute(*mnt_cmd, root_helper=self._root_helper, run_as_root=True, check_exit_code=0) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/remotefs/windows_remotefs.py0000664000175000017500000001261200000000000022647 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Windows remote filesystem client utilities.""" import os import re from oslo_log import log as logging from os_win import utilsfactory from os_brick import exception from os_brick.i18n import _ from os_brick.remotefs import remotefs LOG = logging.getLogger(__name__) class WindowsRemoteFsClient(remotefs.RemoteFsClient): _username_regex = re.compile(r'user(?:name)?=([^, ]+)') _password_regex = re.compile(r'pass(?:word)?=([^, ]+)') _loopback_share_map = {} def __init__(self, mount_type, root_helper=None, execute=None, *args, **kwargs): mount_type_to_option_prefix = { 'cifs': 'smbfs', 'smbfs': 'smbfs', } self._local_path_for_loopback = kwargs.get('local_path_for_loopback', True) if mount_type not in mount_type_to_option_prefix: raise exception.ProtocolNotSupported(protocol=mount_type) self._mount_type = mount_type option_prefix = mount_type_to_option_prefix[mount_type] self._mount_base = kwargs.get(option_prefix + '_mount_point_base') self._mount_options = kwargs.get(option_prefix + '_mount_options') self._smbutils = utilsfactory.get_smbutils() self._pathutils = utilsfactory.get_pathutils() def get_local_share_path(self, share, expect_existing=True): share = self._get_share_norm_path(share) share_name = self.get_share_name(share) share_subdir = self.get_share_subdir(share) is_local_share = self._smbutils.is_local_share(share) if not is_local_share: LOG.debug("Share '%s' is not exposed by the current host.", share) local_share_path = None else: local_share_path = self._smbutils.get_smb_share_path(share_name) if not local_share_path and expect_existing: err_msg = _("Could not find the local " "share path for %(share)s.") raise exception.VolumePathsNotFound(err_msg % dict(share=share)) if local_share_path and share_subdir: local_share_path = os.path.join(local_share_path, share_subdir) return local_share_path def _get_share_norm_path(self, share): return share.replace('/', '\\') def get_share_name(self, share): return self._get_share_norm_path(share).lstrip('\\').split('\\')[1] def get_share_subdir(self, share): return "\\".join( self._get_share_norm_path(share).lstrip('\\').split('\\')[2:]) def mount(self, share, flags=None): share_norm_path = self._get_share_norm_path(share) use_local_path = (self._local_path_for_loopback and self._smbutils.is_local_share(share_norm_path)) if use_local_path: LOG.info("Skipping mounting local share %(share_path)s.", dict(share_path=share_norm_path)) else: mount_options = " ".join( [self._mount_options or '', flags or '']) username, password = self._parse_credentials(mount_options) if not self._smbutils.check_smb_mapping( share_norm_path): self._smbutils.mount_smb_share(share_norm_path, username=username, password=password) if self._mount_base: self._create_mount_point(share, use_local_path) def unmount(self, share): self._smbutils.unmount_smb_share(self._get_share_norm_path(share)) def _create_mount_point(self, share, use_local_path): # The mount point will contain a hash of the share so we're # intentionally preserving the original share path as this is # what the caller will expect. mnt_point = self.get_mount_point(share) share_norm_path = self._get_share_norm_path(share) symlink_dest = (share_norm_path if not use_local_path else self.get_local_share_path(share)) if not os.path.isdir(self._mount_base): os.makedirs(self._mount_base) if os.path.exists(mnt_point): if not self._pathutils.is_symlink(mnt_point): raise exception.BrickException(_("Link path already exists " "and it's not a symlink")) else: self._pathutils.create_sym_link(mnt_point, symlink_dest) def _parse_credentials(self, opts_str): if not opts_str: return None, None match = self._username_regex.findall(opts_str) username = match[0] if match and match[0] != 'guest' else None match = self._password_regex.findall(opts_str) password = match[0] if match else None return username, password ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/tests/0000775000175000017500000000000000000000000016213 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/__init__.py0000664000175000017500000000000000000000000020312 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/base.py0000664000175000017500000001042000000000000017474 0ustar00zuulzuul00000000000000# Copyright 2010-2011 OpenStack Foundation # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import logging import os import testtools import fixtures import mock from oslo_concurrency import lockutils from oslo_config import fixture as config_fixture from oslo_utils import strutils class TestCase(testtools.TestCase): """Test case base class for all unit tests.""" SENTINEL = object() def setUp(self): """Run before each test method to initialize test environment.""" super(TestCase, self).setUp() test_timeout = os.environ.get('OS_TEST_TIMEOUT', 0) try: test_timeout = int(test_timeout) except ValueError: # If timeout value is invalid do not set a timeout. test_timeout = 0 if test_timeout > 0: self.useFixture(fixtures.Timeout(test_timeout, gentle=True)) self.useFixture(fixtures.NestedTempfile()) self.useFixture(fixtures.TempHomeDir()) environ_enabled = (lambda var_name: strutils.bool_from_string(os.environ.get(var_name))) if environ_enabled('OS_STDOUT_CAPTURE'): stdout = self.useFixture(fixtures.StringStream('stdout')).stream self.useFixture(fixtures.MonkeyPatch('sys.stdout', stdout)) if environ_enabled('OS_STDERR_CAPTURE'): stderr = self.useFixture(fixtures.StringStream('stderr')).stream self.useFixture(fixtures.MonkeyPatch('sys.stderr', stderr)) if environ_enabled('OS_LOG_CAPTURE'): log_format = '%(levelname)s [%(name)s] %(message)s' if environ_enabled('OS_DEBUG'): level = logging.DEBUG else: level = logging.INFO self.useFixture(fixtures.LoggerFixture(nuke_handlers=False, format=log_format, level=level)) # At runtime this would be set by the library user: Cinder, Nova, etc. self.useFixture(fixtures.NestedTempfile()) lock_path = self.useFixture(fixtures.TempDir()).path self.fixture = self.useFixture(config_fixture.Config(lockutils.CONF)) self.fixture.config(lock_path=lock_path, group='oslo_concurrency') lockutils.set_defaults(lock_path) def _common_cleanup(self): """Runs after each test method to tear down test environment.""" # Stop any timers for x in self.injected: try: x.stop() except AssertionError: pass # Delete attributes that don't start with _ so they don't pin # memory around unnecessarily for the duration of the test # suite for key in [k for k in self.__dict__.keys() if k[0] != '_']: del self.__dict__[key] def log_level(self, level): """Set logging level to the specified value.""" log_root = logging.getLogger(None).logger log_root.setLevel(level) def mock_object(self, obj, attr_name, new_attr=SENTINEL, **kwargs): """Use python mock to mock an object attribute Mocks the specified objects attribute with the given value. Automatically performs 'addCleanup' for the mock. """ args = [obj, attr_name] if new_attr is not self.SENTINEL: args.append(new_attr) patcher = mock.patch.object(*args, **kwargs) mocked = patcher.start() self.addCleanup(patcher.stop) return mocked def patch(self, path, *args, **kwargs): """Use python mock to mock a path with automatic cleanup.""" patcher = mock.patch(path, *args, **kwargs) result = patcher.start() self.addCleanup(patcher.stop) return result ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/tests/encryptors/0000775000175000017500000000000000000000000020423 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/encryptors/__init__.py0000664000175000017500000000000000000000000022522 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/encryptors/test_base.py0000664000175000017500000001754700000000000022764 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from castellan.tests.unit.key_manager import fake import mock from os_brick import encryptors from os_brick.tests import base class VolumeEncryptorTestCase(base.TestCase): def _create(self): pass def setUp(self): super(VolumeEncryptorTestCase, self).setUp() self.connection_info = { "data": { "device_path": "/dev/disk/by-path/" "ip-192.0.2.0:3260-iscsi-iqn.2010-10.org.openstack" ":volume-fake_uuid-lun-1", }, } self.root_helper = None self.keymgr = fake.fake_api() self.encryptor = self._create() class BaseEncryptorTestCase(VolumeEncryptorTestCase): def _test_get_encryptor(self, provider, expected_provider_class): encryption = {'control_location': 'front-end', 'provider': provider} encryptor = encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) self.assertIsInstance(encryptor, expected_provider_class) def test_get_encryptors(self): self._test_get_encryptor('luks', encryptors.luks.LuksEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('os_brick.encryptors.luks.LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('nova.volume.encryptors.luks.LuksEncryptor', encryptors.luks.LuksEncryptor) self._test_get_encryptor('plain', encryptors.cryptsetup.CryptsetupEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor( 'os_brick.encryptors.cryptsetup.CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor( 'nova.volume.encryptors.cryptsetup.CryptsetupEncryptor', encryptors.cryptsetup.CryptsetupEncryptor) self._test_get_encryptor(None, encryptors.nop.NoOpEncryptor) # TODO(lyarwood): Remove the following in Pike self._test_get_encryptor('NoOpEncryptor', encryptors.nop.NoOpEncryptor) self._test_get_encryptor('os_brick.encryptors.nop.NoOpEncryptor', encryptors.nop.NoOpEncryptor) self._test_get_encryptor('nova.volume.encryptors.nop.NoopEncryptor', encryptors.nop.NoOpEncryptor) def test_get_error_encryptors(self): encryption = {'control_location': 'front-end', 'provider': 'ErrorEncryptor'} self.assertRaises(ValueError, encryptors.get_volume_encryptor, root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) @mock.patch('os_brick.encryptors.LOG') def test_error_log(self, log): encryption = {'control_location': 'front-end', 'provider': 'TestEncryptor'} provider = 'TestEncryptor' try: encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) except Exception as e: log.error.assert_called_once_with("Error instantiating " "%(provider)s: " "%(exception)s", {'provider': provider, 'exception': e}) @mock.patch('os_brick.encryptors.LOG') def test_get_missing_out_of_tree_encryptor_log(self, log): provider = 'TestEncryptor' encryption = {'control_location': 'front-end', 'provider': provider} try: encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) except Exception as e: log.error.assert_called_once_with("Error instantiating " "%(provider)s: " "%(exception)s", {'provider': provider, 'exception': e}) log.warning.assert_called_once_with("Use of the out of tree " "encryptor class %(provider)s " "will be blocked with the " "Queens release of os-brick.", {'provider': provider}) @mock.patch('os_brick.encryptors.LOG') def test_get_direct_encryptor_log(self, log): encryption = {'control_location': 'front-end', 'provider': 'LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) encryption = {'control_location': 'front-end', 'provider': 'os_brick.encryptors.luks.LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) encryption = {'control_location': 'front-end', 'provider': 'nova.volume.encryptors.luks.LuksEncryptor'} encryptors.get_volume_encryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr, **encryption) log.warning.assert_has_calls([ mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'LuksEncryptor'}), mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'os_brick.encryptors.luks.LuksEncryptor'}), mock.call("Use of the in tree encryptor class %(provider)s by " "directly referencing the implementation class will be " "blocked in the Queens release of os-brick.", {'provider': 'nova.volume.encryptors.luks.LuksEncryptor'})]) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/encryptors/test_cryptsetup.py0000664000175000017500000001770600000000000024271 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import binascii import copy import mock import six from castellan.common.objects import symmetric_key as key from castellan.tests.unit.key_manager import fake from os_brick.encryptors import cryptsetup from os_brick import exception from os_brick.tests.encryptors import test_base from oslo_concurrency import processutils as putils def fake__get_key(context, passphrase): raw = bytes(binascii.unhexlify(passphrase)) symmetric_key = key.SymmetricKey('AES', len(raw) * 8, raw) return symmetric_key class CryptsetupEncryptorTestCase(test_base.VolumeEncryptorTestCase): @mock.patch('os.path.exists', return_value=False) def _create(self, mock_exists): return cryptsetup.CryptsetupEncryptor( connection_info=self.connection_info, root_helper=self.root_helper, keymgr=self.keymgr) def setUp(self): super(CryptsetupEncryptorTestCase, self).setUp() self.dev_path = self.connection_info['data']['device_path'] self.dev_name = 'crypt-%s' % self.dev_path.split('/')[-1] self.symlink_path = self.dev_path @mock.patch('os_brick.executor.Executor._execute') def test__open_volume(self, mock_execute): self.encryptor._open_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input='passphrase', run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume(self, mock_execute): fake_key = 'e8b76872e3b04c18b3b6656bbf6f5089' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = fake__get_key(None, fake_key) self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test__close_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'remove', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4]), ]) @mock.patch('os_brick.executor.Executor._execute') def test_detach_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'remove', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4]), ]) def test_init_volume_encryption_not_supported(self): # Tests that creating a CryptsetupEncryptor fails if there is no # device_path key. type = 'unencryptable' data = dict(volume_id='a194699b-aa07-4433-a945-a5d23802043e') connection_info = dict(driver_volume_type=type, data=data) exc = self.assertRaises(exception.VolumeEncryptionNotSupported, cryptsetup.CryptsetupEncryptor, root_helper=self.root_helper, connection_info=connection_info, keymgr=fake.fake_api()) self.assertIn(type, six.text_type(exc)) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os.path.exists', return_value=True) def test_init_volume_encryption_with_old_name(self, mock_exists, mock_execute): # If an old name crypt device exists, dev_path should be the old name. old_dev_name = self.dev_path.split('/')[-1] encryptor = cryptsetup.CryptsetupEncryptor( root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) self.assertFalse(encryptor.dev_name.startswith('crypt-')) self.assertEqual(old_dev_name, encryptor.dev_name) self.assertEqual(self.dev_path, encryptor.dev_path) self.assertEqual(self.symlink_path, encryptor.symlink_path) mock_exists.assert_called_once_with('/dev/mapper/%s' % old_dev_name) mock_execute.assert_called_once_with( 'cryptsetup', 'status', old_dev_name, run_as_root=True) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os.path.exists', side_effect=[False, True]) def test_init_volume_encryption_with_wwn(self, mock_exists, mock_execute): # If an wwn name crypt device exists, dev_path should be based on wwn. old_dev_name = self.dev_path.split('/')[-1] wwn = 'fake_wwn' connection_info = copy.deepcopy(self.connection_info) connection_info['data']['multipath_id'] = wwn encryptor = cryptsetup.CryptsetupEncryptor( root_helper=self.root_helper, connection_info=connection_info, keymgr=fake.fake_api(), execute=mock_execute) self.assertFalse(encryptor.dev_name.startswith('crypt-')) self.assertEqual(wwn, encryptor.dev_name) self.assertEqual(self.dev_path, encryptor.dev_path) self.assertEqual(self.symlink_path, encryptor.symlink_path) mock_exists.assert_has_calls([ mock.call('/dev/mapper/%s' % old_dev_name), mock.call('/dev/mapper/%s' % wwn)]) mock_execute.assert_called_once_with( 'cryptsetup', 'status', wwn, run_as_root=True) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_unmangle_passphrase(self, mock_execute): fake_key = '0725230b' fake_key_mangled = '72523b' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = fake__get_key(None, fake_key) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=2), # luksOpen mock.DEFAULT, mock.DEFAULT, ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'create', '--key-file=-', self.dev_name, self.dev_path, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) self.assertEqual(3, mock_execute.call_count) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/encryptors/test_luks.py0000664000175000017500000003447300000000000023025 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import binascii import mock from castellan.common.objects import symmetric_key as key from os_brick.encryptors import luks from os_brick.tests.encryptors import test_cryptsetup from oslo_concurrency import processutils as putils class LuksEncryptorTestCase(test_cryptsetup.CryptsetupEncryptorTestCase): def _create(self): return luks.LuksEncryptor(root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) @mock.patch('os_brick.executor.Executor._execute') def test_is_luks(self, mock_execute): luks.is_luks(self.root_helper, self.dev_path, execute=mock_execute) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') @mock.patch('os_brick.encryptors.luks.LOG') def test_is_luks_with_error(self, mock_log, mock_execute): error_msg = "Device %s is not a valid LUKS device." % self.dev_path mock_execute.side_effect = putils.ProcessExecutionError( exit_code=1, stderr=error_msg) luks.is_luks(self.root_helper, self.dev_path, execute=mock_execute) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, run_as_root=True, root_helper=self.root_helper, check_exit_code=True), ]) self.assertEqual(1, mock_log.warning.call_count) # warning logged @mock.patch('os_brick.executor.Executor._execute') def test__format_volume(self, mock_execute): self.encryptor._format_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--type', 'luks1', '--key-file=-', self.dev_path, process_input='passphrase', root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), ]) @mock.patch('os_brick.executor.Executor._execute') def test__open_volume(self, mock_execute): self.encryptor._open_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input='passphrase', root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume(self, mock_execute): fake_key = '0c84146034e747639b698368807286df' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_not_formatted(self, mock_execute): fake_key = 'bc37c5eccebe403f9cc2d0dd20dac2bc' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=1), # luksOpen putils.ProcessExecutionError(exit_code=1), # isLuks mock.DEFAULT, # luksFormat mock.DEFAULT, # luksOpen mock.DEFAULT, # ln ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--type', 'luks1', '--key-file=-', self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_fail(self, mock_execute): fake_key = 'ea6c2e1b8f7f4f84ae3560116d659ba2' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=1), # luksOpen mock.DEFAULT, # isLuks ] self.assertRaises(putils.ProcessExecutionError, self.encryptor.attach_volume, None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) @mock.patch('os_brick.executor.Executor._execute') def test__close_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, attempts=3, run_as_root=True, check_exit_code=[0, 4]), ]) @mock.patch('os_brick.executor.Executor._execute') def test_detach_volume(self, mock_execute): self.encryptor.detach_volume() mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, attempts=3, run_as_root=True, check_exit_code=[0, 4]), ]) def test_get_mangled_passphrase(self): # Confirm that a mangled passphrase is provided as per bug#1633518 unmangled_raw_key = bytes(binascii.unhexlify('0725230b')) symmetric_key = key.SymmetricKey('AES', len(unmangled_raw_key) * 8, unmangled_raw_key) unmangled_encoded_key = symmetric_key.get_encoded() self.assertEqual(self.encryptor._get_mangled_passphrase( unmangled_encoded_key), '72523b') @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_unmangle_passphrase(self, mock_execute): fake_key = '0725230b' fake_key_mangled = '72523b' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = \ test_cryptsetup.fake__get_key(None, fake_key) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=2), # luksOpen mock.DEFAULT, # luksOpen mock.DEFAULT, # luksClose mock.DEFAULT, # luksAddKey mock.DEFAULT, # luksOpen mock.DEFAULT, # luksClose mock.DEFAULT, # luksRemoveKey mock.DEFAULT, # luksOpen mock.DEFAULT, # ln ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4], attempts=3), mock.call('cryptsetup', 'luksAddKey', self.dev_path, '--force-password', process_input=''.join([fake_key_mangled, '\n', fake_key, '\n', fake_key]), root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksClose', self.dev_name, root_helper=self.root_helper, run_as_root=True, check_exit_code=[0, 4], attempts=3), mock.call('cryptsetup', 'luksRemoveKey', self.dev_path, process_input=fake_key_mangled, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) self.assertEqual(9, mock_execute.call_count) class Luks2EncryptorTestCase(LuksEncryptorTestCase): def _create(self): return luks.Luks2Encryptor(root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) @mock.patch('os_brick.executor.Executor._execute') def test__format_volume(self, mock_execute): self.encryptor._format_volume("passphrase") mock_execute.assert_has_calls([ mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--type', 'luks2', '--key-file=-', self.dev_path, process_input='passphrase', root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), ]) @mock.patch('os_brick.executor.Executor._execute') def test_attach_volume_not_formatted(self, mock_execute): fake_key = 'bc37c5eccebe403f9cc2d0dd20dac2bc' self.encryptor._get_key = mock.MagicMock() self.encryptor._get_key.return_value = ( test_cryptsetup.fake__get_key(None, fake_key)) mock_execute.side_effect = [ putils.ProcessExecutionError(exit_code=1), # luksOpen putils.ProcessExecutionError(exit_code=1), # isLuks mock.DEFAULT, # luksFormat mock.DEFAULT, # luksOpen mock.DEFAULT, # ln ] self.encryptor.attach_volume(None) mock_execute.assert_has_calls([ mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', 'isLuks', '--verbose', self.dev_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('cryptsetup', '--batch-mode', 'luksFormat', '--type', 'luks2', '--key-file=-', self.dev_path, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True, attempts=3), mock.call('cryptsetup', 'luksOpen', '--key-file=-', self.dev_path, self.dev_name, process_input=fake_key, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), mock.call('ln', '--symbolic', '--force', '/dev/mapper/%s' % self.dev_name, self.symlink_path, root_helper=self.root_helper, run_as_root=True, check_exit_code=True), ], any_order=False) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/encryptors/test_nop.py0000664000175000017500000000264700000000000022641 0ustar00zuulzuul00000000000000# Copyright (c) 2013 The Johns Hopkins University/Applied Physics Laboratory # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.encryptors import nop from os_brick.tests.encryptors import test_base class NoOpEncryptorTestCase(test_base.VolumeEncryptorTestCase): def _create(self): return nop.NoOpEncryptor(root_helper=self.root_helper, connection_info=self.connection_info, keymgr=self.keymgr) def test_attach_volume(self): test_args = { 'control_location': 'front-end', 'provider': 'NoOpEncryptor', } self.encryptor.attach_volume(None, **test_args) def test_detach_volume(self): test_args = { 'control_location': 'front-end', 'provider': 'NoOpEncryptor', } self.encryptor.detach_volume(**test_args) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1486776 os-brick-3.0.8/os_brick/tests/initiator/0000775000175000017500000000000000000000000020215 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/__init__.py0000664000175000017500000000000000000000000022314 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1526778 os-brick-3.0.8/os_brick/tests/initiator/connectors/0000775000175000017500000000000000000000000022372 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/__init__.py0000664000175000017500000000000000000000000024471 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_aoe.py0000664000175000017500000001020700000000000024547 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os from os_brick import exception from os_brick.initiator.connectors import aoe from os_brick.tests.initiator import test_connector class AoEConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for AoE initiator class.""" def setUp(self): super(AoEConnectorTestCase, self).setUp() self.connector = aoe.AoEConnector('sudo') self.connection_properties = {'target_shelf': 'fake_shelf', 'target_lun': 'fake_lun'} def test_get_search_path(self): expected = "/dev/etherd" actual_path = self.connector.get_search_path() self.assertEqual(expected, actual_path) @mock.patch.object(os.path, 'exists', return_value=True) def test_get_volume_paths(self, mock_exists): expected = ["/dev/etherd/efake_shelf.fake_lun"] paths = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, paths) def test_get_connector_properties(self): props = aoe.AoEConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) @mock.patch.object(os.path, 'exists', side_effect=[True, True]) def test_connect_volume(self, exists_mock): """Ensure that if path exist aoe-revalidate was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.connector.connect_volume(self.connection_properties) @mock.patch.object(os.path, 'exists', side_effect=[False, True]) def test_connect_volume_without_path(self, exists_mock): """Ensure that if path doesn't exist aoe-discovery was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) expected_info = { 'type': 'block', 'device': aoe_device, 'path': aoe_path, } with mock.patch.object(self.connector, '_execute', return_value=["", ""]): volume_info = self.connector.connect_volume( self.connection_properties) self.assertDictEqual(volume_info, expected_info) @mock.patch.object(os.path, 'exists', return_value=False) def test_connect_volume_could_not_discover_path(self, exists_mock): _aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.assertRaises(exception.VolumeDeviceNotFound, self.connector.connect_volume, self.connection_properties) @mock.patch.object(os.path, 'exists', return_value=True) def test_disconnect_volume(self, mock_exists): """Ensure that if path exist aoe-revaliadte was called.""" aoe_device, aoe_path = self.connector._get_aoe_info( self.connection_properties) with mock.patch.object(self.connector, '_execute', return_value=["", ""]): self.connector.disconnect_volume(self.connection_properties, {}) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_base_iscsi.py0000664000175000017500000000760200000000000026114 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import base_iscsi from os_brick.initiator.connectors import fake from os_brick.tests import base as test_base class BaseISCSIConnectorTestCase(test_base.TestCase): def setUp(self): super(BaseISCSIConnectorTestCase, self).setUp() self.connector = fake.FakeBaseISCSIConnector(None) @mock.patch.object(base_iscsi.BaseISCSIConnector, '_get_all_targets') def test_iterate_all_targets(self, mock_get_all_targets): # extra_property cannot be a sentinel, a copied sentinel will not # identical to the original one. connection_properties = { 'target_portals': mock.sentinel.target_portals, 'target_iqns': mock.sentinel.target_iqns, 'target_luns': mock.sentinel.target_luns, 'extra_property': 'extra_property'} mock_get_all_targets.return_value = [( mock.sentinel.portal, mock.sentinel.iqn, mock.sentinel.lun)] # method is a generator, and it yields dictionaries. list() will # iterate over all of the method's items. list_props = list( self.connector._iterate_all_targets(connection_properties)) mock_get_all_targets.assert_called_once_with(connection_properties) self.assertEqual(1, len(list_props)) expected_props = {'target_portal': mock.sentinel.portal, 'target_iqn': mock.sentinel.iqn, 'target_lun': mock.sentinel.lun, 'extra_property': 'extra_property'} self.assertEqual(expected_props, list_props[0]) def test_get_all_targets(self): portals = [mock.sentinel.portals1, mock.sentinel.portals2] iqns = [mock.sentinel.iqns1, mock.sentinel.iqns2] luns = [mock.sentinel.luns1, mock.sentinel.luns2] connection_properties = {'target_portals': portals, 'target_iqns': iqns, 'target_luns': luns} all_targets = self.connector._get_all_targets(connection_properties) expected_targets = zip(portals, iqns, luns) self.assertEqual(list(expected_targets), list(all_targets)) def test_get_all_targets_no_target_luns(self): portals = [mock.sentinel.portals1, mock.sentinel.portals2] iqns = [mock.sentinel.iqns1, mock.sentinel.iqns2] lun = mock.sentinel.luns connection_properties = {'target_portals': portals, 'target_iqns': iqns, 'target_lun': lun} all_targets = self.connector._get_all_targets(connection_properties) expected_targets = zip(portals, iqns, [lun, lun]) self.assertEqual(list(expected_targets), list(all_targets)) def test_get_all_targets_single_target(self): connection_properties = { 'target_portal': mock.sentinel.target_portal, 'target_iqn': mock.sentinel.target_iqn, 'target_lun': mock.sentinel.target_lun} all_targets = self.connector._get_all_targets(connection_properties) expected_target = (mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun) self.assertEqual([expected_target], all_targets) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_disco.py0000664000175000017500000001266600000000000025117 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import glob import os from os_brick import exception from os_brick.initiator.connectors import disco from os_brick.tests.initiator import test_connector class DISCOConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for DISCO connector.""" # Fake volume information volume = { 'name': 'a-disco-volume', 'disco_id': '1234567' } # Conf for test conf = { 'ip': test_connector.MY_IP, 'port': 9898 } def setUp(self): super(DISCOConnectorTestCase, self).setUp() self.fake_connection_properties = { 'name': self.volume['name'], 'disco_id': self.volume['disco_id'], 'conf': { 'server_ip': self.conf['ip'], 'server_port': self.conf['port']} } self.fake_volume_status = {'attached': True, 'detached': False} self.fake_request_status = {'success': None, 'fail': 'ERROR'} self.volume_status = 'detached' self.request_status = 'success' # Patch the request and os calls to fake versions self.mock_object(disco.DISCOConnector, '_send_disco_vol_cmd', self.perform_disco_request) self.mock_object(os.path, 'exists', self.is_volume_attached) self.mock_object(glob, 'glob', self.list_disco_volume) # The actual DISCO connector self.connector = disco.DISCOConnector( 'sudo', execute=self.fake_execute) def perform_disco_request(self, *cmd, **kwargs): """Fake the socket call.""" return self.fake_request_status[self.request_status] def is_volume_attached(self, *cmd, **kwargs): """Fake volume detection check.""" return self.fake_volume_status[self.volume_status] def list_disco_volume(self, *cmd, **kwargs): """Fake the glob call.""" path_dir = self.connector.get_search_path() volume_id = self.volume['disco_id'] volume_items = [path_dir, '/', self.connector.DISCO_PREFIX, volume_id] volume_path = ''.join(volume_items) return [volume_path] def test_get_connector_properties(self): props = disco.DISCOConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): """DISCO volumes should be under /dev.""" expected = "/dev" actual = self.connector.get_search_path() self.assertEqual(expected, actual) def test_get_volume_paths(self): """Test to get all the path for a specific volume.""" expected = ['/dev/dms1234567'] self.volume_status = 'attached' actual = self.connector.get_volume_paths( self.fake_connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): """Attach a volume.""" self.connector.connect_volume(self.fake_connection_properties) def test_connect_volume_already_attached(self): """Make sure that we don't issue the request.""" self.request_status = 'fail' self.volume_status = 'attached' self.test_connect_volume() def test_connect_volume_request_fail(self): """Fail the attach request.""" self.volume_status = 'detached' self.request_status = 'fail' self.assertRaises(exception.BrickException, self.test_connect_volume) def test_disconnect_volume(self): """Detach a volume.""" self.connector.disconnect_volume(self.fake_connection_properties, None) def test_disconnect_volume_attached(self): """Detach a volume attached.""" self.request_status = 'success' self.volume_status = 'attached' self.test_disconnect_volume() def test_disconnect_volume_already_detached(self): """Ensure that we don't issue the request.""" self.request_status = 'fail' self.volume_status = 'detached' self.test_disconnect_volume() def test_disconnect_volume_request_fail(self): """Fail the detach request.""" self.volume_status = 'attached' self.request_status = 'fail' self.assertRaises(exception.BrickException, self.test_disconnect_volume) def test_get_all_available_volumes(self): """Test to get all the available DISCO volumes.""" expected = ['/dev/dms1234567'] actual = self.connector.get_all_available_volumes(None) self.assertItemsEqual(expected, actual) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.fake_connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_drbd.py0000664000175000017500000000520000000000000024713 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import drbd from os_brick.tests.initiator import test_connector class DRBDConnectorTestCase(test_connector.ConnectorTestCase): RESOURCE_TEMPLATE = ''' resource r0 { on host1 { } net { shared-secret "%(shared-secret)s"; } } ''' def setUp(self): super(DRBDConnectorTestCase, self).setUp() self.connector = drbd.DRBDConnector( None, execute=self._fake_exec) self.execs = [] def _fake_exec(self, *cmd, **kwargs): self.execs.append(cmd) # out, err return ('', '') def test_get_connector_properties(self): props = drbd.DRBDConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume(self): """Test connect_volume.""" cprop = { 'provider_auth': 'my-secret', 'config': self.RESOURCE_TEMPLATE, 'name': 'my-precious', 'device': '/dev/drbd951722', 'data': {}, } res = self.connector.connect_volume(cprop) self.assertEqual(cprop['device'], res['path']) self.assertEqual('adjust', self.execs[0][1]) self.assertEqual(cprop['name'], self.execs[0][4]) def test_disconnect_volume(self): """Test the disconnect volume case.""" cprop = { 'provider_auth': 'my-secret', 'config': self.RESOURCE_TEMPLATE, 'name': 'my-precious', 'device': '/dev/drbd951722', 'data': {}, } dev_info = {} self.connector.disconnect_volume(cprop, dev_info) self.assertEqual('down', self.execs[0][1]) def test_extend_volume(self): cprop = {'name': 'something'} self.assertRaises(NotImplementedError, self.connector.extend_volume, cprop) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_fibre_channel.py0000664000175000017500000011242500000000000026567 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock import os import six from os_brick import exception from os_brick.initiator.connectors import base from os_brick.initiator.connectors import fibre_channel from os_brick.initiator import linuxfc from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector @ddt.ddt class FibreChannelConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorTestCase, self).setUp() self.connector = fibre_channel.FibreChannelConnector( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertIsNotNone(self.connector._linuxscsi) def fake_get_fc_hbas(self): return [{'ClassDevice': 'host1', 'ClassDevicePath': '/sys/devices/pci0000:00/0000:00:03.0' '/0000:05:00.2/host1/fc_host/host1', 'dev_loss_tmo': '30', 'fabric_name': '0x1000000533f55566', 'issue_lip': '', 'max_npiv_vports': '255', 'maxframe_size': '2048 bytes', 'node_name': '0x200010604b019419', 'npiv_vports_inuse': '0', 'port_id': '0x680409', 'port_name': '0x100010604b019419', 'port_state': 'Online', 'port_type': 'NPort (fabric via point-to-point)', 'speed': '10 Gbit', 'supported_classes': 'Class 3', 'supported_speeds': '10 Gbit', 'symbolic_name': 'Emulex 554M FV4.0.493.0 DV8.3.27', 'tgtid_bind_type': 'wwpn (World Wide Port Name)', 'uevent': None, 'vport_create': '', 'vport_delete': ''}] def fake_get_fc_hbas_with_platform(self): return [{'ClassDevice': 'host1', 'ClassDevicePath': '/sys/devices/platform/smb' '/smb:motherboard/80040000000.peu0-c0' '/pci0000:00/0000:00:03.0' '/0000:05:00.2/host1/fc_host/host1', 'dev_loss_tmo': '30', 'fabric_name': '0x1000000533f55566', 'issue_lip': '', 'max_npiv_vports': '255', 'maxframe_size': '2048 bytes', 'node_name': '0x200010604b019419', 'npiv_vports_inuse': '0', 'port_id': '0x680409', 'port_name': '0x100010604b019419', 'port_state': 'Online', 'port_type': 'NPort (fabric via point-to-point)', 'speed': '10 Gbit', 'supported_classes': 'Class 3', 'supported_speeds': '10 Gbit', 'symbolic_name': 'Emulex 554M FV4.0.493.0 DV8.3.27', 'tgtid_bind_type': 'wwpn (World Wide Port Name)', 'uevent': None, 'vport_create': '', 'vport_delete': ''}] def fake_get_fc_hbas_info(self): hbas = self.fake_get_fc_hbas() info = [{'port_name': hbas[0]['port_name'].replace('0x', ''), 'node_name': hbas[0]['node_name'].replace('0x', ''), 'host_device': hbas[0]['ClassDevice'], 'device_path': hbas[0]['ClassDevicePath']}] return info def fake_get_fc_hbas_info_with_platform(self): hbas = self.fake_get_fc_hbas_with_platform() info = [{'port_name': hbas[0]['port_name'].replace('0x', ''), 'node_name': hbas[0]['node_name'].replace('0x', ''), 'host_device': hbas[0]['ClassDevice'], 'device_path': hbas[0]['ClassDevicePath']}] return info def fibrechan_connection(self, volume, location, wwn, lun=1): return {'driver_volume_type': 'fibrechan', 'data': { 'volume_id': volume['id'], 'target_portal': location, 'target_wwn': wwn, 'target_lun': lun, }} @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') def test_get_connector_properties(self, mock_hbas): mock_hbas.return_value = self.fake_get_fc_hbas() multipath = True enforce_multipath = True props = fibre_channel.FibreChannelConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) hbas = self.fake_get_fc_hbas() expected_props = {'wwpns': [hbas[0]['port_name'].replace('0x', '')], 'wwnns': [hbas[0]['node_name'].replace('0x', '')]} self.assertEqual(expected_props, props) def test_get_search_path(self): search_path = self.connector.get_search_path() expected = "/dev/disk/by-path" self.assertEqual(expected, search_path) def test_get_pci_num(self): hba = {'device_path': "/sys/devices/pci0000:00/0000:00:03.0" "/0000:05:00.3/host2/fc_host/host2"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:05:00.3", pci_num) self.assertIsNone(platform) hba = {'device_path': "/sys/devices/pci0000:00/0000:00:03.0" "/0000:05:00.3/0000:06:00.6/host2/fc_host/host2"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:06:00.6", pci_num) self.assertIsNone(platform) hba = {'device_path': "/sys/devices/pci0000:20/0000:20:03.0" "/0000:21:00.2/net/ens2f2/ctlr_2/host3" "/fc_host/host3"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:21:00.2", pci_num) self.assertIsNone(platform) def test_get_pci_num_with_platform(self): hba = {'device_path': "/sys/devices/platform/smb/smb:motherboard/" "80040000000.peu0-c0/pci0000:00/0000:00:03.0" "/0000:05:00.3/host2/fc_host/host2"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:05:00.3", pci_num) self.assertEqual("platform-80040000000.peu0-c0", platform) hba = {'device_path': "/sys/devices/platform/smb/smb:motherboard" "/80040000000.peu0-c0/pci0000:00/0000:00:03.0" "/0000:05:00.3/0000:06:00.6/host2/fc_host/host2"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:06:00.6", pci_num) self.assertEqual("platform-80040000000.peu0-c0", platform) hba = {'device_path': "/sys/devices/platform/smb" "/smb:motherboard/80040000000.peu0-c0/pci0000:20" "/0000:20:03.0/0000:21:00.2" "/net/ens2f2/ctlr_2/host3/fc_host/host3"} platform, pci_num = self.connector._get_pci_num(hba) self.assertEqual("0000:21:00.2", pci_num) self.assertEqual("platform-80040000000.peu0-c0", platform) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') def test_get_volume_paths(self, fake_fc_hbas_info, fake_fc_hbas, fake_exists): fake_fc_hbas.side_effect = self.fake_get_fc_hbas fake_fc_hbas_info.side_effect = self.fake_get_fc_hbas_info name = 'volume-00000001' vol = {'id': 1, 'name': name} location = '10.0.2.15:3260' wwn = '1234567890123456' connection_info = self.fibrechan_connection(vol, location, wwn) conn_data = self.connector._add_targets_to_connection_properties( connection_info['data'] ) volume_paths = self.connector.get_volume_paths(conn_data) expected = ['/dev/disk/by-path/pci-0000:05:00.2' '-fc-0x1234567890123456-lun-1'] self.assertEqual(expected, volume_paths) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') def test_get_volume_paths_with_platform(self, fake_fc_hbas_info, fake_fc_hbas, fake_exists): fake_fc_hbas.side_effect = self.fake_get_fc_hbas_with_platform fake_fc_hbas_info.side_effect \ = self.fake_get_fc_hbas_info_with_platform name = 'volume-00000001' vol = {'id': 1, 'name': name} location = '10.0.2.15:3260' wwn = '1234567890123456' connection_info = self.fibrechan_connection(vol, location, wwn) conn_data = self.connector._add_targets_to_connection_properties( connection_info['data'] ) volume_paths = self.connector.get_volume_paths(conn_data) expected = ['/dev/disk/by-path' '/platform-80040000000.peu0-c0-pci-0000:05:00.2' '-fc-0x1234567890123456-lun-1'] self.assertEqual(expected, volume_paths) @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, remove_device_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock): check_valid_device_mock.return_value = True get_fc_hbas_mock.side_effect = self.fake_get_fc_hbas get_fc_hbas_info_mock.side_effect = self.fake_get_fc_hbas_info wwn = '1234567890' multipath_devname = '/dev/md-1' devices = {"device": multipath_devname, "id": wwn, "devices": [{'device': '/dev/sdb', 'address': '1:0:0:1', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}]} get_device_info_mock.return_value = devices['devices'][0] get_scsi_wwn_mock.return_value = wwn location = '10.0.2.15:3260' name = 'volume-00000001' vol = {'id': 1, 'name': name} # Should work for string, unicode, and list wwns_luns = [ ('1234567890123456', 1), (six.text_type('1234567890123456'), 1), (['1234567890123456', '1234567890123457'], 1), (['1234567890123456', '1234567890123457'], 1), ] for wwn, lun in wwns_luns: connection_info = self.fibrechan_connection(vol, location, wwn, lun) dev_info = self.connector.connect_volume(connection_info['data']) exp_wwn = wwn[0] if isinstance(wwn, list) else wwn dev_str = ('/dev/disk/by-path/pci-0000:05:00.2-fc-0x%s-lun-1' % exp_wwn) self.assertEqual(dev_info['type'], 'block') self.assertEqual(dev_info['path'], dev_str) self.assertNotIn('multipath_id', dev_info) self.assertNotIn('devices', dev_info) self.connector.disconnect_volume(connection_info['data'], dev_info) expected_commands = [] self.assertEqual(expected_commands, self.cmds) # Should not work for anything other than string, unicode, and list connection_info = self.fibrechan_connection(vol, location, 123) self.assertRaises(exception.VolumePathsNotFound, self.connector.connect_volume, connection_info['data']) get_fc_hbas_mock.side_effect = [[]] get_fc_hbas_info_mock.side_effect = [[]] self.assertRaises(exception.VolumePathsNotFound, self.connector.connect_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') def _test_connect_volume_multipath(self, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, access_mode, should_wait_for_rw, find_mp_device_path_mock): self.connector.use_multipath = True get_fc_hbas_mock.side_effect = self.fake_get_fc_hbas get_fc_hbas_info_mock.side_effect = self.fake_get_fc_hbas_info wwn = '1234567890' multipath_devname = '/dev/md-1' devices = {"device": multipath_devname, "id": wwn, "devices": [{'device': '/dev/sdb', 'address': '1:0:0:1', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}, {'device': '/dev/sdc', 'address': '1:0:0:2', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}]} get_device_info_mock.side_effect = devices['devices'] get_scsi_wwn_mock.return_value = wwn location = '10.0.2.15:3260' name = 'volume-00000001' vol = {'id': 1, 'name': name} initiator_wwn = ['1234567890123456', '1234567890123457'] find_mp_device_path_mock.return_value = '/dev/mapper/mpatha' find_mp_dev_mock.return_value = {"device": "dm-3", "id": wwn, "name": "mpatha"} connection_info = self.fibrechan_connection(vol, location, initiator_wwn) connection_info['data']['access_mode'] = access_mode self.connector.connect_volume(connection_info['data']) self.assertEqual(should_wait_for_rw, wait_for_rw_mock.called) self.connector.disconnect_volume(connection_info['data'], devices['devices'][0]) expected_commands = [ 'multipath -f ' + find_mp_device_path_mock.return_value, 'tee -a /sys/block/sdb/device/delete', 'tee -a /sys/block/sdc/device/delete', ] self.assertEqual(expected_commands, self.cmds) return connection_info @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_rw(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', True) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_no_access_mode(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, None, True) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_ro(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self._test_connect_volume_multipath(get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'ro', False) @mock.patch.object(base.BaseLinuxConnector, '_discover_mpath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_multipath_not_found(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, discover_mp_dev_mock): check_valid_device_mock.return_value = True discover_mp_dev_mock.return_value = ("/dev/disk/by-path/something", None) connection_info = self._test_connect_volume_multipath( get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', False) self.assertNotIn('multipathd_id', connection_info['data']) # Ensure we don't call it with the real path device_name = discover_mp_dev_mock.call_args[0][-1] self.assertNotEqual(realpath_mock.return_value, device_name) @mock.patch.object(fibre_channel.FibreChannelConnector, 'get_volume_paths') def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] volume = {'id': 'fake_uuid'} wwn = '1234567890123456' connection_info = self.fibrechan_connection(volume, "10.0.2.15:3260", wwn) self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(fibre_channel.FibreChannelConnector, 'get_volume_paths') def test_extend_volume(self, mock_volume_paths, mock_scsi_extend): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} wwn = '1234567890123456' connection_info = self.fibrechan_connection(volume, "10.0.2.15:3260", wwn) new_size = self.connector.extend_volume(connection_info['data']) self.assertEqual(fake_new_size, new_size) @mock.patch.object(os.path, 'isdir') def test_get_all_available_volumes_path_not_dir(self, mock_isdir): mock_isdir.return_value = False expected = [] actual = self.connector.get_all_available_volumes() self.assertItemsEqual(expected, actual) @mock.patch('eventlet.greenthread.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_connect_volume_device_not_valid(self, check_valid_device_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = False self.assertRaises(exception.NoFibreChannelVolumeDeviceFound, self._test_connect_volume_multipath, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock, 'rw', True) @ddt.data( { "target_info": { "target_lun": 1, "target_wwn": '1234567890123456', }, "expected_targets": [ ('1234567890123456', 1) ] }, { "target_info": { "target_lun": 1, "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 1), ] }, { "target_info": { "target_luns": [1, 1], "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 1), ] }, { "target_info": { "target_luns": [1, 2], "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 2), ] }, { "target_info": { "target_luns": [1, 1], "target_wwns": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 1), ] }, { "target_info": { "target_lun": 7, "target_luns": [1, 1], "target_wwn": 'foo', "target_wwns": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 1), ] }, # Add the zone map in now { "target_info": { "target_lun": 1, "target_wwn": '1234567890123456', }, "expected_targets": [ ('1234567890123456', 1) ], "itmap": { '0004567890123456': ['1234567890123456'] }, "expected_map": { '0004567890123456': [('1234567890123456', 1)] } }, { "target_info": { "target_lun": 1, "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 1), ], "itmap": { '0004567890123456': ['1234567890123456', '1234567890123457'] }, "expected_map": { '0004567890123456': [('1234567890123456', 1), ('1234567890123457', 1)] } }, { "target_info": { "target_luns": [1, 2], "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 2), ], "itmap": { '0004567890123456': ['1234567890123456'], '1004567890123456': ['1234567890123457'], }, "expected_map": { '0004567890123456': [('1234567890123456', 1)], '1004567890123456': [('1234567890123457', 2)], } }, { "target_info": { "target_luns": [1, 2], "target_wwn": ['1234567890123456', '1234567890123457'], }, "expected_targets": [ ('1234567890123456', 1), ('1234567890123457', 2), ], "itmap": { '0004567890123456': ['1234567890123456', '1234567890123457'] }, "expected_map": { '0004567890123456': [('1234567890123456', 1), ('1234567890123457', 2)] } }, { "target_info": { "target_lun": 1, "target_wwn": ['20320002AC01E166', '21420002AC01E166', '20410002AC01E166', '21410002AC01E166'] }, "expected_targets": [ ('20320002ac01e166', 1), ('21420002ac01e166', 1), ('20410002ac01e166', 1), ('21410002ac01e166', 1) ], "itmap": { '10001409DCD71FF6': ['20320002AC01E166', '21420002AC01E166'], '10001409DCD71FF7': ['20410002AC01E166', '21410002AC01E166'] }, "expected_map": { '10001409dcd71ff6': [('20320002ac01e166', 1), ('21420002ac01e166', 1)], '10001409dcd71ff7': [('20410002ac01e166', 1), ('21410002ac01e166', 1)] } }, ) @ddt.unpack def test__add_targets_to_connection_properties(self, target_info, expected_targets, itmap=None, expected_map=None): volume = {'id': 'fake_uuid'} wwn = '1234567890123456' conn = self.fibrechan_connection(volume, "10.0.2.15:3260", wwn) conn['data'].update(target_info) conn['data']['initiator_target_map'] = itmap connection_info = self.connector._add_targets_to_connection_properties( conn['data']) self.assertIn('targets', connection_info) self.assertEqual(expected_targets, connection_info['targets']) # Check that we turn to lowercase target wwns key = 'target_wwns' if 'target_wwns' in target_info else 'target_wwn' wwns = target_info.get(key) wwns = [wwns] if isinstance(wwns, six.string_types) else wwns wwns = [w.lower() for w in wwns] if wwns: self.assertEqual(wwns, conn['data'][key]) if itmap: self.assertIn('initiator_target_lun_map', connection_info) self.assertEqual(expected_map, connection_info['initiator_target_lun_map']) @ddt.data(('/dev/mapper/', True), ('/dev/mapper/mpath0', True), # Check real devices are properly detected as non multipaths ('/dev/sda', False), ('/dev/disk/by-path/pci-1-fc-1-lun-1', False)) @ddt.unpack @mock.patch('os_brick.initiator.linuxscsi.LinuxSCSI.remove_scsi_device') @mock.patch('os_brick.initiator.linuxscsi.LinuxSCSI.requires_flush') @mock.patch('os_brick.initiator.linuxscsi.LinuxSCSI.get_dev_path') def test__remove_devices(self, path_used, was_multipath, get_dev_path_mock, flush_mock, remove_mock): get_dev_path_mock.return_value = path_used self.connector._remove_devices(mock.sentinel.con_props, [{'device': '/dev/sda'}], mock.sentinel.device_info) get_dev_path_mock.assert_called_once_with(mock.sentinel.con_props, mock.sentinel.device_info) flush_mock.assert_called_once_with('/dev/sda', path_used, was_multipath) remove_mock.assert_called_once_with('/dev/sda', flush=flush_mock.return_value) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_rw') @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(os.path, 'realpath', return_value='/dev/sdb') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_hbas_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(base.BaseLinuxConnector, 'check_valid_device') def test_disconnect_volume(self, check_valid_device_mock, find_mp_device_path_mock, get_device_info_mock, get_scsi_wwn_mock, get_fc_hbas_info_mock, get_fc_hbas_mock, realpath_mock, exists_mock, wait_for_rw_mock, find_mp_dev_mock): check_valid_device_mock.return_value = True self.connector.use_multipath = True get_fc_hbas_mock.side_effect = self.fake_get_fc_hbas get_fc_hbas_info_mock.side_effect = self.fake_get_fc_hbas_info wwn = '1234567890' multipath_devname = '/dev/md-1' devices = {"device": multipath_devname, "id": wwn, "devices": [{'device': '/dev/sdb', 'address': '1:0:0:1', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}, {'device': '/dev/sdc', 'address': '1:0:0:2', 'host': 1, 'channel': 0, 'id': 0, 'lun': 1}]} get_device_info_mock.side_effect = devices['devices'] get_scsi_wwn_mock.return_value = wwn location = '10.0.2.15:3260' name = 'volume-00000001' vol = {'id': 1, 'name': name} initiator_wwn = ['1234567890123456', '1234567890123457'] find_mp_device_path_mock.return_value = '/dev/mapper/mpatha' find_mp_dev_mock.return_value = {"device": "dm-3", "id": wwn, "name": "mpatha"} connection_info = self.fibrechan_connection(vol, location, initiator_wwn) self.connector.disconnect_volume(connection_info['data'], devices['devices'][0]) expected_commands = [ 'multipath -f ' + find_mp_device_path_mock.return_value, 'tee -a /sys/block/sdb/device/delete', 'tee -a /sys/block/sdc/device/delete', ] self.assertEqual(expected_commands, self.cmds) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_fibre_channel_ppc64.py0000664000175000017500000000440500000000000027601 0ustar00zuulzuul00000000000000# (c) Copyright 2013 IBM Company # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import fibre_channel_ppc64 from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector class FibreChannelConnectorPPC64TestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorPPC64TestCase, self).setUp() self.connector = fibre_channel_ppc64.FibreChannelConnectorPPC64( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertEqual(self.connector._linuxfc.__class__.__name__, "LinuxFibreChannel") self.assertIsNotNone(self.connector._linuxscsi) @mock.patch.object(linuxscsi.LinuxSCSI, 'process_lun_id', return_value='2') def test_get_host_devices(self, mock_process_lun_id): lun = 2 possible_devs = [(3, "0x5005076802232ade"), (3, "0x5005076802332ade"), ] devices = self.connector._get_host_devices(possible_devs, lun) self.assertEqual(2, len(devices)) device_path = "/dev/disk/by-path/fc-0x5005076802332ade-lun-2" self.assertIn(device_path, devices) device_path = "/dev/disk/by-path/fc-0x5005076802232ade-lun-2" self.assertIn(device_path, devices) # test duplicates possible_devs = [(3, "0x5005076802232ade"), (3, "0x5005076802232ade"), ] devices = self.connector._get_host_devices(possible_devs, lun) self.assertEqual(1, len(devices)) device_path = "/dev/disk/by-path/fc-0x5005076802232ade-lun-2" self.assertIn(device_path, devices) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_fibre_channel_s390x.py0000664000175000017500000000717200000000000027537 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import fibre_channel_s390x from os_brick.initiator import linuxfc from os_brick.tests.initiator import test_connector class FibreChannelConnectorS390XTestCase(test_connector.ConnectorTestCase): def setUp(self): super(FibreChannelConnectorS390XTestCase, self).setUp() self.connector = fibre_channel_s390x.FibreChannelConnectorS390X( None, execute=self.fake_execute, use_multipath=False) self.assertIsNotNone(self.connector) self.assertIsNotNone(self.connector._linuxfc) self.assertEqual(self.connector._linuxfc.__class__.__name__, "LinuxFibreChannelS390X") self.assertIsNotNone(self.connector._linuxscsi) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'configure_scsi_device') def test_get_host_devices(self, mock_configure_scsi_device): possible_devs = [(3, 5, 2), ] devices = self.connector._get_host_devices(possible_devs) mock_configure_scsi_device.assert_called_with(3, 5, "0x0002000000000000") self.assertEqual(3, len(devices)) device_path = "/dev/disk/by-path/ccw-3-zfcp-5:0x0002000000000000" self.assertEqual(devices[0], device_path) device_path = "/dev/disk/by-path/ccw-3-fc-5-lun-2" self.assertEqual(devices[1], device_path) device_path = "/dev/disk/by-path/ccw-3-fc-5-lun-0x0002000000000000" self.assertEqual(devices[2], device_path) def test_get_lun_string(self): lun = 1 lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x0001000000000000") lun = 0xff lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x00ff000000000000") lun = 0x101 lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x0101000000000000") lun = 0x4020400a lunstring = self.connector._get_lun_string(lun) self.assertEqual(lunstring, "0x4020400a00000000") @mock.patch.object(fibre_channel_s390x.FibreChannelConnectorS390X, '_get_possible_devices', return_value=[('', 3, 5, 2), ]) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'get_fc_hbas_info', return_value=[]) @mock.patch.object(linuxfc.LinuxFibreChannelS390X, 'deconfigure_scsi_device') def test_remove_devices(self, mock_deconfigure_scsi_device, mock_get_fc_hbas_info, mock_get_possible_devices): connection_properties = {'targets': [5, 2]} self.connector._remove_devices(connection_properties, devices=None, device_info=None) mock_deconfigure_scsi_device.assert_called_with(3, 5, "0x0002000000000000") mock_get_fc_hbas_info.assert_called_once_with() mock_get_possible_devices.assert_called_once_with([], [5, 2]) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_gpfs.py0000664000175000017500000000272500000000000024750 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import gpfs from os_brick.tests.initiator.connectors import test_local class GPFSConnectorTestCase(test_local.LocalConnectorTestCase): def setUp(self): super(GPFSConnectorTestCase, self).setUp() self.connection_properties = {'name': 'foo', 'device_path': '/tmp/bar'} self.connector = gpfs.GPFSConnector(None) def test_connect_volume(self): cprops = self.connection_properties dev_info = self.connector.connect_volume(cprops) self.assertEqual(dev_info['type'], 'gpfs') self.assertEqual(dev_info['path'], cprops['device_path']) def test_connect_volume_with_invalid_connection_data(self): cprops = {} self.assertRaises(ValueError, self.connector.connect_volume, cprops) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_hgst.py0000664000175000017500000002077000000000000024756 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator import connector from os_brick.initiator.connectors import hgst from os_brick.tests.initiator import test_connector class HGSTConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for HGST initiator class.""" IP_OUTPUT = """ 1: lo: mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 169.254.169.254/32 scope link lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: em1: mtu 1500 qdisc mq master link/ether 00:25:90:d9:18:08 brd ff:ff:ff:ff:ff:ff inet6 fe80::225:90ff:fed9:1808/64 scope link valid_lft forever preferred_lft forever 3: em2: mtu 1500 qdisc mq state link/ether 00:25:90:d9:18:09 brd ff:ff:ff:ff:ff:ff inet 192.168.0.23/24 brd 192.168.0.255 scope global em2 valid_lft forever preferred_lft forever inet6 fe80::225:90ff:fed9:1809/64 scope link valid_lft forever preferred_lft forever """ DOMAIN_OUTPUT = """localhost""" DOMAIN_FAILED = """this.better.not.resolve.to.a.name.or.else""" SET_APPHOST_OUTPUT = """ VLVM_SET_APPHOSTS0000000395 Request Succeeded """ def setUp(self): super(HGSTConnectorTestCase, self).setUp() self.connector = hgst.HGSTConnector( None, execute=self._fake_exec) self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False def _fake_exec_set_apphosts(self, *cmd): if self._fail_set_apphosts: raise putils.ProcessExecutionError(None, None, 1) else: return self.SET_APPHOST_OUTPUT, '' def _fake_exec_ip(self, *cmd): if self._fail_ip: # Remove localhost so there is no IP match return self.IP_OUTPUT.replace("127.0.0.1", "x.x.x.x"), '' else: return self.IP_OUTPUT, '' def _fake_exec_domain_list(self, *cmd): if self._fail_domain_list: return self.DOMAIN_FAILED, '' else: return self.DOMAIN_OUTPUT, '' def _fake_exec(self, *cmd, **kwargs): self.cmdline = " ".join(cmd) if cmd[0] == "ip": return self._fake_exec_ip(*cmd) elif cmd[0] == "vgc-cluster": if cmd[1] == "domain-list": return self._fake_exec_domain_list(*cmd) elif cmd[1] == "space-set-apphosts": return self._fake_exec_set_apphosts(*cmd) else: return '', '' def test_factory(self): """Can we instantiate a HGSTConnector of the right kind?""" obj = connector.InitiatorConnector.factory('HGST', None, arch='x86_64') self.assertEqual("HGSTConnector", obj.__class__.__name__) def test_get_search_path(self): expected = "/dev" actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(os.path, 'exists', return_value=True) def test_get_volume_paths(self, mock_exists): cprops = {'name': 'space', 'noremovehost': 'stor1'} path = "/dev/%s" % cprops['name'] expected = [path] actual = self.connector.get_volume_paths(cprops) self.assertEqual(expected, actual) def test_connect_volume(self): """Tests that a simple connection succeeds""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} dev_info = self.connector.connect_volume(cprops) self.assertEqual('block', dev_info['type']) self.assertEqual('space', dev_info['device']) self.assertEqual('/dev/space', dev_info['path']) def test_get_connector_properties(self): props = hgst.HGSTConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume_nohost_fail(self): """This host should not be found, connect should fail.""" self._fail_set_apphosts = False self._fail_ip = True self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) def test_connect_volume_nospace_fail(self): """The space command will fail, exception to be thrown""" self._fail_set_apphosts = True self._fail_ip = False self._fail_domain_list = False cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) def test_disconnect_volume(self): """Simple disconnection should pass and disconnect me""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'stor1'} self.connector.disconnect_volume(cprops, None) exp_cli = ("vgc-cluster space-set-apphosts -n space " "-A localhost --action DELETE") self.assertEqual(exp_cli, self.cmdline) def test_disconnect_volume_nohost(self): """Should not run a setapphosts because localhost will""" """be the noremotehost""" self._fail_set_apphosts = False self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'localhost'} self.connector.disconnect_volume(cprops, None) # The last command should be the IP listing, not set apphosts exp_cli = ("ip addr list") self.assertEqual(exp_cli, self.cmdline) def test_disconnect_volume_fails(self): """The set-apphosts should fail, exception to be thrown""" self._fail_set_apphosts = True self._fail_ip = False self._fail_domain_list = False self._cmdline = "" cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) def test_bad_connection_properties(self): """Send in connection_properties missing required fields""" # Invalid connection_properties self.assertRaises(exception.BrickException, self.connector.connect_volume, None) # Name required for connect_volume cprops = {'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.connect_volume, cprops) # Invalid connection_properties self.assertRaises(exception.BrickException, self.connector.disconnect_volume, None, None) # Name and noremovehost needed for disconnect_volume cprops = {'noremovehost': 'stor1'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) cprops = {'name': 'space'} self.assertRaises(exception.BrickException, self.connector.disconnect_volume, cprops, None) def test_extend_volume(self): cprops = {'name': 'space', 'noremovehost': 'stor1'} self.assertRaises(NotImplementedError, self.connector.extend_volume, cprops) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_huawei.py0000664000175000017500000002443100000000000025271 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os import tempfile from os_brick import exception from os_brick.initiator.connectors import huawei from os_brick.tests.initiator import test_connector class HuaweiStorHyperConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for StorHyper initiator class.""" attached = False def setUp(self): super(HuaweiStorHyperConnectorTestCase, self).setUp() self.fake_sdscli_file = tempfile.mktemp() self.addCleanup(os.remove, self.fake_sdscli_file) newefile = open(self.fake_sdscli_file, 'w') newefile.write('test') newefile.close() self.connector = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute) self.connector.cli_path = self.fake_sdscli_file self.connector.iscliexist = True self.connector_fail = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute_fail) self.connector_fail.cli_path = self.fake_sdscli_file self.connector_fail.iscliexist = True self.connector_nocli = huawei.HuaweiStorHyperConnector( None, execute=self.fake_execute_fail) self.connector_nocli.cli_path = self.fake_sdscli_file self.connector_nocli.iscliexist = False self.connection_properties = { 'access_mode': 'rw', 'qos_specs': None, 'volume_id': 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f' } self.device_info = {'type': 'block', 'path': '/dev/vdxxx'} HuaweiStorHyperConnectorTestCase.attached = False def fake_execute(self, *cmd, **kwargs): method = cmd[2] self.cmds.append(" ".join(cmd)) if 'attach' == method: HuaweiStorHyperConnectorTestCase.attached = True return 'ret_code=0', None if 'querydev' == method: if HuaweiStorHyperConnectorTestCase.attached: return 'ret_code=0\ndev_addr=/dev/vdxxx', None else: return 'ret_code=1\ndev_addr=/dev/vdxxx', None if 'detach' == method: HuaweiStorHyperConnectorTestCase.attached = False return 'ret_code=0', None def fake_execute_fail(self, *cmd, **kwargs): method = cmd[2] self.cmds.append(" ".join(cmd)) if 'attach' == method: HuaweiStorHyperConnectorTestCase.attached = False return 'ret_code=330151401', None if 'querydev' == method: if HuaweiStorHyperConnectorTestCase.attached: return 'ret_code=0\ndev_addr=/dev/vdxxx', None else: return 'ret_code=1\ndev_addr=/dev/vdxxx', None if 'detach' == method: HuaweiStorHyperConnectorTestCase.attached = True return 'ret_code=330155007', None def test_get_connector_properties(self): props = huawei.HuaweiStorHyperConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): actual = self.connector.get_search_path() self.assertIsNone(actual) @mock.patch.object(huawei.HuaweiStorHyperConnector, '_query_attached_volume') def test_get_volume_paths(self, mock_query_attached): path = self.device_info['path'] mock_query_attached.return_value = {'ret_code': 0, 'dev_addr': path} expected = [path] actual = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): """Test the basic connect volume case.""" retval = self.connector.connect_volume(self.connection_properties) self.assertEqual(self.device_info, retval) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_disconnect_volume(self): """Test the basic disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.connector.disconnect_volume(self.connection_properties, self.device_info) self.assertEqual(False, HuaweiStorHyperConnectorTestCase.attached) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_is_volume_connected(self): """Test if volume connected to host case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) is_connected = self.connector.is_volume_connected( 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f') self.assertEqual(HuaweiStorHyperConnectorTestCase.attached, is_connected) self.connector.disconnect_volume(self.connection_properties, self.device_info) self.assertEqual(False, HuaweiStorHyperConnectorTestCase.attached) is_connected = self.connector.is_volume_connected( 'volume-b2911673-863c-4380-a5f2-e1729eecfe3f') self.assertEqual(HuaweiStorHyperConnectorTestCase.attached, is_connected) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test__analyze_output(self): cliout = 'ret_code=0\ndev_addr=/dev/vdxxx\nret_desc="success"' analyze_result = {'dev_addr': '/dev/vdxxx', 'ret_desc': '"success"', 'ret_code': '0'} result = self.connector._analyze_output(cliout) self.assertEqual(analyze_result, result) def test_connect_volume_fail(self): """Test the fail connect volume case.""" self.assertRaises(exception.BrickException, self.connector_fail.connect_volume, self.connection_properties) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_disconnect_volume_fail(self): """Test the fail disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.assertRaises(exception.BrickException, self.connector_fail.disconnect_volume, self.connection_properties, self.device_info) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c detach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_connect_volume_nocli(self): """Test the fail connect volume case.""" self.assertRaises(exception.BrickException, self.connector_nocli.connect_volume, self.connection_properties) def test_disconnect_volume_nocli(self): """Test the fail disconnect volume case.""" self.connector.connect_volume(self.connection_properties) self.assertEqual(True, HuaweiStorHyperConnectorTestCase.attached) self.assertRaises(exception.BrickException, self.connector_nocli.disconnect_volume, self.connection_properties, self.device_info) expected_commands = [self.fake_sdscli_file + ' -c attach' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f', self.fake_sdscli_file + ' -c querydev' ' -v volume-b2911673-863c-4380-a5f2-e1729eecfe3f'] self.assertEqual(expected_commands, self.cmds) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_iscsi.py0000664000175000017500000025725100000000000025131 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import collections import mock import os import ddt from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator.connectors import iscsi from os_brick.initiator import linuxscsi from os_brick.initiator import utils from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests.initiator import test_connector @ddt.ddt class ISCSIConnectorTestCase(test_connector.ConnectorTestCase): SINGLE_CON_PROPS = {'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1', 'encryption': False, 'target_lun': '1'} CON_PROPS = { 'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1', 'target_lun': 4, 'target_portals': ['ip1:port1', 'ip2:port2', 'ip3:port3', 'ip4:port4'], 'target_iqns': ['tgt1', 'tgt2', 'tgt3', 'tgt4'], 'target_luns': [4, 5, 6, 7], } def setUp(self): super(ISCSIConnectorTestCase, self).setUp() self.connector = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=False) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) self.mock_object(self.connector._linuxscsi, 'get_name_from_path', return_value="/dev/sdb") self._fake_iqn = 'iqn.1234-56.foo.bar:01:23456789abc' self._name = 'volume-00000001' self._iqn = 'iqn.2010-10.org.openstack:%s' % self._name self._location = '10.0.2.15:3260' self._lun = 1 @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsi_session') def test_get_iscsi_sessions_full(self, sessions_mock): iscsiadm_result = ('tcp: [session1] ip1:port1,1 tgt1 (non-flash)\n' 'tcp: [session2] ip2:port2,-1 tgt2 (non-flash)\n' 'tcp: [session3] ip3:port3,1 tgt3\n') sessions_mock.return_value = (iscsiadm_result, '') res = self.connector._get_iscsi_sessions_full() expected = [('tcp:', 'session1', 'ip1:port1', '1', 'tgt1'), ('tcp:', 'session2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', 'session3', 'ip3:port3', '1', 'tgt3')] self.assertListEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsi_session') def test_get_iscsi_sessions_full_stderr(self, sessions_mock): iscsiadm_result = ('tcp: [session1] ip1:port1,1 tgt1 (non-flash)\n' 'tcp: [session2] ip2:port2,-1 tgt2 (non-flash)\n' 'tcp: [session3] ip3:port3,1 tgt3\n') sessions_mock.return_value = (iscsiadm_result, 'error') res = self.connector._get_iscsi_sessions_full() expected = [('tcp:', 'session1', 'ip1:port1', '1', 'tgt1'), ('tcp:', 'session2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', 'session3', 'ip3:port3', '1', 'tgt3')] self.assertListEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_get_iscsi_sessions(self, sessions_mock): sessions_mock.return_value = [ ('tcp:', 'session1', 'ip1:port1', '1', 'tgt1'), ('tcp:', 'session2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', 'session3', 'ip3:port3', '1', 'tgt3')] res = self.connector._get_iscsi_sessions() expected = ['ip1:port1', 'ip2:port2', 'ip3:port3'] self.assertListEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full', return_value=[]) def test_get_iscsi_sessions_no_sessions(self, sessions_mock): res = self.connector._get_iscsi_sessions() self.assertListEqual([], res) sessions_mock.assert_called() @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_iscsi_nodes(self, exec_mock): iscsiadm_result = ('ip1:port1,1 tgt1\nip2:port2,-1 tgt2\n' 'ip3:port3,1 tgt3\n') exec_mock.return_value = (iscsiadm_result, '') res = self.connector._get_iscsi_nodes() expected = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] self.assertListEqual(expected, res) exec_mock.assert_called_once_with( 'iscsiadm', '-m', 'node', run_as_root=True, root_helper=self.connector._root_helper, check_exit_code=False) @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_iscsi_nodes_error(self, exec_mock): exec_mock.return_value = (None, 'error') res = self.connector._get_iscsi_nodes() self.assertEqual([], res) @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_iscsi_nodes_corrupt(self, exec_mock): iscsiadm_result = ('ip1:port1,-1 tgt1\n' 'ip2:port2,-1 tgt2\n' '[]:port3,-1\n' 'ip4:port4,-1 tgt4\n') exec_mock.return_value = (iscsiadm_result, '') res = self.connector._get_iscsi_nodes() expected = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip4:port4', 'tgt4')] self.assertListEqual(expected, res) exec_mock.assert_called_once_with( 'iscsiadm', '-m', 'node', run_as_root=True, root_helper=self.connector._root_helper, check_exit_code=False) @mock.patch.object(iscsi.ISCSIConnector, '_get_ips_iqns_luns') @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices(self, nodes_mock, sessions_mock, glob_mock, iql_mock): iql_mock.return_value = self.connector._get_all_targets(self.CON_PROPS) # List sessions from other targets and non tcp sessions sessions_mock.return_value = [ ('non-tcp:', '0', 'ip1:port1', '1', 'tgt1'), ('tcp:', '1', 'ip1:port1', '1', 'tgt1'), ('tcp:', '2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', '3', 'ip1:port1', '1', 'tgt4'), ('tcp:', '4', 'ip2:port2', '-1', 'tgt5')] # List 1 node without sessions nodes_mock.return_value = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session1/target6/1:2:6:4/block/sda', sys_cls + '1/device/session1/target6/1:2:6:4/block/sda1'], [sys_cls + '2/device/session2/target7/2:2:7:5/block/sdb', sys_cls + '2/device/session2/target7/2:2:7:4/block/sdc'], ] res = self.connector._get_connection_devices(self.CON_PROPS) expected = {('ip1:port1', 'tgt1'): ({'sda'}, set()), ('ip2:port2', 'tgt2'): ({'sdb'}, {'sdc'}), ('ip3:port3', 'tgt3'): (set(), set())} self.assertDictEqual(expected, res) iql_mock.assert_called_once_with(self.CON_PROPS, discover=False, is_disconnect_call=False) @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices_with_iqns(self, nodes_mock, sessions_mock, glob_mock): ips_iqns_luns = self.connector._get_all_targets(self.CON_PROPS) # List sessions from other targets and non tcp sessions sessions_mock.return_value = [ ('non-tcp:', '0', 'ip1:port1', '1', 'tgt1'), ('tcp:', '1', 'ip1:port1', '1', 'tgt1'), ('tcp:', '2', 'ip2:port2', '-1', 'tgt2'), ('tcp:', '3', 'ip1:port1', '1', 'tgt4'), ('tcp:', '4', 'ip2:port2', '-1', 'tgt5')] # List 1 node without sessions nodes_mock.return_value = [('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2'), ('ip3:port3', 'tgt3')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session1/target6/1:2:6:4/block/sda', sys_cls + '1/device/session1/target6/1:2:6:4/block/sda1'], [sys_cls + '2/device/session2/target7/2:2:7:5/block/sdb', sys_cls + '2/device/session2/target7/2:2:7:4/block/sdc'], ] with mock.patch.object(iscsi.ISCSIConnector, '_get_all_targets') as get_targets_mock: res = self.connector._get_connection_devices(mock.sentinel.props, ips_iqns_luns) expected = {('ip1:port1', 'tgt1'): ({'sda'}, set()), ('ip2:port2', 'tgt2'): ({'sdb'}, {'sdc'}), ('ip3:port3', 'tgt3'): (set(), set())} self.assertDictEqual(expected, res) get_targets_mock.assert_not_called() def generate_device(self, location, iqn, transport=None, lun=1): dev_format = "ip-%s-iscsi-%s-lun-%s" % (location, iqn, lun) if transport: dev_format = "pci-0000:00:00.0-" + dev_format fake_dev_path = "/dev/disk/by-path/" + dev_format return fake_dev_path def iscsi_connection(self, volume, location, iqn): return { 'driver_volume_type': 'iscsi', 'data': { 'volume_id': volume['id'], 'target_portal': location, 'target_iqn': iqn, 'target_lun': 1, } } def iscsi_connection_multipath(self, volume, locations, iqns, luns): return { 'driver_volume_type': 'iscsi', 'data': { 'volume_id': volume['id'], 'target_portals': locations, 'target_iqns': iqns, 'target_luns': luns, } } def iscsi_connection_chap(self, volume, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password): return { 'driver_volume_type': 'iscsi', 'data': { 'auth_method': auth_method, 'auth_username': auth_username, 'auth_password': auth_password, 'discovery_auth_method': discovery_auth_method, 'discovery_auth_username': discovery_auth_username, 'discovery_auth_password': discovery_auth_password, 'target_lun': 1, 'volume_id': volume['id'], 'target_iqn': iqn, 'target_portal': location, } } def _initiator_get_text(self, *arg, **kwargs): text = ('## DO NOT EDIT OR REMOVE THIS FILE!\n' '## If you remove this file, the iSCSI daemon ' 'will not start.\n' '## If you change the InitiatorName, existing ' 'access control lists\n' '## may reject this initiator. The InitiatorName must ' 'be unique\n' '## for each iSCSI initiator. Do NOT duplicate iSCSI ' 'InitiatorNames.\n' 'InitiatorName=%s' % self._fake_iqn) return text, None def test_get_initiator(self): def initiator_no_file(*args, **kwargs): raise putils.ProcessExecutionError('No file') self.connector._execute = initiator_no_file initiator = self.connector.get_initiator() self.assertIsNone(initiator) self.connector._execute = self._initiator_get_text initiator = self.connector.get_initiator() self.assertEqual(initiator, self._fake_iqn) def test_get_connector_properties(self): with mock.patch.object(priv_rootwrap, 'execute') as mock_exec: mock_exec.return_value = self._initiator_get_text() multipath = True enforce_multipath = True props = iscsi.ISCSIConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'initiator': self._fake_iqn} self.assertEqual(expected_props, props) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_brick_iscsi_validate_transport(self, mock_iscsiadm): sample_output = ('# BEGIN RECORD 2.0-872\n' 'iface.iscsi_ifacename = %s.fake_suffix\n' 'iface.net_ifacename = \n' 'iface.ipaddress = \n' 'iface.hwaddress = 00:53:00:00:53:00\n' 'iface.transport_name = %s\n' 'iface.initiatorname = \n' '# END RECORD') for tport in self.connector.supported_transports: mock_iscsiadm.return_value = (sample_output % (tport, tport), '') self.assertEqual(tport + '.fake_suffix', self.connector._validate_iface_transport( tport + '.fake_suffix')) mock_iscsiadm.return_value = ("", 'iscsiadm: Could not ' 'read iface fake_transport (6)') self.assertEqual('default', self.connector._validate_iface_transport( 'fake_transport')) def test_get_search_path(self): search_path = self.connector.get_search_path() expected = "/dev/disk/by-path" self.assertEqual(expected, search_path) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(iscsi.ISCSIConnector, '_get_potential_volume_paths') def test_get_volume_paths(self, mock_potential_paths, mock_exists): name1 = 'volume-00000001-1' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn = 'iqn.2010-10.org.openstack:%s' % name1 fake_path = ("/dev/disk/by-path/ip-%(ip)s-iscsi-%(iqn)s-lun-%(lun)s" % {'ip': '10.0.2.15', 'iqn': iqn, 'lun': 1}) fake_devices = [fake_path] expected = fake_devices mock_potential_paths.return_value = fake_devices connection_properties = self.iscsi_connection(vol, [location], [iqn]) volume_paths = self.connector.get_volume_paths( connection_properties['data']) self.assertEqual(expected, volume_paths) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') def test_discover_mpath_device(self, mock_multipath_device, mock_multipath_device_path): location1 = '10.0.2.15:3260' location2 = '[2001:db8::1]:3260' name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 fake_multipath_dev = '/dev/mapper/fake-multipath-dev' fake_raw_dev = '/dev/disk/by-path/fake-raw-lun' vol = {'id': 1, 'name': name1} connection_properties = self.iscsi_connection_multipath( vol, [location1, location2], [iqn1, iqn2], [1, 2]) mock_multipath_device_path.return_value = fake_multipath_dev mock_multipath_device.return_value = test_connector.FAKE_SCSI_WWN (result_path, result_mpath_id) = ( self.connector_with_multipath._discover_mpath_device( test_connector.FAKE_SCSI_WWN, connection_properties['data'], fake_raw_dev)) result = {'path': result_path, 'multipath_id': result_mpath_id} expected_result = {'path': fake_multipath_dev, 'multipath_id': test_connector.FAKE_SCSI_WWN} self.assertEqual(expected_result, result) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device') @mock.patch.object(os.path, 'realpath') def test_discover_mpath_device_by_realpath(self, mock_realpath, mock_multipath_device, mock_multipath_device_path): FAKE_SCSI_WWN = '1234567890' location1 = '10.0.2.15:3260' location2 = '[2001:db8::1]:3260' name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 fake_multipath_dev = None fake_raw_dev = '/dev/disk/by-path/fake-raw-lun' vol = {'id': 1, 'name': name1} connection_properties = self.iscsi_connection_multipath( vol, [location1, location2], [iqn1, iqn2], [1, 2]) mock_multipath_device_path.return_value = fake_multipath_dev mock_multipath_device.return_value = { 'device': '/dev/mapper/%s' % FAKE_SCSI_WWN} mock_realpath.return_value = '/dev/sdvc' (result_path, result_mpath_id) = ( self.connector_with_multipath._discover_mpath_device( FAKE_SCSI_WWN, connection_properties['data'], fake_raw_dev)) mock_multipath_device.assert_called_with('/dev/sdvc') result = {'path': result_path, 'multipath_id': result_mpath_id} expected_result = {'path': '/dev/mapper/%s' % FAKE_SCSI_WWN, 'multipath_id': FAKE_SCSI_WWN} self.assertEqual(expected_result, result) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_mp(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = True res = self.connector.connect_volume(self.CON_PROPS) self.assertEqual(con_mp_mock.return_value, res) con_single_mock.assert_not_called() con_mp_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_mp_failure(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = True con_mp_mock.side_effect = exception.BrickException self.assertRaises(exception.BrickException, self.connector.connect_volume, self.CON_PROPS) con_single_mock.assert_not_called() con_mp_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_called_once_with(self.CON_PROPS, force=True) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_sp(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = False res = self.connector.connect_volume(self.CON_PROPS) self.assertEqual(con_single_mock.return_value, res) con_mp_mock.assert_not_called() con_single_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch.object(iscsi.ISCSIConnector, '_connect_multipath_volume') @mock.patch.object(iscsi.ISCSIConnector, '_connect_single_volume') def test_connect_volume_sp_failure(self, con_single_mock, con_mp_mock, clean_mock): self.connector.use_multipath = False con_single_mock.side_effect = exception.BrickException self.assertRaises(exception.BrickException, self.connector.connect_volume, self.CON_PROPS) con_mp_mock.assert_not_called() con_single_mock.assert_called_once_with(self.CON_PROPS) clean_mock.assert_called_once_with(self.CON_PROPS, force=True) def test_discover_iscsi_portals(self): location = '10.0.2.15:3260' name = 'volume-00000001' iqn = 'iqn.2010-10.org.openstack:%s' % name vol = {'id': 1, 'name': name} auth_method = 'CHAP' auth_username = 'fake_chap_username' auth_password = 'fake_chap_password' discovery_auth_method = 'CHAP' discovery_auth_username = 'fake_chap_username' discovery_auth_password = 'fake_chap_password' connection_properties = self.iscsi_connection_chap( vol, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) for transport in ['default', 'iser', 'badTransport']: interface = 'iser' if transport == 'iser' else 'default' self.mock_object(self.connector_with_multipath, '_get_transport', mock.Mock(return_value=interface)) self.connector_with_multipath._discover_iscsi_portals( connection_properties['data']) expected_cmds = [ 'iscsiadm -m discoverydb -t sendtargets -I %(iface)s ' '-p %(location)s --op update ' '-n discovery.sendtargets.auth.authmethod -v %(auth_method)s ' '-n discovery.sendtargets.auth.username -v %(username)s ' '-n discovery.sendtargets.auth.password -v %(password)s' % {'iface': interface, 'location': location, 'auth_method': discovery_auth_method, 'username': discovery_auth_username, 'password': discovery_auth_password}, 'iscsiadm -m node --op show -p %s' % location, 'iscsiadm -m discoverydb -t sendtargets -I %(iface)s' ' -p %(location)s --discover' % {'iface': interface, 'location': location}, 'iscsiadm -m node --op show -p %s' % location] self.assertEqual(expected_cmds, self.cmds) # Reset to run with a different transport type self.cmds = list() @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_update_discoverydb') @mock.patch.object(os.path, 'exists', return_value=True) def test_iscsi_portals_with_chap_discovery( self, exists, update_discoverydb): location = '10.0.2.15:3260' name = 'volume-00000001' iqn = 'iqn.2010-10.org.openstack:%s' % name vol = {'id': 1, 'name': name} auth_method = 'CHAP' auth_username = 'fake_chap_username' auth_password = 'fake_chap_password' discovery_auth_method = 'CHAP' discovery_auth_username = 'fake_chap_username' discovery_auth_password = 'fake_chap_password' connection_properties = self.iscsi_connection_chap( vol, location, iqn, auth_method, auth_username, auth_password, discovery_auth_method, discovery_auth_username, discovery_auth_password) self.connector_with_multipath = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=True) self.cmds = [] # The first call returns an error code = 6, mocking an empty # discovery db. The second one mocks a successful return and the # third one a dummy exit code, which will trigger the # TargetPortalNotFound exception in connect_volume update_discoverydb.side_effect = [ putils.ProcessExecutionError(None, None, 6), ("", ""), putils.ProcessExecutionError(None, None, 9)] self.connector_with_multipath._discover_iscsi_portals( connection_properties['data']) update_discoverydb.assert_called_with(connection_properties['data']) expected_cmds = [ 'iscsiadm -m discoverydb -t sendtargets -p %s -I default' ' --op new' % location, 'iscsiadm -m node --op show -p %s' % location, 'iscsiadm -m discoverydb -t sendtargets -I default -p %s' ' --discover' % location, 'iscsiadm -m node --op show -p %s' % location] self.assertEqual(expected_cmds, self.cmds) self.assertRaises(exception.TargetPortalNotFound, self.connector_with_multipath.connect_volume, connection_properties['data']) def test_get_target_portals_from_iscsiadm_output(self): connector = self.connector test_output = '''10.15.84.19:3260,1 iqn.1992-08.com.netapp:sn.33615311 10.15.85.19:3260,2 iqn.1992-08.com.netapp:sn.33615311 ''' res = connector._get_target_portals_from_iscsiadm_output(test_output) ips = ['10.15.84.19:3260', '10.15.85.19:3260'] iqns = ['iqn.1992-08.com.netapp:sn.33615311', 'iqn.1992-08.com.netapp:sn.33615311'] expected = (ips, iqns) self.assertEqual(expected, res) @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') def test_disconnect_volume(self, cleanup_mock): res = self.connector.disconnect_volume(mock.sentinel.con_props, mock.sentinel.dev_info, mock.sentinel.Force, mock.sentinel.ignore_errors) self.assertEqual(cleanup_mock.return_value, res) cleanup_mock.assert_called_once_with( mock.sentinel.con_props, force=mock.sentinel.Force, ignore_errors=mock.sentinel.ignore_errors, device_info=mock.sentinel.dev_info, is_disconnect_call=True) @ddt.data(True, False) @mock.patch.object(iscsi.ISCSIConnector, '_get_transport') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals(self, is_iser, iscsiadm_mock, transport_mock): params = { 'iqn1': self.SINGLE_CON_PROPS['target_iqn'], 'iqn2': 'iqn.2004-04.com.qnap:ts-831x:iscsi.cinder-2017.9ef', 'addr': self.SINGLE_CON_PROPS['target_portal'].replace(':', ','), 'ip1': self.SINGLE_CON_PROPS['target_portal'], 'ip2': '192.168.1.3:3260', 'transport': 'iser' if is_iser else 'default', 'other_transport': 'default' if is_iser else 'iser', } iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: %(addr)s\n' 'Target: %(iqn1)s\n' ' Portal: %(ip2)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(other_transport)s\n' 'Target: %(iqn2)s\n' ' Portal: %(ip2)s,1\n' ' Iface Name: %(transport)s\n' ' Portal: %(ip1)s,1\n' ' Iface Name: %(transport)s\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n' % params, None) transport_mock.return_value = 'iser' if is_iser else 'non-iser' res = self.connector._get_discoverydb_portals(self.SINGLE_CON_PROPS) expected = [(params['ip2'], params['iqn1'], self.SINGLE_CON_PROPS['target_lun']), (params['ip1'], params['iqn1'], self.SINGLE_CON_PROPS['target_lun']), (params['ip2'], params['iqn2'], self.SINGLE_CON_PROPS['target_lun']), (params['ip1'], params['iqn2'], self.SINGLE_CON_PROPS['target_lun'])] self.assertListEqual(expected, res) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_get_transport', return_value='') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals_error(self, iscsiadm_mock, transport_mock): """DiscoveryAddress is not present.""" iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n', None) self.assertRaises(exception.TargetPortalsNotFound, self.connector._get_discoverydb_portals, self.SINGLE_CON_PROPS) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_transport', return_value='') @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_discoverydb_portals_error_is_present(self, iscsiadm_mock, transport_mock): """DiscoveryAddress is present but wrong iterface.""" params = { 'iqn': self.SINGLE_CON_PROPS['target_iqn'], 'addr': self.SINGLE_CON_PROPS['target_portal'].replace(':', ','), 'ip': self.SINGLE_CON_PROPS['target_portal'], } iscsiadm_mock.return_value = ( 'SENDTARGETS:\n' 'DiscoveryAddress: 192.168.1.33,3260\n' 'DiscoveryAddress: %(addr)s\n' 'Target: %(iqn)s\n' ' Portal: %(ip)s,1\n' ' Iface Name: iser\n' 'DiscoveryAddress: 192.168.1.38,3260\n' 'iSNS:\n' 'No targets found.\n' 'STATIC:\n' 'No targets found.\n' 'FIRMWARE:\n' 'No targets found.\n' % params, None) self.assertRaises(exception.TargetPortalsNotFound, self.connector._get_discoverydb_portals, self.SINGLE_CON_PROPS) iscsiadm_mock.assert_called_once_with( ['-m', 'discoverydb', '-o', 'show', '-P', 1]) transport_mock.assert_called_once_with() @ddt.data(('/dev/sda', False), ('/dev/disk/by-id/scsi-WWID', False), ('/dev/dm-11', True), ('/dev/disk/by-id/dm-uuid-mpath-MPATH', True)) @ddt.unpack @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dev_path') @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_connection') @mock.patch.object(iscsi.ISCSIConnector, '_get_connection_devices') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_connection', return_value=None) def test_cleanup_connection(self, path_used, was_multipath, remove_mock, flush_mock, con_devs_mock, discon_mock, get_dev_path_mock): get_dev_path_mock.return_value = path_used # Return an ordered dicts instead of normal dict for discon_mock.assert con_devs_mock.return_value = collections.OrderedDict(( (('ip1:port1', 'tgt1'), ({'sda'}, set())), (('ip2:port2', 'tgt2'), ({'sdb'}, {'sdc'})), (('ip3:port3', 'tgt3'), (set(), set())))) self.connector._cleanup_connection( self.CON_PROPS, ips_iqns_luns=mock.sentinel.ips_iqns_luns, force=False, ignore_errors=False, device_info=mock.sentinel.device_info) get_dev_path_mock.called_once_with(self.CON_PROPS, mock.sentinel.device_info) con_devs_mock.assert_called_once_with(self.CON_PROPS, mock.sentinel.ips_iqns_luns, False) remove_mock.assert_called_once_with({'sda', 'sdb'}, False, mock.ANY, path_used, was_multipath) discon_mock.assert_called_once_with( self.CON_PROPS, [('ip1:port1', 'tgt1'), ('ip3:port3', 'tgt3')], False, mock.ANY) flush_mock.assert_not_called() @mock.patch('os_brick.exception.ExceptionChainer.__nonzero__', mock.Mock(return_value=True)) @mock.patch('os_brick.exception.ExceptionChainer.__bool__', mock.Mock(return_value=True)) @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_connection') @mock.patch.object(iscsi.ISCSIConnector, '_get_connection_devices') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_connection', return_value=mock.sentinel.mp_name) def test_cleanup_connection_force_failure(self, remove_mock, flush_mock, con_devs_mock, discon_mock): # Return an ordered dicts instead of normal dict for discon_mock.assert con_devs_mock.return_value = collections.OrderedDict(( (('ip1:port1', 'tgt1'), ({'sda'}, set())), (('ip2:port2', 'tgt2'), ({'sdb'}, {'sdc'})), (('ip3:port3', 'tgt3'), (set(), set())))) self.assertRaises(exception.ExceptionChainer, self.connector._cleanup_connection, self.CON_PROPS, ips_iqns_luns=mock.sentinel.ips_iqns_luns, force=mock.sentinel.force, ignore_errors=False) con_devs_mock.assert_called_once_with(self.CON_PROPS, mock.sentinel.ips_iqns_luns, False) remove_mock.assert_called_once_with({'sda', 'sdb'}, mock.sentinel.force, mock.ANY, '', False) discon_mock.assert_called_once_with( self.CON_PROPS, [('ip1:port1', 'tgt1'), ('ip3:port3', 'tgt3')], mock.sentinel.force, mock.ANY) flush_mock.assert_called_once_with(mock.sentinel.mp_name) def test_cleanup_connection_no_data_discoverydb(self): self.connector.use_multipath = True with mock.patch.object(self.connector, '_get_discoverydb_portals', side_effect=exception.TargetPortalsNotFound), \ mock.patch.object(self.connector._linuxscsi, 'remove_connection') as mock_remove: # This will not raise and exception self.connector._cleanup_connection(self.SINGLE_CON_PROPS) mock_remove.assert_not_called() @ddt.data({'do_raise': False, 'force': False}, {'do_raise': True, 'force': True}, {'do_raise': True, 'force': False}) @ddt.unpack @mock.patch.object(iscsi.ISCSIConnector, '_disconnect_from_iscsi_portal') def test_disconnect_connection(self, disconnect_mock, do_raise, force): will_raise = do_raise and not force actual_call_args = [] # Since we reuse the copied dictionary on _disconnect_connection # changing its values we cannot use mock's assert_has_calls def my_disconnect(con_props): actual_call_args.append(con_props.copy()) if do_raise: raise exception.ExceptionChainer() disconnect_mock.side_effect = my_disconnect connections = (('ip1:port1', 'tgt1'), ('ip2:port2', 'tgt2')) original_props = self.CON_PROPS.copy() exc = exception.ExceptionChainer() if will_raise: self.assertRaises(exception.ExceptionChainer, self.connector._disconnect_connection, self.CON_PROPS, connections, force=force, exc=exc) else: self.connector._disconnect_connection(self.CON_PROPS, connections, force=force, exc=exc) # Passed properties should not be altered by the method call self.assertDictEqual(original_props, self.CON_PROPS) expected = [original_props.copy(), original_props.copy()] for i, (ip, iqn) in enumerate(connections): expected[i].update(target_portal=ip, target_iqn=iqn) # If we are failing and not forcing we won't make all the alls if will_raise: expected = expected[:1] self.assertListEqual(expected, actual_call_args) # No exceptions have been caught by ExceptionChainer context manager self.assertEqual(do_raise, bool(exc)) def test_disconnect_from_iscsi_portal(self): self.connector._disconnect_from_iscsi_portal(self.CON_PROPS) expected_prefix = ('iscsiadm -m node -T %s -p %s ' % (self.CON_PROPS['target_iqn'], self.CON_PROPS['target_portal'])) expected = [ expected_prefix + '--op update -n node.startup -v manual', expected_prefix + '--logout', expected_prefix + '--op delete', ] self.assertListEqual(expected, self.cmds) def test_iscsiadm_discover_parsing(self): # Ensure that parsing iscsiadm discover ignores cruft. ips = ["192.168.204.82:3260", "192.168.204.82:3261"] iqns = ["iqn.2010-10.org.openstack:volume-" "f9b12623-6ce3-4dac-a71f-09ad4249bdd3", "iqn.2010-10.org.openstack:volume-" "f9b12623-6ce3-4dac-a71f-09ad4249bdd4"] # This slight wonkiness brought to you by pep8, as the actual # example output runs about 97 chars wide. sample_input = """Loading iscsi modules: done Starting iSCSI initiator service: done Setting up iSCSI targets: unused %s %s %s %s """ % (ips[0] + ',1', iqns[0], ips[1] + ',1', iqns[1]) out = self.connector.\ _get_target_portals_from_iscsiadm_output(sample_input) self.assertEqual((ips, iqns), out) def test_sanitize_log_run_iscsiadm(self): # Tests that the parameters to the _run_iscsiadm function # are sanitized for when passwords are logged. def fake_debug(*args, **kwargs): self.assertIn('node.session.auth.password', args[0]) self.assertNotIn('scrubme', args[0]) volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") iscsi_properties = connection_info['data'] with mock.patch.object(iscsi.LOG, 'debug', side_effect=fake_debug) as debug_mock: self.connector._iscsiadm_update(iscsi_properties, 'node.session.auth.password', 'scrubme') # we don't care what the log message is, we just want to make sure # our stub method is called which asserts the password is scrubbed self.assertTrue(debug_mock.called) @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume(self, mock_volume_paths, mock_scsi_extend): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection(volume, "10.0.2.15:3260", "fake_iqn") new_size = self.connector.extend_volume(connection_info['data']) self.assertEqual(fake_new_size, new_size) @mock.patch.object(iscsi.LOG, 'info') @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_mask_password(self, mock_volume_paths, mock_scsi_extend, mock_log_info): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection_chap( volume, "10.0.2.15:3260", "fake_iqn", 'CHAP', 'fake_user', 'fake_password', 'CHAP1', 'fake_user1', 'fake_password1') self.connector.extend_volume(connection_info['data']) self.assertEqual(2, mock_log_info.call_count) self.assertIn("'auth_password': '***'", str(mock_log_info.call_args_list[0])) self.assertIn("'discovery_auth_password': '***'", str(mock_log_info.call_args_list[0])) @mock.patch.object(iscsi.LOG, 'warning') @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume') @mock.patch.object(iscsi.ISCSIConnector, 'get_volume_paths') def test_extend_volume_mask_password_no_paths(self, mock_volume_paths, mock_scsi_extend, mock_log_warning): fake_new_size = 1024 mock_volume_paths.return_value = [] mock_scsi_extend.return_value = fake_new_size volume = {'id': 'fake_uuid'} connection_info = self.iscsi_connection_chap( volume, "10.0.2.15:3260", "fake_iqn", 'CHAP', 'fake_user', 'fake_password', 'CHAP1', 'fake_user1', 'fake_password1') self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_info['data']) self.assertEqual(1, mock_log_warning.call_count) self.assertIn("'auth_password': '***'", str(mock_log_warning.call_args_list[0])) self.assertIn("'discovery_auth_password': '***'", str(mock_log_warning.call_args_list[0])) @mock.patch.object(os.path, 'isdir') def test_get_all_available_volumes_path_not_dir(self, mock_isdir): mock_isdir.return_value = False expected = [] actual = self.connector.get_all_available_volumes() self.assertItemsEqual(expected, actual) @mock.patch.object(iscsi.ISCSIConnector, '_get_device_path') def test_get_potential_paths_mpath(self, get_path_mock): self.connector.use_multipath = True res = self.connector._get_potential_volume_paths(self.CON_PROPS) get_path_mock.assert_called_once_with(self.CON_PROPS) self.assertEqual(get_path_mock.return_value, res) self.assertEqual([], self.cmds) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions') @mock.patch.object(iscsi.ISCSIConnector, '_get_device_path') def test_get_potential_paths_single_path(self, get_path_mock, get_sessions_mock): get_path_mock.side_effect = [['path1'], ['path2'], ['path3', 'path4']] get_sessions_mock.return_value = [ 'ip1:port1', 'ip2:port2', 'ip3:port3'] self.connector.use_multipath = False res = self.connector._get_potential_volume_paths(self.CON_PROPS) self.assertEqual({'path1', 'path2', 'path3', 'path4'}, set(res)) get_sessions_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_with_target_iqns(self, discover_mock): res = self.connector._get_ips_iqns_luns(self.CON_PROPS) expected = list(self.connector._get_all_targets(self.CON_PROPS)) self.assertListEqual(expected, res) discover_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_discoverydb_portals') @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_discoverydb(self, discover_mock, db_portals_mock): db_portals_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS, discover=False) self.assertListEqual(db_portals_mock.return_value, res) db_portals_mock.assert_called_once_with(self.SINGLE_CON_PROPS) discover_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_all_targets') @mock.patch.object(iscsi.ISCSIConnector, '_get_discoverydb_portals') @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_disconnect_single_path(self, discover_mock, db_portals_mock, get_targets_mock): db_portals_mock.side_effect = exception.TargetPortalsNotFound res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS, discover=False, is_disconnect_call=True) db_portals_mock.assert_called_once_with(self.SINGLE_CON_PROPS) discover_mock.assert_not_called() get_targets_mock.assert_called_once_with(self.SINGLE_CON_PROPS) self.assertEqual(get_targets_mock.return_value, res) @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_no_target_iqns_share_iqn(self, discover_mock): discover_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip1:port1', 'tgt2', '1'), ('ip2:port2', 'tgt1', '2'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS) expected = {('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt1', '2')} self.assertEqual(expected, set(res)) @mock.patch.object(iscsi.ISCSIConnector, '_discover_iscsi_portals') def test_get_ips_iqns_luns_no_target_iqns_diff_iqn(self, discover_mock): discover_mock.return_value = [('ip1:port1', 'tgt1', '1'), ('ip2:port2', 'tgt2', '2')] res = self.connector._get_ips_iqns_luns(self.SINGLE_CON_PROPS) self.assertEqual(discover_mock.return_value, res) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_all_new(self, get_sessions_mock): """Connect creating node and session.""" session = 'session2' get_sessions_mock.side_effect = [ [('tcp:', 'session1', 'ip1:port1', '1', 'tgt')], [('tcp:', 'session1', 'ip1:port1', '1', 'tgt'), ('tcp:', session, 'ip1:port1', '-1', 'tgt1')] ] utils.ISCSI_SUPPORTS_MANUAL_SCAN = None with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', 'error'), ('', None), ('', None), ('', None), ('', None)] res = self.connector._connect_to_iscsi_portal(self.CON_PROPS) # True refers to "manual scans", since the call to update # node.session.scan didn't fail they are set to manual self.assertEqual((session, True), res) self.assertTrue(utils.ISCSI_SUPPORTS_MANUAL_SCAN) prefix = 'iscsiadm -m node -T tgt1 -p ip1:port1' expected_cmds = [ prefix, prefix + ' --interface default --op new', prefix + ' --op update -n node.session.scan -v manual', prefix + ' --login', prefix + ' --op update -n node.startup -v automatic' ] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) self.assertEqual(2, get_sessions_mock.call_count) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_ip_case_insensitive(self, get_sessions_mock): """Connect creating node and session.""" session = 'session2' get_sessions_mock.side_effect = [ [('tcp:', 'session1', 'iP1:port1', '1', 'tgt')], [('tcp:', 'session1', 'Ip1:port1', '1', 'tgt'), ('tcp:', session, 'IP1:port1', '-1', 'tgt1')] ] utils.ISCSI_SUPPORTS_MANUAL_SCAN = None with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', 'error'), ('', None), ('', None), ('', None), ('', None)] res = self.connector._connect_to_iscsi_portal(self.CON_PROPS) # True refers to "manual scans", since the call to update # node.session.scan didn't fail they are set to manual self.assertEqual((session, True), res) self.assertTrue(utils.ISCSI_SUPPORTS_MANUAL_SCAN) prefix = 'iscsiadm -m node -T tgt1 -p ip1:port1' expected_cmds = [ prefix, prefix + ' --interface default --op new', prefix + ' --op update -n node.session.scan -v manual', prefix + ' --login', prefix + ' --op update -n node.startup -v automatic' ] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) self.assertEqual(2, get_sessions_mock.call_count) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_all_exists_chap(self, get_sessions_mock): """Node and session already exists and we use chap authentication.""" session = 'session2' get_sessions_mock.return_value = [('tcp:', session, 'ip1:port1', '-1', 'tgt1')] con_props = self.CON_PROPS.copy() con_props.update(auth_method='CHAP', auth_username='user', auth_password='pwd') utils.ISCSI_SUPPORTS_MANUAL_SCAN = None res = self.connector._connect_to_iscsi_portal(con_props) # False refers to "manual scans", so we have manual iscsi scans self.assertEqual((session, True), res) self.assertTrue(utils.ISCSI_SUPPORTS_MANUAL_SCAN) prefix = 'iscsiadm -m node -T tgt1 -p ip1:port1' expected_cmds = [ prefix, prefix + ' --op update -n node.session.scan -v manual', prefix + ' --op update -n node.session.auth.authmethod -v CHAP', prefix + ' --op update -n node.session.auth.username -v user', prefix + ' --op update -n node.session.auth.password -v pwd', ] self.assertListEqual(expected_cmds, self.cmds) get_sessions_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') def test_connect_to_iscsi_portal_fail_login(self, get_sessions_mock): get_sessions_mock.return_value = [] with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', None), ('', None), putils.ProcessExecutionError] res = self.connector._connect_to_iscsi_portal(self.CON_PROPS) self.assertEqual((None, None), res) expected_cmds = ['iscsiadm -m node -T tgt1 -p ip1:port1', 'iscsiadm -m node -T tgt1 -p ip1:port1 ' '--op update -n node.session.scan -v manual', 'iscsiadm -m node -T tgt1 -p ip1:port1 --login'] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) get_sessions_mock.assert_called_once_with() @mock.patch.object(iscsi.ISCSIConnector, '_iscsiadm_update') @mock.patch.object(iscsi.ISCSIConnector, '_get_transport', return_value='default') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch('time.sleep') def test_connect_to_iscsi_portal_fail_op_new(self, sleep_mock, get_sessions_mock, get_transport_mock, iscsiadm_update_mock): get_sessions_mock.return_value = [] with mock.patch.object(self.connector, '_execute') as exec_mock: exec_mock.side_effect = [('', 21), ('', 6), ('', 21), ('', 6), ('', 21), ('', 6)] self.assertRaises(exception.BrickException, self.connector._connect_to_iscsi_portal, self.CON_PROPS) expected_cmds = ['iscsiadm -m node -T tgt1 -p ip1:port1', 'iscsiadm -m node -T tgt1 -p ip1:port1 ' '--interface default --op new', 'iscsiadm -m node -T tgt1 -p ip1:port1', 'iscsiadm -m node -T tgt1 -p ip1:port1 ' '--interface default --op new', 'iscsiadm -m node -T tgt1 -p ip1:port1', 'iscsiadm -m node -T tgt1 -p ip1:port1 ' '--interface default --op new'] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) iscsiadm_update_mock.assert_not_called() # Called twice by the retry mechanism self.assertEqual(2, sleep_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', side_effect=(None, 'tgt2')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch('time.sleep') def test_connect_single_volume(self, sleep_mock, cleanup_mock, connect_mock, get_wwn_mock): def my_connect(rescans, props, data): if props['target_iqn'] == 'tgt2': # Succeed on second call data['found_devices'].append('sdz') connect_mock.side_effect = my_connect res = self.connector._connect_single_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'tgt2', 'path': '/dev/sdz'} self.assertEqual(expected, res) get_wwn_mock.assert_has_calls([mock.call(['sdz']), mock.call(['sdz'])]) sleep_mock.assert_called_once_with(1) cleanup_mock.assert_called_once_with( {'target_lun': 4, 'volume_id': 'vol_id', 'target_portal': 'ip1:port1', 'target_iqn': 'tgt1'}, (('ip1:port1', 'tgt1', 4),), force=True, ignore_errors=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch('time.sleep') def test_connect_single_volume_no_wwn(self, sleep_mock, cleanup_mock, connect_mock, get_wwn_mock): def my_connect(rescans, props, data): data['found_devices'].append('sdz') connect_mock.side_effect = my_connect res = self.connector._connect_single_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': '', 'path': '/dev/sdz'} self.assertEqual(expected, res) get_wwn_mock.assert_has_calls([mock.call(['sdz'])] * 10) self.assertEqual(10, get_wwn_mock.call_count) sleep_mock.assert_has_calls([mock.call(1)] * 10) self.assertEqual(10, sleep_mock.call_count) cleanup_mock.assert_not_called() @staticmethod def _get_connect_vol_data(): return {'stop_connecting': False, 'num_logins': 0, 'failed_logins': 0, 'stopped_threads': 0, 'found_devices': [], 'just_added_devices': []} @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', side_effect=(None, 'tgt2')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch.object(iscsi.ISCSIConnector, '_cleanup_connection') @mock.patch('time.sleep') def test_connect_single_volume_not_found(self, sleep_mock, cleanup_mock, connect_mock, get_wwn_mock): self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_single_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() # Called twice by the retry mechanism self.assertEqual(2, sleep_mock.call_count) props = list(self.connector._get_all_targets(self.CON_PROPS)) calls_per_try = [ mock.call({'target_portal': prop[0], 'target_iqn': prop[1], 'target_lun': prop[2], 'volume_id': 'vol_id'}, (prop,), force=True, ignore_errors=True) for prop in props ] cleanup_mock.assert_has_calls(calls_per_try * 3) data = self._get_connect_vol_data() calls_per_try = [mock.call(self.connector.device_scan_attempts, {'target_portal': prop[0], 'target_iqn': prop[1], 'target_lun': prop[2], 'volume_id': 'vol_id'}, data) for prop in props] connect_mock.assert_has_calls(calls_per_try * 3) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_all_succeed(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': 'sda', 'tgt2': 'sdb', 'tgt3': 'sdc', 'tgt4': 'sdd'} data['stopped_threads'] += 1 data['num_logins'] += 1 dev = devs[props['target_iqn']] data['found_devices'].append(dev) data['just_added_devices'].append(dev) connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'multipath_id': 'wwn', 'path': '/dev/dm-0'} self.assertEqual(expected, res) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sda', 'sdb', 'sdc', 'sdd'], result) # Check we pass the mpath self.assertIsNone(get_wwn_mock.call_args[0][1]) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 2) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_no_wwid(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): # Even if we don't have the wwn we'll be able to find the multipath def my_connect(rescans, props, data): devs = {'tgt1': 'sda', 'tgt2': 'sdb', 'tgt3': 'sdc', 'tgt4': 'sdd'} data['stopped_threads'] += 1 data['num_logins'] += 1 dev = devs[props['target_iqn']] data['found_devices'].append(dev) data['just_added_devices'].append(dev) connect_mock.side_effect = my_connect with mock.patch.object(self.connector, 'use_multipath'): res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': '', 'multipath_id': '', 'path': '/dev/dm-0'} self.assertEqual(expected, res) self.assertEqual(3, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sda', 'sdb', 'sdc', 'sdd'], result) # Initially mpath we pass is None, but on last call is the mpath mpath_values = [c[1][1] for c in get_wwn_mock._mock_mock_calls] self.assertEqual([None, None, 'dm-0'], mpath_values) add_wwid_mock.assert_not_called() add_path_mock.assert_not_called() self.assertGreaterEqual(find_dm_mock.call_count, 2) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_all_fail(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): data['stopped_threads'] += 1 data['failed_logins'] += 1 connect_mock.side_effect = my_connect self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_multipath_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() add_wwid_mock.assert_not_called() add_path_mock.assert_not_called() find_dm_mock.assert_not_called() self.assertEqual(4 * 3, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', side_effect=[None, 'dm-0']) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_some_fail_mp_found(self, sleep_mock, connect_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': '', 'tgt2': 'sdb', 'tgt3': '', 'tgt4': 'sdd'} data['stopped_threads'] += 1 dev = devs[props['target_iqn']] if dev: data['num_logins'] += 1 data['found_devices'].append(dev) data['just_added_devices'].append(dev) else: data['failed_logins'] += 1 connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'multipath_id': 'wwn', 'path': '/dev/dm-0'} self.assertEqual(expected, res) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sdb', 'sdd'], result) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 2) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', return_value=None) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.time, 'time', side_effect=(0, 0, 11, 0)) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep') def test_connect_multipath_volume_some_fail_mp_not_found(self, sleep_mock, connect_mock, time_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): devs = {'tgt1': '', 'tgt2': 'sdb', 'tgt3': '', 'tgt4': 'sdd'} data['stopped_threads'] += 1 dev = devs[props['target_iqn']] if dev: data['num_logins'] += 1 data['found_devices'].append(dev) data['just_added_devices'].append(dev) else: data['failed_logins'] += 1 connect_mock.side_effect = my_connect res = self.connector._connect_multipath_volume(self.CON_PROPS) expected = [{'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sdb'}, {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sdd'}] # It can only be one of the 2 self.assertIn(res, expected) self.assertEqual(1, get_wwn_mock.call_count) result = list(get_wwn_mock.call_args[0][0]) result.sort() self.assertEqual(['sdb', 'sdd'], result) add_wwid_mock.assert_called_once_with('wwn') self.assertNotEqual(0, add_path_mock.call_count) self.assertGreaterEqual(find_dm_mock.call_count, 4) self.assertEqual(4, connect_mock.call_count) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', return_value=None) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwn', return_value='wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_add_wwid') @mock.patch.object(iscsi.time, 'time', side_effect=(0, 0, 11, 0)) @mock.patch.object(iscsi.ISCSIConnector, '_connect_vol') @mock.patch('time.sleep', mock.Mock()) def test_connect_multipath_volume_all_loging_not_found(self, connect_mock, time_mock, add_wwid_mock, add_path_mock, get_wwn_mock, find_dm_mock): def my_connect(rescans, props, data): data['stopped_threads'] += 1 data['num_logins'] += 1 connect_mock.side_effect = my_connect self.assertRaises(exception.VolumeDeviceNotFound, self.connector._connect_multipath_volume, self.CON_PROPS) get_wwn_mock.assert_not_called() add_wwid_mock.assert_not_called() add_path_mock.assert_not_called() find_dm_mock.assert_not_called() self.assertEqual(12, connect_mock.call_count) @mock.patch('time.sleep') @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', return_value='sda') @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol(self, connect_mock, dev_name_mock, scan_mock, sleep_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] connect_mock.return_value = (mock.sentinel.session, False) with mock.patch.object(lscsi, 'get_hctl', side_effect=(None, hctl)) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, found_devices=['sda'], just_added_devices=['sda']) self.assertDictEqual(expected, data) connect_mock.assert_called_once_with(self.CON_PROPS) hctl_mock.assert_has_calls([mock.call(mock.sentinel.session, self.CON_PROPS['target_lun']), mock.call(mock.sentinel.session, self.CON_PROPS['target_lun'])]) scan_mock.assert_not_called() dev_name_mock.assert_called_once_with(mock.sentinel.session, hctl) sleep_mock.assert_called_once_with(1) @mock.patch('time.sleep') @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', side_effect=(None, None, None, None, 'sda')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_rescan(self, connect_mock, dev_name_mock, scan_mock, sleep_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] connect_mock.return_value = (mock.sentinel.session, False) with mock.patch.object(lscsi, 'get_hctl', return_value=hctl) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, found_devices=['sda'], just_added_devices=['sda']) self.assertDictEqual(expected, data) connect_mock.assert_called_once_with(self.CON_PROPS) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) scan_mock.assert_called_once_with(*hctl) self.assertEqual(5, dev_name_mock.call_count) self.assertEqual(4, sleep_mock.call_count) @mock.patch('time.sleep') @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', side_effect=(None, None, None, None, 'sda')) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_manual(self, connect_mock, dev_name_mock, scan_mock, sleep_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] # Simulate manual scan connect_mock.return_value = (mock.sentinel.session, True) with mock.patch.object(lscsi, 'get_hctl', return_value=hctl) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, found_devices=['sda'], just_added_devices=['sda']) self.assertDictEqual(expected, data) connect_mock.assert_called_once_with(self.CON_PROPS) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) self.assertEqual(2, scan_mock.call_count) self.assertEqual(5, dev_name_mock.call_count) self.assertEqual(4, sleep_mock.call_count) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal', return_value=(None, False)) def test_connect_vol_no_session(self, connect_mock): data = self._get_connect_vol_data() self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(failed_logins=1, stopped_threads=1) self.assertDictEqual(expected, data) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_with_connection_failure(self, connect_mock): data = self._get_connect_vol_data() connect_mock.side_effect = Exception() self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(failed_logins=1, stopped_threads=1) self.assertDictEqual(expected, data) @mock.patch('time.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(linuxscsi.LinuxSCSI, 'device_name_by_hctl', return_value=None) @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_not_found(self, connect_mock, dev_name_mock, scan_mock): lscsi = self.connector._linuxscsi data = self._get_connect_vol_data() hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] # True because we are simulating we have manual scans connect_mock.return_value = (mock.sentinel.session, True) with mock.patch.object(lscsi, 'get_hctl', side_effect=(hctl,)) as hctl_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1) self.assertDictEqual(expected, data) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) # We have 3 scans because on manual mode we also scan on connect scan_mock.assert_has_calls([mock.call(*hctl)] * 3) dev_name_mock.assert_has_calls( [mock.call(mock.sentinel.session, hctl), mock.call(mock.sentinel.session, hctl)]) @mock.patch('time.sleep', mock.Mock()) @mock.patch.object(linuxscsi.LinuxSCSI, 'scan_iscsi') @mock.patch.object(iscsi.ISCSIConnector, '_connect_to_iscsi_portal') def test_connect_vol_stop_connecting(self, connect_mock, scan_mock): data = self._get_connect_vol_data() def device_name_by_hctl(session, hctl): data['stop_connecting'] = True return None lscsi = self.connector._linuxscsi hctl = [mock.sentinel.host, mock.sentinel.channel, mock.sentinel.target, mock.sentinel.lun] connect_mock.return_value = (mock.sentinel.session, False) with mock.patch.object(lscsi, 'get_hctl', return_value=hctl) as hctl_mock, \ mock.patch.object( lscsi, 'device_name_by_hctl', side_effect=device_name_by_hctl) as dev_name_mock: self.connector._connect_vol(3, self.CON_PROPS, data) expected = self._get_connect_vol_data() expected.update(num_logins=1, stopped_threads=1, stop_connecting=True) self.assertDictEqual(expected, data) hctl_mock.assert_called_once_with(mock.sentinel.session, self.CON_PROPS['target_lun']) scan_mock.assert_not_called() dev_name_mock.assert_called_once_with(mock.sentinel.session, hctl) @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link') def test__get_connect_result(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = False res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb']) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/sda'} self.assertDictEqual(expected, res) get_link_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link') def test__get_connect_result_mpath(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = False res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb'], 'mpath') expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': '/dev/mpath', 'multipath_id': 'wwn'} self.assertDictEqual(expected, res) get_link_mock.assert_not_called() @mock.patch.object(iscsi.ISCSIConnector, '_get_device_link', return_value='/dev/disk/by-id/scsi-wwn') def test__get_connect_result_encrypted(self, get_link_mock): props = self.CON_PROPS.copy() props['encrypted'] = True res = self.connector._get_connect_result(props, 'wwn', ['sda', 'sdb']) expected = {'type': 'block', 'scsi_wwn': 'wwn', 'path': get_link_mock.return_value} self.assertDictEqual(expected, res) get_link_mock.assert_called_once_with('wwn', '/dev/sda', None) @mock.patch('os.path.realpath', return_value='/dev/sda') def test__get_device_link(self, realpath_mock): symlink = '/dev/disk/by-id/scsi-wwn' res = self.connector._get_device_link('wwn', '/dev/sda', None) self.assertEqual(symlink, res) realpath_mock.assert_called_once_with(symlink) @mock.patch('os.path.realpath', return_value='/dev/dm-0') def test__get_device_link_multipath(self, realpath_mock): symlink = '/dev/disk/by-id/dm-uuid-mpath-wwn' res = self.connector._get_device_link('wwn', '/dev/dm-0', 'wwn') self.assertEqual(symlink, res) realpath_mock.assert_called_once_with(symlink) @mock.patch('os.path.realpath', side_effect=('/dev/sdz', '/dev/sdy', '/dev/sda', '/dev/sdx')) @mock.patch('os.listdir', return_value=['dm-...', 'scsi-wwn', 'scsi-...']) def test__get_device_link_check_links(self, listdir_mock, realpath_mock): res = self.connector._get_device_link('wwn', '/dev/sda', None) self.assertEqual(res, '/dev/disk/by-id/scsi-wwn') listdir_mock.assert_called_once_with('/dev/disk/by-id/') realpath_mock.assert_has_calls([ mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-wwn')]) @mock.patch('time.sleep') @mock.patch('os.path.realpath', return_value='/dev/sdz') @mock.patch('os.listdir', return_value=['dm-...', 'scsi-...']) def test__get_device_link_not_found(self, listdir_mock, realpath_mock, mock_time): self.assertRaises(exception.VolumeDeviceNotFound, self.connector._get_device_link, 'wwn', '/dev/sda', None) listdir_mock.assert_has_calls(3 * [mock.call('/dev/disk/by-id/')]) self.assertEqual(3, listdir_mock.call_count) realpath_mock.assert_has_calls( 3 * [mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-...')]) self.assertEqual(9, realpath_mock.call_count) @mock.patch('time.sleep') @mock.patch('os.path.realpath') @mock.patch('os.listdir', return_value=['dm-...', 'scsi-...']) def test__get_device_link_symlink_found_after_retry(self, mock_listdir, mock_realpath, mock_time): # Return the expected realpath on the third retry mock_realpath.side_effect = [ None, None, None, None, None, None, '/dev/sda'] # Assert that VolumeDeviceNotFound isn't raised self.connector._get_device_link('wwn', '/dev/sda', None) # Assert that listdir and realpath have been called correctly mock_listdir.assert_has_calls(2 * [mock.call('/dev/disk/by-id/')]) self.assertEqual(2, mock_listdir.call_count) mock_realpath.assert_has_calls( 2 * [mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-...')] + [mock.call('/dev/disk/by-id/scsi-wwn')]) self.assertEqual(7, mock_realpath.call_count) @mock.patch('time.sleep') @mock.patch('os.path.realpath') @mock.patch('os.listdir', return_value=['dm-...', 'scsi-...']) def test__get_device_link_symlink_found_after_retry_by_listdir( self, mock_listdir, mock_realpath, mock_time): # Return the expected realpath on the second retry while looping over # the devices returned by listdir mock_realpath.side_effect = [ None, None, None, None, None, '/dev/sda'] # Assert that VolumeDeviceNotFound isn't raised self.connector._get_device_link('wwn', '/dev/sda', None) # Assert that listdir and realpath have been called correctly mock_listdir.assert_has_calls(2 * [mock.call('/dev/disk/by-id/')]) self.assertEqual(2, mock_listdir.call_count) mock_realpath.assert_has_calls( 2 * [mock.call('/dev/disk/by-id/scsi-wwn'), mock.call('/dev/disk/by-id/dm-...'), mock.call('/dev/disk/by-id/scsi-...')]) self.assertEqual(6, mock_realpath.call_count) @mock.patch.object(iscsi.ISCSIConnector, '_run_iscsiadm_bare') def test_get_node_startup_values(self, run_iscsiadm_bare_mock): name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' name3 = 'volume-00000001-3' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 iqn3 = 'iqn.2010-10.org.openstack:%s' % name3 connection_properties = self.iscsi_connection(vol, [location], [iqn1]) node_startup1 = "manual" node_startup2 = "automatic" node_startup3 = "manual" node_values = ( '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n' '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n' '# BEGIN RECORD 2.0-873\n' 'node.name = %s\n' 'node.tpgt = 1\n' 'node.startup = %s\n' 'iface.hwaddress = \n' '# END RECORD\n') % (iqn1, node_startup1, iqn2, node_startup2, iqn3, node_startup3) run_iscsiadm_bare_mock.return_value = (node_values, None) node_startups =\ self.connector._get_node_startup_values( connection_properties['data']) expected_node_startups = {iqn1: node_startup1, iqn2: node_startup2, iqn3: node_startup3} self.assertEqual(node_startups, expected_node_startups) @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_get_node_startup_values_no_nodes(self, exec_mock): connection_properties = {'target_portal': 'ip1:port1'} no_nodes_output = '' no_nodes_err = 'iscsiadm: No records found\n' exec_mock.return_value = (no_nodes_output, no_nodes_err) res = self.connector._get_node_startup_values(connection_properties) self.assertEqual({}, res) exec_mock.assert_called_once_with( 'iscsiadm', '-m', 'node', '--op', 'show', '-p', connection_properties['target_portal'], root_helper=self.connector._root_helper, run_as_root=True, check_exit_code=(0, 21)) @mock.patch.object(iscsi.ISCSIConnector, '_get_node_startup_values') @mock.patch.object(iscsi.ISCSIConnector, '_iscsiadm_update') def test_recover_node_startup_values(self, iscsiadm_update_mock, get_node_startup_values_mock): name1 = 'volume-00000001-1' name2 = 'volume-00000001-2' name3 = 'volume-00000001-3' vol = {'id': 1, 'name': name1} location = '10.0.2.15:3260' iqn1 = 'iqn.2010-10.org.openstack:%s' % name1 iqn2 = 'iqn.2010-10.org.openstack:%s' % name2 iqn3 = 'iqn.2010-10.org.openstack:%s' % name3 connection_properties = self.iscsi_connection(vol, [location], iqn1) recover_connection = self.iscsi_connection(vol, [location], iqn2) node_startup1 = "manual" node_startup2 = "automatic" node_startup3 = "manual" get_node_startup_values_mock.return_value = {iqn1: node_startup1, iqn2: node_startup2, iqn3: node_startup3} old_node_startup_values = {iqn1: node_startup1, iqn2: "manual", iqn3: node_startup3} self.connector._recover_node_startup_values( connection_properties['data'], old_node_startup_values) iscsiadm_update_mock.assert_called_once_with( recover_connection['data'], "node.startup", "manual") ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_iser.py0000664000175000017500000000653000000000000024751 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import iscsi from os_brick.tests.initiator import test_connector class ISERConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(ISERConnectorTestCase, self).setUp() self.connector = iscsi.ISCSIConnector( None, execute=self.fake_execute, use_multipath=False) self.connection_data = { 'volume_id': 'volume_id', 'target_portal': 'ip:port', 'target_iqn': 'target_1', 'target_lun': 1, 'target_portals': ['ip:port'], 'target_iqns': ['target_1'], 'target_luns': [1] } @mock.patch.object(iscsi.ISCSIConnector, '_get_ips_iqns_luns') @mock.patch('glob.glob') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_nodes') def test_get_connection_devices( self, nodes_mock, sessions_mock, glob_mock, iql_mock): self.connector.use_multipath = True iql_mock.return_value = \ self.connector._get_all_targets(self.connection_data) # mocked iSCSI sessions sessions_mock.return_value = \ [('iser:', '0', 'ip:port', '1', 'target_1')] # mocked iSCSI nodes nodes_mock.return_value = [('ip:port', 'target_1')] sys_cls = '/sys/class/scsi_host/host' glob_mock.side_effect = [ [sys_cls + '1/device/session/target/1:1:1:1/block/sda'] ] res = self.connector._get_connection_devices(self.connection_data) expected = {('ip:port', 'target_1'): ({'sda'}, set())} self.assertDictEqual(expected, res) iql_mock.assert_called_once_with(self.connection_data, discover=False, is_disconnect_call=False) @mock.patch.object(iscsi.ISCSIConnector, '_get_iscsi_sessions_full') @mock.patch.object(iscsi.ISCSIConnector, '_execute') def test_connect_to_iscsi_portal(self, exec_mock, sessions_mock): """Connect to portal while session already established""" # connected sessions sessions_mock.side_effect = [ [('iser:', 'session_iser', 'ip:port', '1', 'target_1')] ] exec_mock.side_effect = [('', None), ('', None), ('', None)] res = self.connector._connect_to_iscsi_portal(self.connection_data) # session name is expected to be in the result. self.assertEqual(("session_iser", True), res) prefix = 'iscsiadm -m node -T target_1 -p ip:port' expected_cmds = [ prefix, prefix + ' --op update -n node.session.scan -v manual' ] actual_cmds = [' '.join(args[0]) for args in exec_mock.call_args_list] self.assertListEqual(expected_cmds, actual_cmds) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_local.py0000664000175000017500000000435400000000000025103 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.connectors import local from os_brick.tests.initiator import test_connector class LocalConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(LocalConnectorTestCase, self).setUp() self.connection_properties = {'name': 'foo', 'device_path': '/tmp/bar'} self.connector = local.LocalConnector(None) def test_get_connector_properties(self): props = local.LocalConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): actual = self.connector.get_search_path() self.assertIsNone(actual) def test_get_volume_paths(self): expected = [self.connection_properties['device_path']] actual = self.connector.get_volume_paths( self.connection_properties) self.assertEqual(expected, actual) def test_connect_volume(self): cprops = self.connection_properties dev_info = self.connector.connect_volume(cprops) self.assertEqual(dev_info['type'], 'local') self.assertEqual(dev_info['path'], cprops['device_path']) def test_connect_volume_with_invalid_connection_data(self): cprops = {} self.assertRaises(ValueError, self.connector.connect_volume, cprops) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_nvmeof.py0000664000175000017500000004752000000000000025305 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import ddt from oslo_concurrency import processutils as putils from os_brick import exception from os_brick.initiator.connectors import nvmeof from os_brick.initiator import linuxscsi from os_brick.tests.initiator import test_connector FAKE_NVME_LIST_OUTPUT = """ Node SN Model \ Namespace Usage Format FW Rev\n ---------------- -------------------- ---------------------------------------\ - --------- -------------------------- ---------------- --------\n /dev/nvme0n1 67ff9467da6e5567 Linux \ 10 1.07 GB / 1.07 GB 512 B + 0 B 4.8.0-58\n /dev/nvme11n12 fecc8e73584753d7 Linux \ 1 3.22 GB / 3.22 GB 512 B + 0 B 4.8.0-56\n """ FAKE_NVME_LIST_SUBSYS = """ { "Subsystems" : [ { "Name" : "nvme-subsys0", "NQN" : "nqn.fake:cnode1" }, { "Paths" : [ { "Name" : "nvme0", "Transport" : "rdma", "Address" : "traddr=10.0.2.15 trsvcid=4420" } ] }, { "Name" : "nvme-subsys1", "NQN" : "nqn.2016-06.io.spdk:cnode1" }, { "Paths" : [ { "Name" : "nvme1", "Transport" : "rdma", "Address" : "traddr=10.0.2.15 trsvcid=4420" } ] } ] } """ NVME_DATA1 = {'nvme_transport_type': 'rdma', 'conn_nqn': 'nqn.2016-06.io.spdk:cnode1', 'target_portal': '10.0.2.15', 'port': '4420'} NVME_DATA2 = {'nvme_transport_type': 'rdma', 'conn_nqn': 'nqn.2016-06.io.spdk:cnode2', 'target_portal': '10.0.2.15', 'port': '4420'} @ddt.ddt class NVMeOFConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for NVMe initiator class.""" def setUp(self): super(NVMeOFConnectorTestCase, self).setUp() self.connector = nvmeof.NVMeOFConnector(None, execute=self.fake_execute, use_multipath=False) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) def test_get_sysuuid_without_newline(self, mock_execute): mock_execute.return_value = ( "9126E942-396D-11E7-B0B7-A81E84C186D1\n", "") uuid = self.connector._get_system_uuid() expected_uuid = "9126E942-396D-11E7-B0B7-A81E84C186D1" self.assertEqual(expected_uuid, uuid) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) def test_get_connector_properties_without_sysuuid( self, mock_execute): mock_execute.side_effect = putils.ProcessExecutionError props = self.connector.get_connector_properties('sudo') expected_props = {} self.assertEqual(expected_props, props) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_system_uuid', autospec=True) def test_get_connector_properties_with_sysuuid( self, mock_sysuuid): mock_sysuuid.return_value = "9126E942-396D-11E7-B0B7-A81E84C186D1" props = self.connector.get_connector_properties('sudo') expected_props = { "system uuid": "9126E942-396D-11E7-B0B7-A81E84C186D1"} self.assertEqual(expected_props, props) def _nvmeof_list_cmd(self, *args, **kwargs): return FAKE_NVME_LIST_OUTPUT, None def test__get_nvme_devices(self): expected = ['/dev/nvme0n1', '/dev/nvme11n12'] self.connector._execute = self._nvmeof_list_cmd actual = self.connector._get_nvme_devices() self.assertEqual(expected, actual) @ddt.unpack @ddt.data({'expected': True, 'nvme': NVME_DATA1, 'list_subsys': FAKE_NVME_LIST_SUBSYS, 'nvme_list': ['/dev/nvme0n1', '/dev/nvme1n1']}, {'expected': False, 'nvme': NVME_DATA2, 'list_subsys': FAKE_NVME_LIST_SUBSYS, 'nvme_list': ['/dev/nvme1n1']}, {'expected': False, 'nvme': NVME_DATA1, 'list_subsys': '{}', 'nvme_list': ['dev/nvme1n1']}) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_subsys', autospec=True) @mock.patch('time.sleep', autospec=True) def test__wait_for_blk(self, mock_sleep, mock_nvme_subsys, mock_nvme_dev, expected, nvme, list_subsys, nvme_list): mock_nvme_subsys.return_value = (list_subsys, "") mock_nvme_dev.return_value = nvme_list actual = self.connector._wait_for_blk(**nvme) self.assertEqual(expected, actual) @ddt.unpack @ddt.data({'expected': False, 'nvme': NVME_DATA1, 'list_subsys': FAKE_NVME_LIST_SUBSYS, 'nvme_list': ['/dev/nvme0n1']}) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_subsys', autospec=True) @mock.patch('time.sleep', autospec=True) def test__wait_for_blk_raise(self, mock_sleep, mock_nvme_subsys, mock_nvme_dev, expected, nvme, list_subsys, nvme_list): mock_nvme_subsys.return_value = (list_subsys, "") mock_nvme_dev.return_value = nvme_list self.assertRaises(exception.NotFound, self.connector._wait_for_blk, **nvme) @ddt.unpack @ddt.data({'expected': True, 'nvme': NVME_DATA1, 'list_subsys': FAKE_NVME_LIST_SUBSYS, 'nvme_list': ['dev/nvme0n1', '/dev/nvme1n1']}) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_subsys', autospec=True) @mock.patch('time.sleep', autospec=True) def test__wait_for_blk_retry_success(self, mock_sleep, mock_nvme_subsys, mock_nvme_dev, expected, nvme, list_subsys, nvme_list): mock_nvme_subsys.return_value = (list_subsys, "") mock_nvme_dev.side_effect = [[], nvme_list] actual = self.connector._wait_for_blk(**nvme) self.assertEqual(expected, actual) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_get_nvme_devices_raise(self, mock_sleep, mock_execute): mock_execute.side_effect = putils.ProcessExecutionError self.assertRaises(exception.CommandExecutionFailed, self.connector._get_nvme_devices) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume(self, mock_sleep, mock_execute, mock_devices, mock_blk): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.side_effect = [ ['/dev/nvme0n1'], ['/dev/nvme0n2']] mock_blk.return_value = True device_info = self.connector.connect_volume( connection_properties) self.assertEqual('/dev/nvme0n2', device_info['path']) self.assertEqual('block', device_info['type']) self.assertEqual(2, mock_devices.call_count) mock_execute.assert_called_once_with( self.connector, 'nvme', 'connect', '-t', connection_properties['transport_type'], '-n', 'nqn.volume_123', '-a', connection_properties['target_portal'], '-s', connection_properties['target_port'], root_helper=None, run_as_root=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume_hostnqn( self, mock_sleep, mock_execute, mock_devices, mock_blk): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma', 'host_nqn': 'nqn.host_456'} mock_devices.side_effect = [ ['/dev/nvme0n1'], ['/dev/nvme0n2']] mock_blk.return_value = True device_info = self.connector.connect_volume( connection_properties) self.assertEqual('/dev/nvme0n2', device_info['path']) self.assertEqual('block', device_info['type']) self.assertEqual(2, mock_devices.call_count) mock_execute.assert_called_once_with( self.connector, 'nvme', 'connect', '-t', connection_properties['transport_type'], '-n', connection_properties['nqn'], '-a', connection_properties['target_portal'], '-s', connection_properties['target_port'], '-q', connection_properties['host_nqn'], root_helper=None, run_as_root=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume_raise(self, mock_sleep, mock_execute): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_execute.side_effect = putils.ProcessExecutionError self.assertRaises(exception.CommandExecutionFailed, self.connector.connect_volume, connection_properties) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_subsys', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume_wait_for_blk_raise(self, mock_sleep, mock_blk, mock_subsys, mock_devices, mock_execute): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_blk.side_effect = exception.NotFound self.assertRaises(exception.NotFound, self.connector.connect_volume, connection_properties) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume_max_retry( self, mock_sleep, mock_execute, mock_devices, mock_blk): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.return_value = '/dev/nvme0n1' mock_blk.return_value = True self.assertRaises(exception.VolumePathsNotFound, self.connector.connect_volume, connection_properties) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_volume_nvmelist_retry_success( self, mock_sleep, mock_execute, mock_devices, mock_blk): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.side_effect = [ ['/dev/nvme0n1'], ['/dev/nvme0n1'], ['/dev/nvme0n1', '/dev/nvme0n2']] mock_blk.return_value = True device_info = self.connector.connect_volume( connection_properties) self.assertEqual('/dev/nvme0n2', device_info['path']) self.assertEqual('block', device_info['type']) @mock.patch.object(nvmeof.NVMeOFConnector, '_wait_for_blk', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_connect_nvme_retry_success( self, mock_sleep, mock_execute, mock_devices, mock_blk): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} mock_devices.side_effect = [ ['/dev/nvme0n1'], ['/dev/nvme0n1', '/dev/nvme0n2']] mock_blk.return_value = True device_info = self.connector.connect_volume( connection_properties) mock_execute.side_effect = [ putils.ProcessExecutionError, putils.ProcessExecutionError, None] self.assertEqual('/dev/nvme0n2', device_info['path']) self.assertEqual('block', device_info['type']) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_disconnect_volume_nova( self, mock_sleep, mock_execute, mock_devices): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '/dev/nvme0n1', 'transport_type': 'rdma'} mock_devices.return_value = '/dev/nvme0n1' self.connector.disconnect_volume(connection_properties, None) mock_execute.assert_called_once_with( self.connector, 'nvme', 'disconnect', '-n', 'nqn.volume_123', root_helper=None, run_as_root=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_disconnect_volume_cinder( self, mock_sleep, mock_execute, mock_devices): connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'transport_type': 'rdma'} device_info = {'path': '/dev/nvme0n1'} mock_devices.return_value = '/dev/nvme0n1' self.connector.disconnect_volume(connection_properties, device_info, ignore_errors=True) mock_execute.assert_called_once_with( self.connector, 'nvme', 'disconnect', '-n', 'nqn.volume_123', root_helper=None, run_as_root=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_get_nvme_devices', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, '_execute', autospec=True) @mock.patch('time.sleep', autospec=True) def test_disconnect_volume_raise( self, mock_sleep, mock_execute, mock_devices): mock_execute.side_effect = putils.ProcessExecutionError mock_devices.return_value = '/dev/nvme0n1' connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '/dev/nvme0n1', 'transport_type': 'rdma'} self.assertRaises(putils.ProcessExecutionError, self.connector.disconnect_volume, connection_properties, None) @mock.patch.object(nvmeof.NVMeOFConnector, 'get_volume_paths', autospec=True) def test_extend_volume_no_path(self, mock_volume_paths): mock_volume_paths.return_value = [] connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} self.assertRaises(exception.VolumePathsNotFound, self.connector.extend_volume, connection_properties) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path', autospec=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'extend_volume', autospec=True) @mock.patch.object(nvmeof.NVMeOFConnector, 'get_volume_paths', autospec=True) def test_extend_volume(self, mock_volume_paths, mock_scsi_extend, mock_scsi_find_mpath): fake_new_size = 1024 mock_volume_paths.return_value = ['/dev/vdx'] mock_scsi_extend.return_value = fake_new_size connection_properties = {'target_portal': 'portal', 'target_port': 1, 'nqn': 'nqn.volume_123', 'device_path': '', 'transport_type': 'rdma'} new_size = self.connector.extend_volume(connection_properties) self.assertEqual(fake_new_size, new_size) self.assertFalse(mock_scsi_find_mpath.called) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_rbd.py0000664000175000017500000003515400000000000024562 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import exception from os_brick.initiator.connectors import rbd from os_brick.initiator import linuxrbd from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests.initiator import test_connector from os_brick import utils @ddt.ddt class RBDConnectorTestCase(test_connector.ConnectorTestCase): def setUp(self): super(RBDConnectorTestCase, self).setUp() self.user = 'fake_user' self.pool = 'fake_pool' self.volume = 'fake_volume' self.clustername = 'fake_ceph' self.hosts = ['192.168.10.2'] self.ports = ['6789'] self.keyring = "[client.cinder]\n key = test\n" self.connection_properties = { 'auth_username': self.user, 'name': '%s/%s' % (self.pool, self.volume), 'cluster_name': self.clustername, 'hosts': self.hosts, 'ports': self.ports, 'keyring': self.keyring, } def test_get_search_path(self): rbd_connector = rbd.RBDConnector(None) path = rbd_connector.get_search_path() self.assertIsNone(path) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') def test_get_volume_paths(self, mock_rados, mock_rbd): rbd_connector = rbd.RBDConnector(None) expected = [] actual = rbd_connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) def test_get_connector_properties(self): props = rbd.RBDConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {'do_local_attach': False} self.assertEqual(expected_props, props) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(rbd.RBDConnector, '_create_ceph_conf') @mock.patch('os.path.exists') def test_connect_volume(self, mock_path, mock_conf, mock_rados, mock_rbd): """Test the connect volume case.""" rbd_connector = rbd.RBDConnector(None) mock_path.return_value = False mock_conf.return_value = "/tmp/fake_dir/fake_ceph.conf" device_info = rbd_connector.connect_volume(self.connection_properties) # Ensure rados is instantiated correctly mock_rados.Rados.assert_called_once_with( clustername=self.clustername, rados_id=utils.convert_str(self.user), conffile='/tmp/fake_dir/fake_ceph.conf') # Ensure correct calls to connect to cluster self.assertEqual(1, mock_rados.Rados.return_value.connect.call_count) mock_rados.Rados.return_value.open_ioctx.assert_called_once_with( utils.convert_str(self.pool)) # Ensure rbd image is instantiated correctly mock_rbd.Image.assert_called_once_with( mock_rados.Rados.return_value.open_ioctx.return_value, utils.convert_str(self.volume), read_only=False, snapshot=None) # Ensure expected object is returned correctly self.assertIsInstance(device_info['path'], linuxrbd.RBDVolumeIOWrapper) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(rbd.RBDConnector, '_create_ceph_conf') @mock.patch('os.path.exists') def test_provided_keyring(self, mock_path, mock_conf, mock_rados, mock_rbd): conn = rbd.RBDConnector(None) mock_path.return_value = False mock_conf.return_value = "/tmp/fake_dir/fake_ceph.conf" self.connection_properties['keyring'] = self.keyring conn.connect_volume(self.connection_properties) mock_conf.assert_called_once_with(self.hosts, self.ports, self.clustername, self.user, self.keyring) def test_keyring_is_none(self): conn = rbd.RBDConnector(None) keyring = None keyring_data = "[client.cinder]\n key = test\n" mockopen = mock.mock_open(read_data=keyring_data) mockopen.return_value.__exit__ = mock.Mock() with mock.patch('os_brick.initiator.connectors.rbd.open', mockopen, create=True): self.assertEqual( conn._check_or_get_keyring_contents(keyring, 'cluster', 'user'), keyring_data) self.assertEqual( conn._check_or_get_keyring_contents(keyring, 'cluster', None), '') def test_keyring_raise_error(self): conn = rbd.RBDConnector(None) keyring = None mockopen = mock.mock_open() mockopen.return_value = "" with mock.patch('os_brick.initiator.connectors.rbd.open', mockopen, create=True) as mock_keyring_file: mock_keyring_file.side_effect = IOError self.assertRaises(exception.BrickException, conn._check_or_get_keyring_contents, keyring, 'cluster', 'user') @ddt.data((['192.168.1.1', '192.168.1.2'], ['192.168.1.1', '192.168.1.2']), (['3ffe:1900:4545:3:200:f8ff:fe21:67cf', 'fe80:0:0:0:200:f8ff:fe21:67cf'], ['[3ffe:1900:4545:3:200:f8ff:fe21:67cf]', '[fe80:0:0:0:200:f8ff:fe21:67cf]']), (['foobar', 'fizzbuzz'], ['foobar', 'fizzbuzz']), (['192.168.1.1', '3ffe:1900:4545:3:200:f8ff:fe21:67cf', 'hello, world!'], ['192.168.1.1', '[3ffe:1900:4545:3:200:f8ff:fe21:67cf]', 'hello, world!'])) @ddt.unpack def test_sanitize_mon_host(self, hosts_in, hosts_out): conn = rbd.RBDConnector(None) self.assertEqual(hosts_out, conn._sanitize_mon_hosts(hosts_in)) @mock.patch('os_brick.initiator.connectors.rbd.tempfile.mkstemp') def test_create_ceph_conf(self, mock_mkstemp): mockopen = mock.mock_open() fd = mock.sentinel.fd tmpfile = mock.sentinel.tmpfile mock_mkstemp.return_value = (fd, tmpfile) with mock.patch('os.fdopen', mockopen, create=True): rbd_connector = rbd.RBDConnector(None) conf_path = rbd_connector._create_ceph_conf( self.hosts, self.ports, self.clustername, self.user, self.keyring) self.assertEqual(conf_path, tmpfile) mock_mkstemp.assert_called_once_with(prefix='brickrbd_') # Bug #1865754 - make sure generated config file has a '[global]' # section _, args, _ = mockopen().writelines.mock_calls[0] self.assertIn('[global]', args[0]) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1, execute_call2]) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) @mock.patch('os.path.exists') @mock.patch('os.path.islink') @mock.patch('os.path.realpath') def test_connect_local_volume_dev_exist(self, mock_realpath, mock_islink, mock_exists, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} mock_realpath.return_value = '/dev/rbd0' mock_islink.return_value = True mock_exists.return_value = True device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1]) self.assertFalse(execute_call2 in mock_execute.mock_calls) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume_without_mons(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user'} device_info = rbd_connector.connect_volume(conn) execute_call1 = mock.call('which', 'rbd') cmd = ['rbd', 'map', 'image', '--pool', 'pool', '--id', 'fake_user'] execute_call2 = mock.call(*cmd, root_helper=None, run_as_root=True) mock_execute.assert_has_calls([execute_call1, execute_call2]) expected_info = {'path': '/dev/rbd/pool/image', 'type': 'block'} self.assertEqual(expected_info, device_info) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_connect_local_volume_without_auth(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'hosts': ['192.168.10.2'], 'ports': ['6789']} self.assertRaises(exception.BrickException, rbd_connector.connect_volume, conn) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(linuxrbd.RBDVolumeIOWrapper, 'close') def test_disconnect_volume(self, volume_close, mock_rados, mock_rbd): """Test the disconnect volume case.""" rbd_connector = rbd.RBDConnector(None) device_info = rbd_connector.connect_volume(self.connection_properties) rbd_connector.disconnect_volume( self.connection_properties, device_info) self.assertEqual(1, volume_close.call_count) @ddt.data( """ [{"id":"0","pool":"pool","device":"/dev/rbd0","name":"image"}, {"id":"1","pool":"pool","device":"/dev/rdb1","name":"image_2"}] """, # new-style output (ceph 13.2.0+) """ {"0":{"pool":"pool","device":"/dev/rbd0","name":"image"}, "1":{"pool":"pool","device":"/dev/rdb1","name":"image_2"}} """, # old-style output ) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_disconnect_local_volume(self, rbd_map_out, mock_execute): """Test the disconnect volume case with local attach.""" rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} mock_execute.side_effect = [(rbd_map_out, None), (None, None)] show_cmd = ['rbd', 'showmapped', '--format=json', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] unmap_cmd = ['rbd', 'unmap', '/dev/rbd0', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] rbd_connector.disconnect_volume(conn, None) # Assert that showmapped is used before we unmap the root device mock_execute.assert_has_calls([ mock.call(*show_cmd, root_helper=None, run_as_root=True), mock.call(*unmap_cmd, root_helper=None, run_as_root=True)]) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_disconnect_local_volume_no_mapping(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/not_mapped', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} mock_execute.return_value = (""" {"0":{"pool":"pool","device":"/dev/rbd0","name":"pool-image"}, "1":{"pool":"pool","device":"/dev/rdb1","name":"pool-image_2"}}""", None) show_cmd = ['rbd', 'showmapped', '--format=json', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] rbd_connector.disconnect_volume(conn, None) # Assert that only showmapped is called when no mappings are found mock_execute.called_once_with(*show_cmd, root_helper=None, run_as_root=True) @mock.patch.object(priv_rootwrap, 'execute', return_value=None) def test_disconnect_local_volume_no_mappings(self, mock_execute): rbd_connector = rbd.RBDConnector(None, do_local_attach=True) conn = {'name': 'pool/image', 'auth_username': 'fake_user', 'hosts': ['192.168.10.2'], 'ports': ['6789']} mock_execute.return_value = ("{}", None) show_cmd = ['rbd', 'showmapped', '--format=json', '--id', 'fake_user', '--mon_host', '192.168.10.2:6789'] rbd_connector.disconnect_volume(conn, None) # Assert that only showmapped is called when no mappings are found mock_execute.called_once_with(*show_cmd, root_helper=None, run_as_root=True) def test_extend_volume(self): rbd_connector = rbd.RBDConnector(None) self.assertRaises(NotImplementedError, rbd_connector.extend_volume, self.connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_remotefs.py0000664000175000017500000000635500000000000025640 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator.connectors import remotefs from os_brick.remotefs import remotefs as remotefs_client from os_brick.tests.initiator import test_connector class RemoteFsConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for Remote FS initiator class.""" TEST_DEV = '172.18.194.100:/var/nfs' TEST_PATH = '/mnt/test/df0808229363aad55c27da50c38d6328' TEST_BASE = '/mnt/test' TEST_NAME = '9c592d52-ce47-4263-8c21-4ecf3c029cdb' def setUp(self): super(RemoteFsConnectorTestCase, self).setUp() self.connection_properties = { 'export': self.TEST_DEV, 'name': self.TEST_NAME} self.connector = remotefs.RemoteFsConnector( 'nfs', root_helper='sudo', nfs_mount_point_base=self.TEST_BASE, nfs_mount_options='vers=3') @mock.patch('os_brick.remotefs.remotefs.ScalityRemoteFsClient') def test_init_with_scality(self, mock_scality_remotefs_client): remotefs.RemoteFsConnector('scality', root_helper='sudo') self.assertEqual(1, mock_scality_remotefs_client.call_count) def test_get_connector_properties(self): props = remotefs.RemoteFsConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_get_search_path(self): expected = self.TEST_BASE actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(remotefs_client.RemoteFsClient, 'mount') def test_get_volume_paths(self, mock_mount): path = ("%(path)s/%(name)s" % {'path': self.TEST_PATH, 'name': self.TEST_NAME}) expected = [path] actual = self.connector.get_volume_paths(self.connection_properties) self.assertEqual(expected, actual) @mock.patch.object(remotefs_client.RemoteFsClient, 'mount') @mock.patch.object(remotefs_client.RemoteFsClient, 'get_mount_point', return_value="something") def test_connect_volume(self, mount_point_mock, mount_mock): """Test the basic connect volume case.""" self.connector.connect_volume(self.connection_properties) def test_disconnect_volume(self): """Nothing should happen here -- make sure it doesn't blow up.""" self.connector.disconnect_volume(self.connection_properties, {}) def test_extend_volume(self): self.assertRaises(NotImplementedError, self.connector.extend_volume, self.connection_properties) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_scaleio.py0000664000175000017500000003255300000000000025432 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json import mock import os import requests import six from os_brick import exception from os_brick.initiator.connectors import scaleio from os_brick.tests.initiator import test_connector class ScaleIOConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for ScaleIO connector.""" # Fake volume information vol = { 'id': 'vol1', 'name': 'test_volume', 'provider_id': 'vol1' } # Fake SDC GUID fake_guid = '013a5304-d053-4b30-a34f-ee3ad983236d' def setUp(self): super(ScaleIOConnectorTestCase, self).setUp() self.fake_connection_properties = { 'hostIP': test_connector.MY_IP, 'serverIP': test_connector.MY_IP, 'scaleIO_volname': self.vol['name'], 'scaleIO_volume_id': self.vol['provider_id'], 'serverPort': 443, 'serverUsername': 'test', 'config_group': 'test', 'failed_over': False, 'iopsLimit': None, 'bandwidthLimit': None } # Formatting string for REST API calls self.action_format = "instances/Volume::{}/action/{{}}".format( self.vol['id']) self.get_volume_api = 'types/Volume/instances/getByName::{}'.format( self.vol['name']) # Map of REST API calls to responses self.mock_calls = { self.get_volume_api: self.MockHTTPSResponse(json.dumps(self.vol['id'])), self.action_format.format('addMappedSdc'): self.MockHTTPSResponse(''), self.action_format.format('setMappedSdcLimits'): self.MockHTTPSResponse(''), self.action_format.format('removeMappedSdc'): self.MockHTTPSResponse(''), } # Default error REST response self.error_404 = self.MockHTTPSResponse(content=dict( errorCode=0, message='HTTP 404', ), status_code=404) # Patch the request and os calls to fake versions self.mock_object(requests, 'get', self.handle_scaleio_request) self.mock_object(requests, 'post', self.handle_scaleio_request) self.mock_object(os.path, 'isdir', return_value=True) self.mock_object(os, 'listdir', return_value=["emc-vol-{}".format(self.vol['id'])]) # Patch scaleio privileged calls self.get_password_mock = self.mock_object(scaleio.priv_scaleio, 'get_connector_password', return_value='fake_password') self.get_guid_mock = self.mock_object(scaleio.priv_scaleio, 'get_guid', return_value=self.fake_guid) self.rescan_vols_mock = self.mock_object(scaleio.priv_scaleio, 'rescan_vols') # The actual ScaleIO connector self.connector = scaleio.ScaleIOConnector( 'sudo', execute=self.fake_execute) class MockHTTPSResponse(requests.Response): """Mock HTTP Response Defines the https replies from the mocked calls to do_request() """ def __init__(self, content, status_code=200): super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).__init__() self._content = content self.encoding = 'UTF-8' self.status_code = status_code def json(self, **kwargs): if isinstance(self._content, six.string_types): return super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).json(**kwargs) return self._content @property def text(self): if not isinstance(self._content, six.string_types): return json.dumps(self._content) self._content = self._content.encode('utf-8') return super(ScaleIOConnectorTestCase.MockHTTPSResponse, self).text def handle_scaleio_request(self, url, *args, **kwargs): """Fake REST server""" api_call = url.split(':', 2)[2].split('/', 1)[1].replace('api/', '') if 'setMappedSdcLimits' in api_call: self.assertNotIn("iops_limit", kwargs['data']) if "iopsLimit" not in kwargs['data']: self.assertIn("bandwidthLimitInKbps", kwargs['data']) elif "bandwidthLimitInKbps" not in kwargs['data']: self.assertIn("iopsLimit", kwargs['data']) else: self.assertIn("bandwidthLimitInKbps", kwargs['data']) self.assertIn("iopsLimit", kwargs['data']) try: return self.mock_calls[api_call] except KeyError: return self.error_404 def test_get_search_path(self): expected = "/dev/disk/by-id" actual = self.connector.get_search_path() self.assertEqual(expected, actual) @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(scaleio.ScaleIOConnector, '_wait_for_volume_path') def test_get_volume_paths(self, mock_wait_for_path, mock_exists): mock_wait_for_path.return_value = "emc-vol-vol1" expected = ['/dev/disk/by-id/emc-vol-vol1'] actual = self.connector.get_volume_paths( self.fake_connection_properties) self.assertEqual(expected, actual) def test_get_connector_properties(self): props = scaleio.ScaleIOConnector.get_connector_properties( 'sudo', multipath=True, enforce_multipath=True) expected_props = {} self.assertEqual(expected_props, props) def test_connect_volume(self): """Successful connect to volume""" self.connector.connect_volume(self.fake_connection_properties) self.get_guid_mock.assert_called_once_with( self.connector.GET_GUID_OP_CODE) self.get_password_mock.assert_called_once() def test_connect_volume_old_connection_properties(self): """Successful connect to volume""" connection_properties = { 'hostIP': test_connector.MY_IP, 'serverIP': test_connector.MY_IP, 'scaleIO_volname': self.vol['name'], 'scaleIO_volume_id': self.vol['provider_id'], 'serverPort': 443, 'serverUsername': 'test', 'serverPassword': 'fake', 'serverToken': 'fake_token', 'iopsLimit': None, 'bandwidthLimit': None } self.connector.connect_volume(connection_properties) self.get_guid_mock.assert_called_once_with( self.connector.GET_GUID_OP_CODE) self.get_password_mock.assert_not_called() def test_connect_volume_without_volume_id(self): """Successful connect to volume without a Volume Id""" connection_properties = dict(self.fake_connection_properties) connection_properties.pop('scaleIO_volume_id') self.connector.connect_volume(connection_properties) self.get_guid_mock.assert_called_once_with( self.connector.GET_GUID_OP_CODE) def test_connect_with_bandwidth_limit(self): """Successful connect to volume with bandwidth limit""" self.fake_connection_properties['bandwidthLimit'] = '500' self.test_connect_volume() def test_connect_with_iops_limit(self): """Successful connect to volume with iops limit""" self.fake_connection_properties['iopsLimit'] = '80' self.test_connect_volume() def test_connect_with_iops_and_bandwidth_limits(self): """Successful connect with iops and bandwidth limits""" self.fake_connection_properties['bandwidthLimit'] = '500' self.fake_connection_properties['iopsLimit'] = '80' self.test_connect_volume() def test_disconnect_volume(self): """Successful disconnect from volume""" self.connector.disconnect_volume(self.fake_connection_properties, None) self.get_guid_mock.assert_called_once_with( self.connector.GET_GUID_OP_CODE) def test_disconnect_volume_without_volume_id(self): """Successful disconnect from volume without a Volume Id""" connection_properties = dict(self.fake_connection_properties) connection_properties.pop('scaleIO_volume_id') self.connector.disconnect_volume(connection_properties, None) self.get_guid_mock.assert_called_once_with( self.connector.GET_GUID_OP_CODE) def test_error_id(self): """Fail to connect with bad volume name""" self.fake_connection_properties['scaleIO_volume_id'] = 'bad_id' self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( dict(errorCode='404', message='Test volume not found'), 404) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_no_volume_id(self): """Faile to connect with no volume id""" self.fake_connection_properties['scaleIO_volume_id'] = None self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( 'null', 200) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_bad_login(self): """Fail to connect with bad authentication""" self.mock_calls[self.get_volume_api] = self.MockHTTPSResponse( 'null', 401) self.mock_calls['login'] = self.MockHTTPSResponse('null', 401) self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=401, message='bad login'), 401) self.assertRaises(exception.BrickException, self.test_connect_volume) def test_error_map_volume(self): """Fail to connect with REST API failure""" self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_NOT_MAPPED_ERROR, message='Test error map volume'), 500) self.assertRaises(exception.BrickException, self.test_connect_volume) @mock.patch('time.sleep') def test_error_path_not_found(self, sleep_mock): """Timeout waiting for volume to map to local file system""" self.mock_object(os, 'listdir', return_value=["emc-vol-no-volume"]) self.assertRaises(exception.BrickException, self.test_connect_volume) self.assertTrue(sleep_mock.called) def test_map_volume_already_mapped(self): """Ignore REST API failure for volume already mapped""" self.mock_calls[self.action_format.format( 'addMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_ALREADY_MAPPED_ERROR, message='Test error map volume'), 500) self.test_connect_volume() def test_error_disconnect_volume(self): """Fail to disconnect with REST API failure""" self.mock_calls[self.action_format.format( 'removeMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_ALREADY_MAPPED_ERROR, message='Test error map volume'), 500) self.assertRaises(exception.BrickException, self.test_disconnect_volume) def test_disconnect_volume_not_mapped(self): """Ignore REST API failure for volume not mapped""" self.mock_calls[self.action_format.format( 'removeMappedSdc')] = self.MockHTTPSResponse( dict(errorCode=self.connector.VOLUME_NOT_MAPPED_ERROR, message='Test error map volume'), 500) self.test_disconnect_volume() @mock.patch.object(os.path, 'exists', return_value=True) @mock.patch.object(scaleio.ScaleIOConnector, '_find_volume_path') @mock.patch.object(scaleio.ScaleIOConnector, 'get_device_size') def test_extend_volume(self, mock_device_size, mock_find_volume_path, mock_exists): mock_device_size.return_value = 16 mock_find_volume_path.return_value = "emc-vol-vol1" extended_size = self.connector.extend_volume( self.fake_connection_properties) self.assertEqual(extended_size, mock_device_size.return_value) self.rescan_vols_mock.assert_called_once_with( self.connector.RESCAN_VOLS_OP_CODE) def test_connection_properties_without_failed_over(self): """Handle connection properties with 'failed_over' missing""" connection_properties = dict(self.fake_connection_properties) connection_properties.pop('failed_over') self.connector.connect_volume(connection_properties) self.get_password_mock.assert_called_once_with( scaleio.CONNECTOR_CONF_PATH, connection_properties['config_group'], False) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_storpool.py0000664000175000017500000001524100000000000025667 0ustar00zuulzuul00000000000000# Copyright (c) 2015 - 2017 StorPool # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick import exception from os_brick.initiator.connectors import storpool as connector from os_brick.tests.initiator import test_connector def volumeNameExt(vid): return 'os--volume--{id}'.format(id=vid) class MockStorPoolADB(object): def __init__(self, log): self.requests = {} self.attached = {} def api(self): pass def add(self, req_id, req): if req_id in self.requests: raise Exception('Duplicate MockStorPool request added') self.requests[req_id] = req def remove(self, req_id): req = self.requests.get(req_id, None) if req is None: raise Exception('Unknown MockStorPool request removed') elif req['volume'] in self.attached: raise Exception('Removing attached MockStorPool volume') del self.requests[req_id] def sync(self, req_id, detached): req = self.requests.get(req_id, None) if req is None: raise Exception('Unknown MockStorPool request synced') volume = req.get('volume', None) if volume is None: raise Exception('MockStorPool request without volume') if detached is None: if volume in self.attached: raise Exception('Duplicate MockStorPool request synced') self.attached[volume] = req else: if volume != detached: raise Exception( 'Mismatched volumes on a MockStorPool request removal') elif detached not in self.attached: raise Exception('MockStorPool request not attached yet') del self.attached[detached] def volumeName(self, vid): return volumeNameExt(vid) spopenstack = mock.Mock() spopenstack.AttachDB = MockStorPoolADB connector.spopenstack = spopenstack class StorPoolConnectorTestCase(test_connector.ConnectorTestCase): def volumeName(self, vid): return volumeNameExt(vid) def get_fake_size(self): return self.fakeSize def execute(self, *cmd, **kwargs): if cmd[0] == 'blockdev': self.assertEqual(len(cmd), 3) self.assertEqual(cmd[1], '--getsize64') self.assertEqual(cmd[2], '/dev/storpool/' + self.volumeName(self.fakeProp['volume'])) return (str(self.get_fake_size()) + '\n', None) raise Exception("Unrecognized command passed to " + type(self).__name__ + ".execute(): " + str.join(", ", map(lambda s: "'" + s + "'", cmd))) def setUp(self): super(StorPoolConnectorTestCase, self).setUp() self.fakeProp = { 'volume': 'sp-vol-1', 'client_id': 1, 'access_mode': 'rw', } self.fakeConnection = None self.fakeSize = 1024 * 1024 * 1024 self.connector = connector.StorPoolConnector( None, execute=self.execute) self.adb = self.connector._attach def test_connect_volume(self): self.assertNotIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) conn = self.connector.connect_volume(self.fakeProp) self.assertIn('type', conn) self.assertIn('path', conn) self.assertIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) self.assertEqual(self.connector.get_search_path(), '/dev/storpool') paths = self.connector.get_volume_paths(self.fakeProp) self.assertEqual(len(paths), 1) self.assertEqual(paths[0], "/dev/storpool/" + self.volumeName(self.fakeProp['volume'])) self.fakeConnection = conn def test_disconnect_volume(self): if self.fakeConnection is None: self.test_connect_volume() self.assertIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) self.connector.disconnect_volume(self.fakeProp, None) self.assertNotIn(self.volumeName(self.fakeProp['volume']), self.adb.attached) def test_connect_exceptions(self): """Raise exceptions on missing connection information""" fake = self.fakeProp for key in fake.keys(): c = dict(fake) del c[key] self.assertRaises(exception.BrickException, self.connector.connect_volume, c) if key != 'access_mode': self.assertRaises(exception.BrickException, self.connector.disconnect_volume, c, None) def test_extend_volume(self): if self.fakeConnection is None: self.test_connect_volume() self.fakeSize += 1024 * 1024 * 1024 size_list = [self.fakeSize, self.fakeSize - 1, self.fakeSize - 2] vdata = mock.MagicMock(spec=['size']) vdata.size = self.fakeSize vdata_list = [[vdata]] def fake_volume_list(name): self.assertEqual( name, self.adb.volumeName(self.fakeProp['volume']) ) return vdata_list.pop() api = mock.MagicMock(spec=['volumeList']) api.volumeList = mock.MagicMock(spec=['__call__']) with mock.patch.object( self.adb, attribute='api', spec=['__call__'] ) as fake_api, mock.patch.object( self, attribute='get_fake_size', spec=['__call__'] ) as fake_size, mock.patch('time.sleep') as fake_sleep: fake_api.return_value = api api.volumeList.side_effect = fake_volume_list fake_size.side_effect = size_list.pop newSize = self.connector.extend_volume(self.fakeProp) self.assertEqual(fake_api.call_count, 1) self.assertEqual(api.volumeList.call_count, 1) self.assertListEqual(vdata_list, []) self.assertEqual(fake_size.call_count, 3) self.assertListEqual(size_list, []) self.assertEqual(fake_sleep.call_count, 2) self.assertEqual(newSize, self.fakeSize) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_vmware.py0000664000175000017500000004352700000000000025317 0ustar00zuulzuul00000000000000# Copyright (c) 2016 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from oslo_utils import units from oslo_vmware.objects import datastore from oslo_vmware import vim_util from os_brick import exception from os_brick.initiator.connectors import vmware from os_brick.tests.initiator import test_connector @ddt.ddt class VmdkConnectorTestCase(test_connector.ConnectorTestCase): IP = '127.0.0.1' PORT = 443 USERNAME = 'username' PASSWORD = 'password' API_RETRY_COUNT = 3 TASK_POLL_INTERVAL = 5.0 CA_FILE = "/etc/ssl/rui-ca-cert.pem" TMP_DIR = "/vmware-tmp" IMG_TX_TIMEOUT = 10 VMDK_CONNECTOR = vmware.VmdkConnector def setUp(self): super(VmdkConnectorTestCase, self).setUp() self._connector = vmware.VmdkConnector(None) self._connector._ip = self.IP self._connector._port = self.PORT self._connector._username = self.USERNAME self._connector._password = self.PASSWORD self._connector._api_retry_count = self.API_RETRY_COUNT self._connector._task_poll_interval = self.TASK_POLL_INTERVAL self._connector._ca_file = self.CA_FILE self._connector._insecure = True self._connector._tmp_dir = self.TMP_DIR self._connector._timeout = self.IMG_TX_TIMEOUT def test_load_config(self): config = { 'vmware_host_ip': 'localhost', 'vmware_host_port': 1234, 'vmware_host_username': 'root', 'vmware_host_password': 'pswd', 'vmware_api_retry_count': 1, 'vmware_task_poll_interval': 1.0, 'vmware_ca_file': None, 'vmware_insecure': False, 'vmware_tmp_dir': '/tmp', 'vmware_image_transfer_timeout_secs': 5, } self._connector._load_config({'config': config}) self.assertEqual('localhost', self._connector._ip) self.assertEqual(1234, self._connector._port) self.assertEqual('root', self._connector._username) self.assertEqual('pswd', self._connector._password) self.assertEqual(1, self._connector._api_retry_count) self.assertEqual(1.0, self._connector._task_poll_interval) self.assertIsNone(self._connector._ca_file) self.assertFalse(self._connector._insecure) self.assertEqual('/tmp', self._connector._tmp_dir) self.assertEqual(5, self._connector._timeout) @mock.patch('oslo_vmware.api.VMwareAPISession') def test_create_session(self, session): session.return_value = mock.sentinel.session ret = self._connector._create_session() self.assertEqual(mock.sentinel.session, ret) session.assert_called_once_with( self._connector._ip, self._connector._username, self._connector._password, self._connector._api_retry_count, self._connector._task_poll_interval, port=self._connector._port, cacert=self._connector._ca_file, insecure=self._connector._insecure) @mock.patch('oslo_utils.fileutils.ensure_tree') @mock.patch('tempfile.mkstemp') @mock.patch('os.close') def test_create_temp_file( self, close, mkstemp, ensure_tree): fd = mock.sentinel.fd tmp = mock.sentinel.tmp mkstemp.return_value = (fd, tmp) prefix = ".vmdk" suffix = "test" ret = self._connector._create_temp_file(prefix=prefix, suffix=suffix) self.assertEqual(tmp, ret) ensure_tree.assert_called_once_with(self._connector._tmp_dir) mkstemp.assert_called_once_with(dir=self._connector._tmp_dir, prefix=prefix, suffix=suffix) close.assert_called_once_with(fd) @mock.patch('os_brick.initiator.connectors.vmware.open', create=True) @mock.patch('oslo_vmware.image_transfer.copy_stream_optimized_disk') def test_download_vmdk(self, copy_disk, file_open): file_open_ret = mock.Mock() tmp_file = mock.sentinel.tmp_file file_open_ret.__enter__ = mock.Mock(return_value=tmp_file) file_open_ret.__exit__ = mock.Mock(return_value=None) file_open.return_value = file_open_ret tmp_file_path = mock.sentinel.tmp_file_path session = mock.sentinel.session backing = mock.sentinel.backing vmdk_path = mock.sentinel.vmdk_path vmdk_size = mock.sentinel.vmdk_size self._connector._download_vmdk( tmp_file_path, session, backing, vmdk_path, vmdk_size) file_open.assert_called_once_with(tmp_file_path, 'wb') copy_disk.assert_called_once_with(None, self._connector._timeout, tmp_file, session=session, host=self._connector._ip, port=self._connector._port, vm=backing, vmdk_file_path=vmdk_path, vmdk_size=vmdk_size) def _create_connection_properties(self): return {'volume_id': 'ed083474-d325-4a99-b301-269111654f0d', 'volume': 'ref-1', 'vmdk_path': '[ds] foo/bar.vmdk', 'vmdk_size': units.Gi, 'datastore': 'ds-1', 'datacenter': 'dc-1', } @mock.patch.object(VMDK_CONNECTOR, '_load_config') @mock.patch.object(VMDK_CONNECTOR, '_create_session') @mock.patch.object(VMDK_CONNECTOR, '_create_temp_file') @mock.patch('oslo_vmware.vim_util.get_moref') @mock.patch.object(VMDK_CONNECTOR, '_download_vmdk') @mock.patch('os.path.getmtime') def test_connect_volume( self, getmtime, download_vmdk, get_moref, create_temp_file, create_session, load_config): session = mock.Mock() create_session.return_value = session tmp_file_path = mock.sentinel.tmp_file_path create_temp_file.return_value = tmp_file_path backing = mock.sentinel.backing get_moref.return_value = backing last_modified = mock.sentinel.last_modified getmtime.return_value = last_modified props = self._create_connection_properties() ret = self._connector.connect_volume(props) self.assertEqual(tmp_file_path, ret['path']) self.assertEqual(last_modified, ret['last_modified']) load_config.assert_called_once_with(props) create_session.assert_called_once_with() create_temp_file.assert_called_once_with( suffix=".vmdk", prefix=props['volume_id']) download_vmdk.assert_called_once_with( tmp_file_path, session, backing, props['vmdk_path'], props['vmdk_size']) session.logout.assert_called_once_with() @ddt.data((None, False), ([mock.sentinel.snap], True)) @ddt.unpack def test_snapshot_exists(self, snap_list, exp_return_value): snapshot = mock.Mock(rootSnapshotList=snap_list) session = mock.Mock() session.invoke_api.return_value = snapshot backing = mock.sentinel.backing ret = self._connector._snapshot_exists(session, backing) self.assertEqual(exp_return_value, ret) session.invoke_api.assert_called_once_with( vim_util, 'get_object_property', session.vim, backing, 'snapshot') def test_create_temp_ds_folder(self): session = mock.Mock() ds_folder_path = mock.sentinel.ds_folder_path dc_ref = mock.sentinel.dc_ref self._connector._create_temp_ds_folder(session, ds_folder_path, dc_ref) session.invoke_api.assert_called_once_with( session.vim, 'MakeDirectory', session.vim.service_content.fileManager, name=ds_folder_path, datacenter=dc_ref) @mock.patch('oslo_vmware.objects.datastore.get_datastore_by_ref') @mock.patch.object(VMDK_CONNECTOR, '_create_temp_ds_folder') @mock.patch('os_brick.initiator.connectors.vmware.open', create=True) @mock.patch.object(VMDK_CONNECTOR, '_upload_vmdk') @mock.patch('os.path.getsize') @mock.patch.object(VMDK_CONNECTOR, '_get_disk_device') @mock.patch.object(VMDK_CONNECTOR, '_detach_disk_from_backing') @mock.patch.object(VMDK_CONNECTOR, '_attach_disk_to_backing') def test_disconnect( self, attach_disk_to_backing, detach_disk_from_backing, get_disk_device, getsize, upload_vmdk, file_open, create_temp_ds_folder, get_ds_by_ref): ds_ref = mock.sentinel.ds_ref ds_name = 'datastore-1' dstore = datastore.Datastore(ds_ref, ds_name) get_ds_by_ref.return_value = dstore file_open_ret = mock.Mock() tmp_file = mock.sentinel.tmp_file file_open_ret.__enter__ = mock.Mock(return_value=tmp_file) file_open_ret.__exit__ = mock.Mock(return_value=None) file_open.return_value = file_open_ret dc_name = mock.sentinel.dc_name copy_task = mock.sentinel.copy_vdisk_task delete_file_task = mock.sentinel.delete_file_task session = mock.Mock() session.invoke_api.side_effect = [dc_name, copy_task, delete_file_task] getsize.return_value = units.Gi disk_device = mock.sentinel.disk_device get_disk_device.return_value = disk_device backing = mock.sentinel.backing tmp_file_path = '/tmp/foo.vmdk' dc_ref = mock.sentinel.dc_ref vmdk_path = mock.sentinel.vmdk_path self._connector._disconnect( backing, tmp_file_path, session, ds_ref, dc_ref, vmdk_path) tmp_folder_path = self._connector.TMP_IMAGES_DATASTORE_FOLDER_PATH ds_folder_path = '[%s] %s' % (ds_name, tmp_folder_path) create_temp_ds_folder.assert_called_once_with( session, ds_folder_path, dc_ref) file_open.assert_called_once_with(tmp_file_path, "rb") self.assertEqual( mock.call(vim_util, 'get_object_property', session.vim, dc_ref, 'name'), session.invoke_api.call_args_list[0]) exp_rel_path = '%s/foo.vmdk' % tmp_folder_path upload_vmdk.assert_called_once_with( tmp_file, self._connector._ip, self._connector._port, dc_name, ds_name, session.vim.client.options.transport.cookiejar, exp_rel_path, units.Gi, self._connector._ca_file, self._connector._timeout) get_disk_device.assert_called_once_with(session, backing) detach_disk_from_backing.assert_called_once_with( session, backing, disk_device) src = '[%s] %s' % (ds_name, exp_rel_path) disk_mgr = session.vim.service_content.virtualDiskManager self.assertEqual( mock.call(session.vim, 'CopyVirtualDisk_Task', disk_mgr, sourceName=src, sourceDatacenter=dc_ref, destName=vmdk_path, destDatacenter=dc_ref), session.invoke_api.call_args_list[1]) self.assertEqual(mock.call(copy_task), session.wait_for_task.call_args_list[0]) attach_disk_to_backing.assert_called_once_with( session, backing, disk_device) file_mgr = session.vim.service_content.fileManager self.assertEqual( mock.call(session.vim, 'DeleteDatastoreFile_Task', file_mgr, name=src, datacenter=dc_ref), session.invoke_api.call_args_list[2]) self.assertEqual(mock.call(delete_file_task), session.wait_for_task.call_args_list[1]) @mock.patch('os.path.exists') def test_disconnect_volume_with_missing_temp_file(self, path_exists): path_exists.return_value = False path = mock.sentinel.path self.assertRaises(exception.NotFound, self._connector.disconnect_volume, mock.ANY, {'path': path}) path_exists.assert_called_once_with(path) @mock.patch('os.path.exists') @mock.patch('os.path.getmtime') @mock.patch.object(VMDK_CONNECTOR, '_disconnect') @mock.patch('os.remove') def test_disconnect_volume_with_unmodified_file( self, remove, disconnect, getmtime, path_exists): path_exists.return_value = True mtime = 1467802060 getmtime.return_value = mtime path = mock.sentinel.path self._connector.disconnect_volume(mock.ANY, {'path': path, 'last_modified': mtime}) path_exists.assert_called_once_with(path) getmtime.assert_called_once_with(path) disconnect.assert_not_called() remove.assert_called_once_with(path) @mock.patch('os.path.exists') @mock.patch('os.path.getmtime') @mock.patch.object(VMDK_CONNECTOR, '_load_config') @mock.patch.object(VMDK_CONNECTOR, '_create_session') @mock.patch('oslo_vmware.vim_util.get_moref') @mock.patch.object(VMDK_CONNECTOR, '_snapshot_exists') @mock.patch.object(VMDK_CONNECTOR, '_disconnect') @mock.patch('os.remove') def test_disconnect_volume( self, remove, disconnect, snapshot_exists, get_moref, create_session, load_config, getmtime, path_exists): path_exists.return_value = True mtime = 1467802060 getmtime.return_value = mtime session = mock.Mock() create_session.return_value = session snapshot_exists.return_value = False backing = mock.sentinel.backing ds_ref = mock.sentinel.ds_ref dc_ref = mock.sentinel.dc_ref get_moref.side_effect = [backing, ds_ref, dc_ref] props = self._create_connection_properties() path = mock.sentinel.path self._connector.disconnect_volume(props, {'path': path, 'last_modified': mtime - 1}) path_exists.assert_called_once_with(path) getmtime.assert_called_once_with(path) load_config.assert_called_once_with(props) create_session.assert_called_once_with() snapshot_exists.assert_called_once_with(session, backing) disconnect.assert_called_once_with( backing, path, session, ds_ref, dc_ref, props['vmdk_path']) remove.assert_called_once_with(path) session.logout.assert_called_once_with() def test_get_disk_device(self): disk_device = mock.Mock() disk_device.__class__.__name__ = 'VirtualDisk' controller_device = mock.Mock() controller_device.__class__.__name__ = 'VirtualLSILogicController' devices = mock.Mock() devices.__class__.__name__ = "ArrayOfVirtualDevice" devices.VirtualDevice = [disk_device, controller_device] session = mock.Mock() session.invoke_api.return_value = devices backing = mock.sentinel.backing self.assertEqual(disk_device, self._connector._get_disk_device(session, backing)) session.invoke_api.assert_called_once_with( vim_util, 'get_object_property', session.vim, backing, 'config.hardware.device') def test_create_spec_for_disk_remove(self): disk_spec = mock.Mock() session = mock.Mock() session.vim.client.factory.create.return_value = disk_spec disk_device = mock.sentinel.disk_device self._connector._create_spec_for_disk_remove(session, disk_device) session.vim.client.factory.create.assert_called_once_with( 'ns0:VirtualDeviceConfigSpec') self.assertEqual('remove', disk_spec.operation) self.assertEqual('destroy', disk_spec.fileOperation) self.assertEqual(disk_device, disk_spec.device) @mock.patch.object(VMDK_CONNECTOR, '_create_spec_for_disk_remove') @mock.patch.object(VMDK_CONNECTOR, '_reconfigure_backing') def test_detach_disk_from_backing(self, reconfigure_backing, create_spec): disk_spec = mock.sentinel.disk_spec create_spec.return_value = disk_spec reconfig_spec = mock.Mock() session = mock.Mock() session.vim.client.factory.create.return_value = reconfig_spec backing = mock.sentinel.backing disk_device = mock.sentinel.disk_device self._connector._detach_disk_from_backing( session, backing, disk_device) create_spec.assert_called_once_with(session, disk_device) session.vim.client.factory.create.assert_called_once_with( 'ns0:VirtualMachineConfigSpec') self.assertEqual([disk_spec], reconfig_spec.deviceChange) reconfigure_backing.assert_called_once_with( session, backing, reconfig_spec) @mock.patch.object(VMDK_CONNECTOR, '_reconfigure_backing') def test_attach_disk_to_backing(self, reconfigure_backing): reconfig_spec = mock.Mock() disk_spec = mock.Mock() session = mock.Mock() session.vim.client.factory.create.side_effect = [ reconfig_spec, disk_spec] backing = mock.Mock() disk_device = mock.sentinel.disk_device self._connector._attach_disk_to_backing(session, backing, disk_device) self.assertEqual([disk_spec], reconfig_spec.deviceChange) self.assertEqual('add', disk_spec.operation) self.assertEqual(disk_device, disk_spec.device) reconfigure_backing.assert_called_once_with( session, backing, reconfig_spec) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/connectors/test_vrtshyperscale.py0000664000175000017500000001315500000000000027066 0ustar00zuulzuul00000000000000# Copyright (c) 2017 Veritas Technologies LLC # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import json from oslo_concurrency import processutils from os_brick import exception from os_brick.initiator.connectors import vrtshyperscale from os_brick.tests.initiator import test_connector DEVICE_NAME = '{8ee71c33-dcd0-4267-8f2b-e0742ecabe9f}' DEVICE_PATH = '/dev/8ee71c33-dcd0-4267-8f2b-e0742ec' class HyperScaleConnectorTestCase(test_connector.ConnectorTestCase): """Test cases for Veritas HyperScale os-brick connector.""" def _fake_execute_success(self, *cmd, **kwargs): """Mock successful execution of hscli""" result_json = "" err = 0 args = json.loads(cmd[1]) if args['operation'] == 'connect_volume': result = {} payload = {} payload['vsa_ip'] = '192.0.2.2' payload['refl_factor'] = '2' payload['refl_targets'] = '192.0.2.3,192.0.2.4' result['payload'] = payload result_json = json.dumps(result) return (result_json, err) def _fake_execute_hscli_missing(self, *cmd, **kwargs): """Mock attempt to execute missing hscli""" raise processutils.ProcessExecutionError() return ("", 0) def _fake_execute_hscli_err(self, *cmd, **kwargs): """Mock hscli returning error""" result_json = "" err = 'fake_hscli_error_msg' return (result_json, err) def _fake_execute_hscli_res_inval(self, *cmd, **kwargs): """Mock hscli returning unexpected values""" result_json = "" err = 0 result = {} payload = {} payload['unexpected'] = 'junk' result['payload'] = payload result_json = json.dumps(result) return (result_json, err) def test_connect_volume_normal(self): """Test results of successful connect_volume()""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = { 'name': DEVICE_NAME } device_info = connector.connect_volume(fake_connection_properties) self.assertEqual('192.0.2.2', device_info['vsa_ip']) self.assertEqual('2', device_info['refl_factor']) self.assertEqual('192.0.2.3,192.0.2.4', device_info['refl_targets']) self.assertEqual(DEVICE_PATH, device_info['path']) def test_connect_volume_arg_missing(self): """Test connect_volume with missing missing arguments""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = {} self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_missing(self): """Test connect_volume that can't call hscli""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_missing) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_err(self): """Test connect_volume when hscli returns an error""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_err) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_connect_volume_hscli_res_inval(self): """Test connect_volume if hscli returns an invalid result""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_hscli_res_inval) fake_connection_properties = { 'name': DEVICE_NAME } self.assertRaises(exception.BrickException, connector.connect_volume, fake_connection_properties) def test_disconnect_volume_normal(self): """Test successful disconnect_volume call""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = { 'name': DEVICE_NAME } fake_device_info = {} connector.disconnect_volume(fake_connection_properties, fake_device_info) def test_disconnect_volume_arg_missing(self): """Test disconnect_volume with missing arguments""" connector = vrtshyperscale.HyperScaleConnector( 'sudo', execute=self._fake_execute_success) fake_connection_properties = {} fake_device_info = {} self.assertRaises(exception.BrickException, connector.disconnect_volume, fake_connection_properties, fake_device_info) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_connector.py0000664000175000017500000003042100000000000023620 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import platform import sys import mock from oslo_concurrency import processutils as putils from oslo_service import loopingcall from os_brick import exception from os_brick.initiator import connector from os_brick.initiator.connectors import base from os_brick.initiator.connectors import fake from os_brick.initiator.connectors import iscsi from os_brick.initiator.connectors import nvmeof from os_brick.initiator import linuxfc from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base as test_base MY_IP = '10.0.0.1' FAKE_SCSI_WWN = '1234567890' class ZeroIntervalLoopingCall(loopingcall.FixedIntervalLoopingCall): def start(self, interval, initial_delay=None, stop_on_exception=True): return super(ZeroIntervalLoopingCall, self).start( 0, 0, stop_on_exception) class ConnectorUtilsTestCase(test_base.TestCase): @mock.patch.object(nvmeof.NVMeOFConnector, '_get_system_uuid', return_value=None) @mock.patch.object(iscsi.ISCSIConnector, 'get_initiator', return_value='fakeinitiator') @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_wwpns', return_value=None) @mock.patch.object(linuxfc.LinuxFibreChannel, 'get_fc_wwnns', return_value=None) @mock.patch.object(platform, 'machine', mock.Mock(return_value='s390x')) @mock.patch('sys.platform', 'linux2') def _test_brick_get_connector_properties(self, multipath, enforce_multipath, multipath_result, mock_wwnns, mock_wwpns, mock_initiator, mock_sysuuid, host='fakehost'): props_actual = connector.get_connector_properties('sudo', MY_IP, multipath, enforce_multipath, host=host) os_type = 'linux2' platform = 's390x' props = {'initiator': 'fakeinitiator', 'host': host, 'ip': MY_IP, 'multipath': multipath_result, 'os_type': os_type, 'platform': platform, 'do_local_attach': False} self.assertEqual(props, props_actual) def test_brick_get_connector_properties_connectors_called(self): """Make sure every connector is called.""" mock_list = [] # Make sure every connector is called for item in connector._get_connector_list(): patched = mock.MagicMock() patched.platform = platform.machine() patched.os_type = sys.platform patched.__name__ = item patched.get_connector_properties.return_value = {} patcher = mock.patch(item, new=patched) patcher.start() self.addCleanup(patcher.stop) mock_list.append(patched) connector.get_connector_properties('sudo', MY_IP, True, True) for item in mock_list: assert item.get_connector_properties.called def test_brick_get_connector_properties(self): self._test_brick_get_connector_properties(False, False, False) @mock.patch.object(priv_rootwrap, 'execute', return_value=('', '')) def test_brick_get_connector_properties_multipath(self, mock_execute): self._test_brick_get_connector_properties(True, True, True) mock_execute.assert_called_once_with('multipathd', 'show', 'status', run_as_root=True, root_helper='sudo') @mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError) def test_brick_get_connector_properties_fallback(self, mock_execute): self._test_brick_get_connector_properties(True, False, False) mock_execute.assert_called_once_with('multipathd', 'show', 'status', run_as_root=True, root_helper='sudo') @mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError) def test_brick_get_connector_properties_raise(self, mock_execute): self.assertRaises(putils.ProcessExecutionError, self._test_brick_get_connector_properties, True, True, None) def test_brick_connector_properties_override_hostname(self): override_host = 'myhostname' self._test_brick_get_connector_properties(False, False, False, host=override_host) class ConnectorTestCase(test_base.TestCase): def setUp(self): super(ConnectorTestCase, self).setUp() self.cmds = [] self.mock_object(loopingcall, 'FixedIntervalLoopingCall', ZeroIntervalLoopingCall) def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def fake_connection(self): return { 'driver_volume_type': 'fake', 'data': { 'volume_id': 'fake_volume_id', 'target_portal': 'fake_location', 'target_iqn': 'fake_iqn', 'target_lun': 1, } } def test_connect_volume(self): self.connector = fake.FakeConnector(None) device_info = self.connector.connect_volume(self.fake_connection()) self.assertIn('type', device_info) self.assertIn('path', device_info) def test_disconnect_volume(self): self.connector = fake.FakeConnector(None) def test_get_connector_properties(self): with mock.patch.object(priv_rootwrap, 'execute') as mock_exec: mock_exec.return_value = ('', '') multipath = True enforce_multipath = True props = base.BaseLinuxConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'multipath': True} self.assertEqual(expected_props, props) multipath = False enforce_multipath = True props = base.BaseLinuxConnector.get_connector_properties( 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) expected_props = {'multipath': False} self.assertEqual(expected_props, props) with mock.patch.object(priv_rootwrap, 'execute', side_effect=putils.ProcessExecutionError): multipath = True enforce_multipath = True self.assertRaises( putils.ProcessExecutionError, base.BaseLinuxConnector.get_connector_properties, 'sudo', multipath=multipath, enforce_multipath=enforce_multipath) @mock.patch('sys.platform', 'win32') def test_get_connector_mapping_win32(self): mapping_win32 = connector.get_connector_mapping() self.assertTrue('ISCSI' in mapping_win32) self.assertFalse('RBD' in mapping_win32) self.assertFalse('STORPOOL' in mapping_win32) @mock.patch('os_brick.initiator.connector.platform.machine') def test_get_connector_mapping(self, mock_platform_machine): mock_platform_machine.return_value = 'x86_64' mapping_x86 = connector.get_connector_mapping() mock_platform_machine.return_value = 'ppc64le' mapping_ppc = connector.get_connector_mapping() self.assertNotEqual(mapping_x86, mapping_ppc) mock_platform_machine.return_value = 's390x' mapping_s390 = connector.get_connector_mapping() self.assertNotEqual(mapping_x86, mapping_s390) self.assertNotEqual(mapping_ppc, mapping_s390) def test_factory(self): obj = connector.InitiatorConnector.factory('iscsi', None) self.assertEqual("ISCSIConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('iscsi', None, arch='ppc64le') self.assertEqual("ISCSIConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('fibre_channel', None, arch='x86_64') self.assertEqual("FibreChannelConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('fibre_channel', None, arch='s390x') self.assertEqual("FibreChannelConnectorS390X", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('aoe', None, arch='x86_64') self.assertEqual("AoEConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'nfs', None, nfs_mount_point_base='/mnt/test') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'glusterfs', None, glusterfs_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'scality', None, scality_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('local', None) self.assertEqual("LocalConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory('gpfs', None) self.assertEqual("GPFSConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'huaweisdshypervisor', None, arch='x86_64') self.assertEqual("HuaweiStorHyperConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( "scaleio", None, arch='x86_64') self.assertEqual("ScaleIOConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( 'quobyte', None, quobyte_mount_point_base='/mnt/test', arch='x86_64') self.assertEqual("RemoteFsConnector", obj.__class__.__name__) obj = connector.InitiatorConnector.factory( "disco", None, arch='x86_64') self.assertEqual("DISCOConnector", obj.__class__.__name__) self.assertRaises(exception.InvalidConnectorProtocol, connector.InitiatorConnector.factory, "bogus", None) def test_check_valid_device_with_wrong_path(self): self.connector = fake.FakeConnector(None) self.connector._execute = \ lambda *args, **kwargs: ("", None) self.assertFalse(self.connector.check_valid_device('/d0v')) def test_check_valid_device(self): self.connector = fake.FakeConnector(None) self.connector._execute = \ lambda *args, **kwargs: ("", "") self.assertTrue(self.connector.check_valid_device('/dev')) def test_check_valid_device_with_cmd_error(self): def raise_except(*args, **kwargs): raise putils.ProcessExecutionError self.connector = fake.FakeConnector(None) with mock.patch.object(self.connector, '_execute', side_effect=putils.ProcessExecutionError): self.assertFalse(self.connector.check_valid_device('/dev')) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_host_driver.py0000664000175000017500000000325000000000000024156 0ustar00zuulzuul00000000000000# Copyright (c) 2015 Scality # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import errno import mock from os_brick.initiator import host_driver from os_brick.tests import base class HostDriverTestCase(base.TestCase): def test_get_all_block_devices(self): fake_dev = ['device1', 'device2'] expected = ['/dev/disk/by-path/' + dev for dev in fake_dev] driver = host_driver.HostDriver() with mock.patch('os.listdir', return_value=fake_dev): actual = driver.get_all_block_devices() self.assertEqual(expected, actual) def test_get_all_block_devices_when_oserror_is_enoent(self): driver = host_driver.HostDriver() oserror = OSError(errno.ENOENT, "") with mock.patch('os.listdir', side_effect=oserror): block_devices = driver.get_all_block_devices() self.assertEqual([], block_devices) def test_get_all_block_devices_when_oserror_is_not_enoent(self): driver = host_driver.HostDriver() oserror = OSError(errno.ENOMEM, "") with mock.patch('os.listdir', side_effect=oserror): self.assertRaises(OSError, driver.get_all_block_devices) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_linuxfc.py0000664000175000017500000006323400000000000023306 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os.path import mock from os_brick.initiator import linuxfc from os_brick.tests import base class LinuxFCTestCase(base.TestCase): def setUp(self): super(LinuxFCTestCase, self).setUp() self.cmds = [] self.mock_object(os.path, 'exists', return_value=True) self.mock_object(os.path, 'isdir', return_value=True) self.lfc = linuxfc.LinuxFibreChannel(None, execute=self.fake_execute) def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def test_has_fc_support(self): self.mock_object(os.path, 'isdir', return_value=False) has_fc = self.lfc.has_fc_support() self.assertFalse(has_fc) self.mock_object(os.path, 'isdir', return_value=True) has_fc = self.lfc.has_fc_support() self.assertTrue(has_fc) @staticmethod def __get_rescan_info(zone_manager=False): connection_properties = { 'initiator_target_map': {'50014380186af83c': ['514f0c50023f6c00'], '50014380186af83e': ['514f0c50023f6c01']}, 'initiator_target_lun_map': { '50014380186af83c': [('514f0c50023f6c00', 1)], '50014380186af83e': [('514f0c50023f6c01', 1)] }, 'target_discovered': False, 'target_lun': 1, 'target_wwn': ['514f0c50023f6c00', '514f0c50023f6c01'], 'targets': [ ('514f0c50023f6c00', 1), ('514f0c50023f6c01', 1), ] } hbas = [ {'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.0/host6/fc_host/host6'), 'host_device': 'host6', 'node_name': '50014380186af83d', 'port_name': '50014380186af83c'}, {'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.1/host7/fc_host/host7'), 'host_device': 'host7', 'node_name': '50014380186af83f', 'port_name': '50014380186af83e'}, ] if not zone_manager: del connection_properties['initiator_target_map'] del connection_properties['initiator_target_lun_map'] return hbas, connection_properties def test__get_hba_channel_scsi_target_lun_single_wwpn(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info() con_props['target_wwn'] = con_props['target_wwn'][0] con_props['targets'] = con_props['targets'][0:1] with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = ([['0', '1', 1]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_with_initiator_target_map(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info(zone_manager=True) con_props['target_wwn'] = con_props['target_wwn'][0] con_props['targets'] = con_props['targets'][0:1] hbas[0]['port_name'] = '50014380186af83e' with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c01" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = ([['0', '1', 1]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_with_initiator_target_map_none( self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info() con_props['target_wwn'] = con_props['target_wwn'][0] con_props['targets'] = con_props['targets'][0:1] con_props['initiator_target_map'] = None hbas[0]['port_name'] = '50014380186af83e' with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = ([['0', '1', 1]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_multiple_wwpn(self): execute_results = [ ['/sys/class/fc_transport/target6:0:1/port_name\n', ''], ['/sys/class/fc_transport/target6:0:2/port_name\n', ''], ] hbas, con_props = self.__get_rescan_info() with mock.patch.object(self.lfc, '_execute', side_effect=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) expected_cmds = [ mock.call('grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), mock.call('grep -Gil "514f0c50023f6c01" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), ] execute_mock.assert_has_calls(expected_cmds) expected = ([['0', '1', 1], ['0', '2', 1]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_multiple_wwpn_and_luns(self): execute_results = [ ['/sys/class/fc_transport/target6:0:1/port_name\n', ''], ['/sys/class/fc_transport/target6:0:2/port_name\n', ''], ] hbas, con_props = self.__get_rescan_info() con_props['target_lun'] = [1, 7] con_props['targets'] = [ ('514f0c50023f6c00', 1), ('514f0c50023f6c01', 7), ] with mock.patch.object(self.lfc, '_execute', side_effect=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) expected_cmds = [ mock.call('grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), mock.call('grep -Gil "514f0c50023f6c01" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), ] execute_mock.assert_has_calls(expected_cmds) expected = ([['0', '1', 1], ['0', '2', 7]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_zone_manager(self): execute_results = ('/sys/class/fc_transport/target6:0:1/port_name\n', '') hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=execute_results) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) expected = ([['0', '1', 1]], set()) self.assertEqual(expected, res) def test__get_hba_channel_scsi_target_lun_not_found(self): hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=('', '')) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) self.assertEqual(([], set()), res) def test__get_hba_channel_scsi_target_lun_exception(self): hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', side_effect=Exception) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_called_once_with( 'grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True) self.assertEqual(([], {1}), res) def test__get_hba_channel_scsi_target_lun_some_exception(self): execute_effects = [ ('/sys/class/fc_transport/target6:0:1/port_name\n', ''), Exception() ] expected_cmds = [ mock.call('grep -Gil "514f0c50023f6c00" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), mock.call('grep -Gil "514f0c50023f6c01" ' '/sys/class/fc_transport/target6:*/port_name', shell=True), ] hbas, con_props = self.__get_rescan_info() with mock.patch.object(self.lfc, '_execute', side_effect=execute_effects) as execute_mock: res = self.lfc._get_hba_channel_scsi_target_lun(hbas[0], con_props) execute_mock.assert_has_calls(expected_cmds) expected = ([['0', '1', 1]], {1}) self.assertEqual(expected, res) def test_rescan_hosts_initiator_map(self): """Test FC rescan with initiator map and not every HBA connected.""" get_chan_results = [([['2', '3', 1], ['4', '5', 1]], set()), ([['6', '7', 1]], set())] hbas, con_props = self.__get_rescan_info(zone_manager=True) # This HBA is not in the initiator map, so we should not scan it or try # to get the channel and target hbas.append({'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.2/host8/fc_host/host8'), 'host_device': 'host8', 'node_name': '50014380186af83g', 'port_name': '50014380186af83h'}) with mock.patch.object(self.lfc, '_execute', return_value=None) as execute_mock, \ mock.patch.object(self.lfc, '_get_hba_channel_scsi_target_lun', side_effect=get_chan_results) as mock_get_chan: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='2 3 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='4 5 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host7/scan', process_input='6 7 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) expected_calls = [mock.call(hbas[0], con_props), mock.call(hbas[1], con_props)] mock_get_chan.assert_has_calls(expected_calls) def test_rescan_hosts_single_wwnn(self): """Test FC rescan with no initiator map and single WWNN for ports.""" get_chan_results = [ [[['2', '3', 1], ['4', '5', 1]], set()], [[['6', '7', 1]], set()], [[], {1}], ] hbas, con_props = self.__get_rescan_info(zone_manager=False) # This HBA is the one that is not included in the single WWNN. hbas.append({'device_path': ('/sys/devices/pci0000:00/0000:00:02.0/' '0000:04:00.2/host8/fc_host/host8'), 'host_device': 'host8', 'node_name': '50014380186af83g', 'port_name': '50014380186af83h'}) with mock.patch.object(self.lfc, '_execute', return_value=None) as execute_mock, \ mock.patch.object(self.lfc, '_get_hba_channel_scsi_target_lun', side_effect=get_chan_results) as mock_get_chan: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='2 3 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='4 5 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host7/scan', process_input='6 7 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) expected_calls = [mock.call(hbas[0], con_props), mock.call(hbas[1], con_props)] mock_get_chan.assert_has_calls(expected_calls) def test_rescan_hosts_initiator_map_single_wwnn(self): """Test FC rescan with initiator map and single WWNN.""" get_chan_results = [([['2', '3', 1], ['4', '5', 1]], set()), ([], {1})] hbas, con_props = self.__get_rescan_info(zone_manager=True) with mock.patch.object(self.lfc, '_execute', return_value=None) as execute_mock, \ mock.patch.object(self.lfc, '_get_hba_channel_scsi_target_lun', side_effect=get_chan_results) as mock_get_chan: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='2 3 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='4 5 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) expected_calls = [mock.call(hbas[0], con_props), mock.call(hbas[1], con_props)] mock_get_chan.assert_has_calls(expected_calls) def test_rescan_hosts_port_not_found(self): """Test when we don't find the target ports.""" get_chan_results = [([], {1}), ([], {1})] hbas, con_props = self.__get_rescan_info(zone_manager=True) # Remove the initiator map con_props.pop('initiator_target_map') con_props.pop('initiator_target_lun_map') with mock.patch.object(self.lfc, '_get_hba_channel_scsi_target_lun', side_effect=get_chan_results), \ mock.patch.object(self.lfc, '_execute', side_effect=None) as execute_mock: self.lfc.rescan_hosts(hbas, con_props) expected_commands = [ mock.call('tee', '-a', '/sys/class/scsi_host/host6/scan', process_input='- - 1', root_helper=None, run_as_root=True), mock.call('tee', '-a', '/sys/class/scsi_host/host7/scan', process_input='- - 1', root_helper=None, run_as_root=True)] execute_mock.assert_has_calls(expected_commands) self.assertEqual(len(expected_commands), execute_mock.call_count) def test_rescan_hosts_port_not_found_driver_disables_wildcards(self): """Test when we don't find the target ports but driver forces scan.""" get_chan_results = [([], {1}), ([], {1})] hbas, con_props = self.__get_rescan_info(zone_manager=True) con_props['enable_wildcard_scan'] = False with mock.patch.object(self.lfc, '_get_hba_channel_scsi_target_lun', side_effect=get_chan_results), \ mock.patch.object(self.lfc, '_execute', side_effect=None) as execute_mock: self.lfc.rescan_hosts(hbas, con_props) execute_mock.assert_not_called() def test_get_fc_hbas_fail(self): def fake_exec1(a, b, c, d, run_as_root=True, root_helper='sudo'): raise OSError def fake_exec2(a, b, c, d, run_as_root=True, root_helper='sudo'): return None, 'None found' self.lfc._execute = fake_exec1 hbas = self.lfc.get_fc_hbas() self.assertEqual(0, len(hbas)) self.lfc._execute = fake_exec2 hbas = self.lfc.get_fc_hbas() self.assertEqual(0, len(hbas)) def test_get_fc_hbas(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec hbas = self.lfc.get_fc_hbas() self.assertEqual(2, len(hbas)) hba1 = hbas[0] self.assertEqual("host0", hba1["ClassDevice"]) hba2 = hbas[1] self.assertEqual("host2", hba2["ClassDevice"]) def test_get_fc_hbas_info(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec hbas_info = self.lfc.get_fc_hbas_info() expected_info = [{'device_path': '/sys/devices/pci0000:20/' '0000:20:03.0/0000:21:00.0/' 'host0/fc_host/host0', 'host_device': 'host0', 'node_name': '50014380242b9751', 'port_name': '50014380242b9750'}, {'device_path': '/sys/devices/pci0000:20/' '0000:20:03.0/0000:21:00.1/' 'host2/fc_host/host2', 'host_device': 'host2', 'node_name': '50014380242b9753', 'port_name': '50014380242b9752'}, ] self.assertEqual(expected_info, hbas_info) def test_get_fc_wwpns(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec wwpns = self.lfc.get_fc_wwpns() expected_wwpns = ['50014380242b9750', '50014380242b9752'] self.assertEqual(expected_wwpns, wwpns) def test_get_fc_wwnns(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC, None self.lfc._execute = fake_exec wwnns = self.lfc.get_fc_wwpns() expected_wwnns = ['50014380242b9750', '50014380242b9752'] self.assertEqual(expected_wwnns, wwnns) SYSTOOL_FC = """ Class = "fc_host" Class Device = "host0" Class Device path = "/sys/devices/pci0000:20/0000:20:03.0/\ 0000:21:00.0/host0/fc_host/host0" dev_loss_tmo = "16" fabric_name = "0x100000051ea338b9" issue_lip = max_npiv_vports = "0" node_name = "0x50014380242b9751" npiv_vports_inuse = "0" port_id = "0x960d0d" port_name = "0x50014380242b9750" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QMH2572 FW:v4.04.04 DVR:v8.03.07.12-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = vport_create = vport_delete = Device = "host0" Device path = "/sys/devices/pci0000:20/0000:20:03.0/0000:21:00.0/host0" edc = optrom_ctl = reset = uevent = "DEVTYPE=scsi_host" Class Device = "host2" Class Device path = "/sys/devices/pci0000:20/0000:20:03.0/\ 0000:21:00.1/host2/fc_host/host2" dev_loss_tmo = "16" fabric_name = "0x100000051ea33b79" issue_lip = max_npiv_vports = "0" node_name = "0x50014380242b9753" npiv_vports_inuse = "0" port_id = "0x970e09" port_name = "0x50014380242b9752" port_state = "Online" port_type = "NPort (fabric via point-to-point)" speed = "8 Gbit" supported_classes = "Class 3" supported_speeds = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit" symbolic_name = "QMH2572 FW:v4.04.04 DVR:v8.03.07.12-k" system_hostname = "" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = vport_create = vport_delete = Device = "host2" Device path = "/sys/devices/pci0000:20/0000:20:03.0/0000:21:00.1/host2" edc = optrom_ctl = reset = uevent = "DEVTYPE=scsi_host" """ class LinuxFCS390XTestCase(LinuxFCTestCase): def setUp(self): super(LinuxFCS390XTestCase, self).setUp() self.cmds = [] self.lfc = linuxfc.LinuxFibreChannelS390X(None, execute=self.fake_execute) def test_get_fc_hbas_info(self): def fake_exec(a, b, c, d, run_as_root=True, root_helper='sudo'): return SYSTOOL_FC_S390X, None self.lfc._execute = fake_exec hbas_info = self.lfc.get_fc_hbas_info() expected = [{'device_path': '/sys/devices/css0/0.0.02ea/' '0.0.3080/host0/fc_host/host0', 'host_device': 'host0', 'node_name': '1234567898765432', 'port_name': 'c05076ffe680a960'}] self.assertEqual(expected, hbas_info) @mock.patch.object(os.path, 'exists', return_value=False) def test_configure_scsi_device(self, mock_execute): device_number = "0.0.2319" target_wwn = "0x50014380242b9751" lun = 1 self.lfc.configure_scsi_device(device_number, target_wwn, lun) expected_commands = [('tee -a /sys/bus/ccw/drivers/zfcp/0.0.2319/' 'port_rescan'), ('tee -a /sys/bus/ccw/drivers/zfcp/0.0.2319/' '0x50014380242b9751/unit_add')] self.assertEqual(expected_commands, self.cmds) def test_deconfigure_scsi_device(self): device_number = "0.0.2319" target_wwn = "0x50014380242b9751" lun = 1 self.lfc.deconfigure_scsi_device(device_number, target_wwn, lun) expected_commands = [('tee -a /sys/bus/ccw/drivers/zfcp/' '0.0.2319/0x50014380242b9751/unit_remove')] self.assertEqual(expected_commands, self.cmds) SYSTOOL_FC_S390X = """ Class = "fc_host" Class Device = "host0" Class Device path = "/sys/devices/css0/0.0.02ea/0.0.3080/host0/fc_host/host0" active_fc4s = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 " dev_loss_tmo = "60" maxframe_size = "2112 bytes" node_name = "0x1234567898765432" permanent_port_name = "0xc05076ffe6803081" port_id = "0x010014" port_name = "0xc05076ffe680a960" port_state = "Online" port_type = "NPIV VPORT" serial_number = "IBM00000000000P30" speed = "8 Gbit" supported_classes = "Class 2, Class 3" supported_fc4s = "0x00 0x00 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 \ 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 " supported_speeds = "2 Gbit, 4 Gbit" symbolic_name = "IBM 2827 00000000000P30 \ PCHID: 0308 NPIV UlpId: 01EA0A00 DEVNO: 0.0.1234 NAME: dummy" tgtid_bind_type = "wwpn (World Wide Port Name)" uevent = Device = "host0" Device path = "/sys/devices/css0/0.0.02ea/0.0.3080/host0" uevent = "DEVTYPE=scsi_host" """ ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_linuxrbd.py0000664000175000017500000001720200000000000023457 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may not # use this file except in compliance with the License. You may obtain a copy of # the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations under # the License. import mock from os_brick import exception from os_brick.initiator import linuxrbd from os_brick.tests import base from os_brick import utils class MockRados(object): class Error(Exception): pass class ioctx(object): def __init__(self, *args, **kwargs): pass def __enter__(self, *args, **kwargs): return self def __exit__(self, *args, **kwargs): return False def close(self, *args, **kwargs): pass class Rados(object): def __init__(self, *args, **kwargs): pass def __enter__(self, *args, **kwargs): return self def __exit__(self, *args, **kwargs): return False def connect(self, *args, **kwargs): pass def open_ioctx(self, *args, **kwargs): return MockRados.ioctx() def shutdown(self, *args, **kwargs): pass class RBDClientTestCase(base.TestCase): def setUp(self): super(RBDClientTestCase, self).setUp() @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') def test_with_client(self, mock_rados, mock_rbd): with linuxrbd.RBDClient('test_user', 'test_pool') as client: # Verify object attributes are assigned as expected self.assertEqual('/etc/ceph/ceph.conf', client.rbd_conf) self.assertEqual(utils.convert_str('test_user'), client.rbd_user) self.assertEqual(utils.convert_str('test_pool'), client.rbd_pool) # Assert connect is called with correct paramaters mock_rados.Rados.assert_called_once_with( clustername='ceph', rados_id=utils.convert_str('test_user'), conffile='/etc/ceph/ceph.conf') # Ensure correct calls to connect to cluster self.assertEqual( 1, mock_rados.Rados.return_value.connect.call_count) mock_rados.Rados.return_value.open_ioctx.assert_called_once_with( utils.convert_str('test_pool')) self.assertEqual(1, mock_rados.Rados.return_value.shutdown.call_count) @mock.patch.object(MockRados.Rados, 'connect', side_effect=MockRados.Error) def test_with_client_error(self, _): linuxrbd.rados = MockRados linuxrbd.rados.Error = MockRados.Error def test(): with linuxrbd.RBDClient('test_user', 'test_pool'): pass self.assertRaises(exception.BrickException, test) class RBDVolumeIOWrapperTestCase(base.TestCase): def setUp(self): super(RBDVolumeIOWrapperTestCase, self).setUp() self.mock_volume = mock.Mock() self.mock_volume_wrapper = \ linuxrbd.RBDVolumeIOWrapper(self.mock_volume) self.data_length = 1024 self.full_data = 'abcd' * 256 def test_init(self): self.assertEqual(self.mock_volume, self.mock_volume_wrapper._rbd_volume) self.assertEqual(0, self.mock_volume_wrapper._offset) def test_inc_offset(self): self.mock_volume_wrapper._inc_offset(10) self.mock_volume_wrapper._inc_offset(10) self.assertEqual(20, self.mock_volume_wrapper._offset) def test_read(self): def mock_read(offset, length): return self.full_data[offset:length] self.mock_volume.image.read.side_effect = mock_read self.mock_volume.image.size.return_value = self.data_length data = self.mock_volume_wrapper.read() self.assertEqual(self.full_data, data) data = self.mock_volume_wrapper.read() self.assertEqual(b'', data) self.mock_volume_wrapper.seek(0) data = self.mock_volume_wrapper.read() self.assertEqual(self.full_data, data) self.mock_volume_wrapper.seek(0) data = self.mock_volume_wrapper.read(10) self.assertEqual(self.full_data[:10], data) def test_write(self): self.mock_volume_wrapper.write(self.full_data) self.assertEqual(1024, self.mock_volume_wrapper._offset) def test_seekable(self): self.assertTrue(self.mock_volume_wrapper.seekable) def test_seek(self): self.assertEqual(0, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10) self.assertEqual(10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10) self.assertEqual(10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10, 1) self.assertEqual(20, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(0) self.mock_volume_wrapper.write(self.full_data) self.mock_volume.image.size.return_value = self.data_length self.mock_volume_wrapper.seek(0) self.assertEqual(0, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(10, 2) self.assertEqual(self.data_length + 10, self.mock_volume_wrapper._offset) self.mock_volume_wrapper.seek(-10, 2) self.assertEqual(self.data_length - 10, self.mock_volume_wrapper._offset) # test exceptions. self.assertRaises(IOError, self.mock_volume_wrapper.seek, 0, 3) self.assertRaises(IOError, self.mock_volume_wrapper.seek, -1) # offset should not have been changed by any of the previous # operations. self.assertEqual(self.data_length - 10, self.mock_volume_wrapper._offset) def test_tell(self): self.assertEqual(0, self.mock_volume_wrapper.tell()) self.mock_volume_wrapper._inc_offset(10) self.assertEqual(10, self.mock_volume_wrapper.tell()) def test_flush(self): with mock.patch.object(linuxrbd, 'LOG') as mock_logger: self.mock_volume.image.flush = mock.Mock() self.mock_volume_wrapper.flush() self.assertEqual(1, self.mock_volume.image.flush.call_count) self.mock_volume.image.flush.reset_mock() # this should be caught and logged silently. self.mock_volume.image.flush.side_effect = AttributeError self.mock_volume_wrapper.flush() self.assertEqual(1, self.mock_volume.image.flush.call_count) self.assertEqual(1, mock_logger.warning.call_count) def test_fileno(self): self.assertRaises(IOError, self.mock_volume_wrapper.fileno) @mock.patch('os_brick.initiator.linuxrbd.rbd') @mock.patch('os_brick.initiator.linuxrbd.rados') @mock.patch.object(linuxrbd.RBDClient, 'disconnect') def test_close(self, rbd_disconnect, mock_rados, mock_rbd): rbd_client = linuxrbd.RBDClient('user', 'pool') rbd_volume = linuxrbd.RBDVolume(rbd_client, 'volume') rbd_handle = linuxrbd.RBDVolumeIOWrapper( linuxrbd.RBDImageMetadata(rbd_volume, 'pool', 'user', None)) rbd_handle.close() self.assertEqual(1, rbd_disconnect.call_count) class RBDVolumeTestCase(base.TestCase): def test_name_attribute(self): mock_client = mock.Mock() rbd_volume = linuxrbd.RBDVolume(mock_client, 'volume') self.assertEqual('volume', rbd_volume.name) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_linuxscsi.py0000664000175000017500000016633200000000000023662 0ustar00zuulzuul00000000000000# (c) Copyright 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import os.path import textwrap import time import ddt import mock from oslo_concurrency import processutils as putils from oslo_log import log as logging from os_brick import exception from os_brick.initiator import linuxscsi from os_brick.tests import base LOG = logging.getLogger(__name__) @ddt.ddt class LinuxSCSITestCase(base.TestCase): def setUp(self): super(LinuxSCSITestCase, self).setUp() self.cmds = [] self.realpath = os.path.realpath self.mock_object(os.path, 'realpath', return_value='/dev/sdc') self.mock_object(os, 'stat', returns=os.stat(__file__)) self.linuxscsi = linuxscsi.LinuxSCSI(None, execute=self.fake_execute) def fake_execute(self, *cmd, **kwargs): self.cmds.append(" ".join(cmd)) return "", None def test_echo_scsi_command(self): self.linuxscsi.echo_scsi_command("/some/path", "1") expected_commands = ['tee -a /some/path'] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(os.path, 'realpath') def test_get_name_from_path(self, realpath_mock): device_name = "/dev/sdc" realpath_mock.return_value = device_name disk_path = ("/dev/disk/by-path/ip-10.10.220.253:3260-" "iscsi-iqn.2000-05.com.3pardata:21810002ac00383d-lun-0") name = self.linuxscsi.get_name_from_path(disk_path) self.assertEqual(device_name, name) disk_path = ("/dev/disk/by-path/pci-0000:00:00.0-ip-10.9.8.7:3260-" "iscsi-iqn.2000-05.com.openstack:2180002ac00383d-lun-0") name = self.linuxscsi.get_name_from_path(disk_path) self.assertEqual(device_name, name) realpath_mock.return_value = "bogus" name = self.linuxscsi.get_name_from_path(disk_path) self.assertIsNone(name) @mock.patch.object(os.path, 'exists', return_value=False) def test_remove_scsi_device(self, exists_mock): self.linuxscsi.remove_scsi_device("/dev/sdc") expected_commands = [] self.assertEqual(expected_commands, self.cmds) exists_mock.return_value = True self.linuxscsi.remove_scsi_device("/dev/sdc") expected_commands = [ ('blockdev --flushbufs /dev/sdc'), ('tee -a /sys/block/sdc/device/delete')] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'echo_scsi_command') @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_device_io') @mock.patch.object(os.path, 'exists', return_value=True) def test_remove_scsi_device_force(self, exists_mock, flush_mock, echo_mock): """With force we'll always call delete even if flush fails.""" exc = exception.ExceptionChainer() flush_mock.side_effect = Exception() echo_mock.side_effect = Exception() device = '/dev/sdc' self.linuxscsi.remove_scsi_device(device, force=True, exc=exc) # The context manager has caught the exceptions self.assertTrue(exc) flush_mock.assert_called_once_with(device) echo_mock.assert_called_once_with('/sys/block/sdc/device/delete', '1') @mock.patch.object(os.path, 'exists', return_value=False) def test_remove_scsi_device_no_flush(self, exists_mock): self.linuxscsi.remove_scsi_device("/dev/sdc") expected_commands = [] self.assertEqual(expected_commands, self.cmds) exists_mock.return_value = True self.linuxscsi.remove_scsi_device("/dev/sdc", flush=False) expected_commands = [('tee -a /sys/block/sdc/device/delete')] self.assertEqual(expected_commands, self.cmds) @mock.patch('time.sleep') @mock.patch('os.path.exists', return_value=True) def test_wait_for_volumes_removal_failure(self, exists_mock, sleep_mock): retries = 61 names = ('sda', 'sdb') self.assertRaises(exception.VolumePathNotRemoved, self.linuxscsi.wait_for_volumes_removal, names) exists_mock.assert_has_calls([mock.call('/dev/' + name) for name in names] * retries) self.assertEqual(retries - 1, sleep_mock.call_count) @mock.patch('time.sleep') @mock.patch('os.path.exists', side_effect=(True, True, False, False)) def test_wait_for_volumes_removal_retry(self, exists_mock, sleep_mock): names = ('sda', 'sdb') self.linuxscsi.wait_for_volumes_removal(names) exists_mock.assert_has_calls([mock.call('/dev/' + name) for name in names] * 2) self.assertEqual(1, sleep_mock.call_count) def test_flush_multipath_device(self): dm_map_name = '3600d0230000000000e13955cc3757800' with mock.patch.object(self.linuxscsi, '_execute') as exec_mock: self.linuxscsi.flush_multipath_device(dm_map_name) exec_mock.assert_called_once_with( 'multipath', '-f', dm_map_name, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self.linuxscsi._root_helper) def test_get_scsi_wwn(self): fake_path = '/dev/disk/by-id/somepath' fake_wwn = '1234567890' def fake_execute(*cmd, **kwargs): return fake_wwn, None self.linuxscsi._execute = fake_execute wwn = self.linuxscsi.get_scsi_wwn(fake_path) self.assertEqual(fake_wwn, wwn) @mock.patch('six.moves.builtins.open') def test_get_dm_name(self, open_mock): dm_map_name = '3600d0230000000000e13955cc3757800' cm_open = open_mock.return_value.__enter__.return_value cm_open.read.return_value = dm_map_name res = self.linuxscsi.get_dm_name('dm-0') self.assertEqual(dm_map_name, res) open_mock.assert_called_once_with('/sys/block/dm-0/dm/name') @mock.patch('six.moves.builtins.open', side_effect=IOError) def test_get_dm_name_failure(self, open_mock): self.assertEqual('', self.linuxscsi.get_dm_name('dm-0')) @mock.patch('glob.glob', side_effect=[[], ['/sys/block/sda/holders/dm-9']]) def test_find_sysfs_multipath_dm(self, glob_mock): device_names = ('sda', 'sdb') res = self.linuxscsi.find_sysfs_multipath_dm(device_names) self.assertEqual('dm-9', res) glob_mock.assert_has_calls([mock.call('/sys/block/sda/holders/dm-*'), mock.call('/sys/block/sdb/holders/dm-*')]) @mock.patch('glob.glob', return_value=[]) def test_find_sysfs_multipath_dm_not_found(self, glob_mock): device_names = ('sda', 'sdb') res = self.linuxscsi.find_sysfs_multipath_dm(device_names) self.assertIsNone(res) glob_mock.assert_has_calls([mock.call('/sys/block/sda/holders/dm-*'), mock.call('/sys/block/sdb/holders/dm-*')]) @mock.patch.object(linuxscsi.LinuxSCSI, '_execute') @mock.patch('os.path.exists', return_value=True) def test_flush_device_io(self, exists_mock, exec_mock): device = '/dev/sda' self.linuxscsi.flush_device_io(device) exists_mock.assert_called_once_with(device) exec_mock.assert_called_once_with( 'blockdev', '--flushbufs', device, run_as_root=True, attempts=3, timeout=300, interval=10, root_helper=self.linuxscsi._root_helper) @mock.patch('os.path.exists', return_value=False) def test_flush_device_io_non_existent(self, exists_mock): device = '/dev/sda' self.linuxscsi.flush_device_io(device) exists_mock.assert_called_once_with(device) @mock.patch.object(os.path, 'exists', return_value=True) def test_find_multipath_device_path(self, exists_mock): fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) expected_path = '/dev/disk/by-id/dm-uuid-mpath-%s' % fake_wwn self.assertEqual(expected_path, found_path) @mock.patch('time.sleep') @mock.patch.object(os.path, 'exists') def test_find_multipath_device_path_mapper(self, exists_mock, sleep_mock): # the wait loop tries 3 times before it gives up # we want to test failing to find the # /dev/disk/by-id/dm-uuid-mpath- path # but finding the # /dev/mapper/ path exists_mock.side_effect = [False, False, False, True] fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) expected_path = '/dev/mapper/%s' % fake_wwn self.assertEqual(expected_path, found_path) self.assertTrue(sleep_mock.called) @mock.patch.object(os.path, 'exists', return_value=False) @mock.patch.object(time, 'sleep') def test_find_multipath_device_path_fail(self, exists_mock, sleep_mock): fake_wwn = '1234567890' found_path = self.linuxscsi.find_multipath_device_path(fake_wwn) self.assertIsNone(found_path) @mock.patch.object(os.path, 'exists', return_value=False) @mock.patch.object(time, 'sleep') def test_wait_for_path_not_found(self, exists_mock, sleep_mock): path = "/dev/disk/by-id/dm-uuid-mpath-%s" % '1234567890' self.assertRaisesRegex(exception.VolumeDeviceNotFound, r'Volume device not found at %s' % path, self.linuxscsi.wait_for_path, path) @ddt.data({'do_raise': False, 'force': False}, {'do_raise': True, 'force': True}) @ddt.unpack @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_del_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'is_multipath_running', return_value=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dm_name') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_multipath_complete(self, remove_mock, wait_mock, find_dm_mock, get_dm_name_mock, flush_mp_mock, is_mp_running_mock, mp_del_path_mock, remove_link_mock, do_raise, force): if do_raise: flush_mp_mock.side_effect = Exception devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() mp_name = self.linuxscsi.remove_connection(devices_names, force=mock.sentinel.Force, exc=exc) find_dm_mock.assert_called_once_with(devices_names) get_dm_name_mock.assert_called_once_with(find_dm_mock.return_value) flush_mp_mock.assert_called_once_with(get_dm_name_mock.return_value) self.assertEqual(get_dm_name_mock.return_value if do_raise else None, mp_name) is_mp_running_mock.assert_not_called() mp_del_path_mock.assert_has_calls([ mock.call('/dev/sda'), mock.call('/dev/sdb')]) remove_mock.assert_has_calls([ mock.call('/dev/sda', mock.sentinel.Force, exc, False), mock.call('/dev/sdb', mock.sentinel.Force, exc, False)]) wait_mock.assert_called_once_with(devices_names) self.assertEqual(do_raise, bool(exc)) remove_link_mock.assert_called_once_with(devices_names) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_del_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'is_multipath_running', return_value=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dm_name') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm', return_value=None) @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_multipath_complete_no_dm(self, remove_mock, wait_mock, find_dm_mock, get_dm_name_mock, flush_mp_mock, is_mp_running_mock, mp_del_path_mock, remove_link_mock): devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() mp_name = self.linuxscsi.remove_connection(devices_names, force=mock.sentinel.Force, exc=exc) find_dm_mock.assert_called_once_with(devices_names) get_dm_name_mock.assert_not_called() flush_mp_mock.assert_not_called() self.assertIsNone(mp_name) is_mp_running_mock.assert_called_once() mp_del_path_mock.assert_has_calls([ mock.call('/dev/sda'), mock.call('/dev/sdb')]) remove_mock.assert_has_calls([ mock.call('/dev/sda', mock.sentinel.Force, exc, False), mock.call('/dev/sdb', mock.sentinel.Force, exc, False)]) wait_mock.assert_called_once_with(devices_names) self.assertFalse(bool(exc)) remove_link_mock.assert_called_once_with(devices_names) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_del_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'is_multipath_running', return_value=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'flush_multipath_device', side_effect=Exception) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_dm_name') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_multipath_fail(self, remove_mock, wait_mock, find_dm_mock, get_dm_name_mock, flush_mp_mock, is_mp_running_mock, mp_del_path_mock, remove_link_mock): flush_mp_mock.side_effect = exception.ExceptionChainer devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() self.assertRaises(exception.ExceptionChainer, self.linuxscsi.remove_connection, devices_names, force=False, exc=exc) find_dm_mock.assert_called_once_with(devices_names) get_dm_name_mock.assert_called_once_with(find_dm_mock.return_value) flush_mp_mock.assert_called_once_with(get_dm_name_mock.return_value) is_mp_running_mock.assert_not_called() mp_del_path_mock.assert_not_called() remove_mock.assert_not_called() wait_mock.assert_not_called() remove_link_mock.assert_not_called() self.assertTrue(bool(exc)) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_del_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'is_multipath_running', return_value=True) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_singlepath_no_path(self, remove_mock, wait_mock, find_dm_mock, is_mp_running_mock, mp_del_path_mock, remove_link_mock): # Test remove connection when we didn't form a multipath and didn't # even use any of the devices that were found. This means that we # don't flush any of the single paths when removing them. find_dm_mock.return_value = None devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() self.linuxscsi.remove_connection(devices_names, force=mock.sentinel.Force, exc=exc) find_dm_mock.assert_called_once_with(devices_names) is_mp_running_mock.assert_called_once() mp_del_path_mock.assert_has_calls([ mock.call('/dev/sda'), mock.call('/dev/sdb')]) remove_mock.assert_has_calls( [mock.call('/dev/sda', mock.sentinel.Force, exc, False), mock.call('/dev/sdb', mock.sentinel.Force, exc, False)]) wait_mock.assert_called_once_with(devices_names) remove_link_mock.assert_called_once_with(devices_names) @mock.patch.object(linuxscsi.LinuxSCSI, '_remove_scsi_symlinks') @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_del_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'is_multipath_running', return_value=False) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_sysfs_multipath_dm') @mock.patch.object(linuxscsi.LinuxSCSI, 'wait_for_volumes_removal') @mock.patch.object(linuxscsi.LinuxSCSI, 'remove_scsi_device') def test_remove_connection_singlepath_used(self, remove_mock, wait_mock, find_dm_mock, is_mp_running_mock, mp_del_path_mock, remove_link_mock): # Test remove connection when we didn't form a multipath and just used # one of the single paths that were found. This means that we don't # flush any of the single paths when removing them. find_dm_mock.return_value = None devices_names = ('sda', 'sdb') exc = exception.ExceptionChainer() # realpath was mocked on test setup with mock.patch('os.path.realpath', side_effect=self.realpath): self.linuxscsi.remove_connection(devices_names, force=mock.sentinel.Force, exc=exc, path_used='/dev/sdb', was_multipath=False) find_dm_mock.assert_called_once_with(devices_names) is_mp_running_mock.assert_called_once() mp_del_path_mock.assert_not_called() remove_mock.assert_has_calls( [mock.call('/dev/sda', mock.sentinel.Force, exc, False), mock.call('/dev/sdb', mock.sentinel.Force, exc, True)]) wait_mock.assert_called_once_with(devices_names) remove_link_mock.assert_called_once_with(devices_names) def test_find_multipath_device_3par_ufn(self): def fake_execute(*cmd, **kwargs): out = ("mpath6 (350002ac20398383d) dm-3 3PARdata,VV\n" "size=2.0G features='0' hwhandler='0' wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 0:0:0:1 sde 8:64 active undef running\n" " `- 2:0:0:1 sdf 8:80 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sde') self.assertEqual("350002ac20398383d", info["id"]) self.assertEqual("mpath6", info["name"]) self.assertEqual("/dev/mapper/mpath6", info["device"]) self.assertEqual("/dev/sde", info['devices'][0]['device']) self.assertEqual("0", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("1", info['devices'][0]['lun']) self.assertEqual("/dev/sdf", info['devices'][1]['device']) self.assertEqual("2", info['devices'][1]['host']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("0", info['devices'][1]['channel']) self.assertEqual("1", info['devices'][1]['lun']) def test_find_multipath_device_svc(self): def fake_execute(*cmd, **kwargs): out = ("36005076da00638089c000000000004d5 dm-2 IBM,2145\n" "size=954M features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "|-+- policy='round-robin 0' prio=-1 status=active\n" "| |- 6:0:2:0 sde 8:64 active undef running\n" "| `- 6:0:4:0 sdg 8:96 active undef running\n" "`-+- policy='round-robin 0' prio=-1 status=enabled\n" " |- 6:0:3:0 sdf 8:80 active undef running\n" " `- 6:0:5:0 sdh 8:112 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sde') self.assertEqual("36005076da00638089c000000000004d5", info["id"]) self.assertEqual("36005076da00638089c000000000004d5", info["name"]) self.assertEqual("/dev/mapper/36005076da00638089c000000000004d5", info["device"]) self.assertEqual("/dev/sde", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdf", info['devices'][2]['device']) self.assertEqual("6", info['devices'][2]['host']) self.assertEqual("0", info['devices'][2]['channel']) self.assertEqual("3", info['devices'][2]['id']) self.assertEqual("0", info['devices'][2]['lun']) def test_find_multipath_device_ds8000(self): def fake_execute(*cmd, **kwargs): out = ("36005076303ffc48e0000000000000101 dm-2 IBM,2107900\n" "size=1.0G features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 6:0:2:0 sdd 8:64 active undef running\n" " `- 6:1:0:3 sdc 8:32 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') self.assertEqual("36005076303ffc48e0000000000000101", info["id"]) self.assertEqual("36005076303ffc48e0000000000000101", info["name"]) self.assertEqual("/dev/mapper/36005076303ffc48e0000000000000101", info["device"]) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) def test_find_multipath_device_with_error(self): def fake_execute(*cmd, **kwargs): out = ("Oct 13 10:24:01 | /lib/udev/scsi_id exited with 1\n" "36005076303ffc48e0000000000000101 dm-2 IBM,2107900\n" "size=1.0G features='1 queue_if_no_path' hwhandler='0'" " wp=rw\n" "`-+- policy='round-robin 0' prio=-1 status=active\n" " |- 6:0:2:0 sdd 8:64 active undef running\n" " `- 6:1:0:3 sdc 8:32 active undef running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') self.assertEqual("36005076303ffc48e0000000000000101", info["id"]) self.assertEqual("36005076303ffc48e0000000000000101", info["name"]) self.assertEqual("/dev/mapper/36005076303ffc48e0000000000000101", info["device"]) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) def test_find_multipath_device_with_multiple_errors(self): def fake_execute(*cmd, **kwargs): out = ("Jun 21 04:39:26 | 8:160: path wwid appears to have " "changed. Using old wwid.\n\n" "Jun 21 04:39:26 | 65:208: path wwid appears to have " "changed. Using old wwid.\n\n" "Jun 21 04:39:26 | 65:208: path wwid appears to have " "changed. Using old wwid.\n" "3624a93707edcfde1127040370004ee62 dm-84 PURE ," "FlashArray\n" "size=100G features='0' hwhandler='0' wp=rw\n" "`-+- policy='queue-length 0' prio=1 status=active\n" " |- 8:0:0:9 sdaa 65:160 active ready running\n" " `- 8:0:1:9 sdac 65:192 active ready running\n" ) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdaa') self.assertEqual("3624a93707edcfde1127040370004ee62", info["id"]) self.assertEqual("3624a93707edcfde1127040370004ee62", info["name"]) self.assertEqual("/dev/mapper/3624a93707edcfde1127040370004ee62", info["device"]) self.assertEqual("/dev/sdaa", info['devices'][0]['device']) self.assertEqual("8", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("0", info['devices'][0]['id']) self.assertEqual("9", info['devices'][0]['lun']) self.assertEqual("/dev/sdac", info['devices'][1]['device']) self.assertEqual("8", info['devices'][1]['host']) self.assertEqual("0", info['devices'][1]['channel']) self.assertEqual("1", info['devices'][1]['id']) self.assertEqual("9", info['devices'][1]['lun']) @mock.patch.object(time, 'sleep') def test_wait_for_rw(self, mock_sleep): lsblk_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.return_value = (lsblk_output, None) self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd120014a2bb' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn # Ensure no exception is raised and no sleep is called self.linuxscsi.wait_for_rw(wwn, path) self.assertFalse(mock_sleep.called) @mock.patch.object(time, 'sleep') def test_wait_for_rw_needs_retry(self, mock_sleep): lsblk_ro_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 1 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" lsblk_rw_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 0 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.side_effect = [(lsblk_ro_output, None), ('', None), # multipath -r output (lsblk_rw_output, None)] self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd1200143029' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn self.linuxscsi.wait_for_rw(wwn, path) self.assertEqual(1, mock_sleep.call_count) @mock.patch.object(time, 'sleep') def test_wait_for_rw_always_readonly(self, mock_sleep): lsblk_output = """3624a93709a738ed78583fd1200143029 (dm-2) 0 sdb 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdc 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdd 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sde 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdf 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdg 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdh 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdi 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdj 0 3624a93709a738ed78583fd1200143029 (dm-2) 0 sdk 0 3624a93709a738ed78583fd120014724e (dm-1) 0 sdl 0 3624a93709a738ed78583fd120014a2bb (dm-0) 1 sdm 0 vda1 0 vdb 0 vdb1 0 loop0 0""" mock_execute = mock.Mock() mock_execute.return_value = (lsblk_output, None) self.linuxscsi._execute = mock_execute wwn = '3624a93709a738ed78583fd120014a2bb' path = '/dev/disk/by-id/dm-uuid-mpath-' + wwn self.assertRaises(exception.BlockDeviceReadOnly, self.linuxscsi.wait_for_rw, wwn, path) self.assertEqual(4, mock_sleep.call_count) def test_find_multipath_device_with_action(self): def fake_execute(*cmd, **kwargs): out = textwrap.dedent(""" create: 36005076303ffc48e0000000000000101 dm-2 IBM,2107900 size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=-1 status=active |- 6:0:2:0 sdd 8:64 active undef running `- 6:1:0:3 sdc 8:32 active undef running """) return out, None self.linuxscsi._execute = fake_execute info = self.linuxscsi.find_multipath_device('/dev/sdd') LOG.error("Device info: %s", info) self.assertEqual('36005076303ffc48e0000000000000101', info['id']) self.assertEqual('36005076303ffc48e0000000000000101', info['name']) self.assertEqual('/dev/mapper/36005076303ffc48e0000000000000101', info['device']) self.assertEqual("/dev/sdd", info['devices'][0]['device']) self.assertEqual("6", info['devices'][0]['host']) self.assertEqual("0", info['devices'][0]['channel']) self.assertEqual("2", info['devices'][0]['id']) self.assertEqual("0", info['devices'][0]['lun']) self.assertEqual("/dev/sdc", info['devices'][1]['device']) self.assertEqual("6", info['devices'][1]['host']) self.assertEqual("1", info['devices'][1]['channel']) self.assertEqual("0", info['devices'][1]['id']) self.assertEqual("3", info['devices'][1]['lun']) def test_get_device_size(self): mock_execute = mock.Mock() self.linuxscsi._execute = mock_execute size = '1024' mock_execute.return_value = (size, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertEqual(int(size), ret_size) size = 'junk' mock_execute.return_value = (size, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertIsNone(ret_size) size_bad = '1024\n' size_good = 1024 mock_execute.return_value = (size_bad, None) ret_size = self.linuxscsi.get_device_size('/dev/fake') self.assertEqual(size_good, ret_size) def test_multipath_reconfigure(self): self.linuxscsi.multipath_reconfigure() expected_commands = ['multipathd reconfigure'] self.assertEqual(expected_commands, self.cmds) def test_multipath_resize_map(self): wwn = '1234567890123456' self.linuxscsi.multipath_resize_map(wwn) expected_commands = ['multipathd resize map %s' % wwn] self.assertEqual(expected_commands, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_no_mpath(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path): """Test extending a volume where there is no multipath device.""" fake_device = {'host': '0', 'channel': '0', 'id': '0', 'lun': '1'} mock_device_info.return_value = fake_device first_size = 1024 second_size = 2048 mock_device_size.side_effect = [first_size, second_size] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = None ret_size = self.linuxscsi.extend_volume(['/dev/fake']) self.assertEqual(second_size, ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan'] self.assertEqual(expected_cmds, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_with_mpath(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path): """Test extending a volume where there is a multipath device.""" mock_device_info.side_effect = [{'host': host, 'channel': '0', 'id': '0', 'lun': '1'} for host in ['0', '1']] mock_device_size.side_effect = [1024, 2048, 1024, 2048, 1024, 2048] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = ('/dev/mapper/dm-uuid-mpath-%s' % wwn) ret_size = self.linuxscsi.extend_volume(['/dev/fake1', '/dev/fake2'], use_multipath=True) self.assertEqual(2048, ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan', 'tee -a /sys/bus/scsi/drivers/sd/1:0:0:1/rescan', 'multipathd reconfigure', 'multipathd resize map %s' % wwn] self.assertEqual(expected_cmds, self.cmds) @mock.patch.object(linuxscsi.LinuxSCSI, 'multipath_resize_map') @mock.patch.object(linuxscsi.LinuxSCSI, 'find_multipath_device_path') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_scsi_wwn') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_size') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_device_info') def test_extend_volume_with_mpath_fail(self, mock_device_info, mock_device_size, mock_scsi_wwn, mock_find_mpath_path, mock_mpath_resize_map): """Test extending a volume where there is a multipath device fail.""" mock_device_info.side_effect = [{'host': host, 'channel': '0', 'id': '0', 'lun': '1'} for host in ['0', '1']] mock_device_size.side_effect = [1024, 2048, 1024, 2048, 1024, 2048] wwn = '1234567890123456' mock_scsi_wwn.return_value = wwn mock_find_mpath_path.return_value = ('/dev/mapper/dm-uuid-mpath-%s' % wwn) mock_mpath_resize_map.return_value = 'fail' ret_size = self.linuxscsi.extend_volume(['/dev/fake1', '/dev/fake2'], use_multipath=True) self.assertIsNone(ret_size) # because we don't mock out the echo_scsi_command expected_cmds = ['tee -a /sys/bus/scsi/drivers/sd/0:0:0:1/rescan', 'tee -a /sys/bus/scsi/drivers/sd/1:0:0:1/rescan', 'multipathd reconfigure'] self.assertEqual(expected_cmds, self.cmds) def test_process_lun_id_list(self): lun_list = [2, 255, 88, 370, 5, 256] result = self.linuxscsi.process_lun_id(lun_list) expected = [2, 255, 88, '0x0172000000000000', 5, '0x0100000000000000'] self.assertEqual(expected, result) def test_process_lun_id_single_val_make_hex(self): lun_id = 499 result = self.linuxscsi.process_lun_id(lun_id) expected = '0x01f3000000000000' self.assertEqual(expected, result) def test_process_lun_id_single_val_make_hex_border_case(self): lun_id = 256 result = self.linuxscsi.process_lun_id(lun_id) expected = '0x0100000000000000' self.assertEqual(expected, result) def test_process_lun_id_single_var_return(self): lun_id = 13 result = self.linuxscsi.process_lun_id(lun_id) expected = 13 self.assertEqual(expected, result) @mock.patch('os_brick.privileged.rootwrap.execute', return_value=('', '')) def test_is_multipath_running_default_executor(self, mock_exec): res = linuxscsi.LinuxSCSI.is_multipath_running(False, None, mock_exec) self.assertTrue(res) mock_exec.assert_called_once_with( 'multipathd', 'show', 'status', run_as_root=True, root_helper=None) @mock.patch('os_brick.privileged.rootwrap.execute') def test_is_multipath_running_failure_exit_code_0(self, mock_exec): mock_exec.return_value = ('error receiving packet', '') self.assertRaises(putils.ProcessExecutionError, linuxscsi.LinuxSCSI.is_multipath_running, True, None, mock_exec) mock_exec.assert_called_once_with( 'multipathd', 'show', 'status', run_as_root=True, root_helper=None) def test_get_device_info(self): ret = "/dev/sg0 scsi1 channel=1 id=0 lun=0 [em]\n" with mock.patch.object(self.linuxscsi, '_execute') as exec_mock: exec_mock.return_value = (ret, "") info = self.linuxscsi.get_device_info('/dev/adevice') exec_mock.assert_called_once_with('sg_scan', '/dev/adevice', root_helper=None, run_as_root=True) self.assertEqual(info, {'channel': '1', 'device': '/dev/adevice', 'host': '1', 'id': '0', 'lun': '0'}) @mock.patch('six.moves.builtins.open') def test_get_sysfs_wwn_mpath(self, open_mock): wwn = '3600d0230000000000e13955cc3757800' cm_open = open_mock.return_value.__enter__.return_value cm_open.read.return_value = 'mpath-' + wwn res = self.linuxscsi.get_sysfs_wwn(mock.sentinel.device_names, 'dm-1') open_mock.assert_called_once_with('/sys/block/dm-1/dm/uuid') self.assertEqual(wwn, res) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_single_designator(self, get_wwid_mock, glob_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'wwid1' res = self.linuxscsi.get_sysfs_wwn(mock.sentinel.device_names) self.assertEqual('wwid1', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(mock.sentinel.device_names) @mock.patch('six.moves.builtins.open', side_effect=Exception) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_mpath_exc(self, get_wwid_mock, glob_mock, open_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'wwid1' res = self.linuxscsi.get_sysfs_wwn(mock.sentinel.device_names, 'dm-1') open_mock.assert_called_once_with('/sys/block/dm-1/dm/uuid') self.assertEqual('wwid1', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(mock.sentinel.device_names) @mock.patch('os.listdir', return_value=['sda', 'sdd']) @mock.patch('os.path.realpath', side_effect=('/other/path', '/dev/dm-5', '/dev/sda', '/dev/sdb')) @mock.patch('os.path.islink', side_effect=(False,) + (True,) * 5) @mock.patch('os.stat', side_effect=(False,) + (True,) * 4) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_multiple_designators(self, get_wwid_mock, glob_mock, stat_mock, islink_mock, realpath_mock, listdir_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-fail-link', '/dev/disk/by-id/scsi-fail-stat', '/dev/disk/by-id/scsi-non-dev', '/dev/disk/by-id/scsi-another-dm', '/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'pre-wwid' devices = ['sdb', 'sdc'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('wwid2', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') listdir_mock.assert_called_once_with('/sys/class/block/dm-5/slaves') get_wwid_mock.assert_called_once_with(devices) @mock.patch('os.listdir', side_effect=[['sda', 'sdb'], ['sdc', 'sdd']]) @mock.patch('os.path.realpath', side_effect=('/dev/sde', '/dev/dm-5', '/dev/dm-6')) @mock.patch('os.path.islink', mock.Mock()) @mock.patch('os.stat', mock.Mock()) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid', return_value='') def test_get_sysfs_wwn_dm_link(self, get_wwid_mock, glob_mock, realpath_mock, listdir_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-another-dm', '/dev/disk/by-id/scsi-our-dm'] devices = ['sdc', 'sdd'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('our-dm', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') listdir_mock.assert_has_calls( [mock.call('/sys/class/block/dm-5/slaves'), mock.call('/sys/class/block/dm-6/slaves')]) get_wwid_mock.assert_called_once_with(devices) @mock.patch('os.path.realpath', side_effect=('/dev/sda', '/dev/sdb')) @mock.patch('os.path.islink', return_value=True) @mock.patch('os.stat', return_value=True) @mock.patch('glob.glob') @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_not_found(self, get_wwid_mock, glob_mock, stat_mock, islink_mock, realpath_mock): glob_mock.return_value = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] get_wwid_mock.return_value = 'pre-wwid' devices = ['sdc'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(devices) @mock.patch('glob.glob', return_value=[]) @mock.patch.object(linuxscsi.LinuxSCSI, 'get_sysfs_wwid') def test_get_sysfs_wwn_no_links(self, get_wwid_mock, glob_mock): get_wwid_mock.return_value = '' devices = ['sdc'] res = self.linuxscsi.get_sysfs_wwn(devices) self.assertEqual('', res) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') get_wwid_mock.assert_called_once_with(devices) @ddt.data({'wwn_type': 't10.', 'num_val': '1'}, {'wwn_type': 'eui.', 'num_val': '2'}, {'wwn_type': 'naa.', 'num_val': '3'}) @ddt.unpack @mock.patch('six.moves.builtins.open') def test_get_sysfs_wwid(self, open_mock, wwn_type, num_val): read_fail = mock.MagicMock() read_fail.__enter__.return_value.read.side_effect = IOError read_data = mock.MagicMock() read_data.__enter__.return_value.read.return_value = (wwn_type + 'wwid1\n') open_mock.side_effect = (IOError, read_fail, read_data) res = self.linuxscsi.get_sysfs_wwid(['sda', 'sdb', 'sdc']) self.assertEqual(num_val + 'wwid1', res) open_mock.assert_has_calls([mock.call('/sys/block/sda/device/wwid'), mock.call('/sys/block/sdb/device/wwid'), mock.call('/sys/block/sdc/device/wwid')]) @mock.patch('six.moves.builtins.open', side_effect=IOError) def test_get_sysfs_wwid_not_found(self, open_mock): res = self.linuxscsi.get_sysfs_wwid(['sda', 'sdb']) self.assertEqual('', res) open_mock.assert_has_calls([mock.call('/sys/block/sda/device/wwid'), mock.call('/sys/block/sdb/device/wwid')]) @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=['/dev/sda', '/dev/sdb', '/dev/sdc']) def test_remove_scsi_symlinks(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2', '/dev/disk/by-id/scsi-wwid3'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sdb', 'sdc', 'sdd']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_called_once_with(no_errors=True, *paths[1:]) @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=['/dev/sda', '/dev/sdb']) def test_remove_scsi_symlinks_no_links(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sdd', 'sde']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_not_called() @mock.patch.object(linuxscsi.priv_rootwrap, 'unlink_root') @mock.patch('glob.glob') @mock.patch('os.path.realpath', side_effect=[OSError, '/dev/sda']) def test_remove_scsi_symlinks_race_condition(self, realpath_mock, glob_mock, unlink_mock): paths = ['/dev/disk/by-id/scsi-wwid1', '/dev/disk/by-id/scsi-wwid2'] glob_mock.return_value = paths self.linuxscsi._remove_scsi_symlinks(['sda']) glob_mock.assert_called_once_with('/dev/disk/by-id/scsi-*') realpath_mock.assert_has_calls([mock.call(g) for g in paths]) unlink_mock.assert_called_once_with(paths[1], no_errors=True) @mock.patch('glob.glob') def test_get_hctl_with_target(self, glob_mock): glob_mock.return_value = [ '/sys/class/iscsi_host/host3/device/session1/target3:4:5', '/sys/class/iscsi_host/host3/device/session1/target3:4:6'] res = self.linuxscsi.get_hctl('1', '2') self.assertEqual(('3', '4', '5', '2'), res) glob_mock.assert_called_once_with( '/sys/class/iscsi_host/host*/device/session1/target*') @mock.patch('glob.glob') def test_get_hctl_no_target(self, glob_mock): glob_mock.side_effect = [ [], ['/sys/class/iscsi_host/host3/device/session1', '/sys/class/iscsi_host/host3/device/session1']] res = self.linuxscsi.get_hctl('1', '2') self.assertEqual(('3', '-', '-', '2'), res) glob_mock.assert_has_calls( [mock.call('/sys/class/iscsi_host/host*/device/session1/target*'), mock.call('/sys/class/iscsi_host/host*/device/session1')]) @mock.patch('glob.glob', return_value=[]) def test_get_hctl_no_paths(self, glob_mock): res = self.linuxscsi.get_hctl('1', '2') self.assertIsNone(res) glob_mock.assert_has_calls( [mock.call('/sys/class/iscsi_host/host*/device/session1/target*'), mock.call('/sys/class/iscsi_host/host*/device/session1')]) @mock.patch('glob.glob') def test_device_name_by_hctl(self, glob_mock): glob_mock.return_value = [ '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda2', '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda'] res = self.linuxscsi.device_name_by_hctl('1', ('3', '4', '5', '2')) self.assertEqual('sda', res) glob_mock.assert_called_once_with( '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/*') @mock.patch('glob.glob') def test_device_name_by_hctl_wildcards(self, glob_mock): glob_mock.return_value = [ '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda2', '/sys/class/scsi_host/host3/device/session1/target3:4:5/3:4:5:2/' 'block/sda'] res = self.linuxscsi.device_name_by_hctl('1', ('3', '-', '-', '2')) self.assertEqual('sda', res) glob_mock.assert_called_once_with( '/sys/class/scsi_host/host3/device/session1/target3:*:*/3:*:*:2/' 'block/*') @mock.patch('glob.glob', mock.Mock(return_value=[])) def test_device_name_by_hctl_no_devices(self): res = self.linuxscsi.device_name_by_hctl('1', ('4', '5', '6', '2')) self.assertIsNone(res) @mock.patch.object(linuxscsi.LinuxSCSI, 'echo_scsi_command') def test_scsi_iscsi(self, echo_mock): self.linuxscsi.scan_iscsi('host', 'channel', 'target', 'lun') echo_mock.assert_called_once_with('/sys/class/scsi_host/hosthost/scan', 'channel target lun') def test_multipath_add_wwid(self): self.linuxscsi.multipath_add_wwid('wwid1') self.assertEqual(['multipath -a wwid1'], self.cmds) def test_multipath_add_path(self): self.linuxscsi.multipath_add_path('/dev/sda') self.assertEqual(['multipathd add path /dev/sda'], self.cmds) def test_multipath_del_path(self): self.linuxscsi.multipath_del_path('/dev/sda') self.assertEqual(['multipathd del path /dev/sda'], self.cmds) @ddt.data({'con_props': {}, 'dev_info': {'path': mock.sentinel.path}}, {'con_props': None, 'dev_info': {'path': mock.sentinel.path}}, {'con_props': {'device_path': mock.sentinel.device_path}, 'dev_info': {'path': mock.sentinel.path}}) @ddt.unpack def test_get_dev_path_device_info(self, con_props, dev_info): self.assertEqual(mock.sentinel.path, self.linuxscsi.get_dev_path(con_props, dev_info)) @ddt.data({'con_props': {'device_path': mock.sentinel.device_path}, 'dev_info': {'path': None}}, {'con_props': {'device_path': mock.sentinel.device_path}, 'dev_info': {'path': ''}}, {'con_props': {'device_path': mock.sentinel.device_path}, 'dev_info': {}}, {'con_props': {'device_path': mock.sentinel.device_path}, 'dev_info': None}) @ddt.unpack def test_get_dev_path_conn_props(self, con_props, dev_info): self.assertEqual(mock.sentinel.device_path, self.linuxscsi.get_dev_path(con_props, dev_info)) @ddt.data({'con_props': {'device_path': ''}, 'dev_info': {'path': None}}, {'con_props': {'device_path': None}, 'dev_info': {'path': ''}}, {'con_props': {}, 'dev_info': {}}, {'con_props': {}, 'dev_info': None}) @ddt.unpack def test_get_dev_path_no_path(self, con_props, dev_info): self.assertEqual('', self.linuxscsi.get_dev_path(con_props, dev_info)) @ddt.data(('/dev/sda', '/dev/sda', False, True, None), # This checks that we ignore the was_multipath parameter if it # doesn't make sense (because the used path is the one we are # asking about) ('/dev/sda', '/dev/sda', True, True, None), ('/dev/sda', '', True, False, None), # Check for encrypted volume ('/dev/link_sda', '/dev/disk/by-path/pci-XYZ', False, True, ('/dev/sda', '/dev/mapper/crypt-pci-XYZ')), ('/dev/link_sda', '/dev/link_sdb', False, False, ('/dev/sda', '/dev/sdb')), ('/dev/link_sda', '/dev/link2_sda', False, True, ('/dev/sda', '/dev/sda'))) @ddt.unpack def test_requires_flush(self, path, path_used, was_multipath, expected, real_paths): with mock.patch('os.path.realpath', side_effect=real_paths) as mocked: self.assertEqual( expected, self.linuxscsi.requires_flush(path, path_used, was_multipath)) if real_paths: mocked.assert_has_calls([mock.call(path), mock.call(path_used)]) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/initiator/test_utils.py0000664000175000017500000000533000000000000022767 0ustar00zuulzuul00000000000000# Copyright 2018 Red Hat, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_brick.initiator import utils from os_brick.tests import base class InitiatorUtilsTestCase(base.TestCase): @mock.patch('os.name', 'nt') def test_check_manual_scan_windows(self): self.assertFalse(utils.check_manual_scan()) @mock.patch('os.name', 'posix') @mock.patch('oslo_concurrency.processutils.execute') def test_check_manual_scan_supported(self, mock_exec): self.assertTrue(utils.check_manual_scan()) mock_exec.assert_called_once_with('grep', '-F', 'node.session.scan', '/sbin/iscsiadm') @mock.patch('os.name', 'posix') @mock.patch('oslo_concurrency.processutils.execute', side_effect=utils.putils.ProcessExecutionError) def test_check_manual_scan_not_supported(self, mock_exec): self.assertFalse(utils.check_manual_scan()) mock_exec.assert_called_once_with('grep', '-F', 'node.session.scan', '/sbin/iscsiadm') @mock.patch('oslo_concurrency.lockutils.lock') def test_guard_connection_manual_scan_support(self, mock_lock): utils.ISCSI_SUPPORTS_MANUAL_SCAN = True # We confirm that shared_targets is ignored with utils.guard_connection({'shared_targets': True}): mock_lock.assert_not_called() @mock.patch('oslo_concurrency.lockutils.lock') def test_guard_connection_manual_scan_unsupported_not_shared(self, mock_lock): utils.ISCSI_SUPPORTS_MANUAL_SCAN = False with utils.guard_connection({'shared_targets': False}): mock_lock.assert_not_called() @mock.patch('oslo_concurrency.lockutils.lock') def test_guard_connection_manual_scan_unsupported_hared(self, mock_lock): utils.ISCSI_SUPPORTS_MANUAL_SCAN = False with utils.guard_connection({'service_uuid': mock.sentinel.uuid, 'shared_targets': True}): mock_lock.assert_called_once_with(mock.sentinel.uuid, 'os-brick-', external=True) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1526778 os-brick-3.0.8/os_brick/tests/local_dev/0000775000175000017500000000000000000000000020143 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/local_dev/__init__.py0000664000175000017500000000000000000000000022242 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/local_dev/fake_lvm.py0000664000175000017500000000334500000000000022306 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. class FakeBrickLVM(object): """Logs and records calls, for unit tests.""" def __init__(self, vg_name, create, pv_list, vtype, execute=None): super(FakeBrickLVM, self).__init__() self.vg_size = '5.00' self.vg_free_space = '5.00' self.vg_name = vg_name def supports_thin_provisioning(): return False def get_volumes(self): return ['fake-volume'] def get_volume(self, name): return ['name'] def get_all_physical_volumes(vg_name=None): return [] def get_physical_volumes(self): return [] def update_volume_group_info(self): pass def create_thin_pool(self, name=None, size_str=0): pass def create_volume(self, name, size_str, lv_type='default', mirror_count=0): pass def create_lv_snapshot(self, name, source_lv_name, lv_type='default'): pass def delete(self, name): pass def revert(self, snapshot_name): pass def lv_has_snapshot(self, name): return False def activate_lv(self, lv, is_snapshot=False, permanent=False): pass def rename_volume(self, lv_name, new_name): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/local_dev/test_brick_lvm.py0000664000175000017500000004131000000000000023523 0ustar00zuulzuul00000000000000# Copyright 2012 OpenStack Foundation # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from oslo_concurrency import processutils from os_brick import exception from os_brick.local_dev import lvm as brick from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base class BrickLvmTestCase(base.TestCase): def setUp(self): super(BrickLvmTestCase, self).setUp() if not hasattr(self, 'configuration'): self.configuration = mock.Mock() self.configuration.lvm_suppress_fd_warnings = False self.volume_group_name = 'fake-vg' # Stub processutils.execute for static methods self.mock_object(priv_rootwrap, 'execute', self.fake_execute) self.vg = brick.LVM( self.volume_group_name, 'sudo', create_vg=False, physical_volumes=None, lvm_type='default', executor=self.fake_execute, suppress_fd_warn=self.configuration.lvm_suppress_fd_warnings) def failed_fake_execute(obj, *cmd, **kwargs): return ("\n", "fake-error") def fake_pretend_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.03.00 (2012-03-06)\n", "") def fake_old_lvm_version(obj, *cmd, **kwargs): # Does not support thin prov or snap activation return (" LVM version: 2.02.65(2) (2012-03-06)\n", "") def fake_customised_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.02.100(2)-RHEL6 (2013-09-12)\n", "") def fake_f23_lvm_version(obj, *cmd, **kwargs): return (" LVM version: 2.02.132(2) (2015-09-22)\n", "") def fake_execute(obj, *cmd, **kwargs): # TODO(eharney): remove this and move to per-test mocked execute calls if obj.configuration.lvm_suppress_fd_warnings: _lvm_prefix = 'env, LC_ALL=C, LVM_SUPPRESS_FD_WARNINGS=1, ' else: _lvm_prefix = 'env, LC_ALL=C, ' cmd_string = ', '.join(cmd) data = "\n" if (_lvm_prefix + 'vgs, --noheadings, --unit=g, -o, name' == cmd_string): data = " fake-vg\n" data += " some-other-vg\n" elif (_lvm_prefix + 'vgs, --noheadings, -o, name, fake-vg' == cmd_string): data = " fake-vg\n" elif _lvm_prefix + 'vgs, --version' in cmd_string: data = " LVM version: 2.02.95(2) (2012-03-06)\n" elif (_lvm_prefix + 'vgs, --noheadings, -o, uuid, fake-vg' in cmd_string): data = " kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1\n" elif _lvm_prefix + 'vgs, --noheadings, --unit=g, ' \ '-o, name,size,free,lv_count,uuid, ' \ '--separator, :, --nosuffix' in cmd_string: data = (" test-prov-cap-vg-unit:10.00:10.00:0:" "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z4\n") if 'test-prov-cap-vg-unit' in cmd_string: return (data, "") data = (" test-prov-cap-vg-no-unit:10.00:10.00:0:" "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z4\n") if 'test-prov-cap-vg-no-unit' in cmd_string: return (data, "") data = " fake-vg:10.00:10.00:0:"\ "kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1\n" if 'fake-vg' in cmd_string: return (data, "") data += " fake-vg-2:10.00:10.00:0:"\ "lWyauW-dKpG-Rz7E-xtKY-jeju-QsYU-SLG7Z2\n" data += " fake-vg-3:10.00:10.00:0:"\ "mXzbuX-dKpG-Rz7E-xtKY-jeju-QsYU-SLG8Z3\n" elif (_lvm_prefix + 'lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size, --nosuffix, ' 'fake-vg/lv-nothere' in cmd_string): raise processutils.ProcessExecutionError( stderr="One or more specified logical volume(s) not found.") elif (_lvm_prefix + 'lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size, --nosuffix, ' 'fake-vg/lv-newerror' in cmd_string): raise processutils.ProcessExecutionError( stderr="Failed to find logical volume \"fake-vg/lv-newerror\"") elif (_lvm_prefix + 'lvs, --noheadings, ' '--unit=g, -o, vg_name,name,size' in cmd_string): if 'fake-unknown' in cmd_string: raise processutils.ProcessExecutionError( stderr="One or more volume(s) not found." ) if 'test-prov-cap-vg-unit' in cmd_string: data = " fake-vg test-prov-cap-pool-unit 9.50g\n" data += " fake-vg fake-volume-1 1.00g\n" data += " fake-vg fake-volume-2 2.00g\n" elif 'test-prov-cap-vg-no-unit' in cmd_string: data = " fake-vg test-prov-cap-pool-no-unit 9.50\n" data += " fake-vg fake-volume-1 1.00\n" data += " fake-vg fake-volume-2 2.00\n" elif 'test-found-lv-name' in cmd_string: data = " fake-vg test-found-lv-name 9.50\n" else: data = " fake-vg fake-1 1.00g\n" data += " fake-vg fake-2 1.00g\n" elif (_lvm_prefix + 'lvdisplay, --noheading, -C, -o, Attr' in cmd_string): if 'test-volumes' in cmd_string: data = ' wi-a-' else: data = ' owi-a-' elif _lvm_prefix + 'pvs, --noheadings' in cmd_string: data = " fake-vg|/dev/sda|10.00|1.00\n" data += " fake-vg|/dev/sdb|10.00|1.00\n" data += " fake-vg|/dev/sdc|10.00|8.99\n" data += " fake-vg-2|/dev/sdd|10.00|9.99\n" elif _lvm_prefix + 'lvs, --noheadings, --unit=g' \ ', -o, size,data_percent, --separator, :' in cmd_string: if 'test-prov-cap-pool' in cmd_string: data = " 9.5:20\n" else: data = " 9:12\n" elif 'lvcreate, -T, -L, ' in cmd_string: pass elif 'lvcreate, -T, -l, 100%FREE' in cmd_string: pass elif 'lvcreate, -T, -V, ' in cmd_string: pass elif 'lvcreate, -n, ' in cmd_string: pass elif 'lvcreate, --name, ' in cmd_string: pass elif 'lvextend, -L, ' in cmd_string: pass else: raise AssertionError('unexpected command called: %s' % cmd_string) return (data, "") def test_create_lv_snapshot(self): self.assertIsNone(self.vg.create_lv_snapshot('snapshot-1', 'fake-1')) with mock.patch.object(self.vg, 'get_volume', return_value=None): try: self.vg.create_lv_snapshot('snapshot-1', 'fake-non-existent') except exception.VolumeDeviceNotFound as e: self.assertEqual('fake-non-existent', e.kwargs['device']) else: self.fail("Exception not raised") def test_vg_exists(self): self.assertTrue(self.vg._vg_exists()) def test_get_vg_uuid(self): self.assertEqual('kVxztV-dKpG-Rz7E-xtKY-jeju-QsYU-SLG6Z1', self.vg._get_vg_uuid()[0]) def test_get_all_volumes(self): out = self.vg.get_volumes() self.assertEqual('fake-1', out[0]['name']) self.assertEqual('1.00g', out[0]['size']) self.assertEqual('fake-vg', out[0]['vg']) def test_get_volume(self): self.assertEqual('fake-1', self.vg.get_volume('fake-1')['name']) def test_get_volume_none(self): self.assertIsNone(self.vg.get_volume('fake-unknown')) def test_get_lv_info_notfound(self): # lv-nothere will raise lvm < 2.102.112 exception self.assertEqual( [], self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='lv-nothere') ) # lv-newerror will raise lvm > 2.102.112 exception self.assertEqual( [], self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='lv-newerror') ) def test_get_lv_info_found(self): lv_info = [{'size': '9.50', 'name': 'test-found-lv-name', 'vg': 'fake-vg'}] self.assertEqual( lv_info, self.vg.get_lv_info( 'sudo', vg_name='fake-vg', lv_name='test-found-lv-name') ) def test_get_lv_info_no_lv_name(self): lv_info = [{'name': 'fake-1', 'size': '1.00g', 'vg': 'fake-vg'}, {'name': 'fake-2', 'size': '1.00g', 'vg': 'fake-vg'}] self.assertEqual( lv_info, self.vg.get_lv_info( 'sudo', vg_name='fake-vg') ) def test_get_all_physical_volumes(self): # Filtered VG version pvs = self.vg.get_all_physical_volumes('sudo', 'fake-vg') self.assertEqual(3, len(pvs)) # Non-Filtered, all VG's pvs = self.vg.get_all_physical_volumes('sudo') self.assertEqual(4, len(pvs)) def test_get_physical_volumes(self): pvs = self.vg.get_physical_volumes() self.assertEqual(3, len(pvs)) def test_get_volume_groups(self): self.assertEqual(3, len(self.vg.get_all_volume_groups('sudo'))) self.assertEqual(1, len(self.vg.get_all_volume_groups('sudo', 'fake-vg'))) def test_thin_support(self): # lvm.supports_thin() is a static method and doesn't # use the self._executor fake we pass in on init # so we need to stub processutils.execute appropriately with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_execute): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_pretend_lvm_version): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_thin_provisioning('sudo')) with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_customised_lvm_version): self.assertTrue(self.vg.supports_thin_provisioning('sudo')) def test_snapshot_lv_activate_support(self): self.vg._supports_snapshot_lv_activation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_execute): self.assertTrue(self.vg.supports_snapshot_lv_activation) self.vg._supports_snapshot_lv_activation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_snapshot_lv_activation) self.vg._supports_snapshot_lv_activation = None def test_lvchange_ignskipact_support_yes(self): """Tests if lvchange -K is available via a lvm2 version check.""" self.vg._supports_lvchange_ignoreskipactivation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_pretend_lvm_version): self.assertTrue(self.vg.supports_lvchange_ignoreskipactivation) self.vg._supports_lvchange_ignoreskipactivation = None with mock.patch.object(priv_rootwrap, 'execute', side_effect=self.fake_old_lvm_version): self.assertFalse(self.vg.supports_lvchange_ignoreskipactivation) self.vg._supports_lvchange_ignoreskipactivation = None def test_thin_pool_creation_manual(self): # The size of fake-vg volume group is 10g, so the calculated thin # pool size should be 9.5g (95% of 10g). self.vg.create_thin_pool() def test_thin_pool_provisioned_capacity(self): self.vg.vg_thin_pool = "test-prov-cap-pool-unit" self.vg.vg_name = 'test-prov-cap-vg-unit' self.assertIsNone(self.vg.create_thin_pool(name=self.vg.vg_thin_pool)) self.assertEqual(9.50, self.vg.vg_thin_pool_size) self.assertEqual(7.6, self.vg.vg_thin_pool_free_space) self.assertEqual(3.0, self.vg.vg_provisioned_capacity) self.vg.vg_thin_pool = "test-prov-cap-pool-no-unit" self.vg.vg_name = 'test-prov-cap-vg-no-unit' self.assertIsNone(self.vg.create_thin_pool(name=self.vg.vg_thin_pool)) self.assertEqual(9.50, self.vg.vg_thin_pool_size) self.assertEqual(7.6, self.vg.vg_thin_pool_free_space) self.assertEqual(3.0, self.vg.vg_provisioned_capacity) def test_thin_pool_free_space(self): # The size of fake-vg-pool is 9g and the allocated data sums up to # 12% so the calculated free space should be 7.92 self.assertEqual(float("7.92"), self.vg._get_thin_pool_free_space("fake-vg", "fake-vg-pool")) def test_volume_create_after_thin_creation(self): """Test self.vg.vg_thin_pool is set to pool_name See bug #1220286 for more info. """ vg_name = "vg-name" pool_name = vg_name + "-pool" pool_path = "%s/%s" % (vg_name, pool_name) def executor(obj, *cmd, **kwargs): self.assertEqual(pool_path, cmd[-1]) self.vg._executor = executor self.vg.create_thin_pool(pool_name) self.vg.create_volume("test", "1G", lv_type='thin') self.assertEqual(pool_name, self.vg.vg_thin_pool) def test_lv_has_snapshot(self): self.assertTrue(self.vg.lv_has_snapshot('fake-vg')) self.assertFalse(self.vg.lv_has_snapshot('test-volumes')) def test_activate_lv(self): self.vg._supports_lvchange_ignoreskipactivation = True with mock.patch.object(self.vg, '_execute') as mock_exec: self.vg.activate_lv('my-lv') expected = [mock.call('lvchange', '-a', 'y', '--yes', '-K', 'fake-vg/my-lv', root_helper='sudo', run_as_root=True)] self.assertEqual(expected, mock_exec.call_args_list) def test_get_mirrored_available_capacity(self): self.assertEqual(2.0, self.vg.vg_mirror_free_space(1)) def test_lv_extend(self): self.vg.deactivate_lv = mock.MagicMock() # Extend lv with snapshot and make sure deactivate called self.vg.create_volume("test", "1G") self.vg.extend_volume("test", "2G") self.vg.deactivate_lv.assert_called_once_with('test') self.vg.deactivate_lv.reset_mock() # Extend lv without snapshot so deactivate should not be called self.vg.create_volume("test", "1G") self.vg.vg_name = "test-volumes" self.vg.extend_volume("test", "2G") self.assertFalse(self.vg.deactivate_lv.called) def test_lv_deactivate(self): with mock.patch.object(self.vg, '_execute'): is_active_mock = mock.Mock() is_active_mock.return_value = False self.vg._lv_is_active = is_active_mock self.vg.create_volume('test', '1G') self.vg.deactivate_lv('test') @mock.patch('time.sleep') def test_lv_deactivate_timeout(self, mock_sleep): with mock.patch.object(self.vg, '_execute'): is_active_mock = mock.Mock() is_active_mock.return_value = True self.vg._lv_is_active = is_active_mock self.vg.create_volume('test', '1G') self.assertRaises(exception.VolumeNotDeactivated, self.vg.deactivate_lv, 'test') def test_lv_is_active(self): self.vg.create_volume('test', '1G') with mock.patch.object(self.vg, '_execute', return_value=['owi-a---', '']): self.assertTrue(self.vg._lv_is_active('test')) with mock.patch.object(self.vg, '_execute', return_value=['owi-----', '']): self.assertFalse(self.vg._lv_is_active('test')) class BrickLvmTestCaseIgnoreFDWarnings(BrickLvmTestCase): def setUp(self): self.configuration = mock.Mock() self.configuration.lvm_suppress_fd_warnings = True super(BrickLvmTestCaseIgnoreFDWarnings, self).setUp() ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1526778 os-brick-3.0.8/os_brick/tests/privileged/0000775000175000017500000000000000000000000020345 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/privileged/__init__.py0000664000175000017500000000000000000000000022444 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/privileged/test_rootwrap.py0000664000175000017500000001564200000000000023643 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from oslo_concurrency import processutils as putils import six from os_brick import exception from os_brick import privileged from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.tests import base class PrivRootwrapTestCase(base.TestCase): def setUp(self): super(PrivRootwrapTestCase, self).setUp() # Bypass privsep and run these simple functions in-process # (allows reading back the modified state of mocks) privileged.default.set_client_mode(False) self.addCleanup(privileged.default.set_client_mode, True) @mock.patch('os_brick.privileged.rootwrap.execute_root') @mock.patch('oslo_concurrency.processutils.execute') def test_execute(self, mock_putils_exec, mock_exec_root): priv_rootwrap.execute('echo', 'foo', run_as_root=False) self.assertFalse(mock_exec_root.called) priv_rootwrap.execute('echo', 'foo', run_as_root=True, root_helper='baz', check_exit_code=0) mock_exec_root.assert_called_once_with( 'echo', 'foo', check_exit_code=0) @mock.patch('oslo_concurrency.processutils.execute') def test_execute_root(self, mock_putils_exec): priv_rootwrap.execute_root('echo', 'foo', check_exit_code=0) mock_putils_exec.assert_called_once_with( 'echo', 'foo', check_exit_code=0, shell=False, run_as_root=False, delay_on_retry=False, on_completion=mock.ANY, on_execute=mock.ANY) # Exact exception isn't particularly important, but these # should be errors: self.assertRaises(TypeError, priv_rootwrap.execute_root, 'foo', shell=True) self.assertRaises(TypeError, priv_rootwrap.execute_root, 'foo', run_as_root=True) @mock.patch('oslo_concurrency.processutils.execute', side_effect=OSError(42, 'mock error')) def test_oserror_raise(self, mock_putils_exec): self.assertRaises(putils.ProcessExecutionError, priv_rootwrap.execute, 'foo') @mock.patch.object(priv_rootwrap.execute_root.privsep_entrypoint, 'client_mode', False) @mock.patch.object(priv_rootwrap, 'custom_execute') def test_execute_as_root(self, exec_mock): res = priv_rootwrap.execute(mock.sentinel.cmds, run_as_root=True, root_helper=mock.sentinel.root_helper, keyword_arg=mock.sentinel.kwarg) self.assertEqual(exec_mock.return_value, res) exec_mock.assert_called_once_with(mock.sentinel.cmds, shell=False, run_as_root=False, keyword_arg=mock.sentinel.kwarg) def test_custom_execute(self): on_execute = mock.Mock() on_completion = mock.Mock() msg = 'hola' out, err = priv_rootwrap.custom_execute('echo', msg, on_execute=on_execute, on_completion=on_completion) self.assertEqual(msg + '\n', out) self.assertEqual('', err) on_execute.assert_called_once_with(mock.ANY) proc = on_execute.call_args[0][0] on_completion.assert_called_once_with(proc) @mock.patch('time.sleep') def test_custom_execute_timeout_raises_with_retries(self, sleep_mock): on_execute = mock.Mock() on_completion = mock.Mock() self.assertRaises(exception.ExecutionTimeout, priv_rootwrap.custom_execute, 'sleep', '2', timeout=0.05, raise_timeout=True, interval=2, backoff_rate=3, attempts=3, on_execute=on_execute, on_completion=on_completion) sleep_mock.assert_has_calls([mock.call(0), mock.call(6), mock.call(0), mock.call(18), mock.call(0)]) expected_calls = [mock.call(args[0][0]) for args in on_execute.call_args_list] on_execute.assert_has_calls(expected_calls) on_completion.assert_has_calls(expected_calls) def test_custom_execute_timeout_no_raise(self): out, err = priv_rootwrap.custom_execute('sleep', '2', timeout=0.05, raise_timeout=False) self.assertEqual('', out) self.assertIsInstance(err, six.string_types) def test_custom_execute_check_exit_code(self): self.assertRaises(putils.ProcessExecutionError, priv_rootwrap.custom_execute, 'ls', '-y', check_exit_code=True) def test_custom_execute_no_check_exit_code(self): out, err = priv_rootwrap.custom_execute('ls', '-y', check_exit_code=False) self.assertEqual('', out) self.assertIsInstance(err, six.string_types) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] priv_rootwrap.unlink_root(*links, no_errors=True) unlink_mock.assert_has_calls([mock.call(links[0]), mock.call(links[1])]) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root_raise(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] self.assertRaises(IOError, priv_rootwrap.unlink_root, *links, no_errors=False) unlink_mock.assert_called_once_with(links[0]) @mock.patch.object(priv_rootwrap.unlink_root.privsep_entrypoint, 'client_mode', False) @mock.patch('os.unlink', side_effect=IOError) def test_unlink_root_raise_at_end(self, unlink_mock): links = ['/dev/disk/by-id/link1', '/dev/disk/by-id/link2'] self.assertRaises(exception.ExceptionChainer, priv_rootwrap.unlink_root, *links, raise_at_end=True) unlink_mock.assert_has_calls([mock.call(links[0]), mock.call(links[1])]) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1526778 os-brick-3.0.8/os_brick/tests/remotefs/0000775000175000017500000000000000000000000020037 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/remotefs/__init__.py0000664000175000017500000000000000000000000022136 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/remotefs/test_remotefs.py0000664000175000017500000002722100000000000023300 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock import os import tempfile from oslo_concurrency import processutils as putils import six from os_brick import exception from os_brick.privileged import rootwrap as priv_rootwrap from os_brick.remotefs import remotefs from os_brick.tests import base class RemoteFsClientTestCase(base.TestCase): def setUp(self): super(RemoteFsClientTestCase, self).setUp() self.mock_execute = self.mock_object(priv_rootwrap, 'execute', return_value=None) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_cifs(self, mock_read_mounts): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') share = '10.0.0.1:/qwe' mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('mount', '-t', 'cifs', share, mount_point, run_as_root=True, root_helper='true', check_exit_code=0)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_nfs(self, mock_read_mounts): client = remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/mnt') share = '10.0.0.1:/qwe' mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('mount', '-t', 'nfs', '-o', 'vers=4,minorversion=1', share, mount_point, check_exit_code=0, run_as_root=True, root_helper='true')] self.mock_execute.assert_has_calls(calls) def test_read_mounts(self): mounts = """device1 mnt_point1 ext4 rw,seclabel,relatime 0 0 device2 mnt_point2 ext4 rw,seclabel,relatime 0 0""" mockopen = mock.mock_open(read_data=mounts) mockopen.return_value.__iter__ = lambda self: iter(self.readline, '') with mock.patch.object(six.moves.builtins, "open", mockopen, create=True): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') ret = client._read_mounts() self.assertEqual(ret, {'mnt_point1': 'device1', 'mnt_point2': 'device2'}) @mock.patch.object(priv_rootwrap, 'execute') @mock.patch.object(remotefs.RemoteFsClient, '_do_mount') def test_mount_already_mounted(self, mock_do_mount, mock_execute): share = "10.0.0.1:/share" client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/mnt') mounts = {client.get_mount_point(share): 'some_dev'} with mock.patch.object(client, '_read_mounts', return_value=mounts): client.mount(share) self.assertEqual(mock_do_mount.call_count, 0) self.assertEqual(mock_execute.call_count, 0) @mock.patch.object(priv_rootwrap, 'execute') def test_mount_race(self, mock_execute): err_msg = 'mount.nfs: /var/asdf is already mounted' mock_execute.side_effect = putils.ProcessExecutionError(stderr=err_msg) mounts = {'192.0.2.20:/share': '/var/asdf/'} client = remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/var/asdf') with mock.patch.object(client, '_read_mounts', return_value=mounts): client._do_mount('nfs', '192.0.2.20:/share', '/var/asdf') @mock.patch.object(priv_rootwrap, 'execute') def test_mount_failure(self, mock_execute): err_msg = 'mount.nfs: nfs broke' mock_execute.side_effect = putils.ProcessExecutionError(stderr=err_msg) client = remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/var/asdf') self.assertRaises(putils.ProcessExecutionError, client._do_mount, 'nfs', '192.0.2.20:/share', '/var/asdf') def _test_no_mount_point(self, fs_type): self.assertRaises(exception.InvalidParameterValue, remotefs.RemoteFsClient, fs_type, root_helper='true') def test_no_mount_point_nfs(self): self._test_no_mount_point('nfs') def test_no_mount_point_cifs(self): self._test_no_mount_point('cifs') def test_no_mount_point_glusterfs(self): self._test_no_mount_point('glusterfs') def test_no_mount_point_vzstorage(self): self._test_no_mount_point('vzstorage') def test_no_mount_point_quobyte(self): self._test_no_mount_point('quobyte') def test_invalid_fs(self): self.assertRaises(exception.ProtocolNotSupported, remotefs.RemoteFsClient, 'my_fs', root_helper='true') def test_init_sets_mount_base(self): client = remotefs.RemoteFsClient("cifs", root_helper='true', smbfs_mount_point_base='/fake', cifs_mount_point_base='/fake2') # Tests that although the FS type is "cifs", the config option # starts with "smbfs_" self.assertEqual('/fake', client._mount_base) @mock.patch('os_brick.remotefs.remotefs.RemoteFsClient._check_nfs_options') def test_init_nfs_calls_check_nfs_options(self, mock_check_nfs_options): remotefs.RemoteFsClient("nfs", root_helper='true', nfs_mount_point_base='/fake') mock_check_nfs_options.assert_called_once_with() class VZStorageRemoteFSClientTestVase(RemoteFsClientTestCase): @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_by_cluster_name(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') share = 'qwe' cluster_name = share mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_with_auth(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') cluster_name = 'qwe' password = '123456' share = '%s:%s' % (cluster_name, password) mount_point = client.get_mount_point(share) client.mount(share) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('pstorage', '-c', cluster_name, 'auth-node', '-P', process_input=password, root_helper='true', run_as_root=True), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch('os.path.exists', return_value=False) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_with_mds_list(self, mock_read_mounts, mock_exists): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') cluster_name = 'qwe' mds_list = ['10.0.0.1', '10.0.0.2'] share = '%s:/%s' % (','.join(mds_list), cluster_name) mount_point = client.get_mount_point(share) vz_conf_dir = os.path.join('/etc/pstorage/clusters/', cluster_name) tmp_dir = '/tmp/fake_dir/' with mock.patch.object(tempfile, 'mkdtemp', return_value=tmp_dir): mock_open = mock.mock_open() with mock.patch.object(six.moves.builtins, "open", mock_open, create=True): client.mount(share) write_calls = [mock.call(tmp_dir + 'bs_list', 'w'), mock.call().__enter__(), mock.call().write('10.0.0.1\n'), mock.call().write('10.0.0.2\n'), mock.call().__exit__(None, None, None)] mock_open.assert_has_calls(write_calls) calls = [mock.call('mkdir', '-p', mount_point, check_exit_code=0), mock.call('cp', '-rf', tmp_dir, vz_conf_dir, run_as_root=True, root_helper='true'), mock.call('chown', '-R', 'root:root', vz_conf_dir, run_as_root=True, root_helper='true'), mock.call('pstorage-mount', '-c', cluster_name, mount_point, root_helper='true', check_exit_code=0, run_as_root=True)] self.mock_execute.assert_has_calls(calls) @mock.patch.object(remotefs.RemoteFsClient, '_read_mounts', return_value=[]) def test_vzstorage_invalid_share(self, mock_read_mounts): client = remotefs.VZStorageRemoteFSClient( "vzstorage", root_helper='true', vzstorage_mount_point_base='/mnt') self.assertRaises(exception.BrickException, client.mount, ':') class ScalityRemoteFsClientTestCase(base.TestCase): def test_no_mount_point_scality(self): self.assertRaises(exception.InvalidParameterValue, remotefs.ScalityRemoteFsClient, 'scality', root_helper='true') def test_get_mount_point(self): fsclient = remotefs.ScalityRemoteFsClient( 'scality', root_helper='true', scality_mount_point_base='/fake') self.assertEqual('/fake/path/00', fsclient.get_mount_point('path')) @mock.patch('oslo_concurrency.processutils.execute', return_value=None) @mock.patch('os_brick.remotefs.remotefs.RemoteFsClient._do_mount') def test_mount(self, mock_do_mount, mock_execute): fsclient = remotefs.ScalityRemoteFsClient( 'scality', root_helper='true', scality_mount_point_base='/fake', execute=putils.execute) with mock.patch.object(fsclient, '_read_mounts', return_value={}): fsclient.mount('fake') mock_execute.assert_called_once_with( 'mkdir', '-p', '/fake', check_exit_code=0) mock_do_mount.assert_called_once_with( 'sofs', '/etc/sfused.conf', '/fake') ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/remotefs/test_windows_remotefs.py0000664000175000017500000001473100000000000025054 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import exception from os_brick.remotefs import windows_remotefs from os_brick.tests import base @ddt.ddt class WindowsRemotefsClientTestCase(base.TestCase): _FAKE_SHARE_NAME = 'fake_share' _FAKE_SHARE_SERVER = 'fake_share_server' _FAKE_SHARE = '\\\\%s\\%s' % (_FAKE_SHARE_SERVER, _FAKE_SHARE_NAME) @mock.patch.object(windows_remotefs, 'utilsfactory') def setUp(self, mock_utilsfactory): super(WindowsRemotefsClientTestCase, self).setUp() self._remotefs = windows_remotefs.WindowsRemoteFsClient( mount_type='smbfs') self._remotefs._mount_base = mock.sentinel.mount_base self._smbutils = self._remotefs._smbutils self._pathutils = self._remotefs._pathutils @ddt.data({'is_local_share': False}, {'expect_existing': False}) @ddt.unpack def test_get_local_share_path_missing(self, expect_existing=True, is_local_share=True): self._smbutils.get_smb_share_path.return_value = None self._smbutils.is_local_share.return_value = is_local_share if expect_existing: self.assertRaises( exception.VolumePathsNotFound, self._remotefs.get_local_share_path, self._FAKE_SHARE, expect_existing=expect_existing) else: share_path = self._remotefs.get_local_share_path( self._FAKE_SHARE, expect_existing=expect_existing) self.assertIsNone(share_path) self.assertEqual(is_local_share, self._smbutils.get_smb_share_path.called) self._smbutils.is_local_share.assert_called_once_with(self._FAKE_SHARE) @ddt.data({'share': '//addr/share_name/subdir_a/subdir_b', 'exp_path': r'C:\shared_dir\subdir_a\subdir_b'}, {'share': '//addr/share_name', 'exp_path': r'C:\shared_dir'}) @ddt.unpack @mock.patch('os.path.join', lambda *args: '\\'.join(args)) def test_get_local_share_path(self, share, exp_path): fake_local_path = 'C:\\shared_dir' self._smbutils.get_smb_share_path.return_value = fake_local_path share_path = self._remotefs.get_local_share_path(share) self.assertEqual(exp_path, share_path) self._smbutils.get_smb_share_path.assert_called_once_with( 'share_name') def test_get_share_name(self): resulted_name = self._remotefs.get_share_name(self._FAKE_SHARE) self.assertEqual(self._FAKE_SHARE_NAME, resulted_name) @ddt.data(True, False) @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, '_create_mount_point') def test_mount(self, is_local_share, mock_create_mount_point): flags = '-o pass=password' self._remotefs._mount_options = '-o user=username,randomopt' self._remotefs._local_path_for_loopback = True self._smbutils.check_smb_mapping.return_value = False self._smbutils.is_local_share.return_value = is_local_share self._remotefs.mount(self._FAKE_SHARE, flags) if is_local_share: self.assertFalse(self._smbutils.check_smb_mapping.called) self.assertFalse(self._smbutils.mount_smb_share.called) else: self._smbutils.check_smb_mapping.assert_called_once_with( self._FAKE_SHARE) self._smbutils.mount_smb_share.assert_called_once_with( self._FAKE_SHARE, username='username', password='password') mock_create_mount_point.assert_called_once_with(self._FAKE_SHARE, is_local_share) def test_unmount(self): self._remotefs.unmount(self._FAKE_SHARE) self._smbutils.unmount_smb_share.assert_called_once_with( self._FAKE_SHARE) @ddt.data({'use_local_path': True}, {'path_exists': True, 'is_symlink': True}, {'path_exists': True}) @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, 'get_local_share_path') @mock.patch.object(windows_remotefs.WindowsRemoteFsClient, 'get_mount_point') @mock.patch.object(windows_remotefs, 'os') @ddt.unpack def test_create_mount_point(self, mock_os, mock_get_mount_point, mock_get_local_share_path, path_exists=False, is_symlink=False, use_local_path=False): mock_os.path.exists.return_value = path_exists mock_os.isdir.return_value = False self._pathutils.is_symlink.return_value = is_symlink if path_exists and not is_symlink: self.assertRaises(exception.BrickException, self._remotefs._create_mount_point, self._FAKE_SHARE, use_local_path) else: self._remotefs._create_mount_point(self._FAKE_SHARE, use_local_path) mock_get_mount_point.assert_called_once_with(self._FAKE_SHARE) mock_os.path.isdir.assert_called_once_with(mock.sentinel.mount_base) if use_local_path: mock_get_local_share_path.assert_called_once_with( self._FAKE_SHARE) expected_symlink_target = mock_get_local_share_path.return_value else: expected_symlink_target = self._FAKE_SHARE.replace('/', '\\') if path_exists: self._pathutils.is_symlink.assert_called_once_with( mock_get_mount_point.return_value) else: self._pathutils.create_sym_link.assert_called_once_with( mock_get_mount_point.return_value, expected_symlink_target) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/test_brick.py0000664000175000017500000000135000000000000020715 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ test_os_brick ---------------------------------- Tests for `os_brick` module. """ from os_brick.tests import base class TestBrick(base.TestCase): def test_something(self): pass ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/test_exception.py0000664000175000017500000000432400000000000021625 0ustar00zuulzuul00000000000000 # Copyright 2010 United States Government as represented by the # Administrator of the National Aeronautics and Space Administration. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import six from os_brick import exception from os_brick.tests import base class BrickExceptionTestCase(base.TestCase): def test_default_error_msg(self): class FakeBrickException(exception.BrickException): message = "default message" exc = FakeBrickException() self.assertEqual(six.text_type(exc), 'default message') def test_error_msg(self): self.assertEqual(six.text_type(exception.BrickException('test')), 'test') def test_default_error_msg_with_kwargs(self): class FakeBrickException(exception.BrickException): message = "default message: %(code)s" exc = FakeBrickException(code=500) self.assertEqual(six.text_type(exc), 'default message: 500') def test_error_msg_exception_with_kwargs(self): class FakeBrickException(exception.BrickException): message = "default message: %(mispelled_code)s" exc = FakeBrickException(code=500) self.assertEqual(six.text_type(exc), 'default message: %(mispelled_code)s') def test_default_error_code(self): class FakeBrickException(exception.BrickException): code = 404 exc = FakeBrickException() self.assertEqual(exc.kwargs['code'], 404) def test_error_code_from_kwarg(self): class FakeBrickException(exception.BrickException): code = 500 exc = FakeBrickException(code=404) self.assertEqual(exc.kwargs['code'], 404) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/test_executor.py0000664000175000017500000001527700000000000021476 0ustar00zuulzuul00000000000000# encoding=utf8 # (c) Copyright 2015 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import threading import mock from oslo_concurrency import processutils as putils from oslo_context import context as context_utils import six import testtools from os_brick import executor as brick_executor from os_brick.privileged import rootwrap from os_brick.tests import base class TestExecutor(base.TestCase): def test_default_execute(self): executor = brick_executor.Executor(root_helper=None) self.assertEqual(rootwrap.execute, executor._Executor__execute) def test_none_execute(self): executor = brick_executor.Executor(root_helper=None, execute=None) self.assertEqual(rootwrap.execute, executor._Executor__execute) def test_fake_execute(self): mock_execute = mock.Mock() executor = brick_executor.Executor(root_helper=None, execute=mock_execute) self.assertEqual(mock_execute, executor._Executor__execute) @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_str_exception(self, execute_mock, stdin_mock): execute_mock.side_effect = putils.ProcessExecutionError( stdout='España', stderr='Zürich') executor = brick_executor.Executor(root_helper=None) exc = self.assertRaises(putils.ProcessExecutionError, executor._execute) self.assertEqual(u'Espa\xf1a', exc.stdout) self.assertEqual(u'Z\xfcrich', exc.stderr) @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_str(self, execute_mock, stdin_mock): execute_mock.return_value = ('España', 'Zürich') executor = brick_executor.Executor(root_helper=None) stdout, stderr = executor._execute() self.assertEqual(u'Espa\xf1a', stdout) self.assertEqual(u'Z\xfcrich', stderr) @testtools.skipUnless(six.PY3, 'Specific test for Python 3') @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_bytes_exception(self, execute_mock, stdin_mock): execute_mock.side_effect = putils.ProcessExecutionError( stdout=six.binary_type('España', 'utf-8'), stderr=six.binary_type('Zürich', 'utf-8')) executor = brick_executor.Executor(root_helper=None) exc = self.assertRaises(putils.ProcessExecutionError, executor._execute) self.assertEqual(u'Espa\xf1a', exc.stdout) self.assertEqual(u'Z\xfcrich', exc.stderr) @testtools.skipUnless(six.PY3, 'Specific test for Python 3') @mock.patch('sys.stdin', encoding='UTF-8') @mock.patch('os_brick.executor.priv_rootwrap.execute') def test_execute_non_safe_bytes(self, execute_mock, stdin_mock): execute_mock.return_value = (six.binary_type('España', 'utf-8'), six.binary_type('Zürich', 'utf-8')) executor = brick_executor.Executor(root_helper=None) stdout, stderr = executor._execute() self.assertEqual(u'Espa\xf1a', stdout) self.assertEqual(u'Z\xfcrich', stderr) class TestThread(base.TestCase): def _store_context(self, result): """Stores current thread's context in result list.""" result.append(context_utils.get_current()) def _run_threads(self, threads): for thread in threads: thread.start() for thread in threads: thread.join() def _do_test(self, thread_class, expected, result=None): if result is None: result = [] threads = [thread_class(target=self._store_context, args=[result]) for i in range(3)] self._run_threads(threads) self.assertEqual([expected] * len(threads), result) def test_normal_thread(self): """Test normal threads don't inherit parent's context.""" context = context_utils.RequestContext() context.update_store() self._do_test(threading.Thread, None) def test_no_context(self, result=None): """Test when parent has no context.""" context_utils._request_store.context = None self._do_test(brick_executor.Thread, None, result) def test_with_context(self, result=None): """Test that our class actually inherits the context.""" context = context_utils.RequestContext() context.update_store() self._do_test(brick_executor.Thread, context, result) def _run_test(self, test_method, test_args, result): """Run one of the normal tests and store the result. Meant to be run in a different thread, thus the need to store the result, because by the time the join call completes the test's stack is no longer available and the exception will have been lost. """ try: test_method(test_args) result.append(True) except Exception: result.append(False) raise def test_no_cross_mix(self): """Test there's no shared global context between threads.""" result = [] contexts = [[], [], []] threads = [threading.Thread(target=self._run_test, args=[self.test_with_context, contexts[0], result]), threading.Thread(target=self._run_test, args=[self.test_no_context, contexts[1], result]), threading.Thread(target=self._run_test, args=[self.test_with_context, contexts[2], result])] self._run_threads(threads) # Check that all tests run without raising an exception self.assertEqual([True, True, True], result) # Check that the context were not shared self.assertNotEqual(contexts[0], contexts[2]) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/test_utils.py0000664000175000017500000002367700000000000021003 0ustar00zuulzuul00000000000000# (c) Copyright 2015 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import functools import time import mock from os_brick import exception from os_brick.tests import base from os_brick import utils class WrongException(exception.BrickException): pass class TestRetryDecorator(base.TestCase): def test_no_retry_required(self): self.counter = 0 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exceptions=exception.VolumeDeviceNotFound, interval=2, retries=3, backoff_rate=2) def succeeds(): self.counter += 1 return 'success' ret = succeeds() self.assertFalse(mock_sleep.called) self.assertEqual(ret, 'success') self.assertEqual(self.counter, 1) def test_retries_once(self): self.counter = 0 interval = 2 backoff_rate = 2 retries = 3 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exception.VolumeDeviceNotFound, interval, retries, backoff_rate) def fails_once(): self.counter += 1 if self.counter < 2: raise exception.VolumeDeviceNotFound(device='fake') else: return 'success' ret = fails_once() self.assertEqual(ret, 'success') self.assertEqual(self.counter, 2) self.assertEqual(mock_sleep.call_count, 1) mock_sleep.assert_called_with(interval * backoff_rate) def test_limit_is_reached(self): self.counter = 0 retries = 3 interval = 2 backoff_rate = 4 with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exception.VolumeDeviceNotFound, interval, retries, backoff_rate) def always_fails(): self.counter += 1 raise exception.VolumeDeviceNotFound(device='fake') self.assertRaises(exception.VolumeDeviceNotFound, always_fails) self.assertEqual(retries, self.counter) expected_sleep_arg = [] for i in range(retries): if i > 0: interval *= backoff_rate expected_sleep_arg.append(float(interval)) mock_sleep.assert_has_calls( list(map(mock.call, expected_sleep_arg))) def test_wrong_exception_no_retry(self): with mock.patch.object(time, 'sleep') as mock_sleep: @utils.retry(exceptions=exception.VolumeDeviceNotFound) def raise_unexpected_error(): raise WrongException("wrong exception") self.assertRaises(WrongException, raise_unexpected_error) self.assertFalse(mock_sleep.called) class LogTracingTestCase(base.TestCase): """Test out the log tracing.""" def test_utils_trace_method_default_logger(self): mock_log = self.mock_object(utils, 'LOG') @utils.trace def _trace_test_method_custom_logger(*args, **kwargs): return 'OK' result = _trace_test_method_custom_logger() self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) def test_utils_trace_method_inner_decorator(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) def _test_decorator(f): def blah(*args, **kwargs): return f(*args, **kwargs) return blah @_test_decorator @utils.trace def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the correct function name was logged for call in mock_log.debug.call_args_list: self.assertIn('_trace_test_method', str(call)) self.assertNotIn('blah', str(call)) def test_utils_trace_method_outer_decorator(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) def _test_decorator(f): def blah(*args, **kwargs): return f(*args, **kwargs) return blah @utils.trace @_test_decorator def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the incorrect function name was logged for call in mock_log.debug.call_args_list: self.assertNotIn('_trace_test_method', str(call)) self.assertIn('blah', str(call)) def test_utils_trace_method_outer_decorator_with_functools(self): mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True self.mock_object(utils.logging, 'getLogger', mock_log) mock_log = self.mock_object(utils, 'LOG') def _test_decorator(f): @functools.wraps(f) def wraps(*args, **kwargs): return f(*args, **kwargs) return wraps @utils.trace @_test_decorator def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method() self.assertEqual('OK', result) self.assertEqual(2, mock_log.debug.call_count) # Ensure the incorrect function name was logged for call in mock_log.debug.call_args_list: self.assertIn('_trace_test_method', str(call)) self.assertNotIn('wraps', str(call)) def test_utils_trace_method_with_exception(self): self.LOG = self.mock_object(utils, 'LOG') @utils.trace def _trace_test_method(*args, **kwargs): raise exception.VolumeDeviceNotFound('test message') self.assertRaises(exception.VolumeDeviceNotFound, _trace_test_method) exception_log = self.LOG.debug.call_args_list[1] self.assertIn('exception', str(exception_log)) self.assertIn('test message', str(exception_log)) def test_utils_trace_method_with_time(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) mock_time = mock.Mock(side_effect=[3.1, 6]) self.mock_object(time, 'time', mock_time) @utils.trace def _trace_test_method(*args, **kwargs): return 'OK' result = _trace_test_method(self) self.assertEqual('OK', result) return_log = mock_log.debug.call_args_list[1] self.assertIn('2900', str(return_log)) def test_utils_trace_method_with_password_dict(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): return {'something': 'test', 'password': 'Now you see me'} result = _trace_test_method(self) expected_unmasked_dict = {'something': 'test', 'password': 'Now you see me'} self.assertEqual(expected_unmasked_dict, result) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'password': '***'", str(mock_log.debug.call_args_list[1])) def test_utils_trace_method_with_password_str(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): return "'adminPass': 'Now you see me'" result = _trace_test_method(self) expected_unmasked_str = "'adminPass': 'Now you see me'" self.assertEqual(expected_unmasked_str, result) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'adminPass': '***'", str(mock_log.debug.call_args_list[1])) def test_utils_trace_method_with_password_in_formal_params(self): mock_logging = self.mock_object(utils, 'logging') mock_log = mock.Mock() mock_log.isEnabledFor = lambda x: True mock_logging.getLogger = mock.Mock(return_value=mock_log) @utils.trace def _trace_test_method(*args, **kwargs): self.assertEqual('verybadpass', kwargs['connection']['data']['auth_password']) pass connector_properties = { 'data': { 'auth_password': 'verybadpass' } } _trace_test_method(self, connection=connector_properties) self.assertEqual(2, mock_log.debug.call_count) self.assertIn("'auth_password': '***'", str(mock_log.debug.call_args_list[0])) ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1526778 os-brick-3.0.8/os_brick/tests/windows/0000775000175000017500000000000000000000000017705 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/__init__.py0000664000175000017500000000000000000000000022004 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/fake_win_conn.py0000664000175000017500000000225200000000000023060 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from os_brick.initiator.windows import base as win_conn_base class FakeWindowsConnector(win_conn_base.BaseWindowsConnector): def connect_volume(self, connection_properties): return {} def disconnect_volume(self, connection_properties, device_info, force=False, ignore_errors=False): pass def get_volume_paths(self, connection_properties): return [] def get_search_path(self): return None def get_all_available_volumes(self, connection_properties=None): return [] ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_base.py0000664000175000017500000000252300000000000022232 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import mock from os_win import utilsfactory from os_brick.tests import base class WindowsConnectorTestBase(base.TestCase): @mock.patch('sys.platform', 'win32') def setUp(self): super(WindowsConnectorTestBase, self).setUp() # All the Windows connectors use os_win.utilsfactory to fetch Windows # specific utils. During init, those will run methods that will fail # on other platforms. To make testing easier and avoid checking the # platform in the code, we can simply mock this factory method. utilsfactory_patcher = mock.patch.object( utilsfactory, '_get_class') utilsfactory_patcher.start() self.addCleanup(utilsfactory_patcher.stop) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_base_connector.py0000664000175000017500000001315500000000000024307 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock import six from os_brick import exception from os_brick.initiator.windows import base as base_win_conn from os_brick.tests.windows import fake_win_conn from os_brick.tests.windows import test_base @ddt.ddt class BaseWindowsConnectorTestCase(test_base.WindowsConnectorTestBase): def setUp(self): super(BaseWindowsConnectorTestCase, self).setUp() self._diskutils = mock.Mock() self._connector = fake_win_conn.FakeWindowsConnector() self._connector._diskutils = self._diskutils @ddt.data({}, {'feature_available': True}, {'feature_available': False, 'enforce_multipath': True}) @ddt.unpack @mock.patch.object(base_win_conn.utilsfactory, 'get_hostutils') def test_check_multipath_support(self, mock_get_hostutils, feature_available=True, enforce_multipath=False): mock_hostutils = mock_get_hostutils.return_value mock_hostutils.check_server_feature.return_value = feature_available check_mpio = base_win_conn.BaseWindowsConnector.check_multipath_support if feature_available or not enforce_multipath: multipath_support = check_mpio( enforce_multipath=enforce_multipath) self.assertEqual(feature_available, multipath_support) else: self.assertRaises(exception.BrickException, check_mpio, enforce_multipath=enforce_multipath) mock_hostutils.check_server_feature.assert_called_once_with( mock_hostutils.FEATURE_MPIO) @ddt.data({}, {'mpio_requested': False}, {'mpio_available': True}) @mock.patch.object(base_win_conn.BaseWindowsConnector, 'check_multipath_support') @ddt.unpack def test_get_connector_properties(self, mock_check_mpio, mpio_requested=True, mpio_available=True): mock_check_mpio.return_value = mpio_available enforce_multipath = False props = base_win_conn.BaseWindowsConnector.get_connector_properties( multipath=mpio_requested, enforce_multipath=enforce_multipath) self.assertEqual(mpio_requested and mpio_available, props['multipath']) if mpio_requested: mock_check_mpio.assert_called_once_with(enforce_multipath) def test_get_scsi_wwn(self): mock_get_uid_and_type = self._diskutils.get_disk_uid_and_uid_type mock_get_uid_and_type.return_value = (mock.sentinel.disk_uid, mock.sentinel.uid_type) scsi_wwn = self._connector._get_scsi_wwn(mock.sentinel.dev_num) expected_wwn = '%s%s' % (mock.sentinel.uid_type, mock.sentinel.disk_uid) self.assertEqual(expected_wwn, scsi_wwn) mock_get_uid_and_type.assert_called_once_with(mock.sentinel.dev_num) @ddt.data(None, IOError) @mock.patch.object(six.moves.builtins, 'open') def test_check_valid_device(self, exc, mock_open): mock_open.side_effect = exc valid_device = self._connector.check_valid_device( mock.sentinel.dev_path) self.assertEqual(not exc, valid_device) mock_open.assert_any_call(mock.sentinel.dev_path, 'r') mock_read = mock_open.return_value.__enter__.return_value.read if not exc: mock_read.assert_called_once_with(1) def test_check_device_paths(self): # We expect an exception to be raised if the same volume # can be accessed through multiple paths. device_paths = [mock.sentinel.dev_path_0, mock.sentinel.dev_path_1] self.assertRaises(exception.BrickException, self._connector._check_device_paths, device_paths) @mock.patch.object(fake_win_conn.FakeWindowsConnector, 'get_volume_paths') def test_extend_volume(self, mock_get_vol_paths): mock_vol_paths = [mock.sentinel.dev_path] mock_get_vol_paths.return_value = mock_vol_paths self._connector.extend_volume(mock.sentinel.conn_props) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) mock_get_dev_num = self._diskutils.get_device_number_from_device_name mock_get_dev_num.assert_called_once_with(mock.sentinel.dev_path) self._diskutils.refresh_disk.assert_called_once_with( mock_get_dev_num.return_value) @mock.patch.object(fake_win_conn.FakeWindowsConnector, 'get_volume_paths') def test_extend_volume_missing_path(self, mock_get_vol_paths): mock_get_vol_paths.return_value = [] self.assertRaises(exception.NotFound, self._connector.extend_volume, mock.sentinel.conn_props) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_factory.py0000664000175000017500000000301300000000000022762 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_brick import initiator from os_brick.initiator import connector from os_brick.initiator.windows import fibre_channel from os_brick.initiator.windows import iscsi from os_brick.initiator.windows import smbfs from os_brick.tests.windows import test_base @ddt.ddt class WindowsConnectorFactoryTestCase(test_base.WindowsConnectorTestBase): @ddt.data({'proto': initiator.ISCSI, 'expected_cls': iscsi.WindowsISCSIConnector}, {'proto': initiator.FIBRE_CHANNEL, 'expected_cls': fibre_channel.WindowsFCConnector}, {'proto': initiator.SMBFS, 'expected_cls': smbfs.WindowsSMBFSConnector}) @ddt.unpack @mock.patch('sys.platform', 'win32') def test_factory(self, proto, expected_cls): obj = connector.InitiatorConnector.factory(proto, None) self.assertIsInstance(obj, expected_cls) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_fibre_channel.py0000664000175000017500000002655300000000000024110 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_win import exceptions as os_win_exc from os_brick import exception from os_brick.initiator.windows import fibre_channel as fc from os_brick.tests.windows import test_base @ddt.ddt class WindowsFCConnectorTestCase(test_base.WindowsConnectorTestBase): def setUp(self): super(WindowsFCConnectorTestCase, self).setUp() self._connector = fc.WindowsFCConnector( device_scan_interval=mock.sentinel.rescan_interval) self._diskutils = self._connector._diskutils self._fc_utils = self._connector._fc_utils @ddt.data(True, False) @mock.patch.object(fc.utilsfactory, 'get_fc_utils') def test_get_volume_connector_props(self, valid_fc_hba_ports, mock_get_fc_utils): fake_fc_hba_ports = [{'node_name': mock.sentinel.node_name, 'port_name': mock.sentinel.port_name}, {'node_name': mock.sentinel.second_node_name, 'port_name': mock.sentinel.second_port_name}] self._fc_utils = mock_get_fc_utils.return_value self._fc_utils.get_fc_hba_ports.return_value = ( fake_fc_hba_ports if valid_fc_hba_ports else []) props = self._connector.get_connector_properties() self._fc_utils.refresh_hba_configuration.assert_called_once_with() self._fc_utils.get_fc_hba_ports.assert_called_once_with() if valid_fc_hba_ports: expected_props = { 'wwpns': [mock.sentinel.port_name, mock.sentinel.second_port_name], 'wwnns': [mock.sentinel.node_name, mock.sentinel.second_node_name] } else: expected_props = {} self.assertItemsEqual(expected_props, props) @mock.patch.object(fc.WindowsFCConnector, '_get_scsi_wwn') @mock.patch.object(fc.WindowsFCConnector, 'get_volume_paths') def test_connect_volume(self, mock_get_vol_paths, mock_get_scsi_wwn): mock_get_vol_paths.return_value = [mock.sentinel.dev_name] mock_get_dev_num = self._diskutils.get_device_number_from_device_name mock_get_dev_num.return_value = mock.sentinel.dev_num expected_device_info = dict(type='block', path=mock.sentinel.dev_name, number=mock.sentinel.dev_num, scsi_wwn=mock_get_scsi_wwn.return_value) device_info = self._connector.connect_volume(mock.sentinel.conn_props) self.assertEqual(expected_device_info, device_info) mock_get_vol_paths.assert_called_once_with(mock.sentinel.conn_props) mock_get_dev_num.assert_called_once_with(mock.sentinel.dev_name) mock_get_scsi_wwn.assert_called_once_with(mock.sentinel.dev_num) @mock.patch.object(fc.WindowsFCConnector, 'get_volume_paths') def test_connect_volume_not_found(self, mock_get_vol_paths): mock_get_vol_paths.return_value = [] self.assertRaises(exception.NoFibreChannelVolumeDeviceFound, self._connector.connect_volume, mock.sentinel.conn_props) @ddt.data({'volume_mappings': [], 'expected_paths': []}, {'volume_mappings': [dict(device_name='', fcp_lun=mock.sentinel.fcp_lun)] * 3, 'scsi_id_side_eff': os_win_exc.OSWinException, 'expected_paths': []}, {'volume_mappings': [dict(device_name='', fcp_lun=mock.sentinel.fcp_lun), dict(device_name=mock.sentinel.disk_path)], 'expected_paths': [mock.sentinel.disk_path]}, {'volume_mappings': [dict(device_name='', fcp_lun=mock.sentinel.fcp_lun)], 'scsi_id_side_eff': [[mock.sentinel.disk_path]], 'expected_paths': [mock.sentinel.disk_path]}, {'volume_mappings': [dict(device_name=mock.sentinel.disk_path)], 'use_multipath': True, 'is_mpio_disk': True, 'expected_paths': [mock.sentinel.disk_path]}, {'volume_mappings': [dict(device_name=mock.sentinel.disk_path)], 'use_multipath': True, 'is_mpio_disk': False, 'expected_paths': []}) @ddt.unpack @mock.patch('time.sleep') @mock.patch.object(fc.WindowsFCConnector, '_get_fc_volume_mappings') @mock.patch.object(fc.WindowsFCConnector, '_get_disk_paths_by_scsi_id') def test_get_volume_paths(self, mock_get_disk_paths_by_scsi_id, mock_get_fc_mappings, mock_sleep, volume_mappings, expected_paths, scsi_id_side_eff=None, use_multipath=False, is_mpio_disk=False): mock_get_dev_num = self._diskutils.get_device_number_from_device_name mock_get_fc_mappings.return_value = volume_mappings mock_get_disk_paths_by_scsi_id.side_effect = scsi_id_side_eff self._diskutils.is_mpio_disk.return_value = is_mpio_disk self._connector.use_multipath = use_multipath vol_paths = self._connector.get_volume_paths(mock.sentinel.conn_props) self.assertEqual(expected_paths, vol_paths) # In this test case, either the volume is found after the first # attempt, either it's not found at all, in which case we'd expect # the number of retries to be the requested maximum number of rescans. expected_try_count = (1 if expected_paths else self._connector.device_scan_attempts) self._diskutils.rescan_disks.assert_has_calls( [mock.call()] * expected_try_count) mock_get_fc_mappings.assert_has_calls( [mock.call(mock.sentinel.conn_props)] * expected_try_count) mock_sleep.assert_has_calls( [mock.call(mock.sentinel.rescan_interval)] * (expected_try_count - 1)) dev_names = [mapping['device_name'] for mapping in volume_mappings if mapping['device_name']] if volume_mappings and not dev_names: mock_get_disk_paths_by_scsi_id.assert_any_call( mock.sentinel.conn_props, volume_mappings[0]['fcp_lun']) if expected_paths and use_multipath: mock_get_dev_num.assert_called_once_with(expected_paths[0]) self._diskutils.is_mpio_disk.assert_any_call( mock_get_dev_num.return_value) @mock.patch.object(fc.WindowsFCConnector, '_get_fc_hba_mappings') def test_get_fc_volume_mappings(self, mock_get_fc_hba_mappings): fake_target_wwpn = 'FAKE_TARGET_WWPN' fake_conn_props = dict(target_lun=mock.sentinel.target_lun, target_wwn=[fake_target_wwpn]) mock_hba_mappings = {mock.sentinel.node_name: mock.sentinel.hba_ports} mock_get_fc_hba_mappings.return_value = mock_hba_mappings all_target_mappings = [{'device_name': mock.sentinel.dev_name, 'port_name': fake_target_wwpn, 'lun': mock.sentinel.target_lun}, {'device_name': mock.sentinel.dev_name_1, 'port_name': mock.sentinel.target_port_name_1, 'lun': mock.sentinel.target_lun}, {'device_name': mock.sentinel.dev_name, 'port_name': mock.sentinel.target_port_name, 'lun': mock.sentinel.target_lun_1}] expected_mappings = [all_target_mappings[0]] self._fc_utils.get_fc_target_mappings.return_value = ( all_target_mappings) volume_mappings = self._connector._get_fc_volume_mappings( fake_conn_props) self.assertEqual(expected_mappings, volume_mappings) def test_get_fc_hba_mappings(self): fake_fc_hba_ports = [{'node_name': mock.sentinel.node_name, 'port_name': mock.sentinel.port_name}] self._fc_utils.get_fc_hba_ports.return_value = fake_fc_hba_ports resulted_mappings = self._connector._get_fc_hba_mappings() expected_mappings = { mock.sentinel.node_name: [mock.sentinel.port_name]} self.assertEqual(expected_mappings, resulted_mappings) @mock.patch.object(fc.WindowsFCConnector, '_get_dev_nums_by_scsi_id') def test_get_disk_paths_by_scsi_id(self, mock_get_dev_nums): remote_wwpns = [mock.sentinel.remote_wwpn_0, mock.sentinel.remote_wwpn_1] fake_init_target_map = {mock.sentinel.local_wwpn: remote_wwpns} conn_props = dict(initiator_target_map=fake_init_target_map) mock_get_dev_nums.side_effect = [os_win_exc.FCException, [mock.sentinel.dev_num]] mock_get_dev_name = self._diskutils.get_device_name_by_device_number mock_get_dev_name.return_value = mock.sentinel.dev_name disk_paths = self._connector._get_disk_paths_by_scsi_id( conn_props, mock.sentinel.fcp_lun) self.assertEqual([mock.sentinel.dev_name], disk_paths) mock_get_dev_nums.assert_has_calls([ mock.call(mock.sentinel.local_wwpn, remote_wwpn, mock.sentinel.fcp_lun) for remote_wwpn in remote_wwpns]) mock_get_dev_name.assert_called_once_with(mock.sentinel.dev_num) @mock.patch.object(fc.WindowsFCConnector, '_get_fc_hba_wwn_for_port') def test_get_dev_nums_by_scsi_id(self, mock_get_fc_hba_wwn): fake_identifier = dict(id=mock.sentinel.id, type=mock.sentinel.type) mock_get_fc_hba_wwn.return_value = mock.sentinel.local_wwnn self._fc_utils.get_scsi_device_identifiers.return_value = [ fake_identifier] self._diskutils.get_disk_numbers_by_unique_id.return_value = ( mock.sentinel.dev_nums) dev_nums = self._connector._get_dev_nums_by_scsi_id( mock.sentinel.local_wwpn, mock.sentinel.remote_wwpn, mock.sentinel.fcp_lun) self.assertEqual(mock.sentinel.dev_nums, dev_nums) mock_get_fc_hba_wwn.assert_called_once_with(mock.sentinel.local_wwpn) self._fc_utils.get_scsi_device_identifiers.assert_called_once_with( mock.sentinel.local_wwnn, mock.sentinel.local_wwpn, mock.sentinel.remote_wwpn, mock.sentinel.fcp_lun) self._diskutils.get_disk_numbers_by_unique_id.assert_called_once_with( unique_id=mock.sentinel.id, unique_id_format=mock.sentinel.type) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_iscsi.py0000664000175000017500000002162600000000000022437 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import ddt import mock from os_win import exceptions as os_win_exc from os_brick import exception from os_brick.initiator.windows import iscsi from os_brick.tests.windows import test_base @ddt.ddt class WindowsISCSIConnectorTestCase(test_base.WindowsConnectorTestBase): @mock.patch.object(iscsi.WindowsISCSIConnector, 'validate_initiators') def setUp(self, mock_validate_connectors): super(WindowsISCSIConnectorTestCase, self).setUp() self._diskutils = mock.Mock() self._iscsi_utils = mock.Mock() self._connector = iscsi.WindowsISCSIConnector( device_scan_interval=mock.sentinel.rescan_interval) self._connector._diskutils = self._diskutils self._connector._iscsi_utils = self._iscsi_utils @ddt.data({'requested_initiators': [mock.sentinel.initiator_0], 'available_initiators': [mock.sentinel.initiator_0, mock.sentinel.initiator_1]}, {'requested_initiators': [mock.sentinel.initiator_0], 'available_initiators': [mock.sentinel.initiator_1]}, {'requested_initiators': [], 'available_initiators': [mock.sentinel.software_initiator]}) @ddt.unpack def test_validate_initiators(self, requested_initiators, available_initiators): self._iscsi_utils.get_iscsi_initiators.return_value = ( available_initiators) self._connector.initiator_list = requested_initiators expected_valid_initiator = not ( set(requested_initiators).difference(set(available_initiators))) valid_initiator = self._connector.validate_initiators() self.assertEqual(expected_valid_initiator, valid_initiator) def test_get_initiator(self): initiator = self._connector.get_initiator() self.assertEqual(self._iscsi_utils.get_iscsi_initiator.return_value, initiator) @mock.patch.object(iscsi, 'utilsfactory') def test_get_connector_properties(self, mock_utilsfactory): mock_iscsi_utils = ( mock_utilsfactory.get_iscsi_initiator_utils.return_value) props = self._connector.get_connector_properties() expected_props = dict( initiator=mock_iscsi_utils.get_iscsi_initiator.return_value) self.assertEqual(expected_props, props) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') def test_get_all_paths(self, mock_get_all_targets): initiators = [mock.sentinel.initiator_0, mock.sentinel.initiator_1] all_targets = [(mock.sentinel.portal_0, mock.sentinel.target_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.target_1, mock.sentinel.lun_1)] self._connector.initiator_list = initiators mock_get_all_targets.return_value = all_targets expected_paths = [ (initiator_name, target_portal, target_iqn, target_lun) for target_portal, target_iqn, target_lun in all_targets for initiator_name in initiators] all_paths = self._connector._get_all_paths(mock.sentinel.conn_props) self.assertEqual(expected_paths, all_paths) mock_get_all_targets.assert_called_once_with(mock.sentinel.conn_props) @ddt.data(True, False) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_scsi_wwn') @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_paths') def test_connect_volume(self, use_multipath, mock_get_all_paths, mock_get_scsi_wwn): fake_paths = [(mock.sentinel.initiator_name, mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun)] * 3 fake_conn_props = dict(auth_username=mock.sentinel.auth_username, auth_password=mock.sentinel.auth_password) mock_get_all_paths.return_value = fake_paths self._iscsi_utils.login_storage_target.side_effect = [ os_win_exc.OSWinException, None, None] self._iscsi_utils.get_device_number_and_path.return_value = ( mock.sentinel.device_number, mock.sentinel.device_path) self._connector.use_multipath = use_multipath device_info = self._connector.connect_volume(fake_conn_props) expected_device_info = dict(type='block', path=mock.sentinel.device_path, number=mock.sentinel.device_number, scsi_wwn=mock_get_scsi_wwn.return_value) self.assertEqual(expected_device_info, device_info) mock_get_all_paths.assert_called_once_with(fake_conn_props) expected_login_attempts = 3 if use_multipath else 2 self._iscsi_utils.login_storage_target.assert_has_calls( [mock.call(target_lun=mock.sentinel.target_lun, target_iqn=mock.sentinel.target_iqn, target_portal=mock.sentinel.target_portal, auth_username=mock.sentinel.auth_username, auth_password=mock.sentinel.auth_password, mpio_enabled=use_multipath, initiator_name=mock.sentinel.initiator_name, ensure_lun_available=False)] * expected_login_attempts) self._iscsi_utils.get_device_number_and_path.assert_called_once_with( mock.sentinel.target_iqn, mock.sentinel.target_lun, retry_attempts=self._connector.device_scan_attempts, retry_interval=self._connector.device_scan_interval, rescan_disks=True, ensure_mpio_claimed=use_multipath) mock_get_scsi_wwn.assert_called_once_with(mock.sentinel.device_number) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_paths') def test_connect_volume_exc(self, mock_get_all_paths): fake_paths = [(mock.sentinel.initiator_name, mock.sentinel.target_portal, mock.sentinel.target_iqn, mock.sentinel.target_lun)] * 3 mock_get_all_paths.return_value = fake_paths self._iscsi_utils.login_storage_target.side_effect = ( os_win_exc.OSWinException) self._connector.use_multipath = True self.assertRaises(exception.BrickException, self._connector.connect_volume, connection_properties={}) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') def test_disconnect_volume(self, mock_get_all_targets): targets = [ (mock.sentinel.portal_0, mock.sentinel.tg_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.tg_1, mock.sentinel.lun_1)] mock_get_all_targets.return_value = targets self._iscsi_utils.get_target_luns.return_value = [mock.sentinel.lun_0] self._connector.disconnect_volume(mock.sentinel.conn_props, mock.sentinel.dev_info) self._diskutils.rescan_disks.assert_called_once_with() mock_get_all_targets.assert_called_once_with(mock.sentinel.conn_props) self._iscsi_utils.logout_storage_target.assert_called_once_with( mock.sentinel.tg_0) self._iscsi_utils.get_target_luns.assert_has_calls( [mock.call(mock.sentinel.tg_0), mock.call(mock.sentinel.tg_1)]) @mock.patch.object(iscsi.WindowsISCSIConnector, '_get_all_targets') @mock.patch.object(iscsi.WindowsISCSIConnector, '_check_device_paths') def test_get_volume_paths(self, mock_check_dev_paths, mock_get_all_targets): targets = [ (mock.sentinel.portal_0, mock.sentinel.tg_0, mock.sentinel.lun_0), (mock.sentinel.portal_1, mock.sentinel.tg_1, mock.sentinel.lun_1)] mock_get_all_targets.return_value = targets self._iscsi_utils.get_device_number_and_path.return_value = [ mock.sentinel.dev_num, mock.sentinel.dev_path] volume_paths = self._connector.get_volume_paths( mock.sentinel.conn_props) expected_paths = [mock.sentinel.dev_path] self.assertEqual(expected_paths, volume_paths) mock_check_dev_paths.assert_called_once_with(set(expected_paths)) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/tests/windows/test_smbfs.py0000664000175000017500000001705200000000000022435 0ustar00zuulzuul00000000000000# Copyright 2016 Cloudbase Solutions Srl # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import ddt import mock from os_brick.initiator.windows import smbfs from os_brick.remotefs import windows_remotefs from os_brick.tests.windows import test_base @ddt.ddt class WindowsSMBFSConnectorTestCase(test_base.WindowsConnectorTestBase): def setUp(self): super(WindowsSMBFSConnectorTestCase, self).setUp() self._load_connector() @mock.patch.object(windows_remotefs, 'WindowsRemoteFsClient') def _load_connector(self, mock_remotefs_cls, *args, **kwargs): self._connector = smbfs.WindowsSMBFSConnector(*args, **kwargs) self._remotefs = mock_remotefs_cls.return_value self._vhdutils = self._connector._vhdutils self._diskutils = self._connector._diskutils @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') @mock.patch.object(smbfs.WindowsSMBFSConnector, 'ensure_share_mounted') def test_connect_volume(self, mock_ensure_mounted, mock_get_disk_path): device_info = self._connector.connect_volume(mock.sentinel.conn_props) expected_info = dict(type='file', path=mock_get_disk_path.return_value) self.assertEqual(expected_info, device_info) mock_ensure_mounted.assert_called_once_with(mock.sentinel.conn_props) mock_get_disk_path.assert_called_once_with(mock.sentinel.conn_props) @ddt.data(True, False) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') @mock.patch.object(smbfs.WindowsSMBFSConnector, 'ensure_share_mounted') def test_connect_and_mount_volume(self, read_only, mock_ensure_mounted, mock_get_disk_path): self._load_connector(expect_raw_disk=True) fake_conn_props = dict(access_mode='ro' if read_only else 'rw') self._vhdutils.get_virtual_disk_physical_path.return_value = ( mock.sentinel.raw_disk_path) mock_get_disk_path.return_value = mock.sentinel.image_path device_info = self._connector.connect_volume(fake_conn_props) expected_info = dict(type='file', path=mock.sentinel.raw_disk_path) self.assertEqual(expected_info, device_info) self._vhdutils.attach_virtual_disk.assert_called_once_with( mock.sentinel.image_path, read_only=read_only) self._vhdutils.get_virtual_disk_physical_path.assert_called_once_with( mock.sentinel.image_path) get_dev_num = self._diskutils.get_device_number_from_device_name get_dev_num.assert_called_once_with(mock.sentinel.raw_disk_path) self._diskutils.set_disk_offline.assert_called_once_with( get_dev_num.return_value) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_export_path') def test_disconnect_volume(self, mock_get_export_path, mock_get_disk_path): self._connector.disconnect_volume(mock.sentinel.conn_props, mock.sentinel.dev_info) mock_get_disk_path.assert_called_once_with( mock.sentinel.conn_props) self._vhdutils.detach_virtual_disk.assert_called_once_with( mock_get_disk_path.return_value) self._remotefs.unmount.assert_called_once_with( mock_get_export_path.return_value) mock_get_export_path.assert_called_once_with(mock.sentinel.conn_props) def test_get_export_path(self): fake_export = '//ip/share' fake_conn_props = dict(export=fake_export) expected_export = fake_export.replace('/', '\\') export_path = self._connector._get_export_path(fake_conn_props) self.assertEqual(expected_export, export_path) @ddt.data({}, {'mount_base': mock.sentinel.mount_base}, {'is_local_share': True}, {'is_local_share': True, 'local_path_for_loopbk': True}) @ddt.unpack def test_get_disk_path(self, mount_base=None, local_path_for_loopbk=False, is_local_share=False): fake_mount_point = r'C:\\fake_mount_point' fake_share_name = 'fake_share' fake_local_share_path = 'C:\\%s' % fake_share_name fake_export_path = '\\\\host\\%s' % fake_share_name fake_disk_name = 'fake_disk.vhdx' fake_conn_props = dict(name=fake_disk_name, export=fake_export_path) self._remotefs.get_mount_base.return_value = mount_base self._remotefs.get_mount_point.return_value = fake_mount_point self._remotefs.get_local_share_path.return_value = ( fake_local_share_path) self._remotefs.get_share_name.return_value = fake_share_name self._connector._local_path_for_loopback = local_path_for_loopbk self._connector._smbutils.is_local_share.return_value = is_local_share expecting_local = local_path_for_loopbk and is_local_share if mount_base: expected_export_path = fake_mount_point elif expecting_local: # In this case, we expect the local share export path to be # used directly. expected_export_path = fake_local_share_path else: expected_export_path = fake_export_path expected_disk_path = os.path.join(expected_export_path, fake_disk_name) disk_path = self._connector._get_disk_path(fake_conn_props) self.assertEqual(expected_disk_path, disk_path) if mount_base: self._remotefs.get_mount_point.assert_called_once_with( fake_export_path) elif expecting_local: self._connector._smbutils.is_local_share.assert_called_once_with( fake_export_path) self._remotefs.get_local_share_path.assert_called_once_with( fake_export_path) def test_get_search_path(self): search_path = self._connector.get_search_path() self.assertEqual(search_path, self._remotefs.get_mount_base.return_value) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_disk_path') def test_volume_paths(self, mock_get_disk_path): expected_paths = [mock_get_disk_path.return_value] volume_paths = self._connector.get_volume_paths( mock.sentinel.conn_props) self.assertEqual(expected_paths, volume_paths) mock_get_disk_path.assert_called_once_with( mock.sentinel.conn_props) @mock.patch.object(smbfs.WindowsSMBFSConnector, '_get_export_path') def test_ensure_share_mounted(self, mock_get_export_path): fake_conn_props = dict(options=mock.sentinel.mount_opts) self._connector.ensure_share_mounted(fake_conn_props) self._remotefs.mount.assert_called_once_with( mock_get_export_path.return_value, mock.sentinel.mount_opts) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/utils.py0000664000175000017500000001356100000000000016571 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # """Utilities and helper functions.""" import functools import inspect import logging as py_logging import retrying import six import time from oslo_log import log as logging from oslo_utils import encodeutils from oslo_utils import strutils from os_brick.i18n import _ LOG = logging.getLogger(__name__) def retry(exceptions, interval=1, retries=3, backoff_rate=2): def _retry_on_exception(e): return isinstance(e, exceptions) def _backoff_sleep(previous_attempt_number, delay_since_first_attempt_ms): exp = backoff_rate ** previous_attempt_number wait_for = max(0, interval * exp) LOG.debug("Sleeping for %s seconds", wait_for) return wait_for * 1000.0 def _print_stop(previous_attempt_number, delay_since_first_attempt_ms): delay_since_first_attempt = delay_since_first_attempt_ms / 1000.0 LOG.debug("Failed attempt %s", previous_attempt_number) LOG.debug("Have been at this for %s seconds", delay_since_first_attempt) return previous_attempt_number == retries if retries < 1: raise ValueError(_('Retries must be greater than or ' 'equal to 1 (received: %s). ') % retries) def _decorator(f): @six.wraps(f) def _wrapper(*args, **kwargs): r = retrying.Retrying(retry_on_exception=_retry_on_exception, wait_func=_backoff_sleep, stop_func=_print_stop) return r.call(f, *args, **kwargs) return _wrapper return _decorator def platform_matches(current_platform, connector_platform): curr_p = current_platform.upper() conn_p = connector_platform.upper() if conn_p == 'ALL': return True # Add tests against families of platforms if curr_p == conn_p: return True return False def os_matches(current_os, connector_os): curr_os = current_os.upper() conn_os = connector_os.upper() if conn_os == 'ALL': return True # add tests against OSs if (conn_os == curr_os or conn_os in curr_os): return True return False def merge_dict(dict1, dict2): """Try to safely merge 2 dictionaries.""" if type(dict1) is not dict: raise Exception("dict1 is not a dictionary") if type(dict2) is not dict: raise Exception("dict2 is not a dictionary") dict3 = dict1.copy() dict3.update(dict2) return dict3 def trace(f): """Trace calls to the decorated function. This decorator should always be defined as the outermost decorator so it is defined last. This is important so it does not interfere with other decorators. Using this decorator on a function will cause its execution to be logged at `DEBUG` level with arguments, return values, and exceptions. :returns: a function decorator """ func_name = f.__name__ @functools.wraps(f) def trace_logging_wrapper(*args, **kwargs): if len(args) > 0: maybe_self = args[0] else: maybe_self = kwargs.get('self', None) if maybe_self and hasattr(maybe_self, '__module__'): logger = logging.getLogger(maybe_self.__module__) else: logger = LOG # NOTE(ameade): Don't bother going any further if DEBUG log level # is not enabled for the logger. if not logger.isEnabledFor(py_logging.DEBUG): return f(*args, **kwargs) all_args = inspect.getcallargs(f, *args, **kwargs) logger.debug('==> %(func)s: call %(all_args)r', {'func': func_name, # NOTE(mriedem): We have to stringify the dict first # and don't use mask_dict_password because it results in # an infinite recursion failure. 'all_args': strutils.mask_password( six.text_type(all_args))}) start_time = time.time() * 1000 try: result = f(*args, **kwargs) except Exception as exc: total_time = int(round(time.time() * 1000)) - start_time logger.debug('<== %(func)s: exception (%(time)dms) %(exc)r', {'func': func_name, 'time': total_time, 'exc': exc}) raise total_time = int(round(time.time() * 1000)) - start_time if isinstance(result, dict): mask_result = strutils.mask_dict_password(result) elif isinstance(result, six.string_types): mask_result = strutils.mask_password(result) else: mask_result = result logger.debug('<== %(func)s: return (%(time)dms) %(result)r', {'func': func_name, 'time': total_time, 'result': mask_result}) return result return trace_logging_wrapper def convert_str(text): """Convert to native string. Convert bytes and Unicode strings to native strings: * convert to bytes on Python 2: encode Unicode using encodeutils.safe_encode() * convert to Unicode on Python 3: decode bytes from UTF-8 """ if six.PY2: return encodeutils.to_utf8(text) else: if isinstance(text, bytes): return text.decode('utf-8') else: return text ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/os_brick/version.py0000664000175000017500000000131300000000000017106 0ustar00zuulzuul00000000000000# All Rights Reserved # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # import pbr.version version_info = pbr.version.VersionInfo('os-brick') __version__ = version_info.version_string() ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1446776 os-brick-3.0.8/os_brick.egg-info/0000775000175000017500000000000000000000000016543 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/PKG-INFO0000664000175000017500000000510700000000000017643 0ustar00zuulzuul00000000000000Metadata-Version: 1.1 Name: os-brick Version: 3.0.8 Summary: OpenStack Cinder brick library for managing local volume attaches Home-page: https://docs.openstack.org/os-brick/ Author: OpenStack Author-email: openstack-discuss@lists.openstack.org License: UNKNOWN Description: ======================== Team and repository tags ======================== .. image:: https://governance.openstack.org/tc/badges/os-brick.svg :target: https://governance.openstack.org/tc/reference/tags/index.html .. Change things from this point on ===== brick ===== .. image:: https://img.shields.io/pypi/v/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Latest Version .. image:: https://img.shields.io/pypi/dm/os-brick.svg :target: https://pypi.org/project/os-brick/ :alt: Downloads OpenStack Cinder brick library for managing local volume attaches Features -------- * Discovery of volumes being attached to a host for many transport protocols. * Removal of volumes from a host. Hacking ------- Hacking on brick requires python-gdbm (for Debian derived distributions), Python 2.7 and Python 3.4. A recent tox is required, as is a recent virtualenv (13.1.0 or newer). If "tox -e py34" fails with the error "db type could not be determined", remove the .testrepository/ directory and then run "tox -e py34". For any other information, refer to the developer documents: https://docs.openstack.org/os-brick/latest/ OR refer to the parent project, Cinder: https://docs.openstack.org/cinder/latest/ Release notes for the project can be found at: https://docs.openstack.org/releasenotes/os-brick * License: Apache License, Version 2.0 * Source: https://opendev.org/openstack/os-brick * Bugs: https://bugs.launchpad.net/os-brick Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/SOURCES.txt0000664000175000017500000002012700000000000020431 0ustar00zuulzuul00000000000000.coveragerc .mailmap .stestr.conf .zuul.yaml AUTHORS CONTRIBUTING.rst ChangeLog HACKING.rst LICENSE README.rst babel.cfg bindep.txt lower-constraints.txt pylintrc requirements.txt setup.cfg setup.py test-requirements.txt tox.ini doc/requirements.txt doc/source/conf.py doc/source/index.rst doc/source/contributor/contributing.rst doc/source/install/index.rst doc/source/reference/index.rst doc/source/reference/os_brick/exception.rst doc/source/reference/os_brick/index.rst doc/source/reference/os_brick/initiator/connector.rst doc/source/reference/os_brick/initiator/index.rst doc/source/user/tutorial.rst etc/os-brick/rootwrap.d/os-brick.filters os_brick/__init__.py os_brick/exception.py os_brick/executor.py os_brick/i18n.py os_brick/utils.py os_brick/version.py os_brick.egg-info/PKG-INFO os_brick.egg-info/SOURCES.txt os_brick.egg-info/dependency_links.txt os_brick.egg-info/not-zip-safe os_brick.egg-info/pbr.json os_brick.egg-info/requires.txt os_brick.egg-info/top_level.txt os_brick/encryptors/__init__.py os_brick/encryptors/base.py os_brick/encryptors/cryptsetup.py os_brick/encryptors/luks.py os_brick/encryptors/nop.py os_brick/initiator/__init__.py os_brick/initiator/connector.py os_brick/initiator/host_driver.py os_brick/initiator/initiator_connector.py os_brick/initiator/linuxfc.py os_brick/initiator/linuxrbd.py os_brick/initiator/linuxscsi.py os_brick/initiator/utils.py os_brick/initiator/connectors/__init__.py os_brick/initiator/connectors/aoe.py os_brick/initiator/connectors/base.py os_brick/initiator/connectors/base_iscsi.py os_brick/initiator/connectors/disco.py os_brick/initiator/connectors/drbd.py os_brick/initiator/connectors/fake.py os_brick/initiator/connectors/fibre_channel.py os_brick/initiator/connectors/fibre_channel_ppc64.py os_brick/initiator/connectors/fibre_channel_s390x.py os_brick/initiator/connectors/gpfs.py os_brick/initiator/connectors/hgst.py os_brick/initiator/connectors/huawei.py os_brick/initiator/connectors/iscsi.py os_brick/initiator/connectors/local.py os_brick/initiator/connectors/nvmeof.py os_brick/initiator/connectors/rbd.py os_brick/initiator/connectors/remotefs.py os_brick/initiator/connectors/scaleio.py os_brick/initiator/connectors/storpool.py os_brick/initiator/connectors/vmware.py os_brick/initiator/connectors/vrtshyperscale.py os_brick/initiator/windows/__init__.py os_brick/initiator/windows/base.py os_brick/initiator/windows/fibre_channel.py os_brick/initiator/windows/iscsi.py os_brick/initiator/windows/smbfs.py os_brick/local_dev/__init__.py os_brick/local_dev/lvm.py os_brick/privileged/__init__.py os_brick/privileged/rootwrap.py os_brick/privileged/scaleio.py os_brick/remotefs/__init__.py os_brick/remotefs/remotefs.py os_brick/remotefs/windows_remotefs.py os_brick/tests/__init__.py os_brick/tests/base.py os_brick/tests/test_brick.py os_brick/tests/test_exception.py os_brick/tests/test_executor.py os_brick/tests/test_utils.py os_brick/tests/encryptors/__init__.py os_brick/tests/encryptors/test_base.py os_brick/tests/encryptors/test_cryptsetup.py os_brick/tests/encryptors/test_luks.py os_brick/tests/encryptors/test_nop.py os_brick/tests/initiator/__init__.py os_brick/tests/initiator/test_connector.py os_brick/tests/initiator/test_host_driver.py os_brick/tests/initiator/test_linuxfc.py os_brick/tests/initiator/test_linuxrbd.py os_brick/tests/initiator/test_linuxscsi.py os_brick/tests/initiator/test_utils.py os_brick/tests/initiator/connectors/__init__.py os_brick/tests/initiator/connectors/test_aoe.py os_brick/tests/initiator/connectors/test_base_iscsi.py os_brick/tests/initiator/connectors/test_disco.py os_brick/tests/initiator/connectors/test_drbd.py os_brick/tests/initiator/connectors/test_fibre_channel.py os_brick/tests/initiator/connectors/test_fibre_channel_ppc64.py os_brick/tests/initiator/connectors/test_fibre_channel_s390x.py os_brick/tests/initiator/connectors/test_gpfs.py os_brick/tests/initiator/connectors/test_hgst.py os_brick/tests/initiator/connectors/test_huawei.py os_brick/tests/initiator/connectors/test_iscsi.py os_brick/tests/initiator/connectors/test_iser.py os_brick/tests/initiator/connectors/test_local.py os_brick/tests/initiator/connectors/test_nvmeof.py os_brick/tests/initiator/connectors/test_rbd.py os_brick/tests/initiator/connectors/test_remotefs.py os_brick/tests/initiator/connectors/test_scaleio.py os_brick/tests/initiator/connectors/test_storpool.py os_brick/tests/initiator/connectors/test_vmware.py os_brick/tests/initiator/connectors/test_vrtshyperscale.py os_brick/tests/local_dev/__init__.py os_brick/tests/local_dev/fake_lvm.py os_brick/tests/local_dev/test_brick_lvm.py os_brick/tests/privileged/__init__.py os_brick/tests/privileged/test_rootwrap.py os_brick/tests/remotefs/__init__.py os_brick/tests/remotefs/test_remotefs.py os_brick/tests/remotefs/test_windows_remotefs.py os_brick/tests/windows/__init__.py os_brick/tests/windows/fake_win_conn.py os_brick/tests/windows/test_base.py os_brick/tests/windows/test_base_connector.py os_brick/tests/windows/test_factory.py os_brick/tests/windows/test_fibre_channel.py os_brick/tests/windows/test_iscsi.py os_brick/tests/windows/test_smbfs.py releasenotes/notes/add-luks2-support-13563cfe83aba69c.yaml releasenotes/notes/add-vstorage-protocol-b536f4e21d764801.yaml releasenotes/notes/add-windows-fibre-channel-030c095c149da321.yaml releasenotes/notes/add-windows-iscsi-15d6b1392695f978.yaml releasenotes/notes/add-windows-smbfs-d86edaa003130a31.yaml releasenotes/notes/add_custom_keyring_for_rbd_connection-eccbaae9ee5f3491.yaml releasenotes/notes/bug-1722432-2408dab55c903c5b.yaml releasenotes/notes/bug-1823200-scaleio-upgrade-3e83b5c9dd148714.yaml releasenotes/notes/bug-1823200-ussuri-c76aca2514c75a25.yaml releasenotes/notes/bug-1862443-e87ef38b60f9b979.yaml releasenotes/notes/bug-1865754-ceph-octopus-compatibility-0aa9b8bc1b028301.yaml releasenotes/notes/bug-1884052-798094496dccf23c.yaml releasenotes/notes/bug-1915678-901a6bd24ecede72.yaml releasenotes/notes/bug-1924652-2323f905f62ef8ba.yaml releasenotes/notes/bug-1944474-55c5ebb3a37801aa.yaml releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd.yaml releasenotes/notes/deprecate-plain-cryptsetup-encryptor-0a279abc0b0d718c.yaml releasenotes/notes/disconnect-multipath-cfg-changed-637abc5ecf44fb10.yaml releasenotes/notes/drop-py2-7dcde3ccd0e167b0.yaml releasenotes/notes/external-locks-9f015988ebdc37d6.yaml releasenotes/notes/fc-always-check-single-wwnn-1595689da0eb673b.yaml releasenotes/notes/fc-flush-single-path-22ed6cc7b56a6d9b.yaml releasenotes/notes/fix-fc-scan-too-broad-3c576e1846b7f05f.yaml releasenotes/notes/fix-multipath-disconnect-819d01e6e981883e.yaml releasenotes/notes/improve-get_sysfs_wwn-df38ea88cdcdcc94.yaml releasenotes/notes/improve-iscsi-multipath-detection-f36f28a993f61936.yaml releasenotes/notes/introduce-encryption-provider-constants-a7cd0ce58da2bae8.yaml releasenotes/notes/iscsi_manual_scan_support-d64a1c3c8e1986b4.yaml releasenotes/notes/local-attach-in-rbd-connector-c06347fb164b084a.yaml releasenotes/notes/multipath-improvements-596c2c6eadfba6ea.yaml releasenotes/notes/nvme-rsd-support-d487afd77c534fa1.yaml releasenotes/notes/refactor_iscsi_connect-dfbb24305a954783.yaml releasenotes/notes/refactor_iscsi_disconnect-557f4173bc1ae4ed.yaml releasenotes/notes/remove-old-constants-20021f5b30bde890.yaml releasenotes/notes/remove-sheepdog-611257b28bc88934.yaml releasenotes/notes/scaleio-extend-attached-ec44d3a72395882c.yaml releasenotes/notes/start-using-reno-23e8d5f1a30851a1.yaml releasenotes/notes/ussuri-release-979d709dfa7df068.yaml releasenotes/notes/veritas-hyperscale-connector-fe56cec68b1947cd.yaml releasenotes/notes/vmware-vmdk-connector-19e6999e6cae43cd.yaml releasenotes/notes/wallaby-extra-prelude-d8de88e3e11a7b9f.yaml releasenotes/source/conf.py releasenotes/source/index.rst releasenotes/source/mitaka.rst releasenotes/source/newton.rst releasenotes/source/ocata.rst releasenotes/source/pike.rst releasenotes/source/queens.rst releasenotes/source/rocky.rst releasenotes/source/stein.rst releasenotes/source/train.rst releasenotes/source/unreleased.rst releasenotes/source/_static/.placeholder releasenotes/source/_templates/.placeholder tools/fast8.sh tools/generate_connector_list.py tools/lintstack.py tools/lintstack.sh././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/dependency_links.txt0000664000175000017500000000000100000000000022611 0ustar00zuulzuul00000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/not-zip-safe0000664000175000017500000000000100000000000020771 0ustar00zuulzuul00000000000000 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/pbr.json0000664000175000017500000000005600000000000020222 0ustar00zuulzuul00000000000000{"git_version": "9d3ce01", "is_release": true}././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/requires.txt0000664000175000017500000000050000000000000021136 0ustar00zuulzuul00000000000000Babel!=2.4.0,>=2.3.4 eventlet!=0.18.3,!=0.20.1,!=0.21.0,!=0.23.0,!=0.25.0,>=0.18.2 os-win>=3.0.0 oslo.concurrency>=3.26.0 oslo.context>=2.19.2 oslo.i18n>=3.15.3 oslo.log>=3.36.0 oslo.privsep>=1.32.0 oslo.service!=1.28.1,>=1.24.0 oslo.utils>=3.33.0 pbr!=2.1.0,>=2.0.0 requests>=2.14.2 retrying!=1.3.0,>=1.2.3 six>=1.10.0 ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982235.0 os-brick-3.0.8/os_brick.egg-info/top_level.txt0000664000175000017500000000001100000000000021265 0ustar00zuulzuul00000000000000os_brick ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/pylintrc0000664000175000017500000000220300000000000015042 0ustar00zuulzuul00000000000000# The format of this file isn't really documented; just use --generate-rcfile [Messages Control] # NOTE(justinsb): We might want to have a 2nd strict pylintrc in future # C0111: Don't require docstrings on every method # W0511: TODOs in code comments are fine. # W0142: *args and **kwargs are fine. # W0622: Redefining id is fine. disable=C0111,W0511,W0142,W0622 [Basic] # Variable names can be 1 to 31 characters long, with lowercase and underscores variable-rgx=[a-z_][a-z0-9_]{0,30}$ # Argument names can be 2 to 31 characters long, with lowercase and underscores argument-rgx=[a-z_][a-z0-9_]{1,30}$ # Method names should be at least 3 characters long # and be lowercased with underscores method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|tearDown)$ # Don't require docstrings on tests. no-docstring-rgx=((__.*__)|([tT]est.*)|setUp|tearDown)$ [Design] max-public-methods=100 min-public-methods=0 max-args=6 [Variables] dummy-variables-rgx=_ [Typecheck] # Disable warnings on the HTTPSConnection classes because pylint doesn't # support importing from six.moves yet, see: # https://bitbucket.org/logilab/pylint/issue/550/ ignored-classes=HTTPSConnection ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1406777 os-brick-3.0.8/releasenotes/0000775000175000017500000000000000000000000015747 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/releasenotes/notes/0000775000175000017500000000000000000000000017077 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add-luks2-support-13563cfe83aba69c.yaml0000664000175000017500000000033600000000000025563 0ustar00zuulzuul00000000000000--- features: - | A LUKS2 encryptor has been introduced providing support for this latest version of the Linux Unified Key Setup disk encryption format. This requires ``cryptsetup`` version 2.0.0 or greater. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add-vstorage-protocol-b536f4e21d764801.yaml0000664000175000017500000000011400000000000026253 0ustar00zuulzuul00000000000000--- features: - Added vStorage protocol support for RemoteFS connections. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add-windows-fibre-channel-030c095c149da321.yaml0000664000175000017500000000007700000000000026737 0ustar00zuulzuul00000000000000--- features: - Add Windows Fibre Channel connector support. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add-windows-iscsi-15d6b1392695f978.yaml0000664000175000017500000000006700000000000025336 0ustar00zuulzuul00000000000000--- features: - Add Windows iSCSI connector support. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add-windows-smbfs-d86edaa003130a31.yaml0000664000175000017500000000006700000000000025504 0ustar00zuulzuul00000000000000--- features: - Add Windows SMBFS connector support. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/add_custom_keyring_for_rbd_connection-eccbaae9ee5f3491.yaml0000664000175000017500000000022600000000000032271 0ustar00zuulzuul00000000000000--- fixes: - Add support to use custom Ceph keyring files (previously os-brick hardcoded using /etc/ceph/.client..keyring file). ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1722432-2408dab55c903c5b.yaml0000664000175000017500000000046300000000000023572 0ustar00zuulzuul00000000000000--- fixes: - | [`bug 1722432 `_] Changes the supported_transports to support tcp transport. With this change, we can define an custom iface with tcp transport to limit the storage traffic only be transimitted via storage NIC we specified. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1823200-scaleio-upgrade-3e83b5c9dd148714.yaml0000664000175000017500000000101000000000000026553 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1823200 `_: Prior fixes for this bug changed the connection properties but did not take into account an upgrade scenario in which currently attached volumes had the old format connection properties and could fail on detatch with "KeyError: 'config_group'". This release updates the 'scaleio' connector to handle this situation. It is only applicable to deployments using a Dell EMC PowerFlex/VxFlex OS/ScaleIO backend. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1823200-ussuri-c76aca2514c75a25.yaml0000664000175000017500000000272400000000000025117 0ustar00zuulzuul00000000000000--- security: - | Dell EMC VxFlex OS driver: This release contains a fix for `Bug #1823200 `_. See `OSSN-0086 `_ for details. upgrade: - | The fix for `Bug #1823200 `_ requires that a configuration file be deployed on compute nodes, cinder nodes, and anywhere you would perform a volume attachment in your deployment, when using Cinder with a Dell EMC VxFlex OS backend. See the `Dell EMC VxFlex OS (ScaleIO) Storage driver `_ documentation for details about this configuration file. fixes: - | `Bug #1823200 `_: This release contains an updated connector for use with the Dell EMC VxFlex OS backend. It requires that a configuration file be deployed on compute nodes, cinder nodes, and anywhere you would perform a volume attachment in your deployment. See the `Dell EMC VxFlex OS (ScaleIO) Storage driver `_ documentation for details about the configuration file, and see `OSSN-0086 `_ for more information about the security vulnerability. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1862443-e87ef38b60f9b979.yaml0000664000175000017500000000024700000000000023645 0ustar00zuulzuul00000000000000--- fixes: - | [`bug 1862433 `_] Fix an issue where platform id is needed to determine name of scsi disk. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1865754-ceph-octopus-compatibility-0aa9b8bc1b028301.yaml0000664000175000017500000000115400000000000031050 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1865754 `_: the ``RBDConnector`` class generates a temporary configuration file to connect to Ceph. Previously, os-brick did not include a ``[global]`` section to contain the options it sets, but with the Octopus release (15.2.0+), Ceph has begun enforcing the presence of this section marker, which dates back at least to the Hammer release of Ceph. With this release, os-brick includes the ``[global]`` section in the generated configuration file, which should be backward-compatible at least to Ceph Hammer. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1884052-798094496dccf23c.yaml0000664000175000017500000000020400000000000023542 0ustar00zuulzuul00000000000000--- fixes: - | Fix an incompatibility with ceph 13.2.0 (Mimic) or later, caused by a change in the output of ``rbd map``. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1915678-901a6bd24ecede72.yaml0000664000175000017500000000040400000000000023751 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1915678 `_: Fix unhandled exception during iscsi volume attachment with multipath enabled that resulted in the cinder-volume service becoming stuck and requiring a restart. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1924652-2323f905f62ef8ba.yaml0000664000175000017500000000057400000000000023617 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1924652 `_: Fix issue with newer multipathd implementations where path devices are kept in multipathd even after volume detachment completes, preventing it from creating a multipath device when a new device attachment is made shortly with the same volume device or the same device path. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/bug-1944474-55c5ebb3a37801aa.yaml0000664000175000017500000000033000000000000023653 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1944474 `_: Fixed missing retries to reinitiate iSCSI connections with high concurrency of connections and with multipath enabled. ././@PaxHeader0000000000000000000000000000021100000000000011447 xustar0000000000000000115 path=os-brick-3.0.8/releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd.yaml 22 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/delay-legacy-encryption-provider-name-deprecation-c0d07be3f0d92afd0000664000175000017500000000056200000000000033243 0ustar00zuulzuul00000000000000--- deprecations: - | The direct use of the encryption provider classes such as os_brick.encryptors.luks.LuksEncryptor continues to be deprecated and will now be blocked in the Queens release of os-brick. The use of out of tree encryption provider classes also continues to be deprecated and will also be blocked in the Queens release of os-brick. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/deprecate-plain-cryptsetup-encryptor-0a279abc0b0d718c.yaml0000664000175000017500000000044000000000000031540 0ustar00zuulzuul00000000000000--- deprecations: - | The plain CryptsetupEncryptor is deprecated and will be removed in a future release. Existing users are encouraged to retype any existing volumes using this encryptor to the luks LuksEncryptor or luks2 Luks2Encryptor encryptors as soon as possible ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/disconnect-multipath-cfg-changed-637abc5ecf44fb10.yaml0000664000175000017500000000033500000000000030615 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1921381 `_: Fix disconnecting volumes when the use_multipath value is changed from the connect_volume call to the disconnect_volume call. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/drop-py2-7dcde3ccd0e167b0.yaml0000664000175000017500000000025600000000000024070 0ustar00zuulzuul00000000000000--- upgrade: - | Python 2.7 support has been dropped. Beginning with os-brick release 3.0.0, the minimum version of Python supported by os-brick is Python 3.6. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/external-locks-9f015988ebdc37d6.yaml0000664000175000017500000000072300000000000025155 0ustar00zuulzuul00000000000000--- upgrade: - | Services using os-brick need to set the ``lock_path`` configuration option in their ``[oslo_concurrency]`` section since it doesn't have a valid default (related `bug #1947370 `_). fixes: - | `Bug #1947370 `_: Fixed race conditions on iSCSI with shared targets and NVMe ``connect_volume`` and ``disconnect_volume`` calls. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/fc-always-check-single-wwnn-1595689da0eb673b.yaml0000664000175000017500000000075300000000000027345 0ustar00zuulzuul00000000000000--- fixes: - | Always check if we are dealing with a single WWNN Fibre Channel target, even when we receive an initiator_target_map. This allows us to exclude unconnected HBAs from our scan for storage arrays that automatically connect all target ports (due to their architecture and design) even if the Cinder driver returns the initiator_target_map, provided the target has a single WWNN. Excluding these HBAs prevents undesired volumes from being connected. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/fc-flush-single-path-22ed6cc7b56a6d9b.yaml0000664000175000017500000000034200000000000026262 0ustar00zuulzuul00000000000000--- fixes: - | `Bug #1897787 `_: Fix Fibre Channel not flushing volumes on detach when a multipath connection was requested on their attach, but one was not found. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/fix-fc-scan-too-broad-3c576e1846b7f05f.yaml0000664000175000017500000000034500000000000026116 0ustar00zuulzuul00000000000000--- fixes: - | Fix an issue where SCSI LUN scans for FC were unnecessarily too broad. Now OS-Brick will not use wildcards unless it doesn't find any target ports in sysfs and the Cinder driver doesn't disable them. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/fix-multipath-disconnect-819d01e6e981883e.yaml0000664000175000017500000000060500000000000027004 0ustar00zuulzuul00000000000000--- fixes: - | Under certain conditions detaching a multipath device may result in failure when flushing one of the individual paths, but the disconnect should have succeeded, because there were other paths available to flush all the data. The multipath disconnect mechanism is now more robust and will only fail when disconnecting if multipath would lose data. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/improve-get_sysfs_wwn-df38ea88cdcdcc94.yaml0000664000175000017500000000014400000000000027073 0ustar00zuulzuul00000000000000--- fixes: - | Improve WWN detection for arrays with multiple designators. (bug 1881608). ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/improve-iscsi-multipath-detection-f36f28a993f61936.yaml0000664000175000017500000000021000000000000030627 0ustar00zuulzuul00000000000000--- fixes: - | Improve iSCSI multipath detection to work even if we cannot find the volume's WWN in sysfs. (bug 1881619). ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/introduce-encryption-provider-constants-a7cd0ce58da2bae8.yaml0000664000175000017500000000121400000000000032517 0ustar00zuulzuul00000000000000--- features: - | Encryption provider constants have been introduced detailing the supported encryption formats such as LUKs along with their associated in-tree provider implementations. These constants should now be used to identify an encryption provider implementation for a given encryption format. deprecations: - | The direct use of the encryption provider classes such as os_brick.encryptors.luks.LuksEncryptor is now deprecated and will be blocked in the Pike release of os-brick. The use of out of tree encryption provider classes is also deprecated and will be blocked in the Pike release of os-brick. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/iscsi_manual_scan_support-d64a1c3c8e1986b4.yaml0000664000175000017500000000062400000000000027455 0ustar00zuulzuul00000000000000--- features: - | Support for setting the scan mode on the Open-iSCSI initiator. If installed iscsiadm supports this feature OS-Brick will set all it's new sessions to manual scan. fixes: - | On systems with scan mode support on open-iSCSI we'll no longer see unwanted devices polluting our system due to the automatic initiator scan or to AEN/AER messages from the backend. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/local-attach-in-rbd-connector-c06347fb164b084a.yaml0000664000175000017500000000021500000000000027604 0ustar00zuulzuul00000000000000--- features: - Local attach feature in RBD connector. We use RBD kernel module to attach and detach volumes locally without Nova. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/multipath-improvements-596c2c6eadfba6ea.yaml0000664000175000017500000000006300000000000027235 0ustar00zuulzuul00000000000000--- fixes: - Improved multipath device handling. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/nvme-rsd-support-d487afd77c534fa1.yaml0000664000175000017500000000011500000000000025531 0ustar00zuulzuul00000000000000--- features: - | Extended nvme connector to support RSD with NVMe-oF. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/refactor_iscsi_connect-dfbb24305a954783.yaml0000664000175000017500000000014700000000000026633 0ustar00zuulzuul00000000000000--- fixes: - | iSCSI connect mechanism refactoring to be faster, more robust, more reliable. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/refactor_iscsi_disconnect-557f4173bc1ae4ed.yaml0000664000175000017500000000077300000000000027476 0ustar00zuulzuul00000000000000--- features: - | New parameters on `disconnect_volume` named `force` and `ignore_errors` can be used to let OS-Brick know that data loss is secondary to leaving a clean system with no leftover devices. If `force` is not set, or set to False, preventing data loss will take priority. Currently only iSCSI implements these new parameters. fixes: - | iSCSI disconnect refactoring improves reliability, speed, and thoroughness on leaving a cleaner system after disconnection. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/remove-old-constants-20021f5b30bde890.yaml0000664000175000017500000000047400000000000026171 0ustar00zuulzuul00000000000000--- upgrade: - | The location for connector constants was moved in the 1.6.0 release, but their old location was kept for backwards compatibility. These legacy constants are now being removed and any out of tree code should be updated to use the latest location (os_brick.initiator.CONSTANT_NAME). ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/remove-sheepdog-611257b28bc88934.yaml0000664000175000017500000000031500000000000025057 0ustar00zuulzuul00000000000000--- upgrade: - | The Sheepdog project is no longer active and its driver has been removed from Cinder. The connector and Sheepdog related handling has now been removed from os-brick as well. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/scaleio-extend-attached-ec44d3a72395882c.yaml0000664000175000017500000000010300000000000026600 0ustar00zuulzuul00000000000000--- features: - Added ability to extend attached ScaleIO volumes ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/start-using-reno-23e8d5f1a30851a1.yaml0000664000175000017500000000007100000000000025325 0ustar00zuulzuul00000000000000--- other: - Start using reno to manage release notes. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/ussuri-release-979d709dfa7df068.yaml0000664000175000017500000000026100000000000025176 0ustar00zuulzuul00000000000000--- other: - | This release contains some minor driver fixes. - | Please keep in mind that the minum version of Python supported by this release is Python 3.6. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/veritas-hyperscale-connector-fe56cec68b1947cd.yaml0000664000175000017500000000007300000000000030157 0ustar00zuulzuul00000000000000--- features: - Add Veritas HyperScale connector support ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/vmware-vmdk-connector-19e6999e6cae43cd.yaml0000664000175000017500000000021000000000000026523 0ustar00zuulzuul00000000000000--- features: - Added initiator connector 'VmdkConnector' to support backup and restore of vmdk volumes by Cinder backup service. ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/notes/wallaby-extra-prelude-d8de88e3e11a7b9f.yaml0000664000175000017500000000030600000000000026566 0ustar00zuulzuul00000000000000--- prelude: > This release fixes an issue that could cause data loss when the configuration enabling/disabling multipathing is changed on a compute when volumes are currently attached. ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/releasenotes/source/0000775000175000017500000000000000000000000017247 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/releasenotes/source/_static/0000775000175000017500000000000000000000000020675 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/_static/.placeholder0000664000175000017500000000000000000000000023146 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/releasenotes/source/_templates/0000775000175000017500000000000000000000000021404 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/_templates/.placeholder0000664000175000017500000000000000000000000023655 0ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/conf.py0000664000175000017500000000324100000000000020546 0ustar00zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # # os-brick Release Notes documentation build configuration file # # Refer to the Sphinx documentation for advice on configuring this file: # # http://www.sphinx-doc.org/en/stable/config.html # -- General configuration ------------------------------------------------ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'reno.sphinxext', 'openstackdocstheme', ] # The suffix of source filenames. source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. copyright = u'2015, Cinder Developers' # Release notes are unversioned, so we don't need to set version and release version = '' release = '' # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'openstackdocs' # -- Options for openstackdocstheme ------------------------------------------- repository_name = 'openstack/os-brick' bug_project = 'os-brick' bug_tag = '' ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/index.rst0000664000175000017500000000030100000000000021102 0ustar00zuulzuul00000000000000======================== os-brick Release Notes ======================== .. toctree:: :maxdepth: 1 unreleased train stein rocky queens pike ocata newton mitaka ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/mitaka.rst0000664000175000017500000000021100000000000021241 0ustar00zuulzuul00000000000000=========================== Mitaka Series Release Notes =========================== .. release-notes:: :branch: origin/stable/mitaka ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/newton.rst0000664000175000017500000000021600000000000021312 0ustar00zuulzuul00000000000000============================= Newton Series Release Notes ============================= .. release-notes:: :branch: origin/stable/newton ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/ocata.rst0000664000175000017500000000021400000000000021065 0ustar00zuulzuul00000000000000============================= Ocata Series Release Notes ============================= .. release-notes:: :branch: origin/stable/ocata ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/pike.rst0000664000175000017500000000021700000000000020731 0ustar00zuulzuul00000000000000=================================== Pike Series Release Notes =================================== .. release-notes:: :branch: stable/pike ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/queens.rst0000664000175000017500000000020200000000000021273 0ustar00zuulzuul00000000000000=========================== Queens Series Release Notes =========================== .. release-notes:: :branch: stable/queens ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/rocky.rst0000664000175000017500000000022100000000000021123 0ustar00zuulzuul00000000000000=================================== Rocky Series Release Notes =================================== .. release-notes:: :branch: stable/rocky ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/stein.rst0000664000175000017500000000017600000000000021127 0ustar00zuulzuul00000000000000========================== Stein Series Release Notes ========================== .. release-notes:: :branch: stable/stein ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/train.rst0000664000175000017500000000020300000000000021111 0ustar00zuulzuul00000000000000============================ Train Series Release Notes ============================ .. release-notes:: :branch: stable/train ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/releasenotes/source/unreleased.rst0000664000175000017500000000016000000000000022125 0ustar00zuulzuul00000000000000============================== Current Series Release Notes ============================== .. release-notes:: ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/requirements.txt0000664000175000017500000000126400000000000016545 0ustar00zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. pbr!=2.1.0,>=2.0.0 # Apache-2.0 Babel!=2.4.0,>=2.3.4 # BSD eventlet!=0.18.3,!=0.20.1,!=0.21.0,!=0.23.0,!=0.25.0,>=0.18.2 # MIT oslo.concurrency>=3.26.0 # Apache-2.0 oslo.context>=2.19.2 # Apache-2.0 oslo.log>=3.36.0 # Apache-2.0 oslo.i18n>=3.15.3 # Apache-2.0 oslo.privsep>=1.32.0 # Apache-2.0 oslo.service!=1.28.1,>=1.24.0 # Apache-2.0 oslo.utils>=3.33.0 # Apache-2.0 requests>=2.14.2 # Apache-2.0 retrying!=1.3.0,>=1.2.3 # Apache-2.0 six>=1.10.0 # MIT os-win>=3.0.0 # Apache-2.0 ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/setup.cfg0000664000175000017500000000232200000000000015076 0ustar00zuulzuul00000000000000[metadata] name = os-brick summary = OpenStack Cinder brick library for managing local volume attaches description-file = README.rst author = OpenStack author-email = openstack-discuss@lists.openstack.org home-page = https://docs.openstack.org/os-brick/ classifier = Environment :: OpenStack Intended Audience :: Information Technology Intended Audience :: System Administrators License :: OSI Approved :: Apache Software License Operating System :: POSIX :: Linux Programming Language :: Python Programming Language :: Python :: 3 Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 [global] setup-hooks = pbr.hooks.setup_hook [files] packages = os_brick data_files = etc/ = etc/* [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 [build_sphinx] warning-is-error = 1 source-dir = doc/source build-dir = doc/build all_files = 1 [upload_sphinx] upload-dir = doc/build/html [compile_catalog] directory = os_brick/locale domain = os_brick [update_catalog] domain = os_brick output_dir = os_brick/locale input_file = os_brick/locale/os-brick.pot [extract_messages] keywords = _ gettext ngettext l_ lazy_gettext mapping_file = babel.cfg output_file = os_brick/locale/os-brick.pot ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/setup.py0000664000175000017500000000200600000000000014766 0ustar00zuulzuul00000000000000# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT import setuptools # In python < 2.7.4, a lazy loading of package `pbr` will break # setuptools if some other modules registered functions in `atexit`. # solution from: http://bugs.python.org/issue15881#msg170215 try: import multiprocessing # noqa except ImportError: pass setuptools.setup( setup_requires=['pbr>=2.0.0'], pbr=True) ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/test-requirements.txt0000664000175000017500000000111000000000000017510 0ustar00zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. hacking>=3.0.1,<3.1.0 # Apache-2.0 coverage!=4.4,>=4.0 # Apache-2.0 ddt>=1.0.1 # MIT reno>=2.5.0 # Apache-2.0 sphinx!=1.6.6,!=1.6.7,!=2.1.0,>=1.6.2 # BSD openstackdocstheme>=1.20.0 # Apache-2.0 oslotest>=3.2.0 # Apache-2.0 testscenarios>=0.4 # Apache-2.0/BSD testtools>=2.2.0 # MIT stestr>=1.0.0 # Apache-2.0 oslo.vmware>=2.17.0 # Apache-2.0 castellan>=0.16.0 # Apache-2.0 ././@PaxHeader0000000000000000000000000000003400000000000011452 xustar000000000000000028 mtime=1636982235.1606777 os-brick-3.0.8/tools/0000775000175000017500000000000000000000000014416 5ustar00zuulzuul00000000000000././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/tools/fast8.sh0000775000175000017500000000045300000000000016004 0ustar00zuulzuul00000000000000#!/bin/bash cd $(dirname "$0")/.. CHANGED=$(git diff --name-only HEAD~1 | tr '\n' ' ') # Skip files that don't exist # (have been git rm'd) CHECK="" for FILE in $CHANGED; do if [ -f "$FILE" ]; then CHECK="$CHECK $FILE" fi done diff -u --from-file /dev/null $CHECK | flake8 --diff ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/tools/generate_connector_list.py0000775000175000017500000001405400000000000021676 0ustar00zuulzuul00000000000000#! /usr/bin/env python # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Generate list of os-brick connectors""" import argparse import inspect import json import operator import os from pydoc import locate import textwrap from os_brick.initiator import connector parser = argparse.ArgumentParser(prog="generate_connector_list") parser.add_argument("--format", default='str', choices=['str', 'dict'], help="Output format type") # Keep backwards compatibilty with the gate-docs test # The tests pass ['docs'] on the cmdln, but it's never been used. parser.add_argument("output_list", default=None, nargs='?') def _ensure_loaded(connector_list): """Loads everything in a given path. This will make sure all classes have been loaded and therefore all decorators have registered class. :param start_path: The starting path to load. """ classes = [] for conn in connector_list: try: conn_class = locate(conn) classes.append(conn_class) except Exception: pass return classes def get_connectors(): """Get a list of all connectors.""" classes = _ensure_loaded(connector._get_connector_list()) return [DriverInfo(x) for x in classes] class DriverInfo(object): """Information about Connector implementations.""" def __init__(self, cls): self.cls = cls self.desc = cls.__doc__ self.class_name = cls.__name__ self.class_fqn = '{}.{}'.format(inspect.getmodule(cls).__name__, self.class_name) self.platform = getattr(cls, 'platform', None) self.os_type = getattr(cls, 'os_type', None) def __str__(self): return self.class_name def __repr__(self): return self.class_fqn def __hash__(self): return hash(self.class_fqn) class Output(object): def __init__(self, base_dir, output_list): # At this point we don't care what was passed in, just a trigger # to write this out to the doc tree for now self.connector_file = None if output_list: self.connector_file = open( '%s/doc/source/connectors.rst' % base_dir, 'w+') self.connector_file.write('===================\n') self.connector_file.write('Available Connectors\n') self.connector_file.write('===================\n\n') def __enter__(self): return self def __exit__(self, type, value, traceback): if self.connector_file: self.connector_file.close() def write(self, text): if self.connector_file: self.connector_file.write('%s\n' % text) else: print(text) def format_description(desc, output): desc = desc or '' lines = desc.rstrip('\n').split('\n') output.write('* Description: %s' % lines[0]) output.write('') output.write(textwrap.dedent('\n'.join(lines[1:]))) def format_options(connector_options, output): if connector_options and len(connector_options) > 0: output.write('* Driver Configuration Options:') output.write('') output.write('.. list-table:: **Driver configuration options**') output.write(' :header-rows: 1') output.write(' :widths: 14 30') output.write('') output.write(' * - Name = Default Value') output.write(' - (Type) Description') sorted_options = sorted(connector_options, key=operator.attrgetter('name')) for opt in sorted_options: output.write(' * - %s = %s' % (opt.name, opt.default)) output.write(' - (%s) %s' % (opt.type, opt.help)) output.write('') def print_connectors(connectors, config_name, output, section_char='-'): for conn in sorted(connectors, key=lambda x: x.class_name): conn_name = conn.class_name output.write(conn_name) output.write(section_char * len(conn_name)) if conn.platform: output.write('* Platform: %s' % conn.platform) if conn.os_type: output.write('* OS Type: %s' % conn.os_type) output.write('* %s=%s' % (config_name, conn.class_fqn)) format_description(conn.desc, output) output.write('') output.write('') def output_str(cinder_root, args): with Output(cinder_root, args.output_list) as output: output.write('Connectors') output.write('==============') connectors = get_connectors() print_connectors(connectors, 'connector', output, '~') def collect_connector_info(connector): """Build the dictionary that describes this connector.""" info = {'name': connector.class_name, 'fqn': connector.class_fqn, 'description': connector.desc, 'platform': connector.platform, 'os_type': connector.os_type, } return info def output_dict(): """Output the results as a JSON dict.""" connector_list = [] connectors = get_connectors() for conn in connectors: connector_list.append(collect_connector_info(conn)) print(json.dumps(connector_list)) def main(): tools_dir = os.path.dirname(os.path.abspath(__file__)) brick_root = os.path.dirname(tools_dir) cur_dir = os.getcwd() os.chdir(brick_root) args = parser.parse_args() try: if args.format == 'str': output_str(brick_root, args) elif args.format == 'dict': output_dict() finally: os.chdir(cur_dir) if __name__ == '__main__': main() ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/tools/lintstack.py0000775000175000017500000001530400000000000016772 0ustar00zuulzuul00000000000000#!/usr/bin/env python # Copyright (c) 2013, AT&T Labs, Yun Mao # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """pylint error checking.""" from __future__ import print_function import json import re import sys from pylint import lint from pylint.reporters import text from six.moves import cStringIO as StringIO ignore_codes = [ # Note(maoy): E1103 is error code related to partial type inference "E1103" ] ignore_messages = [ # Note(fengqian): this message is the pattern of [E0611]. # It should be ignored because use six module to keep py3.X compatibility. "No name 'urllib' in module '_MovedItems'", # Note(xyang): these error messages are for the code [E1101]. # They should be ignored because 'sha256' and 'sha224' are functions in # 'hashlib'. "Module 'hashlib' has no 'sha256' member", "Module 'hashlib' has no 'sha224' member", ] ignore_modules = ["os_brick/tests/", "tools/lintstack.head.py"] KNOWN_PYLINT_EXCEPTIONS_FILE = "tools/pylint_exceptions" class LintOutput(object): _cached_filename = None _cached_content = None def __init__(self, filename, lineno, line_content, code, message, lintoutput): self.filename = filename self.lineno = lineno self.line_content = line_content self.code = code self.message = message self.lintoutput = lintoutput @classmethod def from_line(cls, line): m = re.search(r"(\S+):(\d+): \[(\S+)(, \S+)?] (.*)", line) matched = m.groups() filename, lineno, code, message = (matched[0], int(matched[1]), matched[2], matched[-1]) if cls._cached_filename != filename: with open(filename) as f: cls._cached_content = list(f.readlines()) cls._cached_filename = filename line_content = cls._cached_content[lineno - 1].rstrip() return cls(filename, lineno, line_content, code, message, line.rstrip()) @classmethod def from_msg_to_dict(cls, msg): """Converts pytlint message to a unique-error dictionary. From the output of pylint msg, to a dict, where each key is a unique error identifier, value is a list of LintOutput """ result = {} for line in msg.splitlines(): obj = cls.from_line(line) if obj.is_ignored(): continue key = obj.key() if key not in result: result[key] = [] result[key].append(obj) return result def is_ignored(self): if self.code in ignore_codes: return True if any(self.filename.startswith(name) for name in ignore_modules): return True return False def key(self): if self.code in ["E1101", "E1103"]: # These two types of errors are like Foo class has no member bar. # We discard the source code so that the error will be ignored # next time another Foo.bar is encountered. return self.message, "" return self.message, self.line_content.strip() def json(self): return json.dumps(self.__dict__) def review_str(self): return ("File %(filename)s\nLine %(lineno)d:%(line_content)s\n" "%(code)s: %(message)s" % {'filename': self.filename, 'lineno': self.lineno, 'line_content': self.line_content, 'code': self.code, 'message': self.message}) class ErrorKeys(object): @classmethod def print_json(cls, errors, output=sys.stdout): print("# automatically generated by tools/lintstack.py", file=output) for i in sorted(errors.keys()): print(json.dumps(i), file=output) @classmethod def from_file(cls, filename): keys = set() for line in open(filename): if line and line[0] != "#": d = json.loads(line) keys.add(tuple(d)) return keys def run_pylint(): buff = StringIO() reporter = text.ParseableTextReporter(output=buff) args = ["--include-ids=y", "-E", "os_brick"] lint.Run(args, reporter=reporter, exit=False) val = buff.getvalue() buff.close() return val def generate_error_keys(msg=None): print("Generating", KNOWN_PYLINT_EXCEPTIONS_FILE) if msg is None: msg = run_pylint() errors = LintOutput.from_msg_to_dict(msg) with open(KNOWN_PYLINT_EXCEPTIONS_FILE, "w") as f: ErrorKeys.print_json(errors, output=f) def validate(newmsg=None): print("Loading", KNOWN_PYLINT_EXCEPTIONS_FILE) known = ErrorKeys.from_file(KNOWN_PYLINT_EXCEPTIONS_FILE) if newmsg is None: print("Running pylint. Be patient...") newmsg = run_pylint() errors = LintOutput.from_msg_to_dict(newmsg) print("Unique errors reported by pylint: was %d, now %d." % (len(known), len(errors))) passed = True for err_key, err_list in errors.items(): for err in err_list: if err_key not in known: print(err.lintoutput) print() passed = False if passed: print("Congrats! pylint check passed.") redundant = known - set(errors.keys()) if redundant: print("Extra credit: some known pylint exceptions disappeared.") for i in sorted(redundant): print(json.dumps(i)) print("Consider regenerating the exception file if you will.") else: print("Please fix the errors above. If you believe they are false " "positives, run 'tools/lintstack.py generate' to overwrite.") sys.exit(1) def usage(): print("""Usage: tools/lintstack.py [generate|validate] To generate pylint_exceptions file: tools/lintstack.py generate To validate the current commit: tools/lintstack.py """) def main(): option = "validate" if len(sys.argv) > 1: option = sys.argv[1] if option == "generate": generate_error_keys() elif option == "validate": validate() else: usage() if __name__ == "__main__": main() ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/tools/lintstack.sh0000775000175000017500000000420600000000000016753 0ustar00zuulzuul00000000000000#!/usr/bin/env bash # Copyright (c) 2012-2013, AT&T Labs, Yun Mao # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # Use lintstack.py to compare pylint errors. # We run pylint twice, once on HEAD, once on the code before the latest # commit for review. set -e TOOLS_DIR=$(cd $(dirname "$0") && pwd) # Get the current branch name. GITHEAD=`git rev-parse --abbrev-ref HEAD` if [[ "$GITHEAD" == "HEAD" ]]; then # In detached head mode, get revision number instead GITHEAD=`git rev-parse HEAD` echo "Currently we are at commit $GITHEAD" else echo "Currently we are at branch $GITHEAD" fi cp -f $TOOLS_DIR/lintstack.py $TOOLS_DIR/lintstack.head.py if git rev-parse HEAD^2 2>/dev/null; then # The HEAD is a Merge commit. Here, the patch to review is # HEAD^2, the master branch is at HEAD^1, and the patch was # written based on HEAD^2~1. PREV_COMMIT=`git rev-parse HEAD^2~1` git checkout HEAD~1 # The git merge is necessary for reviews with a series of patches. # If not, this is a no-op so won't hurt either. git merge $PREV_COMMIT else # The HEAD is not a merge commit. This won't happen on gerrit. # Most likely you are running against your own patch locally. # We assume the patch to examine is HEAD, and we compare it against # HEAD~1 git checkout HEAD~1 fi # First generate tools/pylint_exceptions from HEAD~1 $TOOLS_DIR/lintstack.head.py generate # Then use that as a reference to compare against HEAD git checkout $GITHEAD $TOOLS_DIR/lintstack.head.py echo "Check passed. FYI: the pylint exceptions are:" cat $TOOLS_DIR/pylint_exceptions ././@PaxHeader0000000000000000000000000000002600000000000011453 xustar000000000000000022 mtime=1636982198.0 os-brick-3.0.8/tox.ini0000664000175000017500000000747100000000000014602 0ustar00zuulzuul00000000000000[tox] minversion = 3.1.0 skipsdist = True skip_missing_interpreters = true # python runtimes: https://governance.openstack.org/tc/reference/runtimes/ussuri.html envlist = py37,py36,pep8 # this allows tox to infer the base python from the environment name # and override any basepython configured in this file ignore_basepython_conflict=true [testenv] basepython = python3 usedevelop = True setenv = VIRTUAL_ENV={envdir} OS_TEST_PATH=./os_brick/tests OS_TEST_TIMEOUT=60 OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 install_command = pip install {opts} {packages} deps = -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/ussuri} -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt # By default stestr will set concurrency # to ncpu, to specify something else use # the concurrency= option. # call example: 'tox -epy37 -- --concurrency=4' commands = stestr run {posargs} stestr slowest whitelist_externals = bash find passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY [testenv:debug] commands = find . -type f -name "*.pyc" -delete oslo_debug_helper {posargs} [testenv:pep8] commands = flake8 {posargs} [testenv:fast8] envdir = {toxworkdir}/pep8 commands = {toxinidir}/tools/fast8.sh [testenv:pylint] deps = -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/ussuri} -r{toxinidir}/requirements.txt pylint==0.26.0 commands = bash tools/lintstack.sh [testenv:venv] commands = {posargs} [testenv:cover] # To see the report of missing coverage add to commands # coverage report --show-missing setenv = {[testenv]setenv} PYTHON=coverage run --source os_brick --parallel-mode commands = stestr run {posargs} coverage combine coverage html -d cover coverage xml -o cover/coverage/xml [testenv:docs] deps = -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/ussuri} -r{toxinidir}/doc/requirements.txt commands = rm -fr doc/build doc/source/contributor/api/ .autogenerated sphinx-build -W -b html -d doc/build/doctrees doc/source doc/build/html whitelist_externals = rm [testenv:pdf-docs] deps = {[testenv:docs]deps} commands = {[testenv:docs]commands} sphinx-build -W -b latex doc/source doc/build/pdf make -C doc/build/pdf whitelist_externals = make [testenv:releasenotes] commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html [flake8] # Following checks are ignored on purpose. # # E251 unexpected spaces around keyword / parameter equals # reason: no improvement in readability # W503 line break before binary operator # reason: pep8 itself is not sure about this one and # reversed this rule in 2016 # W504 line break after binary operator # reason: no agreement on this being universally # preferable for our code. Disabled to keep checking # tools from getting in our way with regards to this. # show-source = True ignore = E251,W503,W504 enable-extensions=H106,H203,H204,H205 builtins = _ exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build max-complexity=30 [hacking] import_exceptions = os_brick.i18n [testenv:bindep] # Do not install any requirements. We want this to be fast and work even if # system dependencies are missing, since it's used to tell you what system # dependencies are missing! This also means that bindep must be installed # separately, outside of the requirements files, and develop mode disabled # explicitly to avoid unnecessarily installing the checked-out repo too (this # further relies on "tox.skipsdist = True" above). deps = bindep commands = bindep {posargs} usedevelop = False [testenv:lower-constraints] deps = -c{toxinidir}/lower-constraints.txt -r{toxinidir}/test-requirements.txt -r{toxinidir}/requirements.txt