python-oslo.vmware-0.2/0000775000175000017500000000000012303705362014125 5ustar chuckchuckpython-oslo.vmware-0.2/openstack-common.conf0000664000175000017500000000023612303705326020252 0ustar chuckchuck[DEFAULT] # The list of modules to copy from oslo-incubator.git module=gettextutils # The base module to hold the copy of openstack.common base=oslo.vmware python-oslo.vmware-0.2/HACKING.rst0000664000175000017500000000022712303705325015723 0ustar chuckchuck Style Commandments =============================================== Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/python-oslo.vmware-0.2/LICENSE0000664000175000017500000002363612303705325015143 0ustar chuckchuck Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. python-oslo.vmware-0.2/requirements.txt0000664000175000017500000000033012303705326017405 0ustar chuckchuckstevedore>=0.12 netaddr>=0.7.6 # for timeutils iso8601>=0.1.8 # for jsonutils six>=1.4.1 # used by openstack/common/gettextutils.py Babel>=1.3 # for the routing notifier PyYAML>=3.1.0 suds>=0.4 eventlet>=0.13.0 python-oslo.vmware-0.2/README.rst0000664000175000017500000000037412303705325015617 0ustar chuckchuck=================================== oslo.vmware =================================== Oslo VMware library for OpenStack projects * Free software: Apache license * Documentation: http://docs.openstack.org/developer/oslo.vmware Features -------- * TODOpython-oslo.vmware-0.2/.testr.conf0000664000175000017500000000047612303705325016221 0ustar chuckchuck[DEFAULT] test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \ ${PYTHON:-python} -m subunit.run discover -t ./ . $LISTOPT $IDOPTION test_id_option=--load-list $IDFILE test_list_option=--listpython-oslo.vmware-0.2/oslo.vmware.egg-info/0000775000175000017500000000000012303705362020073 5ustar chuckchuckpython-oslo.vmware-0.2/oslo.vmware.egg-info/top_level.txt0000664000175000017500000000000512303705362022620 0ustar chuckchuckoslo python-oslo.vmware-0.2/oslo.vmware.egg-info/not-zip-safe0000664000175000017500000000000112303705327022322 0ustar chuckchuck python-oslo.vmware-0.2/oslo.vmware.egg-info/requires.txt0000664000175000017500000000015412303705362022473 0ustar chuckchuckstevedore>=0.12 netaddr>=0.7.6 iso8601>=0.1.8 six>=1.4.1 Babel>=1.3 PyYAML>=3.1.0 suds>=0.4 eventlet>=0.13.0python-oslo.vmware-0.2/oslo.vmware.egg-info/SOURCES.txt0000664000175000017500000000242512303705362021762 0ustar chuckchuck.coveragerc .mailmap .testr.conf AUTHORS CONTRIBUTING.rst ChangeLog HACKING.rst LICENSE MANIFEST.in README.rst babel.cfg openstack-common.conf requirements.txt setup.cfg setup.py test-requirements.txt tox.ini doc/source/conf.py doc/source/contributing.rst doc/source/index.rst doc/source/installation.rst doc/source/readme.rst doc/source/usage.rst oslo/__init__.py oslo.vmware.egg-info/PKG-INFO oslo.vmware.egg-info/SOURCES.txt oslo.vmware.egg-info/dependency_links.txt oslo.vmware.egg-info/namespace_packages.txt oslo.vmware.egg-info/not-zip-safe oslo.vmware.egg-info/requires.txt oslo.vmware.egg-info/top_level.txt oslo/vmware/__init__.py oslo/vmware/api.py oslo/vmware/exceptions.py oslo/vmware/image_transfer.py oslo/vmware/rw_handles.py oslo/vmware/vim.py oslo/vmware/vim_util.py oslo/vmware/common/__init__.py oslo/vmware/common/loopingcall.py oslo/vmware/openstack/__init__.py oslo/vmware/openstack/common/__init__.py oslo/vmware/openstack/common/gettextutils.py oslo/vmware/openstack/common/importutils.py oslo/vmware/openstack/common/jsonutils.py oslo/vmware/openstack/common/local.py oslo/vmware/openstack/common/timeutils.py tests/__init__.py tests/base.py tests/test_api.py tests/test_image_transfer.py tests/test_rw_handles.py tests/test_vim.py tests/test_vim_util.py tests/test_vmware.pypython-oslo.vmware-0.2/oslo.vmware.egg-info/PKG-INFO0000664000175000017500000000222212303705362021166 0ustar chuckchuckMetadata-Version: 1.1 Name: oslo.vmware Version: 0.2 Summary: Oslo VMware library for OpenStack projects Home-page: http://launchpad.net/oslo Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description: =================================== oslo.vmware =================================== Oslo VMware library for OpenStack projects * Free software: Apache license * Documentation: http://docs.openstack.org/developer/oslo.vmware Features -------- * TODO Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 python-oslo.vmware-0.2/oslo.vmware.egg-info/dependency_links.txt0000664000175000017500000000000112303705362024141 0ustar chuckchuck python-oslo.vmware-0.2/oslo.vmware.egg-info/namespace_packages.txt0000664000175000017500000000000512303705362024421 0ustar chuckchuckoslo python-oslo.vmware-0.2/doc/0000775000175000017500000000000012303705362014672 5ustar chuckchuckpython-oslo.vmware-0.2/doc/source/0000775000175000017500000000000012303705362016172 5ustar chuckchuckpython-oslo.vmware-0.2/doc/source/installation.rst0000664000175000017500000000025012303705325021421 0ustar chuckchuck============ Installation ============ At the command line:: $ pip install Or, if you have virtualenvwrapper installed:: $ mkvirtualenv $ pip install python-oslo.vmware-0.2/doc/source/conf.py0000775000175000017500000000463212303705325017500 0ustar chuckchuck# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. import os import sys sys.path.insert(0, os.path.abspath('../..')) # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', #'sphinx.ext.intersphinx', 'oslosphinx' ] # autodoc generation is a bit aggressive and a nuisance when doing heavy # text edit cycles. # execute "export SPHINX_DEBUG=1" in your terminal to disable # The suffix of source filenames. source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = u'oslo.vmware' copyright = u'2014, OpenStack Foundation' # If true, '()' will be appended to :func: etc. cross-reference text. add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). add_module_names = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. # html_theme_path = ["."] # html_theme = '_theme' # html_static_path = ['static'] # Output file base name for HTML help builder. htmlhelp_basename = '%sdoc' % project # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', '%s.tex' % project, u'%s Documentation' % project, u'OpenStack Foundation', 'manual'), ] # Example configuration for intersphinx: refer to the Python standard library. #intersphinx_mapping = {'http://docs.python.org/': None}python-oslo.vmware-0.2/doc/source/usage.rst0000664000175000017500000000007712303705325020033 0ustar chuckchuck======== Usage ======== To use in a project:: import vmwarepython-oslo.vmware-0.2/doc/source/contributing.rst0000664000175000017500000000004312303705325021427 0ustar chuckchuck.. include:: ../../CONTRIBUTING.rstpython-oslo.vmware-0.2/doc/source/readme.rst0000664000175000017500000000003212303705325020153 0ustar chuckchuck.. include:: ../README.rstpython-oslo.vmware-0.2/doc/source/index.rst0000664000175000017500000000074212303705325020035 0ustar chuckchuck.. documentation master file, created by sphinx-quickstart on Tue Jul 9 22:26:36 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to 's documentation! ======================================================== Contents: .. toctree:: :maxdepth: 2 readme installation usage contributing Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` python-oslo.vmware-0.2/setup.py0000775000175000017500000000141512303705325015642 0ustar chuckchuck#!/usr/bin/env python # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT import setuptools setuptools.setup( setup_requires=['pbr'], pbr=True) python-oslo.vmware-0.2/test-requirements.txt0000664000175000017500000000062012303705325020363 0ustar chuckchuck# Hacking already pins down pep8, pyflakes and flake8 hacking>=0.8.0,<0.9 discover fixtures>=0.3.14 mock>=1.0 mox3>=0.7.0 python-subunit testrepository>=0.0.17 testscenarios>=0.4 testtools>=0.9.32 # when we can require tox>= 1.4, this can go into tox.ini: # [testenv:cover] # deps = {[testenv]deps} coverage coverage>=3.6 # this is required for the docs build jobs sphinx>=1.1.2,<1.2 oslosphinx python-oslo.vmware-0.2/PKG-INFO0000664000175000017500000000222212303705362015220 0ustar chuckchuckMetadata-Version: 1.1 Name: oslo.vmware Version: 0.2 Summary: Oslo VMware library for OpenStack projects Home-page: http://launchpad.net/oslo Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description: =================================== oslo.vmware =================================== Oslo VMware library for OpenStack projects * Free software: Apache license * Documentation: http://docs.openstack.org/developer/oslo.vmware Features -------- * TODO Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 python-oslo.vmware-0.2/babel.cfg0000664000175000017500000000002012303705325015642 0ustar chuckchuck[python: **.py] python-oslo.vmware-0.2/tests/0000775000175000017500000000000012303705362015267 5ustar chuckchuckpython-oslo.vmware-0.2/tests/test_vim.py0000664000175000017500000002327412303705325017502 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Unit tests for classes to invoke VMware VI SOAP calls. """ import httplib import urllib2 import mock import suds from oslo.vmware import exceptions from oslo.vmware import vim from tests import base class VimMessagePluginTest(base.TestCase): """Test class for VimMessagePlugin.""" def test_add_attribute_for_value(self): node = mock.Mock() node.name = 'value' plugin = vim.VimMessagePlugin() plugin.add_attribute_for_value(node) node.set.assert_called_once_with('xsi:type', 'xsd:string') def test_marshalled(self): plugin = vim.VimMessagePlugin() context = mock.Mock() plugin.marshalled(context) context.envelope.prune.assert_called_once_with() context.envelope.walk.assert_called_once_with( plugin.add_attribute_for_value) class VimTest(base.TestCase): """Test class for Vim.""" def setUp(self): super(VimTest, self).setUp() patcher = mock.patch('suds.client.Client') self.addCleanup(patcher.stop) self.SudsClientMock = patcher.start() @mock.patch.object(vim.Vim, '__getattr__', autospec=True) def test_init(self, getattr_mock): getattr_ret = mock.Mock() getattr_mock.side_effect = lambda *args: getattr_ret vim_obj = vim.Vim() getattr_mock.assert_called_once_with(vim_obj, 'RetrieveServiceContent') getattr_ret.assert_called_once_with('ServiceInstance') self.assertEqual(self.SudsClientMock.return_value, vim_obj.client) self.assertEqual(getattr_ret.return_value, vim_obj.service_content) def test_retrieve_properties_ex_fault_checker_with_empty_response(self): try: vim.Vim._retrieve_properties_ex_fault_checker(None) assert False except exceptions.VimFaultException as ex: self.assertEqual([exceptions.NOT_AUTHENTICATED], ex.fault_list) def test_retrieve_properties_ex_fault_checker(self): fault_list = ['FileFault', 'VimFault'] missing_set = [] for fault in fault_list: missing_elem = mock.Mock() missing_elem.fault.fault.__class__.__name__ = fault missing_set.append(missing_elem) obj_cont = mock.Mock() obj_cont.missingSet = missing_set response = mock.Mock() response.objects = [obj_cont] try: vim.Vim._retrieve_properties_ex_fault_checker(response) assert False except exceptions.VimFaultException as ex: self.assertEqual(fault_list, ex.fault_list) def test_vim_request_handler(self): managed_object = 'VirtualMachine' resp = mock.Mock() def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) return resp vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) ret = vim_obj.powerOn(managed_object) self.assertEqual(resp, ret) def test_vim_request_handler_with_retrieve_properties_ex_fault(self): managed_object = 'Datacenter' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) return None vim_obj = vim.Vim() attr_name = 'retrievePropertiesEx' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimFaultException, lambda: vim_obj.retrievePropertiesEx(managed_object)) def test_vim_request_handler_with_web_fault(self): managed_object = 'VirtualMachine' fault_list = ['Fault'] def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) doc = mock.Mock() detail = doc.childAtPath.return_value child = mock.Mock() child.get.return_value = fault_list[0] detail.getChildren.return_value = [child] raise suds.WebFault(None, doc) vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) try: vim_obj.powerOn(managed_object) except exceptions.VimFaultException as ex: self.assertEqual(fault_list, ex.fault_list) def test_vim_request_handler_with_attribute_error(self): managed_object = 'VirtualMachine' vim_obj = vim.Vim() # no powerOn method in Vim service_mock = mock.Mock(spec=vim.Vim) vim_obj._client.service = service_mock self.assertRaises(exceptions.VimAttributeException, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_http_cannot_send_error(self): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise httplib.CannotSendRequest() vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimSessionOverLoadException, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_http_response_not_ready_error(self): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise httplib.ResponseNotReady() vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimSessionOverLoadException, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_http_cannot_send_header_error(self): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise httplib.CannotSendHeader() vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimSessionOverLoadException, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_url_error(self): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise urllib2.URLError(None) vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimConnectionException, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_http_error(self): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise urllib2.HTTPError(None, None, None, None, None) vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exceptions.VimConnectionException, lambda: vim_obj.powerOn(managed_object)) def _test_vim_request_handler_with_exception(self, message, exception): managed_object = 'VirtualMachine' def side_effect(mo, **kwargs): self.assertEqual(managed_object, mo._type) self.assertEqual(managed_object, mo.value) raise Exception(message) vim_obj = vim.Vim() attr_name = 'powerOn' service_mock = vim_obj._client.service setattr(service_mock, attr_name, side_effect) self.assertRaises(exception, lambda: vim_obj.powerOn(managed_object)) def test_vim_request_handler_with_address_in_use_error(self): self._test_vim_request_handler_with_exception( vim.ADDRESS_IN_USE_ERROR, exceptions.VimSessionOverLoadException) def test_vim_request_handler_with_conn_abort_error(self): self._test_vim_request_handler_with_exception( vim.CONN_ABORT_ERROR, exceptions.VimSessionOverLoadException) def test_vim_request_handler_with_resp_not_xml_error(self): self._test_vim_request_handler_with_exception( vim.RESP_NOT_XML_ERROR, exceptions.VimSessionOverLoadException) def test_vim_request_handler_with_generic_error(self): self._test_vim_request_handler_with_exception( 'GENERIC_ERROR', exceptions.VimException) python-oslo.vmware-0.2/tests/test_rw_handles.py0000664000175000017500000002634112303705325021033 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Unit tests for read and write handles for image transfer. """ import mock from oslo.vmware import exceptions from oslo.vmware import rw_handles from oslo.vmware import vim_util from tests import base class FileHandleTest(base.TestCase): """Tests for FileHandle.""" def test_close(self): file_handle = mock.Mock() vmw_http_file = rw_handles.FileHandle(file_handle) vmw_http_file.close() file_handle.close.assert_called_once_with() def test_del(self): file_handle = mock.Mock() vmw_http_file = rw_handles.FileHandle(file_handle) del(vmw_http_file) file_handle.close.assert_called_once_with() def test_find_vmdk_url(self): device_url_0 = mock.Mock() device_url_0.disk = False device_url_1 = mock.Mock() device_url_1.disk = True device_url_1.url = 'https://*/ds1/vm1.vmdk' lease_info = mock.Mock() lease_info.deviceUrl = [device_url_0, device_url_1] host = '10.1.2.3' exp_url = 'https://%s/ds1/vm1.vmdk' % host vmw_http_file = rw_handles.FileHandle(None) self.assertEqual(exp_url, vmw_http_file._find_vmdk_url(lease_info, host)) class FileWriteHandleTest(base.TestCase): """Tests for FileWriteHandle.""" def setUp(self): super(FileWriteHandleTest, self).setUp() vim_cookie = mock.Mock() vim_cookie.name = 'name' vim_cookie.value = 'value' self._conn = mock.Mock() patcher = mock.patch('httplib.HTTPConnection') self.addCleanup(patcher.stop) HTTPConnectionMock = patcher.start() HTTPConnectionMock.return_value = self._conn self.vmw_http_write_file = rw_handles.FileWriteHandle( '10.1.2.3', 'dc-0', 'ds-0', [vim_cookie], '1.vmdk', 100, 'http') def test_write(self): self.vmw_http_write_file.write(None) self._conn.send.assert_called_once_with(None) def test_close(self): self.vmw_http_write_file.close() self._conn.getresponse.assert_called_once_with() self._conn.close.assert_called_once_with() class VmdkWriteHandleTest(base.TestCase): """Tests for VmdkWriteHandle.""" def setUp(self): super(VmdkWriteHandleTest, self).setUp() self._conn = mock.Mock() patcher = mock.patch('httplib.HTTPConnection') self.addCleanup(patcher.stop) HTTPConnectionMock = patcher.start() HTTPConnectionMock.return_value = self._conn def _create_mock_session(self, disk=True, progress=-1): device_url = mock.Mock() device_url.disk = disk device_url.url = 'http://*/ds/disk1.vmdk' lease_info = mock.Mock() lease_info.deviceUrl = [device_url] session = mock.Mock() def session_invoke_api_side_effect(module, method, *args, **kwargs): if module == session.vim: if method == 'ImportVApp': return mock.Mock() elif method == 'HttpNfcLeaseProgress': self.assertEqual(progress, kwargs['percent']) return return lease_info session.invoke_api.side_effect = session_invoke_api_side_effect vim_cookie = mock.Mock() vim_cookie.name = 'name' vim_cookie.value = 'value' session.vim.client.options.transport.cookiejar = [vim_cookie] return session def test_init_failure(self): session = self._create_mock_session(False) self.assertRaises(exceptions.VimException, lambda: rw_handles.VmdkWriteHandle(session, '10.1.2.3', 'rp-1', 'folder-1', None, 100)) def test_write(self): session = self._create_mock_session() handle = rw_handles.VmdkWriteHandle(session, '10.1.2.3', 'rp-1', 'folder-1', None, 100) data = [1] * 10 handle.write(data) self.assertEqual(len(data), handle._bytes_written) self._conn.send.assert_called_once_with(data) def test_update_progress(self): vmdk_size = 100 data_size = 10 session = self._create_mock_session(True, 10) handle = rw_handles.VmdkWriteHandle(session, '10.1.2.3', 'rp-1', 'folder-1', None, vmdk_size) handle.write([1] * data_size) handle.update_progress() def test_update_progress_with_error(self): session = self._create_mock_session(True, 10) handle = rw_handles.VmdkWriteHandle(session, '10.1.2.3', 'rp-1', 'folder-1', None, 100) session.invoke_api.side_effect = exceptions.VimException(None) self.assertRaises(exceptions.VimException, handle.update_progress) def test_close(self): session = self._create_mock_session() handle = rw_handles.VmdkWriteHandle(session, '10.1.2.3', 'rp-1', 'folder-1', None, 100) def session_invoke_api_side_effect(module, method, *args, **kwargs): if module == vim_util and method == 'get_object_property': return 'ready' self.assertEqual(session.vim, module) self.assertEqual('HttpNfcLeaseComplete', method) session.invoke_api = mock.Mock( side_effect=session_invoke_api_side_effect) handle.close() self.assertEqual(2, session.invoke_api.call_count) class VmdkReadHandleTest(base.TestCase): """Tests for VmdkReadHandle.""" def setUp(self): super(VmdkReadHandleTest, self).setUp() req_patcher = mock.patch('urllib2.Request') self.addCleanup(req_patcher.stop) RequestMock = req_patcher.start() RequestMock.return_value = mock.Mock() urlopen_patcher = mock.patch('urllib2.urlopen') self.addCleanup(urlopen_patcher.stop) urlopen_mock = urlopen_patcher.start() self._conn = mock.Mock() urlopen_mock.return_value = self._conn def _create_mock_session(self, disk=True, progress=-1): device_url = mock.Mock() device_url.disk = disk device_url.url = 'http://*/ds/disk1.vmdk' lease_info = mock.Mock() lease_info.deviceUrl = [device_url] session = mock.Mock() def session_invoke_api_side_effect(module, method, *args, **kwargs): if module == session.vim: if method == 'ExportVm': return mock.Mock() elif method == 'HttpNfcLeaseProgress': self.assertEqual(progress, kwargs['percent']) return return lease_info session.invoke_api.side_effect = session_invoke_api_side_effect vim_cookie = mock.Mock() vim_cookie.name = 'name' vim_cookie.value = 'value' session.vim.client.options.transport.cookiejar = [vim_cookie] return session def test_init_failure(self): session = self._create_mock_session(False) self.assertRaises(exceptions.VimException, lambda: rw_handles.VmdkReadHandle(session, '10.1.2.3', 'vm-1', '[ds] disk1.vmdk', 100)) def test_read(self): chunk_size = rw_handles.READ_CHUNKSIZE session = self._create_mock_session() self._conn.read.return_value = [1] * chunk_size handle = rw_handles.VmdkReadHandle(session, '10.1.2.3', 'vm-1', '[ds] disk1.vmdk', chunk_size * 10) handle.read(chunk_size) self.assertEqual(chunk_size, handle._bytes_read) self._conn.read.assert_called_once_with(chunk_size) def test_update_progress(self): chunk_size = rw_handles.READ_CHUNKSIZE vmdk_size = chunk_size * 10 session = self._create_mock_session(True, 10) self._conn.read.return_value = [1] * chunk_size handle = rw_handles.VmdkReadHandle(session, '10.1.2.3', 'vm-1', '[ds] disk1.vmdk', vmdk_size) handle.read(chunk_size) handle.update_progress() def test_update_progress_with_error(self): session = self._create_mock_session(True, 10) handle = rw_handles.VmdkReadHandle(session, '10.1.2.3', 'vm-1', '[ds] disk1.vmdk', 100) session.invoke_api.side_effect = exceptions.VimException(None) self.assertRaises(exceptions.VimException, handle.update_progress) def test_close(self): session = self._create_mock_session() handle = rw_handles.VmdkReadHandle(session, '10.1.2.3', 'vm-1', '[ds] disk1.vmdk', 100) def session_invoke_api_side_effect(module, method, *args, **kwargs): if module == vim_util and method == 'get_object_property': return 'ready' self.assertEqual(session.vim, module) self.assertEqual('HttpNfcLeaseComplete', method) session.invoke_api = mock.Mock( side_effect=session_invoke_api_side_effect) handle.close() self.assertEqual(2, session.invoke_api.call_count) class ImageReadHandleTest(base.TestCase): """Tests for ImageReadHandle.""" def test_read(self): max_items = 10 item = [1] * 10 class ImageReadIterator: def __init__(self): self.num_items = 0 def __iter__(self): return self def next(self): if (self.num_items < max_items): self.num_items += 1 return item raise StopIteration handle = rw_handles.ImageReadHandle(ImageReadIterator()) for _ in range(0, max_items): self.assertEqual(item, handle.read(10)) self.assertFalse(handle.read(10)) python-oslo.vmware-0.2/tests/test_image_transfer.py0000664000175000017500000001575412303705325021701 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Unit tests for functions and classes for image transfer. """ import math import mock from oslo.vmware import exceptions from oslo.vmware import image_transfer from tests import base class BlockingQueueTest(base.TestCase): """Tests for BlockingQueue.""" def test_read(self): max_size = 10 chunk_size = 10 max_transfer_size = 30 queue = image_transfer.BlockingQueue(max_size, max_transfer_size) def get_side_effect(): return [1] * chunk_size queue.get = mock.Mock(side_effect=get_side_effect) while True: data_item = queue.read(chunk_size) if not data_item: break self.assertEqual(max_transfer_size, queue._transferred) exp_calls = [mock.call()] * int(math.ceil(float(max_transfer_size) / chunk_size)) self.assertEqual(exp_calls, queue.get.call_args_list) def test_write(self): queue = image_transfer.BlockingQueue(10, 30) queue.put = mock.Mock() write_count = 10 for _ in range(0, write_count): queue.write([1]) exp_calls = [mock.call([1])] * write_count self.assertEqual(exp_calls, queue.put.call_args_list) def test_tell(self): max_transfer_size = 30 queue = image_transfer.BlockingQueue(10, 30) self.assertEqual(max_transfer_size, queue.tell()) class ImageWriterTest(base.TestCase): """Tests for ImageWriter class.""" def _create_image_writer(self): self.image_service = mock.Mock() self.context = mock.Mock() self.input_file = mock.Mock() self.image_id = mock.Mock() return image_transfer.ImageWriter(self.context, self.input_file, self.image_service, self.image_id) def test_start(self): writer = self._create_image_writer() status_list = ['queued', 'saving', 'active'] def image_service_show_side_effect(context, image_id): status = status_list.pop(0) return {'status': status} self.image_service.show.side_effect = image_service_show_side_effect exp_calls = [mock.call(self.context, self.image_id)] * len(status_list) with mock.patch.object(image_transfer, 'IMAGE_SERVICE_POLL_INTERVAL', 0): writer.start() self.assertTrue(writer.wait()) self.image_service.update.assert_called_once_with(self.context, self.image_id, {}, data=self.input_file) self.assertEqual(exp_calls, self.image_service.show.call_args_list) def test_start_with_killed_status(self): writer = self._create_image_writer() def image_service_show_side_effect(_context, _image_id): return {'status': 'killed'} self.image_service.show.side_effect = image_service_show_side_effect writer.start() self.assertRaises(exceptions.ImageTransferException, writer.wait) self.image_service.update.assert_called_once_with(self.context, self.image_id, {}, data=self.input_file) self.image_service.show.assert_called_once_with(self.context, self.image_id) def test_start_with_unknown_status(self): writer = self._create_image_writer() def image_service_show_side_effect(_context, _image_id): return {'status': 'unknown'} self.image_service.show.side_effect = image_service_show_side_effect writer.start() self.assertRaises(exceptions.ImageTransferException, writer.wait) self.image_service.update.assert_called_once_with(self.context, self.image_id, {}, data=self.input_file) self.image_service.show.assert_called_once_with(self.context, self.image_id) def test_start_with_image_service_show_exception(self): writer = self._create_image_writer() self.image_service.show.side_effect = RuntimeError() writer.start() self.assertRaises(exceptions.ImageTransferException, writer.wait) self.image_service.update.assert_called_once_with(self.context, self.image_id, {}, data=self.input_file) self.image_service.show.assert_called_once_with(self.context, self.image_id) class FileReadWriteTaskTest(base.TestCase): """Tests for FileReadWriteTask class.""" def test_start(self): data_items = [[1] * 10, [1] * 20, [1] * 5, []] def input_file_read_side_effect(arg): self.assertFalse(arg) data = data_items[input_file_read_side_effect.i] input_file_read_side_effect.i += 1 return data input_file_read_side_effect.i = 0 input_file = mock.Mock() input_file.read.side_effect = input_file_read_side_effect output_file = mock.Mock() rw_task = image_transfer.FileReadWriteTask(input_file, output_file) rw_task.start() self.assertTrue(rw_task.wait()) self.assertEqual(len(data_items), input_file.read.call_count) exp_calls = [] for i in range(0, len(data_items)): exp_calls.append(mock.call(data_items[i])) self.assertEqual(exp_calls, output_file.write.call_args_list) self.assertEqual(len(data_items), input_file.update_progress.call_count) self.assertEqual(len(data_items), output_file.update_progress.call_count) def test_start_with_read_exception(self): input_file = mock.Mock() input_file.read.side_effect = RuntimeError() output_file = mock.Mock() rw_task = image_transfer.FileReadWriteTask(input_file, output_file) rw_task.start() self.assertRaises(exceptions.ImageTransferException, rw_task.wait) input_file.read.assert_called_once_with(None) python-oslo.vmware-0.2/tests/__init__.py0000664000175000017500000000107212303705325017377 0ustar chuckchuck# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. python-oslo.vmware-0.2/tests/test_vim_util.py0000664000175000017500000003172312303705325020535 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Unit tests for VMware API utility module. """ import mock from oslo.vmware import vim_util from tests import base class VimUtilTest(base.TestCase): """Test class for utility methods in vim_util.""" def test_get_moref(self): moref = vim_util.get_moref("vm-0", "VirtualMachine") self.assertEqual("vm-0", moref.value) self.assertEqual("VirtualMachine", moref._type) def test_build_selection_spec(self): client_factory = mock.Mock() sel_spec = vim_util.build_selection_spec(client_factory, "test") self.assertEqual("test", sel_spec.name) def test_build_traversal_spec(self): client_factory = mock.Mock() sel_spec = mock.Mock() traversal_spec = vim_util.build_traversal_spec(client_factory, 'dc_to_hf', 'Datacenter', 'hostFolder', False, [sel_spec]) self.assertEqual("dc_to_hf", traversal_spec.name) self.assertEqual("hostFolder", traversal_spec.path) self.assertEqual([sel_spec], traversal_spec.selectSet) self.assertFalse(traversal_spec.skip) self.assertEqual("Datacenter", traversal_spec.type) @mock.patch.object(vim_util, 'build_selection_spec') def test_build_recursive_traversal_spec(self, build_selection_spec_mock): sel_spec = mock.Mock() rp_to_rp_sel_spec = mock.Mock() rp_to_vm_sel_spec = mock.Mock() def build_sel_spec_side_effect(client_factory, name): if name == 'visitFolders': return sel_spec elif name == 'rp_to_rp': return rp_to_rp_sel_spec elif name == 'rp_to_vm': return rp_to_vm_sel_spec else: return None build_selection_spec_mock.side_effect = build_sel_spec_side_effect traversal_spec_dict = {'dc_to_hf': {'type': 'Datacenter', 'path': 'hostFolder', 'skip': False, 'selectSet': [sel_spec]}, 'dc_to_vmf': {'type': 'Datacenter', 'path': 'vmFolder', 'skip': False, 'selectSet': [sel_spec]}, 'h_to_vm': {'type': 'HostSystem', 'path': 'vm', 'skip': False, 'selectSet': [sel_spec]}, 'cr_to_h': {'type': 'ComputeResource', 'path': 'host', 'skip': False, 'selectSet': []}, 'cr_to_ds': {'type': 'ComputeResource', 'path': 'datastore', 'skip': False, 'selectSet': []}, 'cr_to_rp': {'type': 'ComputeResource', 'path': 'resourcePool', 'skip': False, 'selectSet': [rp_to_rp_sel_spec, rp_to_vm_sel_spec]}, 'cr_to_rp': {'type': 'ComputeResource', 'path': 'resourcePool', 'skip': False, 'selectSet': [rp_to_rp_sel_spec, rp_to_vm_sel_spec]}, 'ccr_to_h': {'type': 'ClusterComputeResource', 'path': 'host', 'skip': False, 'selectSet': []}, 'ccr_to_ds': {'type': 'ClusterComputeResource', 'path': 'datastore', 'skip': False, 'selectSet': []}, 'ccr_to_rp': {'type': 'ClusterComputeResource', 'path': 'resourcePool', 'skip': False, 'selectSet': [rp_to_rp_sel_spec, rp_to_vm_sel_spec]}, 'rp_to_rp': {'type': 'ResourcePool', 'path': 'resourcePool', 'skip': False, 'selectSet': [rp_to_rp_sel_spec, rp_to_vm_sel_spec]}, 'rp_to_vm': {'type': 'ResourcePool', 'path': 'vm', 'skip': False, 'selectSet': [rp_to_rp_sel_spec, rp_to_vm_sel_spec]}, } client_factory = mock.Mock() client_factory.create.side_effect = lambda ns: mock.Mock() trav_spec = vim_util.build_recursive_traversal_spec(client_factory) self.assertEqual("visitFolders", trav_spec.name) self.assertEqual("childEntity", trav_spec.path) self.assertFalse(trav_spec.skip) self.assertEqual("Folder", trav_spec.type) self.assertEqual(len(traversal_spec_dict) + 1, len(trav_spec.selectSet)) for spec in trav_spec.selectSet: if spec.name not in traversal_spec_dict: self.assertEqual(sel_spec, spec) else: exp_spec = traversal_spec_dict[spec.name] self.assertEqual(exp_spec['type'], spec.type) self.assertEqual(exp_spec['path'], spec.path) self.assertEqual(exp_spec['skip'], spec.skip) self.assertEqual(exp_spec['selectSet'], spec.selectSet) def test_build_property_spec(self): client_factory = mock.Mock() prop_spec = vim_util.build_property_spec(client_factory) self.assertFalse(prop_spec.all) self.assertEqual(["name"], prop_spec.pathSet) self.assertEqual("VirtualMachine", prop_spec.type) def test_build_object_spec(self): client_factory = mock.Mock() root_folder = mock.Mock() specs = [mock.Mock()] obj_spec = vim_util.build_object_spec(client_factory, root_folder, specs) self.assertEqual(root_folder, obj_spec.obj) self.assertEqual(specs, obj_spec.selectSet) self.assertFalse(obj_spec.skip) def test_build_property_filter_spec(self): client_factory = mock.Mock() prop_specs = [mock.Mock()] obj_specs = [mock.Mock()] filter_spec = vim_util.build_property_filter_spec(client_factory, prop_specs, obj_specs) self.assertEqual(obj_specs, filter_spec.objectSet) self.assertEqual(prop_specs, filter_spec.propSet) @mock.patch( 'oslo.vmware.vim_util.build_recursive_traversal_spec') def test_get_objects(self, build_recursive_traversal_spec): vim = mock.Mock() trav_spec = mock.Mock() build_recursive_traversal_spec.return_value = trav_spec max_objects = 10 _type = "VirtualMachine" def vim_RetrievePropertiesEx_side_effect(pc, specSet, options): self.assertTrue(pc is vim.service_content.propertyCollector) self.assertEqual(max_objects, options.maxObjects) self.assertEqual(1, len(specSet)) property_filter_spec = specSet[0] propSet = property_filter_spec.propSet self.assertEqual(1, len(propSet)) prop_spec = propSet[0] self.assertFalse(prop_spec.all) self.assertEqual(["name"], prop_spec.pathSet) self.assertEqual(_type, prop_spec.type) objSet = property_filter_spec.objectSet self.assertEqual(1, len(objSet)) obj_spec = objSet[0] self.assertTrue(obj_spec.obj is vim.service_content.rootFolder) self.assertEqual([trav_spec], obj_spec.selectSet) self.assertFalse(obj_spec.skip) vim.RetrievePropertiesEx.side_effect = \ vim_RetrievePropertiesEx_side_effect vim_util.get_objects(vim, _type, max_objects) self.assertEqual(1, vim.RetrievePropertiesEx.call_count) def test_get_object_properties_with_empty_moref(self): vim = mock.Mock() ret = vim_util.get_object_properties(vim, None, None) self.assertEqual(None, ret) @mock.patch('oslo.vmware.vim_util.cancel_retrieval') def test_get_object_properties(self, cancel_retrieval): vim = mock.Mock() moref = mock.Mock() moref._type = "VirtualMachine" retrieve_result = mock.Mock() def vim_RetrievePropertiesEx_side_effect(pc, specSet, options): self.assertTrue(pc is vim.service_content.propertyCollector) self.assertEqual(1, options.maxObjects) self.assertEqual(1, len(specSet)) property_filter_spec = specSet[0] propSet = property_filter_spec.propSet self.assertEqual(1, len(propSet)) prop_spec = propSet[0] self.assertTrue(prop_spec.all) self.assertEqual(['name'], prop_spec.pathSet) self.assertEqual(moref._type, prop_spec.type) objSet = property_filter_spec.objectSet self.assertEqual(1, len(objSet)) obj_spec = objSet[0] self.assertEqual(moref, obj_spec.obj) self.assertEqual([], obj_spec.selectSet) self.assertFalse(obj_spec.skip) return retrieve_result vim.RetrievePropertiesEx.side_effect = \ vim_RetrievePropertiesEx_side_effect res = vim_util.get_object_properties(vim, moref, None) self.assertEqual(1, vim.RetrievePropertiesEx.call_count) self.assertTrue(res is retrieve_result.objects) cancel_retrieval.assert_called_once_with(vim, retrieve_result) def test_get_token(self): retrieve_result = object() self.assertFalse(vim_util._get_token(retrieve_result)) @mock.patch('oslo.vmware.vim_util._get_token') def test_cancel_retrieval(self, get_token): token = mock.Mock() get_token.return_value = token vim = mock.Mock() retrieve_result = mock.Mock() vim_util.cancel_retrieval(vim, retrieve_result) get_token.assert_called_once_with(retrieve_result) vim.CancelRetrievePropertiesEx.assert_called_once_with( vim.service_content.propertyCollector, token=token) @mock.patch('oslo.vmware.vim_util._get_token') def test_continue_retrieval(self, get_token): token = mock.Mock() get_token.return_value = token vim = mock.Mock() retrieve_result = mock.Mock() vim_util.continue_retrieval(vim, retrieve_result) get_token.assert_called_once_with(retrieve_result) vim.ContinueRetrievePropertiesEx.assert_called_once_with( vim.service_content.propertyCollector, token=token) @mock.patch('oslo.vmware.vim_util.get_object_properties') def test_get_object_property(self, get_object_properties): prop = mock.Mock() prop.val = "ubuntu-12.04" properties = mock.Mock() properties.propSet = [prop] properties_list = [properties] get_object_properties.return_value = properties_list vim = mock.Mock() moref = mock.Mock() property_name = 'name' val = vim_util.get_object_property(vim, moref, property_name) self.assertEqual(prop.val, val) get_object_properties.assert_called_once_with( vim, moref, [property_name]) python-oslo.vmware-0.2/tests/base.py0000664000175000017500000000367012303705325016560 0ustar chuckchuck# -*- coding: utf-8 -*- # Copyright 2010-2011 OpenStack Foundation # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import os import fixtures import testtools _TRUE_VALUES = ('true', '1', 'yes') # FIXME(dhellmann) Update this to use oslo.test library class TestCase(testtools.TestCase): """Test case base class for all unit tests.""" def setUp(self): """Run before each test method to initialize test environment.""" super(TestCase, self).setUp() test_timeout = os.environ.get('OS_TEST_TIMEOUT', 0) try: test_timeout = int(test_timeout) except ValueError: # If timeout value is invalid do not set a timeout. test_timeout = 0 if test_timeout > 0: self.useFixture(fixtures.Timeout(test_timeout, gentle=True)) self.useFixture(fixtures.NestedTempfile()) self.useFixture(fixtures.TempHomeDir()) if os.environ.get('OS_STDOUT_CAPTURE') in _TRUE_VALUES: stdout = self.useFixture(fixtures.StringStream('stdout')).stream self.useFixture(fixtures.MonkeyPatch('sys.stdout', stdout)) if os.environ.get('OS_STDERR_CAPTURE') in _TRUE_VALUES: stderr = self.useFixture(fixtures.StringStream('stderr')).stream self.useFixture(fixtures.MonkeyPatch('sys.stderr', stderr)) self.log_fixture = self.useFixture(fixtures.FakeLogger()) python-oslo.vmware-0.2/tests/test_api.py0000664000175000017500000003466412303705325017465 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Unit tests for session management and API invocation classes. """ import mock from oslo.vmware import api from oslo.vmware import exceptions from oslo.vmware import vim_util from tests import base class RetryDecoratorTest(base.TestCase): """Tests for retry decorator class.""" def test_retry(self): result = "RESULT" @api.RetryDecorator() def func(*args, **kwargs): return result self.assertEqual(result, func()) def func2(*args, **kwargs): return result retry = api.RetryDecorator() self.assertEqual(result, retry(func2)()) self.assertTrue(retry._retry_count == 0) def test_retry_with_expected_exceptions(self): result = "RESULT" responses = [exceptions.VimSessionOverLoadException(None), exceptions.VimSessionOverLoadException(None), result] def func(*args, **kwargs): response = responses.pop(0) if isinstance(response, Exception): raise response return response sleep_time_incr = 1 retry_count = 2 retry = api.RetryDecorator(10, sleep_time_incr, 10, (exceptions.VimSessionOverLoadException,)) self.assertEqual(result, retry(func)()) self.assertTrue(retry._retry_count == retry_count) self.assertEqual(retry_count * sleep_time_incr, retry._sleep_time) def test_retry_with_max_retries(self): responses = [exceptions.VimSessionOverLoadException(None), exceptions.VimSessionOverLoadException(None), exceptions.VimSessionOverLoadException(None)] def func(*args, **kwargs): response = responses.pop(0) if isinstance(response, Exception): raise response return response retry = api.RetryDecorator(2, 0, 0, (exceptions.VimSessionOverLoadException,)) self.assertRaises(exceptions.VimSessionOverLoadException, retry(func)) self.assertTrue(retry._retry_count == 2) def test_retry_with_unexpected_exception(self): def func(*args, **kwargs): raise exceptions.VimException(None) retry = api.RetryDecorator() self.assertRaises(exceptions.VimException, retry(func)) self.assertTrue(retry._retry_count == 0) class VMwareAPISessionTest(base.TestCase): """Tests for VMwareAPISession.""" SERVER_IP = '10.1.2.3' USERNAME = 'admin' PASSWORD = 'password' def setUp(self): super(VMwareAPISessionTest, self).setUp() patcher = mock.patch('oslo.vmware.vim.Vim') self.addCleanup(patcher.stop) self.VimMock = patcher.start() self.VimMock.side_effect = lambda *args, **kw: mock.Mock() def _create_api_session(self, _create_session, retry_count=10, task_poll_interval=1): return api.VMwareAPISession(VMwareAPISessionTest.SERVER_IP, VMwareAPISessionTest.USERNAME, VMwareAPISessionTest.PASSWORD, retry_count, task_poll_interval, 'https', _create_session) def test_vim(self): api_session = self._create_api_session(False) api_session.vim self.VimMock.assert_called_with(protocol=api_session._scheme, host=VMwareAPISessionTest.SERVER_IP, wsdl_loc=api_session._wsdl_loc) def test_create_session(self): session = mock.Mock() session.key = "12345" api_session = self._create_api_session(False) vim_obj = api_session.vim vim_obj.Login.return_value = session api_session._create_session() session_manager = vim_obj.service_content.sessionManager vim_obj.Login.assert_called_once_with( session_manager, userName=VMwareAPISessionTest.USERNAME, password=VMwareAPISessionTest.PASSWORD) self.assertFalse(vim_obj.TerminateSession.called) self.assertEqual(session.key, api_session._session_id) def test_create_session_with_existing_session(self): old_session_key = '12345' new_session_key = '67890' session = mock.Mock() session.key = new_session_key api_session = self._create_api_session(False) api_session._session_id = old_session_key vim_obj = api_session.vim vim_obj.Login.return_value = session api_session._create_session() session_manager = vim_obj.service_content.sessionManager vim_obj.Login.assert_called_once_with( session_manager, userName=VMwareAPISessionTest.USERNAME, password=VMwareAPISessionTest.PASSWORD) vim_obj.TerminateSession.assert_called_once_with( session_manager, sessionId=[old_session_key]) self.assertEqual(new_session_key, api_session._session_id) def test_invoke_api(self): api_session = self._create_api_session(True) response = mock.Mock() def api(*args, **kwargs): return response module = mock.Mock() module.api = api ret = api_session.invoke_api(module, 'api') self.assertEqual(response, ret) def test_invoke_api_with_expected_exception(self): api_session = self._create_api_session(True) ret = mock.Mock() responses = [exceptions.VimConnectionException(None), ret] def api(*args, **kwargs): response = responses.pop(0) if isinstance(response, Exception): raise response return response module = mock.Mock() module.api = api self.assertEqual(ret, api_session.invoke_api(module, 'api')) def test_invoke_api_with_vim_fault_exception(self): api_session = self._create_api_session(True) def api(*args, **kwargs): raise exceptions.VimFaultException([], None) module = mock.Mock() module.api = api self.assertRaises(exceptions.VimFaultException, lambda: api_session.invoke_api(module, 'api')) def test_invoke_api_with_empty_response(self): api_session = self._create_api_session(True) vim_obj = api_session.vim vim_obj.SessionIsActive.return_value = True def api(*args, **kwargs): raise exceptions.VimFaultException( [exceptions.NOT_AUTHENTICATED], None) module = mock.Mock() module.api = api ret = api_session.invoke_api(module, 'api') self.assertEqual([], ret) vim_obj.SessionIsActive.assert_called_once_with( vim_obj.service_content.sessionManager, sessionID=api_session._session_id, userName=api_session._session_username) def test_wait_for_task(self): api_session = self._create_api_session(True) task_info_list = [('queued', 0), ('running', 40), ('success', 100)] task_info_list_size = len(task_info_list) def invoke_api_side_effect(module, method, *args, **kwargs): (state, progress) = task_info_list.pop(0) task_info = mock.Mock() task_info.progress = progress task_info.state = state return task_info api_session.invoke_api = mock.Mock(side_effect=invoke_api_side_effect) task = mock.Mock() ret = api_session.wait_for_task(task) self.assertEqual('success', ret.state) self.assertEqual(100, ret.progress) api_session.invoke_api.assert_called_with(vim_util, 'get_object_property', api_session.vim, task, 'info') self.assertEqual(task_info_list_size, api_session.invoke_api.call_count) def test_wait_for_task_with_error_state(self): api_session = self._create_api_session(True) task_info_list = [('queued', 0), ('running', 40), ('error', -1)] task_info_list_size = len(task_info_list) def invoke_api_side_effect(module, method, *args, **kwargs): (state, progress) = task_info_list.pop(0) task_info = mock.Mock() task_info.progress = progress task_info.state = state return task_info api_session.invoke_api = mock.Mock(side_effect=invoke_api_side_effect) task = mock.Mock() self.assertRaises(exceptions.VMwareDriverException, lambda: api_session.wait_for_task(task)) api_session.invoke_api.assert_called_with(vim_util, 'get_object_property', api_session.vim, task, 'info') self.assertEqual(task_info_list_size, api_session.invoke_api.call_count) def test_wait_for_task_with_invoke_api_exception(self): api_session = self._create_api_session(True) api_session.invoke_api = mock.Mock( side_effect=exceptions.VimException(None)) task = mock.Mock() self.assertRaises(exceptions.VimException, lambda: api_session.wait_for_task(task)) api_session.invoke_api.assert_called_once_with(vim_util, 'get_object_property', api_session.vim, task, 'info') def test_wait_for_lease_ready(self): api_session = self._create_api_session(True) lease_states = ['initializing', 'ready'] num_states = len(lease_states) def invoke_api_side_effect(module, method, *args, **kwargs): return lease_states.pop(0) api_session.invoke_api = mock.Mock(side_effect=invoke_api_side_effect) lease = mock.Mock() api_session.wait_for_lease_ready(lease) api_session.invoke_api.assert_called_with(vim_util, 'get_object_property', api_session.vim, lease, 'state') self.assertEqual(num_states, api_session.invoke_api.call_count) def test_wait_for_lease_ready_with_error_state(self): api_session = self._create_api_session(True) responses = ['initializing', 'error', 'error_msg'] def invoke_api_side_effect(module, method, *args, **kwargs): return responses.pop(0) api_session.invoke_api = mock.Mock(side_effect=invoke_api_side_effect) lease = mock.Mock() self.assertRaises(exceptions.VimException, lambda: api_session.wait_for_lease_ready(lease)) exp_calls = [mock.call(vim_util, 'get_object_property', api_session.vim, lease, 'state')] * 2 exp_calls.append(mock.call(vim_util, 'get_object_property', api_session.vim, lease, 'error')) self.assertEqual(exp_calls, api_session.invoke_api.call_args_list) def test_wait_for_lease_ready_with_unknown_state(self): api_session = self._create_api_session(True) def invoke_api_side_effect(module, method, *args, **kwargs): return 'unknown' api_session.invoke_api = mock.Mock(side_effect=invoke_api_side_effect) lease = mock.Mock() self.assertRaises(exceptions.VimException, lambda: api_session.wait_for_lease_ready(lease)) api_session.invoke_api.assert_called_once_with(vim_util, 'get_object_property', api_session.vim, lease, 'state') def test_wait_for_lease_ready_with_invoke_api_exception(self): api_session = self._create_api_session(True) api_session.invoke_api = mock.Mock( side_effect=exceptions.VimException(None)) lease = mock.Mock() self.assertRaises(exceptions.VimException, lambda: api_session.wait_for_lease_ready(lease)) api_session.invoke_api.assert_called_once_with( vim_util, 'get_object_property', api_session.vim, lease, 'state') def _poll_task_well_known_exceptions(self, fault, expected_exception): api_session = self._create_api_session(False) def fake_invoke_api(self, module, method, *args, **kwargs): task_info = mock.Mock() task_info.progress = -1 task_info.state = 'error' error = mock.Mock() error.localizedMessage = "Error message" error_fault = mock.Mock() error_fault.__class__.__name__ = fault error.fault = error_fault task_info.error = error return task_info with ( mock.patch.object(api_session, 'invoke_api', fake_invoke_api) ): self.assertRaises(expected_exception, api_session._poll_task, 'fake-task') def test_poll_task_well_known_exceptions(self): for k, v in exceptions._fault_classes_registry.iteritems(): self._poll_task_well_known_exceptions(k, v) def test_poll_task_unknown_exception(self): _unknown_exceptions = { 'NoDiskSpace': exceptions.VMwareDriverException, 'RuntimeFault': exceptions.VMwareDriverException } for k, v in _unknown_exceptions.iteritems(): self._poll_task_well_known_exceptions(k, v) python-oslo.vmware-0.2/tests/test_vmware.py0000664000175000017500000000136512303705325020205 0ustar chuckchuck# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ test_vmware ---------------------------------- Tests for `vmware` module. """ from tests import base class TestVmware(base.TestCase): def test_something(self): pass python-oslo.vmware-0.2/.coveragerc0000664000175000017500000000015312303705325016244 0ustar chuckchuck[run] branch = True source = vmware omit = vmware/tests/*,vmware/openstack/* [report] ignore-errors = Truepython-oslo.vmware-0.2/tox.ini0000664000175000017500000000112212303705325015433 0ustar chuckchuck[tox] envlist = py26,py27,py33,pep8 [testenv] setenv = VIRTUAL_ENV={envdir} deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt commands = python setup.py testr --slowest --testr-args='{posargs}' [testenv:pep8] commands = flake8 [testenv:cover] setenv = VIRTUAL_ENV={envdir} commands = python setup.py testr --coverage [testenv:venv] commands = {posargs} [flake8] show-source = True exclude=.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,build,__init__.py builtins = _ [hacking] import_exceptions = oslo.vmware.openstack.common.gettextutils._python-oslo.vmware-0.2/CONTRIBUTING.rst0000664000175000017500000000104012303705325016560 0ustar chuckchuckIf you would like to contribute to the development of OpenStack, you must follow the steps in the "If you're a developer, start here" section of this page: http://wiki.openstack.org/HowToContribute Once those steps have been completed, changes to OpenStack should be submitted for review via the Gerrit tool, following the workflow documented at: http://wiki.openstack.org/GerritWorkflow Pull requests submitted through GitHub will be ignored. Bugs should be filed on Launchpad, not GitHub: https://bugs.launchpad.net/oslo.vmwarepython-oslo.vmware-0.2/.mailmap0000664000175000017500000000013012303705325015537 0ustar chuckchuck# Format is: # # python-oslo.vmware-0.2/setup.cfg0000664000175000017500000000231312303705362015745 0ustar chuckchuck[metadata] name = oslo.vmware summary = Oslo VMware library for OpenStack projects description-file = README.rst author = OpenStack author-email = openstack-dev@lists.openstack.org home-page = http://launchpad.net/oslo classifier = Environment :: OpenStack Intended Audience :: Information Technology Intended Audience :: System Administrators License :: OSI Approved :: Apache Software License Operating System :: POSIX :: Linux Programming Language :: Python Programming Language :: Python :: 2 Programming Language :: Python :: 2.7 Programming Language :: Python :: 2.6 Programming Language :: Python :: 3 Programming Language :: Python :: 3.3 [files] packages = oslo namespace_packages = oslo [build_sphinx] source-dir = doc/source build-dir = doc/build all_files = 1 [upload_sphinx] upload-dir = doc/build/html [compile_catalog] directory = oslo.vmware/locale domain = oslo.vmware [update_catalog] domain = oslo.vmware output_dir = oslo.vmware/locale input_file = oslo.vmware/locale/oslo.vmware.pot [extract_messages] keywords = _ gettext ngettext l_ lazy_gettext mapping_file = babel.cfg output_file = oslo.vmware/locale/oslo.vmware.pot [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 python-oslo.vmware-0.2/MANIFEST.in0000664000175000017500000000013512303705325015661 0ustar chuckchuckinclude AUTHORS include ChangeLog exclude .gitignore exclude .gitreview global-exclude *.pycpython-oslo.vmware-0.2/oslo/0000775000175000017500000000000012303705362015101 5ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/0000775000175000017500000000000012303705362016402 5ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/vim_util.py0000664000175000017500000003512312303705325020607 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ The VMware API utility module. """ import suds def get_moref(value, type_): """Get managed object reference. :param value: value of the managed object :param type_: type of the managed object :returns: managed object reference with given value and type """ moref = suds.sudsobject.Property(value) moref._type = type_ return moref def build_selection_spec(client_factory, name): """Builds the selection spec. :param client_factory: factory to get API input specs :param name: name for the selection spec :returns: selection spec """ sel_spec = client_factory.create('ns0:SelectionSpec') sel_spec.name = name return sel_spec def build_traversal_spec(client_factory, name, type_, path, skip, select_set): """Builds the traversal spec. :param client_factory: factory to get API input specs :param name: name for the traversal spec :param type_: type of the managed object :param path: property path of the managed object :param skip: whether or not to filter the object identified by param path :param select_set: set of selection specs specifying additional objects to filter :returns: traversal spec """ traversal_spec = client_factory.create('ns0:TraversalSpec') traversal_spec.name = name traversal_spec.type = type_ traversal_spec.path = path traversal_spec.skip = skip traversal_spec.selectSet = select_set return traversal_spec def build_recursive_traversal_spec(client_factory): """Builds recursive traversal spec to traverse managed object hierarchy. :param client_factory: factory to get API input specs :returns: recursive traversal spec """ visit_folders_select_spec = build_selection_spec(client_factory, 'visitFolders') # Next hop from Datacenter dc_to_hf = build_traversal_spec(client_factory, 'dc_to_hf', 'Datacenter', 'hostFolder', False, [visit_folders_select_spec]) dc_to_vmf = build_traversal_spec(client_factory, 'dc_to_vmf', 'Datacenter', 'vmFolder', False, [visit_folders_select_spec]) # Next hop from HostSystem h_to_vm = build_traversal_spec(client_factory, 'h_to_vm', 'HostSystem', 'vm', False, [visit_folders_select_spec]) # Next hop from ComputeResource cr_to_h = build_traversal_spec(client_factory, 'cr_to_h', 'ComputeResource', 'host', False, []) cr_to_ds = build_traversal_spec(client_factory, 'cr_to_ds', 'ComputeResource', 'datastore', False, []) rp_to_rp_select_spec = build_selection_spec(client_factory, 'rp_to_rp') rp_to_vm_select_spec = build_selection_spec(client_factory, 'rp_to_vm') cr_to_rp = build_traversal_spec(client_factory, 'cr_to_rp', 'ComputeResource', 'resourcePool', False, [rp_to_rp_select_spec, rp_to_vm_select_spec]) # Next hop from ClusterComputeResource ccr_to_h = build_traversal_spec(client_factory, 'ccr_to_h', 'ClusterComputeResource', 'host', False, []) ccr_to_ds = build_traversal_spec(client_factory, 'ccr_to_ds', 'ClusterComputeResource', 'datastore', False, []) ccr_to_rp = build_traversal_spec(client_factory, 'ccr_to_rp', 'ClusterComputeResource', 'resourcePool', False, [rp_to_rp_select_spec, rp_to_vm_select_spec]) # Next hop from ResourcePool rp_to_rp = build_traversal_spec(client_factory, 'rp_to_rp', 'ResourcePool', 'resourcePool', False, [rp_to_rp_select_spec, rp_to_vm_select_spec]) rp_to_vm = build_traversal_spec(client_factory, 'rp_to_vm', 'ResourcePool', 'vm', False, [rp_to_rp_select_spec, rp_to_vm_select_spec]) # Get the assorted traversal spec which takes care of the objects to # be searched for from the rootFolder traversal_spec = build_traversal_spec(client_factory, 'visitFolders', 'Folder', 'childEntity', False, [visit_folders_select_spec, h_to_vm, dc_to_hf, dc_to_vmf, cr_to_ds, cr_to_h, cr_to_rp, ccr_to_h, ccr_to_ds, ccr_to_rp, rp_to_rp, rp_to_vm]) return traversal_spec def build_property_spec(client_factory, type_='VirtualMachine', properties_to_collect=None, all_properties=False): """Builds the property spec. :param client_factory: factory to get API input specs :param type_: type of the managed object :param properties_to_collect: names of the managed object properties to be collected while traversal filtering :param all_properties: whether all properties of the managed object need to be collected :returns: property spec """ if not properties_to_collect: properties_to_collect = ['name'] property_spec = client_factory.create('ns0:PropertySpec') property_spec.all = all_properties property_spec.pathSet = properties_to_collect property_spec.type = type_ return property_spec def build_object_spec(client_factory, root_folder, traversal_specs): """Builds the object spec. :param client_factory: factory to get API input specs :param root_folder: root folder reference; the starting point of traversal :param traversal_specs: filter specs required for traversal :returns: object spec """ object_spec = client_factory.create('ns0:ObjectSpec') object_spec.obj = root_folder object_spec.skip = False object_spec.selectSet = traversal_specs return object_spec def build_property_filter_spec(client_factory, property_specs, object_specs): """Builds the property filter spec. :param client_factory: factory to get API input specs :param property_specs: property specs to be collected for filtered objects :param object_specs: object specs to identify objects to be filtered :returns: property filter spec """ property_filter_spec = client_factory.create('ns0:PropertyFilterSpec') property_filter_spec.propSet = property_specs property_filter_spec.objectSet = object_specs return property_filter_spec def get_objects(vim, type_, max_objects, properties_to_collect=None, all_properties=False): """Get all managed object references of the given type. It is the caller's responsibility to continue or cancel retrieval. :param vim: Vim object :param type_: type of the managed object :param max_objects: maximum number of objects that should be returned in a single call :param properties_to_collect: names of the managed object properties to be collected :param all_properties: whether all properties of the managed object need to be collected :returns: all managed object references of the given type :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ if not properties_to_collect: properties_to_collect = ['name'] client_factory = vim.client.factory recur_trav_spec = build_recursive_traversal_spec(client_factory) object_spec = build_object_spec(client_factory, vim.service_content.rootFolder, [recur_trav_spec]) property_spec = build_property_spec( client_factory, type_=type_, properties_to_collect=properties_to_collect, all_properties=all_properties) property_filter_spec = build_property_filter_spec(client_factory, [property_spec], [object_spec]) options = client_factory.create('ns0:RetrieveOptions') options.maxObjects = max_objects return vim.RetrievePropertiesEx(vim.service_content.propertyCollector, specSet=[property_filter_spec], options=options) def get_object_properties(vim, moref, properties_to_collect): """Get properties of the given managed object. :param vim: Vim object :param moref: managed object reference :param properties_to_collect: names of the managed object properties to be collected :returns: properties of the given managed object :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ if moref is None: return None client_factory = vim.client.factory all_properties = (properties_to_collect is None or len(properties_to_collect) == 0) property_spec = build_property_spec( client_factory, type_=moref._type, properties_to_collect=properties_to_collect, all_properties=all_properties) object_spec = build_object_spec(client_factory, moref, []) property_filter_spec = build_property_filter_spec(client_factory, [property_spec], [object_spec]) options = client_factory.create('ns0:RetrieveOptions') options.maxObjects = 1 retrieve_result = vim.RetrievePropertiesEx( vim.service_content.propertyCollector, specSet=[property_filter_spec], options=options) cancel_retrieval(vim, retrieve_result) return retrieve_result.objects def _get_token(retrieve_result): """Get token from result to obtain next set of results. :retrieve_result: Result of RetrievePropertiesEx API call :returns: token to obtain next set of results; None if no more results. """ return getattr(retrieve_result, 'token', None) def cancel_retrieval(vim, retrieve_result): """Cancels the retrieve operation if necessary. :param vim: Vim object :param retrieve_result: result of RetrievePropertiesEx API call :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ token = _get_token(retrieve_result) if token: collector = vim.service_content.propertyCollector vim.CancelRetrievePropertiesEx(collector, token=token) def continue_retrieval(vim, retrieve_result): """Continue retrieving results, if available. :param vim: Vim object :param retrieve_result: result of RetrievePropertiesEx API call :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ token = _get_token(retrieve_result) if token: collector = vim.service_content.propertyCollector return vim.ContinueRetrievePropertiesEx(collector, token=token) def get_object_property(vim, moref, property_name): """Get property of the given managed object. :param vim: Vim object :param moref: managed object reference :param property_name: name of the property to be retrieved :returns: property of the given managed object :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ props = get_object_properties(vim, moref, [property_name]) prop_val = None if props: prop = None if hasattr(props[0], 'propSet'): # propSet will be set only if the server provides value # for the field prop = props[0].propSet if prop: prop_val = prop[0].val return prop_val python-oslo.vmware-0.2/oslo/vmware/openstack/0000775000175000017500000000000012303705362020371 5ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/openstack/__init__.py0000664000175000017500000000000012303705325022467 0ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/openstack/common/0000775000175000017500000000000012303705362021661 5ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/openstack/common/jsonutils.py0000664000175000017500000001510612303705325024267 0ustar chuckchuck# Copyright 2010 United States Government as represented by the # Administrator of the National Aeronautics and Space Administration. # Copyright 2011 Justin Santa Barbara # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. ''' JSON related utilities. This module provides a few things: 1) A handy function for getting an object down to something that can be JSON serialized. See to_primitive(). 2) Wrappers around loads() and dumps(). The dumps() wrapper will automatically use to_primitive() for you if needed. 3) This sets up anyjson to use the loads() and dumps() wrappers if anyjson is available. ''' import datetime import functools import inspect import itertools import json try: import xmlrpclib except ImportError: # NOTE(jaypipes): xmlrpclib was renamed to xmlrpc.client in Python3 # however the function and object call signatures # remained the same. This whole try/except block should # be removed and replaced with a call to six.moves once # six 1.4.2 is released. See http://bit.ly/1bqrVzu import xmlrpc.client as xmlrpclib import six from oslo.vmware.openstack.common import gettextutils from oslo.vmware.openstack.common import importutils from oslo.vmware.openstack.common import timeutils netaddr = importutils.try_import("netaddr") _nasty_type_tests = [inspect.ismodule, inspect.isclass, inspect.ismethod, inspect.isfunction, inspect.isgeneratorfunction, inspect.isgenerator, inspect.istraceback, inspect.isframe, inspect.iscode, inspect.isbuiltin, inspect.isroutine, inspect.isabstract] _simple_types = (six.string_types + six.integer_types + (type(None), bool, float)) def to_primitive(value, convert_instances=False, convert_datetime=True, level=0, max_depth=3): """Convert a complex object into primitives. Handy for JSON serialization. We can optionally handle instances, but since this is a recursive function, we could have cyclical data structures. To handle cyclical data structures we could track the actual objects visited in a set, but not all objects are hashable. Instead we just track the depth of the object inspections and don't go too deep. Therefore, convert_instances=True is lossy ... be aware. """ # handle obvious types first - order of basic types determined by running # full tests on nova project, resulting in the following counts: # 572754 # 460353 # 379632 # 274610 # 199918 # 114200 # 51817 # 26164 # 6491 # 283 # 19 if isinstance(value, _simple_types): return value if isinstance(value, datetime.datetime): if convert_datetime: return timeutils.strtime(value) else: return value # value of itertools.count doesn't get caught by nasty_type_tests # and results in infinite loop when list(value) is called. if type(value) == itertools.count: return six.text_type(value) # FIXME(vish): Workaround for LP bug 852095. Without this workaround, # tests that raise an exception in a mocked method that # has a @wrap_exception with a notifier will fail. If # we up the dependency to 0.5.4 (when it is released) we # can remove this workaround. if getattr(value, '__module__', None) == 'mox': return 'mock' if level > max_depth: return '?' # The try block may not be necessary after the class check above, # but just in case ... try: recursive = functools.partial(to_primitive, convert_instances=convert_instances, convert_datetime=convert_datetime, level=level, max_depth=max_depth) if isinstance(value, dict): return dict((k, recursive(v)) for k, v in six.iteritems(value)) elif isinstance(value, (list, tuple)): return [recursive(lv) for lv in value] # It's not clear why xmlrpclib created their own DateTime type, but # for our purposes, make it a datetime type which is explicitly # handled if isinstance(value, xmlrpclib.DateTime): value = datetime.datetime(*tuple(value.timetuple())[:6]) if convert_datetime and isinstance(value, datetime.datetime): return timeutils.strtime(value) elif isinstance(value, gettextutils.Message): return value.data elif hasattr(value, 'iteritems'): return recursive(dict(value.iteritems()), level=level + 1) elif hasattr(value, '__iter__'): return recursive(list(value)) elif convert_instances and hasattr(value, '__dict__'): # Likely an instance of something. Watch for cycles. # Ignore class member vars. return recursive(value.__dict__, level=level + 1) elif netaddr and isinstance(value, netaddr.IPAddress): return six.text_type(value) else: if any(test(value) for test in _nasty_type_tests): return six.text_type(value) return value except TypeError: # Class objects are tricky since they may define something like # __iter__ defined but it isn't callable as list(). return six.text_type(value) def dumps(value, default=to_primitive, **kwargs): return json.dumps(value, default=default, **kwargs) def loads(s): return json.loads(s) def load(s): return json.load(s) try: import anyjson except ImportError: pass else: anyjson._modules.append((__name__, 'dumps', TypeError, 'loads', ValueError, 'load')) anyjson.force_implementation(__name__) python-oslo.vmware-0.2/oslo/vmware/openstack/common/timeutils.py0000664000175000017500000001424112303705325024253 0ustar chuckchuck# Copyright 2011 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Time related utilities and helper functions. """ import calendar import datetime import time import iso8601 import six # ISO 8601 extended time format with microseconds _ISO8601_TIME_FORMAT_SUBSECOND = '%Y-%m-%dT%H:%M:%S.%f' _ISO8601_TIME_FORMAT = '%Y-%m-%dT%H:%M:%S' PERFECT_TIME_FORMAT = _ISO8601_TIME_FORMAT_SUBSECOND def isotime(at=None, subsecond=False): """Stringify time in ISO 8601 format.""" if not at: at = utcnow() st = at.strftime(_ISO8601_TIME_FORMAT if not subsecond else _ISO8601_TIME_FORMAT_SUBSECOND) tz = at.tzinfo.tzname(None) if at.tzinfo else 'UTC' st += ('Z' if tz == 'UTC' else tz) return st def parse_isotime(timestr): """Parse time from ISO 8601 format.""" try: return iso8601.parse_date(timestr) except iso8601.ParseError as e: raise ValueError(six.text_type(e)) except TypeError as e: raise ValueError(six.text_type(e)) def strtime(at=None, fmt=PERFECT_TIME_FORMAT): """Returns formatted utcnow.""" if not at: at = utcnow() return at.strftime(fmt) def parse_strtime(timestr, fmt=PERFECT_TIME_FORMAT): """Turn a formatted time back into a datetime.""" return datetime.datetime.strptime(timestr, fmt) def normalize_time(timestamp): """Normalize time in arbitrary timezone to UTC naive object.""" offset = timestamp.utcoffset() if offset is None: return timestamp return timestamp.replace(tzinfo=None) - offset def is_older_than(before, seconds): """Return True if before is older than seconds.""" if isinstance(before, six.string_types): before = parse_strtime(before).replace(tzinfo=None) else: before = before.replace(tzinfo=None) return utcnow() - before > datetime.timedelta(seconds=seconds) def is_newer_than(after, seconds): """Return True if after is newer than seconds.""" if isinstance(after, six.string_types): after = parse_strtime(after).replace(tzinfo=None) else: after = after.replace(tzinfo=None) return after - utcnow() > datetime.timedelta(seconds=seconds) def utcnow_ts(): """Timestamp version of our utcnow function.""" if utcnow.override_time is None: # NOTE(kgriffs): This is several times faster # than going through calendar.timegm(...) return int(time.time()) return calendar.timegm(utcnow().timetuple()) def utcnow(): """Overridable version of utils.utcnow.""" if utcnow.override_time: try: return utcnow.override_time.pop(0) except AttributeError: return utcnow.override_time return datetime.datetime.utcnow() def iso8601_from_timestamp(timestamp): """Returns a iso8601 formatted date from timestamp.""" return isotime(datetime.datetime.utcfromtimestamp(timestamp)) utcnow.override_time = None def set_time_override(override_time=None): """Overrides utils.utcnow. Make it return a constant time or a list thereof, one at a time. :param override_time: datetime instance or list thereof. If not given, defaults to the current UTC time. """ utcnow.override_time = override_time or datetime.datetime.utcnow() def advance_time_delta(timedelta): """Advance overridden time using a datetime.timedelta.""" assert(not utcnow.override_time is None) try: for dt in utcnow.override_time: dt += timedelta except TypeError: utcnow.override_time += timedelta def advance_time_seconds(seconds): """Advance overridden time by seconds.""" advance_time_delta(datetime.timedelta(0, seconds)) def clear_time_override(): """Remove the overridden time.""" utcnow.override_time = None def marshall_now(now=None): """Make an rpc-safe datetime with microseconds. Note: tzinfo is stripped, but not required for relative times. """ if not now: now = utcnow() return dict(day=now.day, month=now.month, year=now.year, hour=now.hour, minute=now.minute, second=now.second, microsecond=now.microsecond) def unmarshall_time(tyme): """Unmarshall a datetime dict.""" return datetime.datetime(day=tyme['day'], month=tyme['month'], year=tyme['year'], hour=tyme['hour'], minute=tyme['minute'], second=tyme['second'], microsecond=tyme['microsecond']) def delta_seconds(before, after): """Return the difference between two timing objects. Compute the difference in seconds between two date, time, or datetime objects (as a float, to microsecond resolution). """ delta = after - before return total_seconds(delta) def total_seconds(delta): """Return the total seconds of datetime.timedelta object. Compute total seconds of datetime.timedelta, datetime.timedelta doesn't have method total_seconds in Python2.6, calculate it manually. """ try: return delta.total_seconds() except AttributeError: return ((delta.days * 24 * 3600) + delta.seconds + float(delta.microseconds) / (10 ** 6)) def is_soon(dt, window): """Determines if time is going to happen in the next window seconds. :param dt: the time :param window: minimum seconds to remain to consider the time not soon :return: True if expiration is within the given duration """ soon = (utcnow() + datetime.timedelta(seconds=window)) return normalize_time(dt) <= soon python-oslo.vmware-0.2/oslo/vmware/openstack/common/__init__.py0000664000175000017500000000000012303705325023757 0ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/openstack/common/gettextutils.py0000664000175000017500000004460512303705325025010 0ustar chuckchuck# Copyright 2012 Red Hat, Inc. # Copyright 2013 IBM Corp. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ gettext for openstack-common modules. Usual usage in an openstack.common module: from oslo.vmware.openstack.common.gettextutils import _ """ import copy import functools import gettext import locale from logging import handlers import os import re from babel import localedata import six _localedir = os.environ.get('oslo.vmware'.upper() + '_LOCALEDIR') _t = gettext.translation('oslo.vmware', localedir=_localedir, fallback=True) # We use separate translation catalogs for each log level, so set up a # mapping between the log level name and the translator. The domain # for the log level is project_name + "-log-" + log_level so messages # for each level end up in their own catalog. _t_log_levels = dict( (level, gettext.translation('oslo.vmware' + '-log-' + level, localedir=_localedir, fallback=True)) for level in ['info', 'warning', 'error', 'critical'] ) _AVAILABLE_LANGUAGES = {} USE_LAZY = False def enable_lazy(): """Convenience function for configuring _() to use lazy gettext Call this at the start of execution to enable the gettextutils._ function to use lazy gettext functionality. This is useful if your project is importing _ directly instead of using the gettextutils.install() way of importing the _ function. """ global USE_LAZY USE_LAZY = True def _(msg): if USE_LAZY: return Message(msg, domain='oslo.vmware') else: if six.PY3: return _t.gettext(msg) return _t.ugettext(msg) def _log_translation(msg, level): """Build a single translation of a log message """ if USE_LAZY: return Message(msg, domain='oslo.vmware' + '-log-' + level) else: translator = _t_log_levels[level] if six.PY3: return translator.gettext(msg) return translator.ugettext(msg) # Translators for log levels. # # The abbreviated names are meant to reflect the usual use of a short # name like '_'. The "L" is for "log" and the other letter comes from # the level. _LI = functools.partial(_log_translation, level='info') _LW = functools.partial(_log_translation, level='warning') _LE = functools.partial(_log_translation, level='error') _LC = functools.partial(_log_translation, level='critical') def install(domain, lazy=False): """Install a _() function using the given translation domain. Given a translation domain, install a _() function using gettext's install() function. The main difference from gettext.install() is that we allow overriding the default localedir (e.g. /usr/share/locale) using a translation-domain-specific environment variable (e.g. NOVA_LOCALEDIR). :param domain: the translation domain :param lazy: indicates whether or not to install the lazy _() function. The lazy _() introduces a way to do deferred translation of messages by installing a _ that builds Message objects, instead of strings, which can then be lazily translated into any available locale. """ if lazy: # NOTE(mrodden): Lazy gettext functionality. # # The following introduces a deferred way to do translations on # messages in OpenStack. We override the standard _() function # and % (format string) operation to build Message objects that can # later be translated when we have more information. def _lazy_gettext(msg): """Create and return a Message object. Lazy gettext function for a given domain, it is a factory method for a project/module to get a lazy gettext function for its own translation domain (i.e. nova, glance, cinder, etc.) Message encapsulates a string so that we can translate it later when needed. """ return Message(msg, domain=domain) from six import moves moves.builtins.__dict__['_'] = _lazy_gettext else: localedir = '%s_LOCALEDIR' % domain.upper() if six.PY3: gettext.install(domain, localedir=os.environ.get(localedir)) else: gettext.install(domain, localedir=os.environ.get(localedir), unicode=True) class Message(six.text_type): """A Message object is a unicode object that can be translated. Translation of Message is done explicitly using the translate() method. For all non-translation intents and purposes, a Message is simply unicode, and can be treated as such. """ def __new__(cls, msgid, msgtext=None, params=None, domain='oslo.vmware', *args): """Create a new Message object. In order for translation to work gettext requires a message ID, this msgid will be used as the base unicode text. It is also possible for the msgid and the base unicode text to be different by passing the msgtext parameter. """ # If the base msgtext is not given, we use the default translation # of the msgid (which is in English) just in case the system locale is # not English, so that the base text will be in that locale by default. if not msgtext: msgtext = Message._translate_msgid(msgid, domain) # We want to initialize the parent unicode with the actual object that # would have been plain unicode if 'Message' was not enabled. msg = super(Message, cls).__new__(cls, msgtext) msg.msgid = msgid msg.domain = domain msg.params = params return msg def translate(self, desired_locale=None): """Translate this message to the desired locale. :param desired_locale: The desired locale to translate the message to, if no locale is provided the message will be translated to the system's default locale. :returns: the translated message in unicode """ translated_message = Message._translate_msgid(self.msgid, self.domain, desired_locale) if self.params is None: # No need for more translation return translated_message # This Message object may have been formatted with one or more # Message objects as substitution arguments, given either as a single # argument, part of a tuple, or as one or more values in a dictionary. # When translating this Message we need to translate those Messages too translated_params = _translate_args(self.params, desired_locale) translated_message = translated_message % translated_params return translated_message @staticmethod def _translate_msgid(msgid, domain, desired_locale=None): if not desired_locale: system_locale = locale.getdefaultlocale() # If the system locale is not available to the runtime use English if not system_locale[0]: desired_locale = 'en_US' else: desired_locale = system_locale[0] locale_dir = os.environ.get(domain.upper() + '_LOCALEDIR') lang = gettext.translation(domain, localedir=locale_dir, languages=[desired_locale], fallback=True) if six.PY3: translator = lang.gettext else: translator = lang.ugettext translated_message = translator(msgid) return translated_message def __mod__(self, other): # When we mod a Message we want the actual operation to be performed # by the parent class (i.e. unicode()), the only thing we do here is # save the original msgid and the parameters in case of a translation params = self._sanitize_mod_params(other) unicode_mod = super(Message, self).__mod__(params) modded = Message(self.msgid, msgtext=unicode_mod, params=params, domain=self.domain) return modded def _sanitize_mod_params(self, other): """Sanitize the object being modded with this Message. - Add support for modding 'None' so translation supports it - Trim the modded object, which can be a large dictionary, to only those keys that would actually be used in a translation - Snapshot the object being modded, in case the message is translated, it will be used as it was when the Message was created """ if other is None: params = (other,) elif isinstance(other, dict): params = self._trim_dictionary_parameters(other) else: params = self._copy_param(other) return params def _trim_dictionary_parameters(self, dict_param): """Return a dict that only has matching entries in the msgid.""" # NOTE(luisg): Here we trim down the dictionary passed as parameters # to avoid carrying a lot of unnecessary weight around in the message # object, for example if someone passes in Message() % locals() but # only some params are used, and additionally we prevent errors for # non-deepcopyable objects by unicoding() them. # Look for %(param) keys in msgid; # Skip %% and deal with the case where % is first character on the line keys = re.findall('(?:[^%]|^)?%\((\w*)\)[a-z]', self.msgid) # If we don't find any %(param) keys but have a %s if not keys and re.findall('(?:[^%]|^)%[a-z]', self.msgid): # Apparently the full dictionary is the parameter params = self._copy_param(dict_param) else: params = {} # Save our existing parameters as defaults to protect # ourselves from losing values if we are called through an # (erroneous) chain that builds a valid Message with # arguments, and then does something like "msg % kwds" # where kwds is an empty dictionary. src = {} if isinstance(self.params, dict): src.update(self.params) src.update(dict_param) for key in keys: params[key] = self._copy_param(src[key]) return params def _copy_param(self, param): try: return copy.deepcopy(param) except TypeError: # Fallback to casting to unicode this will handle the # python code-like objects that can't be deep-copied return six.text_type(param) def __add__(self, other): msg = _('Message objects do not support addition.') raise TypeError(msg) def __radd__(self, other): return self.__add__(other) def __str__(self): # NOTE(luisg): Logging in python 2.6 tries to str() log records, # and it expects specifically a UnicodeError in order to proceed. msg = _('Message objects do not support str() because they may ' 'contain non-ascii characters. ' 'Please use unicode() or translate() instead.') raise UnicodeError(msg) def get_available_languages(domain): """Lists the available languages for the given translation domain. :param domain: the domain to get languages for """ if domain in _AVAILABLE_LANGUAGES: return copy.copy(_AVAILABLE_LANGUAGES[domain]) localedir = '%s_LOCALEDIR' % domain.upper() find = lambda x: gettext.find(domain, localedir=os.environ.get(localedir), languages=[x]) # NOTE(mrodden): en_US should always be available (and first in case # order matters) since our in-line message strings are en_US language_list = ['en_US'] # NOTE(luisg): Babel <1.0 used a function called list(), which was # renamed to locale_identifiers() in >=1.0, the requirements master list # requires >=0.9.6, uncapped, so defensively work with both. We can remove # this check when the master list updates to >=1.0, and update all projects list_identifiers = (getattr(localedata, 'list', None) or getattr(localedata, 'locale_identifiers')) locale_identifiers = list_identifiers() for i in locale_identifiers: if find(i) is not None: language_list.append(i) # NOTE(luisg): Babel>=1.0,<1.3 has a bug where some OpenStack supported # locales (e.g. 'zh_CN', and 'zh_TW') aren't supported even though they # are perfectly legitimate locales: # https://github.com/mitsuhiko/babel/issues/37 # In Babel 1.3 they fixed the bug and they support these locales, but # they are still not explicitly "listed" by locale_identifiers(). # That is why we add the locales here explicitly if necessary so that # they are listed as supported. aliases = {'zh': 'zh_CN', 'zh_Hant_HK': 'zh_HK', 'zh_Hant': 'zh_TW', 'fil': 'tl_PH'} for (locale, alias) in six.iteritems(aliases): if locale in language_list and alias not in language_list: language_list.append(alias) _AVAILABLE_LANGUAGES[domain] = language_list return copy.copy(language_list) def translate(obj, desired_locale=None): """Gets the translated unicode representation of the given object. If the object is not translatable it is returned as-is. If the locale is None the object is translated to the system locale. :param obj: the object to translate :param desired_locale: the locale to translate the message to, if None the default system locale will be used :returns: the translated object in unicode, or the original object if it could not be translated """ message = obj if not isinstance(message, Message): # If the object to translate is not already translatable, # let's first get its unicode representation message = six.text_type(obj) if isinstance(message, Message): # Even after unicoding() we still need to check if we are # running with translatable unicode before translating return message.translate(desired_locale) return obj def _translate_args(args, desired_locale=None): """Translates all the translatable elements of the given arguments object. This method is used for translating the translatable values in method arguments which include values of tuples or dictionaries. If the object is not a tuple or a dictionary the object itself is translated if it is translatable. If the locale is None the object is translated to the system locale. :param args: the args to translate :param desired_locale: the locale to translate the args to, if None the default system locale will be used :returns: a new args object with the translated contents of the original """ if isinstance(args, tuple): return tuple(translate(v, desired_locale) for v in args) if isinstance(args, dict): translated_dict = {} for (k, v) in six.iteritems(args): translated_v = translate(v, desired_locale) translated_dict[k] = translated_v return translated_dict return translate(args, desired_locale) class TranslationHandler(handlers.MemoryHandler): """Handler that translates records before logging them. The TranslationHandler takes a locale and a target logging.Handler object to forward LogRecord objects to after translating them. This handler depends on Message objects being logged, instead of regular strings. The handler can be configured declaratively in the logging.conf as follows: [handlers] keys = translatedlog, translator [handler_translatedlog] class = handlers.WatchedFileHandler args = ('/var/log/api-localized.log',) formatter = context [handler_translator] class = openstack.common.log.TranslationHandler target = translatedlog args = ('zh_CN',) If the specified locale is not available in the system, the handler will log in the default locale. """ def __init__(self, locale=None, target=None): """Initialize a TranslationHandler :param locale: locale to use for translating messages :param target: logging.Handler object to forward LogRecord objects to after translation """ # NOTE(luisg): In order to allow this handler to be a wrapper for # other handlers, such as a FileHandler, and still be able to # configure it using logging.conf, this handler has to extend # MemoryHandler because only the MemoryHandlers' logging.conf # parsing is implemented such that it accepts a target handler. handlers.MemoryHandler.__init__(self, capacity=0, target=target) self.locale = locale def setFormatter(self, fmt): self.target.setFormatter(fmt) def emit(self, record): # We save the message from the original record to restore it # after translation, so other handlers are not affected by this original_msg = record.msg original_args = record.args try: self._translate_and_log_record(record) finally: record.msg = original_msg record.args = original_args def _translate_and_log_record(self, record): record.msg = translate(record.msg, self.locale) # In addition to translating the message, we also need to translate # arguments that were passed to the log method that were not part # of the main message e.g., log.info(_('Some message %s'), this_one)) record.args = _translate_args(record.args, self.locale) self.target.emit(record) python-oslo.vmware-0.2/oslo/vmware/openstack/common/importutils.py0000664000175000017500000000452012303705325024626 0ustar chuckchuck# Copyright 2011 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Import related utilities and helper functions. """ import sys import traceback def import_class(import_str): """Returns a class from a string including module and class.""" mod_str, _sep, class_str = import_str.rpartition('.') try: __import__(mod_str) return getattr(sys.modules[mod_str], class_str) except (ValueError, AttributeError): raise ImportError('Class %s cannot be found (%s)' % (class_str, traceback.format_exception(*sys.exc_info()))) def import_object(import_str, *args, **kwargs): """Import a class and return an instance of it.""" return import_class(import_str)(*args, **kwargs) def import_object_ns(name_space, import_str, *args, **kwargs): """Tries to import object from default namespace. Imports a class and return an instance of it, first by trying to find the class in a default namespace, then failing back to a full path if not found in the default namespace. """ import_value = "%s.%s" % (name_space, import_str) try: return import_class(import_value)(*args, **kwargs) except ImportError: return import_class(import_str)(*args, **kwargs) def import_module(import_str): """Import a module.""" __import__(import_str) return sys.modules[import_str] def import_versioned_module(version, submodule=None): module = 'oslo.vmware.v%s' % version if submodule: module = '.'.join((module, submodule)) return import_module(module) def try_import(import_str, default=None): """Try to import a module and if it fails return default.""" try: return import_module(import_str) except ImportError: return default python-oslo.vmware-0.2/oslo/vmware/openstack/common/local.py0000664000175000017500000000321512303705325023325 0ustar chuckchuck# Copyright 2011 OpenStack Foundation. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """Local storage of variables using weak references""" import threading import weakref class WeakLocal(threading.local): def __getattribute__(self, attr): rval = super(WeakLocal, self).__getattribute__(attr) if rval: # NOTE(mikal): this bit is confusing. What is stored is a weak # reference, not the value itself. We therefore need to lookup # the weak reference and return the inner value here. rval = rval() return rval def __setattr__(self, attr, value): value = weakref.ref(value) return super(WeakLocal, self).__setattr__(attr, value) # NOTE(mikal): the name "store" should be deprecated in the future store = WeakLocal() # A "weak" store uses weak references and allows an object to fall out of scope # when it falls out of scope in the code that uses the thread local storage. A # "strong" store will hold a reference to the object so that it never falls out # of scope. weak_store = WeakLocal() strong_store = threading.local() python-oslo.vmware-0.2/oslo/vmware/__init__.py0000664000175000017500000000000012303705325020500 0ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/vim.py0000664000175000017500000002356712303705326017564 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Classes for making VMware VI SOAP calls. """ import httplib import logging import urllib2 import suds from oslo.vmware import exceptions from oslo.vmware.openstack.common.gettextutils import _ from oslo.vmware import vim_util ADDRESS_IN_USE_ERROR = 'Address already in use' CONN_ABORT_ERROR = 'Software caused connection abort' RESP_NOT_XML_ERROR = "Response is 'text/html', not 'text/xml'" LOG = logging.getLogger(__name__) class VimMessagePlugin(suds.plugin.MessagePlugin): """Suds plug-in handling some special cases while calling VI SDK.""" def add_attribute_for_value(self, node): """Helper to handle AnyType. Suds does not handle AnyType properly. But VI SDK requires type attribute to be set when AnyType is used. :param node: XML value node """ if node.name == 'value': node.set('xsi:type', 'xsd:string') def marshalled(self, context): """Modifies the envelope document before it is sent. This method provides the plug-in with the opportunity to prune empty nodes and fix nodes before sending it to the server. :param context: send context """ # Suds builds the entire request object based on the WSDL schema. # VI SDK throws server errors if optional SOAP nodes are sent # without values; e.g., as opposed to test. context.envelope.prune() context.envelope.walk(self.add_attribute_for_value) class Vim(object): """VIM API Client.""" def __init__(self, protocol='https', host='localhost', wsdl_loc=None): """Create communication interfaces for initiating SOAP transactions. :param protocol: http or https :param host: server IP address[:port] or host name[:port] :param wsdl_loc: WSDL file location :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ if not wsdl_loc: wsdl_loc = Vim._get_wsdl_loc(protocol, host) soap_url = Vim._get_soap_url(protocol, host) self._client = suds.client.Client(wsdl_loc, location=soap_url, plugins=[VimMessagePlugin()]) self._service_content = self.RetrieveServiceContent('ServiceInstance') @staticmethod def _get_wsdl_loc(protocol, host): """Get the default WSDL file location hosted at the server. :param protocol: http or https :param host: server IP address[:port] or host name[:port] :returns: default WSDL file location hosted at the server """ return '%s://%s/sdk/vimService.wsdl' % (protocol, host) @staticmethod def _get_soap_url(protocol, host): """Get ESX/VC server's SOAP service URL. :param protocol: http or https :param host: server IP address[:port] or host name[:port] :returns: URL of ESX/VC server's SOAP service """ return '%s://%s/sdk' % (protocol, host) @property def service_content(self): return self._service_content @property def client(self): return self._client @staticmethod def _retrieve_properties_ex_fault_checker(response): """Checks the RetrievePropertiesEx API response for errors. Certain faults are sent in the SOAP body as a property of missingSet. This method raises VimFaultException when a fault is found in the response. :param response: response from RetrievePropertiesEx API call :raises: VimFaultException """ LOG.debug(_("Checking RetrievePropertiesEx API response for faults.")) fault_list = [] if not response: # This is the case when the session has timed out. ESX SOAP # server sends an empty RetrievePropertiesExResponse. Normally # missingSet in the response objects has the specifics about # the error, but that's not the case with a timed out idle # session. It is as bad as a terminated session for we cannot # use the session. Therefore setting fault to NotAuthenticated # fault. LOG.debug(_("RetrievePropertiesEx API response is empty; setting " "fault to %s."), exceptions.NOT_AUTHENTICATED) fault_list = [exceptions.NOT_AUTHENTICATED] else: for obj_cont in response.objects: if hasattr(obj_cont, 'missingSet'): for missing_elem in obj_cont.missingSet: fault_type = missing_elem.fault.fault.__class__ fault_list.append(fault_type.__name__) if fault_list: LOG.error(_("Faults %s found in RetrievePropertiesEx API " "response."), fault_list) raise exceptions.VimFaultException(fault_list, _("Error occurred while calling" " RetrievePropertiesEx.")) LOG.debug(_("No faults found in RetrievePropertiesEx API response.")) def __getattr__(self, attr_name): """Returns the method to invoke API identified by param attr_name.""" def vim_request_handler(managed_object, **kwargs): """Handler for VIM API calls. Invokes the API and parses the response for fault checking and other errors. :param managed_object: managed object reference argument of the API call :param kwargs: keyword arguments of the API call :returns: response of the API call :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ try: if isinstance(managed_object, str): # For strings, use string value for value and type # of the managed object. managed_object = vim_util.get_moref(managed_object, managed_object) request = getattr(self.client.service, attr_name) LOG.debug(_("Invoking %(attr_name)s on %(moref)s."), {'attr_name': attr_name, 'moref': managed_object}) response = request(managed_object, **kwargs) if (attr_name.lower() == 'retrievepropertiesex'): Vim._retrieve_properties_ex_fault_checker(response) LOG.debug(_("Invocation of %(attr_name)s on %(moref)s " "completed successfully."), {'attr_name': attr_name, 'moref': managed_object}) return response except exceptions.VimFaultException: # Catch the VimFaultException that is raised by the fault # check of the SOAP response. raise except suds.WebFault as excep: doc = excep.document detail = doc.childAtPath('/Envelope/Body/Fault/detail') fault_list = [] if detail: for child in detail.getChildren(): fault_list.append(child.get('type')) raise exceptions.VimFaultException( fault_list, _("Web fault in %s.") % attr_name, excep) except AttributeError as excep: raise exceptions.VimAttributeException( _("No such SOAP method %s.") % attr_name, excep) except (httplib.CannotSendRequest, httplib.ResponseNotReady, httplib.CannotSendHeader) as excep: raise exceptions.VimSessionOverLoadException( _("httplib error in %s.") % attr_name, excep) except (urllib2.URLError, urllib2.HTTPError) as excep: raise exceptions.VimConnectionException( _("urllib2 error in %s.") % attr_name, excep) except Exception as excep: # TODO(vbala) should catch specific exceptions and raise # appropriate VimExceptions. # Socket errors which need special handling; some of these # might be caused by server API call overload. if (unicode(excep).find(ADDRESS_IN_USE_ERROR) != -1 or unicode(excep).find(CONN_ABORT_ERROR)) != -1: raise exceptions.VimSessionOverLoadException( _("Socket error in %s.") % attr_name, excep) # Type error which needs special handling; it might be caused # by server API call overload. elif unicode(excep).find(RESP_NOT_XML_ERROR) != -1: raise exceptions.VimSessionOverLoadException( _("Type error in %s.") % attr_name, excep) else: raise exceptions.VimException( _("Exception in %s.") % attr_name, excep) return vim_request_handler def __repr__(self): return "VIM Object." def __str__(self): return "VIM Object." python-oslo.vmware-0.2/oslo/vmware/image_transfer.py0000664000175000017500000005151712303705326021753 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Functions and classes for image transfer between ESX/VC & image service. """ import errno import logging from eventlet import event from eventlet import greenthread from eventlet import queue from eventlet import timeout from oslo.vmware import exceptions from oslo.vmware.openstack.common.gettextutils import _ from oslo.vmware import rw_handles LOG = logging.getLogger(__name__) IMAGE_SERVICE_POLL_INTERVAL = 5 FILE_READ_WRITE_TASK_SLEEP_TIME = 0.01 BLOCKING_QUEUE_SIZE = 10 class BlockingQueue(queue.LightQueue): """Producer-Consumer queue to share data between reader/writer threads.""" def __init__(self, max_size, max_transfer_size): """Initializes the queue with the given parameters. :param max_size: maximum queue size; if max_size is less than zero or None, the queue size is infinite. :param _max_transfer_size: maximum amount of data that can be _transferred using this queue """ queue.LightQueue.__init__(self, max_size) self._max_transfer_size = max_transfer_size self._transferred = 0 def read(self, chunk_size): """Read data from the queue. This method blocks until data is available. The input chunk size is ignored since we have ensured that the data chunks written to the pipe by the image reader thread is the same as the chunks asked for by the image writer thread. """ if (self._max_transfer_size is 0 or self._transferred < self._max_transfer_size): data_item = self.get() self._transferred += len(data_item) return data_item else: LOG.debug(_("Completed transfer of size %s."), self._transferred) return "" def write(self, data): """Write data into the queue. :param data: data to be written """ LOG.debug(_("Writing %d data items into the queue."), len(data)) self.put(data) # Below methods are provided in order to enable treating the queue # as a file handle. # Note(vui): When file transfer size is not specified, we raise IOError to # prevent incorrect predetermination of file length by readers. def seek(self, offset, whence=0): if self._max_transfer_size is 0: raise IOError(errno.ESPIPE, "Illegal seek") def tell(self): """Get size of the file to be read. We interpret _max_transfer_size=0 as stream mode and raise IOError to prevent incorrect predetermination of file length by readers. """ if self._max_transfer_size is 0: raise IOError(errno.ESPIPE, "Illegal seek") return self._max_transfer_size def close(self): pass def __str__(self): return "blocking queue" class ImageWriter(object): """Class to write the image to the image service from an input file.""" def __init__(self, context, input_file, image_service, image_id, image_meta=None): """Initializes the image writer instance with given parameters. :param context: write context needed by the image service :param input_file: file to read the image data from :param image_service: handle to image service :param image_id: ID of the image in the image service :param image_meta: image meta-data """ if not image_meta: image_meta = {} self._context = context self._input_file = input_file self._image_service = image_service self._image_id = image_id self._image_meta = image_meta self._running = False def start(self): """Start the image write task. :returns: the event indicating the status of the write task """ self._done = event.Event() def _inner(): """Task performing the image write operation. This method performs image data transfer through an update call. After the update, it waits until the image state becomes 'active', 'killed' or unknown. If the final state is not 'active' an instance of ImageTransferException is thrown. :raises: ImageTransferException """ LOG.debug(_("Calling image service update on image: %(image)s " "with meta: %(meta)s"), {'image': self._image_id, 'meta': self._image_meta}) try: self._image_service.update(self._context, self._image_id, self._image_meta, data=self._input_file) self._running = True while self._running: LOG.debug(_("Retrieving status of image: %s."), self._image_id) image_meta = self._image_service.show(self._context, self._image_id) image_status = image_meta.get('status') if image_status == 'active': self.stop() LOG.debug(_("Image: %s is now active."), self._image_id) self._done.send(True) elif image_status == 'killed': self.stop() excep_msg = (_("Image: %s is in killed state.") % self._image_id) LOG.error(excep_msg) excep = exceptions.ImageTransferException(excep_msg) self._done.send_exception(excep) elif image_status in ['saving', 'queued']: LOG.debug(_("Image: %(image)s is in %(state)s state; " "sleeping for %(sleep)d seconds."), {'image': self._image_id, 'state': image_status, 'sleep': IMAGE_SERVICE_POLL_INTERVAL}) greenthread.sleep(IMAGE_SERVICE_POLL_INTERVAL) else: self.stop() excep_msg = _("Image: %(image)s is in unknown state: " "%(state)s.") % {'image': self._image_id, 'state': image_status} LOG.error(excep_msg) excep = exceptions.ImageTransferException(excep_msg) self._done.send_exception(excep) except Exception as excep: self.stop() excep_msg = (_("Error occurred while writing image: %s") % self._image_id) LOG.exception(excep_msg) excep = exceptions.ImageTransferException(excep_msg, excep) self._done.send_exception(excep) LOG.debug(_("Starting image write task for image: %(image)s with" " source: %(source)s."), {'source': self._input_file, 'image': self._image_id}) greenthread.spawn(_inner) return self._done def stop(self): """Stop the image writing task.""" LOG.debug(_("Stopping the writing task for image: %s."), self._image_id) self._running = False def wait(self): """Wait for the image writer task to complete. This method returns True if the writer thread completes successfully. In case of error, it raises ImageTransferException. :raises ImageTransferException """ return self._done.wait() def close(self): """This is a NOP.""" pass def __str__(self): string = "Image Writer " % (self._input_file, self._image_id) return string class FileReadWriteTask(object): """Task which reads data from the input file and writes to the output file. This class defines the task which copies the given input file to the given output file. The copy operation involves reading chunks of data from the input file and writing the same to the output file. """ def __init__(self, input_file, output_file): """Initializes the read-write task with the given input parameters. :param input_file: the input file handle :param output_file: the output file handle """ self._input_file = input_file self._output_file = output_file self._running = False def start(self): """Start the file read - file write task. :returns: the event indicating the status of the read-write task """ self._done = event.Event() def _inner(): """Task performing the file read-write operation.""" self._running = True while self._running: try: data = self._input_file.read(None) if not data: LOG.debug(_("File read-write task is done.")) self.stop() self._done.send(True) self._output_file.write(data) # update lease progress if applicable if hasattr(self._input_file, "update_progress"): self._input_file.update_progress() if hasattr(self._output_file, "update_progress"): self._output_file.update_progress() greenthread.sleep(FILE_READ_WRITE_TASK_SLEEP_TIME) except Exception as excep: self.stop() excep_msg = _("Error occurred during file read-write " "task.") LOG.exception(excep_msg) excep = exceptions.ImageTransferException(excep_msg, excep) self._done.send_exception(excep) LOG.debug(_("Starting file read-write task with source: %(source)s " "and destination: %(dest)s."), {'source': self._input_file, 'dest': self._output_file}) greenthread.spawn(_inner) return self._done def stop(self): """Stop the read-write task.""" LOG.debug(_("Stopping the file read-write task.")) self._running = False def wait(self): """Wait for the file read-write task to complete. This method returns True if the read-write thread completes successfully. In case of error, it raises ImageTransferException. :raises: ImageTransferException """ return self._done.wait() def __str__(self): string = ("File Read-Write Task " % (self._input_file, self._output_file)) return string # Functions to perform image transfer between VMware servers and image service. def _start_transfer(context, timeout_secs, read_file_handle, max_data_size, write_file_handle=None, image_service=None, image_id=None, image_meta=None): """Start the image transfer. The image reader reads the data from the image source and writes to the blocking queue. The image source is always a file handle (VmdkReadHandle or ImageReadHandle); therefore, a FileReadWriteTask is created for this transfer. The image writer reads the data from the blocking queue and writes it to the image destination. The image destination is either a file or VMDK in VMware datastore or an image in the image service. If the destination is a file or VMDK in VMware datastore, the method creates a FileReadWriteTask which reads from the blocking queue and writes to either FileWriteHandle or VmdkWriteHandle. In the case of image service as the destination, an instance of ImageWriter task is created which reads from the blocking queue and writes to the image service. :param context: write context needed for the image service :param timeout_secs: time in seconds to wait for the transfer to complete :param read_file_handle: handle to read data from :param max_data_size: maximum transfer size :param write_file_handle: handle to write data to; if this is None, then param image_service and param image_id should be set. :param image_service: image service handle :param image_id: ID of the image in the image service :param image_meta: image meta-data :raises: ImageTransferException, ValueError """ # Create the blocking queue blocking_queue = BlockingQueue(BLOCKING_QUEUE_SIZE, max_data_size) # Create the image reader reader = FileReadWriteTask(read_file_handle, blocking_queue) # Create the image writer if write_file_handle: # File or VMDK in VMware datastore is the image destination writer = FileReadWriteTask(blocking_queue, write_file_handle) elif image_service and image_id: # Image service image is the destination writer = ImageWriter(context, blocking_queue, image_service, image_id, image_meta) else: excep_msg = _("No image destination given.") LOG.error(excep_msg) raise ValueError(excep_msg) # Start the reader and writer LOG.debug(_("Starting image transfer with reader: %(reader)s and writer: " "%(writer)s"), {'reader': reader, 'writer': writer}) reader.start() writer.start() timer = timeout.Timeout(timeout_secs) try: # Wait for the reader and writer to complete reader.wait() writer.wait() except (timeout.Timeout, exceptions.ImageTransferException) as excep: excep_msg = (_("Error occurred during image transfer with reader: " "%(reader)s and writer: %(writer)s") % {'reader': reader, 'writer': writer}) LOG.exception(excep_msg) reader.stop() writer.stop() if isinstance(excep, exceptions.ImageTransferException): raise raise exceptions.ImageTransferException(excep_msg, excep) finally: timer.cancel() read_file_handle.close() if write_file_handle: write_file_handle.close() def download_flat_image(context, timeout_secs, image_service, image_id, **kwargs): """Download flat image from the image service to VMware server. :param context: image service write context :param timeout_secs: time in seconds to wait for the download to complete :param image_service: image service handle :param image_id: ID of the image to be downloaded :param kwargs: keyword arguments to configure the destination file write handle :raises: VimConnectionException, ImageTransferException, ValueError """ LOG.debug(_("Downloading image: %s from image service as a flat file."), image_id) # TODO(vbala) catch specific exceptions raised by download call read_iter = image_service.download(context, image_id) read_handle = rw_handles.ImageReadHandle(read_iter) file_size = int(kwargs.get('image_size')) write_handle = rw_handles.FileWriteHandle(kwargs.get('host'), kwargs.get('data_center_name'), kwargs.get('datastore_name'), kwargs.get('cookies'), kwargs.get('file_path'), file_size) _start_transfer(context, timeout_secs, read_handle, file_size, write_file_handle=write_handle) LOG.debug(_("Downloaded image: %s from image service as a flat file."), image_id) def download_stream_optimized_image(context, timeout_secs, image_service, image_id, **kwargs): """Download stream optimized image from image service to VMware server. :param context: image service write context :param timeout_secs: time in seconds to wait for the download to complete :param image_service: image service handle :param image_id: ID of the image to be downloaded :param kwargs: keyword arguments to configure the destination VMDK write handle :returns: managed object reference of the VM created for import to VMware server :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException, ImageTransferException, ValueError """ LOG.debug(_("Downloading image: %s from image service as a stream " "optimized file."), image_id) # TODO(vbala) catch specific exceptions raised by download call read_iter = image_service.download(context, image_id) read_handle = rw_handles.ImageReadHandle(read_iter) file_size = int(kwargs.get('image_size')) write_handle = rw_handles.VmdkWriteHandle(kwargs.get('session'), kwargs.get('host'), kwargs.get('resource_pool'), kwargs.get('vm_folder'), kwargs.get('vm_import_spec'), file_size) _start_transfer(context, timeout_secs, read_handle, file_size, write_file_handle=write_handle) LOG.debug(_("Downloaded image: %s from image service as a stream " "optimized file."), image_id) return write_handle.get_imported_vm() def upload_image(context, timeout_secs, image_service, image_id, owner_id, **kwargs): """Upload the VM's disk file to image service. :param context: image service write context :param timeout_secs: time in seconds to wait for the upload to complete :param image_service: image service handle :param image_id: upload destination image ID :param kwargs: keyword arguments to configure the source VMDK read handle :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException, ImageTransferException, ValueError """ LOG.debug(_("Uploading to image: %s."), image_id) file_size = kwargs.get('vmdk_size') read_handle = rw_handles.VmdkReadHandle(kwargs.get('session'), kwargs.get('host'), kwargs.get('vm'), kwargs.get('vmdk_file_path'), file_size) # Set the image properties. It is important to set the 'size' to 0. # Otherwise, the image service client will use the VM's disk capacity # which will not be the image size after upload, since it is converted # to a stream-optimized sparse disk. image_metadata = {'disk_format': 'vmdk', 'is_public': kwargs.get('is_public'), 'name': kwargs.get('image_name'), 'status': 'active', 'container_format': 'bare', 'size': 0, 'properties': {'vmware_image_version': kwargs.get('image_version'), 'vmware_disktype': 'streamOptimized', 'owner_id': owner_id}} # Passing 0 as the file size since data size to be transferred cannot be # predetermined. _start_transfer(context, timeout_secs, read_handle, 0, image_service=image_service, image_id=image_id, image_meta=image_metadata) LOG.debug(_("Uploaded image: %s."), image_id) python-oslo.vmware-0.2/oslo/vmware/rw_handles.py0000664000175000017500000006411112303705326021105 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Classes defining read and write handles for image transfer. This module defines various classes for reading and writing files including VMDK files in VMware servers. It also contains a class to read images from glance server. """ import httplib import logging import socket import urllib import urllib2 import urlparse import netaddr from oslo.vmware import exceptions from oslo.vmware.openstack.common.gettextutils import _ from oslo.vmware import vim_util LOG = logging.getLogger(__name__) READ_CHUNKSIZE = 65536 USER_AGENT = 'OpenStack-ESX-Adapter' class FileHandle(object): """Base class for VMware server file (including VMDK) access over HTTP. This class wraps a backing file handle and provides utility methods for various sub-classes. """ def __init__(self, file_handle): """Initializes the file handle. :param _file_handle: backing file handle """ self._eof = False self._file_handle = file_handle def close(self): """Close the file handle.""" try: self._file_handle.close() except Exception: LOG.warn(_("Error occurred while closing the file handle"), exc_info=True) def __del__(self): """Close the file handle on garbage collection.""" self.close() def _build_vim_cookie_header(self, vim_cookies): """Build ESX host session cookie header.""" cookie_header = "" for vim_cookie in vim_cookies: cookie_header = vim_cookie.name + '=' + vim_cookie.value break return cookie_header def write(self, data): """Write data to the file. :param data: data to be written :raises: NotImplementedError """ raise NotImplementedError() def read(self, chunk_size): """Read a chunk of data. :param chunk_size: read chunk size :raises: NotImplementedError """ raise NotImplementedError() def get_size(self): """Get size of the file to be read. :raises: NotImplementedError """ raise NotImplementedError() def _is_valid_ipv6(self, address): """Checks whether the given host address is a valid IPv6 address.""" try: return netaddr.valid_ipv6(address) except Exception: return False def _get_soap_url(self, scheme, host): """Returns the IPv4/v6 compatible SOAP URL for the given host.""" if self._is_valid_ipv6(host): return '%s://[%s]' % (scheme, host) return '%s://%s' % (scheme, host) def _fix_esx_url(self, url, host): """Fix netloc in the case of an ESX host. In the case of an ESX host, the netloc is set to '*' in the URL returned in HttpNfcLeaseInfo. It should be replaced with host name or IP address. """ urlp = urlparse.urlparse(url) if urlp.netloc == '*': scheme, netloc, path, params, query, fragment = urlp url = urlparse.urlunparse((scheme, host, path, params, query, fragment)) return url def _find_vmdk_url(self, lease_info, host): """Find the URL corresponding to a VMDK file in lease info.""" LOG.debug(_("Finding VMDK URL from lease info.")) url = None for deviceUrl in lease_info.deviceUrl: if deviceUrl.disk: url = self._fix_esx_url(deviceUrl.url, host) break if not url: excep_msg = _("Could not retrieve VMDK URL from lease info.") LOG.error(excep_msg) raise exceptions.VimException(excep_msg) LOG.debug(_("Found VMDK URL: %s from lease info."), url) return url class FileWriteHandle(FileHandle): """Write handle for a file in VMware server.""" def __init__(self, host, data_center_name, datastore_name, cookies, file_path, file_size, scheme='https'): """Initializes the write handle with given parameters. :param host: ESX/VC server IP address[:port] or host name[:port] :param data_center_name: name of the data center in the case of a VC server :param datastore_name: name of the datastore where the file is stored :param cookies: cookies to build the vim cookie header :param file_path: datastore path where the file is written :param file_size: size of the file in bytes :param scheme: protocol-- http or https :raises: VimConnectionException, ValueError """ soap_url = self._get_soap_url(scheme, host) param_list = {'dcPath': data_center_name, 'dsName': datastore_name} self._url = '%s/folder/%s' % (soap_url, file_path) self._url = self._url + '?' + urllib.urlencode(param_list) self.conn = self._create_connection(self._url, file_size, cookies) FileHandle.__init__(self, self.conn) def _create_connection(self, url, file_size, cookies): """Create HTTP connection to write to the file with given URL.""" LOG.debug(_("Creating HTTP connection to write to file with " "size = %(file_size)d and URL = %(url)s."), {'file_size': file_size, 'url': url}) _urlparse = urlparse.urlparse(url) scheme, netloc, path, params, query, fragment = _urlparse try: if scheme == 'http': conn = httplib.HTTPConnection(netloc) elif scheme == 'https': conn = httplib.HTTPSConnection(netloc) else: excep_msg = _("Invalid scheme: %s.") % scheme LOG.error(excep_msg) raise ValueError(excep_msg) conn.putrequest('PUT', path + '?' + query) conn.putheader('User-Agent', USER_AGENT) conn.putheader('Content-Length', file_size) conn.putheader('Cookie', self._build_vim_cookie_header(cookies)) conn.endheaders() LOG.debug(_("Created HTTP connection to write to file with " "URL = %s."), url) return conn except (httplib.InvalidURL, httplib.CannotSendRequest, httplib.CannotSendHeader) as excep: excep_msg = _("Error occurred while creating HTTP connection " "to write to file with URL = %s.") % url LOG.exception(excep_msg) raise exceptions.VimConnectionException(excep_msg, excep) def write(self, data): """Write data to the file. :param data: data to be written :raises: VimConnectionException, VimException """ LOG.debug(_("Writing data to %s."), self._url) try: self._file_handle.send(data) except (socket.error, httplib.NotConnected) as excep: excep_msg = _("Connection error occurred while writing data to" " %s.") % self._url LOG.exception(excep_msg) raise exceptions.VimConnectionException(excep_msg, excep) except Exception as excep: # TODO(vbala) We need to catch and raise specific exceptions # related to connection problems, invalid request and invalid # arguments. excep_msg = _("Error occurred while writing data to" " %s.") % self._url LOG.exception(excep_msg) raise exceptions.VimException(excep_msg, excep) def close(self): """Get the response and close the connection.""" LOG.debug(_("Closing write handle for %s."), self._url) try: self.conn.getresponse() except Exception: LOG.warn(_("Error occurred while reading the HTTP response."), exc_info=True) super(FileWriteHandle, self).close() LOG.debug(_("Closed write handle for %s."), self._url) def __str__(self): return "File write handle for %s" % self._url class VmdkWriteHandle(FileHandle): """VMDK write handle based on HttpNfcLease. This class creates a vApp in the specified resource pool and uploads the virtual disk contents. """ def __init__(self, session, host, rp_ref, vm_folder_ref, import_spec, vmdk_size): """Initializes the VMDK write handle with input parameters. :param session: valid API session to ESX/VC server :param host: ESX/VC server IP address[:port] or host name[:port] :param rp_ref: resource pool into which the backing VM is imported :param vm_folder_ref: VM folder in ESX/VC inventory to use as parent of backing VM :param import_spec: import specification of the backing VM :param vmdk_size: size of the backing VM's VMDK file :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException, ValueError """ self._session = session self._vmdk_size = vmdk_size self._bytes_written = 0 # Get lease and its info for vApp import self._lease = self._create_and_wait_for_lease(session, rp_ref, import_spec, vm_folder_ref) LOG.debug(_("Invoking VIM API for reading info of lease: %s."), self._lease) lease_info = session.invoke_api(vim_util, 'get_object_property', session.vim, self._lease, 'info') # Find VMDK URL where data is to be written self._url = self._find_vmdk_url(lease_info, host) self._vm_ref = lease_info.entity # Create HTTP connection to write to VMDK URL self._conn = self._create_connection(session, self._url, vmdk_size) FileHandle.__init__(self, self._conn) def get_imported_vm(self): """"Get managed object reference of the VM created for import.""" return self._vm_ref def _create_and_wait_for_lease(self, session, rp_ref, import_spec, vm_folder_ref): """Create and wait for HttpNfcLease lease for vApp import.""" LOG.debug(_("Creating HttpNfcLease lease for vApp import into resource" " pool: %s."), rp_ref) lease = session.invoke_api(session.vim, 'ImportVApp', rp_ref, spec=import_spec, folder=vm_folder_ref) LOG.debug(_("Lease: %(lease)s obtained for vApp import into resource" " pool %(rp_ref)s."), {'lease': lease, 'rp_ref': rp_ref}) session.wait_for_lease_ready(lease) return lease def _create_connection(self, session, url, vmdk_size): """Create HTTP connection to write to VMDK file.""" LOG.debug(_("Creating HTTP connection to write to VMDK file with " "size = %(vmdk_size)d and URL = %(url)s."), {'vmdk_size': vmdk_size, 'url': url}) cookies = session.vim.client.options.transport.cookiejar _urlparse = urlparse.urlparse(url) scheme, netloc, path, params, query, fragment = _urlparse try: if scheme == 'http': conn = httplib.HTTPConnection(netloc) elif scheme == 'https': conn = httplib.HTTPSConnection(netloc) else: excep_msg = _("Invalid scheme: %s.") % scheme LOG.error(excep_msg) raise ValueError(excep_msg) if query: path = path + '?' + query conn.putrequest('PUT', path) conn.putheader('User-Agent', USER_AGENT) conn.putheader('Content-Length', str(vmdk_size)) conn.putheader('Overwrite', 't') conn.putheader('Cookie', self._build_vim_cookie_header(cookies)) conn.putheader('Content-Type', 'binary/octet-stream') conn.endheaders() LOG.debug(_("Created HTTP connection to write to VMDK file with " "URL = %s."), url) return conn except (httplib.InvalidURL, httplib.CannotSendRequest, httplib.CannotSendHeader) as excep: excep_msg = _("Error occurred while creating HTTP connection " "to write to VMDK file with URL = %s.") % url LOG.exception(excep_msg) raise exceptions.VimConnectionException(excep_msg, excep) def write(self, data): """Write data to the file. :param data: data to be written :raises: VimConnectionException, VimException """ LOG.debug(_("Writing data to VMDK file with URL = %s."), self._url) try: self._file_handle.send(data) self._bytes_written += len(data) LOG.debug(_("Total %(bytes_written)d bytes written to VMDK file " "with URL = %(url)s."), {'bytes_written': self._bytes_written, 'url': self._url}) except (socket.error, httplib.NotConnected) as excep: excep_msg = _("Connection error occurred while writing data to" " %s.") % self._url LOG.exception(excep_msg) raise exceptions.VimConnectionException(excep_msg, excep) except Exception as excep: # TODO(vbala) We need to catch and raise specific exceptions # related to connection problems, invalid request and invalid # arguments. excep_msg = _("Error occurred while writing data to" " %s.") % self._url LOG.exception(excep_msg) raise exceptions.VimException(excep_msg, excep) def update_progress(self): """Updates progress to lease. This call back to the lease is essential to keep the lease alive across long running write operations. :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ percent = int(float(self._bytes_written) / self._vmdk_size * 100) LOG.debug(_("Calling VIM API to update write progress of VMDK file" " with URL = %(url)s to %(percent)d%%."), {'url': self._url, 'percent': percent}) try: self._session.invoke_api(self._session.vim, 'HttpNfcLeaseProgress', self._lease, percent=percent) LOG.debug(_("Updated write progress of VMDK file with " "URL = %(url)s to %(percent)d%%."), {'url': self._url, 'percent': percent}) except exceptions.VimException as excep: LOG.exception(_("Error occurred while updating the write progress " "of VMDK file with URL = %s."), self._url) raise excep def close(self): """Releases the lease and close the connection. :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ LOG.debug(_("Getting lease state for %s."), self._url) try: state = self._session.invoke_api(vim_util, 'get_object_property', self._session.vim, self._lease, 'state') LOG.debug(_("Lease for %(url)s is in state: %(state)s."), {'url': self._url, 'state': state}) if state == 'ready': LOG.debug(_("Releasing lease for %s."), self._url) self._session.invoke_api(self._session.vim, 'HttpNfcLeaseComplete', self._lease) LOG.debug(_("Lease for %s released."), self._url) else: LOG.debug(_("Lease for %(url)s is in state: %(state)s; no " "need to release."), {'url': self._url, 'state': state}) except exceptions.VimException: LOG.warn(_("Error occurred while releasing the lease for %s."), self._url, exc_info=True) super(VmdkWriteHandle, self).close() LOG.debug(_("Closed VMDK write handle for %s."), self._url) def __str__(self): return "VMDK write handle for %s" % self._url class VmdkReadHandle(FileHandle): """VMDK read handle based on HttpNfcLease.""" def __init__(self, session, host, vm_ref, vmdk_path, vmdk_size): """Initializes the VMDK read handle with the given parameters. During the read (export) operation, the VMDK file is converted to a stream-optimized sparse disk format. Therefore, the size of the VMDK file read may be smaller than the actual VMDK size. :param session: valid api session to ESX/VC server :param host: ESX/VC server IP address[:port] or host name[:port] :param vm_ref: managed object reference of the backing VM whose VMDK is to be exported :param vmdk_path: path of the VMDK file to be exported :param vmdk_size: actual size of the VMDK file :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ self._session = session self._vmdk_size = vmdk_size self._bytes_read = 0 # Obtain lease for VM export self._lease = self._create_and_wait_for_lease(session, vm_ref) LOG.debug(_("Invoking VIM API for reading info of lease: %s."), self._lease) lease_info = session.invoke_api(vim_util, 'get_object_property', session.vim, self._lease, 'info') # find URL of the VMDK file to be read and open connection self._url = self._find_vmdk_url(lease_info, host) self._conn = self._create_connection(session, self._url) FileHandle.__init__(self, self._conn) def _create_and_wait_for_lease(self, session, vm_ref): """Create and wait for HttpNfcLease lease for VM export.""" LOG.debug(_("Creating HttpNfcLease lease for exporting VM: %s."), vm_ref) lease = session.invoke_api(session.vim, 'ExportVm', vm_ref) LOG.debug(_("Lease: %(lease)s obtained for exporting VM: %(vm_ref)s."), {'lease': lease, 'vm_ref': vm_ref}) session.wait_for_lease_ready(lease) return lease def _create_connection(self, session, url): LOG.debug(_("Opening URL: %s for reading."), url) try: cookies = session.vim.client.options.transport.cookiejar headers = {'User-Agent': USER_AGENT, 'Cookie': self._build_vim_cookie_header(cookies)} request = urllib2.Request(url, None, headers) conn = urllib2.urlopen(request) LOG.debug(_("URL: %s opened for reading."), url) return conn except Exception as excep: # TODO(vbala) We need to catch and raise specific exceptions # related to connection problems, invalid request and invalid # arguments. excep_msg = _("Error occurred while opening URL: %s for " "reading.") % url LOG.exception(excep_msg) raise exceptions.VimException(excep_msg, excep) def read(self, chunk_size): """Read a chunk of data from the VMDK file. :param chunk_size: size of read chunk :returns: the data :raises: VimException """ LOG.debug(_("Reading data from VMDK file with URL = %s."), self._url) try: data = self._file_handle.read(READ_CHUNKSIZE) self._bytes_read += len(data) LOG.debug(_("Total %(bytes_read)d bytes read from VMDK file " "with URL = %(url)s."), {'bytes_read': self._bytes_read, 'url': self._url}) return data except Exception as excep: # TODO(vbala) We need to catch and raise specific exceptions # related to connection problems, invalid request and invalid # arguments. excep_msg = _("Error occurred while reading data from" " %s.") % self._url LOG.exception(excep_msg) raise exceptions.VimException(excep_msg, excep) def update_progress(self): """Updates progress to lease. This call back to the lease is essential to keep the lease alive across long running read operations. :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ percent = int(float(self._bytes_read) / self._vmdk_size * 100) LOG.debug(_("Calling VIM API to update read progress of VMDK file" " with URL = %(url)s to %(percent)d%%."), {'url': self._url, 'percent': percent}) try: self._session.invoke_api(self._session.vim, 'HttpNfcLeaseProgress', self._lease, percent=percent) LOG.debug(_("Updated read progress of VMDK file with " "URL = %(url)s to %(percent)d%%."), {'url': self._url, 'percent': percent}) except exceptions.VimException as excep: LOG.exception(_("Error occurred while updating the read progress " "of VMDK file with URL = %s."), self._url) raise excep def close(self): """Releases the lease and close the connection. :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ LOG.debug(_("Getting lease state for %s."), self._url) try: state = self._session.invoke_api(vim_util, 'get_object_property', self._session.vim, self._lease, 'state') LOG.debug(_("Lease for %(url)s is in state: %(state)s."), {'url': self._url, 'state': state}) if state == 'ready': LOG.debug(_("Releasing lease for %s."), self._url) self._session.invoke_api(self._session.vim, 'HttpNfcLeaseComplete', self._lease) LOG.debug(_("Lease for %s released."), self._url) else: LOG.debug(_("Lease for %(url)s is in state: %(state)s; no " "need to release."), {'url': self._url, 'state': state}) except exceptions.VimException: LOG.warn(_("Error occurred while releasing the lease for %s."), self._url, exc_info=True) raise super(VmdkReadHandle, self).close() LOG.debug(_("Closed VMDK read handle for %s."), self._url) def __str__(self): return "VMDK read handle for %s" % self._url class ImageReadHandle(object): """Read handle for glance images.""" def __init__(self, glance_read_iter): """Initializes the read handle with given parameters. :param glance_read_iter: iterator to read data from glance image """ self._glance_read_iter = glance_read_iter self._iter = self.get_next() def read(self, chunk_size): """Read an item from the image data iterator. The input chunk size is ignored since the client ImageBodyIterator uses its own chunk size. """ try: data = self._iter.next() LOG.debug(_("Read %d bytes from the image iterator."), len(data)) return data except StopIteration: LOG.debug(_("Completed reading data from the image iterator.")) return "" def get_next(self): """Get the next item from the image iterator.""" for data in self._glance_read_iter: yield data def close(self): """Close the read handle. This is a NOP. """ pass def __str__(self): return "Image read handle" python-oslo.vmware-0.2/oslo/vmware/common/0000775000175000017500000000000012303705362017672 5ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/common/__init__.py0000664000175000017500000000000012303705326021771 0ustar chuckchuckpython-oslo.vmware-0.2/oslo/vmware/common/loopingcall.py0000664000175000017500000001076212303705326022555 0ustar chuckchuck# Copyright 2010 United States Government as represented by the # Administrator of the National Aeronautics and Space Administration. # Copyright 2011 Justin Santa Barbara # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import logging import sys from eventlet import event from eventlet import greenthread from oslo.vmware.openstack.common.gettextutils import _ from oslo.vmware.openstack.common import timeutils LOG = logging.getLogger(__name__) class LoopingCallDone(Exception): """Exception to break out and stop a LoopingCall. The poll-function passed to LoopingCall can raise this exception to break out of the loop normally. This is somewhat analogous to StopIteration. An optional return-value can be included as the argument to the exception; this return-value will be returned by LoopingCall.wait() """ def __init__(self, retvalue=True): """:param retvalue: Value that LoopingCall.wait() should return.""" self.retvalue = retvalue class LoopingCallBase(object): def __init__(self, f=None, *args, **kw): self.args = args self.kw = kw self.f = f self._running = False self.done = None def stop(self): self._running = False def wait(self): return self.done.wait() class FixedIntervalLoopingCall(LoopingCallBase): """A fixed interval looping call.""" def start(self, interval, initial_delay=None): self._running = True done = event.Event() def _inner(): if initial_delay: greenthread.sleep(initial_delay) try: while self._running: start = timeutils.utcnow() self.f(*self.args, **self.kw) end = timeutils.utcnow() if not self._running: break delay = interval - timeutils.delta_seconds(start, end) if delay <= 0: LOG.warn(_('task run outlasted interval by %s sec') % -delay) greenthread.sleep(delay if delay > 0 else 0) except LoopingCallDone as e: self.stop() done.send(e.retvalue) except Exception: LOG.exception(_('in fixed duration looping call')) done.send_exception(*sys.exc_info()) return else: done.send(True) self.done = done greenthread.spawn_n(_inner) return self.done # TODO(mikal): this class name is deprecated in Havana and should be removed # in the I release LoopingCall = FixedIntervalLoopingCall class DynamicLoopingCall(LoopingCallBase): """A looping call which sleeps until the next known event. The function called should return how long to sleep for before being called again. """ def start(self, initial_delay=None, periodic_interval_max=None): self._running = True done = event.Event() def _inner(): if initial_delay: greenthread.sleep(initial_delay) try: while self._running: idle = self.f(*self.args, **self.kw) if not self._running: break if periodic_interval_max is not None: idle = min(idle, periodic_interval_max) LOG.debug(_('Dynamic looping call sleeping for %.02f ' 'seconds'), idle) greenthread.sleep(idle) except LoopingCallDone as e: self.stop() done.send(e.retvalue) except Exception: LOG.exception(_('in dynamic looping call')) done.send_exception(*sys.exc_info()) return else: done.send(True) self.done = done greenthread.spawn(_inner) return self.done python-oslo.vmware-0.2/oslo/vmware/api.py0000664000175000017500000005110112303705326017523 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Session and API call management for VMware ESX/VC server. This module contains classes to invoke VIM APIs. It supports automatic session re-establishment and retry of API invocations in case of connection problems or server API call overload. """ import logging from oslo.vmware.common import loopingcall from oslo.vmware import exceptions from oslo.vmware.openstack.common.gettextutils import _ from oslo.vmware import vim from oslo.vmware import vim_util LOG = logging.getLogger(__name__) # TODO(vbala) Move this class to excutils.py. class RetryDecorator(object): """Decorator for retrying a function upon suggested exceptions. The decorated function is retried for the given number of times, and the sleep time between the retries is incremented until max sleep time is reached. If the max retry count is set to -1, then the decorated function is invoked indefinitely until an exception is thrown, and the caught exception is not in the list of suggested exceptions. """ def __init__(self, max_retry_count=-1, inc_sleep_time=10, max_sleep_time=60, exceptions=()): """Configure the retry object using the input params. :param max_retry_count: maximum number of times the given function must be retried when one of the input 'exceptions' is caught. When set to -1, it will be retried indefinitely until an exception is thrown and the caught exception is not in param exceptions. :param inc_sleep_time: incremental time in seconds for sleep time between retries :param max_sleep_time: max sleep time in seconds beyond which the sleep time will not be incremented using param inc_sleep_time. On reaching this threshold, max_sleep_time will be used as the sleep time. :param exceptions: suggested exceptions for which the function must be retried """ self._max_retry_count = max_retry_count self._inc_sleep_time = inc_sleep_time self._max_sleep_time = max_sleep_time self._exceptions = exceptions self._retry_count = 0 self._sleep_time = 0 def __call__(self, f): def _func(*args, **kwargs): func_name = f.__name__ try: LOG.debug(_("Invoking %(func_name)s; retry count is " "%(retry_count)d."), {'func_name': func_name, 'retry_count': self._retry_count}) result = f(*args, **kwargs) LOG.debug(_("Function %(func_name)s returned successfully " "after %(retry_count)d retries."), {'func_name': func_name, 'retry_count': self._retry_count}) except self._exceptions as excep: LOG.warn(_("Exception which is in the suggested list of " "exceptions occurred while invoking function:" " %s."), func_name, exc_info=True) if (self._max_retry_count != -1 and self._retry_count >= self._max_retry_count): LOG.error(_("Cannot retry upon suggested exception since " "retry count (%(retry_count)d) reached " "max retry count (%(max_retry_count)d)."), {'retry_count': self._retry_count, 'max_retry_count': self._max_retry_count}) raise excep else: self._retry_count += 1 self._sleep_time += self._inc_sleep_time return self._sleep_time except Exception as excep: LOG.exception(_("Exception which is not in the suggested list " "of exceptions occurred while invoking %s."), func_name) raise excep raise loopingcall.LoopingCallDone(result) def func(*args, **kwargs): loop = loopingcall.DynamicLoopingCall(_func, *args, **kwargs) evt = loop.start(periodic_interval_max=self._max_sleep_time) LOG.debug(_("Waiting for function %s to return."), f.__name__) return evt.wait() return func class VMwareAPISession(object): """Setup a session with the server and handles all calls made to it. Example: api_session = VMwareAPISession('10.1.2.3', 'administrator', 'password', 10, 0.1, _create_session=False) result = api_session.invoke_api(vim_util, 'get_objects', api_session.vim, 'HostSystem', 100) """ def __init__(self, host, server_username, server_password, api_retry_count, task_poll_interval, scheme='https', create_session=True, wsdl_loc=None): """Initializes the API session with given parameters. :param host: ESX/VC server IP address[:port] or host name[:port] :param server_username: username of ESX/VC server admin user :param server_password: password for param server_username :param api_retry_count: number of times an API must be retried upon session/connection related errors :param task_poll_interval: sleep time in seconds for polling an on-going async task as part of the API call :param scheme: protocol-- http or https :param _create_session: whether to setup a connection at the time of instance creation :param wsdl_loc: WSDL file location for invoking SOAP calls on server using suds :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException """ self._host = host self._server_username = server_username self._server_password = server_password self._api_retry_count = api_retry_count self._task_poll_interval = task_poll_interval self._scheme = scheme self._wsdl_loc = wsdl_loc self._session_id = None self._session_username = None self._vim = None if create_session: self._create_session() @property def vim(self): if not self._vim: self._vim = vim.Vim(protocol=self._scheme, host=self._host, wsdl_loc=self._wsdl_loc) return self._vim @RetryDecorator(exceptions=(exceptions.VimConnectionException,)) def _create_session(self): """Establish session with the server.""" session_manager = self.vim.service_content.sessionManager # Login and create new session with the server for making API calls. LOG.debug(_("Logging in with username = %s."), self._server_username) session = self.vim.Login(session_manager, userName=self._server_username, password=self._server_password) prev_session_id, self._session_id = self._session_id, session.key # We need to save the username in the session since we may need it # later to check active session. The SessionIsActive method requires # the username parameter to be exactly same as that in the session # object. We can't use the username used for login since the Login # method ignores the case. self._session_username = session.userName LOG.info(_("Successfully established new session; session ID is %s."), self._session_id) # Terminate the previous session (if exists) for preserving sessions # as there is a limit on the number of sessions we can have. if prev_session_id: try: LOG.info(_("Terminating the previous session with ID = %s"), prev_session_id) self.vim.TerminateSession(session_manager, sessionId=[prev_session_id]) except Exception: # This exception is something we can live with. It is # just an extra caution on our side. The session might # have been cleared already. We could have made a call to # SessionIsActive, but that is an overhead because we # anyway would have to call TerminateSession. LOG.warn(_("Error occurred while terminating the previous " "session with ID = %s."), prev_session_id, exc_info=True) def __del__(self): """Log out and terminate the current session.""" if self._session_id: LOG.info(_("Logging out and terminating the current session with " "ID = %s."), self._session_id) try: self.vim.Logout(self.vim.service_content.sessionManager) except Exception: LOG.exception(_("Error occurred while logging out and " "terminating the current session with " "ID = %s."), self._session_id) else: LOG.debug(_("No session exists to log out.")) def invoke_api(self, module, method, *args, **kwargs): """Wrapper method for invoking APIs. The API call is retried in the event of exceptions due to session overload or connection problems. :param module: module corresponding to the VIM API call :param method: method in the module which corresponds to the VIM API call :param args: arguments to the method :param kwargs: keyword arguments to the method :returns: response from the API call :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ @RetryDecorator(max_retry_count=self._api_retry_count, exceptions=(exceptions.VimSessionOverLoadException, exceptions.VimConnectionException)) def _invoke_api(module, method, *args, **kwargs): LOG.debug(_("Invoking method %(module)s.%(method)s."), {'module': module, 'method': method}) try: api_method = getattr(module, method) return api_method(*args, **kwargs) except exceptions.VimFaultException as excep: # If this is due to an inactive session, we should re-create # the session and retry. if exceptions.NOT_AUTHENTICATED in excep.fault_list: # The NotAuthenticated fault is set by the fault checker # due to an empty response. An empty response could be a # valid response; for e.g., response for the query to # return the VMs in an ESX server which has no VMs in it. # Also, the server responds with an empty response in the # case of an inactive session. Therefore, we need a way to # differentiate between these two cases. if self._is_current_session_active(): LOG.debug(_("Returning empty response for " "%(module)s.%(method)s invocation."), {'module': module, 'method': method}) return [] else: # empty response is due to an inactive session excep_msg = ( _("Current session: %(session)s is inactive; " "re-creating the session while invoking " "method %(module)s.%(method)s.") % {'session': self._session_id, 'module': module, 'method': method}) LOG.warn(excep_msg, exc_info=True) self._create_session() raise exceptions.VimConnectionException(excep_msg, excep) else: # no need to retry for other VIM faults like # InvalidArgument # Raise specific exceptions here if possible if excep.fault_list: LOG.debug(_("Fault list: %s"), excep.fault_list) raise exceptions.get_fault_class(excep.fault_list[0]) raise except exceptions.VimConnectionException: # Re-create the session during connection exception. LOG.warn(_("Re-creating session due to connection problems " "while invoking method %(module)s.%(method)s."), {'module': module, 'method': method}, exc_info=True) self._create_session() raise return _invoke_api(module, method, *args, **kwargs) def _is_current_session_active(self): """Check if current session is active. :returns: True if the session is active; False otherwise """ LOG.debug(_("Checking if the current session: %s is active."), self._session_id) is_active = False try: is_active = self.vim.SessionIsActive( self.vim.service_content.sessionManager, sessionID=self._session_id, userName=self._session_username) except exceptions.VimException: LOG.warn(_("Error occurred while checking whether the " "current session: %s is active."), self._session_id, exc_info=True) return is_active def wait_for_task(self, task): """Waits for the given task to complete and returns the result. The task is polled until it is done. The method returns the task information upon successful completion. In case of any error, appropriate exception is raised. :param task: managed object reference of the task :returns: task info upon successful completion of the task :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ loop = loopingcall.FixedIntervalLoopingCall(self._poll_task, task) evt = loop.start(self._task_poll_interval) LOG.debug(_("Waiting for the task: %s to complete."), task) return evt.wait() def _poll_task(self, task): """Poll the given task until completion. If the task completes successfully, the method returns the task info using the input event (param done). In case of any error, appropriate exception is set in the event. :param task: managed object reference of the task """ LOG.debug(_("Invoking VIM API to read info of task: %s."), task) try: task_info = self.invoke_api(vim_util, 'get_object_property', self.vim, task, 'info') except exceptions.VimException as excep: LOG.exception(_("Error occurred while reading info of task: %s."), task) raise excep else: if task_info.state in ['queued', 'running']: if hasattr(task_info, 'progress'): LOG.debug(_("Task: %(task)s progress is %(progress)s%%."), {'task': task, 'progress': task_info.progress}) elif task_info.state == 'success': LOG.debug(_("Task: %s status is success."), task) raise loopingcall.LoopingCallDone(task_info) else: error_msg = unicode(task_info.error.localizedMessage) excep_msg = _("Task: %(task)s failed with error: " "%(error)s.") % {'task': task, 'error': error_msg} LOG.error(excep_msg) error = task_info.error name = error.fault.__class__.__name__ task_ex = exceptions.get_fault_class(name)(error_msg) raise task_ex def wait_for_lease_ready(self, lease): """Waits for the given lease to be ready. This method return when the lease is ready. In case of any error, appropriate exception is raised. :param lease: lease to be checked for :raises: VimException, VimFaultException, VimAttributeException, VimSessionOverLoadException, VimConnectionException """ loop = loopingcall.FixedIntervalLoopingCall(self._poll_lease, lease) evt = loop.start(self._task_poll_interval) LOG.debug(_("Waiting for the lease: %s to be ready."), lease) evt.wait() def _poll_lease(self, lease): """Poll the state of the given lease. When the lease is ready, the event (param done) is notified. In case of any error, appropriate exception is set in the event. :param lease: lease whose state is to be polled """ LOG.debug(_("Invoking VIM API to read state of lease: %s."), lease) try: state = self.invoke_api(vim_util, 'get_object_property', self.vim, lease, 'state') except exceptions.VimException as excep: LOG.exception(_("Error occurred while checking state of lease: " "%s."), lease) raise excep else: if state == 'ready': LOG.debug(_("Lease: %s is ready."), lease) raise loopingcall.LoopingCallDone() elif state == 'initializing': LOG.debug(_("Lease: %s is initializing."), lease) elif state == 'error': LOG.debug(_("Invoking VIM API to read lease: %s error."), lease) error_msg = self._get_error_message(lease) excep_msg = _("Lease: %(lease)s is in error state. Details: " "%(error_msg)s.") % {'lease': lease, 'error_msg': error_msg} LOG.error(excep_msg) raise exceptions.VimException(excep_msg) else: # unknown state excep_msg = _("Unknown state: %(state)s for lease: " "%(lease)s.") % {'state': state, 'lease': lease} LOG.error(excep_msg) raise exceptions.VimException(excep_msg) def _get_error_message(self, lease): """Get error message associated with the given lease.""" try: return self.invoke_api(vim_util, 'get_object_property', self.vim, lease, 'error') except exceptions.VimException: LOG.warn(_("Error occurred while reading error message for lease: " "%s."), lease, exc_info=True) return "Unknown" python-oslo.vmware-0.2/oslo/vmware/exceptions.py0000664000175000017500000001133512303705326021140 0ustar chuckchuck# Copyright (c) 2014 VMware, Inc. # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ Exception definitions. """ import logging import six from oslo.vmware.openstack.common.gettextutils import _ LOG = logging.getLogger(__name__) ALREADY_EXISTS = 'AlreadyExists' CANNOT_DELETE_FILE = 'CannotDeleteFile' FILE_ALREADY_EXISTS = 'FileAlreadyExists' FILE_FAULT = 'FileFault' FILE_LOCKED = 'FileLocked' FILE_NOT_FOUND = 'FileNotFound' INVALID_PROPERTY = 'InvalidProperty' NOT_AUTHENTICATED = 'NotAuthenticated' class VimException(Exception): """The base exception class for all exceptions this library raises.""" def __init__(self, message, cause=None): self.msg = message self.cause = cause def __str__(self): descr = self.msg if self.cause: descr += '\nCause: ' + str(self.cause) return descr class VimSessionOverLoadException(VimException): """Thrown when there is an API call overload at the VMware server.""" pass class VimConnectionException(VimException): """Thrown when there is a connection problem.""" pass class VimAttributeException(VimException): """Thrown when a particular attribute cannot be found.""" pass class VimFaultException(VimException): """Exception thrown when there are faults during VIM API calls.""" def __init__(self, fault_list, message, cause=None): super(VimFaultException, self).__init__(message, cause) self.fault_list = fault_list def __str__(self): descr = VimException.__str__(self) if self.fault_list: descr += '\nFaults: ' + str(self.fault_list) return descr class ImageTransferException(VimException): """Thrown when there is an error during image transfer.""" pass class VMwareDriverException(Exception): """Base VMware Driver Exception To correctly use this class, inherit from it and define a 'msg_fmt' property. That msg_fmt will get printf'd with the keyword arguments provided to the constructor. """ msg_fmt = _("An unknown exception occurred.") def __init__(self, message=None, **kwargs): self.kwargs = kwargs if not message: try: message = self.msg_fmt % kwargs except Exception: # kwargs doesn't match a variable in the message # log the issue and the kwargs LOG.exception(_('Exception in string format operation')) for name, value in six.iteritems(kwargs): LOG.error("%s: %s" % (name, value)) # at least get the core message out if something happened message = self.msg_fmt super(VMwareDriverException, self).__init__(message) class AlreadyExistsException(VMwareDriverException): msg_fmt = _("Resource already exists.") class CannotDeleteFileException(VMwareDriverException): msg_fmt = _("Cannot delete file.") class FileAlreadyExistsException(VMwareDriverException): msg_fmt = _("File already exists.") class FileFaultException(VMwareDriverException): msg_fmt = _("File fault.") class FileLockedException(VMwareDriverException): msg_fmt = _("File locked.") class FileNotFoundException(VMwareDriverException): msg_fmt = _("File not found.") class InvalidPropertyException(VMwareDriverException): msg_fmt = _("Invalid property.") class NotAuthenticatedException(VMwareDriverException): msg_fmt = _("Not Authenticated.") # Populate the fault registry with the exceptions that have # special treatment. _fault_classes_registry = { ALREADY_EXISTS: AlreadyExistsException, CANNOT_DELETE_FILE: CannotDeleteFileException, FILE_ALREADY_EXISTS: FileAlreadyExistsException, FILE_FAULT: FileFaultException, FILE_LOCKED: FileLockedException, FILE_NOT_FOUND: FileNotFoundException, INVALID_PROPERTY: InvalidPropertyException, NOT_AUTHENTICATED: NotAuthenticatedException, } def get_fault_class(name): """Get a named subclass of NovaException.""" name = str(name) fault_class = _fault_classes_registry.get(name) if not fault_class: LOG.debug(_('Fault %s not matched.'), name) fault_class = VMwareDriverException return fault_class python-oslo.vmware-0.2/oslo/__init__.py0000664000175000017500000000116512303705325017214 0ustar chuckchuck# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. __import__('pkg_resources').declare_namespace(__name__) python-oslo.vmware-0.2/AUTHORS0000664000175000017500000000000112303705362015164 0ustar chuckchuck python-oslo.vmware-0.2/ChangeLog0000664000175000017500000000105712303705362015702 0ustar chuckchuckCHANGES ======= 0.2 --- * Remove dependency on log.py from openstack/common * Remove vim header 0.1 --- * VMware: raise more specific exceptions * Move image transfer code in VMware drivers to OSLO * Move read/write handles in VMware drivers to OSLO * Move API invocation code in VMware drivers to OSLO * Move VIM API client code in VMware drivers to OSLO * Add eventlet requirement * Move utility methods in VMware drivers to OSLO * Import necessary files from openstack.common * Initial checkin for Oslo VMware Library generated using oslo-cookiecutter