pax_global_header 0000666 0000000 0000000 00000000064 13535423452 0014520 g ustar 00root root 0000000 0000000 52 comment=2d7b5f75db690fbfffe601ecc8657fb750a7977a
cs-2.7.1/ 0000775 0000000 0000000 00000000000 13535423452 0012134 5 ustar 00root root 0000000 0000000 cs-2.7.1/.gitignore 0000664 0000000 0000000 00000000100 13535423452 0014113 0 ustar 00root root 0000000 0000000 dist
build
.venv
.coverage
.eggs
.tox
*.egg-info
*.pyc
.*_cache
cs-2.7.1/.travis.yml 0000664 0000000 0000000 00000000230 13535423452 0014240 0 ustar 00root root 0000000 0000000 dist: xenial
sudo: required
language: python
python:
- 2.7
- 3.5
- 3.6
- 3.7
- 3.8-dev
install:
- pip install tox-travis
script:
- tox
cs-2.7.1/LICENSE 0000664 0000000 0000000 00000002735 13535423452 0013150 0 ustar 00root root 0000000 0000000 Copyright (c) 2014, Bruno Renié and contributors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
cs-2.7.1/MANIFEST.in 0000664 0000000 0000000 00000000052 13535423452 0013667 0 ustar 00root root 0000000 0000000 recursive-include cs *.py
include LICENSE
cs-2.7.1/README.rst 0000664 0000000 0000000 00000016303 13535423452 0013626 0 ustar 00root root 0000000 0000000 CS
==
.. image:: https://travis-ci.org/exoscale/cs.svg?branch=master
:alt: Build Status
:target: https://travis-ci.org/exoscale/cs
.. image:: https://img.shields.io/pypi/l/cs.svg
:alt: License
:target: https://pypi.org/project/cs/
.. image:: https://img.shields.io/pypi/pyversions/cs.svg
:alt: Python versions
:target: https://pypi.org/project/cs/
A simple, yet powerful CloudStack API client for python and the command-line.
* Python 2.7+ and 3.4+ support.
* Async support for Python 3.5+.
* All present and future CloudStack API calls and parameters are supported.
* Syntax highlight in the command-line client if Pygments is installed.
* BSD license.
Installation
------------
::
pip install cs
# with the colored output
pip install cs[highlight]
# with the async support (Python 3.5+)
pip install cs[async]
# with both
pip install cs[async,highlight]
Usage
-----
In Python:
.. code-block:: python
from cs import CloudStack
cs = CloudStack(endpoint='https://api.exoscale.ch/compute',
key='cloudstack api key',
secret='cloudstack api secret')
vms = cs.listVirtualMachines()
cs.createSecurityGroup(name='web', description='HTTP traffic')
From the command-line, this requires some configuration:
.. code-block:: console
cat $HOME/.cloudstack.ini
.. code-block:: ini
[cloudstack]
endpoint = https://api.exoscale.ch/compute
key = cloudstack api key
secret = cloudstack api secret
# Optional ca authority certificate
verify = /path/to/certs/exoscale_ca.crt
# Optional client PEM certificate
cert = /path/to/client_exoscale.pem
Then:
.. code-block:: console
$ cs listVirtualMachines
.. code-block:: json
{
"count": 1,
"virtualmachine": [
{
"account": "...",
...
}
]
}
.. code-block:: console
$ cs authorizeSecurityGroupIngress \
cidrlist="0.0.0.0/0" endport=443 startport=443 \
securitygroupname="blah blah" protocol=tcp
The command-line client polls when async results are returned. To disable
polling, use the ``--async`` flag.
To find the list CloudStack API calls go to
http://cloudstack.apache.org/api.html
Configuration
-------------
Configuration is read from several locations, in the following order:
* The ``CLOUDSTACK_ENDPOINT``, ``CLOUDSTACK_KEY``, ``CLOUDSTACK_SECRET`` and
``CLOUDSTACK_METHOD`` environment variables,
* A ``CLOUDSTACK_CONFIG`` environment variable pointing to an ``.ini`` file,
* A ``CLOUDSTACK_VERIFY`` (optional) environment variable pointing to a CA authority cert file,
* A ``CLOUDSTACK_CERT`` (optional) environment variable pointing to a client PEM cert file,
* A ``cloudstack.ini`` file in the current working directory,
* A ``.cloudstack.ini`` file in the home directory.
To use that configuration scheme from your Python code:
.. code-block:: python
from cs import CloudStack, read_config
cs = CloudStack(**read_config())
Note that ``read_config()`` can raise ``SystemExit`` if no configuration is
found.
``CLOUDSTACK_METHOD`` or the ``method`` entry in the configuration file can be
used to change the HTTP verb used to make CloudStack requests. By default,
requests are made with the GET method but CloudStack supports POST requests.
POST can be useful to overcome some length limits in the CloudStack API.
``CLOUDSTACK_TIMEOUT`` or the ``timeout`` entry in the configuration file can
be used to change the HTTP timeout when making CloudStack requests (in
seconds). The default value is 10.
``CLOUDSTACK_RETRY`` or the ``retry`` entry in the configuration file
(integer) can be used to retry ``list`` and ``queryAsync`` requests on
failure. The default value is 0, meaning no retry.
``CLOUDSTACK_JOB_TIMEOUT`` or the `job_timeout`` entry in the configuration file
(float) can be used to set how long an async call is retried assuming ``fetch_result`` is set to true). The default value is ``None``, it waits forever.
``CLOUDSTACK_POLL_INTERVAL`` or the ``poll_interval`` entry in the configuration file (number of seconds, float) can be used to set how frequently polling an async job result is done. The default value is 2.
``CLOUDSTACK_EXPIRATION`` or the ``expiration`` entry in the configuration file
(integer) can be used to set how long a signature is valid. By default, it picks
10 minutes but may be deactivated using any negative value, e.g. -1.
``CLOUDSTACK_DANGEROUS_NO_TLS_VERIFY`` or the ``dangerous_no_tls_verify`` entry
in the configuration file (boolean) can be used to deactivate the TLS verification
made when using the HTTPS protocol.
Multiple credentials can be set in ``.cloudstack.ini``. This allows selecting
the credentials or endpoint to use with a command-line flag.
.. code-block:: ini
[cloudstack]
endpoint = https://some-host/api/compute
key = api key
secret = api secret
[exoscale]
endpoint = https://api.exoscale.ch/compute
key = api key
secret = api secret
Usage::
$ cs listVirtualMachines --region=exoscale
Optionally ``CLOUDSTACK_REGION`` can be used to overwrite the default region ``cloudstack``.
For the power users that don't want to put any secrets on disk,
``CLOUDSTACK_OVERRIDES`` let you pick which key will be set from the
environment even if present in the ini file.
Pagination
----------
CloudStack paginates requests. ``cs`` is able to abstract away the pagination
logic to allow fetching large result sets in one go. This is done with the
``fetch_list`` parameter::
$ cs listVirtualMachines fetch_list=true
Or in Python::
cs.listVirtualMachines(fetch_list=True)
Tracing HTTP requests
---------------------
Once in a while, it could be useful to understand, see what HTTP calls are made
under the hood. The ``trace`` flag (or ``CLOUDSTACK_TRACE``) does just that::
$ cs --trace listVirtualMachines
$ cs -t listZones
Async client
------------
``cs`` provides the ``AIOCloudStack`` class for async/await calls in Python
3.5+.
.. code-block:: python
import asyncio
from cs import AIOCloudStack, read_config
cs = AIOCloudStack(**read_config())
async def main():
vms = await cs.listVirtualMachines(fetch_list=True)
print(vms)
asyncio.run(main())
Async deployment of multiple VMs
________________________________
.. code-block:: python
import asyncio
from cs import AIOCloudStack, read_config
cs = AIOCloudStack(**read_config())
machine = {"zoneid": ..., "serviceofferingid": ..., "templateid": ...}
async def main():
tasks = asyncio.gather(*(cs.deployVirtualMachine(name=f"vm-{i}",
**machine,
fetch_result=True)
for i in range(5)))
results = await tasks
# Destroy all of them, but skip waiting on the job results
await asyncio.gather(*(cs.destroyVirtualMachine(id=result['virtualmachine']['id'])
for result in results))
asyncio.run(main())
Links
-----
* CloudStack API: http://cloudstack.apache.org/api.html
* Example of use: `Get Started with the exoscale API client `_
cs-2.7.1/cs/ 0000775 0000000 0000000 00000000000 13535423452 0012541 5 ustar 00root root 0000000 0000000 cs-2.7.1/cs/__init__.py 0000664 0000000 0000000 00000010510 13535423452 0014647 0 ustar 00root root 0000000 0000000 import argparse
import json
import os
import sys
from collections import defaultdict
try:
from configparser import NoSectionError
except ImportError: # python 2
from ConfigParser import NoSectionError
try:
import pygments
from pygments.lexers import JsonLexer
from pygments.styles import get_style_by_name
from pygments.formatters import Terminal256Formatter
except ImportError:
pygments = None
from .client import read_config, CloudStack, CloudStackApiException, CloudStackException # noqa
__all__ = ['read_config', 'CloudStack', 'CloudStackException']
if sys.version_info >= (3, 5):
try:
import aiohttp # noqa
except ImportError:
pass
else:
from ._async import AIOCloudStack # noqa
__all__.append('AIOCloudStack')
def _format_json(data, theme):
"""Pretty print a dict as a JSON, with colors if pygments is present."""
output = json.dumps(data, indent=2, sort_keys=True)
if pygments and sys.stdout.isatty():
style = get_style_by_name(theme)
formatter = Terminal256Formatter(style=style)
return pygments.highlight(output, JsonLexer(), formatter)
return output
def main(args=None):
parser = argparse.ArgumentParser(description='Cloustack client.')
parser.add_argument('--region', '-r', metavar='REGION',
help='Cloudstack region in ~/.cloudstack.ini',
default=os.environ.get('CLOUDSTACK_REGION',
'cloudstack'))
parser.add_argument('--theme', metavar='THEME',
help='Pygments style',
default=os.environ.get('CLOUDSTACK_THEME',
'default'))
parser.add_argument('--post', action='store_true', default=False,
help='use POST instead of GET')
parser.add_argument('--async', action='store_true', default=False,
help='do not wait for async result')
parser.add_argument('--quiet', '-q', action='store_true', default=False,
help='do not display additional status messages')
parser.add_argument('--trace', '-t', action='store_true',
default=os.environ.get('CLOUDSTACK_TRACE', False),
help='trace the HTTP requests done on stderr')
parser.add_argument('command', metavar="COMMAND",
help='Cloudstack API command to execute')
def parse_option(x):
if '=' not in x:
raise ValueError("{!r} is not a correctly formatted "
"option".format(x))
return x.split('=', 1)
parser.add_argument('arguments', metavar="OPTION=VALUE",
nargs='*', type=parse_option,
help='Cloudstack API argument')
options = parser.parse_args(args=args)
command = options.command
kwargs = defaultdict(set)
for arg in options.arguments:
key, value = arg
kwargs[key].add(value.strip(" \"'"))
try:
config = read_config(ini_group=options.region)
except NoSectionError:
raise SystemExit("Error: region '%s' not in config" % options.region)
theme = config.pop('theme', 'default')
fetch_result = 'Async' not in command and not getattr(options, 'async')
if options.post:
config['method'] = 'post'
if options.trace:
config['trace'] = True
cs = CloudStack(**config)
ok = True
response = None
try:
response = getattr(cs, command)(fetch_result=fetch_result,
**kwargs)
except CloudStackException as e:
ok = False
if e.response is not None:
if not options.quiet:
sys.stderr.write("CloudStack error: ")
sys.stderr.write("\n".join((str(arg) for arg in e.args)))
sys.stderr.write("\n")
try:
response = json.loads(e.response.text)
except ValueError:
sys.stderr.write(e.response.text)
sys.stderr.write("\n")
else:
message, data = (e.args[0], e.args[0:])
sys.stderr.write("Error: {0}\n{1}\n".format(message, data))
if response:
sys.stdout.write(_format_json(response, theme=theme))
sys.stdout.write('\n')
return not ok
cs-2.7.1/cs/__main__.py 0000664 0000000 0000000 00000000073 13535423452 0014633 0 ustar 00root root 0000000 0000000 from . import main
if __name__ == '__main__':
main()
cs-2.7.1/cs/_async.py 0000664 0000000 0000000 00000011032 13535423452 0014364 0 ustar 00root root 0000000 0000000 import asyncio
import ssl
import aiohttp
from . import CloudStack, CloudStackApiException, CloudStackException
from .client import PENDING, SUCCESS, transform
class AIOCloudStack(CloudStack):
def __getattr__(self, command):
def handler(**kwargs):
return self._request(command, **kwargs)
return handler
async def _request(self, command, json=True, opcode_name='command',
fetch_list=False, headers=None, **params):
fetch_result = params.pop("fetch_result", self.fetch_result)
kwarg, kwargs = self._prepare_request(command, json, opcode_name,
fetch_list, **params)
ssl_context = None
if self.cert:
ssl_context = ssl.create_default_context(cafile=self.cert)
connector = aiohttp.TCPConnector(verify_ssl=self.verify,
ssl_context=ssl_context)
async with aiohttp.ClientSession(read_timeout=self.timeout,
conn_timeout=self.timeout,
connector=connector) as session:
handler = getattr(session, self.method)
done = False
final_data = []
page = 1
while not done:
if fetch_list:
kwargs['page'] = page
transform(kwargs)
kwargs.pop('signature', None)
self._sign(kwargs)
response = await handler(self.endpoint,
headers=headers,
**{kwarg: kwargs})
ctype = response.headers['content-type'].split(';')[0]
try:
data = await response.json(content_type=ctype)
except ValueError as e:
msg = "Make sure endpoint URL {!r} is correct.".format(
self.endpoint)
raise CloudStackException(
"HTTP {0} response from CloudStack".format(
response.status),
"{}. {}".format(e, msg),
response=response,
)
[key] = data.keys()
data = data[key]
if response.status != 200:
raise CloudStackApiException(
"HTTP {0} response from CloudStack".format(
response.status), error=data, response=response)
if fetch_list:
try:
[key] = [k for k in data.keys() if k != 'count']
except ValueError:
done = True
else:
final_data.extend(data[key])
page += 1
elif fetch_result and 'jobid' in data:
try:
final_data = await asyncio.wait_for(
self._jobresult(data['jobid'], response),
self.job_timeout)
except asyncio.TimeoutError:
raise CloudStackException(
"Timeout waiting for async job result",
data['jobid'],
response=response)
done = True
else:
final_data = data
done = True
return final_data
async def _jobresult(self, jobid, response):
failures = 0
while True:
try:
j = await self.queryAsyncJobResult(jobid=jobid,
fetch_result=False)
failures = 0
if j['jobstatus'] != PENDING:
if j['jobresultcode'] != 0 or j['jobstatus'] != SUCCESS:
raise CloudStackApiException("Job failure", j,
error=j['jobresult'],
response=response)
if 'jobresult' not in j:
raise CloudStackException("Unknown job result", j,
response=response)
return j['jobresult']
except CloudStackException:
raise
except Exception:
failures += 1
if failures > 10:
raise
await asyncio.sleep(self.poll_interval)
cs-2.7.1/cs/client.py 0000664 0000000 0000000 00000043420 13535423452 0014374 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
from __future__ import print_function
import base64
import hashlib
import hmac
import os
import re
import sys
import time
from datetime import datetime, timedelta
from fnmatch import fnmatch
try:
from configparser import ConfigParser
except ImportError: # python 2
from ConfigParser import ConfigParser
try:
from urllib.parse import quote
except ImportError: # python 2
from urllib import quote
import pytz
import requests
from requests.structures import CaseInsensitiveDict
PY2 = sys.version_info < (3, 0)
if PY2:
text_type = unicode # noqa
string_type = basestring # noqa
integer_types = int, long # noqa
binary_type = str
else:
text_type = str
string_type = str
integer_types = int
binary_type = bytes
if sys.version_info >= (3, 5):
try:
from . import AIOCloudStack # noqa
except ImportError:
pass
TIMEOUT = 10
PAGE_SIZE = 500
POLL_INTERVAL = 2.0
EXPIRATION = timedelta(minutes=10)
EXPIRES_FORMAT = "%Y-%m-%dT%H:%M:%S%z"
REQUIRED_CONFIG_KEYS = {"endpoint", "key", "secret", "method", "timeout"}
ALLOWED_CONFIG_KEYS = {"verify", "cert", "retry", "theme", "expiration",
"poll_interval", "trace", "dangerous_no_tls_verify",
"header_*"}
DEFAULT_CONFIG = {
"timeout": 10,
"method": "get",
"retry": 0,
"verify": None,
"cert": None,
"name": None,
"expiration": 600,
"poll_interval": POLL_INTERVAL,
"trace": None,
"dangerous_no_tls_verify": False,
}
PENDING = 0
SUCCESS = 1
FAILURE = 2
def strtobool(val):
"""Convert a string representation of truth to true (1) or false (0).
True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values
are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if
'val' is anything else.
This function has been borrowed from distutils.util module in order
to avoid pulling a dependency on deprecated module "imp".
"""
val = val.lower()
if val in ('y', 'yes', 't', 'true', 'on', '1'):
return 1
elif val in ('n', 'no', 'f', 'false', 'off', '0'):
return 0
else:
raise ValueError("invalid truth value %r" % (val,))
def check_key(key, allowed):
"""
Validate that the specified key is allowed according the provided
list of patterns.
"""
if key in allowed:
return True
for pattern in allowed:
if fnmatch(key, pattern):
return True
return False
def cs_encode(s):
"""Encode URI component like CloudStack would do before signing.
java.net.URLEncoder.encode(s).replace('+', '%20')
"""
if PY2 and isinstance(s, text_type):
s = s.encode("utf-8")
return quote(s, safe="*")
def transform(params):
"""
Transforms an heterogeneous map of params into a CloudStack
ready mapping of parameter to values.
It handles lists and dicts.
>>> p = {"a": 1, "b": "foo", "c": ["eggs", "spam"], "d": {"key": "value"}}
>>> transform(p)
>>> print(p)
{'a': '1', 'b': 'foo', 'c': 'eggs,spam', 'd[0].key': 'value'}
"""
for key, value in list(params.items()):
if value is None:
params.pop(key)
continue
if isinstance(value, (string_type, binary_type)):
continue
if isinstance(value, integer_types):
params[key] = text_type(value)
elif isinstance(value, (list, tuple, set, dict)):
if not value:
params.pop(key)
else:
if isinstance(value, dict):
value = [value]
if isinstance(value, set):
value = list(value)
if not isinstance(value[0], dict):
params[key] = ",".join(value)
else:
params.pop(key)
for index, val in enumerate(value):
for name, v in val.items():
k = "%s[%d].%s" % (key, index, name)
params[k] = text_type(v)
else:
raise ValueError(type(value))
class CloudStackException(Exception):
"""Exception nicely wrapping a request response."""
def __init__(self, *args, **kwargs):
self.response = kwargs.pop('response')
super(CloudStackException, self).__init__(*args, **kwargs)
class CloudStackApiException(CloudStackException):
def __init__(self, *args, **kwargs):
self.error = kwargs.pop('error')
super(CloudStackApiException, self).__init__(*args, **kwargs)
class CloudStack(object):
def __init__(self, endpoint, key, secret, timeout=10, method='get',
verify=None, cert=None, name=None, retry=0,
job_timeout=None, poll_interval=POLL_INTERVAL,
expiration=timedelta(minutes=10), trace=False,
dangerous_no_tls_verify=False, headers=None,
session=None, fetch_result=False):
self.endpoint = endpoint
self.key = key
self.secret = secret
self.timeout = int(timeout)
self.method = method.lower()
if verify:
self.verify = verify
else:
self.verify = not dangerous_no_tls_verify
if headers is None:
headers = {}
self.headers = headers
self.session = session if session is not None else requests.Session()
self.cert = cert
self.name = name
self.retry = int(retry)
self.job_timeout = int(job_timeout) if job_timeout else None
self.poll_interval = float(poll_interval)
if not hasattr(expiration, "seconds"):
expiration = timedelta(seconds=int(expiration))
self.expiration = expiration
self.trace = bool(trace)
self.fetch_result = fetch_result
def __repr__(self):
return ''.format(self.name or self.endpoint)
def __getattr__(self, command):
def handler(**kwargs):
return self._request(command, **kwargs)
return handler
def _prepare_request(self, command, json=True, opcode_name='command',
fetch_list=False, **kwargs):
params = CaseInsensitiveDict(**kwargs)
params.update({
'apiKey': self.key,
opcode_name: command,
})
if json:
params['response'] = 'json'
if 'page' in kwargs or fetch_list:
params.setdefault('pagesize', PAGE_SIZE)
if 'expires' not in params and self.expiration.total_seconds() >= 0:
params['signatureVersion'] = '3'
tz = pytz.utc
expires = tz.localize(datetime.utcnow() + self.expiration)
params['expires'] = expires.astimezone(tz).strftime(EXPIRES_FORMAT)
kind = 'params' if self.method == 'get' else 'data'
return kind, dict(params.items())
def _request(self, command, json=True, opcode_name='command',
fetch_list=False, headers=None, **params):
fetch_result = params.pop('fetch_result', self.fetch_result)
kind, params = self._prepare_request(command, json, opcode_name,
fetch_list, **params)
if headers is None:
headers = {}
headers.update(self.headers)
done = False
max_retry = self.retry
final_data = []
page = 1
while not done:
if fetch_list:
params['page'] = page
transform(params)
params.pop('signature', None)
self._sign(params)
req = requests.Request(self.method,
self.endpoint,
headers=headers,
**{kind: params})
prepped = req.prepare()
if self.trace:
print(prepped.method, prepped.url, file=sys.stderr)
if prepped.headers:
print(prepped.headers, "\n", file=sys.stderr)
if prepped.body:
print(prepped.body, file=sys.stderr)
else:
print(file=sys.stderr)
try:
with self.session as session:
response = session.send(prepped,
timeout=self.timeout,
verify=self.verify,
cert=self.cert)
except requests.exceptions.ConnectionError:
max_retry -= 1
if (
max_retry < 0 or
not command.startswith(('list', 'queryAsync'))
):
raise
continue
max_retry = self.retry
if self.trace:
print(response.status_code, response.reason, file=sys.stderr)
headersTrace = "\n".join("{}: {}".format(k, v)
for k, v in response.headers.items())
print(headersTrace, "\n", file=sys.stderr)
print(response.text, "\n", file=sys.stderr)
data = self._response_value(response, json)
if fetch_list:
try:
[key] = [k for k in data.keys() if k != 'count']
except ValueError:
done = True
else:
final_data.extend(data[key])
page += 1
if len(final_data) >= data.get('count', PAGE_SIZE):
done = True
elif fetch_result and 'jobid' in data:
final_data = self._jobresult(jobid=data['jobid'],
headers=headers)
done = True
else:
final_data = data
done = True
return final_data
def _response_value(self, response, json=True):
"""Parses the HTTP response as a the cloudstack value.
It throws an exception if the server didn't answer with a 200.
"""
if json:
contentType = response.headers.get("Content-Type", "")
if not contentType.startswith(("application/json",
"text/javascript")):
if response.status_code == 200:
raise CloudStackException(
"JSON (application/json) was expected, got {!r}"
.format(contentType),
response=response)
raise CloudStackException(
"HTTP {0.status_code} {0.reason}"
.format(response),
"Make sure endpoint URL {!r} is correct."
.format(self.endpoint),
response=response)
try:
data = response.json()
except ValueError as e:
raise CloudStackException(
"HTTP {0.status_code} {0.reason}"
.format(response),
"{0!s}. Malformed JSON document".format(e),
response=response)
[key] = data.keys()
data = data[key]
else:
data = response.text
if response.status_code != 200:
raise CloudStackApiException(
"HTTP {0} response from CloudStack".format(
response.status_code),
error=data,
response=response)
return data
def _jobresult(self, jobid, json=True, headers=None):
"""Poll the async job result.
To be run via in a Thread, the result is put within
the result list which is a hack.
"""
failures = 0
total_time = self.job_timeout or 2**30
remaining = timedelta(seconds=total_time)
endtime = datetime.now() + remaining
while remaining.total_seconds() > 0:
timeout = max(min(self.timeout, remaining.total_seconds()), 1)
try:
kind, params = self._prepare_request('queryAsyncJobResult',
jobid=jobid)
transform(params)
self._sign(params)
req = requests.Request(self.method,
self.endpoint,
headers=headers,
**{kind: params})
prepped = req.prepare()
if self.trace:
print(prepped.method, prepped.url, file=sys.stderr)
if prepped.headers:
print(prepped.headers, "\n", file=sys.stderr)
if prepped.body:
print(prepped.body, file=sys.stderr)
else:
print(file=sys.stderr)
with self.session as session:
response = session.send(prepped,
timeout=timeout,
verify=self.verify,
cert=self.cert)
j = self._response_value(response, json)
if self.trace:
print(response.status_code, response.reason,
file=sys.stderr)
headersTrace = "\n".join(
"{}: {}".format(k, v)
for k, v in response.headers.items())
print(headersTrace, "\n", file=sys.stderr)
print(response.text, "\n", file=sys.stderr)
failures = 0
if j['jobstatus'] != PENDING:
if j['jobresultcode'] or j['jobstatus'] != SUCCESS:
raise CloudStackApiException("Job failure",
error=j['jobresult'],
response=response)
if 'jobresult' not in j:
raise CloudStackException("Unknown job result",
response=response)
return j['jobresult']
except CloudStackException:
raise
except Exception:
failures += 1
if failures > 10:
raise
time.sleep(self.poll_interval)
remaining = endtime - datetime.now()
if response:
response.status_code = 408
raise CloudStackException("Timeout waiting for async job result",
jobid,
response=response)
def _sign(self, data):
"""
Compute a signature string according to the CloudStack
signature method (hmac/sha1).
"""
# Python2/3 urlencode aren't good enough for this task.
params = "&".join(
"=".join((key, cs_encode(value)))
for key, value in sorted(data.items())
)
digest = hmac.new(
self.secret.encode('utf-8'),
msg=params.lower().encode('utf-8'),
digestmod=hashlib.sha1).digest()
data['signature'] = base64.b64encode(digest).decode('utf-8').strip()
def read_config_from_ini(ini_group=None):
# Config file: $PWD/cloudstack.ini or $HOME/.cloudstack.ini
# Last read wins in configparser
paths = [
os.path.join(os.path.expanduser('~'), '.cloudstack.ini'),
os.path.join(os.getcwd(), 'cloudstack.ini'),
]
# Look at CLOUDSTACK_CONFIG first if present
if 'CLOUDSTACK_CONFIG' in os.environ:
paths.append(os.path.expanduser(os.environ['CLOUDSTACK_CONFIG']))
if not any([os.path.exists(c) for c in paths]):
raise SystemExit("Config file not found. Tried {0}".format(
", ".join(paths)))
conf = ConfigParser()
conf.read(paths)
if not ini_group:
ini_group = os.getenv('CLOUDSTACK_REGION', 'cloudstack')
if not conf.has_section(ini_group):
return dict(name=None)
ini_config = {
k: v
for k, v in conf.items(ini_group)
if v and check_key(k, REQUIRED_CONFIG_KEYS.union(ALLOWED_CONFIG_KEYS))
}
ini_config["name"] = ini_group
# Convert individual header_* settings into a single dict
for k in list(ini_config):
if k.startswith("header_"):
ini_config.setdefault("headers", {})
ini_config["headers"][k[len("header_"):]] = ini_config.pop(k)
return ini_config
def read_config(ini_group=None):
"""
Read the configuration from the environment, or config.
First it try to go for the environment, then it overrides
those with the cloudstack.ini file.
"""
env_conf = dict(DEFAULT_CONFIG)
for key in REQUIRED_CONFIG_KEYS.union(ALLOWED_CONFIG_KEYS):
env_key = "CLOUDSTACK_{0}".format(key.upper())
value = os.getenv(env_key)
if value:
env_conf[key] = value
# overrides means we have a .ini to read
overrides = os.getenv('CLOUDSTACK_OVERRIDES', '').strip()
if not overrides and set(env_conf).issuperset(REQUIRED_CONFIG_KEYS):
return env_conf
ini_conf = read_config_from_ini(ini_group)
overrides = {s.lower() for s in re.split(r'\W+', overrides)}
config = dict(dict(env_conf, **ini_conf),
**{k: v for k, v in env_conf.items() if k in overrides})
missings = REQUIRED_CONFIG_KEYS.difference(config)
if missings:
raise ValueError("the configuration is missing the following keys: " +
", ".join(missings))
# convert booleans values.
bool_keys = ('dangerous_no_tls_verify',)
for bool_key in bool_keys:
if isinstance(config[bool_key], string_type):
try:
config[bool_key] = strtobool(config[bool_key])
except ValueError:
pass
return config
cs-2.7.1/setup.cfg 0000664 0000000 0000000 00000000301 13535423452 0013747 0 ustar 00root root 0000000 0000000 [wheel]
universal = 1
[aliases]
test = pytest
[tool:pytest]
addopts = --cov=cs --cov-report=term-missing cs tests.py
[check-manifest]
ignore =
tox.ini
tests.py
[isort]
style = pep8
cs-2.7.1/setup.py 0000664 0000000 0000000 00000003423 13535423452 0013650 0 ustar 00root root 0000000 0000000 # coding: utf-8
"""
A simple yet powerful CloudStack API client for Python and the command-line.
"""
from __future__ import unicode_literals
import sys
from codecs import open
from setuptools import find_packages, setup
with open('README.rst', 'r', encoding='utf-8') as f:
long_description = f.read()
install_requires = ['pytz', 'requests']
extras_require = {
'highlight': ['pygments'],
}
tests_require = [
'pytest',
'pytest-cache',
'pytest-cov',
]
if sys.version_info < (3, 0):
tests_require.append("mock")
elif sys.version_info >= (3, 5):
extras_require["async"] = ["aiohttp"]
tests_require.append("aiohttp")
setup(
name='cs',
version='2.7.1',
url='https://github.com/exoscale/cs',
license='BSD',
author='Bruno Renié',
description=__doc__.strip(),
long_description=long_description,
packages=find_packages(exclude=['tests']),
zip_safe=False,
include_package_data=True,
platforms='any',
classifiers=(
'Intended Audience :: Developers',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: BSD License',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
),
setup_requires=['pytest-runner'],
install_requires=install_requires,
extras_require=extras_require,
tests_require=tests_require,
entry_points={
'console_scripts': [
'cs = cs:main',
],
},
)
cs-2.7.1/tests.py 0000664 0000000 0000000 00000036757 13535423452 0013672 0 ustar 00root root 0000000 0000000 # coding: utf-8
import datetime
import os
from contextlib import contextmanager
from functools import partial
from unittest import TestCase
try:
from unittest.mock import patch
except ImportError:
from mock import patch
try:
from urllib.parse import urlparse, parse_qs
except ImportError:
from urlparse import urlparse, parse_qs
from cs import CloudStack, CloudStackException, read_config
from cs.client import EXPIRES_FORMAT
from requests.structures import CaseInsensitiveDict
@contextmanager
def env(**kwargs):
old_env = {}
for key in kwargs:
if key in os.environ:
old_env[key] = os.environ[key]
os.environ.update(kwargs)
try:
yield
finally:
for key in kwargs:
if key in old_env:
os.environ[key] = old_env[key]
else:
del os.environ[key]
@contextmanager
def cwd(path):
initial = os.getcwd()
os.chdir(path)
try:
with patch('os.path.expanduser', new=lambda x: path):
yield
finally:
os.chdir(initial)
class ConfigTest(TestCase):
def test_env_vars(self):
with env(CLOUDSTACK_KEY='test key from env',
CLOUDSTACK_SECRET='test secret from env',
CLOUDSTACK_ENDPOINT='https://api.example.com/from-env'):
conf = read_config()
self.assertEqual({
'key': 'test key from env',
'secret': 'test secret from env',
'endpoint': 'https://api.example.com/from-env',
'expiration': 600,
'method': 'get',
'trace': None,
'timeout': 10,
'poll_interval': 2.0,
'verify': None,
'dangerous_no_tls_verify': False,
'cert': None,
'name': None,
'retry': 0,
}, conf)
with env(CLOUDSTACK_KEY='test key from env',
CLOUDSTACK_SECRET='test secret from env',
CLOUDSTACK_ENDPOINT='https://api.example.com/from-env',
CLOUDSTACK_METHOD='post',
CLOUDSTACK_TIMEOUT='99',
CLOUDSTACK_RETRY='5',
CLOUDSTACK_VERIFY='/path/to/ca.pem',
CLOUDSTACK_CERT='/path/to/cert.pem'):
conf = read_config()
self.assertEqual({
'key': 'test key from env',
'secret': 'test secret from env',
'endpoint': 'https://api.example.com/from-env',
'expiration': 600,
'method': 'post',
'timeout': '99',
'trace': None,
'poll_interval': 2.0,
'verify': '/path/to/ca.pem',
'cert': '/path/to/cert.pem',
'dangerous_no_tls_verify': False,
'name': None,
'retry': '5',
}, conf)
def test_env_var_combined_with_dir_config(self):
with open('/tmp/cloudstack.ini', 'w') as f:
f.write('[hanibal]\n'
'endpoint = https://api.example.com/from-file\n'
'key = test key from file\n'
'secret = secret from file\n'
'theme = monokai\n'
'other = please ignore me\n'
'timeout = 50')
self.addCleanup(partial(os.remove, '/tmp/cloudstack.ini'))
# Secret gets read from env var
with env(CLOUDSTACK_ENDPOINT='https://api.example.com/from-env',
CLOUDSTACK_KEY='test key from env',
CLOUDSTACK_SECRET='test secret from env',
CLOUDSTACK_REGION='hanibal',
CLOUDSTACK_DANGEROUS_NO_TLS_VERIFY='1',
CLOUDSTACK_OVERRIDES='endpoint,secret'), cwd('/tmp'):
conf = read_config()
self.assertEqual({
'endpoint': 'https://api.example.com/from-env',
'key': 'test key from file',
'secret': 'test secret from env',
'expiration': 600,
'theme': 'monokai',
'timeout': '50',
'trace': None,
'poll_interval': 2.0,
'name': 'hanibal',
'poll_interval': 2.0,
'verify': None,
'dangerous_no_tls_verify': True,
'retry': 0,
'method': 'get',
'cert': None,
}, conf)
def test_current_dir_config(self):
with open('/tmp/cloudstack.ini', 'w') as f:
f.write('[cloudstack]\n'
'endpoint = https://api.example.com/from-file\n'
'key = test key from file\n'
'secret = test secret from file\n'
'dangerous_no_tls_verify = true\n'
'theme = monokai\n'
'other = please ignore me\n'
'header_x-custom-header1 = foo\n'
'header_x-custom-header2 = bar\n'
'timeout = 50')
self.addCleanup(partial(os.remove, '/tmp/cloudstack.ini'))
with cwd('/tmp'):
conf = read_config()
self.assertEqual({
'endpoint': 'https://api.example.com/from-file',
'key': 'test key from file',
'secret': 'test secret from file',
'expiration': 600,
'theme': 'monokai',
'timeout': '50',
'trace': None,
'poll_interval': 2.0,
'name': 'cloudstack',
'poll_interval': 2.0,
'verify': None,
'dangerous_no_tls_verify': True,
'retry': 0,
'method': 'get',
'cert': None,
'headers': {
'x-custom-header1': 'foo',
'x-custom-header2': 'bar',
},
}, conf)
def test_incomplete_config(self):
with open('/tmp/cloudstack.ini', 'w') as f:
f.write('[hanibal]\n'
'endpoint = https://api.example.com/from-file\n'
'secret = secret from file\n'
'theme = monokai\n'
'other = please ignore me\n'
'timeout = 50')
self.addCleanup(partial(os.remove, '/tmp/cloudstack.ini'))
# Secret gets read from env var
with cwd('/tmp'):
self.assertRaises(ValueError, read_config)
class RequestTest(TestCase):
@patch("requests.Session.send")
def test_request_params(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
timeout=20, expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {},
}
machines = cs.listVirtualMachines(listall='true',
headers={'Accept-Encoding': 'br'})
self.assertEqual(machines, {})
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=20, verify=True), kwargs)
self.assertEqual('GET', request.method)
self.assertEqual('br', request.headers['Accept-Encoding'])
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('listVirtualMachines', qs['command'][0])
self.assertEqual('B0d6hBsZTcFVCiioSxzwKA9Pke8=', qs['signature'][0])
self.assertEqual('true', qs['listall'][0])
@patch("requests.Session.send")
def test_request_params_casing(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
timeout=20, expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {},
}
machines = cs.listVirtualMachines(zoneId=2, templateId='3',
temPlateidd='4', pageSize='10',
fetch_list=True)
self.assertEqual(machines, [])
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=20, verify=True), kwargs)
self.assertEqual('GET', request.method)
self.assertFalse(request.headers)
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('listVirtualMachines', qs['command'][0])
self.assertEqual('mMS7XALuGkCXk7kj5SywySku0Z0=', qs['signature'][0])
self.assertEqual('3', qs['templateId'][0])
self.assertEqual('4', qs['temPlateidd'][0])
@patch("requests.Session.send")
def test_encoding(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {},
}
cs.listVirtualMachines(listall=1, unicode_param=u'éèààû')
self.assertEqual(1, mock.call_count)
[request], _ = mock.call_args
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('listVirtualMachines', qs['command'][0])
self.assertEqual('gABU/KFJKD3FLAgKDuxQoryu4sA=', qs['signature'][0])
self.assertEqual('éèààû', qs['unicode_param'][0])
@patch("requests.Session.send")
def test_transform(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {},
}
cs.listVirtualMachines(foo=["foo", "bar"],
bar=[{'baz': 'blah', 'foo': 1000}],
bytes_param=b'blah')
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=10, verify=True), kwargs)
self.assertEqual('GET', request.method)
self.assertFalse(request.headers)
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('listVirtualMachines', qs['command'][0])
self.assertEqual('ImJ/5F0P2RDL7yn4LdLnGcEx5WE=', qs['signature'][0])
self.assertEqual('1000', qs['bar[0].foo'][0])
self.assertEqual('blah', qs['bar[0].baz'][0])
self.assertEqual('blah', qs['bytes_param'][0])
self.assertEqual('foo,bar', qs['foo'][0])
@patch("requests.Session.send")
def test_transform_dict(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'scalevirtualmachineresponse': {},
}
cs.scaleVirtualMachine(id='a',
details={'cpunumber': 1000, 'memory': '640k'})
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=10, verify=True), kwargs)
self.assertEqual('GET', request.method)
self.assertFalse(request.headers)
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('scaleVirtualMachine', qs['command'][0])
self.assertEqual('ZNl66z3gFhnsx2Eo3vvCIM0kAgI=', qs['signature'][0])
self.assertEqual('1000', qs['details[0].cpunumber'][0])
self.assertEqual('640k', qs['details[0].memory'][0])
@patch("requests.Session.send")
def test_transform_empty(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'createnetworkresponse': {},
}
cs.createNetwork(name="", display_text="")
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=10, verify=True), kwargs)
self.assertEqual('GET', request.method)
self.assertFalse(request.headers)
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('createNetwork', qs['command'][0])
self.assertEqual('CistTEiPt/4Rv1v4qSyILvPbhmg=', qs['signature'][0])
self.assertEqual('', qs['name'][0])
self.assertEqual('', qs['display_text'][0])
@patch("requests.Session.send")
def test_method(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
method='post', expiration=-1)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {},
}
cs.listVirtualMachines(blah='brah')
self.assertEqual(1, mock.call_count)
[request], kwargs = mock.call_args
self.assertEqual(dict(cert=None, timeout=10, verify=True), kwargs)
self.assertEqual('POST', request.method)
self.assertEqual('application/x-www-form-urlencoded',
request.headers['Content-Type'])
qs = parse_qs(request.body, True)
self.assertEqual('listVirtualMachines', qs['command'][0])
self.assertEqual('58VvLSaVUqHnG9DhXNOAiDFwBoA=', qs['signature'][0])
self.assertEqual('brah', qs['blah'][0])
@patch("requests.Session.send")
def test_error(self, mock):
mock.return_value.status_code = 530
mock.return_value.json.return_value = {
'listvirtualmachinesresponse': {'errorcode': 530,
'uuidList': [],
'cserrorcode': 9999,
'errortext': 'Fail'}}
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar')
self.assertRaises(CloudStackException, cs.listVirtualMachines)
@patch("requests.Session.send")
def test_bad_content_type(self, get):
get.return_value.status_code = 502
get.return_value.headers = CaseInsensitiveDict(**{
"content-type": "text/html;charset=utf-8"})
get.return_value.text = ("502"
"Gateway timeout
")
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar')
self.assertRaises(CloudStackException, cs.listVirtualMachines)
@patch("requests.Session.send")
def test_signature_v3(self, mock):
cs = CloudStack(endpoint='https://localhost', key='foo', secret='bar',
expiration=600)
mock.return_value.status_code = 200
mock.return_value.json.return_value = {
'createnetworkresponse': {},
}
cs.createNetwork(name="", display_text="")
self.assertEqual(1, mock.call_count)
[request], _ = mock.call_args
url = urlparse(request.url)
qs = parse_qs(url.query, True)
self.assertEqual('createNetwork', qs['command'][0])
self.assertEqual('3', qs['signatureVersion'][0])
expires = qs['expires'][0]
# we ignore the timezone for Python2's lack of %z
expires = datetime.datetime.strptime(expires[:19],
EXPIRES_FORMAT[:-2])
self.assertTrue(expires > datetime.datetime.utcnow(), expires)
cs-2.7.1/tox.ini 0000664 0000000 0000000 00000000447 13535423452 0013454 0 ustar 00root root 0000000 0000000 [tox]
envlist =
py{27,34,35,36,37,38}
lint
skip_missing_interpreters = True
[travis]
python =
3.6: py36, lint
[testenv]
deps=
check-manifest
flake8
flake8-import-order
commands =
pip install pytest pytest-cache pytest-cov
python setup.py test
check-manifest
lint: flake8 cs tests.py