maas-test-0.1+bzr147.orig/MANIFEST.in0000644000000000000000000000003112321262143015106 0ustar 00000000000000include requirements.txt maas-test-0.1+bzr147.orig/Makefile0000644000000000000000000000403412321262143015017 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). PYTHON := python define virtualenv virtualenv --python=$(PYTHON) --quiet endef define apt-get sudo DEBIAN_FRONTEND=noninteractive \ apt-get --assume-yes --no-install-recommends endef # --- all: build lint doc build: bin/maas-test dist: bin/python setup.py bin/python setup.py --quiet egg_info -r sdist test: setup.py bin/tox tox.ini @bin/tox clean: $(RM) -r bin build dist include lib local TAGS tags find . -name '*.py[co]' -print0 | xargs -r0 $(RM) -r find . -name '__pycache__' -print0 | xargs -r0 $(RM) -r find . -name '*.egg' -print0 | xargs -r0 $(RM) -r find . -name '*.egg-info' -print0 | xargs -r0 $(RM) -r find . -name '*~' -print0 | xargs -r0 $(RM) $(RM) -r .tox .deps lint: bin/flake8 @bin/flake8 maastest install-dependencies: packages.txt @xargs --verbose --no-run-if-empty $(apt-get) install < packages.txt update-requirements: requirements.txt.new mv $< requirements.txt # Generate manpage in man/ man: $(patsubst docs/man/%.rst,man/%,$(wildcard docs/man/*.rst)) man/%: bin/rst2man.py docs/man/%.rst bin/rst2man.py docs/man/$*.rst $@ # Generate documentation. doc: man # --- bin/maas-test: bin/python setup.py bin/python setup.py --quiet develop # Generated executables. define executables bin/flake8 bin/pyflakes bin/pep8 bin/rst2man.py bin/tox endef $(strip $(executables)): bin/python requirements.txt bin/python -m pip install --quiet --ignore-installed -r requirements.txt bin/python: $(virtualenv) --system-site-packages $(PWD) requirements.txt.new: requirements.txt $(virtualenv) --no-site-packages .deps .deps/bin/python -m pip install --quiet \ --ignore-installed -r requirements.txt .deps/bin/python -m pip freeze --local | \ egrep -v ^virtualenv= > $@ @$(RM) -r .deps @diff -u $< $@ || true # --- define phony all build clean dist doc install-dependencies lint man test update-requirements endef .PHONY: $(strip $(phony)) maas-test-0.1+bzr147.orig/README.txt0000644000000000000000000000261512321262143015060 0ustar 00000000000000.. -*- mode: rst -*- ********* maas-test ********* A utility to test if a particular piece of hardware is compatible with MAAS. For more information see the `Launchpad project page`_. .. _Launchpad project page: https://launchpad.net/maas-test Setting up for development -------------------------- To get maas-test running should be fairly easy, assuming you're on Ubuntu 13.10 or later:: # Add uvtool's dev PPA in order to get a recent enough package. sudo add-apt-repository -y ppa:uvtool-dev/trunk sudo apt-get update # Get a local copy of lp:maas-test. bzr branch lp:maas-test cd maas-test # Install the dependencies and build the environment. make install-dependencies make build The ``make install-dependencies`` step ensures that maas-test's dependencies are installed as system packages, and will not later be installed by pip. Dependencies ------------ The policy is: * All production dependencies must be satisfied from system packages. * Development-only dependencies can be installed from PyPI. That means that production dependencies need to go in ``packages.txt`` and development-only ones into ``requirements.txt``. The latter can be maintained like so:: make update-requirements Then **carefully** review the changes; libraries installed elsewhere on the system can cause ``freeze --local`` to come up with radically different answers to those on another machine. maas-test-0.1+bzr147.orig/docs/0000755000000000000000000000000012321262143014306 5ustar 00000000000000maas-test-0.1+bzr147.orig/maastest/0000755000000000000000000000000012321262143015177 5ustar 00000000000000maas-test-0.1+bzr147.orig/man/0000755000000000000000000000000012321262143014131 5ustar 00000000000000maas-test-0.1+bzr147.orig/packages.txt0000644000000000000000000000053412321262143015677 0ustar 00000000000000cpu-checker polipo python-distro-info python-fixtures python-lxml python-maas-client python-mock python-netaddr python-netifaces python-six python-testresources python-testtools python-virtualenv python3-fixtures python3-lxml python3-mock python3-netaddr python3-six python3-testresources python3-testtools qemu-kvm uvtool uvtool-libvirt virt-what maas-test-0.1+bzr147.orig/requirements.txt0000644000000000000000000000013512321262143016641 0ustar 00000000000000docutils==0.11 flake8==2.1.0 mccabe==0.2.1 pep8==1.4.6 py==1.4.18 pyflakes==0.7.3 tox==1.6.1 maas-test-0.1+bzr147.orig/setup.py0000755000000000000000000000210212321262143015066 0ustar 00000000000000#!/usr/bin/env python # Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Distutils installer for maas-test.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type from setuptools import setup import six setup( name='maas-test', version="0.1", url="https://launchpad.net/maas-test", maintainer="Canonical", maintainer_email="maas-devel@lists.launchpad.net", packages={'maastest' if six.PY3 else b'maastest'}, package_dir={'maastest': 'maastest'}, install_requires=[], # It's complicated; see README.txt tests_require=[], # It's complicated; see README.txt test_suite="maastest.tests", include_package_data=True, zip_safe=False, description=( "A utility to test if a particular piece of hardware is " "compatible with MAAS."), entry_points={ "console_scripts": [ "maas-test = maastest.script:entry_point", ], }, ) maas-test-0.1+bzr147.orig/tox.ini0000644000000000000000000000027112321262143014671 0ustar 00000000000000[tox] # Python 3 is disabled until maas-client is packaged for Python 3. #envlist = py27, py33 envlist = py27 [testenv] commands = {envpython} setup.py test sitepackages = True deps = maas-test-0.1+bzr147.orig/docs/man/0000755000000000000000000000000012321262143015061 5ustar 00000000000000maas-test-0.1+bzr147.orig/docs/man/maas-test.8.rst0000644000000000000000000002663512321262143017673 0ustar 00000000000000========= maas-test ========= ------------------------------------------------------ test a server for compatibility running as a MAAS node ------------------------------------------------------ :Author: MAAS engineering team at Canonical, Ltd. :Copyright: Copyright (c) 2013, Canonical Ltd. :Manual section: 8 Synopsis ======== maas-test --interactive [options...] maas-test --bmc-mac=
[options...] Description =========== Use `maas-test` to test whether a server can be used as a MAAS node. It must be run as root. The test will set up a MAAS instance and attempt to manage the node. Do not run `maas-test` on the same system that you wish to test. Two systems are involved: * The *testing system*. Run `maas-test` here. It will create a virtual MAAS server for the duration of the test. * The *node*. The MAAS server running on the testing system will control it as a node, running it through various test steps. MAAS controls the node remotely though IPMI. This may include powering it on or off, booting it, and even installing an operating system. **CAUTION:** Future versions of `maas-test` may wipe the node's disks and install a new operating system. And in general, accidents may happen. **Be prepared to lose any data stored there, and to re-install the node after the test.** In addition to this, even in the present version, MAAS will modify the node's firmware netboot settings. Network configuration ===================== Run the test in a dedicated testing network consisting of just these two systems. This network should be as isolated as possible, and does not need a route to the internet. For the node, that means that both the node's own network interface card (NIC) and its baseboard management controller (BMC) should be on the testing network. The testing system only needs one NIC on the testing network. In addition to being on the testing network, the testing system must also have internet access. maas-test supports two different network architectures: Network config #1: the IPMI NIC is connected to the interface managed by MAAS:: +----------+ +---------------+ | Internet | | |-----+ | | | |eth0 |+------------>| | | |-----+ +----------+ | Host | | (where | NIC's MAC: aa:bb:cc:dd:ee:ff | maas-test | | | is installed) |-----+ | | |eth1 | +------+ | | | |<-->|Router| V +--------+ | |-----+ | |<-->+----| | +---------------+ +------+ |IPMI| | ^ |eth1| | | +----| Node | | | being | | +----| tested | +---->|eth0| | | | | +----| | +--------+ In this case, one needs to pass the IPMI NIC's MAC address to maas-test. The invocation of maas-test will look something like: $ maas-test --bmc-mac aa:bb:cc:dd:ee:ff --bmc-username username --bmc-password password eth1 Network config #2: the IPMI NIC is *not* connected to the interface managed by MAAS and has a fixed IP address:: +----------+ +---------------+ | Internet | | |-----+ | | | |eth0 |+------------>+----------+ | |-----+ | Host | +---------+ | (where |-----+ +----| | | maas-test |eth1 |<-------->|eth0| | | is installed) |-----+ +----| Node | | | | being | | |-----+ +----| tested | | |eth2 |<-------->|eth1| | | |-----+ |IPMI| | +---------------+ +----| | ^ +---------+ | Fixed IP address AA.BB.CC.DD In this case, one needs to pass the IPMI NIC's IP address to maas-test. The invocation of maas-test will look something like: $ maas-test --bmc-ip AA.BB.CC.DD --bmc-username username --bmc-password password eth1 Preparing to run ================ The test will run MAAS in a virtual machine. It will not be installed on your physical system. Nevertheless there are a few things you need to be aware of: 1. Prepare to lose any data on the node's disks. 2. Ensure that your node (both its BMC and its own NIC) is connected only to the testing network. 3. Make sure that there is no DHCP server running on the testing network. The test program will also check for this on startup. MAAS will act as a DHCP server on the testing network. 4. Select a network interface on the testing system that provides access to the testing network. You will be passing this interface to `maas-test`. 5. Depending on caching, the test may download and store large amounts of data on the testing system. Make sure you have sufficient disk space and network bandwidth. The data that needs downloading and/or caching consists mostly of system images for the virtual machine, and for booting the node. As a rule of thumb, count on half a gigabyte as a baseline, plus a quarter gigabyte for each combination of architecture and Ubuntu release that will run on the node. Running ======= There is one required argument: the network interface which connects the testing system to the testing network, e.g. `eth1`. The test will need to power up the node. It can do that in two ways: a. Through IPMI commands to the node's BMC. You'll need to specify its address and authentication information using the `--bmc-*` options. b. Manually when running in `interactive mode`. The test will stop and ask you to power up the node. Test results ============ Once maas-test has finished testing the node it will upload the test results to Launchpad (unless told otherwise; see the `reporting options`_). This allows you to share the test results with others, including the MAAS developers, by filing a Launchpad bug which includes the test results as an attachment. By default, the results are also written to timestamped log files under `/var/log/maas-test`. Options ======= -h, --help Show help and exit. --bmc-mac=MAC MAC address for the node's baseboard management controller. MAAS will control the node's power and boot sequence through this controller. It must be attached to the testing network. This is mutually exclusive with **--bmc-ip**. This option is not needed in interactive mode. In non-interactive mode, either **--bmc-mac** or **--bmc-ip** is required. --bmc-ip=IP IP address of the node's baseboard management controller. Use this if the BMC is not connected to the interface given as argument. Note that the IP address must not change for the duration of the testing. This is mutually exclusive with **--bmc-mac**. This option is not needed in interactive mode. In non-interactive mode, either **--bmc-mac** or **--bmc-ip** is required. --bmc-password=password Password for IPMI authentication on the BMC. Use with **--bmc-user**. Not needed in interactive mode. --bmc-user=user Username for IPMI authentication. Use with **--bmc-password**. Not needed in interactive mode. --ipmi-driver=driver Specify IPMI driver version. Default is LAN_2_0 (IPMI v2.0), which is what most modern BMCs support. Use the LAN option if your BMC only supports IPMI version 1.5. --interactive Interactive mode. Instead of powering up the node automatically through IPMI, prompt the user to turn it on manually. In this mode there is no need to specify BMC details; the MAAS enlistment process will discover them automatically. --archive=archive Optional package repository name. If given, the virtual machine may install packages from this additional archive as well as from the main Ubuntu archive. The archive is added to the virtual machine using 'add-apt-repository'; it may be either a line in the format of apt's sources.list, or a personal package archive identifier in the form 'ppa:/', or a distribution component that should be enabled. This is typically used to test recent versions of MAAS that are only available in a PPA such as "ppa:maas-maintainers/dailybuilds". This can be specified multiple times. --series=codename Code name for the Ubuntu release series that should be run on the node during enlistment and commissioning, e.g. "saucy" for 13.10 Saucy Salamander. Defaults to the latest long-term support release of Ubuntu. --architecture=architecture CPU architecture for the node. MAAS will import boot images for this architecture only. The architecture may include a sub-architecture name, which defaults to `generic`, so e.g. `i386/generic` may be abbreviated to `i386`. The default architecture is `amd64`. --maas-series=codename Code name for the Ubuntu release series to install on the virtual machine (where the MAAS server will be installed). Defaults to the latest stable Ubuntu series. --http-proxy=URL Use the given HTTP proxy for all downloads, both on the testing system and on the nodes: KVM images, MAAS boot images, and Ubuntu packages. Like `--disable-cache`, this also disables the caching proxy that `maas-test` runs by default. --disable-cache Do not run a caching HTTP proxy on the testing system. This cache is normally used for all downloads, both on the testing system and on the nodes: KVM images, MAAS boot images, and Ubuntu packages. It speeds up subsequent test runs, but also caches a large amount of data on the testing system's filesystem. The proxy software used is `polipo`. The cache will be stored under `/var/cache/maas-test`. --dry-run Bring up the MAAS region controller in a virtual machine, but don't attempt to boot any machine on its network or do any destructive testing. .. _`reporting options`: --no-reporting Turn off all reporting of test results. Results will be written to stdout but not recorded elsewhere. --log-results-only Write test results to a file, but don't upload them to Launchpad. Results will be written to a timestamped log file under `/var/log/maas-test`. Reporting bugs ============== Report bugs in Launchpad: https://bugs.launchpad.net/maas-test/ See also ======== The maas-test program is part of the MAAS project. Find out more about MAAS at http://maas.ubuntu.com/ For maas-test development, see https://launchpad.net/maas-test/ Polipo caching proxy: http://www.pps.univ-paris-diderot.fr/~jch/software/polipo/ Ubuntu virtualization tools (uvtool): https://launchpad.net/uvtool Files ===== State and configuration for `maas-test` is stored in `/var/cache/maas-test`. This includes an ssh key pair for communicating with the virtual machine. Pidfiles are stored in `/run/maas-test`, and logs and test results are written to `/var/log/maas-test`. If you choose to run a local proxy, downloaded data will also be cached in the `/var/cache/maas-test`. It can quickly grow to gigabyte sizes. maas-test-0.1+bzr147.orig/maastest/__init__.py0000644000000000000000000000044312321262143017311 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """maastest.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ ] maas-test-0.1+bzr147.orig/maastest/cases.py0000644000000000000000000001611612321262143016654 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Test whether or not a node is compatible with MAAS.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "TestOneNode", ] import httplib import json import sys from maastest import utils from maastest.maas_enums import NODE_STATUS import testresources import testtools from testtools.testcase import gather_details class TestMAAS(testtools.TestCase, testresources.ResourcedTestCase): """Base class for testing machine compatibility with MAAS.""" @classmethod def configure(cls, args, maas): """Set test parameters. These are class variables, but they are set only on ad-hoc subclasses, not on `TestMAAS` itself. So this class stays in its original state. :param args: An arguments object for the program run, as returned by the arguments parser. :param maas: `MAASFixture` with a running MAAS. """ assert cls != TestMAAS, "TestMAAS.configure called on TestMAAS class." cls.args = args cls.maas = maas cls.details = {} def shortDescription(self): """Overridable from `unittest`: Describe the ongoing test. By default this shows the first line of the test's docstring (if available), but this `TestCase` is basically our UI so we want full, user-friendly docstrings. """ # Cribbed from unittest2.TestCase. The difference is that we return # the whole docstring, instead of just the first line. # Future maintainers beware: the docstring may also be None. return self._testMethodDoc def unify_details(self): """Merge the details of the fixture into the details of the TestMAAS. """ # Make sure that the MAAS logs are included in the test report. self.maas.collect_logs() gather_details(self.maas.kvm_fixture.getDetails(), self.getDetails()) # This is a bit of a horrible hack, but it allows us to gather # up the details in the case of successful runs (which # testtools, rather frustratingly, doesn't support). gather_details(self.getDetails(), self.details) def setUp(self): super(TestMAAS, self).setUp() self.addCleanup(self.unify_details) def get_node_list(self, status): """Return the list of nodes with the given status.""" # TODO: Move this to MAASFixture? uri = utils.get_uri('nodes/') response = self.maas.admin_maas_client.get(uri, op="list") self.assertEqual( httplib.OK, response.code, "Failed to get the node list.") nodes = json.loads(response.read()) relevant_nodes = [node for node in nodes if node['status'] == status] return relevant_nodes class TestOneNode(TestMAAS): """Test a single node for compatibility with MAAS.""" def test_power_up_node(self): """Power up the node under test. If maas-test is running in interactive mode, this will ask the user to power up the node manually. Otherwise it will use the provided power parameters to power up the node by running 'ipmipower' in the virtual machine.. """ if self.args.dry_run: self.skip("Dry run.") if self.args.interactive: self.power_up_node_interactive() else: self.power_up_node_noninteractive() def power_up_node_interactive(self): """Asks the user to power up the node manually.""" print( "Power up the node under test and press enter to proceed " "with the testing.") sys.stdin.readline() def power_up_node_noninteractive(self): """Power up the node using the provided power parameters.""" # Power-cycle the node. # TODO: Move these into MAASFixture? power_address = self.find_bmc_ip_address() self.maas.kvm_fixture.run_command([ 'ipmipower', '-h', power_address, '-W', 'opensesspriv', '-D', self.args.ipmi_driver, '-u', self.args.bmc_username, '-p', self.args.bmc_password, '--off'], check_call=True) self.maas.kvm_fixture.run_command([ 'ipmipower', '-h', power_address, '-W', 'opensesspriv', '-D', self.args.ipmi_driver, '-u', self.args.bmc_username, '-p', self.args.bmc_password, '--on'], check_call=True) def find_bmc_ip_address(self): """Returns the BMC IP address. This is done by scanning the network it the BMC MAC address was given or by simply returning the IP address of the BMC is it was passed. """ if self.args.bmc_mac is not None: # The BMC's MAC address was passed: scan the network to find the # IP address of the node's BMC. for retry in utils.retries(delay=20, timeout=5 * 60): ip_scan = self.maas.kvm_fixture.get_ip_from_network_scan power_address = ip_scan(self.args.bmc_mac) if power_address is not None: return power_address else: # The BMC IP address was passed: just return that. return self.args.bmc_ip if power_address is None: self.fail("Failed to get the IP address of the BMC.") def test_enlist_node(self): """Enlist node.""" if self.args.dry_run: self.skip("Dry run.") for retry in utils.retries(delay=10, timeout=10 * 60): nb_enlisted_nodes = len(self.get_node_list(NODE_STATUS.DECLARED)) if nb_enlisted_nodes == 1: return self.fail("Failed to enlist node.") def test_wait_node_down(self): """Wait for the node to go down.""" if self.args.dry_run: self.skip("Dry run.") power_address = self.find_bmc_ip_address() for retry in utils.retries(delay=10, timeout=60): _, result, _ = self.maas.kvm_fixture.run_command([ 'ipmipower', '-h', power_address, '-W', 'opensesspriv', '-D', self.args.ipmi_driver, '-u', self.args.bmc_username, '-p', self.args.bmc_password, '--stat'], check_call=True) if result.strip() == "%s: off" % power_address: return self.fail("Node wasn't powered down after enlistment.") def test_commission_node(self): """Commission node.""" if self.args.dry_run: self.skip("Dry run.") uri = utils.get_uri('nodes/') response = self.maas.admin_maas_client.post(uri, op="accept_all") self.assertEqual( httplib.OK, response.code, "Failed to accept node.") for retry in utils.retries(delay=10, timeout=10 * 60): nb_enlisted_nodes = len(self.get_node_list(NODE_STATUS.READY)) if nb_enlisted_nodes == 1: return self.fail("Failed to commission node.") maas-test-0.1+bzr147.orig/maastest/console.py0000644000000000000000000000144712321262143017221 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Console-based front-end to maas-test.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "run_console", ] import unittest from maastest import utils def run_console(testcase, output_stream): """Run the tests found on `testcase`, reporting to the console. :type testcase: `unittest.TestCase` :return: `unittest.TestResult` """ loader = utils.CasesLoader() suite = loader.loadTestsFromTestCase(testcase) runner = unittest.TextTestRunner( verbosity=2, descriptions=False, failfast=True, stream=output_stream) return runner.run(suite) maas-test-0.1+bzr147.orig/maastest/detect_dhcp.py0000644000000000000000000001632512321262143020026 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Discover DHCP servers on your network. This code was copied over from the MAAS codebase. Update with care. """ from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [ 'probe_dhcp', ] from contextlib import contextmanager from errno import EADDRNOTAVAIL import fcntl import logging from random import randint import socket import struct def make_transaction_ID(): """Generate a random DHCP transaction identifier.""" transaction_id = b'' for _ in range(4): transaction_id += struct.pack(b'!B', randint(0, 255)) return transaction_id class DHCPDiscoverPacket: """A representation of a DHCP_DISCOVER packet. :param my_mac: The MAC address to which the dhcp server should respond. Normally this is the MAC of the interface you're using to send the request. """ def __init__(self, my_mac): self.transaction_ID = make_transaction_ID() self.packed_mac = self.string_mac_to_packed(my_mac) self._build() @classmethod def string_mac_to_packed(cls, mac): """Convert a string MAC address to 6 hex octets. :param mac: A MAC address in the format AA:BB:CC:DD:EE:FF :return: a byte string of length 6 """ packed = b'' for pair in mac.split(':'): hex_octet = int(pair, 16) packed += struct.pack(b'!B', hex_octet) return packed def _build(self): self.packet = b'' self.packet += b'\x01' # Message type: Boot Request (1) self.packet += b'\x01' # Hardware type: Ethernet self.packet += b'\x06' # Hardware address length: 6 self.packet += b'\x00' # Hops: 0 self.packet += self.transaction_ID self.packet += b'\x00\x00' # Seconds elapsed: 0 # Bootp flags: 0x8000 (Broadcast) + reserved flags self.packet += b'\x80\x00' self.packet += b'\x00\x00\x00\x00' # Client IP address: 0.0.0.0 self.packet += b'\x00\x00\x00\x00' # Your (client) IP address: 0.0.0.0 self.packet += b'\x00\x00\x00\x00' # Next server IP address: 0.0.0.0 self.packet += b'\x00\x00\x00\x00' # Relay agent IP address: 0.0.0.0 self.packet += self.packed_mac # Client hardware address padding: 00000000000000000000 self.packet += b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' self.packet += b'\x00' * 67 # Server host name not given self.packet += b'\x00' * 125 # Boot file name not given self.packet += b'\x63\x82\x53\x63' # Magic cookie: DHCP # Option: (t=53,l=1) DHCP Message Type = DHCP Discover self.packet += b'\x35\x01\x01' self.packet += b'\x3d\x06' + self.packed_mac # Option: (t=55,l=3) Parameter Request List self.packet += b'\x37\x03\x03\x01\x06' self.packet += b'\xff' # End Option class DHCPOfferPacket: """A representation of a DHCP_OFFER packet.""" def __init__(self, data): self.transaction_ID = data[4:8] self.dhcp_server_ID = socket.inet_ntoa(data[245:249]) # UDP ports for the BOOTP protocol. Used for discovery requests. BOOTP_SERVER_PORT = 67 BOOTP_CLIENT_PORT = 68 # ioctl request for requesting IP address. SIOCGIFADDR = 0x8915 # ioctl request for requesting hardware (MAC) address. SIOCGIFHWADDR = 0x8927 def get_interface_MAC(sock, interface): """Obtain a network interface's MAC address, as a string.""" ifreq = struct.pack(b'256s', interface.encode('ascii')[:15]) info = fcntl.ioctl(sock.fileno(), SIOCGIFHWADDR, ifreq) mac = ''.join(['%02x:' % ord(char) for char in info[18:24]])[:-1] return mac def get_interface_IP(sock, interface): """Obtain an IP address for a network interface, as a string.""" ifreq = struct.pack( b'16sH14s', interface.encode('ascii')[:15], socket.AF_INET, b'\x00' * 14) try: info = fcntl.ioctl(sock, SIOCGIFADDR, ifreq) except IOError as e: if e.errno == EADDRNOTAVAIL: # Interface has no IP address. return None else: raise ip = struct.unpack(b'16sH2x4s8x', info)[2] return socket.inet_ntoa(ip) @contextmanager def udp_socket(): """Open, and later close, a UDP socket.""" sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # We're going to bind to the BOOTP/DHCP client socket, where dhclient may # also be listening, even if it's operating on a different interface! # The SO_REUSEADDR option makes this possible. sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) yield sock sock.close() def request_dhcp(interface): """Broadcast a DHCP discovery request. Returns DHCP transaction ID, or `None` in the special case where the network interface has no IP address. """ with udp_socket() as sock: sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1) mac = get_interface_MAC(sock, interface) bind_address = get_interface_IP(sock, interface) if bind_address is None: return None discover = DHCPDiscoverPacket(mac) sock.bind((bind_address, BOOTP_CLIENT_PORT)) sock.sendto(discover.packet, ('', BOOTP_SERVER_PORT)) return discover.transaction_ID def receive_offers(transaction_id): """Receive DHCP offers. Return set of offering servers.""" servers = set() with udp_socket() as sock: # The socket we use for receiving DHCP offers must be bound to IF_ANY. sock.bind(('', BOOTP_CLIENT_PORT)) try: while True: sock.settimeout(3) data = sock.recv(1024) offer = DHCPOfferPacket(data) if offer.transaction_ID == transaction_id: servers.add(offer.dhcp_server_ID) except socket.timeout: # No more offers. Done. return servers def probe_dhcp(interface): """Look for a DHCP server on the network. This must be run with provileges to broadcast from the BOOTP port, which typically requires root. It may fail to bind to that port if a DHCP client is running on that same interface. :param interface: Network interface name, e.g. "eth0", attached to the network you wish to probe. :return: Set of discovered DHCP servers. :exception IOError: If the interface does not have an IP address. """ logging.info("Scanning for unexpected DHCP servers on testing network...") # There is a small race window here, after we close the first socket and # before we bind the second one. Hopefully executing a few lines of code # will be faster than communication over the network. # UDP is not reliable at any rate. If detection is important, we should # send out repeated requests. transaction_id = request_dhcp(interface) if transaction_id is None: # Network interface has no IP address. Our DHCP detection code does # not work for this case. A second check, made through the virtual # machine, will cover that case. return set() return receive_offers(transaction_id) maas-test-0.1+bzr147.orig/maastest/kvmfixture.py0000644000000000000000000004746212321262143017772 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Manage a KVM machine.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "KVMFixture", ] from datetime import timedelta import itertools import logging import os.path from pipes import quote import textwrap from uuid import uuid1 from fixtures import Fixture from lxml import etree from maastest.utils import ( DEFAULT_STATE_DIR, extract_mac_ip_mapping, retries, run_command, ) import netaddr from six import text_type from testtools.content import text_content # Maximum wait for the virtual machine to complete its cloud-init # initialization. It can take quite a while, because it performs an # apt-get update. CLOUDINIT_TIMEOUT = timedelta(minutes=10) # The virtual machine will create this file on its filesystem once cloud-init # is done, and the machine is ready for use. CLOUDINIT_COMPLETION_MARKER = '/var/lib/cloud/instance/boot-finished' # Locale to use on the virtual machine. It needs to be one that is defined in # the virtual machine, or postgres installation will break, as may other # things. # # See bug 969462 and this discussion: # https://lists.ubuntu.com/archives/ubuntu-devel/2013-June/037195.html VM_LOCALE = "en_US.UTF-8" # Standard, non-interactive, "apt-get install" line. APT_GET_INSTALL = [ 'sudo', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', 'install', '-y', ] # Interface NATed to the host's network. NATED_INTERFACE = 'eth0' # Interface connected to the 'direct' network. DIRECT_INTERFACE = 'eth1' class KVMFixture(Fixture): """A fixture for a Kernel-based Virtual Machine. This fixture uses `uvtool` to get the images and create the virtual machine. """ def __init__(self, series, architecture, proxy_url, direct_interface=None, direct_network=None, name=None, archives=None, ssh_key_dir=None, kvm_timeout=None, simplestreams_filter=None): """Initialise a KVM-based Virtual Machine fixture. The VM's network is configured like this: - eth0 is a NATed interface which gets its IP from the KVM-provided dnsmasq server. - eth1 only exists if `direct_interface` is supplied. In that case eth1 is bridged to the interface `direct_interface` on the host, and has a fixed IP (`direct_ip`). :param series: The Ubuntu series to run in VM (e.g. 'saucy'). :type series: string :param architecture: The architecture of the VM (e.g. 'amd64'). :type architecture: string :param proxy_url: The URL of an HTTP proxy. :type proxy_url: LocalProxyFixture :param name: An optional name for the VM. If not provided, a random one will be generated. :type name: string :param direct_interface: An optional network interface name. If provided, the VM will have a second interface (eth1) directly attached to the named physical interface of the host machine. If this is provided, the caller can provide a for 'direct_network' to govern how the interface's network will be configured. :type direct_interface: string :param direct_network: An optional network definition. This parameter is ignored if the parameter 'direct_interface' is not provided. It is used to configure the network of the the eth1 interace of the VM instance. If 'direct_interface' is provided and 'direct_network' is not provided, this will default to the network 192.168.2.0/24. :type direct_network: netaddr.IPNetwork :param archives: An optional list of repository names. If provided, each repository will be added onto the VM. :type archives: list :param ssh_key_dir: Optional directory in which to store the SSH keys for connecting to this KVM instance. Defaults to `/var/cache/maas-test`. :type ssh_key_dir: string. :param kvm_timeout: The number of seconds to wait for a KVM instance to start. :type kvm_timeout: int :param simplestreams_filter: Optional simplestreams filter use to retrieve the VM images. This is a string using the standard simplestreams format: space-separated filter constraints (e.g. "label=alpha1 filter2=value"). :type simplestreams_filter: string """ super(KVMFixture, self).__init__() self.command_count = itertools.count(1) self.archives = archives self.series = series self.simplestreams_filter = simplestreams_filter self.architecture = architecture self.proxy_url = proxy_url self.direct_interface = direct_interface self.direct_network = direct_network self._ip_address = None if direct_interface is not None: if direct_network is None: # Generate a 'direct' private network. self.direct_network = netaddr.IPNetwork('192.168.2.0/24') self.direct_ip = "%s" % ( netaddr.IPAddress(self.direct_network.first + 1)) else: self.direct_ip = None if name is None: name = text_type(uuid1()) self.name = name if ssh_key_dir is None: ssh_key_dir = DEFAULT_STATE_DIR self.ssh_private_key_file = os.path.join( ssh_key_dir, 'vm_ssh_id_rsa') self.kvm_timeout = kvm_timeout self.running = False def setUp(self): super(KVMFixture, self).setUp() self.import_image() self.generate_ssh_key() self.start() # From this point, if further setup fails, we'll need to clean up. self.addCleanup(self.destroy) self.wait_for_vm() self.wait_for_cloudinit() self.configure_network() self.configure_archives() self.install_base_packages() logging.info("Virtual machine %s is ready.", self.name) def direct_first_available_ip(self): """Returns the first available IP on the 'direct' network. The 'direct' network is the network directly connected to the host's machine.""" return "%s" % netaddr.IPAddress(self.direct_network.first + 2) def identity(self): """Return the identity (user@ip-address) for this machine. This is the identity that can be used to ssh into this machine.""" return "ubuntu@%s" % self.ip_address() def ip_address(self): """Return the IP address of the KVM instance. The IP address is discovered by calling uvt-kvm ip for the KVM instance. Once the IP address has been fetched once it is cached here, so uvt-kvm ip need not be called again. """ if self._ip_address is None: _, ip_address, _ = run_command( ["uvt-kvm", "ip", self.name], check_call=True) self._ip_address = ip_address.strip().decode("ascii") return self._ip_address def get_ssh_public_key_file(self): """Return path to public SSH key file, for VM sshd configuration.""" return self.ssh_private_key_file + '.pub' def get_simplestreams_filter(self): """Return a list with the simplestreams filters for this VM.""" filters = [ "arch=%s" % self.architecture, "release=%s" % self.series, ] if self.simplestreams_filter is not None: filters.extend(self.simplestreams_filter.split()) return filters def import_image(self): """Download the image required to create the virtual machine.""" logging.info( "Downloading KVM image for %s..." % ' '.join( self.get_simplestreams_filter())) command = [ "sudo", "http_proxy=%s" % self.proxy_url, "uvt-simplestreams-libvirt", "sync", ] + self.get_simplestreams_filter() run_command(command, check_call=True) logging.info( "Done downloading KVM image.") def generate_ssh_key(self): """Set up an ssh key pair for accessing the virtual machine. The key pair will be stored in the SSH key directory `ssh_key_dir`. If this did not exist yet, it will be created. """ key_dir = os.path.dirname(self.ssh_private_key_file) if not os.path.exists(key_dir): os.makedirs(key_dir, mode=0o700) if not os.path.exists(self.ssh_private_key_file): run_command( [ 'ssh-keygen', '-t', 'rsa', '-f', self.ssh_private_key_file, '-N', '', ], check_call=True) def wait_for_vm(self): """Wait for the virtual machine to come up.""" # Block until uvtool gives us the machine. This really only blocks # until its SSH server comes up. logging.info("Waiting for the virtual machine to come up...") if self.kvm_timeout is not None: command = [ 'uvt-kvm', 'wait', '--timeout', unicode(self.kvm_timeout), self.name] else: command = ['uvt-kvm', 'wait', self.name] run_command(command, check_call=True) logging.info("Virtual machine is running.") def wait_for_cloudinit(self): """Wait for cloud-init to finish its work on the virtual machine.""" logging.info("Waiting for cloud-init to finish its work...") for _ in retries(timeout=CLOUDINIT_TIMEOUT.total_seconds(), delay=5): return_code, _, stderr = self.run_command( ['test', '-f', CLOUDINIT_COMPLETION_MARKER]) if return_code == 0: logging.info("Cloud-init run finished.") return # If the file does not exist, "test" returns 1. If we get some # other nonzero return code, that means there was definitely a # real error. In which case, let's not keep the user waiting for # the full timeout period. if return_code != 1: raise RuntimeError( "Error contacting virtual machine %s. " "Error output was: '%s'" % (self.name, stderr)) raise RuntimeError( "Cloud-init initialization on virtual machine %s timed out. " "Error output was: '%s'" % (self.name, stderr)) def start(self): """Create and start the virtual machine. When this method returns, the VM is ready to be used.""" logging.info( "Creating virtual machine %s, arch=%s...", self.name, self.architecture) template = make_kvm_template(self.direct_interface) command = [ "sudo", "uvt-kvm", "create", "--ssh-public-key-file=%s" % self.get_ssh_public_key_file(), "--unsafe-caching", # On i386 hosts, qemu errors when the memory is > 2047 MB. "--memory", "2047", "--disk", "20", self.name ] + self.get_simplestreams_filter() command = command + ["--template", "-"] run_command(command, input=template, check_call=True) self.running = True logging.info( "Done creating virtual machine %s, arch=%s.", self.name, self.architecture) def configure_archives(self): """Add repositories to the VM and update the list of packages.""" if self.archives is not None: for archive in self.archives: self.run_command( ["sudo", "add-apt-repository", "-y", archive], check_call=True) self.run_command( ["sudo", "apt-get", "update"], check_call=True) def configure_network(self): """Configure the virtual machine network. If this fixture was configured to create a bridged interface on the VM, this method configures the network: it configures and brings up the bridged network interface, eth1. """ # Configure the bridged network if self.direct_interface is not # None. if self.direct_interface is not None: logging.info( "Configuring network interface on virtual machine %s...", self.name) network_config_snippet = make_network_interface_config( DIRECT_INTERFACE, self.direct_ip, self.direct_network, NATED_INTERFACE) self.run_command( ["sudo", "tee", "-a", "/etc/network/interfaces"], input=network_config_snippet, check_call=True) self.run_command( ["sudo", "ifup", DIRECT_INTERFACE], check_call=True) logging.info( "Done configuring network interface on virtual machine %s", self.name) def destroy(self): """Destroy the virtual machine.""" logging.info("Destroying virtual machine %s...", self.name) run_command( ["sudo", "uvt-kvm", "destroy", self.name], check_call=True) self.running = False logging.info("Done destroying virtual machine %s.", self.name) def _get_base_ssh_options(self): """Return a list of ssh options for connecting to the VM. These are minimal options that are needed to connect at all, whether using `ssh` or `scp`. """ return [ "-i", self.ssh_private_key_file, "-o", "LogLevel=quiet", "-o", "UserKnownHostsFile=/dev/null", "-o", "StrictHostKeyChecking=no", ] def _make_ssh_command(self, command_line): """Compose an `ssh` command line to execute `command_line` in the VM. :param command_line: Shell command to be executed on the virtual machine, in the form of a sequence of arguments (starting with the executable). :return: An `ssh` command line to execute `command_line` on the virtual machine, in the form of a list. """ remote_command_line = ' '.join(map(quote, command_line)) return [ "ssh", ] + self._get_base_ssh_options() + [ self.identity(), "LC_ALL=%s" % VM_LOCALE, remote_command_line, ] def _make_apt_get_install_command(self, packages): """Install the given packages on the virtual machine.""" return [ 'sudo', 'http_proxy=%s' % self.proxy_url, 'https_proxy=%s' % self.proxy_url, 'DEBIAN_FRONTEND=noninteractive', 'apt-get', 'install', '-y' ] + list(packages) def install_packages(self, packages): """Install the given packages on the virtual machine.""" self.run_command( self._make_apt_get_install_command(packages), check_call=True) def install_base_packages(self): """Install the packages needed for this instance to work.""" # Instal nmap so that 'get_ip_from_network_scan' can work. self.install_packages(['nmap']) def run_command(self, args, input=None, check_call=False): """Run the given command in the VM.""" args = self._make_ssh_command(args) retcode, stdout, stderr = run_command( args, check_call=check_call, input=input) count = next(self.command_count) cmd_prefix = 'cmd #%04d' % count self.addDetail(cmd_prefix, text_content(' '.join(args))) self.addDetail(cmd_prefix + ' retcode', text_content(str(retcode))) input_detail = '' if input is None else input self.addDetail(cmd_prefix + ' input', text_content(input_detail)) u_stdout = unicode(stdout, errors='replace') self.addDetail(cmd_prefix + ' stdout', text_content(u_stdout)) u_stderr = unicode(stderr, errors='replace') self.addDetail(cmd_prefix + ' stderr', text_content(u_stderr)) return retcode, stdout, stderr def upload_file(self, source, dest=None): """Copy the given file into the virtual machine's filesystem. :param source: File to copy. :param dest: Optional destination. May be an absolute path, or one relative to the `ubuntu` user's home directory. Defaults to the same base name as `source`, directly in the user's home directory. """ if dest is None: dest = os.path.basename(source) run_command( ['scp'] + self._get_base_ssh_options() + [source, dest], check_call=True) def get_ip_from_network_scan(self, mac_address): """Return the IP address associated with a MAC address. The IP address is found by scanning the direct network using nmap. """ network_repr = "%s" % self.direct_network nmap_scan_cmd = ['sudo', 'nmap', '-sP', network_repr, '-oX', '-'] _, output, _ = self.run_command( nmap_scan_cmd, check_call=True) mapping = extract_mac_ip_mapping(output) ip = mapping.get(mac_address.upper()) if ip is not None: return ip return None KVM_TEMPLATE = """ hvm """ KVM_DIRECT_INTERFACE_TEMPLATE = """ """ def make_kvm_template(direct_interface=None): """Return a KVM template suitable for 'uvt-kvm create'. :param direct_interface: An optional interface name. If provided, the returned template will include provision for a direct attachment of the virtual machine's NIC to the named physical interface. See http://libvirt.org/formatdomain.html#elementsNICSDirect for details. """ parser = etree.XMLParser(remove_blank_text=True) xml_template = etree.fromstring(KVM_TEMPLATE, parser) if direct_interface is not None: xml_interface_template = etree.fromstring( KVM_DIRECT_INTERFACE_TEMPLATE % direct_interface, parser) interface_tags = xml_template.xpath("/domain/devices/interface") interface_tags[-1].addnext(xml_interface_template) return etree.tostring(xml_template) NETWORK_CONFIG_TEMPLATE = textwrap.dedent(""" auto %(direct_interface)s iface %(direct_interface)s inet static address %(ip_address)s network %(network)s netmask %(netmask)s broadcast %(broadcast)s post-up sysctl -w net.ipv4.ip_forward=1 post-up iptables -t nat -A POSTROUTING -o %(interface)s -j MASQUERADE post-up iptables -A FORWARD -i %(interface)s -o %(direct_interface)s \ -m state --state RELATED,ESTABLISHED -j ACCEPT post-up iptables -A FORWARD -i %(direct_interface)s -o %(interface)s \ -j ACCEPT """) def make_network_interface_config(direct_interface, ip_address, network, interface): """Return a network configuration snippet. The returned snippet defines a fixed-IP address network configuration for the given interface suitable for inclusion in /etc/network/interfaces. """ return NETWORK_CONFIG_TEMPLATE % { 'interface': interface, 'ip_address': ip_address, 'network': network.network, 'netmask': network.netmask, 'broadcast': network.broadcast, 'direct_interface': direct_interface, } maas-test-0.1+bzr147.orig/maastest/maas_enums.py0000644000000000000000000000162012321262143017700 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Enumerations copied from MAAS. These don't have to be complete, but anything they do define must be consistent with the matching enum definitions in the MAAS source tree itself. """ from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [ 'NODEGROUPINTERFACE_MANAGEMENT', 'NODE_STATUS', ] class NODE_STATUS: """The vocabulary of a `Node`'s possible statuses.""" DECLARED = 0 COMMISSIONING = 1 FAILED_TESTS = 2 MISSING = 3 READY = 4 RESERVED = 5 ALLOCATED = 6 RETIRED = 7 class NODEGROUPINTERFACE_MANAGEMENT: """The vocabulary of a `NodeGroupInterface`'s possible statuses.""" UNMANAGED = 0 DHCP = 1 DHCP_AND_DNS = 2 maas-test-0.1+bzr147.orig/maastest/maasfixture.py0000644000000000000000000003701612321262143020110 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Manage a MAAS installation.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [ 'MAASFixture', ] from datetime import timedelta import httplib import json import logging from random import choice from string import ascii_letters from textwrap import dedent from apiclient.creds import convert_string_to_tuple from apiclient.maas_client import ( MAASClient, MAASDispatcher, MAASOAuth, urllib2, ) from fixtures import Fixture from maastest import utils from maastest.maas_enums import NODEGROUPINTERFACE_MANAGEMENT import netaddr from testtools.monkey import MonkeyPatcher MAAS_ADMIN_USER = 'admin' def compose_rewrite_for_default_maas_url(config_file, default_maas_url): """Return a shell command to rewrite `DEFAULT_MAAS_URL` in config.""" # Comment out existing DEFAULT_MAAS_URL, and add new one. replacement = dedent(""" # Replaced by maas-test: # \\1 DEFAULT_MAAS_URL = "%s" """ % default_maas_url).replace('\n', '\\n') return [ 'sed', '--in-place=.bak', 's|^\\(DEFAULT_MAAS_URL *=.*\\)$|%s|' % replacement, config_file, ] # Log files that are interesting when debugging MAAS issues. They will # be collected just before the fixture is disposed of. LOG_FILES = [ # Syslog contains DHCP requests. '/var/log/syslog', # DHCP lease file. '/var/lib/maas/dhcp/dhcpd.leases', # MAAS logs. '/var/log/maas/maas.log', '/var/log/maas/pserv.log', '/var/log/maas/celery-region.log', '/var/log/maas/celery.log', # Apache logs. '/var/log/apache2/access.log', '/var/log/apache2/error.log', ] class NoBootImagesError(Exception): """No boot images present on the master cluster.""" class MAASFixture(Fixture): """A fixture for a MAAS server.""" def __init__(self, kvmfixture, proxy_url, series, architecture, simplestreams_filter): """Initialise a MAAS server installed in a KVM instance. :param kvmfixture: The `maastest.KVMFixture` corresponding to the KVM instance on which the MAAS server will be installed. :param series: The Ubuntu series that this MAAS server should use to enlist and commission nodes. :param architecture: The architecture that this MAAS server should import images for, e.g. 'amd64', 'arm/highbank'. If the subarchitecture isn't specified, 'generic' is assumed. """ self.kvm_fixture = kvmfixture self.proxy_url = proxy_url self.series = series self.simplestreams_filter = simplestreams_filter self.architecture = architecture self.installed = False self._maas_admin = None def install_maas(self): """Install and configure MAAS in the virtual machine. This method is idempotent. It's OK to run it more than once. """ if self.installed: # Already installed. Nothing to do. return # Install the English language pack first. If this is not done # before postgres is installed, on some systems it won't be able to # set up its main cluster. # TODO: Is there no better way to ensure this, e.g. through a # dependency? self.kvm_fixture.install_packages(['language-pack-en']) maas_version = self.get_maas_version() logging.info("Installing MAAS (version %s)..." % maas_version) # Now we can install maas (which also installs postgres). self.kvm_fixture.install_packages(['maas', 'maas-dhcp', 'maas-dns']) logging.info("Done installing MAAS.") self.installed = True def get_maas_version(self): """Return the version of MAAS that is to be installed.""" _, policy, _ = self.kvm_fixture.run_command( ['apt-cache', 'policy', 'maas'], check_call=True) return utils.extract_package_version(policy) @property def maas_admin(self): """Return the correct maas admin command. If maas-region-admin is available, use it (on trusty or later), otherwise fall back to the old 'maas' command. """ if self._maas_admin is None: retcode, _, _ = self.kvm_fixture.run_command( ['which', 'maas-region-admin']) if retcode == 0: self._maas_admin = 'maas-region-admin' else: self._maas_admin = 'maas' return self._maas_admin def configure_default_maas_url(self): """Set `DEFAULT_MAAS_URL` in the virtual machine. We'd prefer to do this by preseed, before installing MAAS, but that doesn't currently work (bug 1251175). Instead, we patch up configuration after installation. """ assert self.kvm_fixture.direct_ip is not None, ( "configure_default_maas_url should only be called on a " "machine that has a 'direct interface.'") rewrite_command = compose_rewrite_for_default_maas_url( '/etc/maas/maas_local_settings.py', 'http://%s/MAAS' % self.kvm_fixture.direct_ip) self.kvm_fixture.run_command( ['sudo'] + rewrite_command, check_call=True) # Restarting apache2 sometimes times out: retry the command a # couple of times before giving up. See bug #1260363. args = ['sudo', 'service', 'apache2', 'restart'] for retry in utils.retries(delay=10, timeout=60): retcode, stdout, stderr = self.kvm_fixture.run_command( args, check_call=False) if retcode == 0: break else: raise utils.make_exception(args, retcode, stdout, stderr) def query_api_key(self, username): """Return the API key for the given MAAS user.""" # The "apikey" command prints the user's API key to stdout. return_code, stdout, stderr = self.kvm_fixture.run_command([ 'sudo', self.maas_admin, 'apikey', '--username=%s' % username, ], check_call=True) return stdout.strip() def create_maas_admin(self): """Create a MAAS admin user. Invoke this after MAAS has been installed on the virtual machine. :return: User name and password for the new admin. """ username = MAAS_ADMIN_USER password = ''.join(choice(ascii_letters) for counter in range(8)) # Email address really does not matter. MAAS never sends anything # there. Its hostname doesn't have to resolve, although it has to # look like a FQDN. email = 'root@localhost.local' return_code, _, stderr = self.kvm_fixture.run_command([ 'sudo', self.maas_admin, 'createadmin', '--username=%s' % username, '--password=%s' % password, '--email=%s' % email, ], check_call=True) return username, password def dump_data(self): """Dump the NodeCommissionResult table to stdout. This allows us to capture the lshw results for commissioned nodes. This method will be called when the fixture get cleaned up. """ return_code, stdout, _ = self.kvm_fixture.run_command([ 'sudo', self.maas_admin, 'dumpdata', 'metadataserver.NodeCommissionResult' ]) def import_maas_images(self, series, architecture, simplestreams_filter=None): """Import boot images into the MAAS instance. This download gigabytes of data to the virtual machine's disk. :param series: The Ubuntu series for which images should be imported. :param archtecture: The architecture for which images should be imported, e.g. 'amd64', 'arm/hightbank'. If the subarchitecture isn't specified, 'generic' is assumed. """ arch_list = utils.mipf_arch_list(architecture) arch_string = ' '.join(arch_list) logging.info( "Importing boot images series=%s, architectures=%s..." % ( series, arch_string)) # Import boot images. # XXX jtv 2014-04-03: Configure /etc/maas/bootresources.yaml to import # just the right series & architecture. self.kvm_fixture.run_command( [ 'sudo', 'http_proxy=%s' % self.proxy_url, 'https_proxy=%s' % self.proxy_url, 'maas-import-pxe-files', ], check_call=True) logging.info("Done importing boot images.") def wait_until_boot_images_scanned(self): """Wait until the master cluster has reported boot images.""" # Poll until the boot images have been reported. timeout = timedelta(minutes=10).total_seconds() for _ in utils.retries(timeout=timeout, delay=10): boot_images = self.list_boot_images() if len(boot_images) > 0: return raise NoBootImagesError( "Boot image download timed out: cluster reported no images.") def list_boot_images(self): """Return the boot images of the master cluster.""" ng_uuid = self.get_master_ng_uuid() uri = utils.get_uri('nodegroups/%s/boot-images/' % ng_uuid) response = self.admin_maas_client.get(uri) if response.code != httplib.OK: raise Exception( "Error getting boot images for cluster '%s'" % ng_uuid) return json.loads(response.read()) def configure(self, name, value): """Set a config value in MAAS.""" uri = utils.get_uri('maas/') response = self.admin_maas_client.post( uri, op='set_config', name=name, value=value) if response.code != httplib.OK: # TODO: include the response's content in the exception # message, here and in other places in this file. raise Exception("Error configuring '%s'" % name) def configure_default_series(self, series): """Set the default series used for enlistment and commissioning.""" self.configure('commissioning_distro_series', series) def configure_http_proxy(self, proxy_url): """Set the proxy to be used by MAAS.""" self.configure('http_proxy', proxy_url) def setUp(self): """Install and configure MAAS inside the virtual machine. :param kvm_fixture: `KVMFixture` that controls the virtual machine. :return: Username, password and API key for the MAAS admin user. """ super(MAASFixture, self).setUp() # MAAS installation and configuration. self.install_maas() if self.kvm_fixture.direct_ip is not None: self.configure_default_maas_url() # Admin creation. admin_user, admin_password = self.create_maas_admin() api_key = self.query_api_key(admin_user) maas_client = self.get_maas_api_client(api_key) self.admin_user = admin_user self.admin_password = admin_password self.admin_api_key = api_key self.admin_maas_client = maas_client # MAAS setup. if self.proxy_url: # None or '' indicates no proxy. self.configure_http_proxy(self.proxy_url) # We should have a working MAAS at this point, so if things break from # here on, try to provide relevant information for debugging. self.log_connection_details() self.addCleanup(self.collect_logs) self.addCleanup(self.dump_data) # Now do the rest of the setup: import images, configure the # controllers, and so on. self.import_maas_images( series=self.series, architecture=self.architecture, simplestreams_filter=self.simplestreams_filter) self.wait_until_boot_images_scanned() self.configure_default_series(self.series) if self.kvm_fixture.direct_ip is not None: self.check_cluster_connected() self.configure_cluster() def log_connection_details(self): """Log the details on how to connect to this MAAS server.""" logging.info( "MAAS server URL: http://%s/MAAS/ username:%s, password:%s" % ( self.kvm_fixture.ip_address(), self.admin_user, self.admin_password, ) ) logging.info( "SSH login: sudo ssh -i %s %s" % ( self.kvm_fixture.ssh_private_key_file, self.kvm_fixture.identity(), ) ) def collect_logs(self): for filename in LOG_FILES: # The result of every command run on the VM goes through # testtools.TestCase.addDetail() so we just need to # 'cat' the files to get their content included in the # report. self.kvm_fixture.run_command([ 'sudo', 'cat', filename], check_call=False) def get_maas_api_client(self, api_key): """Create and return a MAASClient. :param kvm_fixture: `KVMFixture` that controls the virtual machine. :return: `MAASClient` instance for the MAAS admin user. """ credentials = convert_string_to_tuple(api_key) auth = MAASOAuth(*credentials) return MAASClient( auth, MAASDispatcher(), "http://%s/MAAS" % self.kvm_fixture.ip_address()) def get_master_ng_uuid(self): """Get the UUID of the master nodegroup from the API.""" uri = utils.get_uri('nodegroups/') response = self.admin_maas_client.get(uri, op='list') if response.code != httplib.OK: raise Exception("Error listing the clusters") nodegroups = json.loads(response.read()) if len(nodegroups) != 1: raise Exception( "Expected exactly 1 nodegroup, but saw %d." % len(nodegroups)) return nodegroups[0]['uuid'] def check_cluster_connected(self): for retry in utils.retries(timeout=3 * 60): name = self.get_master_ng_uuid() if name != 'master': # The master cluster is connected. The master nodegroup # had its uuid field updated from 'master' to its UUID. return raise Exception("Master cluster failed to connect.") def configure_cluster(self): network = self.kvm_fixture.direct_network first_ip = self.kvm_fixture.direct_first_available_ip() last_ip = "%s" % netaddr.IPAddress( self.kvm_fixture.direct_network.last - 1) dhcp_config = { "ip": self.kvm_fixture.direct_ip, "interface": "eth1", "subnet_mask": "%s" % network.netmask, "broadcast_ip": "%s" % network.broadcast, "router_ip": self.kvm_fixture.direct_ip, "management": '%s' % NODEGROUPINTERFACE_MANAGEMENT.DHCP_AND_DNS, "ip_range_low": first_ip, "ip_range_high": last_ip, } uuid = self.get_master_ng_uuid() uri = utils.get_uri( 'nodegroups/%s/interfaces/%s/' % (uuid, 'eth1')) # XXX: rvb 2013-11-14 bug=1251214: workaround a bug affecting # 'PUT' requests in urllib2 (which is used by MAASClient). patcher = MonkeyPatcher() patcher.add_patch(urllib2.Request, "get_method", lambda self: 'PUT') response = patcher.run_with_patches( self.admin_maas_client.put, uri, **dhcp_config) if response.code != httplib.OK: raise Exception("Error configure the master cluster") maas-test-0.1+bzr147.orig/maastest/main.py0000644000000000000000000003060712321262143016503 0ustar 00000000000000# Copyright 2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Test whether or not a node is compatible with MAAS.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "main", ] import logging import os import sys from maastest import utils from testtools.testresult.real import _details_to_str # Most imports are done as needed in main() to improve responsiveness # when maas-test is run interactively, especially when --help is passed. class RETURN_CODES: """Return codes for this program. Not particularly useful in practice, but they help the test suite check for the right error diagnosis. """ SUCCESS = 0 TEST_FAILED = 1 NOT_ROOT = 2 NO_KVM_EXTENSIONS = 3 UNEXPECTED_DHCP_SERVER = 4 class ProgramFailure(Exception): """A known kind of program failure. These are fatal, program-level failures that we know how to present to the user, including which error code to return. :ivar return_code: Return code, from `RETURN_CODES`, which the program should return to the shell. """ def __init__(self, return_code, message): super(ProgramFailure, self).__init__(message) self.return_code = return_code def check_for_root(): """Verify that the program is being run as root. :raise ProgramFailure: if the program is not running as root. """ if os.geteuid() != 0: raise ProgramFailure( RETURN_CODES.NOT_ROOT, "This test must be run as root. Use sudo.") def check_for_virtualisation_support(): """Verify that the system supports efficient virtual machines. :raise ProgramFailure: if the CPU lacks virtualisation support, or support has been disabled in firmware. """ if not utils.check_kvm_ok(): raise ProgramFailure( RETURN_CODES.NO_KVM_EXTENSIONS, "Unable to continue without KVM extensions.") def check_against_virtual_host(): """Warn if the program is running in a virtual machine. For acceptable performance, maas-test should be run on a physical system. This function warns, but does not fail, if the system is virtual. """ virt_type = utils.virtualization_type() if virt_type is not None: logging.info( "This machine is running on %s virtual hardware." % virt_type) logging.warning( "Running maas-test on this machine will result in significantly " "poorer performance than on physical hardware.") def check_against_dhcp_servers(interface): """Check for unexpected DHCP servers on the testing network. :param interface: Network interface to the testing network. This will send out a DHCP discovery request on that interface. :raise ProgramFailure: if any DHCP servers were detected. """ # Importing locally for responsiveness. from maastest.detect_dhcp import probe_dhcp dhcp_servers = probe_dhcp(interface) if len(dhcp_servers) > 0: raise ProgramFailure( RETURN_CODES.UNEXPECTED_DHCP_SERVER, "DHCP server(s) detected on %s: %s. " "Pass an interface that is connected to the testing network. " "Ensure that the testing network connects only to the node and " "its BMC, and has no DHCP service running. " "See the maas-test manual for details." % (interface, ', '.join(sorted(dhcp_servers)))) def has_maas_probe_dhcp(machine): """Does the given virtual machine have `maas-probe-dhcp` installed?""" return_code, _, _ = machine.run_command(['which', 'maas-probe-dhcp']) return (return_code == 0) def check_against_dhcp_servers_from_vm(machine): """Check for unexpected DHCP servers, from within the virtual machine. This complements `check_against_dhcp_servers`, adding detection for two situations that it doesn't handle: - The interface has no IP address (the VM's interface gets a static one). - The DHCP server is running on the same interface we use. This second DHCP check uses the `maas-probe-dhcp` script that is installed on the VM as part of MAAS. :param machine: Virtual-machine fixture, with MAAS installed. :raise ProgramFailure: if any DHCP servers were detected. """ if machine.direct_interface is None: # We need a direct interface to perform this check. return if not has_maas_probe_dhcp(machine): # Not all released MAAS versions have the maas-probe-dhcp tool. If we # don't have it, skip this check. return # Importing locally for responsiveness. from maastest.kvmfixture import DIRECT_INTERFACE return_code, stdout, stderr = machine.run_command( ['sudo', 'maas-probe-dhcp', DIRECT_INTERFACE], check_call=False) if return_code == 0: # OK. No DHCP servers detected. return # Importing locally for responsiveness. import re match = re.match("DHCP servers detected: (.*)$", stdout) if match is None: # Not the output we expect when maas-probe-dhcp detects DHCP servers, # so either something else went wrong, or the output has changed. raise Exception( "Call to maas-probe-dhcp failed in virtual machine: '%s'" % stderr) ips = match.group(1) if ips == machine.direct_ip: # A DHCP server was detected, but it's running on the virtual machine's # bridged IP address. It's probably just the MAAS DHCP server running # in the virtual machine itself. return # The virtual machine detected a DHCP server that we missed before. raise ProgramFailure( RETURN_CODES.UNEXPECTED_DHCP_SERVER, "The virtual machine detected a DHCP server on the network: %s. " "Pass an interface that is connected to the testing network. " "Ensure that the testing network connects only to the node and " "its BMC, and has no DHCP service running. " "See the maas-test manual for details." % ips) def make_local_proxy_fixture(): """Create a `LocalProxyFixture`. Defined here so that the import can be deferred for responsiveness, while still allowing tests to patch it. """ from maastest.proxyfixture import LocalProxyFixture return LocalProxyFixture() def set_up_proxy(args): """Obtain or set up an http/https proxy, as appropriate. The command-line arguments may disable this, or make it use an existing proxy, or require us to set up our own. This function will do whichever fits, and set up the process environment as needed. :return: A tuple of (proxy URL, proxy fixture). The fixture may be `None`. If it is not `None`, it will need to be cleaned up later. The proxy URL may be the empty string if no proxy is to be used. """ proxy_fixture = None proxy_is_set_up = False try: if args.http_proxy is not None: # The user has passed an external proxy; don't start polipo, # just use that proxy for everything. proxy_url = args.http_proxy logging.info("Using external proxy %s." % proxy_url) elif args.disable_cache: # The user has passed --disable-cache, so don't start polipo # and don't try to use an external proxy. proxy_url = '' logging.info("Caching disabled.") else: # The default case: start polipo and use that as our caching # proxy. proxy_fixture = make_local_proxy_fixture() proxy_fixture.setUp() proxy_is_set_up = True proxy_url = proxy_fixture.get_url() if proxy_url != '': os.putenv('http_proxy', proxy_url) os.putenv('https_proxy', proxy_url) except: if proxy_is_set_up: # We have a proxy fixture set up, but we're not going to return. # That means nobody else is responsible for cleaning it up. proxy_fixture.cleanUp() raise return proxy_url, proxy_fixture def create_vm(args, proxy_url=None): """Create the virtual machine fixture, but do not set it up yet. :param args: Arguments object as returned by the arguments parser. :param proxy_url: Optional http/https proxy URL for the VM to use, as returned by `set_up_proxy`. :return: Virtual-machine fixture. If you begin to set it up, clean it up as well once you're done. """ # Importing locally for responsiveness. from maastest.kvmfixture import KVMFixture # TODO: series and architecture should be script parameters. architecture = utils.determine_vm_architecture() fixture = KVMFixture( series=args.maas_series, architecture=architecture, proxy_url=proxy_url, direct_interface=args.interface, archives=args.archive, kvm_timeout=args.kvm_timeout, simplestreams_filter=args.maas_simplestreams_filter) return fixture def install_maas(args, machine, proxy_url): """Install MAAS in the virtual machine. :return: MAAS fixture. It still needs to be set up, but it will already have MAAS installed in the virtual machine. Clean up after use. """ # Importing locally for responsiveness. from maastest.maasfixture import MAASFixture fixture = MAASFixture( machine, proxy_url=proxy_url, series=args.series, architecture=args.architecture, simplestreams_filter=args.simplestreams_filter) fixture.install_maas() return fixture def main(args): # We tie everything onto one output stream so that we can capture # things for reporting. output_stream = utils.CachingOutputStream(sys.stdout) logging.basicConfig( level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s', stream=output_stream) # TODO: Use Python 3's contextlib.ExitStack. Not too hard to fake in 2.7. proxy_fixture = None machine_fixture = None maas_fixture = None maas_is_set_up = False try: check_for_root() check_for_virtualisation_support() check_against_virtual_host() check_against_dhcp_servers(args.interface) proxy_url, proxy_fixture = set_up_proxy(args) machine_fixture = create_vm(args, proxy_url) machine_fixture.setUp() maas_fixture = install_maas(args, machine_fixture, proxy_url) # Before we go on to download PXE images etc. use the virtual MAAS # installation to check for DHCP servers. check_against_dhcp_servers_from_vm(machine_fixture) # Now go through the rest of MAAS setup, and the actual tests. maas_fixture.setUp() maas_is_set_up = True from maastest.cases import TestOneNode class ConfiguredTestMAAS(TestOneNode): """A configured version of TestInteractiveOneNode. ConfiguredTestMAAS is a TestMAAS use to configure it by calling cls.configure. We need to configure the class itself and not the instance because the KVMFixture TestMAAS instantiates and needs to configure is created at the class level. """ ConfiguredTestMAAS.configure(args, maas_fixture) from maastest.console import run_console result = run_console(ConfiguredTestMAAS, output_stream) from maastest import report encoded_results = ( output_stream.cache.getvalue().encode('utf-8')) if result.wasSuccessful(): # If the test succeeded, we have to gather the details by # hand and include them with the output. Testtools won't do # it it for us. details = _details_to_str( ConfiguredTestMAAS.details).encode('utf-8') encoded_results = encoded_results + details if not args.no_reporting: report.write_test_results(encoded_results) if not args.no_reporting and not args.log_results_only: report.report_test_results(encoded_results, result.wasSuccessful()) if result.wasSuccessful(): return RETURN_CODES.SUCCESS else: return RETURN_CODES.TEST_FAILED except ProgramFailure as e: logging.error(e) return e.return_code finally: if maas_is_set_up: maas_fixture.cleanUp() if machine_fixture is not None: machine_fixture.cleanUp() if proxy_fixture is not None: proxy_fixture.cleanUp() maas-test-0.1+bzr147.orig/maastest/parser.py0000644000000000000000000001713712321262143017056 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Argument parser for `maastest.main`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "prepare_parser", ] import argparse import distro_info from maastest import utils from six import text_type class MAASTestArgumentParser(argparse.ArgumentParser): def parse_args(self, *args, **kwargs): args = super(MAASTestArgumentParser, self).parse_args(*args, **kwargs) bmc_details = (args.bmc_mac, args.bmc_username, args.bmc_password) if args.interactive: if bmc_details != (None, None, None): self.error( "None of the BMC details should be provided when using " "--interactive.") else: if args.bmc_mac is not None and args.bmc_ip is not None: self.error( "BMC MAC address and IP address cannot be both provided") bmc_details_all_set = ( (args.bmc_mac is not None or args.bmc_ip is not None) and args.bmc_username is not None and args.bmc_password is not None) if not bmc_details_all_set: self.error( "All the BMC details (MAC address or IP address, username " "and password) must be provided.") return args latest_LTS_series = distro_info.UbuntuDistroInfo().lts() latest_released_series = distro_info.UbuntuDistroInfo().stable() def prepare_parser(): # TODO: This is not tested sinc how maas-test will interface with # checkbox is still in flux. Right now, this is just to get the # testing going. parser = MAASTestArgumentParser( description="Test whether or not a node is compatible with MAAS.") parser.add_argument( 'interface', type=text_type, help="Network interface that connects the testing system to the " "dedicated testing network. MAAS will serve DHCP on this " "interface.") parser.add_argument( '--archive', type=text_type, default=None, action='append', help="Additional personal package archive, 'sources.list' entry, or " "distribution component to install on the virtual MAAS system. " "This can be specified multiple times.") parser.add_argument( '--interactive', action='store_true', help="Interactive mode. Prompt the user to power up the node " "manually instead of doing it automatically through the BMC.") # BMC details. parser.add_argument( '--bmc-mac', type=text_type, help="MAC address of the node's baseboard management controller. " "Use this if the BMC is connected to the interface given as " "argument. In non-interactive mode, either --bmc-mac or " "--bmc-ip. Not needed in interactive mode.") parser.add_argument( '--bmc-ip', type=text_type, help="IP address of the node's baseboard management controller. " "Use this if the BMC is not connected to the inteface given as " "argument. Note that the IP address must not change for the " "duration of the testing. In non-interactive mode, either " "--bmc-mac or --bmc-ip. Not needed in interactive mode.") parser.add_argument( '--bmc-username', type=text_type, help="Username for authenticating to the node's BMC. " "Not needed in interactive mode.") parser.add_argument( '--bmc-password', type=text_type, help="Password for authenticating to the node's BMC. " "Not needed in interactive mode.") parser.add_argument( '--ipmi-driver', type=text_type, default='LAN_2_0', help="IPMI driver type for the node's BMC. Defaults to LAN_2_0 " "(IPMI v2.0). " "Other option is LAN (IPMI version 1.5). Many BMCs can work " "with either IPMI 1.5 or 2.0, but, some BMCs such as HP's iLO " "only use IPMI 2.0. " "You may need to pass LAN if your BMC only supports IPMI 1.5.") # VM details. parser.add_argument( '--maas-series', type=text_type, choices=utils.get_supported_maas_series(), default=utils.get_supported_maas_series()[0], help="Code name for the Ubuntu release series to install on the " "virtual machine (where the MAAS server will be installed). " "Defaults to the latest stable Ubuntu series (%(default)s).") parser.add_argument( '--maas-simplestreams-filter', type=text_type, default=None, help="Optional simplestreams filter used to retrieve the image for " "the virtual machine (e.g. 'label=beta2 other_filter=value'). " "maas-test will set the value for the 'arch' filter and for " "the 'series' filter (see option '--maas-series'). This is " "an advanced option used to specify additional simplestreams " "filter.") # MAAS images. parser.add_argument( '--series', type=text_type, choices=utils.get_supported_node_series(), # Hardcode 'trusty' until it's effectively the latest LTS. # This is the series used for commissioning and deployment and # MAAS doesn't support commissioning with precise. default='trusty', help="Code name for the Ubuntu release series that the node should " "run. Defaults to the latest LTS (%(default)s).") parser.add_argument( '--simplestreams-filter', type=text_type, default=None, help="Optional simplestreams filter used to retrieve the ephemeral " "images MAAS downloads (e.g. 'label=beta2 other_filter=value'). " "MAAS will set the value for the 'arch' filter and for " "the 'series' filter (see option '--series'). This is " "an advanced option used to specify additional simplestreams " "filter.") parser.add_argument( '--architecture', type=text_type, default='amd64', help="Node's CPU architecture, e.g. armhf/highbank or amd64. " "Defaults to %(default)s.") # Proxy. parser.add_argument( '--http-proxy', type=text_type, help="HTTP proxy to use for all downloads. Implies --disable-cache.") parser.add_argument( '--disable-cache', action='store_true', help="By default, maas-test uses polipo to cache all KVM images, " "MAAS images and apt packages. If --disable-cache is specified, " "polipo will be disabled.") # Reporting. parser.add_argument( '--no-reporting', action='store_true', help="Log results only; do not store or submit them to Launchpad.") parser.add_argument( '--log-results-only', action='store_true', help="Store results, but do not submit them to Launchpad.") # Dry run parser.add_argument( '--dry-run', action='store_true', help="Don't attempt to boot any machines or do any destructive " "testing.") # VM Timeout parser.add_argument( # We use 300.0 as the default here because it's slightly longer # than uvtool's default 120 seconds. It's still not enough on # some virtual hardware, but we don't care too much about those. '--kvm-timeout', type=int, default=300.0, help="The number of seconds to wait for the maas-test virtual " "machine to come up.") return parser maas-test-0.1+bzr147.orig/maastest/proxyfixture.py0000644000000000000000000001442112321262143020343 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Test whether or not a node is compatible with MAAS.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "LocalProxyFixture", ] import errno import logging import os.path import re from fixtures import Fixture from maastest import utils from netaddr import IPNetwork import netifaces DISK_CACHE_ROOT = '%(config_dir)s/proxy_cache/' PROXY_CONFIG_ITEMS = { 'daemonise': 'true', 'logFile': '%(log_file)s', 'pidFile': '%(pid_file)s', 'proxyAddress': "::0", 'proxyPort': '%(port)d', 'allowedClients': '127.0.0.1,%(ip_address)s/%(netmask)s', 'diskCacheRoot': DISK_CACHE_ROOT } IP_ADDRESS_RE = re.compile( ".*inet\s+(\d+\.\d+\.\d+\.\d+)/(\d+)?.*", flags=re.DOTALL) class LocalProxyFixture(Fixture): """A test fixture for using a caching proxy in maas-test. If no external proxy URL is passed to __init__(), LocalProxyFixture spins up a local instance of polipo, a lightweight caching proxy. We use polipo here because it's particularly cheap to set up a mayfly instance; all of the configuration options can be passed on the command line, meaning that we don't need to care about maintaining a config file somewhere. If LocalProxyFixture is given an external proxy URL, it won't start a local proxy and will simply return the external URL when asked. """ #TODO: port should be determined at runtime from the set of # available ports rather than being an arbitrary one as it is here. def __init__(self, config_dir=None, port=42676, pidfile_dir=None, log_dir=None): """Create a `LocalProxyFixture` object. :param url: The URL of the proxy to use. If None, LocalProxyFixture will create a local caching proxy. :type url: string :param port: The port on which to start a local caching proxy. If URL is specified, this parameter is ignored. :type port: int :param config_dir: The directory in which to put the proxy's configuration and cache. :type config_dir: string :param pidfile_dir: Optional directory in which to write the proxy's pidfile. Will be created if not present. Defaults to `/run/maas-test`. :type pidfile_dir: string :param log_dir: Optional directory in which to write the proxy's log files. Will be created if not present. Defaults to `/var/log/maas-test`. :type log_dir: string """ super(LocalProxyFixture, self).__init__() self.port = port if config_dir is None: config_dir = utils.DEFAULT_STATE_DIR self.config_dir = config_dir if pidfile_dir is None: pidfile_dir = utils.DEFAULT_PIDFILE_DIR if log_dir is None: log_dir = utils.DEFAULT_LOG_DIR self.pid_file = os.path.join(pidfile_dir, 'proxy.pid') self.log_file = os.path.join(log_dir, 'proxy.log') self._network_address = None def _get_config_arguments(self): """Return the proxy config as a sorted list of key=value strings.""" ip_address = self._get_network_address() substitutions = { 'config_dir': self.config_dir, 'port': self.port, 'ip_address': ip_address.ip.format(), 'netmask': ip_address.prefixlen, 'pid_file': self.pid_file, 'log_file': self.log_file, } config_args = [] for key, value in PROXY_CONFIG_ITEMS.items(): arg_string = "%s=%s" % (key, value % substitutions) config_args.append(arg_string) return sorted(config_args) def _get_network_address(self): """Return an `IPNetwork` instance for the KVM virtual bridge. This allows us to determine which IP address(es) polipo should accept connections on. """ if self._network_address is None: ipv4_address = netifaces.ifaddresses( 'virbr0')[netifaces.AF_INET][0] self._network_address = IPNetwork( "%s/%s" % (ipv4_address['addr'], ipv4_address['netmask'])) return self._network_address def get_url(self): """Return the URL of the proxy.""" ip_address = self._get_network_address() return "http://%s:%d" % (ip_address.ip.format(), self.port) def setUp(self): """Configure the polipo caching http proxy.""" super(LocalProxyFixture, self).setUp() self.create_cache_root() # From this point, if further setup fails, we'll need to clean up. self.addCleanup(self.kill_running_proxy) # Create pidfile and log directories if they did not already exist. pidfile_dir = os.path.dirname(self.pid_file) if not os.path.isdir(pidfile_dir): os.makedirs(pidfile_dir, 0o755) log_dir = os.path.dirname(self.log_file) if not os.path.isdir(log_dir): os.makedirs(log_dir) self.start_proxy() def start_proxy(self): logging.info("Checking for running proxy instance...") if os.path.exists(self.pid_file): raise Exception( "Found pid file %s for running proxy process." % self.pid_file) logging.info("Starting proxy...") utils.run_command( ['polipo'] + self._get_config_arguments(), check_call=True) logging.info("Done starting proxy.") def create_cache_root(self): cache_root = DISK_CACHE_ROOT % {'config_dir': self.config_dir} try: os.makedirs(cache_root) except OSError as error: if error.errno != errno.EEXIST: raise # Directory already exists. def kill_running_proxy(self): """Kill a running proxy.""" logging.info("Killing proxy...") try: pid = utils.read_file(self.pid_file).strip() utils.run_command(['kill', pid], check_call=True) logging.info("Done killing proxy.") except Exception as error: logging.error("Unable to kill proxy:") logging.error(error.message) maas-test-0.1+bzr147.orig/maastest/report.py0000644000000000000000000001402712321262143017070 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Reporting functions for MAAS test.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "create_launchpad_blob", "upload_blob", ] from datetime import datetime import email.mime from io import BytesIO import logging import os from urllib2 import ( build_opener, HTTPSHandler, ProxyHandler, Request, ) from apiclient import multipart from maastest.utils import DEFAULT_LOG_DIR # Used to disable proxies when opening the connection to # Launchpad (see the XXX re bug 1267443 below). NO_PROXY_HANDLER = ProxyHandler(proxies={}) # This is largely cargo-culted from apiclient.multipart because # Launchpad only pays attention to the first Content-Disposition header # on an attachment, and if we want the content to actually be attached # to the bug rather than lost in the ether we have to make it # Content-Disposition: attachment. By default, # apiclient.multipart.make_file_payload() gives the payload a # Content-Disposition of "form-data". def make_file_payload(name, content): payload = email.mime.application.MIMEApplication(content) payload.add_header( "Content-Disposition", "attachment", name=name, filename=name) names = name, getattr(content, "name", None) payload.set_type(multipart.get_content_type(*names)) return payload def create_launchpad_blob(test_results, test_succeeded): """Create an RFC822-formatted blob of data to upload to Launchpad. """ launchpad_form_data = multipart.build_multipart_message([]) launchpad_form_data['private'] = 'yes' launchpad_form_data['subscribers'] = "private-canonical-maas" if test_succeeded: launchpad_form_data['subject'] = "maas-test success" launchpad_form_data['tags'] = 'success' else: launchpad_form_data['subject'] = "maas-test failure" launchpad_form_data['tags'] = 'failure' results_payload = make_file_payload( 'maas-test.log', test_results.encode('utf-8', 'replace')) launchpad_form_data.attach(results_payload) return launchpad_form_data # And this is largely cargo-culted from apport.crashdb_impl.launchpad, # because it does all the form-submission dance that we need to do to # make this actually work. def build_upload_mime_multipart(blob): launchpad_form_data = email.mime.multipart.MIMEMultipart() submit = email.mime.Text.MIMEText('1') submit.add_header( 'Content-Disposition', 'form-data; name="FORM_SUBMIT"') launchpad_form_data.attach(submit) form_blob_field = email.mime.Base.MIMEBase('application', 'octet-stream') form_blob_field.add_header( 'Content-Disposition', 'form-data; name="field.blob"; filename="x"') form_blob_field.set_payload(blob.as_string().encode('ascii')) launchpad_form_data.attach(form_blob_field) return launchpad_form_data def build_upload_request(url, launchpad_form_data): # These next three lines are cargo-culted from # apport.crashdb_impl.launchpad. Launchpad won't recognise the blob # as something it can parse if we supress the leading headers. data_flat = BytesIO() generator = email.generator.Generator(data_flat, mangle_from_=False) generator.flatten(launchpad_form_data) # do the request; we need to explicitly set the content type here, as it # defaults to x-www-form-urlencoded request = Request(url, data_flat.getvalue()) request.add_header( 'Content-Type', 'multipart/form-data; boundary=' + launchpad_form_data.get_boundary()) return request def upload_blob(blob, hostname='launchpad.net'): """Upload blob (file-like object) to Launchpad. :param blob: """ token = None url = 'https://%s/+storeblob' % hostname form = build_upload_mime_multipart(blob) request = build_upload_request(url, form) # XXX 2014-01-09 gmb bug=1267443: # We explicitly disable proxies here because connecting to # Launchpad fails if https_proxy is set in the environment (see # linked bug). opener = build_opener(HTTPSHandler, NO_PROXY_HANDLER) try: result = opener.open(request) except Exception: logging.exception("Unable to connect to Launchpad.") return None else: token = result.info().get('X-Launchpad-Blob-Token') return token def write_test_results(results, log_file_name=None, log_dir=None, now=None): """Write maas-test results to a file. :param results: The results from maas-test. :param log_file_name: The filename to which to write the results (used for testing only). :param log_dir: The directory under which to create `log_file_name`. :param now: The current date/time (used for testing only). :type now: datetime.datetime. """ if now is None: now = datetime.utcnow() if log_dir is None: log_dir = DEFAULT_LOG_DIR if not os.path.exists(log_dir): os.makedirs(log_dir) if log_file_name is None: log_file_name = ( "maas-test.%s.log" % now.strftime("%Y-%m-%d_%H:%M:%S")) log_file_path = os.path.join(log_dir, log_file_name) with open(log_file_path, 'w') as log_file: log_file.write(results) logging.info("Test log saved to %s." % log_file_path) def report_test_results(results, test_succeeded): """Handle the reporting of maas-test results. :param results: The results to report. :param test_succeeded: True if maas-test passed, False otherwise. """ blob = create_launchpad_blob(results, test_succeeded) blob_token = upload_blob(blob) if blob_token is not None: logging.info("Test results uploaded to Launchpad.") logging.info( "Visit https://bugs.launchpad.net/maas-test-reports/+filebug/%s " "to file a bug and complete the maas-test reporting process." % blob_token) else: logging.info("Unable to store test results on Launchpad.") maas-test-0.1+bzr147.orig/maastest/script.py0000755000000000000000000000137412321262143017065 0ustar 00000000000000#!/usr/bin/env python2.7 # Copyright 2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Test a node for compatibility with MAAS.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [] import sys from maastest.parser import prepare_parser def entry_point(): """Main entry point for `maas-test`.""" args = prepare_parser().parse_args() # If the user passed --help, we've already exited. This is where we can # import the rest of the code without hurting responsiveness for that case. from maastest.main import main return_code = main(args) sys.exit(return_code) maas-test-0.1+bzr147.orig/maastest/testing/0000755000000000000000000000000012321262143016654 5ustar 00000000000000maas-test-0.1+bzr147.orig/maastest/tests/0000755000000000000000000000000012321262143016341 5ustar 00000000000000maas-test-0.1+bzr147.orig/maastest/utils.py0000644000000000000000000002544312321262143016721 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """maas-test utilities.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [ "BINARY", "binary_content", "DEFAULT_STATE_DIR", "determine_vm_series", "determine_vm_architecture", "get_uri", "DEFAULT_PIDFILE_DIR", "read_file", "retries", "run_command", "CasesLoader", ] import inspect from io import BytesIO import logging from pipes import quote import platform import re from subprocess import ( PIPE, Popen, ) from time import ( sleep, time, ) import unittest import distro_info from lxml import etree from six.moves import input from testtools.content import Content from testtools.content_type import ContentType # Default location for maas-test state, such as SSH keys for the virtual # machine, and the http proxy cache. DEFAULT_STATE_DIR = '/var/cache/maas-test' # Default location for pidfiles. DEFAULT_PIDFILE_DIR = '/run/maas-test' # Default location for log files. DEFAULT_LOG_DIR = '/var/log/maas-test' def read_file(path): """Return a given file's contents, as `bytes`.""" with open(path, "rb") as f: return f.read() def make_exception(args, retcode, stdout, stderr): """Create an exception from the information about a failed command.""" cmd = " ".join(quote(arg) for arg in args) return Exception( "Command '%s' failed (%d):\n%s\n%s" % ( cmd, retcode, stdout.decode('utf8', errors='replace'), stderr.decode('utf8', errors='replace'))) def run_command(args, input=None, check_call=False): """A wrapper to Popen to run commands in the command-line.""" process = Popen(args, stdout=PIPE, stderr=PIPE, stdin=PIPE, shell=False) stdout, stderr = process.communicate(input) retcode = process.returncode if check_call and retcode != 0: raise make_exception(args, retcode, stdout, stderr) return retcode, stdout, stderr def get_uri(path): """GET an API V1 URI. :return: The API URI. """ api_root = '/api/1.0/' return api_root + path BINARY = ContentType("application", "octet-stream") def binary_content(data): """Create a `Content` object from a byte string. This is useful for adding details which are blobs of binary data. """ return Content(BINARY, lambda: [data]) def retries(timeout=30, delay=1): """Helper for retrying something, sleeping between attempts. Yields ``(elapsed, remaining)`` tuples, giving times in seconds. @param timeout: From now, how long to keep iterating, in seconds. @param delay: The sleep between each iteration, in seconds. """ start = time() end = start + timeout for now in iter(time, None): if now < end: yield now - start, end - now sleep(min(delay, end - now)) else: break class UnknownCPUArchitecture(RuntimeError): """Could not determine CPU architecture.""" def determine_vm_architecture(): """Figure out CPU architecture for the virtual machine that will run MAAS. :return: Architecture string, e.g. 'i386'. :raise UnknownCPUArchitecture: if the CPU architecture could not be determined. """ # Ubuntu sometimes uses different architecture names than we get from # the platform module. This dict translates for those cases. ubuntu_names = { 'i686': 'i386', 'x86_64': 'amd64', } raw_arch = platform.machine() if raw_arch == '': raise UnknownCPUArchitecture( "Could not determine system CPU architecture. " "Please report a bug against http://launchpad.net/maas-test " "with this system's details. To work around the problem, " "try the --vm-arch option.") return ubuntu_names.get(raw_arch, raw_arch) def determine_vm_series(): """Figure out the Ubuntu release series to run MAAS on. :return: Series codename, e.g. "precise" for 12.04 Precise Pangolin. """ distro, version, series = platform.linux_distribution() return series def get_supported_node_series(): """List of MAAS-supported Ubuntu series. These are the series that MAAS can deploy on the nodes it manages. """ # Return all series above and including Precise. return _get_supported_series('precise') def get_supported_maas_series(): """List of Ubuntu series for MAAS. These are the series on which maas-test can install a a MAAS instance. """ # Return all series above and including Trusty. return _get_supported_series('trusty') def _get_supported_series(cut_off_series): """Return the list of supported series, starting from the given `cut_off_series`. """ udi = distro_info.UbuntuDistroInfo() supported = udi.supported(result="codename") cut_off_index = ( supported.index(cut_off_series) if cut_off_series in supported else 0) return supported[cut_off_index:] class CasesLoader(unittest.TestLoader): """A test loader specialised for loading maas-test cases.""" sortTestMethodsUsing = property( doc="Unused; sorting is implemented in getTestCaseNames()") def getTestCaseNames(self, testCaseClass): """Return test case names in order of line number. This assumes that all test methods have been defined in the *same file*, and in the order they should be run. """ def getlineno(method): _, lineno = inspect.findsource(method) return lineno method_starts = { name: getlineno(method) for name, method in inspect.getmembers(testCaseClass, inspect.ismethod) if name.startswith(self.testMethodPrefix) } return sorted(method_starts, key=method_starts.get) def extract_mac_ip_mapping(nmap_xml_scan): """Extract the IP->MAC address mapping from the result of a nmap scan. The returned mapping uses uppercase MAC addresses. :param nmap_xml_scan: The XML result of scanning a network using nmap. Scanning a network using nmap is typically done by running: `nmap -sP -oX -`. :type nmap_xml_scan: string """ parser = etree.XMLParser(remove_blank_text=True) xml_doc = etree.fromstring(nmap_xml_scan, parser) hosts = xml_doc.xpath("/nmaprun/host") mapping = {} for host in hosts: ips = host.xpath("address[@addrtype='ipv4']/@addr") macs = host.xpath("address[@addrtype='mac']/@addr") if len(ips) == 1 and len(macs) == 1: mapping[macs[0].upper()] = ips[0] return mapping def mipf_arch_list(architecture): """Return the architectures that must be downloaded by m-i-p-f. This converts an architecture name (e.g. 'amd64' or 'armhf/generic') into the list of architectures that m-i-p-f must download in a format that this script understands. More precisely, this means doing two things: - default the subarchitecture 'generic' if not specified. - work around bug 1181334 by including 'i386/generic' in the list of architectures to be downloaded if it's not explicitely specified. """ # If subarchitecture is not provided, assume 'generic'. if '/' not in architecture: architecture = '%s/generic' % architecture arch_list = [architecture] # Include i386, see # https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1181334 if architecture != "i386/generic": arch_list.append("i386/generic") return arch_list def check_kvm_ok(): """Check that this machine is capable of running KVM. Uses ``sudo`` to run ``kvm-ok`` as root. From kvm_ok(1): If running as root, it will check your CPU's MSRs to see if VT is disabled in the BIOS. :return: True if KVM extensions are found, False otherwise. """ logging.info("Checking for KVM extensions.") retcode, stdout, _ = run_command(['sudo', 'kvm-ok'], check_call=True) if retcode != 0: logging.debug(stdout.strip()) return False else: return True def extract_package_version(policy): """Get the version of the package to be installed from policy settings. The given policy is assumed to be of the form returned by running: `apt-cache policy `. This method returns None if the given policy string cannot be parsed. """ match = re.search('Candidate: (.*)\n', policy) if match is not None: return match.group(1) else: return None def virtualization_type(): """Returns the name of the virtualization type of this machine. :return: the name of the virtualization type if the machine is running on virtual hardware. None otherwise. """ logging.info("Checking for virtualised hardware...") retcode, stdout, _ = run_command(['sudo', 'virt-what'], check_call=True) if stdout == '': # The hardware is physical. return None return stdout.strip() def get_user_input(message): """A wrapper around raw_input() that allows us to sanely mock it. """ return input(message) class CachingOutputStream: """An object that caches output sent to a stream.""" def __init__(self, stream): """Create a CachingOutputStream. :param stream: A file-like object to which content should be written after caching. """ self.cache = BytesIO() self.stream = stream def __getattr__(self, attr): """Pass-through any attribute ther than write(). :param attr: The attribute to return. This will be looked up on and returned from the output stream to which this CachingOutputStream writes. :type attr: string """ return getattr(self.stream, attr) def write(self, output): """Write to the internal cache and to the output stream. :param output: The content to write to store in the CachingOutputStream's cache and then write to the output stream. :type output: string """ self.cache.write(output.encode('utf-8')) self.stream.write(output) def compose_filter(key, values): """Create a simplestreams filter string. The filter string that is returned performs a regex match of `key` against given literal `values`, e.g. "arch~(i386|amd64|armhf)" (for key 'arch' with values 'i386' etc.). :param key: The simplestreams key that is to be filtered. :param values: Iterable of strings. Any of these literal values for `key` will match the simplestream filter; nothing else will. :return: A regex string suitable for passing to simplestreams. """ return "%s~(%s)" % ( key, '|'.join(re.escape(literal) for literal in values), ) maas-test-0.1+bzr147.orig/maastest/testing/__init__.py0000644000000000000000000000000012321262143020753 0ustar 00000000000000maas-test-0.1+bzr147.orig/maastest/testing/factory.py0000644000000000000000000000145112321262143020676 0ustar 00000000000000# Copyright 2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Factory helpers for testing.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [ 'make_file', ] from fixtures import TempDir import os.path def make_file(testcase, name=None, content=None): """Write a file. Attach cleanup to `testcase`.""" if content is None: content = testcase.getUniqueString() testcase.assertIsInstance(content, bytes) if name is None: name = testcase.getUniqueString() path = os.path.join(testcase.useFixture(TempDir()).path, name) with open(path, 'wb') as f: f.write(content) return path maas-test-0.1+bzr147.orig/maastest/tests/__init__.py0000644000000000000000000000000012321262143020440 0ustar 00000000000000maas-test-0.1+bzr147.orig/maastest/tests/test_detect_dhcp.py0000644000000000000000000002662712321262143022235 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for dhcp/detect.py""" from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [] from errno import EADDRNOTAVAIL import fcntl import os from random import randint import socket import string from maastest.detect_dhcp import ( BOOTP_CLIENT_PORT, BOOTP_SERVER_PORT, DHCPDiscoverPacket, DHCPOfferPacket, get_interface_IP, get_interface_MAC, make_transaction_ID, probe_dhcp, receive_offers, request_dhcp, udp_socket, ) import maastest.detect_dhcp as detect_module import mock from testtools import TestCase def make_MAC(): """Return an arbitrary MAC address.""" return ':'.join('%x' % randint(0, 255) for _ in range(6)) def make_IP(): """Return an arbitrary IP address.""" return '.'.join('%d' % randint(1, 254) for _ in range(4)) def pick_item(items): """Return an arbitrary item from container`.""" index = randint(0, len(items) - 1) return items[index] def make_name(prefix=None, length=10, sep='_'): """Return an arbitrary identifier-style string.""" characters = string.letters + string.digits if prefix is None: name = pick_item(string.letters) else: name = prefix + sep while len(name) < length: name += pick_item(characters) return name def make_bytes(length=None): """Return an arbitrary `bytes`.""" if length is None: length = randint(1, 50) return os.urandom(length) class TestMakeTransactionID(TestCase): """Tests for `make_transaction_ID`.""" def test_produces_well_formed_ID(self): # The dhcp transaction should be 4 bytes long. transaction_id = make_transaction_ID() self.assertIsInstance(transaction_id, bytes) self.assertEqual(4, len(transaction_id)) def test_randomises(self): self.assertNotEqual( make_transaction_ID(), make_transaction_ID()) class TestDHCPDiscoverPacket(TestCase): def test_init_sets_transaction_ID(self): transaction_id = make_transaction_ID() self.patch( detect_module, 'make_transaction_ID', mock.MagicMock(return_value=transaction_id)) discover = DHCPDiscoverPacket(make_MAC()) self.assertEqual(transaction_id, discover.transaction_ID) def test_init_sets_packed_mac(self): mac = make_MAC() discover = DHCPDiscoverPacket(mac) self.assertEqual( discover.string_mac_to_packed(mac), discover.packed_mac) def test_init_sets_packet(self): discover = DHCPDiscoverPacket(make_MAC()) self.assertIsNotNone(discover.packet) def test_string_mac_to_packed(self): discover = DHCPDiscoverPacket expected = b"\x01\x22\x33\x99\xaa\xff" input = "01:22:33:99:aa:ff" self.assertEqual(expected, discover.string_mac_to_packed(input)) def test__build(self): mac = make_MAC() discover = DHCPDiscoverPacket(mac) discover._build() expected = ( b'\x01\x01\x06\x00' + discover.transaction_ID + b'\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00' + b'\x00\x00\x00\x00\x00\x00\x00\x00' + discover.packed_mac + b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' + b'\x00' * 67 + b'\x00' * 125 + b'\x63\x82\x53\x63\x35\x01\x01\x3d\x06' + discover.packed_mac + b'\x37\x03\x03\x01\x06\xff') self.assertEqual(expected, discover.packet) class TestDHCPOfferPacket(TestCase): def test_decodes_dhcp_server(self): buffer = b'\x00' * 245 + b'\x10\x00\x00\xaa' offer = DHCPOfferPacket(buffer) self.assertEqual('16.0.0.170', offer.dhcp_server_ID) class TestGetInterfaceMAC(TestCase): """Tests for `get_interface_MAC`.""" def test_loopback_has_zero_MAC(self): # It's a lame test, but what other network interfaces can we reliably # test this on? sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.assertEqual('00:00:00:00:00:00', get_interface_MAC(sock, 'lo')) class TestGetInterfaceIP(TestCase): """Tests for `get_interface_IP`.""" def test_loopback_has_localhost_address(self): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.assertEqual('127.0.0.1', get_interface_IP(sock, 'lo')) def test_returns_None_if_no_address(self): failure = IOError(EADDRNOTAVAIL, "Interface has no address") self.patch(fcntl, 'ioctl', mock.MagicMock(side_effect=failure)) sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.assertIsNone(get_interface_IP(sock, make_name('itf'))) def patch_socket(testcase): """Patch `socket.socket` to return a mock.""" sock = mock.MagicMock() testcase.patch(socket, 'socket', mock.MagicMock(return_value=sock)) return sock class TestUDPSocket(TestCase): """Tests for `udp_socket`.""" def test_yields_open_socket(self): patch_socket(self) with udp_socket() as sock: socket_calls = list(socket.socket.mock_calls) close_calls = list(sock.close.mock_calls) self.assertEqual( [mock.call(socket.AF_INET, socket.SOCK_DGRAM)], socket_calls) self.assertEqual([], close_calls) def test_closes_socket_on_exit(self): patch_socket(self) with udp_socket() as sock: pass self.assertEqual([mock.call()], sock.close.mock_calls) def test_sets_reuseaddr(self): patch_socket(self) with udp_socket() as sock: pass self.assertEqual( [mock.call(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)], sock.setsockopt.mock_calls) class TestRequestDHCP(TestCase): """Tests for `request_dhcp`.""" def patch_interface_MAC(self): """Patch `get_interface_MAC` to return a fixed value.""" mac = make_MAC() self.patch( detect_module, 'get_interface_MAC', mock.MagicMock(return_value=mac)) return mac def patch_interface_IP(self): """Patch `get_interface_IP` to return a fixed value.""" ip = make_IP() self.patch( detect_module, 'get_interface_IP', mock.MagicMock(return_value=ip)) return ip def patch_transaction_ID(self): """Patch `make_transaction_ID` to return a fixed value.""" transaction_id = make_transaction_ID() self.patch( detect_module, 'make_transaction_ID', mock.MagicMock(return_value=transaction_id)) return transaction_id def test_returns_None_if_no_IP_address(self): self.patch( detect_module, 'get_interface_MAC', mock.MagicMock(return_value=make_MAC())) self.patch( detect_module, 'get_interface_IP', mock.MagicMock(return_value=None)) self.assertIsNone(request_dhcp(make_name('itf'))) def test_sends_discover_packet(self): sock = patch_socket(self) self.patch_interface_MAC() self.patch_interface_IP() request_dhcp(make_name('itf')) [call] = sock.sendto.mock_calls _, args, _ = call self.assertEqual( ('', BOOTP_SERVER_PORT), args[1]) def test_returns_transaction_id(self): patch_socket(self) self.patch_interface_MAC() self.patch_interface_IP() transaction_id = self.patch_transaction_ID() interface = make_name('itf') self.assertEqual(transaction_id, request_dhcp(interface)) class FakePacketReceiver: """Fake callable to substitute for a socket's `recv`. Returns the given packets on successive calls. When it runs out, raises a timeout. """ def __init__(self, packets=None): if packets is None: packets = [] self.calls = [] self.packets = list(packets) def __call__(self, recv_size): self.calls.append(recv_size) if len(self.packets) == 0: raise socket.timeout() else: return self.packets.pop(0) def patch_recv(testcase, sock, num_packets=0): """Patch up socket's `recv` to return `num_packets` arbitrary packets. After that, further calls to `recv` will raise a timeout. """ packets = [make_bytes() for _ in range(num_packets)] receiver = FakePacketReceiver(packets) testcase.patch(sock, 'recv', receiver) return receiver def patch_offer_packet(testcase): """Patch a mock `DHCPOfferPacket`.""" transaction_id = make_bytes(4) packet = mock.MagicMock() packet.transaction_ID = transaction_id packet.dhcp_server_ID = make_IP() testcase.patch( detect_module, 'DHCPOfferPacket', mock.MagicMock(return_value=packet)) return packet class TestReceiveOffers(TestCase): """Tests for `receive_offers`.""" def test_receives_from_socket(self): sock = patch_socket(self) receiver = patch_recv(self, sock) transaction_id = patch_offer_packet(self).transaction_ID receive_offers(transaction_id) self.assertEqual( [mock.call(socket.AF_INET, socket.SOCK_DGRAM)], socket.socket.mock_calls) self.assertEqual( [mock.call(('', BOOTP_CLIENT_PORT))], sock.bind.mock_calls) self.assertEqual([1024], receiver.calls) def test_returns_empty_if_nothing_received(self): sock = patch_socket(self) patch_recv(self, sock) transaction_id = patch_offer_packet(self).transaction_ID self.assertEqual(set(), receive_offers(transaction_id)) def test_processes_offer(self): sock = patch_socket(self) patch_recv(self, sock, 1) packet = patch_offer_packet(self) self.assertEqual( {packet.dhcp_server_ID}, receive_offers(packet.transaction_ID)) def test_ignores_other_transactions(self): sock = patch_socket(self) patch_recv(self, sock, 1) patch_offer_packet(self) other_transaction_id = make_bytes(4) self.assertEqual(set(), receive_offers(other_transaction_id)) def test_propagates_errors_other_than_timeout(self): class InducedError(Exception): """Deliberately induced error for testing.""" sock = patch_socket(self) sock.recv = mock.MagicMock(side_effect=InducedError) self.assertRaises( InducedError, receive_offers, make_bytes(4)) class TestProbeDHCP(TestCase): def test_detects_dhcp_servers(self): sock = patch_socket(self) patch_recv(self, sock, 1) packet = patch_offer_packet(self) transaction_id = packet.transaction_ID server = packet.dhcp_server_ID self.patch( detect_module, 'request_dhcp', mock.MagicMock(return_value=transaction_id)) servers = probe_dhcp(make_name('itf')) self.assertEqual({server}, servers) def test_tolerates_interface_without_address(self): self.patch( detect_module, 'get_interface_MAC', mock.MagicMock(return_value=make_MAC())) self.patch( detect_module, 'get_interface_IP', mock.MagicMock(return_value=None)) servers = probe_dhcp(make_name('itf')) self.assertEqual(set(), servers) maas-test-0.1+bzr147.orig/maastest/tests/test_kvmfixture.py0000644000000000000000000007510612321262143022167 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for `maastest.kvmfixture`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [] import itertools import os.path import random from random import randint from tempfile import gettempdir import textwrap import fixtures from lxml import etree from maastest import kvmfixture from maastest.testing.factory import make_file from maastest.utils import read_file import mock import netaddr from six import text_type import testtools from testtools.matchers import ( FileContains, FileExists, HasPermissions, MatchesRegex, Not, StartsWith, ) class TestKVMFixture(testtools.TestCase): def setUp(self): super(TestKVMFixture, self).setUp() self.logger = self.useFixture(fixtures.FakeLogger()) self.patch(kvmfixture, 'sleep', mock.MagicMock()) # KVMFixture.generate_ssh_key() generates an ssh key in maas-test's # configuration directory, creating that directory if necessary. Set # up a temporary directory so it can do what it needs to. self.state_dir = self.useFixture(fixtures.TempDir()).path def patch_run_command(self, return_value=0, stdout=b'', stderr=b''): """Patch out `kvmfixture` calls to `run_command`. Call this so your test doesn't accidentally make `KVMFixture` download machine images, ssh into it, etc. Returns the `MagicMock` fake it has installed. """ self.assertIsInstance(stdout, bytes) self.assertIsInstance(stderr, bytes) fake = mock.MagicMock(return_value=(return_value, stdout, stderr)) self.patch(kvmfixture, 'run_command', fake) return fake def make_KVMFixture(self, series=None, architecture=None, name=None, direct_interface=None, direct_network=None, archives=None, proxy_url=None, kvm_timeout=None, simplestreams_filter=None): if series is None: series = self.getUniqueString() if architecture is None: architecture = self.getUniqueString() if proxy_url is None: proxy_url = "http://example.com:%s" % ( random.randint(1, 65535)) self.proxy_url = proxy_url # We don't bother dealing with name being None, since KVMFixture # deals with that itself. fixture = kvmfixture.KVMFixture( series, architecture, name=name, direct_interface=direct_interface, direct_network=direct_network, archives=archives, proxy_url=proxy_url, ssh_key_dir=self.state_dir, kvm_timeout=kvm_timeout, simplestreams_filter=simplestreams_filter) # Hack to get fixture.addDetail to work without actually calling # fixture.setUp(). fixture._details = {} return fixture def find_matching_call(self, fake_run_command, command_prefix): """Find the first `run_command` call with a matching command line. This assumes that you have called `patch_run_command` first. :param fake_run_command: fake for `run_command` as returned by `patch_run_command`. :param command_prefix: A sequence of items on the command line. A call to `run_commanad` will match this when this forms a prefix of the executed command line. For instance, a `command_prefix` of `['ls']` will match a command line `['ls', '/tmp']` but not a command line `['sudo', 'ls', '/tmp']`. On the other hand, a prefix `['sudo', 'ls']` will match `['sudo', 'ls', '/tmp']` but not `['ls', '/tmp']`. :rtype: `mock.call` """ command_prefix = list(command_prefix) prefix_length = len(command_prefix) for invocation in fake_run_command.mock_calls: _, args, _ = invocation command_line = args[0] if command_line[:prefix_length] == command_prefix: return invocation return None def test_init_sets_properties(self): series = "test-series" architecture = "test-arch" fixture = self.make_KVMFixture(series, architecture) self.assertEqual( (series, architecture, 36), (fixture.series, fixture.architecture, len(fixture.name))) def test_ip_address_calls_out_for_ip(self): series = "test-series" architecture = "test-arch" ip_address = '192.168.2.1' self.patch_run_command(0, stdout=ip_address.encode("ascii")) with self.make_KVMFixture(series, architecture) as fixture: self.assertEqual(ip_address, fixture.ip_address()) def test_ip_address_caches_ip_after_acquisition(self): series = "test-series" architecture = "test-arch" ip_address = '192.168.2.1' self.patch_run_command(0, stdout=ip_address.encode("ascii")) with self.make_KVMFixture(series, architecture) as fixture: fixture.ip_address() self.assertEqual(ip_address, fixture._ip_address) def test_ip_address_returns_cached_address(self): series = "test-series" architecture = "test-arch" ip_address = '192.168.2.1' self.patch_run_command() with self.make_KVMFixture(series, architecture) as fixture: fixture._ip_address = ip_address self.assertEqual(ip_address, fixture.ip_address()) def test_identity(self): series = "test-series" architecture = "test-arch" ip_address = '192.168.2.1' self.patch_run_command(stdout=ip_address.encode("ascii")) with self.make_KVMFixture(series, architecture) as fixture: self.assertEqual("ubuntu@%s" % ip_address, fixture.identity()) def test_setUp_calls_machine_creation_methods(self): self.patch_run_command() mock_import_image = mock.MagicMock() self.patch(kvmfixture.KVMFixture, 'import_image', mock_import_image) mock_start = mock.MagicMock() self.patch(kvmfixture.KVMFixture, 'start', mock_start) mock_install_base_packages = mock.MagicMock() self.patch( kvmfixture.KVMFixture, 'install_base_packages', mock_install_base_packages) mock_configure_network = mock.MagicMock() self.patch( kvmfixture.KVMFixture, 'configure_network', mock_configure_network) mock_configure_archives = mock.MagicMock() self.patch( kvmfixture.KVMFixture, 'configure_archives', mock_configure_archives) series = "test-series" architecture = "test-arch" with self.make_KVMFixture(series, architecture): self.assertEqual( [ mock.call(), mock.call(), mock.call(), mock.call(), mock.call(), ], [ mock_import_image.mock_calls, mock_start.mock_calls, mock_configure_network.mock_calls, mock_configure_archives.mock_calls, mock_install_base_packages.mock_calls, ]) def test_setUp_awaits_boot_and_cloudinit_completion(self): fake_run_command = self.patch_run_command() self.patch( kvmfixture.KVMFixture, 'ip_address', mock.MagicMock(return_value='127.127.127.127')) with self.make_KVMFixture() as fixture: name = fixture.name fake_run_command.assert_has_calls([ mock.call(['uvt-kvm', 'wait', name], check_call=True), mock.call( fixture._make_ssh_command( ['test', '-f', kvmfixture.CLOUDINIT_COMPLETION_MARKER]), check_call=False, input=None), ]) def test_running_is_set_while_running_only(self): self.patch_run_command() fixture = self.make_KVMFixture() running_before = fixture.running with fixture: running_during = fixture.running running_after = fixture.running self.assertEqual( (False, True, False), (running_before, running_during, running_after)) def test_setUp_registers_cleanup_method(self): class InducedError(RuntimeError): """Deliberately induced error for testing.""" self.patch_run_command() fixture = self.make_KVMFixture() self.patch( fixture, 'wait_for_vm', mock.MagicMock(side_effect=InducedError)) self.assertRaises(InducedError, fixture.setUp) fixture.cleanUp() self.assertFalse(fixture.running) def test_get_simplestreams_filter_with_filters(self): filter1 = "%s=%s" % (self.getUniqueString(), self.getUniqueString()) filter2 = "%s=%s" % (self.getUniqueString(), self.getUniqueString()) fixture = self.make_KVMFixture( simplestreams_filter="%s %s" % (filter1, filter2)) self.assertItemsEqual( [ 'arch=%s' % fixture.architecture, 'release=%s' % fixture.series, filter1, filter2, ], fixture.get_simplestreams_filter()) def test_get_simplestreams_filter_without_filters(self): fixture = self.make_KVMFixture(simplestreams_filter=None) self.assertItemsEqual( [ 'arch=%s' % fixture.architecture, 'release=%s' % fixture.series, ], fixture.get_simplestreams_filter()) def test_start_starts_vm(self): fake_run_command = self.patch_run_command() series = "test-series" architecture = "test-arch" self.patch(kvmfixture.KVMFixture, 'generate_ssh_key', mock.MagicMock()) mock_make_kvm_template = mock.MagicMock() kvm_template = self.getUniqueString() mock_make_kvm_template.return_value = kvm_template self.patch(kvmfixture, 'make_kvm_template', mock_make_kvm_template) ssfilter = self.getUniqueString() kvm_fixture = self.make_KVMFixture( series, architecture, simplestreams_filter=ssfilter) with kvm_fixture as fixture: name = fixture.name public_key = fixture.get_ssh_public_key_file() fake_run_command.assert_has_calls( [ mock.call([ 'sudo', 'http_proxy=%s' % self.proxy_url, 'uvt-simplestreams-libvirt', 'sync', 'arch=%s' % architecture, 'release=%s' % series, ssfilter], check_call=True), mock.call([ 'sudo', 'uvt-kvm', 'create', '--ssh-public-key-file=%s' % public_key, '--unsafe-caching', '--memory', '2047', '--disk', '20', name, 'arch=%s' % architecture, 'release=%s' % series, ssfilter, '--template', '-'], input=kvm_template, check_call=True), ]) self.assertEqual( [ mock.call(None), ], mock_make_kvm_template.mock_calls) def test_import_image_logs_to_root_logger(self): self.patch_run_command() fixture = self.make_KVMFixture() fixture.import_image() self.assertEqual( [ "Downloading KVM image for %s..." % ' '.join( fixture.get_simplestreams_filter()), "Done downloading KVM image." ], self.logger.output.strip().split("\n")) def test_generate_ssh_key_sets_up_temporary_key_pair(self): fixture = self.make_KVMFixture() fixtures.Fixture.setUp(fixture) fixture.generate_ssh_key() self.assertThat(fixture.ssh_private_key_file, FileExists()) self.assertThat(fixture.get_ssh_public_key_file(), FileExists()) self.assertThat( fixture.ssh_private_key_file, StartsWith(self.state_dir + '/')) self.assertThat( fixture.ssh_private_key_file, FileContains( matcher=StartsWith('-----BEGIN RSA PRIVATE KEY-----'))) self.assertThat( fixture.get_ssh_public_key_file(), FileContains(matcher=StartsWith('ssh-rsa'))) # Other users are not allowed to read the private key. self.assertThat(fixture.ssh_private_key_file, HasPermissions('0600')) # The key pair is not protected by a passphrase, as indicated by # the absence of this line. self.assertThat( fixture.ssh_private_key_file, Not(FileContains(matcher=MatchesRegex("^Proc-Type: .*ENCRYPTED")))) def test_generate_ssh_key_reuses_existing_key_pair(self): earlier_fixture = self.make_KVMFixture() fixtures.Fixture.setUp(earlier_fixture) earlier_fixture.generate_ssh_key() # We decode the file contents here to work around bug #1250483 # in FileContains later. This is for Python 3 compatibility. earlier_private_key_contents = read_file( earlier_fixture.ssh_private_key_file).decode("ascii") earlier_public_key_contents = read_file( earlier_fixture.get_ssh_public_key_file()).decode("ascii") later_fixture = self.make_KVMFixture() fixtures.Fixture.setUp(later_fixture) later_fixture.generate_ssh_key() self.assertEqual( earlier_fixture.ssh_private_key_file, later_fixture.ssh_private_key_file) self.assertEqual( earlier_fixture.get_ssh_public_key_file(), later_fixture.get_ssh_public_key_file()) self.assertThat( later_fixture.ssh_private_key_file, FileContains(earlier_private_key_contents)) self.assertThat( later_fixture.get_ssh_public_key_file(), FileContains(earlier_public_key_contents)) def test_start_passes_public_ssh_key(self): fake_run_command = self.patch_run_command() with self.make_KVMFixture() as fixture: fixture.start() uvt_call = self.find_matching_call( fake_run_command, ['sudo', 'uvt-kvm']) self.assertIsNotNone(uvt_call) _, args, _ = uvt_call command_line = args[0] self.assertIn( '--ssh-public-key-file=%s' % fixture.get_ssh_public_key_file(), command_line) def test_wait_for_cloudinit_looks_for_completion_marker(self): ssh_run_command = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', ssh_run_command) fixture = self.make_KVMFixture() fixture.wait_for_cloudinit() self.assertEqual( [mock.call( ['test', '-f', kvmfixture.CLOUDINIT_COMPLETION_MARKER])], ssh_run_command.mock_calls) self.assertEqual([], kvmfixture.sleep.mock_calls) def test_wait_for_cloudinit_times_out_eventually(self): attempts = [(0, 15), (5, 10), (10, 5), (15, 0)] self.patch( kvmfixture, 'retries', mock.MagicMock(return_value=attempts)) stderr = self.getUniqueString() ssh_run_command = mock.MagicMock(return_value=(1, '', stderr)) self.patch(kvmfixture.KVMFixture, 'run_command', ssh_run_command) fixture = self.make_KVMFixture() exception = self.assertRaises(RuntimeError, fixture.wait_for_cloudinit) message = text_type(exception) self.assertIn( "Cloud-init initialization on virtual machine %s timed out" % fixture.name, message) self.assertIn(stderr, message) self.assertEqual(len(attempts), len(ssh_run_command.mock_calls)) def test_wait_for_cloudinit_fails_immediately_on_unexpected_error(self): stderr = self.getUniqueString() ssh_run_command = mock.MagicMock(return_value=(99, '', stderr)) self.patch(kvmfixture.KVMFixture, 'run_command', ssh_run_command) fixture = self.make_KVMFixture() exception = self.assertRaises(RuntimeError, fixture.wait_for_cloudinit) message = text_type(exception) self.assertIn( "Error contacting virtual machine %s" % fixture.name, message) self.assertIn(stderr, message) self.assertEqual(1, len(ssh_run_command.mock_calls)) self.assertEqual([], kvmfixture.sleep.mock_calls) def test_get_ssh_public_key_file_returns_public_key(self): self.patch_run_command() with self.make_KVMFixture() as fixture: public_key = fixture.get_ssh_public_key_file() self.assertEqual(fixture.ssh_private_key_file + '.pub', public_key) def test_start_logs_to_root_logger(self): self.patch_run_command() fixture = self.make_KVMFixture() fixture.start() self.assertEqual( [ "Creating virtual machine %s, arch=%s..." % (fixture.name, fixture.architecture), "Done creating virtual machine %s, arch=%s." % (fixture.name, fixture.architecture), ], self.logger.output.strip().split("\n")) def test_destroy_logs_to_root_logger(self): self.patch_run_command() fixture = self.make_KVMFixture() fixture.destroy() self.assertEqual( [ "Destroying virtual machine %s..." % fixture.name, "Done destroying virtual machine %s." % fixture.name, ], self.logger.output.strip().split("\n")) def test_network_generated_if_not_specified(self): self.patch_run_command() series = "test-series" architecture = "test-arch" interface = self.getUniqueString() fixture = self.make_KVMFixture( series, architecture, direct_interface=interface) self.assertEqual( (fixture.direct_network, fixture.direct_ip), (netaddr.IPNetwork('192.168.2.0/24'), '192.168.2.1')) def test_direct_first_available_ip(self): self.patch_run_command() series = "test-series" architecture = "test-arch" interface = self.getUniqueString() network = netaddr.IPNetwork('192.168.12.0/24') fixture = self.make_KVMFixture( series, architecture, direct_interface=interface, direct_network=network) self.assertEqual( fixture.direct_first_available_ip(), "192.168.12.2") def test_configure_archives_adds_archives_and_updates(self): mock_identity = mock.MagicMock() mock_identity.return_value = "ubuntu@test" self.patch(kvmfixture.KVMFixture, 'identity', mock_identity) run_command_mock = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', run_command_mock) archives = [self.getUniqueString(), self.getUniqueString()] fixture = self.make_KVMFixture( 'series', 'architecture', archives=archives) fixture.configure_archives() add_archive_cmds = [ ["sudo", "add-apt-repository", "-y", archives[0]], ["sudo", "add-apt-repository", "-y", archives[1]], ["sudo", "apt-get", "update"], ] self.assertEqual( [mock.call(cmd, check_call=True) for cmd in add_archive_cmds], run_command_mock.mock_calls) def test_configure_archives_always_updates(self): mock_identity = mock.MagicMock() mock_identity.return_value = "ubuntu@test" self.patch(kvmfixture.KVMFixture, 'identity', mock_identity) run_command_mock = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', run_command_mock) fixture = self.make_KVMFixture( 'series', 'architecture', archives=None) fixture.configure_archives() self.assertEqual( [mock.call(["sudo", "apt-get", "update"], check_call=True)], run_command_mock.mock_calls) def test_configure_network_configures_network(self): series = "test-series" architecture = "test-arch" mock_identity = mock.MagicMock() mock_identity.return_value = "ubuntu@test" self.patch(kvmfixture.KVMFixture, 'identity', mock_identity) network = netaddr.IPNetwork('192.168.12.0/24') ip = "192.168.12.1" interface = self.getUniqueString() fixture = self.make_KVMFixture( series, architecture, direct_interface=interface, direct_network=network) kvm_run_command = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', kvm_run_command) fixture.configure_network() netconfig_command = ["sudo", "tee", "-a", "/etc/network/interfaces"] input = kvmfixture.make_network_interface_config( 'eth1', ip, network, 'eth0') netup_command = ["sudo", "ifup", "eth1"] self.assertEqual( [ mock.call(netconfig_command, input=input, check_call=True), mock.call(netup_command, check_call=True), ], kvm_run_command.mock_calls) def test_machine_is_destroyed(self): fake_run_command = self.patch_run_command() series = "test-series" architecture = "test-arch" with self.make_KVMFixture(series, architecture) as fixture: pass self.assertEqual( mock.call([ "sudo", "uvt-kvm", "destroy", fixture.name], check_call=True), fake_run_command.mock_calls[-1]) def test_install_packages_runs_apt_get_noninteractively_with_check(self): self.patch_run_command() packages = ['foo', 'bar'] fixture = self.make_KVMFixture('series', 'architecture') kvm_run_command = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', kvm_run_command) fixture.install_packages(packages) self.assertEqual( [mock.call( fixture._make_apt_get_install_command(packages), check_call=True)], kvm_run_command.mock_calls) def test_install_base_packages(self): self.patch_run_command() fixture = self.make_KVMFixture('series', 'architecture') kvm_run_command = mock.MagicMock(return_value=(0, '', '')) self.patch(kvmfixture.KVMFixture, 'run_command', kvm_run_command) fixture.install_base_packages() self.assertEqual( [mock.call( fixture._make_apt_get_install_command(['nmap']), check_call=True)], kvm_run_command.mock_calls) def test_run_command_runs_command_remotely(self): fake_run_command = self.patch_run_command() series = "test-series" architecture = "test-arch" input = self.getUniqueString() command = ["echo", "Hello"] with self.make_KVMFixture(series, architecture) as fixture: expected_command = fixture._make_ssh_command(command) fixture.run_command(command, input=input, check_call=False) ssh_command = fake_run_command.mock_calls[-1] self.assertEqual( mock.call(expected_command, input=input, check_call=False), ssh_command) def test_run_command_adds_details(self): series = "test-series" architecture = "test-arch" return_value = randint(0, 3) input = self.getUniqueString() stdout_content = self.getUniqueString().encode("ascii") stderr_content = self.getUniqueString().encode("ascii") self.patch_run_command( return_value, stdout=stdout_content, stderr=stderr_content) self.patch( kvmfixture.KVMFixture, 'wait_for_cloudinit', mock.MagicMock()) with self.make_KVMFixture(series, architecture) as fixture: fixture.command_count = itertools.count(1) fixture.run_command( ["echo", "Hello"], input=input, check_call=False) details = fixture.getDetails() self.assertThat( details['cmd #0001'].as_text(), MatchesRegex('ssh .* echo Hello$')) self.assertEqual( ( text_type(return_value), input, stdout_content, stderr_content, ), ( details['cmd #0001 retcode'].as_text(), details['cmd #0001 input'].as_text(), details['cmd #0001 stdout'].as_text(), details['cmd #0001 stderr'].as_text(), )) def test_get_ip_from_network_scan(self): self.patch_run_command() mac = self.getUniqueString() ip = self.getUniqueString() network_str = '192.168.12.0/24' network = netaddr.IPNetwork(network_str) fixture = self.make_KVMFixture( 'series', 'architecture', direct_network=network) mapping = {mac.upper(): ip} extract_mac_ip_mapping_mock = mock.MagicMock(return_value=mapping) xml_output = self.getUniqueString() self.patch( kvmfixture, 'extract_mac_ip_mapping', extract_mac_ip_mapping_mock) kvm_run_command = mock.MagicMock(return_value=(0, xml_output, '')) self.patch(kvmfixture.KVMFixture, 'run_command', kvm_run_command) returned_ip = fixture.get_ip_from_network_scan(mac) self.assertEqual( ( ip, [ mock.call( ['sudo', 'nmap', '-sP', network_str, '-oX', '-'], check_call=True) ], [mock.call(xml_output)], ), ( returned_ip, kvm_run_command.mock_calls, extract_mac_ip_mapping_mock.mock_calls, )) def test_wait_for_vm_uses_timeout(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) timeout = randint(1, 1000) fixture = self.make_KVMFixture(kvm_timeout=timeout) fixture.wait_for_vm() self.assertEqual([ mock.call([ 'uvt-kvm', 'wait', '--timeout', unicode(timeout), fixture.name], check_call=True)], kvmfixture.run_command.mock_calls) def test_wait_for_vm_ignores_timeout_if_not_set(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_KVMFixture(kvm_timeout=None) fixture.wait_for_vm() self.assertEqual([ mock.call([ 'uvt-kvm', 'wait', fixture.name], check_call=True)], kvmfixture.run_command.mock_calls) def test_upload_file_uploads_file(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_KVMFixture(kvm_timeout=None) local_file = make_file(self) remote_file = os.path.join(gettempdir(), self.getUniqueString()) fixture.upload_file(local_file, remote_file) self.assertEqual( [ mock.call( ( ['scp'] + fixture._get_base_ssh_options() + [local_file, remote_file] ), check_call=True) ], kvmfixture.run_command.mock_calls) def test_upload_file_defaults_to_base_name_in_default_dir(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_KVMFixture(kvm_timeout=None) local_file = make_file(self) fixture.upload_file(local_file) [command] = kvmfixture.run_command.mock_calls name, args, kwargs = command (command_line,) = args self.assertEqual(os.path.basename(local_file), command_line[-1]) def xml_normalize(snippet): parser = etree.XMLParser(remove_blank_text=True) return etree.tostring(etree.fromstring(snippet, parser)) class TestKVMTemplateGeneration(testtools.TestCase): def test_kvm_template_creates_kvm_template(self): template = kvmfixture.make_kvm_template() expected_template = xml_normalize(kvmfixture.KVM_TEMPLATE) self.assertEqual(expected_template, template) def test_kvm_template_without_direct_interface_has_one_interface(self): parser = etree.XMLParser(remove_blank_text=True) template = kvmfixture.make_kvm_template() xml_template = etree.fromstring(template, parser) interface_tags = xml_template.xpath("/domain/devices/interface") expected_interfaces = [""" """] self.assertEqual( [xml_normalize(iface) for iface in expected_interfaces], [etree.tostring(iface) for iface in interface_tags]) def test_kvm_template_with_direct_interface_has_two_interface(self): interface = self.getUniqueString() parser = etree.XMLParser(remove_blank_text=True) template = kvmfixture.make_kvm_template(interface) xml_template = etree.fromstring(template, parser) interface_tags = xml_template.xpath("/domain/devices/interface") expected_interfaces = [ """ """, """ """ % interface, ] self.assertEqual( [xml_normalize(iface) for iface in expected_interfaces], [etree.tostring(iface) for iface in interface_tags]) class TestNetworkConfigGeneration(testtools.TestCase): def test_make_network_interface_config(self): interface = 'eth0' direct_interface = 'eth1' ip = "192.168.12.22" network = netaddr.IPNetwork('192.168.12.3/24') expected_config = textwrap.dedent(""" auto eth1 iface eth1 inet static address 192.168.12.22 network 192.168.12.0 netmask 255.255.255.0 broadcast 192.168.12.255 post-up sysctl -w net.ipv4.ip_forward=1 post-up iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE post-up iptables -A FORWARD -i eth0 -o eth1 -m state --state \ RELATED,ESTABLISHED -j ACCEPT post-up iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT """) config = kvmfixture.make_network_interface_config( direct_interface, ip, network, interface) self.assertEqual(expected_config, config) maas-test-0.1+bzr147.orig/maastest/tests/test_maasfixture.py0000644000000000000000000007635612321262143022323 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for the "prepare" command.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) str = None __metaclass__ = type __all__ = [] import httplib from itertools import ( chain, repeat, ) import json from random import randint from subprocess import check_call from textwrap import dedent from apiclient.maas_client import MAASClient from maastest import ( kvmfixture, maasfixture, utils, ) from maastest.maas_enums import NODEGROUPINTERFACE_MANAGEMENT from maastest.maasfixture import NoBootImagesError from maastest.testing.factory import make_file import mock import netaddr from six import text_type from testtools import TestCase from testtools.matchers import FileContains class FakeResponse(object): """A fake http response used in testing.""" def __init__(self, content, code): self.content = content self._code = code def read(self): return self.content @property def code(self): return self._code class TestMAASFixture(TestCase): def make_kvm_fixture(self): """Create a `KVMFixture` with arbitrary series/architecture. Its `run_command` method will be replaced by a `MagicMock`, which returns success by default. Its `ip_address` will be patched to return an arbitrary IP address. """ fixture = kvmfixture.KVMFixture( 'droopy', 'i860', proxy_url="http://example.com:8080") fixture.run_command = mock.MagicMock( return_value=(0, 'stdout output', 'stderr output')) ip_address = '.'.join('%d' % randint(1, 254) for octets in range(4)) fixture.ip_address = mock.MagicMock(return_value=ip_address) return fixture def make_maas_fixture(self, kvm_fixture=None, proxy_url=None): """Create a `MAASFixture` with arbitrary series/architecture. The maasfixture.MAASClient's post/get/put methods are replaced by a `MagicMock`, which returns success (i.e. a FakeResponse with a return code of 200) by default. """ if kvm_fixture is None: kvm_fixture = self.make_kvm_fixture() series = self.getUniqueString() arch = self.getUniqueString() if proxy_url is None: proxy_url = self.getUniqueString() filter = self.getUniqueString() fixture = maasfixture.MAASFixture( kvm_fixture, proxy_url, series, arch, filter) ok_response = FakeResponse('', httplib.OK) for method in ('get', 'post', 'put'): self.patch( maasfixture.MAASClient, method, mock.MagicMock(return_value=ok_response)) return fixture def make_maas_local_settings(self, default_maas_url): """Compose a `maas_local_settings.py`, as installed.""" return ('DEFAULT_MAAS_URL = "%s"' % default_maas_url).encode('ascii') def make_rewritten_maas_local_settings(self, old_url, new_url): """Compose a `maas_local_settings.py` as rewritten. We rewrite this file to change its `DEFAULT_MAAS_URL` setting. It's done in a shell command, so needs careful testing. """ return dedent(""" # Replaced by maas-test: # DEFAULT_MAAS_URL = "%s" DEFAULT_MAAS_URL = "%s" """ % (old_url, new_url)).encode('ascii') def make_file(self, name=None, content=None): return make_file(self, name=name, content=content) def patch_MAASClient(self, client, method, content, code=httplib.OK): """Patch a MAASClient so that it will return a pre-cooked response.""" json_content = json.dumps(content) response = FakeResponse(json_content, code) mock_object = mock.MagicMock(return_value=response) self.patch(client, method, mock_object) return mock_object def test_compose_rewrite_for_default_maas_url_rewrites_url(self): old_url = 'http://1.1.1.1/' new_url = 'http://2.2.2.2/' config_file = make_file( self, content=self.make_maas_local_settings(old_url)) command = maasfixture.compose_rewrite_for_default_maas_url( config_file, new_url) # Test the generated sed command, by running it locally. In real use # this runs only on the virtual machine. # (But make sure it doesn't sudo. That won't do in tests.) self.assertNotIn('sudo', command) check_call(command) new_content = self.make_rewritten_maas_local_settings(old_url, new_url) self.assertThat(config_file, FileContains(new_content.decode('ascii'))) def test_compose_rewrite_for_default_maas_url_accepts_path(self): old_url = 'http://old-host.com/OLDMAAS' new_url = 'http://new-host.local/NEWMAAS' config_file = make_file( self, content=self.make_maas_local_settings(old_url)) check_call( maasfixture.compose_rewrite_for_default_maas_url( config_file, new_url)) new_content = self.make_rewritten_maas_local_settings(old_url, new_url) self.assertThat(config_file, FileContains(new_content.decode('ascii'))) def test_compose_rewrite_for_default_maas_url_preserves_other_text(self): top = b"# Text before setting." bottom = b"# Text after setting." old_url = "http://1.1.1.1/" old_content = b'\n'.join( [top, self.make_maas_local_settings(old_url), bottom]) config_file = make_file(self, content=old_content) new_url = "http://2.2.2.2/" check_call( maasfixture.compose_rewrite_for_default_maas_url( config_file, new_url)) new_part = self.make_rewritten_maas_local_settings(old_url, new_url) new_content = b'\n'.join([top, new_part, bottom]) self.assertThat(config_file, FileContains(new_content.decode('ascii'))) def test_compose_rewrite_for_default_maas_url_ignores_commented_URL(self): top = b'#DEFAULT_MAAS_URL = "http://9.9.9.9/"\n' old_url = "http://1.1.1.1/" old_content = top + self.make_maas_local_settings(old_url) config_file = make_file(self, content=old_content) new_url = "http://2.2.2.2/" check_call( maasfixture.compose_rewrite_for_default_maas_url( config_file, new_url)) new_content = top + self.make_rewritten_maas_local_settings( old_url, new_url) self.assertThat(config_file, FileContains(new_content.decode('ascii'))) def test_configure_default_maas_url_sets_config_and_restarts(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_kvm_fixture() fixture.direct_ip = '127.99.88.77' rewrite_command = maasfixture.compose_rewrite_for_default_maas_url( '/etc/maas/maas_local_settings.py', "http://%s/MAAS" % fixture.direct_ip) maas_fixture = self.make_maas_fixture(fixture) maas_fixture.configure_default_maas_url() self.assertEqual( [ mock.call(['sudo'] + rewrite_command, check_call=True), mock.call( ['sudo', 'service', 'apache2', 'restart'], check_call=False), ], fixture.run_command.mock_calls) def test_configure_default_maas_url_retries_if_restart_fails(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_kvm_fixture() # Configure fixture.run_command to simulate a transient failure when # restarting apache. returns = [ # Simluate response for the 'config rewrite' command. (0, 'stdout', 'stderr'), # Simulate first response for the 'apache2 restart' command: # failure. (1, 'stdout', 'an error!'), # Simulate second response for the 'apache2 restart' command: # success. (0, 'stdout', 'stderr'), ] def side_effect(*args, **kwargs): return returns.pop(0) fixture.run_command = mock.MagicMock(side_effect=side_effect) fixture.direct_ip = '127.99.88.77' self.patch( utils, 'retries', mock.MagicMock(return_value=[(0, 1), (0, 1)])) maas_fixture = self.make_maas_fixture(fixture) maas_fixture.configure_default_maas_url() # The first call to fixture.run_command is the rewrite # configuration command, ignore it since it's tested elsewhere # and we're only testing that the 'apache2 restart' command is # retried when it fails here. run_command_calls = fixture.run_command.mock_calls[1:] self.assertEqual( [ # 2 'apache restart' calls. mock.call( ['sudo', 'service', 'apache2', 'restart'], check_call=False), mock.call( ['sudo', 'service', 'apache2', 'restart'], check_call=False), ], run_command_calls) def test_configure_default_maas_url_fails_if_apache_cannot_restart(self): self.patch(kvmfixture, 'run_command', mock.MagicMock()) fixture = self.make_kvm_fixture() error_msg = self.getUniqueString() # Configure fixture.run_command to simulate a failure when # restarting apache. returns = chain([ # Simluate response for the 'config rewrite' command. (0, 'stdout', 'stderr')], repeat((1, 'stdout', error_msg))) def side_effect(*args, **kwargs): return returns.next() fixture.run_command = mock.MagicMock(side_effect=side_effect) fixture.direct_ip = '127.99.88.77' self.patch( utils, 'retries', mock.MagicMock(return_value=[(0, 1), (0, 1)])) maas_fixture = self.make_maas_fixture(fixture) error = self.assertRaises( Exception, maas_fixture.configure_default_maas_url) self.assertIn(error_msg, text_type(error)) def test_maas_admin_property_returns_maas_if_old_maas(self): # Pre-trusty MAAS servers use 'maas' to control the region controller. fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(fixture, 'run_command', mock.MagicMock( return_value=(1, b'stdout output', b'stderr output'))) admin_cmd = maas_fixture.maas_admin self.assertEqual('maas', admin_cmd) create_call = fixture.run_command.call_args_list[0] args, kwargs = create_call command_line = args[0] self.assertEqual( ['which', 'maas-region-admin'], command_line[:2]) def test_maas_admin_property_returns_maas_admin_region_if_new_maas(self): # Trusty and later MAAS servers use 'maas-region-admin' to control the # region controller. fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(fixture, 'run_command', mock.MagicMock( return_value=(0, b'stdout output', b'stderr output'))) admin_cmd = maas_fixture.maas_admin self.assertEqual('maas-region-admin', admin_cmd) create_call = fixture.run_command.call_args_list[0] args, kwargs = create_call command_line = args[0] self.assertEqual( ['which', 'maas-region-admin'], command_line[:2]) def test_create_maas_admin_creates_admin(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) maas_fixture._maas_admin = 'maas-region-admin' username, password = maas_fixture.create_maas_admin() create_call = fixture.run_command.call_args_list[0] args, kwargs = create_call command_line = args[0] self.assertEqual( ['sudo', 'maas-region-admin', 'createadmin'], command_line[:3]) self.assertIn('--username=%s' % username, command_line) self.assertIn('--password=%s' % password, command_line) def test_query_api_key_returns_api_key(self): username = "maas-user" actual_api_key = "API KEY" fixture = self.make_kvm_fixture() fixture.run_command.return_value = (0, actual_api_key, 'stderr output') maas_fixture = self.make_maas_fixture(fixture) maas_fixture._maas_admin = 'maas-region-admin' api_key = maas_fixture.query_api_key(username) args, kwargs = fixture.run_command.call_args_list[0] command_line = args[0] self.assertEqual( ['sudo', 'maas-region-admin', 'apikey', '--username=%s' % username], command_line) self.assertEqual(actual_api_key, api_key) def test_get_maas_api_client_returns_client(self): actual_api_key = "actual:maas:key" self.patch(kvmfixture, 'run_command', mock.MagicMock( return_value=(0, b'stdout output', b'stderr output'))) fixture = self.make_kvm_fixture() fixture.run_command.return_value = ( 0, actual_api_key, b'stderr output') maas_fixture = self.make_maas_fixture(fixture) client = maas_fixture.get_maas_api_client(actual_api_key) self.assertIsInstance(client, MAASClient) def test_install_maas_installs_integrates_with_install_packages(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) maas_fixture.install_maas() fixture.run_command.assert_has_call( mock.call( maas_fixture.kvm_fixture._make_apt_get_install_command( ['language-pack-en']), check_call=True), ) def test_install_maas_logs_maas_version(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) version = self.getUniqueString() mock_get_maas_version = mock.MagicMock(return_value=version) self.patch(maas_fixture, 'get_maas_version', mock_get_maas_version) mock_logging = mock.MagicMock() self.patch(maasfixture.logging, 'info', mock_logging) maas_fixture.install_maas() self.assertEqual([mock.call()], mock_get_maas_version.mock_calls) self.assertEqual( [ mock.call("Installing MAAS (version %s)..." % version), mock.call("Done installing MAAS."), ], mock_logging.mock_calls) def test_install_maas_installs_only_once(self): fixture = self.make_maas_fixture() fixture.kvm_fixture.install_packages = mock.MagicMock() fixture.install_maas() interim_installs = list( fixture.kvm_fixture.install_packages.mock_calls) interim_runs = list(fixture.kvm_fixture.run_command.mock_calls) fixture.install_maas() final_installs = list( fixture.kvm_fixture.install_packages.mock_calls) final_runs = list(fixture.kvm_fixture.run_command.mock_calls) self.assertGreater(len(interim_installs), 0) self.assertEqual(interim_installs, final_installs) self.assertGreater(len(interim_runs), 0) self.assertEqual(interim_runs, final_runs) def test_get_maas_version_returns_maas_version(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) version = self.getUniqueString() policy = "Candidate: %s\n" % version self.patch(fixture, 'run_command', mock.MagicMock( return_value=(0, policy, b'stderr'))) extracted_version = maas_fixture.get_maas_version() self.assertEqual(version, extracted_version) def test_log_connection_details_logs_info(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value='fake:api:key')) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass mock_logging = mock.MagicMock() self.patch(maasfixture.logging, 'info', mock_logging) maas_fixture.log_connection_details() self.assertEqual( [ mock.call( "MAAS server URL: http://%s/MAAS/ " "username:%s, password:%s" % ( fixture.ip_address(), maas_fixture.admin_user, maas_fixture.admin_password, ) ), mock.call( "SSH login: sudo ssh -i %s %s" % ( fixture.ssh_private_key_file, fixture.identity(), ) ), ], mock_logging.mock_calls) def test_install_maas_installs_language_pack_followed_by_maas(self): # We need to do it in this order, or postgres (and other things) may # break during installation. fixture = self.make_kvm_fixture() fake_install = mock.MagicMock() maas_fixture = self.make_maas_fixture(fixture) self.patch(fixture, 'install_packages', fake_install) maas_fixture.install_maas() self.assertEqual( [ mock.call(['language-pack-en']), mock.call(['maas', 'maas-dhcp', 'maas-dns']), ], fake_install.mock_calls) def test_get_master_ng_uuid_returns_uuid(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(maas_fixture, 'install_packages', mock.MagicMock()) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) uuid = self.getUniqueString() response = [{'uuid': uuid}] call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'get', response) self.assertEqual( (uuid, [mock.call('/api/1.0/nodegroups/', op='list')]), (maas_fixture.get_master_ng_uuid(), call_mock.mock_calls)) def test_check_cluster_connected_returns_if_cluster_connected(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(maas_fixture, 'install_packages', mock.MagicMock()) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) uuid = self.getUniqueString() response = [{'uuid': uuid}] self.patch_MAASClient( maas_fixture.admin_maas_client, 'get', response) self.assertEqual(None, maas_fixture.check_cluster_connected()) def test_check_cluster_connected_fails_if_cluster_not_connected(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(maas_fixture, 'install_packages', mock.MagicMock()) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) self.patch( utils, 'retries', mock.MagicMock(return_value=[(0, 1)])) response = [{'uuid': 'master'}] call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'get', response) self.assertRaises(Exception, maas_fixture.check_cluster_connected) self.assertEqual( [mock.call('/api/1.0/nodegroups/', op='list')], call_mock.mock_calls) def test_configure_cluster_updates_dhcp_info(self): fixture = self.make_kvm_fixture() fixture.direct_ip = '192.168.2.0' network = netaddr.IPNetwork('192.168.2.0/24') fixture.direct_network = network maas_fixture = self.make_maas_fixture(fixture) self.patch(maas_fixture, 'install_packages', mock.MagicMock()) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) self.patch( utils, 'retries', mock.MagicMock(return_value=[(0, 1)])) uuid = self.getUniqueString() response = [{'uuid': uuid}] call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'get', response) put_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'put', '') maas_fixture.configure_cluster() interface_url = '/api/1.0/nodegroups/%s/interfaces/%s/' % ( uuid, 'eth1') last_ip = "%s" % netaddr.IPAddress( fixture.direct_network.last - 1) self.assertEqual( ( [mock.call('/api/1.0/nodegroups/', op='list')], [mock.call( interface_url, ip=fixture.direct_ip, interface="eth1", subnet_mask="%s" % network.netmask, broadcast_ip="%s" % network.broadcast, router_ip=fixture.direct_ip, management=( '%s' % NODEGROUPINTERFACE_MANAGEMENT.DHCP_AND_DNS), ip_range_low=fixture.direct_first_available_ip(), ip_range_high=last_ip, )], ), (call_mock.mock_calls, put_mock.mock_calls)) def test_collect_logs_collects_logs(self): kvm_fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(kvm_fixture) maas_fixture.collect_logs() expected_calls = [ mock.call(['sudo', 'cat', filename], check_call=False) for filename in maasfixture.LOG_FILES] self.assertEqual( expected_calls, kvm_fixture.run_command.mock_calls) def test_setUp_sets_admin_details(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) fixture.direct_ip = None user = 'maas-admin-user' password = 'maas-admin-password' self.patch( maas_fixture, 'create_maas_admin', mock.MagicMock(return_value=(user, password))) key = 'fake:api:key' self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value=key)) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass client_key = "%s:%s:%s" % ( maas_fixture.admin_maas_client.auth.consumer_token.key, maas_fixture.admin_maas_client.auth.resource_token.key, maas_fixture.admin_maas_client.auth.resource_token.secret) self.assertEqual( (user, password, key, key), ( maas_fixture.admin_user, maas_fixture.admin_password, maas_fixture.admin_api_key, client_key, )) def test_setUp_calls_cleanup_methods(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value='fake:api:key')) self.patch( maas_fixture, 'collect_logs', mock.MagicMock()) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass self.assertEqual( [ [mock.call()], ], [ maas_fixture.collect_logs.mock_calls, ]) def test_configure_default_series_configures_series(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) series = self.getUniqueString() call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'post', '') maas_fixture.configure_default_series(series) self.assertEqual( [mock.call( '/api/1.0/maas/', op='set_config', name='commissioning_distro_series', value=series)], call_mock.mock_calls) def test_configure_http_proxy_configure_http_proxy(self): fixture = self.make_kvm_fixture() proxy_url = self.getUniqueString() maas_fixture = self.make_maas_fixture(fixture, proxy_url=proxy_url) self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'post', '') maas_fixture.configure_http_proxy(proxy_url) self.assertEqual( [mock.call( '/api/1.0/maas/', op='set_config', name='http_proxy', value=proxy_url)], (call_mock.mock_calls)) def test_setUp_calls_methods_if_direct_ip_set(self): fixture = self.make_kvm_fixture() fixture.direct_ip = '127.11.24.38' maas_fixture = self.make_maas_fixture(fixture) self.patch( maas_fixture, 'configure_cluster', mock.MagicMock()) self.patch( maas_fixture, 'check_cluster_connected', mock.MagicMock()) self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value='fake:api:key')) # Patch methods that we want to check setUp calls. self.patch( maas_fixture, 'configure_default_maas_url', mock.MagicMock()) self.patch( maas_fixture, 'check_cluster_connected', mock.MagicMock()) self.patch( maas_fixture, 'configure_cluster', mock.MagicMock()) self.patch( maas_fixture, 'configure_http_proxy', mock.MagicMock()) self.patch( maas_fixture, 'import_maas_images', mock.MagicMock()) self.patch( maas_fixture, 'log_connection_details', mock.MagicMock()) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass self.assertEqual( [ [mock.call()], [mock.call()], [mock.call()], [mock.call(maas_fixture.proxy_url)], [mock.call( series=maas_fixture.series, architecture=maas_fixture.architecture, simplestreams_filter=maas_fixture.simplestreams_filter)], [mock.call()], ], [ maas_fixture.configure_default_maas_url.mock_calls, maas_fixture.check_cluster_connected.mock_calls, maas_fixture.configure_cluster.mock_calls, maas_fixture.configure_http_proxy.mock_calls, maas_fixture.import_maas_images.mock_calls, maas_fixture.log_connection_details.mock_calls, ]) def test_setUp_does_not_configure_proxy_if_no_proxy(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture, proxy_url='') self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value='fake:api:key')) self.patch( maas_fixture, 'configure_http_proxy', mock.MagicMock()) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass self.assertEqual( [[]], [ maas_fixture.configure_http_proxy.mock_calls, ]) def test_setUp_does_not_call_methods_if_direct_ip_unset(self): fixture = self.make_kvm_fixture() maas_fixture = self.make_maas_fixture(fixture) self.patch( maas_fixture, 'query_api_key', mock.MagicMock(return_value='fake:api:key')) # Patch methods that we want to check setUp doesn't call. self.patch( maas_fixture, 'configure_default_maas_url', mock.MagicMock()) self.patch( maas_fixture, 'check_cluster_connected', mock.MagicMock()) self.patch( maas_fixture, 'configure_cluster', mock.MagicMock()) self.patch( maas_fixture, 'wait_until_boot_images_scanned', mock.MagicMock()) with maas_fixture: pass self.assertEqual( [[], [], []], [ maas_fixture.configure_default_maas_url.mock_calls, maas_fixture.check_cluster_connected.mock_calls, maas_fixture.configure_cluster.mock_calls, ]) def test_import_maas_images_imports_pxe_images(self): kvm_fixture = self.make_kvm_fixture() proxy_url = self.getUniqueString() maas_fixture = self.make_maas_fixture(kvm_fixture, proxy_url) series = self.getUniqueString() arch = self.getUniqueString() filter = self.getUniqueString() returned_arches = ['amd64', 'i386'] mock_mipf_arch_list = mock.MagicMock(return_value=returned_arches) self.patch( utils, 'mipf_arch_list', mock_mipf_arch_list) maas_fixture.import_maas_images(series, arch, filter) self.assertEqual( mock.call([ 'sudo', 'http_proxy=%s' % proxy_url, 'https_proxy=%s' % proxy_url, 'maas-import-pxe-files', ], check_call=True), kvm_fixture.run_command.mock_calls[-1]) self.assertEqual([mock.call(arch)], mock_mipf_arch_list.mock_calls) def test_dump_data_calls_dumpdata(self): maas_fixture = self.make_maas_fixture() maas_fixture._maas_admin = 'maas-region-admin' self.patch( maas_fixture.kvm_fixture, 'run_command', mock.MagicMock(return_value=(0, '', ''))) maas_fixture.dump_data() self.assertIn( mock.call([ 'sudo', 'maas-region-admin', 'dumpdata', 'metadataserver.NodeCommissionResult']), maas_fixture.kvm_fixture.run_command.mock_calls) def test_wait_until_boot_images_scanned_times_out_eventually(self): maas_fixture = self.make_maas_fixture() self.patch( maas_fixture, 'list_boot_images', mock.MagicMock(return_value=[])) self.patch( utils, 'retries', mock.MagicMock(return_value=[(0, 1), (0, 1)])) exception = self.assertRaises( NoBootImagesError, maas_fixture.wait_until_boot_images_scanned) message = text_type(exception) self.assertIn( "Boot image download timed out: cluster reported no images.", message) def test_wait_until_boot_images_scanned_returns_with_error(self): maas_fixture = self.make_maas_fixture() self.patch( maas_fixture, 'list_boot_images', mock.MagicMock(return_value=['non-empty1', 'non-empty2'])) self.assertEqual(None, maas_fixture.wait_until_boot_images_scanned()) def test_list_boot_images_lists_images(self): maas_fixture = self.make_maas_fixture() response = [{'fake-response': self.getUniqueString()}] self.patch(maas_fixture, 'admin_maas_client', mock.MagicMock()) call_mock = self.patch_MAASClient( maas_fixture.admin_maas_client, 'get', response) uuid = self.getUniqueString() self.patch( maas_fixture, 'get_master_ng_uuid', mock.MagicMock(return_value=uuid)) self.assertEqual( ( response, [mock.call('/api/1.0/nodegroups/%s/boot-images/' % uuid)], ), (maas_fixture.list_boot_images(), call_mock.mock_calls)) maas-test-0.1+bzr147.orig/maastest/tests/test_main.py0000644000000000000000000002312112321262143020675 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for `maastest.main`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [] import logging import os from random import randint from maastest import ( detect_dhcp, main, utils, ) import mock import testtools def make_interface_name(): """Generate an arbitrary network interface name.""" return 'itf-%d' % randint(0, 100) class TestTestMAAS(testtools.TestCase): def patch_logging(self): """Shut up logging for the duration of this test.""" # Quick and dirty. Replace with something nicer if appropriate. for logging_function in ['debug', 'info', 'error']: self.patch(logging, logging_function, mock.MagicMock()) def pretend_to_be_user(self, uid): """Make the am-I-root check think we are user no. `uid`. Pass zero for root. """ self.patch(os, 'geteuid', mock.MagicMock(return_value=uid)) def patch_kvm_ok(self, ok=True): """Patch `check_kvm_ok` to return the given value.""" self.patch(utils, 'check_kvm_ok', mock.MagicMock(return_value=ok)) def patch_virtualization_type(self, virt_type=None): """Patch `virtualization_type` to return the given value. The default, type `None`, always says this is a physical system. """ self.patch( utils, 'virtualization_type', mock.MagicMock(return_value=virt_type)) def make_args(self, interface, interactive=True): """Create a fake arguments object with the given parameters.""" args = mock.MagicMock() args.interface = interface args.interactive = interactive return args def test_refuses_to_run_if_interface_detects_dhcp_server(self): self.patch_logging() self.pretend_to_be_user(0) self.patch_kvm_ok() self.patch_virtualization_type() interface = make_interface_name() server = '127.55.33.11' self.patch( detect_dhcp, 'probe_dhcp', mock.MagicMock(return_value={server})) return_value = main.main(self.make_args(interface)) self.assertEqual( main.RETURN_CODES.UNEXPECTED_DHCP_SERVER, return_value) def test_requires_sudo(self): self.pretend_to_be_user(randint(100, 200)) self.patch_kvm_ok() self.patch_virtualization_type() interface = make_interface_name() return_value = main.main([interface, '--interactive']) self.assertEqual(main.RETURN_CODES.NOT_ROOT, return_value) def make_proxy_url(): return 'http://%d.example.com:%d' % (randint(1, 999999), randint(1, 65535)) class TestSetUpProxy(testtools.TestCase): def setUp(self): super(TestSetUpProxy, self).setUp() self.patch(main, 'make_local_proxy_fixture', mock.MagicMock()) self.patch(os, 'putenv', mock.MagicMock()) def make_args(self, http_proxy=None, disable_cache=False): """Create a fake arguments object with the given parameters.""" args = mock.MagicMock() args.http_proxy = http_proxy args.disable_cache = disable_cache return args def test_returns_empty_if_caching_disabled(self): self.assertEqual( ('', None), main.set_up_proxy(self.make_args(disable_cache=True))) def test_returns_existing_proxy_if_set(self): proxy = make_proxy_url() self.assertEqual( (proxy, None), main.set_up_proxy(self.make_args(http_proxy=proxy))) def test_creates_proxy_if_appropriate(self): proxy_url, proxy_fixture = main.set_up_proxy(self.make_args()) self.assertIsNotNone(proxy_fixture) self.addCleanup(proxy_fixture.cleanUp) self.assertEqual(proxy_fixture.get_url(), proxy_url) def test_cleans_up_proxy_on_failure(self): class DeliberateFailure(Exception): """Deliberately induced error for testing.""" self.patch( main.make_local_proxy_fixture.return_value, 'get_url', mock.MagicMock(side_effect=DeliberateFailure)) self.assertRaises( DeliberateFailure, main.set_up_proxy, self.make_args()) self.assertEqual( [mock.call()], main.make_local_proxy_fixture.return_value.cleanUp.mock_calls) def test_sets_env_to_existing_proxy(self): proxy = make_proxy_url() main.set_up_proxy(self.make_args(http_proxy=proxy)) self.assertItemsEqual( [ mock.call('http_proxy', proxy), mock.call('https_proxy', proxy), ], os.putenv.mock_calls) def test_sets_env_to_created_proxy(self): proxy_url, _ = main.set_up_proxy(self.make_args()) self.assertItemsEqual( [ mock.call('http_proxy', proxy_url), mock.call('https_proxy', proxy_url), ], os.putenv.mock_calls) def test_leaves_env_unchanged_if_caching_disabled(self): main.set_up_proxy(self.make_args(disable_cache=True)) self.assertEqual([], os.putenv.mock_calls) class TestCheckAgainstDHCPServersFromVM(testtools.TestCase): def make_fake_vm(self): """Create a minimal fake virtual-machine fixture.""" fake = mock.MagicMock() fake.direct_ip = '192.168.%d.%d' % (randint(0, 255), randint(1, 254)) return fake def patch_to_find_no_servers(self, fake): """Patch a fake VM fixture to find no DHCP servers.""" self.patch( fake, 'run_command', mock.MagicMock(return_value=(0, '', ''))) return fake def patch_to_find_servers(self, fake, servers): """Patch a fake VM fixture to find the given DHCP servers.""" return_code = len(servers) output = "DHCP servers detected: %s" % ', '.join(servers) self.patch( fake, 'run_command', mock.MagicMock(return_value=(return_code, output, ''))) def patch_has_maas_probe_dhcp(self, present=True): """Patch `has_maas_probe_dhcp` to return the given answer.""" self.patch( main, 'has_maas_probe_dhcp', mock.MagicMock(return_value=present)) def make_dhcp_server_ip(self): """Return an IP address for a fictional unexpected DHCP server.""" return '10.%d.%d.%d' % ( randint(0, 254), randint(0, 254), randint(1, 254), ) def test_passes_if_no_dhcp_servers_detected(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = self.make_fake_vm() self.patch_to_find_no_servers(fake_vm) main.check_against_dhcp_servers_from_vm(fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) def test_fails_if_dhcp_server_detected(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = self.make_fake_vm() self.patch_to_find_servers(fake_vm, [self.make_dhcp_server_ip()]) exception = self.assertRaises( main.ProgramFailure, main.check_against_dhcp_servers_from_vm, fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) self.assertEqual( main.RETURN_CODES.UNEXPECTED_DHCP_SERVER, exception.return_code) def test_fails_if_multiple_dhcp_servers_detected(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = self.make_fake_vm() self.patch_to_find_servers( fake_vm, [self.make_dhcp_server_ip() for _ in range(3)]) exception = self.assertRaises( main.ProgramFailure, main.check_against_dhcp_servers_from_vm, fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) self.assertEqual( main.RETURN_CODES.UNEXPECTED_DHCP_SERVER, exception.return_code) def test_passes_if_only_maas_dhcp_server_detected(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = self.make_fake_vm() self.patch_to_find_servers(fake_vm, [fake_vm.direct_ip]) main.check_against_dhcp_servers_from_vm(fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) def test_fails_if_maas_and_other_dhcp_server_detected(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = self.make_fake_vm() self.patch_to_find_servers( fake_vm, [fake_vm.direct_ip, self.make_dhcp_server_ip()]) exception = self.assertRaises( main.ProgramFailure, main.check_against_dhcp_servers_from_vm, fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) self.assertEqual( main.RETURN_CODES.UNEXPECTED_DHCP_SERVER, exception.return_code) def test_propagates_other_failure(self): self.patch_has_maas_probe_dhcp(present=True) fake_vm = mock.MagicMock() fake_vm.run_command = mock.MagicMock(return_value=(1, '', 'Kaboom')) exception = self.assertRaises( Exception, main.check_against_dhcp_servers_from_vm, fake_vm) self.assertEqual(1, len(fake_vm.run_command.mock_calls)) self.assertIn( "Call to maas-probe-dhcp failed in virtual machine", repr(exception)) self.assertIn('Kaboom', repr(exception)) def test_skips_if_maas_probe_dhcp_not_present(self): self.patch_has_maas_probe_dhcp(present=False) fake_vm = self.make_fake_vm() self.patch_to_find_servers(fake_vm, [self.make_dhcp_server_ip()]) self.assertEqual(0, len(fake_vm.run_command.mock_calls)) maas-test-0.1+bzr147.orig/maastest/tests/test_proxyfixture.py0000644000000000000000000001725712321262143022556 0ustar 00000000000000# Copyright 2013-2014 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for `maastest.utils`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [] import logging import os.path import random from fixtures import TempDir from maastest import ( proxyfixture, utils, ) from maastest.proxyfixture import LocalProxyFixture import mock import netifaces import testtools from testtools.matchers import DirExists class TestLocalProxyFixture(testtools.TestCase): def setUp(self): super(TestLocalProxyFixture, self).setUp() mock_ifaddresses = mock.MagicMock() mock_ifaddresses.return_value = { netifaces.AF_INET: [{ 'addr': '192.168.0.1', 'netmask': '255.255.255.0', }]} self.patch(netifaces, 'ifaddresses', mock_ifaddresses) def make_dir(self): """Create a temporary directory for the duration of this test.""" return self.useFixture(TempDir()).path def make_fixture(self, port=None, config_dir=None, pidfile_dir=None, log_dir=None): """Create a `LocalProxyFixture`. :param port: Optional port. Defaults to a random choice. :param config_dir: Optional configuration directory. Defaults to a temporary directory. :param pidfile_dir: Optional pidfile directory. Defaults to a temporary directory. :param log_dir: Optional log directory. Defaults to a temporary directory. """ if port is None: port = self.get_random_port_number() if config_dir is None: config_dir = self.make_dir() if pidfile_dir is None: pidfile_dir = self.make_dir() if log_dir is None: log_dir = self.make_dir() return LocalProxyFixture( port=port, config_dir=config_dir, pidfile_dir=pidfile_dir, log_dir=log_dir) def get_random_port_number(self): return random.randint(1, 65535) def test_init_sets_properties(self): port = self.get_random_port_number() config_dir = self.make_dir() pidfile_dir = self.make_dir() proxy_fixture = LocalProxyFixture( port=port, config_dir=config_dir, pidfile_dir=pidfile_dir) self.assertEqual(port, proxy_fixture.port) self.assertEqual(config_dir, proxy_fixture.config_dir) self.assertEqual( os.path.join(pidfile_dir, 'proxy.pid'), proxy_fixture.pid_file) def test_creates_pidfile_dir(self): pidfile_dir = os.path.join(self.make_dir(), "maas-test-pidfiles") with self.make_fixture(pidfile_dir=pidfile_dir): self.assertThat(pidfile_dir, DirExists()) def test_creates_log_dir(self): log_dir = os.path.join(self.make_dir(), "maas-test-logs") with self.make_fixture(log_dir=log_dir): self.assertThat(log_dir, DirExists()) def test_get_url_returns_proxy_url(self): port = self.get_random_port_number() proxy_fixture = self.make_fixture(port=port) self.assertEqual( "http://192.168.0.1:%s" % port, proxy_fixture.get_url()) def test_set_up_calls_all_setup_methods(self): self.patch(LocalProxyFixture, 'create_cache_root', mock.MagicMock()) self.patch(LocalProxyFixture, 'start_proxy', mock.MagicMock()) with self.make_fixture() as fixture: pass self.assertEqual( [[mock.call()], [mock.call()]], [ fixture.create_cache_root.mock_calls, fixture.start_proxy.mock_calls, ]) def test_start_proxy_starts_proxy(self): mock_run_command = mock.MagicMock() self.patch(utils, 'run_command', mock_run_command) proxy_fixture = self.make_fixture() proxy_fixture.start_proxy() self.assertEqual( mock.call( ['polipo'] + proxy_fixture._get_config_arguments(), check_call=True), mock_run_command.mock_calls[-1]) def test_create_cache_root_creates_directory(self): config_dir = self.make_dir() proxy_fixture = self.make_fixture(config_dir=config_dir) proxy_fixture.create_cache_root() expected_cache_path = proxyfixture.DISK_CACHE_ROOT % { 'config_dir': config_dir} self.assertThat(expected_cache_path, DirExists()) def test_create_cache_root_ignores_existing_directory(self): disk_cache_root = self.make_dir() self.patch(proxyfixture, 'DISK_CACHE_ROOT', disk_cache_root) proxy_fixture = self.make_fixture() proxy_fixture.create_cache_root() self.assertThat(disk_cache_root, DirExists()) def test_get_network_address(self): proxy_adddress = self.make_fixture()._get_network_address() self.assertEqual('192.168.0.1', proxy_adddress.ip.format()) self.assertEqual(24, proxy_adddress.prefixlen) def test_kill_running_proxy_kills_proxy(self): mock_run_command = mock.MagicMock() self.patch(utils, 'run_command', mock_run_command) mock_read_file = mock.MagicMock() mock_read_file.return_value = bytes(self.getUniqueInteger()) self.patch(utils, 'read_file', mock_read_file) self.make_fixture().kill_running_proxy() self.assertEqual( mock.call(['kill', mock_read_file.return_value], check_call=True), mock_run_command.mock_calls[0]) def test_kill_running_proxy_handles_nonexistent_pid_file_gracefully(self): mock_log_error = mock.MagicMock() self.patch(logging, 'error', mock_log_error) mock_read_file = mock.MagicMock() mock_read_file.side_effect = Exception(self.getUniqueString()) self.patch(utils, 'read_file', mock_read_file) self.make_fixture().kill_running_proxy() mock_log_error.assert_has_calls([ mock.call("Unable to kill proxy:"), mock.call(mock_read_file.side_effect.message)]) def test_kill_running_proxy_handles_errors_from_kill_gracefully(self): mock_log_error = mock.MagicMock() self.patch(logging, 'error', mock_log_error) mock_run_command = mock.MagicMock() mock_run_command.side_effect = Exception(self.getUniqueString()) self.patch(utils, 'run_command', mock_run_command) mock_read_file = mock.MagicMock() mock_read_file.return_value = bytes(self.getUniqueInteger()) self.patch(utils, 'read_file', mock_read_file) self.make_fixture().kill_running_proxy() mock_log_error.assert_has_calls([ mock.call("Unable to kill proxy:"), mock.call(mock_run_command.side_effect.message)]) def test_kill_running_proxy_called_on_cleanup(self): mock_run_command = mock.MagicMock() self.patch(utils, 'run_command', mock_run_command) mock_read_file = mock.MagicMock() mock_read_file.return_value = bytes(self.getUniqueInteger()) self.patch(utils, 'read_file', mock_read_file) mock_kill_running_proxy = mock.MagicMock() proxy_fixture = self.make_fixture() self.patch( proxy_fixture, 'kill_running_proxy', mock_kill_running_proxy) proxy_fixture.setUp() proxy_fixture.cleanUp() mock_kill_running_proxy.assert_has_call([mock.call()]) def test_setUp_checks_for_pidfile(self): mock_exists = mock.MagicMock() mock_exists.return_value = True self.patch(os.path, 'exists', mock_exists) proxy_fixture = self.make_fixture() self.assertRaises(Exception, proxy_fixture.setUp) maas-test-0.1+bzr147.orig/maastest/tests/test_report.py0000644000000000000000000002520212321262143021266 0ustar 00000000000000# Copyright 2012 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for `maastest.report`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [] import base64 from datetime import datetime import email import io import logging import os.path import urllib2 from fixtures import TempDir from maastest import report import mock import testtools class TestMakeFilePayload(testtools.TestCase): def test_returns_MIMEApplication(self): payload = report.make_file_payload( self.getUniqueString(), self.getUniqueString()) self.assertIsInstance( payload, email.mime.application.MIMEApplication) def test_sets_content_disposition_header(self): filename = self.getUniqueString() payload = report.make_file_payload( filename, self.getUniqueString()) expected_disposition = ( 'attachment; name="%(name)s"; filename="%(name)s"' % {'name': filename}) self.assertEqual( expected_disposition, payload['Content-Disposition']) def test_sets_payload(self): content = self.getUniqueString() filename = self.getUniqueString() payload = report.make_file_payload(filename, content) encoded_content = base64.b64encode(content) self.assertEqual( encoded_content, payload.get_payload().replace("\n", "")) class TestCreateLaunchpadBlob(testtools.TestCase): def test_returns_MIMEMultipart(self): blob = report.create_launchpad_blob(self.getUniqueString(), False) self.assertIsInstance(blob, email.mime.multipart.MIMEMultipart) def test_sets_tags_header_for_success(self): blob = report.create_launchpad_blob( self.getUniqueString(), test_succeeded=True) self.assertEqual("success", blob['tags']) def test_sets_tags_header_for_failure(self): blob = report.create_launchpad_blob( self.getUniqueString(), test_succeeded=False) self.assertEqual("failure", blob['tags']) def test_sets_subject_header_for_success(self): blob = report.create_launchpad_blob( self.getUniqueString(), test_succeeded=True) self.assertEqual("maas-test success", blob['subject']) def test_sets_subject_header_for_failure(self): blob = report.create_launchpad_blob( self.getUniqueString(), test_succeeded=False) self.assertEqual("maas-test failure", blob['subject']) def test_sets_private_header(self): blob = report.create_launchpad_blob(self.getUniqueString(), False) self.assertEqual("yes", blob['private']) def test_sets_subscribers_header(self): blob = report.create_launchpad_blob(self.getUniqueString(), False) self.assertEqual("private-canonical-maas", blob['subscribers']) def test_attaches_results_as_payload(self): results = self.getUniqueString() blob = report.create_launchpad_blob(results, False) expected_payload = report.make_file_payload("maas-test.log", results) actual_payload = blob.get_payload()[0] self.assertEqual( expected_payload['Content-Disposition'], actual_payload['Content-Disposition']) self.assertEqual( expected_payload.get_payload(), actual_payload.get_payload()) class TestBuildUploadMIMEMultipart(testtools.TestCase): def setUp(self): super(TestBuildUploadMIMEMultipart, self).setUp() self.launchpad_blob = report.create_launchpad_blob( self.getUniqueString(), False) def test_returns_MIMEMultipart(self): upload_multipart = report.build_upload_mime_multipart( self.launchpad_blob) self.assertIsInstance( upload_multipart, email.mime.multipart.MIMEMultipart) def test_adds_submit_payload(self): upload_multipart = report.build_upload_mime_multipart( self.launchpad_blob) submit_payload = upload_multipart.get_payload()[0] self.assertIsInstance(submit_payload, email.mime.Text.MIMEText) self.assertEqual( submit_payload['Content-Disposition'], 'form-data; name="FORM_SUBMIT"') self.assertEqual('1', submit_payload.get_payload()) def test_adds_blob_as_payload(self): upload_multipart = report.build_upload_mime_multipart( self.launchpad_blob) blob_payload = upload_multipart.get_payload()[1] self.assertEqual( blob_payload['Content-Disposition'], 'form-data; name="field.blob"; filename="x"') self.assertEqual( self.launchpad_blob.as_string().encode('ascii'), blob_payload.get_payload()) class TestBuildUploadRequest(testtools.TestCase): def setUp(self): super(TestBuildUploadRequest, self).setUp() self.launchpad_blob = report.create_launchpad_blob( self.getUniqueString(), False) self.launchpad_form_data = report.build_upload_mime_multipart( self.launchpad_blob) def test_returns_http_request(self): request = report.build_upload_request( "http://example.com", self.launchpad_form_data) self.assertIsInstance(request, urllib2.Request) def test_sets_request_content_type_header(self): request = report.build_upload_request( "http://example.com", self.launchpad_form_data) self.assertRegexpMatches( request.get_header("Content-type"), 'multipart/form-data; boundary=.*') def test_sets_request_data(self): flat_buffer = io.BytesIO() generator = email.generator.Generator(flat_buffer, mangle_from_=False) generator.flatten(self.launchpad_form_data) flattened_data = flat_buffer.getvalue() request = report.build_upload_request( "http://example.com", self.launchpad_form_data) self.assertEqual(flattened_data, request.data) class TestUploadBlob(testtools.TestCase): def setUp(self): super(TestUploadBlob, self).setUp() self.mock_opener = mock.MagicMock() self.mock_result = self.mock_opener.open.return_value self.patch( report, 'build_opener', mock.MagicMock(return_value=self.mock_opener)) def test_builds_opener(self): results = self.getUniqueString() blob = report.create_launchpad_blob(results, False) report.upload_blob(blob) report.build_opener.assert_has_calls( [mock.call(urllib2.HTTPSHandler, report.NO_PROXY_HANDLER)]) def test_opens_url(self): results = self.getUniqueString() blob = report.create_launchpad_blob(results, False) mock_request = mock.MagicMock() mock_build_upload_request = mock.MagicMock( return_value=mock_request) self.patch(report, 'build_upload_request', mock_build_upload_request) report.upload_blob(blob) self.mock_opener.open.assert_has_calls([mock.call(mock_request)]) def test_returns_token(self): token = self.getUniqueString() mock_info = mock.MagicMock( return_value={'X-Launchpad-Blob-Token': token}) self.mock_result.info = mock_info blob = report.create_launchpad_blob(self.getUniqueString(), False) self.assertEqual(token, report.upload_blob(blob)) def test_returns_none_if_request(self): self.mock_opener.open.return_value = None self.mock_opener.open.side_effect = Exception(self.getUniqueString()) mock_error = mock.MagicMock() self.patch(logging, 'error', mock_error) blob = report.create_launchpad_blob(self.getUniqueString(), False) expected_message = ('Unable to connect to Launchpad.') token = report.upload_blob(blob) mock_error.assert_has_calls([mock.call(expected_message, exc_info=1)]) self.assertEqual(None, token) class TestReportTestResults(testtools.TestCase): def test_uploads_blob(self): token = self.getUniqueString() blob = email.mime.multipart.MIMEMultipart() mock_logger = mock.MagicMock() mock_create_launchpad_blob = mock.MagicMock(return_value=blob) mock_upload_blob = mock.MagicMock(return_value=token) mock_write_test_results = mock.MagicMock() self.patch(logging, 'info', mock_logger) self.patch(report, 'create_launchpad_blob', mock_create_launchpad_blob) self.patch(report, 'upload_blob', mock_upload_blob) self.patch(report, 'write_test_results', mock_write_test_results) report.report_test_results( self.getUniqueString(), test_succeeded=False) mock_upload_blob.assert_has_calls([mock.call(blob)]) mock_logger.assert_has_calls([ mock.call("Test results uploaded to Launchpad."), mock.call( "Visit " "https://bugs.launchpad.net/maas-test-reports/+filebug/%s " "to file a bug and complete the maas-test reporting process." % token) ]) class TestWriteTestResults(testtools.TestCase): def test_writes_results_to_file(self): temp_dir = self.useFixture(TempDir()).path results = self.getUniqueString() log_file_name = self.getUniqueString() report.write_test_results(results, log_file_name, temp_dir) with open(os.path.join(temp_dir, log_file_name), 'rb') as log_file: self.assertEqual(results, log_file.read().strip()) def test_creates_log_dir(self): temp_dir = self.useFixture(TempDir()).path results = self.getUniqueString() log_dir = os.path.join(temp_dir, self.getUniqueString()) log_file_name = self.getUniqueString() self.assertFalse(os.path.exists(log_dir)) report.write_test_results(results, log_file_name, log_dir) self.assertTrue(os.path.exists(log_dir)) def test_saves_log_with_timestamp(self): temp_dir = self.useFixture(TempDir()).path results = self.getUniqueString() mock_logger = mock.MagicMock() self.patch(logging, 'info', mock_logger) results = self.getUniqueString() now = datetime(2013, 12, 17, 7, 18, 0) now_string = now.strftime("%Y-%m-%d_%H:%M:%S") expected_filename = 'maas-test.%s.log' % now_string expected_file_path = os.path.join(temp_dir, expected_filename) report.write_test_results(results, log_dir=temp_dir, now=now) self.assertIn( mock.call("Test log saved to %s." % expected_file_path), mock_logger.mock_calls ) self.assertTrue(os.path.exists(expected_file_path)) maas-test-0.1+bzr147.orig/maastest/tests/test_utils.py0000644000000000000000000003755512321262143021131 0ustar 00000000000000# Copyright 2013 Canonical Ltd. This software is licensed under the # GNU Affero General Public License version 3 (see the file LICENSE). """Tests for `maastest.utils`.""" from __future__ import ( absolute_import, print_function, unicode_literals, ) __metaclass__ = type __all__ = [] from io import BytesIO import os.path from pipes import quote import platform from subprocess import PIPE from textwrap import dedent import time import re import distro_info from fixtures import TempDir from maastest import utils from maastest.utils import binary_content import mock from six import text_type import testtools from testtools.matchers import ( ContainsAll, MatchesRegex, MatchesStructure, ) class TestRunCommand(testtools.TestCase): def test_read_file_returns_file_contents_as_bytes(self): temp_dir = self.useFixture(TempDir()) sample_file = os.path.join(temp_dir.path, self.getUniqueString()) contents = self.getUniqueString().encode('ascii') with open(sample_file, 'wb') as f: f.write(contents) read_contents = utils.read_file(sample_file) self.assertEqual(contents, read_contents) self.assertIsInstance(contents, bytes) def test_run_command_calls_Popen(self): mock_Popen = mock.MagicMock() expected_retcode = self.getUniqueInteger() expected_stdout = self.getUniqueString() expected_stderr = self.getUniqueString() mock_Popen.return_value.returncode = expected_retcode mock_Popen.return_value.communicate.return_value = ( expected_stdout, expected_stderr) self.patch(utils, 'Popen', mock_Popen) args = ['one', 'two'] retcode, stdout, stderr = utils.run_command(args) self.assertEqual( [ mock.call( ['one', 'two'], stdout=PIPE, stderr=PIPE, stdin=PIPE, shell=False), mock.call().communicate(None), ], mock_Popen.mock_calls) self.assertEqual( (expected_retcode, expected_stdout, expected_stderr), (retcode, stdout, stderr)) def test_run_command_runs_command(self): retcode, stdout, stderr = utils.run_command(['ls', '/']) self.assertEqual((0, b''), (retcode, stderr)) self.assertIn(b"boot", stdout) def test_run_command_checks_return_value(self): mock_Popen = mock.MagicMock() expected_retcode = 2 expected_stdout = self.getUniqueString() expected_stderr = self.getUniqueString() mock_Popen.return_value.returncode = expected_retcode mock_Popen.return_value.communicate.return_value = ( expected_stdout, expected_stderr) self.patch(utils, 'Popen', mock_Popen) args = ['one', 'two'] error = self.assertRaises( Exception, utils.run_command, args, check_call=True) self.assertIn(expected_stdout, text_type(error)) self.assertIn(expected_stderr, text_type(error)) def test_run_command_uses_input(self): input_string = self.getUniqueString().encode("ascii") retcode, stdout, stderr = utils.run_command( ['cat', '-'], input=input_string) self.assertEqual( (0, b'', input_string), (retcode, stderr, stdout)) def test_make_exception_contains_details(self): args = ['ls', '/'] retcode = 58723 stdout = self.getUniqueString() stderr = self.getUniqueString() exception = utils.make_exception(args, retcode, stdout, stderr) self.assertThat( text_type(exception), ContainsAll([ stdout, stderr, str(retcode), " ".join(quote(arg) for arg in args), ])) class TestBinaryContent(testtools.TestCase): """Tests for `binary_content`.""" def test_returns_same_content(self): content = binary_content(b'abc123') self.assertEqual( b''.join(content.iter_bytes()), b'abc123') def test_has_appropriate_content_type(self): content = binary_content(b'abc123') self.assertThat( content.content_type, MatchesStructure.byEquality( type="application", subtype="octet-stream", parameters={}, )) class TestGetURI(testtools.TestCase): """Tests for `get_uri`.""" def test_returns_api_root_plus_path(self): path = "this/is/a/path" self.assertEqual("/api/1.0/" + path, utils.get_uri(path)) class TestRetries(testtools.TestCase): def test_returns_retry_iterator(self): # Patch utils.sleep() so that no time is actually spent # sleeping. mock_sleep = mock.MagicMock() self.patch(utils, 'sleep', mock_sleep) # Patch utils.time() so that it will return [0, 0, 1, 2, ...] thus # simulating a sleep() of one second during each loop. The # double '0' at the beginning of the list is there to cope with # retries() calling time() twice before the first iteration # starts. mock_time = mock.MagicMock() values = iter([0] + range(20)) mock_time.side_effect = lambda: values.next() self.patch(utils, 'time', mock_time) timeout = 20 tries = utils.retries(timeout=timeout) self.assertEquals( (timeout, timeout), (len(list(tries)), len(mock_sleep.mock_calls))) def test_returns_immediately(self): # When iterating over a generator returned by retries(), the # first object is returned immediately; sleep() is only called # between iterations. delay = 10 tries = utils.retries(delay=delay) start = time.time() for _ in tries: elapsed = time.time() - start break self.assertLess(elapsed, delay) class TestDetermineVMArchitecture(testtools.TestCase): def test_determine_vm_architecture_gets_system_arch(self): # Additions may become necessary. supported_architectures = { 'amd64', 'armhf', 'i386', } self.assertIn( utils.determine_vm_architecture(), supported_architectures) def test_determine_vm_architecture_returns_ubuntu_arch_name(self): # Architecture names according to Python's platform module, and their # Ubuntu equivalents. Unknown names are passed on unchanged. arch_names = { 'i386': 'i386', 'i686': 'i386', 'x86_64': 'amd64', 'armhf': 'armhf', 'power': 'power', 'sparc64': 'sparc64', 'm68k': 'm68k', } # See what architectures determine_vm_architecture() reports for these # systems. translations = {} machine_func = mock.MagicMock() self.patch(platform, 'machine', machine_func) for arch in arch_names.keys(): machine_func.return_value = arch translations[arch] = utils.determine_vm_architecture() self.assertEqual(arch_names, translations) def test_determine_vm_architecture_reports_failure(self): # platform.machine() returns an empty string when it can't determine # the architecture. This test simulates that scenario. # In python 3 it returns a unicode string, in python 2 a string of # bytes. return_type = platform.machine().__class__ self.patch(platform, 'machine', lambda: return_type()) self.assertRaises( utils.UnknownCPUArchitecture, utils.determine_vm_architecture) class TestDetermineVMSeries(testtools.TestCase): def test_determine_vm_series_returns_system_series(self): # We don't know the names of future series this test might run on, but # it'll be a lower-case name. The supported releases start with # Precise, and for the forseeable future, progress through the # alphabet from there. self.assertThat( utils.determine_vm_series(), MatchesRegex('[p-z][a-z]+$')) def test_determine_vm_series_uses_distro_information(self): series = 'nerdy' distro = ('Ubuntu', '1.01', series) self.patch(platform, 'linux_distribution', lambda: distro) self.assertEqual(series, utils.determine_vm_series()) class TestGetSupportedNodeSeries(testtools.TestCase): def test_get_supported_node_series_return_series(self): supported = ( distro_info.UbuntuDistroInfo().supported(result="codename")) # The list of supported series will evolve in time. At the time # of this writing, 'lucid' is still a supported series but it's # not supported by MAAS. if 'lucid' in supported: supported.remove('lucid') self.assertEqual(supported, utils.get_supported_node_series()) else: self.assertEqual(supported, utils.get_supported_node_series()) class TestGetSupportedMAASSeries(testtools.TestCase): def test_get_supported_maas_series_return_series(self): self.assertEqual(['trusty'], utils.get_supported_maas_series()) class TestExtractMAPIPMapping(testtools.TestCase): def test_extract_mac_ip_mapping_parses_output(self): NMAP_XML_OUTPUT = dedent("""
""") expected_result = { '00:9C:02:A2:82:74': '192.168.2.2', '00:9C:02:A0:4D:0A': '192.168.2.4', } self.assertEqual( expected_result, utils.extract_mac_ip_mapping(NMAP_XML_OUTPUT)) def test_extract_mac_ip_mapping_returns_uppercase_mac(self): NMAP_XML_OUTPUT = dedent("""
""") expected_result = {'AA:BB:CC:DD:EE:FF': '192.168.2.2'} self.assertEqual( expected_result, utils.extract_mac_ip_mapping(NMAP_XML_OUTPUT)) def test_extract_mac_ip_mapping_with_empty_doc(self): self.assertEqual( {}, utils.extract_mac_ip_mapping('')) def test_extract_mac_ip_mapping_with_missing_info(self): # If either the IP address of the MAC address is missing in a host # definition, the host entry is not considered. NMAP_XML_OUTPUT = dedent("""
""") self.assertEqual({}, utils.extract_mac_ip_mapping(NMAP_XML_OUTPUT)) class TestMipfArchList(testtools.TestCase): def test_defaults_to_generic(self): self.assertItemsEqual(['i386/generic'], utils.mipf_arch_list('i386')) def test_adds_i386_if_missing(self): self.assertEqual( (['amd64/generic', 'i386/generic'], ['i386/generic']), (utils.mipf_arch_list('amd64'), utils.mipf_arch_list('i386'))) class TestVirtualizationType(testtools.TestCase): def test_returns_None_if_not_virtualised(self): mock_run_command = mock.MagicMock() mock_run_command.return_value = (0, '', '') self.patch(utils, 'run_command', mock_run_command) self.assertIsNone(utils.virtualization_type()) def test_returns_virt_type_if_virtualised(self): virt_type = self.getUniqueString() mock_run_command = mock.MagicMock() mock_run_command.return_value = (0, virt_type, '') self.patch(utils, 'run_command', mock_run_command) self.assertEqual(virt_type, utils.virtualization_type()) class TestCheckKVM(testtools.TestCase): def test_returns_false_if_kvm_not_available(self): mock_run_command = mock.MagicMock() mock_run_command.return_value = (1, '', '') self.patch(utils, 'run_command', mock_run_command) self.assertFalse(utils.check_kvm_ok()) def test_returns_true_if_kvm_available(self): mock_run_command = mock.MagicMock() mock_run_command.return_value = (0, '', '') self.patch(utils, 'run_command', mock_run_command) self.assertTrue(utils.check_kvm_ok()) class TestCasesLoader(testtools.TestCase): def test_that_cases_are_sorted_by_lineno(self): class ExampleTest(testtools.TestCase): def test_333(self): pass def test_111(self): pass def test_222(self): pass loader = utils.CasesLoader() suite = loader.loadTestsFromTestCase(ExampleTest) self.assertEqual( ["test_333", "test_111", "test_222"], [test._testMethodName for test in suite]) class TestCachingOutputStream(testtools.TestCase): def test_writes_to_stream(self): stream = BytesIO() caching_stream = utils.CachingOutputStream(stream) output_string = self.getUniqueString() caching_stream.write(output_string) self.assertEqual(output_string, stream.getvalue()) def test_caches_output(self): stream = BytesIO() caching_stream = utils.CachingOutputStream(stream) caching_stream.write(self.getUniqueString()) self.assertEqual( stream.getvalue(), caching_stream.cache.getvalue()) def test_init_sets_values(self): stream = BytesIO() caching_stream = utils.CachingOutputStream(stream) self.assertEqual(stream, caching_stream.stream) self.assertIsInstance(caching_stream.cache, BytesIO) def test_attributes_looked_up_on_stream(self): stream = BytesIO() some_string = self.getUniqueString() stream.foo_bar_baz = mock.MagicMock(return_value=some_string) caching_stream = utils.CachingOutputStream(stream) self.assertEqual(some_string, caching_stream.foo_bar_baz()) class TestExtractPackageVersion(testtools.TestCase): def test_extracts_package_version(self): version = '1.4+bzr1693+dfsg-0ubuntu2.2' policy = dedent(""" maas: Installed: (none) Candidate: %s Version table: 1.4+bzr1693+dfsg-0ubuntu2.2 0 500 http://example.com/ubuntu/ saucy/main amd64 Packages """) % version self.assertEqual(version, utils.extract_package_version(policy)) def test_returns_None_if_policy_is_unparseable(self): self.assertIsNone(utils.extract_package_version("unparsable")) class TestComposeFilter(testtools.TestCase): def test_compose_filter_returns_single_literal(self): key = self.getUniqueString() literal = self.getUniqueString() self.assertEqual( '%s~(%s)' % (key, re.escape(literal)), utils.compose_filter(key, [literal])) def test_compose_filter_combines_literals(self): key = self.getUniqueString() values = (self.getUniqueString(), self.getUniqueString()) self.assertEqual( '%s~(%s|%s)' % ( key, re.escape(values[0]), re.escape(values[1])), utils.compose_filter(key, values)) def test_compose_filter_escapes_literals_for_regex_use(self): key = self.getUniqueString() self.assertEqual( '%s~(x\\.y\\*)' % key, utils.compose_filter(key, ['x.y*'])) maas-test-0.1+bzr147.orig/man/maas-test.80000644000000000000000000003147612321262143016133 0ustar 00000000000000.\" Man page generated from reStructuredText. . .TH MAAS-TEST 8 "" "" "" .SH NAME maas-test \- test a server for compatibility running as a MAAS node . .nr rst2man-indent-level 0 . .de1 rstReportMargin \\$1 \\n[an-margin] level \\n[rst2man-indent-level] level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] - \\n[rst2man-indent0] \\n[rst2man-indent1] \\n[rst2man-indent2] .. .de1 INDENT .\" .rstReportMargin pre: . RS \\$1 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] . nr rst2man-indent-level +1 .\" .rstReportMargin post: .. .de UNINDENT . RE .\" indent \\n[an-margin] .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] .nr rst2man-indent-level -1 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] .in \\n[rst2man-indent\\n[rst2man-indent-level]]u .. .SH SYNOPSIS .sp maas\-test \-\-interactive [options...] .sp maas\-test \-\-bmc\-mac=
[options...] .SH DESCRIPTION .sp Use \fImaas\-test\fP to test whether a server can be used as a MAAS node. It must be run as root. The test will set up a MAAS instance and attempt to manage the node. .sp Do not run \fImaas\-test\fP on the same system that you wish to test. Two systems are involved: .INDENT 0.0 .IP \(bu 2 The \fItesting system\fP\&. Run \fImaas\-test\fP here. It will create a virtual MAAS server for the duration of the test. .IP \(bu 2 The \fInode\fP\&. The MAAS server running on the testing system will control it as a node, running it through various test steps. .UNINDENT .sp MAAS controls the node remotely though IPMI. This may include powering it on or off, booting it, and even installing an operating system. .sp \fBCAUTION:\fP Future versions of \fImaas\-test\fP may wipe the node\(aqs disks and install a new operating system. And in general, accidents may happen. \fBBe prepared to lose any data stored there, and to re\-install the node after the test.\fP .sp In addition to this, even in the present version, MAAS will modify the node\(aqs firmware netboot settings. .SH NETWORK CONFIGURATION .sp Run the test in a dedicated testing network consisting of just these two systems. This network should be as isolated as possible, and does not need a route to the internet. .sp For the node, that means that both the node\(aqs own network interface card (NIC) and its baseboard management controller (BMC) should be on the testing network. The testing system only needs one NIC on the testing network. .sp In addition to being on the testing network, the testing system must also have internet access. .sp maas\-test supports two different network architectures: .sp Network config #1: the IPMI NIC is connected to the interface managed by MAAS: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C +\-\-\-\-\-\-\-\-\-\-+ +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+ | Internet | | |\-\-\-\-\-+ | | | |eth0 |+\-\-\-\-\-\-\-\-\-\-\-\->| | | |\-\-\-\-\-+ +\-\-\-\-\-\-\-\-\-\-+ | Host | | (where | NIC\(aqs MAC: aa:bb:cc:dd:ee:ff | maas\-test | | | is installed) |\-\-\-\-\-+ | | |eth1 | +\-\-\-\-\-\-+ | | | |<\-\->|Router| V +\-\-\-\-\-\-\-\-+ | |\-\-\-\-\-+ | |<\-\->+\-\-\-\-| | +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+ +\-\-\-\-\-\-+ |IPMI| | ^ |eth1| | | +\-\-\-\-| Node | | | being | | +\-\-\-\-| tested | +\-\-\-\->|eth0| | | | | +\-\-\-\-| | +\-\-\-\-\-\-\-\-+ .ft P .fi .UNINDENT .UNINDENT .sp In this case, one needs to pass the IPMI NIC\(aqs MAC address to maas\-test. The invocation of maas\-test will look something like: .sp $ maas\-test \-\-bmc\-mac aa:bb:cc:dd:ee:ff \-\-bmc\-username username \-\-bmc\-password password eth1 .sp Network config #2: the IPMI NIC is \fInot\fP connected to the interface managed by MAAS and has a fixed IP address: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C +\-\-\-\-\-\-\-\-\-\-+ +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+ | Internet | | |\-\-\-\-\-+ | | | |eth0 |+\-\-\-\-\-\-\-\-\-\-\-\->+\-\-\-\-\-\-\-\-\-\-+ | |\-\-\-\-\-+ | Host | +\-\-\-\-\-\-\-\-\-+ | (where |\-\-\-\-\-+ +\-\-\-\-| | | maas\-test |eth1 |<\-\-\-\-\-\-\-\->|eth0| | | is installed) |\-\-\-\-\-+ +\-\-\-\-| Node | | | | being | | |\-\-\-\-\-+ +\-\-\-\-| tested | | |eth2 |<\-\-\-\-\-\-\-\->|eth1| | | |\-\-\-\-\-+ |IPMI| | +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-+ +\-\-\-\-| | ^ +\-\-\-\-\-\-\-\-\-+ | Fixed IP address AA.BB.CC.DD .ft P .fi .UNINDENT .UNINDENT .sp In this case, one needs to pass the IPMI NIC\(aqs IP address to maas\-test. The invocation of maas\-test will look something like: .sp $ maas\-test \-\-bmc\-ip AA.BB.CC.DD \-\-bmc\-username username \-\-bmc\-password password eth1 .SH PREPARING TO RUN .sp The test will run MAAS in a virtual machine. It will not be installed on your physical system. Nevertheless there are a few things you need to be aware of: .INDENT 0.0 .IP 1. 3 Prepare to lose any data on the node\(aqs disks. .UNINDENT .sp 2. Ensure that your node (both its BMC and its own NIC) is connected only to the testing network. .sp 3. Make sure that there is no DHCP server running on the testing network. The test program will also check for this on startup. MAAS will act as a DHCP server on the testing network. .sp 4. Select a network interface on the testing system that provides access to the testing network. You will be passing this interface to \fImaas\-test\fP\&. .sp 5. Depending on caching, the test may download and store large amounts of data on the testing system. Make sure you have sufficient disk space and network bandwidth. .sp The data that needs downloading and/or caching consists mostly of system images for the virtual machine, and for booting the node. As a rule of thumb, count on half a gigabyte as a baseline, plus a quarter gigabyte for each combination of architecture and Ubuntu release that will run on the node. .SH RUNNING .sp There is one required argument: the network interface which connects the testing system to the testing network, e.g. \fIeth1\fP\&. .sp The test will need to power up the node. It can do that in two ways: .INDENT 0.0 .IP a. 3 Through IPMI commands to the node\(aqs BMC. You\(aqll need to specify its address and authentication information using the \fI\-\-bmc\-*\fP options. .IP b. 3 Manually when running in \fIinteractive mode\fP\&. The test will stop and ask you to power up the node. .UNINDENT .SH TEST RESULTS .sp Once maas\-test has finished testing the node it will upload the test results to Launchpad (unless told otherwise; see the \fI\%reporting options\fP). This allows you to share the test results with others, including the MAAS developers, by filing a Launchpad bug which includes the test results as an attachment. .sp By default, the results are also written to timestamped log files under \fI/var/log/maas\-test\fP\&. .SH OPTIONS .INDENT 0.0 .TP .B \-h\fP,\fB \-\-help Show help and exit. .TP .BI \-\-bmc\-mac\fB= MAC MAC address for the node\(aqs baseboard management controller. MAAS will control the node\(aqs power and boot sequence through this controller. It must be attached to the testing network. This is mutually exclusive with \fB\-\-bmc\-ip\fP\&. This option is not needed in interactive mode. In non\-interactive mode, either \fB\-\-bmc\-mac\fP or \fB\-\-bmc\-ip\fP is required. .TP .BI \-\-bmc\-ip\fB= IP IP address of the node\(aqs baseboard management controller. Use this if the BMC is not connected to the interface given as argument. Note that the IP address must not change for the duration of the testing. This is mutually exclusive with \fB\-\-bmc\-mac\fP\&. This option is not needed in interactive mode. In non\-interactive mode, either \fB\-\-bmc\-mac\fP or \fB\-\-bmc\-ip\fP is required. .TP .BI \-\-bmc\-password\fB= password Password for IPMI authentication on the BMC. Use with \fB\-\-bmc\-user\fP\&. Not needed in interactive mode. .TP .BI \-\-bmc\-user\fB= user Username for IPMI authentication. Use with \fB\-\-bmc\-password\fP\&. Not needed in interactive mode. .TP .BI \-\-ipmi\-driver\fB= driver Specify IPMI driver version. Default is LAN_2_0 (IPMI v2.0), which is what most modern BMCs support. Use the LAN option if your BMC only supports IPMI version 1.5. .TP .B \-\-interactive Interactive mode. Instead of powering up the node automatically through IPMI, prompt the user to turn it on manually. In this mode there is no need to specify BMC details; the MAAS enlistment process will discover them automatically. .TP .BI \-\-archive\fB= archive Optional package repository name. If given, the virtual machine may install packages from this additional archive as well as from the main Ubuntu archive. The archive is added to the virtual machine using \(aqadd\-apt\-repository\(aq; it may be either a line in the format of apt\(aqs sources.list, or a personal package archive identifier in the form \(aqppa:/\(aq, or a distribution component that should be enabled. This is typically used to test recent versions of MAAS that are only available in a PPA such as "ppa:maas\-maintainers/dailybuilds". This can be specified multiple times. .TP .BI \-\-series\fB= codename Code name for the Ubuntu release series that should be run on the node during enlistment and commissioning, e.g. "saucy" for 13.10 Saucy Salamander. Defaults to the latest long\-term support release of Ubuntu. .TP .BI \-\-architecture\fB= architecture CPU architecture for the node. MAAS will import boot images for this architecture only. The architecture may include a sub\-architecture name, which defaults to \fIgeneric\fP, so e.g. \fIi386/generic\fP may be abbreviated to \fIi386\fP\&. The default architecture is \fIamd64\fP\&. .TP .BI \-\-maas\-series\fB= codename Code name for the Ubuntu release series to install on the virtual machine (where the MAAS server will be installed). Defaults to the latest stable Ubuntu series. .TP .BI \-\-http\-proxy\fB= URL Use the given HTTP proxy for all downloads, both on the testing system and on the nodes: KVM images, MAAS boot images, and Ubuntu packages. Like \fI\-\-disable\-cache\fP, this also disables the caching proxy that \fImaas\-test\fP runs by default. .TP .B \-\-disable\-cache Do not run a caching HTTP proxy on the testing system. This cache is normally used for all downloads, both on the testing system and on the nodes: KVM images, MAAS boot images, and Ubuntu packages. It speeds up subsequent test runs, but also caches a large amount of data on the testing system\(aqs filesystem. The proxy software used is \fIpolipo\fP\&. The cache will be stored under \fI/var/cache/maas\-test\fP\&. .TP .B \-\-dry\-run Bring up the MAAS region controller in a virtual machine, but don\(aqt attempt to boot any machine on its network or do any destructive testing. .UNINDENT .INDENT 0.0 .TP .B \-\-no\-reporting Turn off all reporting of test results. Results will be written to stdout but not recorded elsewhere. .TP .B \-\-log\-results\-only Write test results to a file, but don\(aqt upload them to Launchpad. Results will be written to a timestamped log file under \fI/var/log/maas\-test\fP\&. .UNINDENT .SH REPORTING BUGS .sp Report bugs in Launchpad: \fI\%https://bugs.launchpad.net/maas\-test/\fP .SH SEE ALSO .sp The maas\-test program is part of the MAAS project. Find out more about MAAS at \fI\%http://maas.ubuntu.com/\fP .sp For maas\-test development, see \fI\%https://launchpad.net/maas\-test/\fP .sp Polipo caching proxy: \fI\%http://www.pps.univ\-paris\-diderot.fr/~jch/software/polipo/\fP .sp Ubuntu virtualization tools (uvtool): \fI\%https://launchpad.net/uvtool\fP .SH FILES .sp State and configuration for \fImaas\-test\fP is stored in \fI/var/cache/maas\-test\fP\&. This includes an ssh key pair for communicating with the virtual machine. Pidfiles are stored in \fI/run/maas\-test\fP, and logs and test results are written to \fI/var/log/maas\-test\fP\&. .sp If you choose to run a local proxy, downloaded data will also be cached in the \fI/var/cache/maas\-test\fP\&. It can quickly grow to gigabyte sizes. .SH AUTHOR MAAS engineering team at Canonical, Ltd. .SH COPYRIGHT Copyright (c) 2013, Canonical Ltd. .\" Generated by docutils manpage writer. .