pax_global_header00006660000000000000000000000064136114062630014514gustar00rootroot0000000000000052 comment=d92315ef25047d3e61164c2be0302f73cab86fb4 debspawn-0.4.0/000077500000000000000000000000001361140626300133205ustar00rootroot00000000000000debspawn-0.4.0/.gitignore000066400000000000000000000022631361140626300153130ustar00rootroot00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # C extensions *.so # Distribution / packaging .Python build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ lib/ lib64/ parts/ sdist/ var/ wheels/ *.egg-info/ .installed.cfg *.egg MANIFEST # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *.cover .hypothesis/ .pytest_cache/ # Translations *.mo *.pot # Django stuff: *.log local_settings.py db.sqlite3 # Flask stuff: instance/ .webassets-cache # Scrapy stuff: .scrapy # Sphinx documentation docs/_build/ # PyBuilder target/ # Jupyter Notebook .ipynb_checkpoints # pyenv .python-version # celery beat schedule file celerybeat-schedule # SageMath parsed files *.sage.py # Environments .env .venv env/ venv/ ENV/ env.bak/ venv.bak/ # Spyder project settings .spyderproject .spyproject # Rope project settings .ropeproject # mkdocs documentation /site # mypy .mypy_cache/ debspawn-0.4.0/.travis.yml000066400000000000000000000010551361140626300154320ustar00rootroot00000000000000# Travis CI config for Debspawn language: python dist: bionic sudo: required addons: apt: update: true packages: - xsltproc - docbook-xsl - docbook-xml - zstd - systemd-container - debootstrap python: - "3.7" - "3.8" - "nightly" matrix: allow_failures: - python: "nightly" install: - pip install flake8 pytest script: - ./setup.py build - ./setup.py install --single-version-externally-managed --root=/tmp - sudo $(which python3) -m pytest - rm -rf build/ - flake8 ./ - flake8 debspawn/dsrun debspawn-0.4.0/AUTHORS000066400000000000000000000000501361140626300143630ustar00rootroot00000000000000Matthias Klumpp debspawn-0.4.0/LICENSE000066400000000000000000000167431361140626300143400ustar00rootroot00000000000000 GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below. 0. Additional Definitions. As used herein, "this License" refers to version 3 of the GNU Lesser General Public License, and the "GNU GPL" refers to version 3 of the GNU General Public License. "The Library" refers to a covered work governed by this License, other than an Application or a Combined Work as defined below. An "Application" is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library. A "Combined Work" is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the "Linked Version". The "Minimal Corresponding Source" for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version. The "Corresponding Application Code" for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work. 1. Exception to Section 3 of the GNU GPL. You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL. 2. Conveying Modified Versions. If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version: a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy. 3. Object Code Incorporating Material from Library Header Files. The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following: a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the object code with a copy of the GNU GPL and this license document. 4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following: a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the Combined Work with a copy of the GNU GPL and this license document. c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document. d) Do one of the following: 0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source. 1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version. e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.) 5. Combined Libraries. You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License. b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 6. Revised Versions of the GNU Lesser General Public License. The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library. debspawn-0.4.0/MANIFEST.in000066400000000000000000000000271361140626300150550ustar00rootroot00000000000000include debspawn/dsrun debspawn-0.4.0/NEWS000066400000000000000000000057221361140626300140250ustar00rootroot00000000000000Version 0.4.0 ~~~~~~~~~~~~~~ Released: 2020-01-20 Features: * Implement an interactive build mode * Store a copy of the build log by default * Allow copying back changes in interactive mode * Use a bit of color in errors and warnings, if possible * Update manual pages * Permit recreation of images, instead of just updating them Bugfixes: * Move dsrun helper into the package itself * Drop some unwanted files from /dev before creating OS tarballs * Remove d/files file if it's created by Debspawn pre-build * Interactive mode and build logs are mutually exclusive for now * Add MANIFEST file Version 0.3.0 ~~~~~~~~~~~~~~ Released: 2020-01-06 Features: * Allow to override temporary directory path explicitly in config * Allow full sources.list customization at image creation time * Add initial test infrastructure * Allow 'b' shorthand for the 'build' subparser (Mo Zhou) * Allow turning on d/rules clean on the host, disable it by default * Allow selected environment variables to survive auto-sudo * Implement way to run Lintian as part of the build * Print pretty error message if configuration JSON is broken * Prefer hardlinks over copies when creating the APT package cache * Implement support for injecting packages * docs: Add a note about how to inject packages * Only install minimal Python in containers * Harmonize project name (= Debspawn spelling everywhere) * Add command to list installed container image details * Update sbuild replacement note Bugfixes: * Ensure we have absolute paths for debspawn run * Don't fail running command without build/artifacts directory * Build packages with epochs correctly when building from source-dir * Sign packages with an epoch correctly * Change HOME when dropping privileges * Don't install arch-indep build-deps on arch-only builds * Shorten nspawn machine name when hostname is exceptionally long * tests: Test container updates * Ensure all data lands in its intended directories when installing Version 0.2.1 ~~~~~~~~~~~~~~ Released: 2019-01-10 Features: * Allow giving the container extra capabilities easily for custom commands * Allow giving the container permission to access the host's /dev * Allow creating an image with a suite and base-suite Version 0.2.0 ~~~~~~~~~~~~~~ Released: 2018-08-28 Features: * Allow specifying enabled archive components at image creation time * Support printing the program version to stdout * Allow diverting the maintainer address * Prepare container for arbitrary run action similarly to package build * Support more build-only choices * Print some basic system info to the log * Log some basic disk space stats before/after build Bugfixes: * random.choices is only available since Python 3.6, replace it * Enforce dsrun to be installed in a location were we can find it * Ensure we don't try to link journals * Force new configuration by default, not old one * Set environment shell Version 0.1.0 ~~~~~~~~~~~~~~ Released: 2018-08-20 Notes: * Initial release debspawn-0.4.0/README.md000066400000000000000000000176301361140626300146060ustar00rootroot00000000000000# Debspawn [![Build Status](https://travis-ci.org/lkorigin/debspawn.svg?branch=master)](https://travis-ci.org/lkorigin/debspawn) Debspawn is a tool to build Debian packages in an isolated environment. Unlike similar tools like `sbuild` or `pbuilder`, `debspawn` uses `systemd-nspawn` instead of plain chroots to manage the isolated environment. This allows Debspawn to isolate builds from the host system much further via container technology. It also allows for more advanced features to manage builds, for example setting resource limits for individual builds. Please keep in mind that Debspawn is *not* a security feature! While it provides a lot of isolation from the host system, you should not run arbitrary untrusted code with it. The usual warnings for all container technology apply here. Debspawn also allows one to run arbitrary custom commands in its environment. This is used by the Laniakea[1] Spark workers to execute a variety of non-package builds and QA actions in the same environment in which we usually build packages. Debspawn was built with simplicity in mind. It should both be usable in an automated environment on large build farms, as well as on a personal workstation by a human user. Due to that, the most common operations are as easily accessible as possible. Additionally, `debspawn` will always try to do the right thing automatically before resorting to a flag that the user has to set. Options which change the build environment are - with one exception - not made available intentionally, so achieving reproducible builds is easier. See the FAQ below for more details. [1]: https://github.com/lkorigin/laniakea ## Usage ### Installing Debspawn #### Via the Debian package On Debian/Ubuntu, simply run ```bash sudo apt install debspawn ``` to start using Debspawn. #### Via the Git repository Clone the Git repository, install the (build and runtime) dependencies of `debspawn`: ```bash sudo apt install xsltproc docbook-xsl python3-setuptools zstd systemd-container debootstrap ``` You can the run `debspawn.py` directly from the Git repository, or choose to install it: ```bash sudo pip3 install --no-binary debspawn . ``` (or use `sudo python3 setup.py install --single-version-externally-managed --root=/` to install without pip) Debspawn requires at least Python 3.5. We try to keep the dependency footprint of this tool as small as possible, so it is not planned to raise that requirement or add any more dependencies anytime soon. ### On superuser permission If `sudo` is available on the system, `debspawn` will automatically request root permission when it needs it, there is no need to run it as root explicitly. If it can not obtain privileges, `debspawn` will exit with the appropriate error message. ### Creating a new image You can easily create images for any suite that has a script in `debootstrap`. For Debian Unstable for example: ```bash $ debspawn create sid ``` This will create a Debian Sid (unstable) image for the current system architecture. To create an image for testing Ubuntu builds: ```bash $ debspawn create --arch=i386 cosmic ``` This creates an `i386` image for Ubuntu 18.10. If you want to use a different mirror than set by default, pass it with the `--mirror` option. ### Refreshing an image Just run `debspawn update` and give the details of the base image that should be updated: ```bash $ debspawn update sid $ debspawn update --arch=i386 cosmic ``` This will update the base image contents and perform other maintenance actions. ### Building a package You can build a package from its source directory, or just by passing a plain `.dsc` file to `debspawn`. If the result should be automatically signed, the `--sign` flag needs to be passed too: ```bash $ cd ~/packages/hello $ debspawn build sid --sign $ debspawn build --arch=i386 cosmic ./hello_2.10-1.dsc ``` Build results are by default returned in `/var/lib/debspawn/results/` ### Building a package - with git-buildpackage You can use a command like this to build your project with gbp and Debspawn: ```bash $ gbp buildpackage --git-builder='debspawn build sid --sign' ``` ### Manual interactive-shell action If you want to, you can log into the container environment and either play around in ephemeral mode with no persistent changes, or pass `--persistent` to `debspawn` so all changes are permanently saved: ```bash $ debspawn login sid # Attention! This may alter the build environment! $ debspawn login --persistent sid ``` ### Deleting a container image At some point, you may want to permanently remove a container image again, for example because the release it was built for went end of life. This is easily done as well: ```bash $ debspawn delete sid $ debspawn delete --arch=i386 cosmic ``` ### Running arbitrary commands This is achieved with the `debspawn run` command and is a bit more involved. Refer to the manual page and help output for more information. ## FAQ #### Why use systemd-nspawn? Why not $other_container? Systemd-nspawn is a very lightweight container solution readily available without much (or any) setup on all Linux systems that are running systemd. It does not need any background daemon and while it is light on features, it fits the relatively simple usecase of building in an isolated environment perfectly. #### Do I need to set up apt-cacher-ng to use this efficiently? No - while `apt-cacher-ng` is generally a useful tool, it is not required for efficient use of `debspawn`. `debspawn` will cache downloaded packages between runs fully automatically, so packages only get downloaded when they have not been retrieved before. #### Is the build environment the same as sbuild? No, unfortunately. Due to the different technology used, there are subtle differences between sbuild chroots and `debspawn` containers. The differences should not have any impact on package builds, and any such occurrence is highly likely a bug in the package's build process. If you think it is not, please file a bug against Debspawn. We try to be as close to sbuild's default environment as possible. One way the build environment differs from Debian's default sbuild setup intentionally is in its consistent use of unicode. By default, `debspawn` will ensure that unicode is always available and default. If you do not want this behavior, you can pass the `--no-unicode` flag to `debspawn` to disable unicode in the tool itself and in the build environment. #### Will this replace sbuild? Not in the foreseeable future on Debian itself. Sbuild is a proven tool that works well for Debian and supports other OSes than Linux, while `debspawn` is Linux-only, a thing that will not change. However, Laniakea-using derivatives such as PureOS use the tool for building all packages and for constructing other build environments to e.g. build disk images. #### What is the relation of this project with Laniakea? The Laniakea job runner uses `debspawn` for a bunch of tasks and the integration with the Laniakea system is generally quite tight. Of course you can use `debspawn` without Laniakea and integrate it with any tool you want. Debspawn will always be usable without Laniakea automation. #### This tool is really fast! What is the secret? Surprisingly, building packages with `debspawn` is often a bit faster than using `pbuilder` and `sbuild` with their default settings. The speed gain comes in large part from the internal use of the Zstandard compression algorithm for base images. Zstd allows for fast decompression of the tarballs, which is exactly why it was chosen (LZ4 would be even faster, but Zstd actually is a good compromise between compression ration and speed). This shaves off a few seconds of time for each build that is used on base image decompression. Additionally, Debspawn uses `eatmydata` to disable fsync & co. by default in a few places, improving the time it takes to set up the build environment by quite a bit as well. If you want, you can configure other tools to make use of the same methods (eatmydata & zstd) as well and see if they run faster. debspawn-0.4.0/RELEASE000066400000000000000000000013141361140626300143220ustar00rootroot00000000000000Debspawn Release Notes 1. Write NEWS entries for Debspawn in the same format as usual. git shortlog v0.3.0.. | grep -i -v trivial | grep -v Merge > NEWS.new -------------------------------------------------------------------------------- Version 0.3.1 ~~~~~~~~~~~~~~ Released: 2020-xx-xx Notes: Features: Bugfixes: -------------------------------------------------------------------------------- 2. Commit changes in Git: git commit -a -m "Release version 0.3.1" git tag -s -f -m "Release 0.3.1" v0.3.1 git push --tags git push 3. Do post release version bump in `RELEASE` and `debspawn/__init__.py` 4. Commit trivial changes: git commit -a -m "trivial: post release version bump" git push debspawn-0.4.0/debspawn.py000077500000000000000000000003531361140626300155010ustar00rootroot00000000000000#!/usr/bin/env python3 import os import sys from debspawn import cli thisfile = __file__ if not os.path.isabs(thisfile): thisfile = os.path.normpath(os.path.join(os.getcwd(), thisfile)) sys.exit(cli.run(thisfile, sys.argv[1:])) debspawn-0.4.0/debspawn/000077500000000000000000000000001361140626300151235ustar00rootroot00000000000000debspawn-0.4.0/debspawn/__init__.py000066400000000000000000000015421361140626300172360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . __appname__ = "debspawn" __version__ = "0.4.0" debspawn-0.4.0/debspawn/aptcache.py000066400000000000000000000063341361140626300172530ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import shutil from pathlib import Path from glob import glob from .utils.misc import hardlink_or_copy class APTCache: ''' Manage cache of APT packages ''' def __init__(self, osbase): self._cache_dir = os.path.join(osbase.global_config.aptcache_dir, osbase.name) def merge_from_dir(self, tmp_cache_dir): ''' Merge in packages from a temporary cache ''' from random import choice from string import ascii_lowercase, digits Path(self._cache_dir).mkdir(parents=True, exist_ok=True) for pkg_fname in glob(os.path.join(tmp_cache_dir, '*.deb')): pkg_basename = os.path.basename(pkg_fname) pkg_cachepath = os.path.join(self._cache_dir, pkg_basename) if not os.path.isfile(pkg_cachepath): pkg_tmp_name = pkg_cachepath + '.tmp-' + ''.join(choice(ascii_lowercase + digits) for _ in range(8)) shutil.copy2(pkg_fname, pkg_tmp_name) try: os.rename(pkg_tmp_name, pkg_cachepath) except OSError: # maybe some other debspawn instance tried to add the package just now, # in that case we give up os.remove(pkg_tmp_name) def create_instance_cache(self, tmp_cache_dir): ''' Copy the cache to a temporary location for use in a new container instance. ''' Path(self._cache_dir).mkdir(parents=True, exist_ok=True) Path(tmp_cache_dir).mkdir(parents=True, exist_ok=True) for pkg_fname in glob(os.path.join(self._cache_dir, '*.deb')): pkg_cachepath = os.path.join(tmp_cache_dir, os.path.basename(pkg_fname)) if not os.path.isfile(pkg_cachepath): hardlink_or_copy(pkg_fname, pkg_cachepath) def clear(self): ''' Remove all cache contents. ''' Path(self._cache_dir).mkdir(parents=True, exist_ok=True) cache_size = len(glob(os.path.join(self._cache_dir, '*.deb'))) old_cache_dir = self._cache_dir.rstrip(os.sep) + '.old' os.rename(self._cache_dir, old_cache_dir) Path(self._cache_dir).mkdir(parents=True, exist_ok=True) shutil.rmtree(old_cache_dir) return cache_size def delete(self): ''' Remove cache completely - only useful when removing a base image completely. ''' shutil.rmtree(self._cache_dir) debspawn-0.4.0/debspawn/build.py000066400000000000000000000510071361140626300165770ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import subprocess import shutil import platform from glob import glob from .utils.env import ensure_root, switch_unprivileged, get_owner_uid_gid, get_free_space, get_tree_size from .utils.misc import temp_dir, cd, format_filesize, version_noepoch from .utils.log import print_header, print_section, print_info, print_warn, print_error, \ capture_console_output, save_captured_console_output from .utils.command import safe_run from .nspawn import nspawn_run_helper_persist, nspawn_run_persist from .injectpkg import PackageInjector def interact_with_build_environment(osbase, instance_dir, machine_name, *, pkg_dir_root, source_pkg_dir, aptcache_tmp, pkginjector, prev_exitcode): ''' Launch an interactive shell in the build environment ''' # find the right directory to switch to pkg_dir = pkg_dir_root for f in glob(os.path.join(pkg_dir, '*')): if os.path.isdir(f): pkg_dir = f break print() print_info('Launching interactive shell in build environment.') if prev_exitcode != 0: print_info('The previous build step failed with exit code {}'.format(prev_exitcode)) else: print_info('The previous build step was successful.') print_info('Temporary location of package files on the host:\n => file://{}'.format(pkg_dir)) print_info('Press CTL+D to exit the interactive shell.') print() nspawn_flags = ['--bind={}:/srv/build/'.format(pkg_dir_root)] nspawn_run_persist(osbase, instance_dir, machine_name, chdir=os.path.join('/srv/build', os.path.basename(pkg_dir)), flags=nspawn_flags, tmp_apt_cache_dir=aptcache_tmp, pkginjector=pkginjector, verbose=True) if source_pkg_dir: print() while True: copy_changes = input(('Should changes to the debian/ directory be copied back to the host?\n' 'This will OVERRIDE all changes made on files on the host. [y/N]: ')) if copy_changes == 'y' or copy_changes == 'Y': copy_changes = True break elif copy_changes == 'n' or copy_changes == 'N': copy_changes = False break elif not copy_changes: copy_changes = False break if copy_changes: print_info('Cleaning up...') # clean the source tree. we intentionally ignore errors here. nspawn_run_persist(osbase, instance_dir, machine_name, chdir=os.path.join('/srv/build', os.path.basename(pkg_dir)), flags=nspawn_flags, command=['dpkg-buildpackage', '-T', 'clean'], tmp_apt_cache_dir=aptcache_tmp, pkginjector=pkginjector) print() print_info('Copying back changes...') known_files = {} dest_debian_dir = os.path.join(source_pkg_dir, 'debian') src_debian_dir = os.path.join(pkg_dir, 'debian') # get uid/gid of the user who invoked us o_uid, o_gid = get_owner_uid_gid() # collect list of existing packages for sdir, _, files in os.walk(dest_debian_dir): for f in files: fname = os.path.join(sdir, f) known_files[os.path.relpath(fname, dest_debian_dir)] = fname # walk through the source files, copying everything to the destination for sdir, _, files in os.walk(src_debian_dir): for f in files: fname = os.path.join(sdir, f) rel_fname = os.path.relpath(fname, src_debian_dir) dest_fname = os.path.normpath(os.path.join(dest_debian_dir, rel_fname)) dest_dir = os.path.dirname(dest_fname) if rel_fname in known_files: del known_files[rel_fname] if os.path.isdir(fname): print('New dir: {}'.format(rel_fname)) with switch_unprivileged(): os.makedirs(dest_fname, exist_ok=True) continue if not os.path.isdir(dest_dir): print('New dir: {}'.format(os.path.relpath(dest_dir, dest_debian_dir))) with switch_unprivileged(): os.makedirs(dest_dir, exist_ok=True) print('Copy: {}'.format(rel_fname)) shutil.copy2(fname, dest_fname, follow_symlinks=False) os.chown(dest_fname, o_uid, o_gid, follow_symlinks=False) for rel_fname, fname in known_files.items(): print('Delete: {}'.format(rel_fname)) os.remove(fname) print() else: print_info('Discarding build environment.') else: print_info('Can not copy back changes as original package directory is unknown.') def internal_execute_build(osbase, pkg_dir, build_only=None, *, qa_lintian=False, interact=False, source_pkg_dir=None, buildflags=[]): ''' Perform the actual build on an extracted package directory ''' assert not build_only or isinstance(build_only, str) if not pkg_dir: raise Exception('Package directory is missing!') pkg_dir = os.path.normpath(pkg_dir) with osbase.new_instance() as (instance_dir, machine_name): # first, check basic requirements # instance dir and pkg dir are both temporary directories, so they # will be on the same filesystem configured as workspace for debspawn. # therefore we only check on directory. free_space = get_free_space(instance_dir) print_info('Free space in workspace: {}'.format(format_filesize(free_space))) # check for at least 512MiB - this is a ridiculously small amount, so the build will likely fail. # but with even less, even attempting a build is pointless. if (free_space / 2048) < 512: print_error('Not enough free space available in workspace.') return 8 # prepare the build. At this point, we only run trusted code and the container # has network access with temp_dir('pkgsync-' + machine_name) as pkgsync_tmp: # create temporary locations set up and APT cache sharing and package injection aptcache_tmp = os.path.join(pkgsync_tmp, 'aptcache') pkginjector = PackageInjector(osbase) if pkginjector.has_injectables(): pkginjector.create_instance_repo(os.path.join(pkgsync_tmp, 'pkginject')) # set up the build environment nspawn_flags = ['--bind={}:/srv/build/'.format(pkg_dir)] prep_flags = ['--build-prepare'] if build_only == 'arch': prep_flags.append('--arch-only') if qa_lintian: # install Lintian from the start, if we run it later prep_flags.append('--lintian') r = nspawn_run_helper_persist(osbase, instance_dir, machine_name, prep_flags, '/srv', nspawn_flags=nspawn_flags, tmp_apt_cache_dir=aptcache_tmp, pkginjector=pkginjector) if r != 0: print_error('Build environment setup failed.') return False # run the actual build. At this point, code is less trusted, and we disable network access. nspawn_flags = ['--bind={}:/srv/build/'.format(pkg_dir), '-u', 'builder', '--private-network'] helper_flags = ['--build-run'] if buildflags: helper_flags.append('--buildflags={}'.format(';'.join(buildflags))) r = nspawn_run_helper_persist(osbase, instance_dir, machine_name, helper_flags, '/srv', nspawn_flags=nspawn_flags, tmp_apt_cache_dir=aptcache_tmp, pkginjector=pkginjector) # exit, unless we are in interactive mode if r != 0 and not interact: return False if qa_lintian and r == 0: # running Lintian was requested, so do so. # we use Lintian from the container, so we validate with the validator from # the OS the package was actually built against nspawn_flags = ['--bind={}:/srv/build/'.format(pkg_dir)] r = nspawn_run_helper_persist(osbase, instance_dir, machine_name, ['--run-qa', '--lintian'], '/srv', nspawn_flags=nspawn_flags, tmp_apt_cache_dir=aptcache_tmp, pkginjector=pkginjector) if r != 0: print_error('QA failed.') return False print() # extra blank line after Lintian output if interact: interact_with_build_environment(osbase, instance_dir, machine_name, pkg_dir_root=pkg_dir, source_pkg_dir=source_pkg_dir, aptcache_tmp=aptcache_tmp, pkginjector=pkginjector, prev_exitcode=r) # exit with status of previous exist code if r != 0: return False build_dir_size = get_tree_size(pkg_dir) print_info('This build required {} of dedicated disk space.'.format(format_filesize(build_dir_size))) return True def print_build_detail(osbase, pkgname, version): print_info('Package: {}'.format(pkgname)) print_info('Version: {}'.format(version)) print_info('Distribution: {}'.format(osbase.suite)) print_info('Architecture: {}'.format(osbase.arch)) print_info() def _read_source_package_details(): out, err, ret = safe_run(['dpkg-parsechangelog']) if ret != 0: raise Exception('Running dpkg-parsechangelog failed: {}{}'.format(out, err)) pkg_sourcename = None pkg_version = None for line in out.split('\n'): if line.startswith('Source: '): pkg_sourcename = line[8:].strip() elif line.startswith('Version: '): pkg_version = line[9:].strip() if not pkg_sourcename or not pkg_version: print_error('Unable to determine source package name or source package version. Can not continue.') return None, None, None pkg_version_dsc = version_noepoch(pkg_version) dsc_fname = '{}_{}.dsc'.format(pkg_sourcename, pkg_version_dsc) return pkg_sourcename, pkg_version, dsc_fname def _get_build_flags(build_only=None, include_orig=False, maintainer=None, extra_flags=[]): import shlex buildflags = [] if build_only: if build_only == 'binary': buildflags.append('-b') elif build_only == 'arch': buildflags.append('-B') elif build_only == 'indep': buildflags.append('-A') elif build_only == 'source': buildflags.append('-S') else: print_error('Invalid build-only flag "{}". Can not continue.'.format(build_only)) return False, [] if include_orig: buildflags.append('-sa') if maintainer: buildflags.append('-m{}'.format(maintainer.replace(';', ','))) buildflags.append('-e{}'.format(maintainer.replace(';', ','))) for flag_raw in extra_flags: buildflags.extend(shlex.split(flag_raw)) return True, buildflags def _retrieve_artifacts(osbase, tmp_dir): print_section('Retrieving build artifacts') o_uid, o_gid = get_owner_uid_gid() acount = 0 for f in glob(os.path.join(tmp_dir, '*.*')): if os.path.isfile(f): target_fname = os.path.join(osbase.results_dir, os.path.basename(f)) shutil.copy2(f, target_fname) os.chown(target_fname, o_uid, o_gid, follow_symlinks=False) acount += 1 print_info('Copied {} files.'.format(acount)) def _sign_result(results_dir, spkg_name, spkg_version, build_arch): print_section('Signing Package') spkg_version_noepoch = version_noepoch(spkg_version) changes_basename = '{}_{}_{}.changes'.format(spkg_name, spkg_version_noepoch, build_arch) with switch_unprivileged(): proc = subprocess.run(['debsign', os.path.join(results_dir, changes_basename)]) if proc.returncode != 0: print_error('Signing failed.') return False return True def _print_system_info(): from . import __version__ from .utils.misc import current_time_string print_info('debspawn {version} on {host} at {time}'.format(version=__version__, host=platform.node(), time=current_time_string())) def build_from_directory(osbase, pkg_dir, *, sign=False, build_only=None, include_orig=False, maintainer=None, clean_source=False, qa_lintian=False, interact=False, log_build=True, extra_dpkg_flags=[]): ensure_root() osbase.ensure_exists() if interact and log_build: print_warn('Build log and interactive mode can not be enabled at the same time. Disabling build log.') print() log_build = False # capture console output if we should log the build if log_build: capture_console_output() if not pkg_dir: pkg_dir = os.getcwd() pkg_dir = os.path.abspath(pkg_dir) r, buildflags = _get_build_flags(build_only, include_orig, maintainer, extra_dpkg_flags) if not r: return False _print_system_info() print_header('Package build (from directory)') print_section('Creating source package') with cd(pkg_dir): with switch_unprivileged(): deb_files_fname = os.path.join(pkg_dir, 'debian', 'files') if os.path.isfile(deb_files_fname): deb_files_fname = None # the file already existed, we don't need to clean it up later pkg_sourcename, pkg_version, dsc_fname = _read_source_package_details() if not pkg_sourcename: return False cmd = ['dpkg-buildpackage', '-S', '--no-sign'] # d/rules clean requires build dependencies installed if run on the host # we avoid that by default, unless explicitly requested if not clean_source: cmd.append('-nc') proc = subprocess.run(cmd) if proc.returncode != 0: return False # remove d/files file that was created when generating the source package. # we only clean up the file if it didn't exist prior to us running the command. if deb_files_fname: try: os.remove(deb_files_fname) except OSError: pass print_header('Package build') print_build_detail(osbase, pkg_sourcename, pkg_version) with temp_dir(pkg_sourcename) as pkg_tmp_dir: with cd(pkg_tmp_dir): cmd = ['dpkg-source', '-x', os.path.join(pkg_dir, '..', dsc_fname)] proc = subprocess.run(cmd) if proc.returncode != 0: return False ret = internal_execute_build(osbase, pkg_tmp_dir, build_only, qa_lintian=qa_lintian, interact=interact, source_pkg_dir=pkg_dir, buildflags=buildflags) if not ret: return False # copy build results _retrieve_artifacts(osbase, pkg_tmp_dir) # save buildlog, if we generated one log_fname = os.path.join(osbase.results_dir, '{}_{}_{}.buildlog'.format(pkg_sourcename, version_noepoch(pkg_version), osbase.arch)) save_captured_console_output(log_fname) # sign the resulting package if sign: r = _sign_result(osbase.results_dir, pkg_sourcename, pkg_version, osbase.arch) if not r: return False print_info('Done.') return True def build_from_dsc(osbase, dsc_fname, *, sign=False, build_only=None, include_orig=False, maintainer=None, qa_lintian=False, interact=False, log_build=True, extra_dpkg_flags=[]): ensure_root() osbase.ensure_exists() if interact and log_build: print_warn('Build log and interactive mode can not be enabled at the same time. Disabling build log.') print() log_build = False # capture console output if we should log the build if log_build: capture_console_output() r, buildflags = _get_build_flags(build_only, include_orig, maintainer, extra_dpkg_flags) if not r: return False _print_system_info() dsc_fname = os.path.abspath(os.path.normpath(dsc_fname)) tmp_prefix = os.path.basename(dsc_fname).replace('.dsc', '').replace(' ', '-') with temp_dir(tmp_prefix) as pkg_tmp_dir: with cd(pkg_tmp_dir): cmd = ['dpkg-source', '-x', dsc_fname] proc = subprocess.run(cmd) if proc.returncode != 0: return False pkg_srcdir = None for f in glob('./*'): if os.path.isdir(f): pkg_srcdir = f break if not pkg_srcdir: print_error('Unable to find source directory of extracted package.') return False with cd(pkg_srcdir): pkg_sourcename, pkg_version, dsc_fname = _read_source_package_details() if not pkg_sourcename: return False print_header('Package build') print_build_detail(osbase, pkg_sourcename, pkg_version) ret = internal_execute_build(osbase, pkg_tmp_dir, build_only, qa_lintian=qa_lintian, interact=interact, buildflags=buildflags) if not ret: return False # copy build results _retrieve_artifacts(osbase, pkg_tmp_dir) # save buildlog, if we generated one log_fname = os.path.join(osbase.results_dir, '{}_{}_{}.buildlog'.format(pkg_sourcename, version_noepoch(pkg_version), osbase.arch)) save_captured_console_output(log_fname) # sign the resulting package if sign: r = _sign_result(osbase.results_dir, pkg_sourcename, pkg_version, osbase.arch) if not r: return False print_info('Done.') return True debspawn-0.4.0/debspawn/cli.py000066400000000000000000000404051361140626300162470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys import logging as log from argparse import ArgumentParser, HelpFormatter from .config import GlobalConfig from .utils.env import set_unicode_allowed, set_owning_user from .osbase import OSBase def init_config(options): ''' Create a new :GlobalConfig from command-line options. ''' gconf = GlobalConfig() gconf.load(options.config) # check if we are forbidden from using unicode - otherwise we build # with unicode enabled by default if options.no_unicode: set_unicode_allowed(False) else: if 'utf-8' not in os.environ.get('LANG', 'utf-8').lower(): log.warning('Building with unicode support, but your environment does not seem to support unicode.') set_unicode_allowed(True) if options.owner: info = options.owner.split(':') if len(info) > 2: print('You can only use one colon to split user:group when using the --owner flag.') sys.exit(1) if len(info) == 2: user = info[0] group = info[1] else: user = info[0] group = None set_owning_user(user, group) return gconf def check_print_version(options): if options.show_version: from . import __version__ print(__version__) sys.exit(0) def command_create(options): ''' Create new container image ''' check_print_version(options) if not options.suite: print('Need at least a suite name to bootstrap!') sys.exit(1) gconf = init_config(options) components = None if options.components: components = options.components.split(',') extra_suites = [] if options.extra_suites: extra_suites = options.extra_suites.strip().split(' ') osbase = OSBase(gconf, options.suite, options.arch, variant=options.variant, base_suite=options.base_suite) r = osbase.create(options.mirror, components, extra_suites, options.extra_source_lines) if not r: sys.exit(2) def command_delete(options): ''' Delete container image ''' check_print_version(options) if not options.suite: print('No suite name was specified!') sys.exit(1) gconf = init_config(options) osbase = OSBase(gconf, options.suite, options.arch, options.variant) r = osbase.delete() if not r: sys.exit(2) def command_update(options): ''' Update container image ''' check_print_version(options) if not options.suite: print('Need at least a suite name for update!') sys.exit(1) gconf = init_config(options) osbase = OSBase(gconf, options.suite, options.arch, options.variant) if options.recreate: r = osbase.recreate() else: r = osbase.update() if not r: sys.exit(2) def command_list(options): ''' List container images ''' from .osbase import print_container_base_image_info check_print_version(options) gconf = init_config(options) print_container_base_image_info(gconf) def command_build(options): ''' Build a package in a new volatile container ''' from .build import build_from_directory, build_from_dsc check_print_version(options) if not options.suite: print('Need at least a suite name for building!') sys.exit(1) gconf = init_config(options) osbase = OSBase(gconf, options.suite, options.arch, options.variant) buildflags = [] if options.buildflags: buildflags = options.buildflags.split(';') if not options.target and os.path.isdir(options.suite): print('A directory is given as parameter, but you are missing a suite parameter to build for.') print('Can not continue.') sys.exit(1) # override globally configured output directory with # a custom one defined on the CLI if options.results_dir: osbase.results_dir = options.results_dir if not options.target or os.path.isdir(options.target): r = build_from_directory(osbase, options.target, sign=options.sign, build_only=options.build_only, include_orig=options.include_orig, maintainer=options.maintainer, clean_source=options.clean_source, qa_lintian=options.lintian, interact=options.interact, log_build=not options.no_buildlog, extra_dpkg_flags=buildflags) else: r = build_from_dsc(osbase, options.target, sign=options.sign, build_only=options.build_only, include_orig=options.include_orig, maintainer=options.maintainer, qa_lintian=options.lintian, interact=options.interact, log_build=not options.no_buildlog, extra_dpkg_flags=buildflags) if not r: sys.exit(2) def command_login(options): ''' Open interactive session in a container ''' check_print_version(options) if not options.suite: print('Need at least a suite name!') sys.exit(1) gconf = init_config(options) osbase = OSBase(gconf, options.suite, options.arch, options.variant) allowed = [] if options.allow: allowed = [s.strip() for s in options.allow.split(',')] r = osbase.login(options.persistent, allowed) if not r: sys.exit(2) def command_run(options, custom_command): ''' Run arbitrary command in container session ''' check_print_version(options) if not options.suite: print('Need at least a suite name!') sys.exit(1) gconf = init_config(options) osbase = OSBase(gconf, options.suite, options.arch, options.variant) allowed = [] if options.allow: allowed = [s.strip() for s in options.allow.split(',')] r = osbase.run(custom_command, options.build_dir, options.artifacts_dir, options.external_commad, options.header, allowed=allowed) if not r: sys.exit(2) class CustomArgparseFormatter(HelpFormatter): def _split_lines(self, text, width): if text.startswith('CF|'): return text[3:].splitlines() return HelpFormatter._split_lines(self, text, width) def add_container_select_arguments(parser): parser.add_argument('--variant', action='store', dest='variant', default=None, help='Set the bootstrap script variant.') parser.add_argument('-a', '--arch', action='store', dest='arch', default=None, help='The architecture of the container.') parser.add_argument('suite', action='store', nargs='?', default=None, help='The suite name of the container.') def create_parser(formatter_class=None): ''' Create debspawn CLI argument parser ''' if not formatter_class: formatter_class = CustomArgparseFormatter parser = ArgumentParser(description='Build in nspawn containers', formatter_class=formatter_class) subparsers = parser.add_subparsers(dest='sp_name', title='subcommands') # generic arguments parser.add_argument('-c', '--config', action='store', dest='config', default=None, help='Path to the global config file.') parser.add_argument('--verbose', action='store_true', dest='verbose', help='Enable debug messages.') parser.add_argument('--no-unicode', action='store_true', dest='no_unicode', help='Disable unicode support.') parser.add_argument('--version', action='store_true', dest='show_version', help='Display the version of debspawn itself.') parser.add_argument('--owner', action='store', dest='owner', default=None, help=('Set the user name/uid and group/gid separated by a colon ' 'whose behalf we are acting.')) # 'create' command sp = subparsers.add_parser('create', help='Create new container image') add_container_select_arguments(sp) sp.add_argument('--mirror', action='store', dest='mirror', default=None, help='Set a specific mirror to bootstrap from.') sp.add_argument('--components', action='store', dest='components', default=None, help='A comma-separated list of archive components to enable in the newly created image.') sp.add_argument('--base-suite', action='store', dest='base_suite', default=None, help='A full suite that forms the base of the selected partial suite (e.g. for -updates and -backports).') sp.add_argument('--extra-suites', action='store', dest='extra_suites', default=None, help='Space-separated list of additional suites that should also be added to the sources.list file.') sp.add_argument('--extra-sourceslist-lines', action='store', dest='extra_source_lines', default=None, help='Lines that should be added to the build environments source.list verbatim. Separate lines by linebreaks.') sp.set_defaults(func=command_create) # 'delete' command sp = subparsers.add_parser('delete', help='Remove a container image') add_container_select_arguments(sp) sp.set_defaults(func=command_delete) # 'update' command sp = subparsers.add_parser('update', help='Update a container image') add_container_select_arguments(sp) sp.add_argument('--recreate', action='store_true', dest='recreate', help='Re-create the container image from scratch using the settings used to create it previously, instead of just updating it.') sp.set_defaults(func=command_update) # 'list' command sp = subparsers.add_parser('list', help='List available container images', aliases=['ls']) sp.set_defaults(func=command_list) # 'build' command sp = subparsers.add_parser('build', help='Build a package in an isolated environment', formatter_class=formatter_class, aliases=['b']) add_container_select_arguments(sp) sp.add_argument('-s', '--sign', action='store_true', dest='sign', help='Sign the resulting package.') sp.add_argument('--only', choices=['binary', 'arch', 'indep', 'source'], dest='build_only', help=('CF|Select only a specific set of packages to be built. Choices are:\n' 'binary: Build only binary packages, no source files are to be built and/or distributed.\n' 'arch: Build only architecture-specific binary packages.\n' 'indep: Build only architecture-independent (arch:all) binary packages.\n' 'source: Do a source-only build, no binary packages are made.')) sp.add_argument('--include-orig', action='store_true', dest='include_orig', help='Forces the inclusion of the original source.') sp.add_argument('--buildflags', action='store', dest='buildflags', help='Set flags passed through to dpkg-buildpackage as semicolon-separated list.') sp.add_argument('--results-dir', action='store', dest='results_dir', help='Override the configured results directory and return artifacts at a custom location.') sp.add_argument('--maintainer', action='store', dest='maintainer', help=('Set the name and email address of the maintainer for this package and upload, rather than using ' 'the information from the source tree\'s control file or changelog.')) sp.add_argument('--clean-source', action='store_true', dest='clean_source', help=('Run the d/rules clean target outside of the container. This means the package build dependencies need to be ' 'installed on the host system when building from a source directory.')) sp.add_argument('--lintian', action='store_true', dest='lintian', help='Run the Lintian static analysis tool for Debian packages after the package is built.') sp.add_argument('--no-buildlog', action='store_true', dest='no_buildlog', help='Do not write a build log.') sp.add_argument('-i', '--interact', action='store_true', dest='interact', help='Run an interactive shell in the build environment after build. This implies `--no-buildlog` and disables the log.') sp.add_argument('target', action='store', nargs='?', default=None, help='The source package file or source directory to build.') sp.set_defaults(func=command_build) # 'login' command sp = subparsers.add_parser('login', help='Open interactive session in a container') add_container_select_arguments(sp) sp.add_argument('--persistent', action='store_true', dest='persistent', help='Make changes done in the session persistent.') sp.add_argument('--allow', action='store', dest='allow', help='List one or more additional permissions to grant the container. Takes a comma-separated list of capability names.') sp.set_defaults(func=command_login) # 'run' command sp = subparsers.add_parser('run', help='Run arbitrary command in an ephemeral container') add_container_select_arguments(sp) sp.add_argument('--artifacts-out', action='store', dest='artifacts_dir', default=None, help='Directory on the host where artifacts can be stored. Mounted to /srv/artifacts in the guest.') sp.add_argument('--build-dir', action='store', dest='build_dir', default=None, help='Select a host directory that gets bind mounted to /srv/build.') sp.add_argument('-x', '--external-command', action='store_true', dest='external_commad', help='If set, the command script will be copied from the host to the container and then executed.') sp.add_argument('--header', action='store', dest='header', default=None, help='Name of the task that is run, will be printed as header.') sp.add_argument('--allow', action='store', dest='allow', help='List one or more additional permissions to grant the container. Takes a comma-separated list of capability names.') sp.add_argument('command', action='store', nargs='*', default=None, help='The command to run.') return parser def run(mainfile, args): if len(args) == 0: print('Need a subcommand to proceed!') sys.exit(1) parser = create_parser() # special case, so 'run' can understand which arguments are for debspawn and which are # for the command to be executed custom_command = None if args[0] == 'run': for i, arg in enumerate(args): if arg == '---': if i + 1 == len(args): print('No command was given after "---", can not continue.') sys.exit(1) custom_command = args[i + 1:] args = args[:i] break args = parser.parse_args(args) check_print_version(args) if args.sp_name == 'run': if not custom_command: custom_command = args.command command_run(args, custom_command) else: args.func(args) debspawn-0.4.0/debspawn/config.py000066400000000000000000000067341361140626300167540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys import json thisfile = __file__ if not os.path.isabs(thisfile): thisfile = os.path.normpath(os.path.join(os.getcwd(), thisfile)) __all__ = ['GlobalConfig'] class GlobalConfig: ''' Global configuration singleton affecting all of Debspawn. ''' _instance = None class __GlobalConfig: def load(self, fname=None): if not fname: fname = '/etc/debspawn/global.json' jdata = {} if os.path.isfile(fname): with open(fname) as json_file: try: jdata = json.load(json_file) except json.JSONDecodeError as e: print('Unable to parse global configuration (global.json): {}'.format(str(e)), file=sys.stderr) sys.exit(8) self._dsrun_path = os.path.normpath(os.path.join(thisfile, '..', 'dsrun')) if not os.path.isfile(self._dsrun_path): print('Debspawn is not set up properly: Unable to find file "{}". Can not continue.'.format(self._dsrun_path), file=sys.stderr) sys.exit(4) self._osroots_dir = jdata.get('OSRootsDir', '/var/lib/debspawn/containers/') self._results_dir = jdata.get('ResultsDir', '/var/lib/debspawn/results/') self._aptcache_dir = jdata.get('APTCacheDir', '/var/lib/debspawn/aptcache/') self._injected_pkgs_dir = jdata.get('InjectedPkgsDir', '/var/lib/debspawn/injected-pkgs/') self._temp_dir = jdata.get('TempDir', '/var/tmp/debspawn/') self._allow_unsafe_perms = jdata.get('AllowUnsafePermissions', False) @property def dsrun_path(self) -> str: return self._dsrun_path @dsrun_path.setter def dsrun_path(self, v) -> str: self._dsrun_path = v @property def osroots_dir(self) -> str: return self._osroots_dir @property def results_dir(self) -> str: return self._results_dir @property def aptcache_dir(self) -> str: return self._aptcache_dir @property def injected_pkgs_dir(self) -> str: return self._injected_pkgs_dir @property def temp_dir(self) -> str: return self._temp_dir @property def allow_unsafe_perms(self) -> bool: return self._allow_unsafe_perms def __init__(self, fname=None): if not GlobalConfig._instance: GlobalConfig._instance = GlobalConfig.__GlobalConfig() GlobalConfig._instance.load(fname) def __getattr__(self, name): return getattr(self._instance, name) debspawn-0.4.0/debspawn/dsrun000077500000000000000000000252711361140626300162130ustar00rootroot00000000000000#!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # SPDX-License-Identifier: LGPL-3.0-or-later # IMPORTANT: This file is placed within a Debspawn container. # The containers only contain a minimal set of packages, and only a reduced # installation of Python is available via the python3-minimal package. # This file must be self-contained and only depend on modules available # in that Python installation. # It must also not depend on any Python 3 feature introduced after version 3.5. # See /usr/share/doc/python3.*-minimal/README.Debian for a list of permitted # modules. # Additionally, the CLI API of this file should remain as stable as possible, # to not introduce odd behavior if a container wasn't updated and is used with # a newer debspawn version. import os import sys import pwd import subprocess from contextlib import contextmanager from argparse import ArgumentParser from glob import glob # the user performing builds in the container BUILD_USER = 'builder' # the directory where we build a package BUILD_DIR = '/srv/build' # additional packages to be used when building EXTRAPKG_DIR = '/srv/extra-packages' # # Globals # unicode_enabled = True color_enabled = True def run_command(cmd, env=None): if isinstance(cmd, str): cmd = cmd.split(' ') proc_env = env if proc_env: proc_env = os.environ.copy() proc_env.update(env) p = subprocess.run(cmd, env=proc_env) if p.returncode != 0: print('Command `{}` failed.'.format(' '.join(cmd))) sys.exit(p.returncode) def run_apt_command(cmd): if isinstance(cmd, str): cmd = cmd.split(' ') env = {'DEBIAN_FRONTEND': 'noninteractive'} apt_cmd = ['apt-get', '-uyq', '-o Dpkg::Options::="--force-confnew"'] apt_cmd.extend(cmd) run_command(apt_cmd, env) def print_textbox(title, tl, hline, tr, vline, bl, br): def write_utf8(s): sys.stdout.buffer.write(s.encode('utf-8')) tlen = len(title) write_utf8('\n{}'.format(tl)) write_utf8(hline * (10 + tlen)) write_utf8('{}\n'.format(tr)) write_utf8('{} {}'.format(vline, title)) write_utf8(' ' * 8) write_utf8('{}\n'.format(vline)) write_utf8(bl) write_utf8(hline * (10 + tlen)) write_utf8('{}\n'.format(br)) sys.stdout.flush() def print_header(title): global unicode_enabled if unicode_enabled: print_textbox(title, '╔', '═', '╗', '║', '╚', '╝') else: print_textbox(title, '+', '═', '+', '|', '+', '+') def print_section(title): global unicode_enabled if unicode_enabled: print_textbox(title, '┌', '─', '┐', '│', '└', '┘') else: print_textbox(title, '+', '-', '+', '|', '+', '+') @contextmanager def eatmydata(): try: # FIXME: We just override the env vars here, appending to them would # be much cleaner. os.environ['LD_LIBRARY_PATH'] = '/usr/lib/libeatmydata' os.environ['LD_PRELOAD'] = 'libeatmydata.so' yield finally: del os.environ['LD_LIBRARY_PATH'] del os.environ['LD_PRELOAD'] def update_container(): with eatmydata(): run_apt_command('update') run_apt_command('full-upgrade') run_apt_command(['install', '--no-install-recommends', 'build-essential', 'dpkg-dev', 'fakeroot', 'eatmydata']) run_apt_command(['--purge', 'autoremove']) run_apt_command('clean') try: pwd.getpwnam(BUILD_USER) except KeyError: print('No "{}" user, creating it.'.format(BUILD_USER)) run_command('adduser --system --no-create-home --disabled-password {}'.format(BUILD_USER)) run_command('mkdir -p /srv/build') run_command('chown {} /srv/build'.format(BUILD_USER)) return True def prepare_run(): print_section('Preparing container') with eatmydata(): run_apt_command('update') run_apt_command('full-upgrade') return True def prepare_package_build(arch_only=False, qa_lintian=False): print_section('Preparing container for build') with eatmydata(): run_apt_command('update') run_apt_command('full-upgrade') run_apt_command(['install', '--no-install-recommends', 'build-essential', 'dpkg-dev', 'fakeroot']) # if we want to run Lintian later, we need to make sure it is installed if qa_lintian: print('Lintian check requested, installing Lintian.') run_apt_command(['install', 'lintian']) # check if we have extra packages to register with APT if os.path.exists(EXTRAPKG_DIR) and os.path.isdir(EXTRAPKG_DIR): if os.listdir(EXTRAPKG_DIR): run_apt_command(['install', '--no-install-recommends', 'apt-utils']) print() print('Using injected packages as additional APT package source.') os.chdir(EXTRAPKG_DIR) with open(os.path.join(EXTRAPKG_DIR, 'Packages'), 'wt') as f: proc = subprocess.Popen(['apt-ftparchive', 'packages', '.'], cwd=EXTRAPKG_DIR, stdout=f) ret = proc.wait() if ret != 0: print('ERROR: Unable to generate temporary APT repository for injected packages.') sys.exit(2) with open('/etc/apt/sources.list', 'a') as f: f.write('deb [trusted=yes] file://{} ./\n'.format(EXTRAPKG_DIR)) # make APT aware of the new packages, update base packages if needed run_apt_command('update') run_apt_command('full-upgrade') # ensure we are in our build directory at this point os.chdir(BUILD_DIR) run_command('chown -R {} /srv/build'.format(BUILD_USER)) for f in glob('./*'): if os.path.isdir(f): os.chdir(f) break print_section('Installing package build-dependencies') with eatmydata(): cmd = ['build-dep'] if arch_only: cmd.append('--arch-only') cmd.append('./') run_apt_command(cmd) return True def build_package(buildflags=None): print_section('Build') os.chdir(BUILD_DIR) for f in glob('./*'): if os.path.isdir(f): os.chdir(f) break cmd = ['dpkg-buildpackage'] if buildflags: cmd.extend(buildflags) run_command(cmd) # run_command will exit the whole program if the command failed, # so we can return True here (everything went fine if we are here) return True def run_qatasks(qa_lintian=True): ''' Run QA tasks on a built package immediately after build (currently Lintian) ''' os.chdir(BUILD_DIR) for f in glob('./*'): if os.path.isdir(f): os.chdir(f) break if qa_lintian: print_section('QA: Lintian') # ensure Lintian is really installed run_apt_command(['install', 'lintian']) # drop privileges pw = pwd.getpwnam(BUILD_USER) os.seteuid(pw.pw_uid) cmd = ['lintian', '-I', # infos by default '--pedantic', # pedantic hints by default, '--no-tag-display-limit' # display all tags found (even if that may be a lot occasionally) ] run_command(cmd) # run_command will exit the whole program if the command failed, # so we can return True here (everything went fine if we are here) return True def setup_environment(use_color=True, use_unicode=True): os.environ['LANG'] = 'C.UTF-8' if use_unicode else 'C' os.environ['HOME'] = '/nonexistent' os.environ['TERM'] = 'xterm-256color' if use_color else 'xterm-mono' os.environ['SHELL'] = '/bin/sh' del os.environ['LOGNAME'] def main(): if not os.environ.get('container'): print('This helper script must be run in a systemd-nspawn container.') return 1 parser = ArgumentParser(description='Debspawn helper script') parser.add_argument('--update', action='store_true', dest='update', help='Initialize the container.') parser.add_argument('--no-color', action='store_true', dest='no_color', help='Disable terminal colors.') parser.add_argument('--no-unicode', action='store_true', dest='no_unicode', help='Disable unicode support.') parser.add_argument('--arch-only', action='store_true', dest='arch_only', default=None, help='Only get arch-dependent packages (used when satisfying build dependencies).') parser.add_argument('--build-prepare', action='store_true', dest='build_prepare', help='Prepare building a Debian package.') parser.add_argument('--build-run', action='store_true', dest='build_run', help='Build a Debian package.') parser.add_argument('--lintian', action='store_true', dest='qa_lintian', help='Run Lintian on the generated package.') parser.add_argument('--buildflags', action='store', dest='buildflags', default=None, help='Flags passed to dpkg-buildpackage.') parser.add_argument('--prepare-run', action='store_true', dest='prepare_run', help='Prepare container image for generic script run.') parser.add_argument('--run-qa', action='store_true', dest='run_qatasks', help='Run QA tasks (only Lintian currently) against a package.') options = parser.parse_args(sys.argv[1:]) # initialize environment defaults global unicode_enabled, color_enabled unicode_enabled = not options.no_unicode color_enabled = not options.no_color setup_environment(color_enabled, unicode_enabled) if options.update: r = update_container() if not r: return 2 elif options.build_prepare: r = prepare_package_build(options.arch_only, options.qa_lintian) if not r: return 2 elif options.build_run: buildflags = [] if options.buildflags: buildflags = [s.strip('\'" ') for s in options.buildflags.split(';')] r = build_package(buildflags) if not r: return 2 elif options.prepare_run: r = prepare_run() if not r: return 2 elif options.run_qatasks: r = run_qatasks(qa_lintian=options.qa_lintian) if not r: return 2 else: print('ERROR: No action specified.') return 1 return 0 if __name__ == '__main__': sys.exit(main()) debspawn-0.4.0/debspawn/injectpkg.py000066400000000000000000000100411361140626300174470ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2019-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os from pathlib import Path from glob import glob from contextlib import contextmanager from .utils import hardlink_or_copy, temp_dir, print_info class PackageInjector: ''' Inject packages from external sources into the container APT environment. ''' def __init__(self, osbase): self._pkgs_basedir = osbase.global_config.injected_pkgs_dir self._pkgs_specific_dir = os.path.join(self._pkgs_basedir, osbase.name) self._has_injectables = None self._instance_repo_dir = None def has_injectables(self): ''' Return True if we actually have any packages ready to inject ''' if type(self._has_injectables) is bool: return self._has_injectables if os.path.exists(self._pkgs_basedir) and os.path.isdir(self._pkgs_basedir): for f in os.listdir(self._pkgs_basedir): if os.path.isfile(os.path.join(self._pkgs_basedir, f)): self._has_injectables = True return True if os.path.exists(self._pkgs_specific_dir) and os.path.isdir(self._pkgs_specific_dir): for f in os.listdir(self._pkgs_specific_dir): if os.path.isfile(os.path.join(self._pkgs_specific_dir, f)): self._has_injectables = True return True self._has_injectables = False return False def create_instance_repo(self, tmp_repo_dir): ''' Create a temporary location where all injected packages for this container are copied to. ''' Path(self._pkgs_basedir).mkdir(parents=True, exist_ok=True) Path(tmp_repo_dir).mkdir(parents=True, exist_ok=True) print_info('Copying injected packages to instance location') self._instance_repo_dir = tmp_repo_dir # copy/link injected packages specific to this environment if os.path.isdir(self._pkgs_specific_dir): for pkg_fname in glob(os.path.join(self._pkgs_specific_dir, '*.deb')): pkg_path = os.path.join(tmp_repo_dir, os.path.basename(pkg_fname)) if not os.path.isfile(pkg_path): hardlink_or_copy(pkg_fname, pkg_path) # copy/link injected packages used by all environments for pkg_fname in glob(os.path.join(self._pkgs_basedir, '*.deb')): pkg_path = os.path.join(tmp_repo_dir, os.path.basename(pkg_fname)) if not os.path.isfile(pkg_path): hardlink_or_copy(pkg_fname, pkg_path) @property def instance_repo_dir(self) -> str: return self._instance_repo_dir @contextmanager def package_injector(osbase, machine_name=None): ''' Create a package injector as context manager and make it create a new temporary instance repo. ''' if not machine_name: from random import choice from string import ascii_lowercase, digits nid = ''.join(choice(ascii_lowercase + digits) for _ in range(4)) machine_name = '{}-{}'.format(osbase.name, nid) pi = PackageInjector(osbase) if not pi.has_injectables(): yield pi else: with temp_dir('pkginject-' + machine_name) as injectrepo_tmp: pi.create_instance_repo(injectrepo_tmp) yield pi debspawn-0.4.0/debspawn/nspawn.py000066400000000000000000000171001361140626300170020ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import platform from .utils import temp_dir, print_error, print_warn, print_info, safe_run, run_forwarded from .utils.env import colored_output_allowed, unicode_allowed def get_nspawn_personality(osbase): ''' Return the syszemd-nspawn container personality for the given combination of host architecture and base OS. This allows running x86 builds on amd64 machines. ''' import fnmatch if platform.machine() == 'x86_64' and fnmatch.filter([osbase.arch], 'i?86'): return 'x86' return None def _execute_sdnspawn(osbase, parameters, machine_name, allow_permissions=[]): ''' Execute systemd-nspawn with the given parameters. Mess around with cgroups if necessary. ''' import sys import time capabilities = [] full_dev_access = False for perm in allow_permissions: perm = perm.lower() if perm.startswith('cap_'): capabilities.append(perm.upper()) elif perm == 'full-dev-access': full_dev_access = True else: print_info('Unknown allowed permission: {}'.format(perm)) if (capabilities or full_dev_access) and not osbase.global_config.allow_unsafe_perms: print_error('Configuration does not permit usage of additional and potentially dangerous additional permissions. Exiting.') sys.exit(9) cmd = ['systemd-nspawn'] cmd.extend(['-M', machine_name]) if full_dev_access: cmd.extend(['--bind', '/dev']) if capabilities: cmd.extend(['--capability', ','.join(capabilities)]) cmd.extend(parameters) if not full_dev_access: proc = run_forwarded(cmd) return proc.returncode else: out, _, _ = safe_run(['systemd-escape', machine_name]) escaped_full_machine_name = out.strip() pid = os.fork() if pid == 0: # child process - edit the cgroup to allow full access to all # devices. Hopefully there won't be too much need for this awful code. parent_pid = os.getppid() print_warn('Giving container access to all host devices.') syscg_devices_allow = '/sys/fs/cgroup/devices/machine.slice/machine-{}.scope/devices.allow'.format(escaped_full_machine_name) tries = 0 while not os.path.exists(syscg_devices_allow): time.sleep(0.5) tries += 1 if tries > 40: break # check if our parent process has died - this is very prone to race conditions, but # the best simple way to perform this check. The Linux kernel has neat features to # make this easier though. if os.getppid() != parent_pid: os._exit(0) if not os.path.isfile(syscg_devices_allow): print_error('Unable to give container full read/write permissions on host /dev!') os._exit(0) with open(syscg_devices_allow, 'w') as sys_f: sys_f.write('c *:* rwm\n') # full access to character devices sys_f.flush() sys_f.write('b *:* rwm\n') # full access to block devices sys_f.flush() os._exit(0) else: proc = run_forwarded(cmd) return proc.returncode def nspawn_run_persist(osbase, base_dir, machine_name, chdir, command=[], flags=[], *, tmp_apt_cache_dir=None, pkginjector=None, allowed=[], verbose=False): if isinstance(command, str): command = command.split(' ') if isinstance(flags, str): flags = flags.split(' ') personality = get_nspawn_personality(osbase) def run_nspawn_with_aptcache(aptcache_tmp_dir): params = ['--chdir={}'.format(chdir), '--link-journal=no', '--bind={}:/var/cache/apt/archives/'.format(aptcache_tmp_dir)] if pkginjector and pkginjector.instance_repo_dir: params.append('--bind={}:/srv/extra-packages/'.format(pkginjector.instance_repo_dir)) if personality: params.append('--personality={}'.format(personality)) params.extend(flags) params.extend(['-a{}D'.format('' if verbose else 'q'), base_dir]) params.extend(command) # ensure the temporary apt cache is up-to-date osbase.aptcache.create_instance_cache(aptcache_tmp_dir) # run command in container ret = _execute_sdnspawn(osbase, params, machine_name, allowed) # archive APT cache, so future runs of this command are faster osbase.aptcache.merge_from_dir(aptcache_tmp_dir) return ret if tmp_apt_cache_dir: ret = run_nspawn_with_aptcache(tmp_apt_cache_dir) else: with temp_dir('aptcache-' + machine_name) as aptcache_tmp: ret = run_nspawn_with_aptcache(aptcache_tmp) return ret def nspawn_run_ephemeral(osbase, base_dir, machine_name, chdir, command=[], flags=[], allowed=[]): if isinstance(command, str): command = command.split(' ') if isinstance(flags, str): flags = flags.split(' ') personality = get_nspawn_personality(osbase) params = ['--chdir={}'.format(chdir), '--link-journal=no'] if personality: params.append('--personality={}'.format(personality)) params.extend(flags) params.extend(['-aqxD', base_dir]) params.extend(command) return _execute_sdnspawn(osbase, params, machine_name, allowed) def nspawn_make_helper_cmd(flags): if isinstance(flags, str): flags = flags.split(' ') cmd = ['/usr/lib/debspawn/dsrun'] if not colored_output_allowed(): cmd.append('--no-color') if not unicode_allowed(): cmd.append('--no-unicode') cmd.extend(flags) return cmd def nspawn_run_helper_ephemeral(osbase, base_dir, machine_name, helper_flags, chdir='/tmp', *, nspawn_flags=[], allowed=[]): cmd = nspawn_make_helper_cmd(helper_flags) return nspawn_run_ephemeral(base_dir, machine_name, chdir, cmd, nspawn_flags, allowed) def nspawn_run_helper_persist(osbase, base_dir, machine_name, helper_flags, chdir='/tmp', *, nspawn_flags=[], tmp_apt_cache_dir=None, pkginjector=None, allowed=[]): cmd = nspawn_make_helper_cmd(helper_flags) return nspawn_run_persist(osbase, base_dir, machine_name, chdir, cmd, nspawn_flags, tmp_apt_cache_dir=tmp_apt_cache_dir, pkginjector=pkginjector, allowed=allowed) debspawn-0.4.0/debspawn/osbase.py000066400000000000000000000542051361140626300167570ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import json import subprocess import shutil from pathlib import Path from contextlib import contextmanager from .utils import temp_dir, print_header, print_section, format_filesize, \ print_info, print_error, print_warn from .utils.env import ensure_root from .utils.command import safe_run from .utils.zstd_tar import compress_directory, decompress_tarball, ensure_tar_zstd from .nspawn import nspawn_run_helper_persist, nspawn_run_persist from .aptcache import APTCache class OSBase: ''' Describes an OS base registered with debspawn ''' def __init__(self, gconf, suite, arch, variant=None, base_suite=None): self._gconf = gconf self._suite = suite self._base_suite = base_suite self._arch = arch self._variant = variant self._name = self._make_name() self._results_dir = self._gconf.results_dir self._aptcache = APTCache(self) # ensure we can (de)compress zstd tarballs ensure_tar_zstd() def _make_name(self): if not self._arch: out, _, ret = safe_run(['dpkg-architecture', '-qDEB_HOST_ARCH']) if ret != 0: raise Exception('Running dpkg-architecture failed: {}'.format(out)) self._arch = out.strip() if self._variant: return '{}-{}-{}'.format(self._suite, self._arch, self._variant) else: return '{}-{}'.format(self._suite, self._arch) @property def name(self) -> str: return self._name @property def suite(self) -> str: return self._suite @property def base_suite(self) -> str: return self._base_suite @property def arch(self) -> str: return self._arch @property def variant(self) -> str: return self._variant @property def global_config(self): return self._gconf @property def aptcache(self): return self._aptcache @property def has_base_suite(self) -> bool: return True if self.base_suite and self.base_suite != self.suite else False @property def results_dir(self): Path(self._results_dir).mkdir(parents=True, exist_ok=True) return self._results_dir @results_dir.setter def results_dir(self, path): self._results_dir = path Path(self._results_dir).mkdir(exist_ok=True) def _copy_helper_script(self, osroot_path): script_location = os.path.join(osroot_path, 'usr', 'lib', 'debspawn') Path(script_location).mkdir(parents=True, exist_ok=True) script_fname = os.path.join(script_location, 'dsrun') if os.path.isfile(script_fname): os.remove(script_fname) shutil.copy2(self._gconf.dsrun_path, script_fname) os.chmod(script_fname, 0o0755) def get_tarball_location(self): return os.path.join(self._gconf.osroots_dir, '{}.tar.zst'.format(self.name)) def get_config_location(self): return os.path.join(self._gconf.osroots_dir, '{}.json'.format(self.name)) def exists(self): return os.path.isfile(self.get_tarball_location()) def ensure_exists(self): ''' Ensure the container image exists, and terminate the program with an error code in case it does not. ''' import sys if not self.exists(): print_error('The container image for "{}" does not exist. Please create it first.'.format(self.name)) sys.exit(3) def new_nspawn_machine_name(self): import platform from random import choice from string import ascii_lowercase, digits nid = ''.join(choice(ascii_lowercase + digits) for _ in range(4)) # on Linux, the maximum hostname length is 64, so we simple set this as general default for # debspawn here. # shorten the hostname part or replace the suffix, depending on what is longer. # This should only ever matter if the hostname of the system already is incredibly long uniq_suffix = '{}-{}'.format(self.name, nid) if len(uniq_suffix) > 48: uniq_suffix = ''.join(choice(ascii_lowercase + digits) for _ in range(12)) node_name_prefix = platform.node()[:63 - len(uniq_suffix)] return '{}-{}'.format(node_name_prefix, uniq_suffix) def _write_config_json(self, mirror, components, extra_suites, extra_source_lines): ''' Create configuration file for this container base image ''' print_info('Saving configuration settings.') data = {'Suite': self.suite, 'Architecture': self.arch} if self.variant: data['Variant'] = self.variant if mirror: data['Mirror'] = mirror if components: data['Components'] = components if extra_suites: data['ExtraSuites'] = extra_suites if extra_source_lines: data['ExtraSourceLines'] = extra_source_lines with open(self.get_config_location(), 'wt') as f: f.write(json.dumps(data, sort_keys=True, indent=4)) def _clear_image_tree(self, image_dir): ''' Clear files from a directory tree that we don't want in the tarball. ''' if os.path.ismount(image_dir): print_warn('Preparing OS tree for compression, but /dev is still mounted.') return for sdir, _, files in os.walk(os.path.join(image_dir, 'dev')): for f in files: fname = os.path.join(sdir, f) if os.path.lexists(fname) and not os.path.isdir(fname) and not os.path.ismount(fname): os.remove(fname) def _create_internal(self, mirror=None, components=None, extra_suites=[], extra_source_lines=None, show_header=True): ''' Create new container base image (internal method) ''' if self.exists(): print_error('An image already exists for this configuration. Can not create a new one.') return False # ensure image location exists Path(self._gconf.osroots_dir).mkdir(parents=True, exist_ok=True) if show_header: print_header('Creating new base: {} [{}]'.format(self.suite, self.arch)) else: print_section('Creating new base: {} [{}]'.format(self.suite, self.arch)) print('Using mirror: {}'.format(mirror if mirror else 'default')) if self.variant: print('variant: {}'.format(self.variant)) cmd = ['debootstrap', '--arch={}'.format(self.arch), '--include=python3-minimal,eatmydata'] if components: cmd.append('--components={}'.format(','.join(components))) if self.variant: cmd.append('--variant={}'.format(self.variant)) with temp_dir() as tdir: bootstrap_suite = self.suite if self.has_base_suite: bootstrap_suite = self.base_suite cmd.extend([bootstrap_suite, tdir]) print('Bootstrap suite: {}'.format(bootstrap_suite)) if extra_suites: print('Additional suites: {}'.format(', '.join(extra_suites))) if extra_source_lines: print('Custom sources.list lines will be added:') for line in extra_source_lines.split('\\n'): print(' {}'.format(line)) if mirror: cmd.append(mirror) print_section('Bootstrap') proc = subprocess.run(cmd) if proc.returncode != 0: return False # create helper script runner self._copy_helper_script(tdir) # if we bootstrapped the base suite, add the primary suite to # sources.list now if self.has_base_suite: import re sourceslist_fname = os.path.join(tdir, 'etc', 'apt', 'sources.list') if not mirror: with open(sourceslist_fname, 'r') as f: contents = f.read() matches = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\\(\\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', contents) if not matches: print_error('Unable to detect default APT repository URL (no regex matches).') return False mirror = matches[0] if not mirror: print_error('Unable to detect default APT repository URL.') return False if not components: components = ['main'] # FIXME: We should really be more clever here, e.g. depend on python-apt and parse sources.list properly with open(sourceslist_fname, 'a') as f: f.write('deb {mirror} {suite} {components}\n'.format(mirror=mirror, suite=self.suite, components=' '.join(components))) if extra_suites: f.write('\n') for esuite in extra_suites: if esuite == self.suite or esuite == bootstrap_suite: # don't add existing suites multiple times continue f.write('deb {mirror} {esuite} {components}\n'.format(mirror=mirror, esuite=esuite, components=' '.join(components))) if extra_source_lines: f.write('\n') for line in extra_source_lines.split('\\n'): f.write('{}\n'.format(line.strip())) print_section('Configure') if nspawn_run_helper_persist(self, tdir, self.new_nspawn_machine_name(), '--update') != 0: return False print_section('Creating Tarball') self._clear_image_tree(tdir) compress_directory(tdir, self.get_tarball_location()) # store configuration settings, so we can later recreate this tarball # or just display information about it self._write_config_json(mirror, components, extra_suites, extra_source_lines) return True def create(self, mirror=None, components=None, extra_suites=[], extra_source_lines=None): ''' Create new container base image (internal method) ''' ensure_root() if self.exists(): print_error('This configuration has already been created. You can only delete or update it.') return False ret = self._create_internal(mirror=mirror, components=components, extra_suites=extra_suites, extra_source_lines=extra_source_lines, show_header=True) if ret: print_info('Done.') return ret def delete(self): ''' Remove container base image ''' ensure_root() if not self.exists(): print_error('Can not delete "{}": The configuration does not exist.'.format(self.name)) return False print_header('Removing base image {}'.format(self.name)) print_section('Deleting cache') cache_size = self._aptcache.clear() print_info('Removed {} cached packages.'.format(cache_size)) self._aptcache.delete() print_info('Cache directory removed.') print_section('Deleting base tarball') os.remove(self.get_tarball_location()) config_fname = self.get_config_location() if os.path.isfile(config_fname): print_section('Deleting configuration manifest') os.remove(config_fname) print_info('Done.') return True @contextmanager def new_instance(self, basename=None): with temp_dir() as tdir: decompress_tarball(self.get_tarball_location(), tdir) yield tdir, self.new_nspawn_machine_name() def make_instance_permanent(self, instance_dir): ''' Add changes done in the current instance to the main tarball of this OS tree, replacing it. ''' # remove unwanted files from the tarball self._clear_image_tree(instance_dir) tarball_name = self.get_tarball_location() tarball_name_old = '{}.old'.format(tarball_name) os.replace(tarball_name, tarball_name_old) compress_directory(instance_dir, tarball_name) os.remove(tarball_name_old) tar_size = os.path.getsize(self.get_tarball_location()) print_info('New compressed tarball size is {}'.format(format_filesize(tar_size))) def update(self): ''' Update container base image ''' ensure_root() if not self.exists(): print_error('Can not update "{}": The configuration does not exist.'.format(self.name)) return False print_header('Updating container image') with self.new_instance() as (instance_dir, machine_name): # ensure helper script runner exists and is up to date self._copy_helper_script(instance_dir) print_section('Update') if nspawn_run_helper_persist(self, instance_dir, self.new_nspawn_machine_name(), '--update') != 0: return False print_section('Recreating tarball') self.make_instance_permanent(instance_dir) print_section('Cleaning up cache') cache_size = self._aptcache.clear() print_info('Removed {} cached packages.'.format(cache_size)) print_info('Done.') return True def recreate(self): ''' Recreate a container base image ''' ensure_root() if not self.exists(): print_error('Can not recreate "{}": The image does not exist.'.format(self.name)) return False config_fname = self.get_config_location() if not os.path.isfile(config_fname): print_error('Can not recreate "{}": Unable to find configuration data for this image.'.format(self.name)) return False print_header('Recreating container image') # read configuration data with open(config_fname, 'rt') as f: cdata = json.loads(f.read()) self._suite = cdata.get('Suite', self.suite) self._arch = cdata.get('Architecture', self.arch) self._variant = cdata.get('Variant', self.variant) mirror = cdata.get('Mirror') components = cdata.get('Components') extra_suites = cdata.get('ExtraSuites', []) extra_source_lines = cdata.get('ExtraSourceLines') print_section('Deleting cache') cache_size = self._aptcache.clear() print_info('Removed {} cached packages.'.format(cache_size)) self._aptcache.delete() print_info('Cache directory removed.') # move old image tarball out of the way image_name = self.get_tarball_location() image_name_old = self.get_tarball_location() + '.old' if os.path.isfile(image_name_old): print_info('Removing cruft image') os.remove(image_name_old) os.rename(image_name, image_name_old) print_info('Old tarball moved.') # ty to create the tarball again try: ret = self._create_internal(mirror=mirror, components=components, extra_suites=extra_suites, extra_source_lines=extra_source_lines, show_header=False) except Exception as e: print_error('Error while trying to create image: {}'.format(str(e))) ret = False if ret: if os.path.isfile(image_name_old): print_info('Removing old image') os.remove(image_name_old) print_info('Done.') return True else: print_info('Restoring old tarball') if os.path.isfile(image_name): print_info('Removing failed new image') os.remove(image_name) os.rename(image_name_old, image_name) print_info('Recreation failed.') return False def login(self, persistent=False, allowed=[]): ''' Interactive shell login into the container ''' ensure_root() if not self.exists(): print_info('Can not enter "{}": The configuration does not exist.'.format(self.name)) return False print_header('Login (persistent changes) for {}'.format(self.name) if persistent else 'Login for {}'.format(self.name)) with self.new_instance() as (instance_dir, machine_name): # ensure helper script runner exists and is up to date self._copy_helper_script(instance_dir) # run an interactive shell in the new container nspawn_run_persist(self, instance_dir, self.new_nspawn_machine_name(), '/srv', verbose=True, allowed=allowed) if persistent: print_section('Recreating tarball') self.make_instance_permanent(instance_dir) else: print_info('Changes discarded.') print_info('Done.') return True def run(self, command, build_dir, artifacts_dir, copy_command=False, header_msg=None, allowed=[]): ''' Run an arbitrary command or script in the container ''' ensure_root() if not self.exists(): print_error('Can not run command in "{}": The base image does not exist.'.format(self.name)) return False if len(command) <= 0: print_error('No command was given. Can not continue.') return False if header_msg: print_header(header_msg) # ensure we have absolute paths if build_dir: build_dir = os.path.abspath(build_dir) if artifacts_dir: artifacts_dir = os.path.abspath(artifacts_dir) with self.new_instance() as (instance_dir, machine_name): # ensure helper script runner exists and is up to date self._copy_helper_script(instance_dir) if copy_command: # copy the script from the host into our container and execute it there host_script = os.path.abspath(command[0]) if not os.path.isfile(host_script): print_error('Unable to find script "{}", can not copy it to the container. Exiting.'.format(host_script)) return False script_location = os.path.join(instance_dir, 'srv', 'tmp') Path(script_location).mkdir(parents=True, exist_ok=True) script_fname = os.path.join(script_location, os.path.basename(host_script)) if os.path.isfile(script_fname): os.remove(script_fname) shutil.copy2(host_script, script_fname) os.chmod(script_fname, 0o0755) command[0] = os.path.join('/srv', 'tmp', os.path.basename(host_script)) r = nspawn_run_helper_persist(self, instance_dir, machine_name, '--prepare-run', '/srv') if r != 0: print_error('Container setup failed.') return False print_section('Running Task') nspawn_flags = [] chdir = '/srv' if artifacts_dir: nspawn_flags.extend(['--bind={}:/srv/artifacts/'.format(os.path.normpath(artifacts_dir))]) if build_dir: nspawn_flags.extend(['--bind={}:/srv/build/'.format(os.path.normpath(build_dir))]) chdir = '/srv/build' r = nspawn_run_persist(self, instance_dir, machine_name, chdir, command, nspawn_flags, allowed=allowed) if r != 0: return False print_info('Done.') return True def print_container_base_image_info(gconf): ''' Search for all available container base images and list information about them. ''' from glob import glob osroots_dir = gconf.osroots_dir tar_files = [] if os.path.isdir(osroots_dir): tar_files = list(glob(os.path.join(osroots_dir, '*.tar.zst'))) if not tar_files: print_info('No container base images have been found!') return False tar_files_len = len(tar_files) for i, tar_fname in enumerate(tar_files): img_basepath = os.path.splitext(os.path.splitext(tar_fname)[0])[0] config_fname = img_basepath + '.json' imgid = os.path.basename(img_basepath) print('[{}]'.format(imgid)) # read configuration data if it exists if os.path.isfile(config_fname): with open(config_fname, 'rt') as f: cdata = json.loads(f.read()) for key, value in cdata.items(): if type(value) is list: value = '; '.join(value) print('{} = {}'.format(key, value)) tar_size = os.path.getsize(tar_fname) print('Size = {}'.format(format_filesize(tar_size))) if i != tar_files_len - 1: print() debspawn-0.4.0/debspawn/utils/000077500000000000000000000000001361140626300162635ustar00rootroot00000000000000debspawn-0.4.0/debspawn/utils/__init__.py000066400000000000000000000025101361140626300203720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . from .log import print_info, print_warn, print_error, print_header, print_section from .env import colored_output_allowed, unicode_allowed from .command import safe_run, run_forwarded from .misc import temp_dir, cd, hardlink_or_copy, format_filesize __all__ = ['print_info', 'print_warn', 'print_error', 'print_header', 'print_section', 'colored_output_allowed', 'unicode_allowed', 'safe_run', 'run_forwarded', 'temp_dir', 'cd', 'hardlink_or_copy', 'format_filesize'] debspawn-0.4.0/debspawn/utils/command.py000066400000000000000000000061131361140626300202540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2016-2020 Matthias Klumpp # Copyright (C) 2012-2013 Paul Tagliamonte # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import sys import shlex import subprocess from .log import TwoStreamLogger class SubprocessError(Exception): def __init__(self, out, err, ret, cmd): self.out = out self.err = err self.ret = ret self.cmd = cmd def __str__(self): return "%s: %d\n%s" % (str(self.cmd), self.ret, str(self.err)) # Input may be a byte string, a unicode string, or a file-like object def run_command(command, input=None): if not isinstance(command, list): command = shlex.split(command) if not input: input = None elif isinstance(input, str): input = input.encode('utf-8') elif not isinstance(input, bytes): input = input.read() try: pipe = subprocess.Popen(command, shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) except OSError: return (None, None, -1) (output, stderr) = pipe.communicate(input=input) (output, stderr) = (c.decode('utf-8', errors='ignore') for c in (output, stderr)) return (output, stderr, pipe.returncode) def safe_run(cmd, input=None, expected=0): if not isinstance(expected, tuple): expected = (expected, ) out, err, ret = run_command(cmd, input=input) if ret not in expected: raise SubprocessError(out, err, ret, cmd) return out, err, ret def run_forwarded(command): ''' Run a command, forwarding all output to the current stdout as well as to our build-logger in case we have one set previously. ''' if not isinstance(command, list): command = shlex.split(command) if isinstance(sys.stdout, TwoStreamLogger): proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) # ensure output is written to our file as well as stdout (as sys.stdout may be a redirect) while True: line = proc.stdout.readline() if proc.poll() is not None: break sys.stdout.write(str(line, 'utf-8', 'replace')) return proc else: return subprocess.run(command) debspawn-0.4.0/debspawn/utils/env.py000066400000000000000000000127141361140626300174320ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2017-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys import shutil from contextlib import contextmanager _unicode_allowed = True # store whether we are allowed to use unicode _owner_uid = 0 # uid of the user on whose behalf we are running _owner_gid = 0 # gid of the user on whose behalf we are running def set_owning_user(user, group=None): ''' Set the user on whose behalf we are running. This is useful so we can drop privileges to the perticular user in many cases. ''' from pwd import getpwnam, getpwuid from grp import getgrnam if user.isdecimal(): uid = int(user) else: uid = getpwnam(user).pw_uid if not group: gid = getpwuid(uid).pw_gid elif group.isdecimal(): gid = int(group) else: gid = getgrnam(group).gr_gid global _owner_uid global _owner_gid _owner_uid = uid _owner_gid = gid def ensure_root(): ''' Ensure we are running as root and all code following this function is privileged. ''' if os.geteuid() == 0: return args = sys.argv.copy() owner_set = any(a.startswith('--owner=') for a in sys.argv) if owner_set: # we don't override an owner explicitly set by the user args = sys.argv.copy() else: args = [sys.argv[0]] # set flag to tell the new process who it can impersonate # for unprivileged actions. It it is root, just omit the flag. uid = os.getuid() gid = os.getgid() if uid != 0 or gid != 0: args.append('--owner={}:{}'.format(uid, gid)) args.extend(sys.argv[1:]) def filter_env_far(result, name): value = os.environ.get(name) if not value: return result.append('{}={}'.format(name, shlex.quote(value))) if shutil.which('sudo'): # Filter "good" environment variables that we want to have after running sudo. # Most of those are standard variables affecting debsign bahevior later, in case # the user has requested signing import shlex env = [] filter_env_far(env, 'DEBEMAIL') filter_env_far(env, 'DEBFULLNAME') filter_env_far(env, 'GPGKEY') filter_env_far(env, 'GPG_AGENT_INFO') os.execvp("sudo", ["sudo"] + env + args) else: print('This command needs to be run as root.') sys.exit(1) @contextmanager def switch_unprivileged(): ''' Run actions using the unprivileged user ID on the behalf of which we are running. This is NOT a security feature! ''' import pwd global _owner_uid global _owner_gid if _owner_uid == 0 and _owner_gid == 0: # we can't really do much here, we have to run # as root, as we don't know an unprivileged user # to switch to yield else: orig_egid = os.getegid() orig_euid = os.geteuid() orig_home = os.environ.get('HOME') if not orig_home: orig_home = pwd.getpwuid(os.getuid()).pw_dir try: os.setegid(_owner_gid) os.seteuid(_owner_uid) os.environ['HOME'] = pwd.getpwuid(_owner_uid).pw_dir yield finally: os.setegid(orig_egid) os.seteuid(orig_euid) os.environ['HOME'] = orig_home def get_owner_uid_gid(): global _owner_uid global _owner_gid return _owner_uid, _owner_gid def colored_output_allowed(): return (hasattr(sys.stdout, "isatty") and sys.stdout.isatty()) or \ ('TERM' in os.environ and os.environ['TERM'] == 'ANSI') def unicode_allowed(): global _unicode_allowed return _unicode_allowed def set_unicode_allowed(val): global _unicode_allowed _unicode_allowed = val def get_free_space(path): ''' Return free space of :path ''' real_path = os.path.realpath(path) stat = os.statvfs(real_path) # get free space in MiB. free_space = float(stat.f_bsize * stat.f_bavail) return free_space def get_tree_size(path): ''' Return total size of files in path and subdirs. If is_dir() or stat() fails, print an error message to stderr and assume zero size (for example, file has been deleted). ''' total = 0 for entry in os.scandir(path): try: is_dir = entry.is_dir(follow_symlinks=False) except OSError as error: print('Error calling is_dir():', error, file=sys.stderr) continue if is_dir: total += get_tree_size(entry.path) else: try: total += entry.stat(follow_symlinks=False).st_size except OSError as error: print('Error calling stat():', error, file=sys.stderr) return total debspawn-0.4.0/debspawn/utils/log.py000066400000000000000000000110141361140626300174130ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2017-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys import re import shutil from .env import unicode_allowed def console_supports_color(): ''' Returns True if the running system's terminal supports color, and False otherwise. ''' is_a_tty = hasattr(sys.stdout, 'isatty') and sys.stdout.isatty() return 'ANSICON' in os.environ or is_a_tty def print_textbox(title, tl, hline, tr, vline, bl, br): def write_utf8(s): sys.stdout.buffer.write(s.encode('utf-8')) tlen = len(title) write_utf8('\n{}'.format(tl)) write_utf8(hline * (10 + tlen)) write_utf8('{}\n'.format(tr)) write_utf8('{} {}'.format(vline, title)) write_utf8(' ' * 8) write_utf8('{}\n'.format(vline)) write_utf8(bl) write_utf8(hline * (10 + tlen)) write_utf8('{}\n'.format(br)) sys.stdout.flush() def print_header(title): if unicode_allowed(): print_textbox(title, '╔', '═', '╗', '║', '╚', '╝') else: print_textbox(title, '+', '═', '+', '|', '+', '+') def print_section(title): if unicode_allowed(): print_textbox(title, '┌', '─', '┐', '│', '└', '┘') else: print_textbox(title, '+', '-', '+', '|', '+', '+') def print_info(*arg): ''' Prints an information message and ensures that it shows up on stdout immediately. ''' print(*arg) sys.stdout.flush() def print_warn(*arg): ''' Prints an information message and ensures that it shows up on stdout immediately. ''' if console_supports_color(): print('\033[93m/!\\\033[0m', *arg) else: print('/!\\', *arg) sys.stdout.flush() def print_error(*arg): ''' Prints an information message and ensures that it shows up on stdout immediately. ''' if console_supports_color(): print('\033[91mERROR:\033[0m', *arg, file=sys.stderr) else: print('ERROR:', *arg, file=sys.stderr) sys.stderr.flush() class TwoStreamLogger: ''' Permits logging messages to stdout/stderr as well as to a file. ''' class Buffer: def __init__(self, fstream, cstream): self._fstream = fstream self._cstream = cstream def write(self, message): self._fstream.write(str(message, 'utf-8', 'replace')) self._cstream.buffer.write(message) def __init__(self, fstream, cstream, fflush_always=False): self._fstream = fstream self._cstream = cstream self._fflush_always = fflush_always self._colorsub = re.compile('\x1b\\[(K|.*?m)') self.buffer = TwoStreamLogger.Buffer(fstream, cstream) def write(self, message): # write message to console self._cstream.write(message) if self._fflush_always: self.flush() # write message to file, stripping ANSI colors self._fstream.write(self._colorsub.sub('', message)) def flush(self): self._cstream.flush() self._fstream.flush() def copy_to(self, fname): self.flush() shutil.copy(self._fstream.name, fname) def isatty(self): return self._cstream.isatty() def capture_console_output(): ''' Direct console output to a file as well as to the original stdout/stderr terminal. ''' from tempfile import NamedTemporaryFile logfile = NamedTemporaryFile(mode='a', prefix='ds_', suffix='.log') nstdout = TwoStreamLogger(logfile, sys.stdout) nstderr = TwoStreamLogger(logfile, sys.stderr, True) sys.stdout = nstdout sys.stderr = nstderr def save_captured_console_output(fname): from .env import get_owner_uid_gid if hasattr(sys.stdout, 'copy_to'): o_uid, o_gid = get_owner_uid_gid() sys.stdout.copy_to(fname) os.chown(fname, o_uid, o_gid) debspawn-0.4.0/debspawn/utils/misc.py000066400000000000000000000050631361140626300175740ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2017-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import shutil from pathlib import Path from contextlib import contextmanager from ..config import GlobalConfig @contextmanager def cd(where): ncwd = os.getcwd() try: yield os.chdir(where) finally: os.chdir(ncwd) @contextmanager def temp_dir(basename=None): from random import choice from string import ascii_lowercase, digits rdm_id = ''.join(choice(ascii_lowercase + digits) for _ in range(8)) if basename: dir_name = '{}-{}'.format(basename, rdm_id) else: dir_name = rdm_id temp_basedir = GlobalConfig().temp_dir if not temp_basedir: temp_basedir = '/var/tmp/debspawn/' tmp_path = os.path.join(temp_basedir, dir_name) Path(tmp_path).mkdir(parents=True, exist_ok=True) try: yield tmp_path finally: shutil.rmtree(tmp_path) def format_filesize(num, suffix='B'): for unit in ['', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi']: if abs(num) < 1024.0: return "%3.1f%s%s" % (num, unit, suffix) num /= 1024.0 return "%.1f%s%s" % (num, 'Yi', suffix) def current_time_string(): ''' Get the current time as human-readable string. ''' from datetime import datetime, timezone utc_dt = datetime.now(timezone.utc) return utc_dt.astimezone().strftime('%Y-%m-%d %H:%M:%S UTC%z') def version_noepoch(version): ''' Return version from :version without epoch. ''' version_noe = version if ':' in version_noe: version_noe = version_noe.split(':', 1)[1] return version_noe def hardlink_or_copy(src, dst): ''' Hardlink a file :src to :dst or copy the file in case linking is not possible ''' try: os.link(src, dst) except (PermissionError, OSError): shutil.copy2(src, dst) debspawn-0.4.0/debspawn/utils/zstd_tar.py000066400000000000000000000036621361140626300204760ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2018-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import shutil from .command import run_command def ensure_tar_zstd(): ''' Check if the required binaries for compression are available ''' if not shutil.which('zstd'): raise Exception('The "zsdt" binary was not found, we can not compress tarballs. Please install zstd to continue!') if not shutil.which('tar'): raise Exception('The "tar" binary was not found, we can not create tarballs. Please install tar to continue!') def compress_directory(dirname, tarname): ''' Compress a directory to a given tarball ''' cmd = ['tar', '-C', dirname, '-I', 'zstd', '-cf', tarname, '.'] out, err, ret = run_command(cmd) if ret != 0: raise Exception('Unable to create tarball "{}":\n{}{}'.format(tarname, out, err)) def decompress_tarball(tarname, dirname): ''' Compress a directory to a given tarball ''' cmd = ['tar', '-C', dirname, '-I', 'zstd', '-xf', tarname] out, err, ret = run_command(cmd) if ret != 0: raise Exception('Unable to decompress tarball "{}":\n{}{}'.format(tarname, out, err)) debspawn-0.4.0/docs/000077500000000000000000000000001361140626300142505ustar00rootroot00000000000000debspawn-0.4.0/docs/__init__.py000066400000000000000000000000001361140626300163470ustar00rootroot00000000000000debspawn-0.4.0/docs/assemble_man.py000066400000000000000000000072161361140626300172560ustar00rootroot00000000000000#!/usr/bin/env python3 import os import sys from xml.sax.saxutils import escape as xml_escape from functools import reduce sys.path.append("..") class DocbookEditor: def __init__(self): self._replacements = {} def add_substvar(self, name, replacement): self._replacements['@{}@'.format(name)] = replacement def register_command_flag_synopsis(self, actions, command_name): flags_text = '' flags_entries = '' for item in actions: options_text = xml_escape('|'.join(item.option_strings)) flags_text += '{}\n'.format(options_text) oid = item.option_strings[0] desc_text = None if oid == '-h': desc_text = 'Print brief help information about available commands.' if command_name != 'create': if oid == '--variant': desc_text = 'Set the variant of the selected image, that was used for bootstrapping.' elif oid == '-a': desc_text = 'The architecture of the base image that should be selected.' if not desc_text: desc_text = item.help desc_text = xml_escape(desc_text) if desc_text.startswith('CF|'): desc_text = desc_text[3:] desc_text = desc_text.replace('binary:', ':', 1) desc_text = desc_text.replace('arch:', ':', 1) desc_text = desc_text.replace('indep:', ':', 1) desc_text = desc_text.replace('source:', ':', 1) flags_entries += ''' {} {} '''.format(options_text, desc_text) self.add_substvar('{}_FLAGS_SYNOPSIS'.format(command_name.upper()), flags_text) self.add_substvar('{}_FLAGS_ENTRIES'.format(command_name.upper()), flags_entries) def process_file(self, input_fname, output_fname): with open(input_fname, 'r') as f: template_content = f.read() result = reduce(lambda x, y: x.replace(y, self._replacements[y]), self._replacements, template_content) with open(output_fname, 'w') as f: f.write(result) return output_fname def generate_docbook_pages(build_dir): from debspawn.cli import create_parser build_dir = os.path.abspath(build_dir) parser = create_parser() editor = DocbookEditor() editor.register_command_flag_synopsis(parser._get_optional_actions(), 'BASE') xml_manpages = [] xml_manpages.append(editor.process_file('docs/debspawn.1.xml', os.path.join(build_dir, 'debspawn.1.xml'))) for command, sp in parser._get_positional_actions()[0]._name_parser_map.items(): editor.register_command_flag_synopsis(sp._get_optional_actions(), command) template_fname = 'docs/debspawn-{}.1.xml'.format(command) if not os.path.isfile(template_fname): if command in ['ls', 'b']: continue # the ls and b shorthands need to manual page print('Manual page template {} is missing! Skipping it.'.format(template_fname)) continue xml_manpages.append(editor.process_file(template_fname, os.path.join(build_dir, os.path.basename(template_fname)))) return xml_manpages if __name__ == '__main__': generate_docbook_pages('/tmp') debspawn-0.4.0/docs/debspawn-build.1.xml000066400000000000000000000130221361140626300200270ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Build Debian packages in a container &command; @BUILD_FLAGS_SYNOPSIS@ SUITE DIR|DSC_FILE Description Build a Debian package from a directory or source package *.dsc file. debspawn will create a new container for the respective build using the base image specified, build the package and return build artifacts in the default output directory /var/lib/debspawn/results/ unless a different location was specified via the flag. Downloaded packages that are build dependencies are cached and will be reused on subsequent builds if possible. You can inject packages into the build environment that are not available in the preconfigured APT repositories by placing them in /var/lib/debspawn/injected-pkgs/${container-name}, or in /var/lib/debspawn/injected-pkgs/ to make a package available in all environments. Internally, debspawn will build a transient package repository with the respective packages and add it as a package source for APT. If you want to debug the package build process, you can pass the flag to debspawn. This will open an interactive root shell in the build environment post-build, no matter whether the build failed or succeeded. After investigating the issue / building the package manually, the shell can be exited and the user is asked whether debspawn should copy back the changes made in the packages' debian/ directory to the host to make them permanent. Please keep in mind that while interactive mode is enabled, no build log can be created. Examples You can build a package from its source directory, or just by passing a plain .dsc file to &command;. If the result should be automatically signed, the flag needs to be passed too: $ cd ~/packages/hello $ &command; sid --sign $ &command; --arch=i386 cosmic ./hello_2.10-1.dsc You can also build packages using git-buildpackage and debspawn. In this case the flag is also used to perform a Lintian static analysis check in the container after build: $ gbp buildpackage --git-builder='debspawn b sid --lintian --sign' To debug a build issue interactively, the flag can be used: $ &command; sid --interact Options @BUILD_FLAGS_ENTRIES@ Differences to sbuild On Debian, sbuild is the primary tool used for package building, which uses different technology. So naturally, the question is whether the sbuild build environments and the debspawn build environments are be identical or at least compatible. Due to the different technology used, there may be subtle differences between sbuild chroots and debspawn containers. The differences should not have any impact on package builds, and any such occurrence is highly likely a bug in the package's build process. If you think it is not, please file a bug against Debspawn. We try to be as close to sbuild's default environment as possible, but unfortunately can not make any guarantees. One way the build environment of debspawn differs from Debian's default sbuild setup intentionally is in its consistent use of unicode. By default, debspawn will ensure that unicode is always available and enabled. If you do not want this behavior, you can pass the flag to &command; to disable unicode in the tool itself and in the build environment. See Also debspawn-update(1), debspawn-create(1), dpkg-buildpackage(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-create.1.xml000066400000000000000000000040521361140626300201760ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Create new container images &command; @CREATE_FLAGS_SYNOPSIS@ SUITE Description Create a new base image for a suite known to debootstrap(1). The image will later be used to spawn ephemeral containers in which packages can be built. Examples You can easily create images for any suite that has a script in debootstrap. For example, to create a Debian Unstable image for your current machine architecture, you can use: $ &command; sid A more advanced example, for building on Ubuntu 18.10 on the x86 architecture: $ &command; --arch=i386 cosmic Options @CREATE_FLAGS_ENTRIES@ See Also debspawn-build(1), debootstrap(1), systemd-nspawn(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-delete.1.xml000066400000000000000000000030411361140626300201720ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Remove a container image &command; @DELETE_FLAGS_SYNOPSIS@ SUITE Description Remove an image known to debspawn and clear all data related to it. This explicitly includes any cached data, but does not include generated build artifacts that may still exist in the results directory. Options @DELETE_FLAGS_ENTRIES@ See Also debspawn-create(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-list.1.xml000066400000000000000000000027461361140626300177160ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; List information about container images &command; @LIST_FLAGS_SYNOPSIS@ SUITE Description This command will list detailed information about all currently registered container images that Debspawn can use as build environments. Options @LIST_FLAGS_ENTRIES@ See Also debspawn-create(1), debspawn-update(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-login.1.xml000066400000000000000000000031631361140626300200450ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Open interactive shell session in a container &command; @LOGIN_FLAGS_SYNOPSIS@ SUITE Description This command enters an interactive shell session in a container that is normally used for building. This can be useful to inspect the build environment, or to manually customize the container image for special applications if the flag is set. Options @LOGIN_FLAGS_ENTRIES@ See Also debspawn(1), systemd-nspawn(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-run.1.xml000066400000000000000000000033531361140626300175420ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Run arbitrary commands in debspawn container session &command; @RUN_FLAGS_SYNOPSIS@ SUITE Description This subcommand allows you to run arbitrary commands in an ephemeral debspawn container, using the same environment that is normally used for building packages. &command; is explicitly designed to be used by other tools for custom applications, and usually, you will want to use debspawn build instead to build Debian packages. Options @RUN_FLAGS_ENTRIES@ See Also debspawn-build(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn-update.1.xml000066400000000000000000000043361361140626300202220ustar00rootroot00000000000000 18 August, 2018"> ]> &command; 2018-2020 Matthias Klumpp Debspawn &date; &pagename; 1 &pagename; Update a container image &command; @UPDATE_FLAGS_SYNOPSIS@ SUITE Description Update a container base image. This achieves the same thing as running apt update && apt full-upgrade on the base image and making the changes permanent. Additionally, &command; will prune all caches and ensure all required packages and scripts are installed in the container image. Running &command; on the images that are in use about once a week ensures builds will happen faster, due to less changes that have to be done prior to each build. Examples Updating images is easy, you just pass the same arguments you used for creating them, but use the update subcommand instead: $ &command; sid $ &command; --arch=i386 cosmic Options @UPDATE_FLAGS_ENTRIES@ See Also debspawn-create(1), debspawn-build(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/docs/debspawn.1.xml000066400000000000000000000120021361140626300167270ustar00rootroot00000000000000 18 August, 2018"> ]> debspawn 2018-2020 Matthias Klumpp Debspawn &date; debspawn 1 &package; Build in nspawn containers &package; @BASE_FLAGS_SYNOPSIS@ Description This manual page documents the &package; command. &package; is a tool to build Debian packages in an isolated environment, using nspawn containers. By using containers, Debspawn can isolate builds from the host system much better than a regular chroot could. It also allows for more advanced features to manage builds, for example setting resource limits for individual builds. Please keep in mind that Debspawn is not a security feature! While it provides a lot of isolation from the host system, you should not run arbitrary untrusted code with it. The usual warnings for all technology based on Linux containers apply here. See systemd-nspawn(1) for more information on the container solution Debspawn uses. Debspawn also allows one to run arbitrary custom commands in its environment. This is useful to execute a variety of non-package build and QA actions that make sense to be run in the same environment in which packages are usually built. For more information about the Debspawn project, you can visit its project page. Subcommands &package; actions are invoked via subcommands. Refer to their individual manual pages for further details. Create a new container base image for a specific suite, architecture and variant. A custom mirror location can also be provided. For details, see debspawn-create(1). List information about all container image that Debspawn knows on the current host. For details, see debspawn-list(1). Delete a container base image and all data associated with it. For details, see debspawn-delete(1). Update a container base image, ensuring all packages are up to date and the image is set up properly for use with debspawn. For details, see debspawn-update(1). Build a Debian package in an isolated environment. For details, see debspawn-build(1). Get an interactive shell session in a container. For details, see debspawn-login(1). Run arbitrary commands in debspawn container session. This is primarily useful for using &package; to isolate non-package build processes. For details, see debspawn-run(1). Flags @BASE_FLAGS_ENTRIES@ See Also dpkg-buildpackage(1), systemd-nspawn(1), sbuild(1). AUTHOR This manual page was written by Matthias Klumpp mak@debian.org. debspawn-0.4.0/setup.cfg000066400000000000000000000001071361140626300151370ustar00rootroot00000000000000[flake8] max-line-length = 180 [metadata] description-file = README.md debspawn-0.4.0/setup.py000077500000000000000000000072431361140626300150430ustar00rootroot00000000000000#!/usr/bin/env python3 import os import sys import platform import shutil from debspawn import __appname__, __version__ from setuptools import setup from setuptools.command.install_scripts import install_scripts as install_scripts_orig from subprocess import check_call from docs.assemble_man import generate_docbook_pages class install_scripts(install_scripts_orig): def _create_manpage(self, xml_src, out_dir): man_name = os.path.splitext(os.path.basename(xml_src))[0] out_fname = os.path.join(out_dir, man_name) print('Generating manual page {}'.format(man_name)) check_call(['xsltproc', '--nonet', '--stringparam', 'man.output.quietly', '1', '--stringparam', 'funcsynopsis.style', 'ansi', '--stringparam', 'man.th.extra1.suppress', '1', '-o', out_fname, 'http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl', xml_src]) return out_fname def run(self): if platform.system() == 'Windows': super().run() return if not self.skip_build: self.run_command('build_scripts') self.outfiles = [] # check for xsltproc, we need it to build manual pages if not shutil.which('xsltproc'): print('The "xsltproc" binary was not found. Please install it to continue!') sys.exit(1) if self.dry_run: return if '--single-version-externally-managed' not in sys.argv: print() print('Attempting to install Debspawn as binary distribution may not yield a working installation.', file=sys.stderr) print('We require a file to be installed in a system location, and manual pages are in an external location as well.', file=sys.stderr) print(('Currently, no workarounds for this issue have been implemented in Debspawn itself, so please run setup.py with ' '`--single-version-externally-managed`.'), file=sys.stderr) print('If you are using pip, try `sudo pip3 install --no-binary debspawn .`', file=sys.stderr) sys.exit(1) self.mkpath(self.install_dir) # We want the files to be installed without a suffix on Unix for infile in self.get_inputs(): infile = os.path.basename(infile) in_built = os.path.join(self.build_dir, infile) in_stripped = infile[:-3] if infile.endswith('.py') else infile outfile = os.path.join(self.install_dir, in_stripped) # NOTE: Mode is preserved by default self.copy_file(in_built, outfile) self.outfiles.append(outfile) # handle generation of manual pages man_dir = os.path.normpath(os.path.join(self.install_dir, '..', 'share', 'man', 'man1')) self.mkpath(man_dir) pages = generate_docbook_pages(self.build_dir) for page in pages: self.outfiles.append(self._create_manpage(page, man_dir)) cmdclass = { 'install_scripts': install_scripts, } packages = [ 'debspawn', 'debspawn.utils', ] package_data = {'': ['debspawn/dsrun']} scripts = ['debspawn.py'] setup( name=__appname__, version=__version__, author="Matthias Klumpp", author_email="matthias@tenstral.net", description='Debian package builder and build helper using systemd-nspawn', license="LGPL-3.0+", url="https://lkorigin.github.io/", python_requires='>=3.5', platforms=['any'], zip_safe=False, include_package_data=True, packages=packages, cmdclass=cmdclass, package_data=package_data, scripts=scripts ) debspawn-0.4.0/tests/000077500000000000000000000000001361140626300144625ustar00rootroot00000000000000debspawn-0.4.0/tests/__init__.py000066400000000000000000000021331361140626300165720ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2019-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys thisfile = __file__ if not os.path.isabs(thisfile): thisfile = os.path.normpath(os.path.join(os.getcwd(), thisfile)) source_root = os.path.normpath(os.path.join(os.path.dirname(thisfile), '..')) sys.path.append(os.path.normpath(source_root)) __all__ = ['source_root'] debspawn-0.4.0/tests/conftest.py000066400000000000000000000060161361140626300166640ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2019-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . import os import sys import pytest @pytest.fixture(scope='session', autouse=True) def gconfig(): ''' Ensure the global config object is set up properly for unit-testing. ''' import shutil import debspawn.cli from . import source_root debspawn.cli.__mainfile = os.path.join(source_root, 'debspawn.py') class MockOptions: config = None no_unicode = False owner = None gconf = debspawn.cli.init_config(MockOptions()) test_tmp_dir = '/tmp/debspawn-test/' shutil.rmtree(test_tmp_dir, ignore_errors=True) os.makedirs(test_tmp_dir) gconf._instance._osroots_dir = os.path.join(test_tmp_dir, 'containers/') gconf._instance._results_dir = os.path.join(test_tmp_dir, 'results/') gconf._instance._aptcache_dir = os.path.join(test_tmp_dir, 'aptcache/') gconf._instance._injected_pkgs_dir = os.path.join(test_tmp_dir, 'injected-pkgs/') return gconf @pytest.fixture(scope='session', autouse=True) def ensure_root(): ''' Ensure we run with superuser permissions. ''' if os.geteuid() != 0: print('The testsuite has to be run with superuser permissions in order to create nspawn instances.') sys.exit(1) @pytest.fixture(scope='session') def build_arch(): ''' Retrieve the current architecture we should build packages for. ''' from debspawn.utils.command import safe_run out, _, ret = safe_run(['dpkg-architecture', '-q', 'DEB_BUILD_ARCH']) assert ret == 0 arch = out.strip() if not arch: arch = 'amd64' # assume arm64 as default return arch @pytest.fixture(scope='session') def testing_container(gconfig, build_arch): ''' Create a container for Debian stable used for default tests ''' from debspawn.osbase import OSBase suite = 'stable' variant = 'minbase' components = ['main', 'contrib', 'non-free'] extra_suites = [] osbase = OSBase(gconfig, suite, build_arch, variant=variant, base_suite=None) r = osbase.create(None, components, extra_suites, None) assert r return (suite, build_arch, variant) debspawn-0.4.0/tests/test_cud.py000066400000000000000000000027701361140626300166540ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2019-2020 Matthias Klumpp # # Licensed under the GNU Lesser General Public License Version 3 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the license, or # (at your option) any later version. # # This software is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this software. If not, see . from debspawn.osbase import OSBase def test_container_create_delete(gconfig, testing_container): # the "default" container is created by a fixture. # what we actually want to do here in future is create and # delete containers with special settings pass def test_container_update(gconfig, testing_container): ''' Update a container ''' suite, arch, variant = testing_container osbase = OSBase(gconfig, suite, arch, variant) assert osbase.update() def test_container_recreate(gconfig, testing_container): ''' Test recreating a container ''' suite, arch, variant = testing_container osbase = OSBase(gconfig, suite, arch, variant) assert osbase.recreate()