pax_global_header00006660000000000000000000000064147751752410014527gustar00rootroot0000000000000052 comment=28a3a159559fb389240b6d14b57dad9b6c9ba93d mmdebstrap/000077500000000000000000000000001477517524100132315ustar00rootroot00000000000000mmdebstrap/.gitignore000066400000000000000000000000071477517524100152160ustar00rootroot00000000000000shared mmdebstrap/.mailmap000066400000000000000000000007141477517524100146540ustar00rootroot00000000000000Johannes Schauer Marin Rodrigues Johannes Schauer Marin Rodrigues Johannes Schauer Marin Rodrigues Johannes Schauer Marin Rodrigues Helmut Grohne Benjamin Drung mmdebstrap/.perltidyrc000066400000000000000000000007261477517524100154200ustar00rootroot00000000000000# mmdebstrap is a tool focused on Debian and derivatives (it relies on apt # after all). Thus, we use a perl style used in other Debian Perl code. The # following options are used in Lintian and devscripts --break-before-all-operators --noblanks-before-comments --cuddled-else --maximum-line-length=79 --paren-tightness=2 --square-bracket-tightness=2 --space-for-semicolon --opening-brace-always-on-right --stack-opening-tokens --stack-closing-tokens --format-skipping mmdebstrap/CHANGELOG.md000066400000000000000000000334311477517524100150460ustar00rootroot000000000000001.5.7 (2025-04-08) ------------------ - support for elxr Debian derivative - support loong64 - mmdebstrap-autopkgtest-build-qemu: add --sshkey=... 1.5.6 (2025-01-11) ------------------ - bugfix release 1.5.5 (2025-01-09) ------------------ - do not run dpkg-preconfigure via DPkg::Pre-Install-Pkgs hook - m-a-b-q: support passing args to mmdebstrap 1.5.4 (2024-10-28) ------------------ - do not generate apt sources.list entry if SUITE is empty 1.5.3 (2024-09-13) ------------------ - tidy up any zombie processes - chrootless hurd-i386 - add --skip=cleanup/reproducible/machine-id - m-a-b-q: replace test_installed by dpkg-checkbuilddeps 1.5.2 (2024-06-26) ------------------ - mmdebstrap-autopkgtest-build-qemu produces bit-by-bit reproducible output 1.5.1 (2024-06-03) ------------------ - in root and unshare mode, run 'mount --make-rprivate /' before bind-mounting - switch apt variant from using 'apt-get dist-upgrade' to apt patterns 1.5.0 (2024-05-14) ------------------ - add --format=ext4 1.4.3 (2024-02-01) ------------------ - take hard links into account when computing disk usage 1.4.2 (2024-01-29) ------------------ - allow for start-stop-daemon to be in either /sbin or /usr/sbin - mmdebstrap-autopkgtest-build-qemu: fix octal mode computation and hostname 1.4.1 (2024-01-09) ------------------ - set DPkg::Chroot-Directory in APT_CONFIG to simplify calling apt in hooks - disallow running chrootless as root without fakeroot unless --skip=check/chrootless is used - only print short --help output if wrong args are passed - read files passed as --aptopt and --dpkgopt outside the unshared namespace 1.4.0 (2023-10-24) ------------------ - add mmdebstrap-autopkgtest-build-qemu - export container=mmdebstrap-unshare env variable in unshare-mode hooks - add new skip options: output/dev, output/mknod, tar-in/mknod, copy-in/mknod, sync-in/mknod - stop copying qemu-$arch-static binary into the chroot - tarfilter: add --type-exclude option - set MMDEBSTRAP_FORMAT in hooks - do not install priority:required in buildd variant following debootstrap 1.3.8 (2023-08-20) ------------------ - hooks/merged-usr: implement post-merging as debootstrap does - exclude ./lost+found from tarball 1.3.7 (2023-06-21) ------------------ - add hooks/copy-host-apt-sources-and-preferences 1.3.6 (2023-06-16) ------------------ - bugfix release 1.3.5 (2023-03-20) ------------------ - bugfix release 1.3.4 (2023-03-16) ------------------ - more safeguards before automatically choosing unshare mode 1.3.3 (2023-02-19) ------------------ - testsuite improvements 1.3.2 (2023-02-16) ------------------ - unshare mode works in privileged docker containers 1.3.1 (2023-01-20) ------------------ - bugfix release 1.3.0 (2023-01-16) ------------------ - add hooks/maybe-jessie-or-older and hooks/maybe-merged-usr - add --skip=check/signed-by - hooks/jessie-or-older: split into two individual hook files - skip running apt-get update if we are very sure that it was already run - be more verbose when 'apt-get update' failed - warn if a hook is named like one but not executable and if a hook is executable but not named like one - to find signed-by value, run gpg on the individual keys to print better error messages in case it fails (gpg doesn't give an indication which file it was unable to read) and print progress bar - allow empty sources.list entries 1.2.5 (2023-01-04) ------------------ - bugfix release 1.2.4 (2022-12-23) ------------------ - bugfix release - add jessie-or-older extract hook 1.2.3 (2022-11-16) ------------------ - use Text::ParseWords::shellwords instead of spawning a new shell - mount and unmount once, instead for each run_chroot() call 1.2.2 (2022-10-27) ------------------ - allow /etc/apt/trusted.gpg.d/ not to exist - always create /var/lib/dpkg/arch to make foreign architecture chrootless tarballs bit-by-bit identical - write an empty /etc/machine-id instead of writing 'uninitialized' - only print progress bars on interactive terminals that are wide enough 1.2.1 (2022-09-08) ------------------ - bugfix release 1.2.0 (2022-09-05) ------------------ - remove proot mode - error out if stdout is an interactive terminal - replace taridshift by tarfilter --idshift - tarfilter: add --transform option - multiple --skip options can be separated by comma or whitespace - also cleanup the contents of /run - support apt patterns and paths with commas and whitespace in --include - hooks: store the values of the --include option in MMDEBSTRAP_INCLUDE - add new --skip options: chroot/start-stop-daemon, chroot/policy-rc.d chroot/mount, chroot/mount/dev, chroot/mount/proc, chroot/mount/sys, cleanup/run 1.1.0 (2022-07-26) ---------------- - mount a new /dev/pts instance into the chroot to make posix_openpt work - adjust merged-/usr hook to work the same way as debootstrap - add no-merged-usr hook 1.0.1 (2022-05-29) ------------------ - bugfix release 1.0.0 (2022-05-28) ------------------ - all documented interfaces are now considered stable - allow file:// mirrors - /var/cache/apt/archives/ is now allowed to contain *.deb packages - add file-mirror-automount hook-dir - set $MMDEBSTRAP_VERBOSITY in hooks - rewrite coverage with multiple individual and skippable shell scripts 0.8.6 (2022-03-25) ------------------ - allow running root mode inside unshare mode 0.8.5 (2022-03-07) ------------------ - improve documentation 0.8.4 (2022-02-11) ------------------ - tarfilter: add --strip-components option - don't install essential packages in run_install() - remove /var/lib/dbus/machine-id 0.8.3 (2022-01-08) ------------------ - allow codenames with apt patterns (requires apt >= 2.3.14) - don't overwrite existing files in setup code - don't copy in qemu-user-static binary if it's not needed 0.8.2 (2021-12-14) ------------------ - use apt patterns to select priority variants (requires apt >= 2.3.10) 0.8.1 (2021-10-07) ------------------ - enforce dpkg >= 1.20.0 and apt >= 2.3.7 - allow working directory be not world readable - do not run xz and zstd with --threads=0 since this is a bad default for machines with more than 100 cores - bit-by-bit identical chrootless mode 0.8.0 (2021-09-21) ------------------ - allow running inside chroot in root mode - allow running without /dev, /sys or /proc - new --format=null which gets automatically selected if the output is /dev/null and doesn't produce a tarball or other permanent output - allow ASCII-armored keyrings (requires gnupg >= 2.2.8) - run zstd with --threads=0 - tarfilter: add --pax-exclude and --pax-include to strip extended attributes - add --skip=setup, --skip=update and --skip=cleanup - add --skip=cleanup/apt/lists and --skip=cleanup/apt/cache - pass extended attributes (excluding system) to tar2sqfs - use apt-get update -error-on=any (requires apt >= 2.1.16) - support Debian 11 Buster - use apt from outside using DPkg::Chroot-Directory (requires apt >= 2.3.7) * build chroots without apt (for example from buildinfo files) * no need to install additional packages like apt-transport-* or ca-certificates inside the chroot * no need for additional key material inside the chroot * possible use of file:// and copy:// - use apt pattern to select essential set - write 'uninitialized' to /etc/machine-id - allow running in root mode without mount working, either because of missing CAP_SYS_ADMIN or missing /usr/bin/mount - make /etc/ld.so.cache under fakechroot mode bit-by-bit identical to root and unshare mode - move hooks/setup00-merged-usr.sh to hooks/merged-usr/setup00.sh - add gpgvnoexpkeysig script for very old snapshot.d.o timestamps with expired signature 0.7.5 (2021-02-06) ------------------ - skip emulation check for extract variant - add new suite name trixie - unset TMPDIR in hooks because there is no value that works inside as well as outside the chroot - expose hook name to hooks via MMDEBSTRAP_HOOK environment variable 0.7.4 (2021-01-16) ------------------ - Optimize mmtarfilter to handle many path exclusions - Set MMDEBSTRAP_APT_CONFIG, MMDEBSTRAP_MODE and MMDEBSTRAP_HOOKSOCK for hook scripts - Do not run an additional env command inside the chroot - Allow unshare mode as root user - Additional checks whether root has the necessary privileges to mount - Make most features work on Debian 10 Buster 0.7.3 (2020-12-02) ------------------ - bugfix release 0.7.2 (2020-11-28) ------------------ - check whether tools like dpkg and apt are installed at startup - make it possible to seed /var/cache/apt/archives with deb packages - if a suite name was specified, use the matching apt index to figure out the package set to install - use Debian::DistroInfo or /usr/share/distro-info/debian.csv (if available) to figure out the security mirror for bullseye and beyond - use argparse in tarfilter and taridshift for proper --help output 0.7.1 (2020-09-18) ------------------ - bugfix release 0.7.0 (2020-08-27) ----------------- - the hook system (setup, extract, essential, customize and hook-dir) is made public and is now a documented interface - tarball is also created if the output is a named pipe or character special - add --format option to control the output format independent of the output filename or in cases where output is directed to stdout - generate ext2 filesystems if output file ends with .ext2 or --format=ext2 - add --skip option to prevent some automatic actions from being carried out - implement dpkg-realpath in perl so that we don't need to run tar inside the chroot anymore for modes other than fakechroot and proot - add ready-to-use hook scripts for eatmydata, merged-usr and busybox - add tarfilter tool - use distro-info-data and debootstrap to help with suite name and keyring discovery - no longer needs to install twice when --depkgopt=path-exclude is given - variant=custom and hooks can be used as a debootstrap wrapper - use File::Find instead of "du" to avoid different results on different filesystems - many, many bugfixes and documentation enhancements 0.6.1 (2020-03-08) ------------------ - replace /etc/machine-id with an empty file - fix deterministic tar with pax and xattr support - support deb822-style format apt sources - mount /sys and /proc as read-only in root mode - unset TMPDIR environment variable for everything running inside the chroot 0.6.0 (2020-01-16) ------------------ - allow multiple --architecture options - allow multiple --include options - enable parallel compression with xz by default - add --man option - add --keyring option overwriting apt's default keyring - preserve extended attributes in tarball - allow running tests on non-amd64 systems - generate squashfs images if output file ends in .sqfs or .squashfs - add --dry-run/--simulate options - add taridshift tool 0.5.1 (2019-10-19) ------------------ - minor bugfixes and documentation clarification - the --components option now takes component names as a comma or whitespace separated list or as multiple --components options - make_mirror.sh now has to be invoked manually before calling coverage.sh 0.5.0 (2019-10-05) ------------------ - do not unconditionally read sources.list stdin anymore * if mmdebstrap is used via ssh without a pseudo-terminal, it will stall forever * as this is unexpected, one now has to explicitly request reading sources.list from stdin in situations where it's ambiguous whether that is requested * thus, the following modes of operation don't work anymore: $ mmdebstrap unstable /output/dir < sources.list $ mmdebstrap unstable /output/dir http://mirror < sources.list * instead, one now has to write: $ mmdebstrap unstable /output/dir - < sources.list $ mmdebstrap unstable /output/dir http://mirror - < sources.list - fix binfmt_misc support on docker - do not use qemu for architectures unequal the native architecture that can be used without it - do not copy /etc/resolv.conf or /etc/hostname if the host system doesn't have them - add --force-check-gpg dummy option - allow hooks to remove start-stop-daemon - add /var/lib/dpkg/arch in chrootless mode when chroot architecture differs - create /var/lib/dpkg/cmethopt for dselect - do not skip package installation in 'custom' variant - fix EDSP output for external solvers so that apt doesn't mark itself as Essential:yes - also re-exec under fakechroot if fakechroot is picked in 'auto' mode - chdir() before 'apt-get update' to accomodate for apt << 1.5 - add Dir::State::Status to apt config for apt << 1.3 - chmod 0755 on qemu-user-static binary - select the right mirror for ubuntu, kali and tanglu 0.4.1 (2019-03-01) ------------------ - re-enable fakechroot mode testing - disable apt sandboxing if necessary - keep apt and dpkg lock files 0.4.0 (2019-02-23) ------------------ - disable merged-usr - add --verbose option that prints apt and dpkg output instead of progress bars - add --quiet/--silent options which print nothing on stderr - add --debug option for even more output than with --verbose - add some no-op options to make mmdebstrap a drop-in replacement for certain debootstrap wrappers like sbuild-createchroot - add --logfile option which outputs to a file what would otherwise be written to stderr - add --version option 0.3.0 (2018-11-21) ------------------ - add chrootless mode - add extract and custom variants - make testsuite unprivileged through qemu and guestfish - allow empty lost+found directory in target - add 54 testcases and fix lots of bugs as a result 0.2.0 (2018-10-03) ------------------ - if no MIRROR was specified but there was data on standard input, then use that data as the sources.list instead of falling back to the default mirror - lots of bug fixes 0.1.0 (2018-09-24) ------------------ - initial release mmdebstrap/README.md000066400000000000000000000152011477517524100145070ustar00rootroot00000000000000mmdebstrap ========== An alternative to debootstrap which uses apt internally and is thus able to use more than one mirror and resolve more complex dependencies. Usage ----- Use like debootstrap: sudo mmdebstrap unstable ./unstable-chroot Without superuser privileges: mmdebstrap unstable unstable-chroot.tar With complex apt options: cat /etc/apt/sources.list | mmdebstrap > unstable-chroot.tar For the full documentation use: pod2man ./mmdebstrap | man -l - Or read a HTML version of the man page in either of these locations: - https://gitlab.mister-muffin.de/josch/mmdebstrap/wiki - https://manpages.debian.org/unstable/mmdebstrap/mmdebstrap.1.en.html The sales pitch in comparison to debootstrap -------------------------------------------- Summary: - more than one mirror possible - security and updates mirror included for Debian stable chroots - twice as fast - chroot with apt in 11 seconds - gzipped tarball with apt is 27M small - bit-by-bit reproducible output - unprivileged operation using Linux user namespaces or fakechroot - can operate on filesystems mounted with nodev - foreign architecture chroots with qemu-user - variant installing only Essential:yes packages and dependencies - temporary chroots by redirecting to /dev/null - chroots without apt inside (for chroot from buildinfo file with debootsnap) The author believes that a chroot of a Debian stable release should include the latest packages including security fixes by default. This has been a wontfix with debootstrap since 2009 (See #543819 and #762222). Since mmdebstrap uses apt internally, support for multiple mirrors comes for free and stable or oldstable **chroots will include security and updates mirrors**. A side-effect of using apt is being twice as fast as debootstrap. The timings were carried out on a laptop with an Intel Core i5-5200U, using a mirror on localhost and a tmpfs. | variant | mmdebstrap | debootstrap | | --------- | ---------- | ------------ | | essential | 9.52 s | n.a | | apt | 10.98 s | n.a | | minbase | 13.54 s | 26.37 s | | buildd | 21.31 s | 34.85 s | | - | 23.01 s | 48.83 s | Apt considers itself an `Essential: yes` package. This feature allows one to create a chroot containing just the `Essential: yes` packages and apt (and their hard dependencies) in **just 11 seconds**. If desired, a most minimal chroot with just the `Essential: yes` packages and their hard dependencies can be created with a gzipped tarball size of just 34M. By using dpkg's `--path-exclude` option to exclude documentation, even smaller gzipped tarballs of 21M in size are possible. If apt is included, the result is a **gzipped tarball of only 27M**. These small sizes are also achieved because apt caches and other cruft is stripped from the chroot. This also makes the result **bit-by-bit reproducible** if the `$SOURCE_DATE_EPOCH` environment variable is set. The author believes, that it should not be necessary to have superuser privileges to create a file (the chroot tarball) in one's home directory. Thus, mmdebstrap provides multiple options to create a chroot tarball with the right permissions **without superuser privileges**. This avoids a whole class of bugs like #921815. Depending on what is available, it uses either Linux user namespaces or fakechroot. Debootstrap supports fakechroot but will not create a tarball with the right permissions by itself. Support for Linux user namespaces is missing (see #829134). When creating a chroot tarball with debootstrap, the temporary chroot directory cannot be on a filesystem that has been mounted with nodev. In unprivileged mode, **mknod is never used**, which means that /tmp can be used as a temporary directory location even if if it's mounted with nodev as a security measure. If the chroot architecture cannot be executed by the current machine, qemu-user is used to allow one to create a **foreign architecture chroot**. Limitations in comparison to debootstrap ---------------------------------------- Debootstrap supports creating a Debian chroot on non-Debian systems but mmdebstrap requires apt and is thus limited to Debian and derivatives. This means that mmdebstrap can never fully replace debootstrap and debootstrap will continue to be relevant in situations where you want to create a Debian chroot from a platform without apt and dpkg. There is no `SCRIPT` argument. The following options, don't exist: `--second-stage`, `--exclude`, `--resolve-deps`, `--force-check-gpg`, `--merged-usr` and `--no-merged-usr`. The quirks from debootstrap are needed to create chroots of Debian unstable from snapshot.d.o before timestamp 20141107T220431Z or Debian 8 (Jessie) or later. Tests ===== The script `coverage.sh` runs mmdebstrap in all kind of scenarios to execute all code paths of the script. It verifies its output in each scenario and displays the results gathered with Devel::Cover. It also compares the output of mmdebstrap with debootstrap in several scenarios. To run the testsuite, run: ./make_mirror.sh CMD=./mmdebstrap ./coverage.sh To also generate perl Devel::Cover data, omit the `CMD` environment variable. But that will also take a lot longer. The `make_mirror.sh` script will be a no-op if nothing changed in Debian unstable. You don't need to run `make_mirror.sh` before every invocation of `coverage.sh`. When you make changes to `make_mirror.sh` and want to regenerate the cache, run: touch -d yesterday shared/cache/debian/dists/unstable/Release The script `coverage.sh` does not need an active internet connection by default. An online connection is only needed by the `make_mirror.sh` script which fills a local cache with a few minimal Debian mirror copies. By default, `coverage.sh` will skip running a single test which tries creating a Ubuntu Focal chroot. To not skip that test, run `coverage.sh` with the environment variable `ONLINE=yes`. If a test fails you can run individual tests by executing `coverage.py` with the test name and optionally limit it to a specific distribution like so: CMD=./mmdebstrap ./coverage.py --dist unstable check-against-debootstrap-dist Bugs ==== mmdebstrap has bugs. Report them here: https://gitlab.mister-muffin.de/josch/mmdebstrap/issues Contributors ============ - Johannes Schauer Marin Rodrigues (main author) - Jochen Sprickerhof - Helmut Grohne - Gioele Barabucci - Benjamin Drung - Josh Triplett - Konstantin Demin - Charles Short - Chris Hofstaedtler - Colin Watson - David Kalnischkies - Emilio Pozuelo Monfort - Francesco Poli - Jakub Wilk - Joe Groocock - Max-Julian Pogner - Nicolas Vigier - Raul Tambre - Steve Dodd - Trent W. Buck - Vagrant Cascadian mmdebstrap/caching_proxy.py000077500000000000000000000105271477517524100164500ustar00rootroot00000000000000#!/usr/bin/env python3 import sys import os import time import http.client import http.server from io import StringIO import pathlib import urllib.parse oldcachedir = None newcachedir = None readonly = False class ProxyRequestHandler(http.server.BaseHTTPRequestHandler): def do_GET(self): assert int(self.headers.get("Content-Length", 0)) == 0 assert self.headers["Host"] pathprefix = "http://" + self.headers["Host"] + "/" assert self.path.startswith(pathprefix) sanitizedpath = urllib.parse.unquote(self.path.removeprefix(pathprefix)) oldpath = oldcachedir / sanitizedpath newpath = newcachedir / sanitizedpath if not readonly: newpath.parent.mkdir(parents=True, exist_ok=True) # just send back to client if newpath.exists(): print(f"proxy cached: {self.path}", file=sys.stderr) self.wfile.write(b"HTTP/1.1 200 OK\r\n") self.send_header("Content-Length", newpath.stat().st_size) self.end_headers() with newpath.open(mode="rb") as new: while True: buf = new.read(64 * 1024) # same as shutil uses if not buf: break self.wfile.write(buf) self.wfile.flush() return if readonly: newpath = pathlib.Path("/dev/null") # copy from oldpath to newpath and send back to client # Only take files from the old cache if they are .deb files or Packages # files in the by-hash directory as only those are unique by their path # name. Other files like InRelease files have to be downloaded afresh. if oldpath.exists() and ( oldpath.suffix == ".deb" or "by-hash" in oldpath.parts ): print(f"proxy cached: {self.path}", file=sys.stderr) self.wfile.write(b"HTTP/1.1 200 OK\r\n") self.send_header("Content-Length", oldpath.stat().st_size) self.end_headers() with oldpath.open(mode="rb") as old, newpath.open(mode="wb") as new: # we are not using shutil.copyfileobj() because we want to # write to two file objects simultaneously while True: buf = old.read(64 * 1024) # same as shutil uses if not buf: break self.wfile.write(buf) new.write(buf) self.wfile.flush() return # download fresh copy try: print(f"\rproxy download: {self.path}", file=sys.stderr) conn = http.client.HTTPConnection(self.headers["Host"], timeout=5) conn.request("GET", self.path, None, dict(self.headers)) res = conn.getresponse() assert (res.status, res.reason) == (200, "OK"), (res.status, res.reason) self.wfile.write(b"HTTP/1.1 200 OK\r\n") for k, v in res.getheaders(): # do not allow a persistent connection if k == "connection": continue self.send_header(k, v) self.end_headers() with newpath.open(mode="wb") as f: # we are not using shutil.copyfileobj() because we want to # write to two file objects simultaneously and throttle the # writing speed to 1024 kB/s while True: buf = res.read(64 * 1024) # same as shutil uses if not buf: break self.wfile.write(buf) f.write(buf) time.sleep(64 / 1024) # 1024 kB/s self.wfile.flush() except Exception as e: self.send_error(502) def main(): global oldcachedir, newcachedir, readonly if sys.argv[1] == "--readonly": readonly = True oldcachedir = pathlib.Path(sys.argv[2]) newcachedir = pathlib.Path(sys.argv[3]) else: oldcachedir = pathlib.Path(sys.argv[1]) newcachedir = pathlib.Path(sys.argv[2]) print(f"starting caching proxy for {newcachedir}", file=sys.stderr) httpd = http.server.ThreadingHTTPServer( server_address=("", 8080), RequestHandlerClass=ProxyRequestHandler ) httpd.serve_forever() if __name__ == "__main__": main() mmdebstrap/coverage.py000077500000000000000000000401701477517524100154030ustar00rootroot00000000000000#!/usr/bin/env python3 from debian.deb822 import Deb822, Release import email.utils import os import sys import shutil import subprocess import argparse import time from datetime import timedelta from collections import defaultdict from itertools import product have_qemu = os.getenv("HAVE_QEMU", "yes") == "yes" have_binfmt = os.getenv("HAVE_BINFMT", "yes") == "yes" run_ma_same_tests = os.getenv("RUN_MA_SAME_TESTS", "yes") == "yes" use_host_apt_config = os.getenv("USE_HOST_APT_CONFIG", "no") == "yes" cmd = os.getenv("CMD", "./mmdebstrap") default_dist = os.getenv("DEFAULT_DIST", "unstable") all_dists = ["oldstable", "stable", "testing", "unstable"] default_mode = "auto" all_modes = ["auto", "root", "unshare", "fakechroot", "chrootless"] default_variant = "apt" all_variants = [ "extract", "custom", "essential", "apt", "minbase", "buildd", "-", "standard", ] default_format = "auto" all_formats = ["auto", "directory", "tar", "squashfs", "ext2", "ext4", "null"] mirror = os.getenv("mirror", "http://127.0.0.1/debian") hostarch = subprocess.check_output(["dpkg", "--print-architecture"]).decode().strip() release_path = f"./shared/cache/debian/dists/{default_dist}/InRelease" if not os.path.exists(release_path): print("path doesn't exist:", release_path, file=sys.stderr) print("run ./make_mirror.sh first", file=sys.stderr) exit(1) if os.getenv("SOURCE_DATE_EPOCH") is not None: s_d_e = os.getenv("SOURCE_DATE_EPOCH") else: with open(release_path) as f: rel = Release(f) s_d_e = str(email.utils.mktime_tz(email.utils.parsedate_tz(rel["Date"]))) separator = ( "------------------------------------------------------------------------------" ) def skip(condition, dist, mode, variant, fmt): if not condition: return "" for line in condition.splitlines(): if not line: continue if eval(line): return line.strip() return "" def parse_config(confname): config_dict = defaultdict(dict) config_order = list() all_vals = { "Dists": all_dists, "Modes": all_modes, "Variants": all_variants, "Formats": all_formats, } with open(confname) as f: for test in Deb822.iter_paragraphs(f): if "Test" not in test.keys(): print("Test without name", file=sys.stderr) exit(1) name = test["Test"] config_order.append(name) for k in test.keys(): v = test[k] if k not in [ "Test", "Dists", "Modes", "Variants", "Formats", "Skip-If", "Needs-QEMU", "Needs-Root", "Needs-APT-Config", ]: print(f"Unknown field name {k} in test {name}") exit(1) if k in all_vals.keys(): if v == "default": print( f"Setting {k} to default in Test {name} is redundant", file=sys.stderr, ) exit(1) if v == "any": v = all_vals[k] else: # else, split the value by whitespace v = v.split() for i in v: if i not in all_vals[k]: print( f"{i} is not a valid value for {k}", file=sys.stderr ) exit(1) config_dict[name][k] = v return config_order, config_dict def format_test(num, total, name, dist, mode, variant, fmt, config_dict): ret = f"({num}/{total}) {name}" if len(config_dict[name].get("Dists", [])) > 1: ret += f" --dist={dist}" if len(config_dict[name].get("Modes", [])) > 1: ret += f" --mode={mode}" if len(config_dict[name].get("Variants", [])) > 1: ret += f" --variant={variant}" if len(config_dict[name].get("Formats", [])) > 1: ret += f" --format={fmt}" return ret def print_time_per_test(time_per_test, name="test"): print( f"average time per {name}:", sum(time_per_test.values(), start=timedelta()) / len(time_per_test), file=sys.stderr, ) print( f"median time per {name}:", sorted(time_per_test.values())[len(time_per_test) // 2], file=sys.stderr, ) head_tail_num = 10 print(f"{head_tail_num} fastests {name}s:", file=sys.stderr) for k, v in sorted(time_per_test.items(), key=lambda i: i[1])[ : min(head_tail_num, len(time_per_test)) ]: print(f" {k}: {v}", file=sys.stderr) print(f"{head_tail_num} slowest {name}s:", file=sys.stderr) for k, v in sorted(time_per_test.items(), key=lambda i: i[1], reverse=True)[ : min(head_tail_num, len(time_per_test)) ]: print(f" {k}: {v}", file=sys.stderr) def main(): parser = argparse.ArgumentParser() parser.add_argument("test", nargs="*", help="only run these tests") parser.add_argument( "-x", "--exitfirst", action="store_const", dest="maxfail", const=1, help="exit instantly on first error or failed test.", ) parser.add_argument( "--maxfail", metavar="num", action="store", type=int, dest="maxfail", default=0, help="exit after first num failures or errors.", ) parser.add_argument( "--mode", metavar="mode", help=f"only run tests with this mode (Default = {default_mode})", ) parser.add_argument( "--dist", metavar="dist", help=f"only run tests with this dist (Default = {default_dist})", ) parser.add_argument( "--variant", metavar="variant", help=f"only run tests with this variant (Default = {default_variant})", ) parser.add_argument( "--format", metavar="format", help=f"only run tests with this format (Default = {default_format})", ) parser.add_argument( "--skip", metavar="test", action="append", help="skip this test" ) args = parser.parse_args() # copy over files from git or as distributed for git, dist, target in [ ("./mmdebstrap", "/usr/bin/mmdebstrap", "mmdebstrap"), ("./tarfilter", "/usr/bin/mmtarfilter", "tarfilter"), ( "./proxysolver", "/usr/lib/apt/solvers/mmdebstrap-dump-solution", "proxysolver", ), ( "./ldconfig.fakechroot", "/usr/libexec/mmdebstrap/ldconfig.fakechroot", "ldconfig.fakechroot", ), ]: if os.path.exists(git): shutil.copy(git, f"shared/{target}") else: shutil.copy(dist, f"shared/{target}") # copy over hooks from git or as distributed if os.path.exists("hooks"): shutil.copytree("hooks", "shared/hooks", dirs_exist_ok=True) else: shutil.copytree( "/usr/share/mmdebstrap/hooks", "shared/hooks", dirs_exist_ok=True ) # parse coverage.txt config_order, config_dict = parse_config("coverage.txt") indirbutnotcovered = set( [d for d in os.listdir("tests") if not d.startswith(".")] ) - set(config_order) if indirbutnotcovered: print( "test(s) missing from coverage.txt: %s" % (", ".join(sorted(indirbutnotcovered))), file=sys.stderr, ) exit(1) coveredbutnotindir = set(config_order) - set( [d for d in os.listdir("tests") if not d.startswith(".")] ) if coveredbutnotindir: print( "test(s) missing from ./tests: %s" % (", ".join(sorted(coveredbutnotindir))), file=sys.stderr, ) exit(1) # produce the list of tests using the cartesian product of all allowed # dists, modes, variants and formats of a given test tests = [] for name in config_order: test = config_dict[name] for dist, mode, variant, fmt in product( test.get("Dists", [default_dist]), test.get("Modes", [default_mode]), test.get("Variants", [default_variant]), test.get("Formats", [default_format]), ): skipreason = skip(test.get("Skip-If"), dist, mode, variant, fmt) if skipreason: tt = ("skip", skipreason) elif ( test.get("Needs-APT-Config", "false") == "true" and use_host_apt_config ): tt = ("skip", "test cannot use host apt config") elif have_qemu: tt = "qemu" elif test.get("Needs-QEMU", "false") == "true": tt = ("skip", "test needs QEMU") elif test.get("Needs-Root", "false") == "true": tt = "sudo" elif mode == "root": tt = "sudo" else: tt = "null" tests.append((tt, name, dist, mode, variant, fmt)) torun = [] num_tests = len(tests) if args.test: # check if all given tests are either a valid name or a valid number for test in args.test: if test in [name for (_, name, _, _, _, _) in tests]: continue if not test.isdigit(): print(f"cannot find test named {test}", file=sys.stderr) exit(1) if int(test) >= len(tests) or int(test) <= 0 or str(int(test)) != test: print(f"test number {test} doesn't exist", file=sys.stderr) exit(1) for i, (_, name, _, _, _, _) in enumerate(tests): # if either the number or test name matches, then we use this test, # otherwise we skip it if name in args.test: torun.append(i) if str(i + 1) in args.test: torun.append(i) num_tests = len(torun) starttime = time.time() skipped = defaultdict(list) failed = [] num_success = 0 num_finished = 0 time_per_test = {} acc_time_per_test = defaultdict(list) for i, (test, name, dist, mode, variant, fmt) in enumerate(tests): if torun and i not in torun: continue print(separator, file=sys.stderr) print("(%d/%d) %s" % (i + 1, len(tests), name), file=sys.stderr) print("dist: %s" % dist, file=sys.stderr) print("mode: %s" % mode, file=sys.stderr) print("variant: %s" % variant, file=sys.stderr) print("format: %s" % fmt, file=sys.stderr) if num_finished > 0: currenttime = time.time() timeleft = timedelta( seconds=int( (num_tests - num_finished) * (currenttime - starttime) / num_finished ) ) print("time left: %s" % timeleft, file=sys.stderr) if failed: print("failed: %d" % len(failed), file=sys.stderr) num_finished += 1 with open("tests/" + name) as fin, open("shared/test.sh", "w") as fout: for line in fin: line = line.replace("{{ CMD }}", cmd) line = line.replace("{{ SOURCE_DATE_EPOCH }}", s_d_e) line = line.replace("{{ DIST }}", dist) line = line.replace("{{ MIRROR }}", mirror) line = line.replace("{{ MODE }}", mode) line = line.replace("{{ VARIANT }}", variant) line = line.replace("{{ FORMAT }}", fmt) line = line.replace("{{ HOSTARCH }}", hostarch) fout.write(line) # ignore: # SC2016 Expressions don't expand in single quotes, use double quotes for that. # SC2050 This expression is constant. Did you forget the $ on a variable? # SC2194 This word is constant. Did you forget the $ on a variable? shellcheck = subprocess.run( [ "shellcheck", "--exclude=SC2050,SC2194,SC2016", "-f", "gcc", "shared/test.sh", ], check=False, stdout=subprocess.PIPE, ).stdout.decode() shfmt = subprocess.run( [ "shfmt", "--posix", "--binary-next-line", "--case-indent", "--indent", "2", "--simplify", "-d", "shared/test.sh", ], check=False, stdout=subprocess.PIPE, ).stdout.decode() argv = None match test: case "qemu": argv = ["./run_qemu.sh"] case "sudo": argv = ["./run_null.sh", "SUDO"] case "null": argv = ["./run_null.sh"] case ("skip", reason): skipped[reason].append( format_test( i + 1, len(tests), name, dist, mode, variant, fmt, config_dict ) ) print(f"skipped because of {reason}", file=sys.stderr) continue print(separator, file=sys.stderr) if args.skip and name in args.skip: print(f"skipping because of --skip={name}", file=sys.stderr) continue if args.dist and args.dist != dist: print(f"skipping because of --dist={args.dist}", file=sys.stderr) continue if args.mode and args.mode != mode: print(f"skipping because of --mode={args.mode}", file=sys.stderr) continue if args.variant and args.variant != variant: print(f"skipping because of --variant={args.variant}", file=sys.stderr) continue if args.format and args.format != fmt: print(f"skipping because of --format={args.format}", file=sys.stderr) continue before = time.time() proc = subprocess.Popen(argv) try: proc.wait() except KeyboardInterrupt: proc.terminate() proc.wait() break after = time.time() walltime = timedelta(seconds=int(after - before)) formated_test_name = format_test( i + 1, len(tests), name, dist, mode, variant, fmt, config_dict ) time_per_test[formated_test_name] = walltime acc_time_per_test[name].append(walltime) print(separator, file=sys.stderr) print(f"duration: {walltime}", file=sys.stderr) if proc.returncode != 0 or shellcheck != "" or shfmt != "": if shellcheck != "": print(shellcheck) if shfmt != "": print(shfmt) failed.append(formated_test_name) print("result: FAILURE", file=sys.stderr) else: print("result: SUCCESS", file=sys.stderr) num_success += 1 if args.maxfail and len(failed) >= args.maxfail: break print(separator, file=sys.stderr) print( "successfully ran %d tests" % num_success, file=sys.stderr, ) if skipped: print("skipped %d:" % sum([len(v) for v in skipped.values()]), file=sys.stderr) for reason, l in skipped.items(): print(f"skipped because of {reason}:", file=sys.stderr) for t in l: print(f" {t}", file=sys.stderr) if len(time_per_test) > 1: print_time_per_test(time_per_test) if len(acc_time_per_test) > 1: print_time_per_test( { f"{len(v)}x {k}": sum(v, start=timedelta()) for k, v in acc_time_per_test.items() }, "accumulated test", ) if failed: print("failed %d:" % len(failed), file=sys.stderr) for f in failed: print(f, file=sys.stderr) currenttime = time.time() walltime = timedelta(seconds=int(currenttime - starttime)) print(f"total runtime: {walltime}", file=sys.stderr) if failed: exit(1) if __name__ == "__main__": main() mmdebstrap/coverage.sh000077500000000000000000000072631477517524100153730ustar00rootroot00000000000000#!/bin/sh set -eu # by default, use the mmdebstrap executable in the current directory together # with perl Devel::Cover but allow to overwrite this : "${CMD:=perl -MDevel::Cover=-silent,-nogcov ./mmdebstrap}" case "$CMD" in "mmdebstrap "* | mmdebstrap | *" mmdebstrap" | *" mmdebstrap "*) MMSCRIPT="$(command -v mmdebstrap 2>/dev/null)" ;; *) MMSCRIPT=./mmdebstrap ;; esac if [ -e "$MMSCRIPT" ]; then TMPFILE=$(mktemp) perltidy <"$MMSCRIPT" >"$TMPFILE" ret=0 diff -u "$MMSCRIPT" "$TMPFILE" || ret=$? if [ "$ret" -ne 0 ]; then echo "perltidy failed" >&2 rm "$TMPFILE" exit 1 fi rm "$TMPFILE" if [ "$(sed -e '/^__END__$/,$d' "$MMSCRIPT" | wc --max-line-length)" -gt 79 ]; then echo "$MMSCRIPT exceeded maximum line length of 79 characters" >&2 exit 1 fi perlcritic --severity 4 --verbose 8 "$MMSCRIPT" pod2man "$MMSCRIPT" >/dev/null fi for f in tarfilter coverage.py caching_proxy.py; do [ -e "./$f" ] || continue black --check "./$f" done shellcheck --exclude=SC2016 coverage.sh make_mirror.sh run_null.sh run_qemu.sh gpgvnoexpkeysig mmdebstrap-autopkgtest-build-qemu hooks/*/*.sh shfmt --binary-next-line --case-indent --indent 2 --simplify -d coverage.sh make_mirror.sh run_null.sh run_qemu.sh mmdebstrap-autopkgtest-build-qemu gpgvnoexpkeysig mirrordir="./shared/cache/debian" if [ ! -e "$mirrordir" ]; then echo "run ./make_mirror.sh before running $0" >&2 exit 1 fi # we use -f because the file might not exist rm -f shared/cover_db.img : "${DEFAULT_DIST:=unstable}" : "${HAVE_QEMU:=yes}" : "${RUN_MA_SAME_TESTS:=yes}" if [ "$HAVE_QEMU" = "yes" ]; then # prepare image for cover_db fallocate -l 64M shared/cover_db.img /usr/sbin/mkfs.vfat shared/cover_db.img if [ ! -e "./shared/cache/debian-$DEFAULT_DIST.ext4" ]; then echo "./shared/cache/debian-$DEFAULT_DIST.ext4 does not exist" >&2 exit 1 fi fi # choose the timestamp of the unstable Release file, so that we get # reproducible results for the same mirror timestamp SOURCE_DATE_EPOCH=$(date --date="$(grep-dctrl -s Date -n '' "$mirrordir/dists/$DEFAULT_DIST/Release")" +%s) # for traditional sort order that uses native byte values export LC_ALL=C.UTF-8 : "${HAVE_BINFMT:=yes}" mirror="http://127.0.0.1/debian" export HAVE_QEMU HAVE_BINFMT RUN_MA_SAME_TESTS DEFAULT_DIST SOURCE_DATE_EPOCH CMD mirror ./coverage.py "$@" if [ -e shared/cover_db.img ]; then # produce report inside the VM to make sure that the versions match or # otherwise we might get: # Can't read shared/cover_db/runs/1598213854.252.64287/cover.14 with Sereal: Sereal: Error: Bad Sereal header: Not a valid Sereal document. at offset 1 of input at srl_decoder.c line 600 at /usr/lib/x86_64-linux-gnu/perl5/5.30/Devel/Cover/DB/IO/Sereal.pm line 34, <$fh> chunk 1. cat <shared/test.sh cover -nogcov -report html_basic cover_db >&2 mkdir -p report for f in common.js coverage.html cover.css css.js mmdebstrap--branch.html mmdebstrap--condition.html mmdebstrap.html mmdebstrap--subroutine.html standardista-table-sorting.js; do cp -a cover_db/\$f report done cover -delete cover_db >&2 END if [ "$HAVE_QEMU" = "yes" ]; then ./run_qemu.sh else ./run_null.sh fi echo echo "open file://$(pwd)/shared/report/coverage.html in a browser" echo fi # check if the wiki has to be updated with pod2markdown output if [ "${DEBEMAIL-}" = "josch@debian.org" ]; then bash -exc "diff -u <(curl --silent https://gitlab.mister-muffin.de/josch/mmdebstrap/wiki/raw/Home | dos2unix; echo) <(pod2markdown < mmdebstrap)" || : fi rm -f shared/test.sh shared/tar1.txt shared/tar2.txt shared/pkglist.txt shared/doc-debian.tar.list shared/mmdebstrap shared/tarfilter shared/proxysolver echo "$0 finished successfully" >&2 mmdebstrap/coverage.txt000066400000000000000000000205151477517524100155700ustar00rootroot00000000000000Test: debootstrap Dists: any Variants: minbase buildd - Needs-Root: true Needs-APT-Config: true Test: check-against-debootstrap-dist Dists: any Variants: minbase buildd - Needs-Root: true Needs-APT-Config: true Test: as-debootstrap-unshare-wrapper Modes: unshare Needs-Root: true Variants: minbase - Needs-APT-Config: true Test: help Test: man Test: version Test: create-directory Needs-Root: true Test: unshare-as-root-user Needs-Root: true Test: dist-using-codename Dists: any Needs-APT-Config: true Test: fail-without-etc-subuid Needs-QEMU: true Test: fail-without-username-in-etc-subuid Needs-QEMU: true Test: unshare-as-root-user-inside-chroot Needs-Root: true Needs-APT-Config: true Test: root-mode-inside-chroot Needs-Root: true Needs-APT-Config: true Test: root-mode-inside-unshare-chroot Modes: unshare Needs-APT-Config: true Test: root-without-cap-sys-admin Needs-Root: true Test: mount-is-missing Needs-QEMU: true Test: mmdebstrap Needs-Root: true Modes: root Formats: tar squashfs ext2 ext4 Variants: essential apt minbase buildd - standard Skip-If: variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558 mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276 Test: check-for-bit-by-bit-identical-format-output Modes: unshare fakechroot Formats: tar squashfs ext2 ext4 Variants: essential apt minbase buildd - standard Skip-If: variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558 mode == "fakechroot" and variant in ["-", "standard"] # no extended attributes variant == "standard" and dist in ["oldstable", "stable"] and hostarch in ["armel", "armhf", "mipsel"] # #1031276 Test: tarfilter-idshift Needs-QEMU: true Test: progress-bars-on-fake-tty Test: debug-output-on-fake-tty Test: existing-empty-directory Needs-Root: true Test: existing-directory-with-lost-found Needs-Root: true Test: fail-installing-to-non-empty-lost-found Test: fail-installing-to-non-empty-target-directory Test: missing-device-nodes-outside-the-chroot Needs-QEMU: true Test: missing-dev-sys-proc-inside-the-chroot Modes: unshare Variants: custom Test: chroot-directory-not-accessible-by-apt-user Needs-Root: true Test: cwd-directory-not-accessible-by-unshared-user Needs-Root: true Modes: unshare Test: create-gzip-compressed-tarball Test: custom-tmpdir Needs-Root: true Modes: unshare Test: xz-compressed-tarball Test: directory-ending-in-tar Modes: root Needs-Root: true Test: auto-mode-without-unshare-capabilities Needs-QEMU: true Test: fail-with-missing-lz4 Test: fail-with-path-with-quotes Test: create-tarball-with-tmp-mounted-nodev Needs-QEMU: true Test: read-from-stdin-write-to-stdout Test: supply-components-manually Modes: root Needs-Root: true Needs-APT-Config: true Test: stable-default-mirror Needs-QEMU: true Test: pass-distribution-but-implicitly-write-to-stdout Needs-QEMU: true Test: aspcud-apt-solver Test: mirror-is-stdin Test: copy-mirror Needs-QEMU: true Test: file-mirror Needs-QEMU: true Test: file-mirror-automount-hook Modes: root unshare fakechroot Needs-QEMU: true Test: mirror-is-deb Test: mirror-is-real-file Needs-APT-Config: true Test: deb822-1-2 Modes: root Needs-Root: true Needs-APT-Config: true Test: deb822-2-2 Modes: root Needs-Root: true Needs-APT-Config: true Test: automatic-mirror-from-suite Needs-QEMU: true Test: invalid-mirror Needs-APT-Config: true Test: fail-installing-to-root Modes: root Needs-Root: true Test: fail-installing-to-existing-file Modes: root Needs-Root: true Test: arm64-without-qemu-support Needs-QEMU: true Skip-If: hostarch != "amd64" Test: i386-which-can-be-executed-without-qemu Needs-QEMU: true Skip-If: hostarch != "amd64" not run_ma_same_tests Test: include-foreign-libmagic-mgc Needs-Root: true Needs-APT-Config: true Skip-If: hostarch not in ["amd64", "arm64"] not run_ma_same_tests Test: include-foreign-libmagic-mgc-with-multiple-arch-options Needs-Root: true Needs-APT-Config: true Skip-If: hostarch not in ["amd64", "arm64"] not run_ma_same_tests Test: aptopt Needs-Root: true Test: keyring Needs-QEMU: true Test: keyring-overwrites Needs-Root: true Needs-APT-Config: true Test: signed-by-without-host-keys Needs-QEMU: true Test: ascii-armored-keys Needs-QEMU: true Test: signed-by-with-host-keys Needs-Root: true Needs-APT-Config: true Test: dpkgopt Needs-Root: true Test: include Needs-Root: true Test: multiple-include Needs-Root: true Test: include-with-multiple-apt-sources Needs-Root: true Test: essential-hook Needs-Root: true Test: customize-hook Needs-Root: true Test: failing-customize-hook Needs-Root: true Test: sigint-during-customize-hook Needs-Root: true Test: hook-directory Needs-Root: true Test: eatmydata-via-hook-dir Needs-Root: true Test: special-hooks-using-helpers Needs-Root: true Needs-APT-Config: true Test: special-hooks-using-helpers-and-env-vars Needs-Root: true Needs-APT-Config: true Test: special-hooks-with-mode-mode Modes: root unshare fakechroot Test: debootstrap-no-op-options Needs-Root: true Test: verbose Variants: standard Skip-If: variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558 Test: debug Variants: standard Skip-If: variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558 Test: quiet Needs-Root: true Test: logfile Needs-Root: true Needs-APT-Config: true Test: without-etc-resolv-conf-and-etc-hostname Needs-QEMU: true Test: preserve-mode-of-etc-resolv-conf-and-etc-hostname Modes: root Needs-QEMU: true Test: not-having-to-install-apt-in-include-because-a-hook-did-it-before Test: remove-start-stop-daemon-and-policy-rc-d-in-hook Test: skip-start-stop-daemon-policy-rc Test: skip-mount Modes: unshare Test: compare-output-with-pre-seeded-var-cache-apt-archives Needs-QEMU: true Variants: any Skip-If: variant == "standard" and dist == "oldstable" # #864082, #1004557, #1004558 Test: create-directory-dry-run Modes: root Test: create-tarball-dry-run Variants: any Modes: any Test: unpack-doc-debian Modes: root fakechroot Variants: extract Needs-APT-Config: true Test: install-doc-debian Modes: chrootless Variants: custom Needs-APT-Config: true Test: chrootless Variants: essential Modes: chrootless Needs-Root: true Skip-If: dist == "oldstable" Test: chrootless-fakeroot Variants: essential Modes: chrootless Skip-If: dist == "oldstable" hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286 Test: chrootless-foreign Variants: essential Modes: chrootless Skip-If: dist == "oldstable" hostarch not in ["amd64", "arm64"] not run_ma_same_tests Needs-QEMU: true Test: install-doc-debian-and-output-tarball Variants: custom Modes: chrootless Needs-APT-Config: true Test: install-doc-debian-and-test-hooks Variants: custom Modes: chrootless Needs-APT-Config: true Test: install-libmagic-mgc-on-foreign Variants: custom Modes: chrootless Skip-If: hostarch not in ["amd64", "arm64"] not have_binfmt Test: install-busybox-based-sub-essential-system Needs-Root: true Test: create-foreign-tarball Modes: root unshare fakechroot Skip-If: hostarch not in ["amd64", "arm64"] mode == "fakechroot" and not run_ma_same_tests mode == "fakechroot" and hostarch == "arm64" # usrmerge postinst under fakechroot wants to copy /lib/ld-linux-x86-64.so.2 (which does not exist) instead of /lib64/ld-linux-x86-64.so.2 not have_binfmt Test: no-sbin-in-path Modes: fakechroot Test: dev-ptmx Modes: root unshare Test: error-if-stdout-is-tty Test: variant-custom-timeout Test: include-deb-file Modes: root unshare fakechroot Needs-APT-Config: true Test: unshare-include-deb Modes: unshare Test: pivot_root Modes: root unshare Needs-APT-Config: true Test: jessie-or-older Needs-Root: true Modes: root unshare fakechroot Variants: essential apt minbase Skip-If: mode == "fakechroot" and hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286 Test: apt-patterns Test: apt-patterns-custom Test: empty-sources.list Test: merged-fakechroot-inside-unmerged-chroot Needs-Root: true Needs-APT-Config: true Skip-If: hostarch in ["i386", "armel", "armhf", "mipsel"] # #1023286 dist in ["testing", "unstable"] # #1053671 Test: auto-mode-as-normal-user Modes: auto Test: skip-output-dev Modes: root unshare Test: skip-output-mknod Modes: root unshare Test: skip-tar-in-mknod Modes: unshare Test: zombie-reaping Modes: unshare Test: empty-suite Needs-APT-Config: true mmdebstrap/examples/000077500000000000000000000000001477517524100150475ustar00rootroot00000000000000mmdebstrap/examples/twb/000077500000000000000000000000001477517524100156435ustar00rootroot00000000000000mmdebstrap/examples/twb/debian-11-minimal.py000066400000000000000000000123621477517524100213060ustar00rootroot00000000000000#!/usr/bin/python3 import argparse import pathlib import subprocess import tempfile import pathlib __author__ = "Trent W. Buck" __copyright__ = "Copyright © 2020 Trent W. Buck" __license__ = "expat" __doc__ = """ build the simplest Debian Live image that can boot This uses mmdebstrap to do the heavy lifting; it can run entirely without root privileges. It emits a USB key disk image that contains a bootable EFI ESP, which in turn includes a bootloader (refind), kernel, ramdisk, and filesystem.squashfs. NOTE: this is the simplest config possible. It lacks CRITICAL SECURITY AND DATA LOSS packages, such as amd64-microcode and smartd. """ parser = argparse.ArgumentParser(description=__doc__) parser.add_argument( "output_file", nargs="?", default=pathlib.Path("filesystem.img"), type=pathlib.Path ) args = parser.parse_args() filesystem_img_size = "256M" # big enough to include filesystem.squashfs + about 64M of bootloader, kernel, and ramdisk. esp_offset = 1024 * 1024 # 1MiB esp_label = "UEFI-ESP" # max 8 bytes for FAT32 live_media_path = "debian-live" with tempfile.TemporaryDirectory(prefix="debian-live-bullseye-amd64-minimal.") as td: td = pathlib.Path(td) subprocess.check_call( [ "mmdebstrap", "--mode=unshare", "--variant=apt", '--aptopt=Acquire::http::Proxy "http://apt-cacher-ng.cyber.com.au:3142"', '--aptopt=Acquire::https::Proxy "DIRECT"', "--dpkgopt=force-unsafe-io", "--include=linux-image-amd64 init initramfs-tools live-boot netbase", "--include=dbus", # https://bugs.debian.org/814758 "--include=live-config iproute2 keyboard-configuration locales sudo user-setup", "--include=ifupdown isc-dhcp-client", # live-config doesn't support systemd-networkd yet. # Do the **BARE MINIMUM** to make a USB key that can boot on X86_64 UEFI. # We use mtools so we do not ever need root privileges. # We can't use mkfs.vfat, as that needs kpartx or losetup (i.e. root). # We can't use mkfs.udf, as that needs mount (i.e. root). # We can't use "refind-install --usedefault" as that runs mount(8) (i.e. root). # We don't use genisoimage because # 1) ISO9660 must die; # 2) incomplete UDF 1.5+ support; # 3) resulting filesystem can't be tweaked after flashing (e.g. debian-live/site.dir/etc/systemd/network/up.network). # # We use refind because 1) I hate grub; and 2) I like refind. # If you want aarch64 or ia32 you need to install their BOOTxxx.EFI files. # If you want kernel+initrd on something other than FAT, you need refind/drivers_xxx/xxx_xxx.EFI. # # FIXME: with qemu in UEFI mode (OVMF), I get dumped into startup.nsh (UEFI REPL). # From there, I can manually type in "FS0:\EFI\BOOT\BOOTX64.EFI" to start refind, tho. # So WTF is its problem? Does it not support fallback bootloader? "--include=refind parted mtools", "--essential-hook=echo refind refind/install_to_esp boolean false | chroot $1 debconf-set-selections", "--customize-hook=echo refind refind/install_to_esp boolean true | chroot $1 debconf-set-selections", "--customize-hook=chroot $1 mkdir -p /boot/USB /boot/EFI/BOOT", "--customize-hook=chroot $1 cp /usr/share/refind/refind/refind_x64.efi /boot/EFI/BOOT/BOOTX64.EFI", f"--customize-hook=chroot $1 truncate --size={filesystem_img_size} /boot/USB/filesystem.img", f"--customize-hook=chroot $1 parted --script --align=optimal /boot/USB/filesystem.img mklabel gpt mkpart {esp_label} {esp_offset}b 100% set 1 esp on", f"--customize-hook=chroot $1 mformat -i /boot/USB/filesystem.img@@{esp_offset} -F -v {esp_label}", f"--customize-hook=chroot $1 mmd -i /boot/USB/filesystem.img@@{esp_offset} ::{live_media_path}", f"""--customize-hook=echo '"Boot with default options" "boot=live live-media-path={live_media_path}"' >$1/boot/refind_linux.conf""", # NOTE: find sidesteps the "glob expands before chroot applies" problem. f"""--customize-hook=chroot $1 find -O3 /boot/ -xdev -mindepth 1 -maxdepth 1 -regextype posix-egrep -iregex '.*/(EFI|refind_linux.conf|vmlinuz.*|initrd.img.*)' -exec mcopy -vsbpm -i /boot/USB/filesystem.img@@{esp_offset} {{}} :: ';'""", # FIXME: copy-out doesn't handle sparseness, so is REALLY slow (about 50 seconds). # Therefore instead leave it in the squashfs, and extract it later. # f'--customize-hook=copy-out /boot/USB/filesystem.img /tmp/', # f'--customize-hook=chroot $1 rm /boot/USB/filesystem.img', "bullseye", td / "filesystem.squashfs", ] ) with args.output_file.open("wb") as f: subprocess.check_call( ["rdsquashfs", "--cat=boot/USB/filesystem.img", td / "filesystem.squashfs"], stdout=f, ) subprocess.check_call( [ "mcopy", "-i", f"{args.output_file}@@{esp_offset}", td / "filesystem.squashfs", f"::{live_media_path}/filesystem.squashfs", ] ) mmdebstrap/examples/twb/debian-sid-zfs.py000066400000000000000000000334731477517524100210260ustar00rootroot00000000000000#!/usr/bin/python3 import argparse import pathlib import subprocess import tempfile import pathlib __author__ = "Trent W. Buck" __copyright__ = "Copyright © 2020 Trent W. Buck" __license__ = "expat" __doc__ = """ build a Debian Live image that can install Debian 11 on ZFS 2 This uses mmdebstrap to do the heavy lifting; it can run entirely without root privileges. It emits a USB key disk image that contains a bootable EFI ESP, which in turn includes a bootloader (refind), kernel, ramdisk, and filesystem.squashfs. """ parser = argparse.ArgumentParser(description=__doc__) parser.add_argument( "output_file", nargs="?", default=pathlib.Path("filesystem.img"), type=pathlib.Path ) parser.add_argument( "--timezone", default="Australia/Melbourne", type=lambda s: s.split("/"), help='NOTE: MUST be "Area/Zone" not e.g. "UTC", for now', ) parser.add_argument( "--locale", default="en_AU.UTF-8", help='NOTE: MUST end in ".UTF-8", for now' ) args = parser.parse_args() filesystem_img_size = "512M" # big enough to include filesystem.squashfs + about 64M of bootloader, kernel, and ramdisk. esp_offset = 1024 * 1024 # 1MiB esp_label = "UEFI-ESP" # max 8 bytes for FAT32 live_media_path = "debian-live" with tempfile.TemporaryDirectory(prefix="debian-sid-zfs.") as td: td = pathlib.Path(td) subprocess.check_call( [ "mmdebstrap", "--mode=unshare", "--variant=apt", '--aptopt=Acquire::http::Proxy "http://apt-cacher-ng.cyber.com.au:3142"', '--aptopt=Acquire::https::Proxy "DIRECT"', "--dpkgopt=force-unsafe-io", "--components=main contrib non-free", # needed for CPU security patches "--include=init initramfs-tools xz-utils live-boot netbase", "--include=dbus", # https://bugs.debian.org/814758 "--include=linux-image-amd64 firmware-linux", # Have ZFS 2.0 support. "--include=zfs-dkms zfsutils-linux zfs-zed build-essential linux-headers-amd64", # ZFS 2 support # Make the initrd a little smaller (41MB -> 20MB), at the expensive of significantly slower image build time. "--include=zstd", "--essential-hook=mkdir -p $1/etc/initramfs-tools/conf.d", "--essential-hook=>$1/etc/initramfs-tools/conf.d/zstd echo COMPRESS=zstd", # Be the equivalent of Debian Live GNOME # '--include=live-task-gnome', #'--include=live-task-xfce', # FIXME: enable this? It makes live-task-xfce go from 1G to 16G... so no. #'--aptopt=Apt::Install-Recommends "true"', # ...cherry-pick instead # UPDATE: debian-installer-launcher DOES NOT WORK because we don't load crap SPECIFICALLY into /live/installer, in the ESP. # UPDATE: network-manager-gnome DOES NOT WORK, nor is systemd-networkd auto-started... WTF? # end result is no networking. #'--include=live-config user-setup sudo firmware-linux haveged', #'--include=calamares-settings-debian udisks2', # 300MB weirdo Qt GUI debian installer #'--include=xfce4-terminal', # x86_64 CPUs are undocumented proprietary RISC chips that EMULATE a documented x86_64 CISC ISA. # The emulator is called "microcode", and is full of security vulnerabilities. # Make sure security patches for microcode for *ALL* CPUs are included. # By default, it tries to auto-detect the running CPU, so only patches the CPU of the build server. "--include=intel-microcode amd64-microcode iucode-tool", "--essential-hook=>$1/etc/default/intel-microcode echo IUCODE_TOOL_INITRAMFS=yes IUCODE_TOOL_SCANCPUS=no", "--essential-hook=>$1/etc/default/amd64-microcode echo AMD64UCODE_INITRAMFS=yes", "--dpkgopt=force-confold", # Work around https://bugs.debian.org/981004 # DHCP/DNS/SNTP clients... # FIXME: use live-config ? "--include=libnss-resolve libnss-myhostname systemd-timesyncd", "--customize-hook=chroot $1 cp -alf /lib/systemd/resolv.conf /etc/resolv.conf", # This probably needs to happen LAST # FIXME: fix resolv.conf to point to resolved, not "copy from the build-time OS" # FIXME: fix hostname & hosts to not exist, not "copy from the build-time OS" "--customize-hook=systemctl --root=$1 enable systemd-networkd systemd-timesyncd", # is this needed? # Run a DHCP client on *ALL* ifaces. # Consider network "up" (start sshd and local login prompt) when *ANY* (not ALL) ifaces are up. "--customize-hook=>$1/etc/systemd/network/up.network printf '%s\n' '[Match]' Name='en*' '[Network]' DHCP=yes", # try DHCP on all ethernet ifaces "--customize-hook=mkdir $1/etc/systemd/system/systemd-networkd-wait-online.service.d", "--customize-hook=>$1/etc/systemd/system/systemd-networkd-wait-online.service.d/any-not-all.conf printf '%s\n' '[Service]' 'ExecStart=' 'ExecStart=/lib/systemd/systemd-networkd-wait-online --any'", # Hope there's a central smarthost SMTP server called "mail" in the local search domain. # FIXME: can live-config do this? "--include=msmtp-mta", "--customize-hook=>$1/etc/msmtprc printf '%s\n' 'account default' 'syslog LOG_MAIL' 'host mail' 'auto_from on'", # Hope there's a central RELP logserver called "logserv" in the local domain. # FIXME: can live-config do this? "--include=rsyslog-relp", """--customize-hook=>$1/etc/rsyslog.conf printf '%s\n' 'module(load="imuxsock")' 'module(load="imklog")' 'module(load="omrelp")' 'action(type="omrelp" target="logserv" port="2514" template="RSYSLOG_SyslogProtocol23Format")'""", # Run self-tests on all discoverable hard disks, and (try to) email if something goes wrong. "--include=smartmontools bsd-mailx", "--customize-hook=>$1/etc/smartd.conf echo 'DEVICESCAN -n standby,15 -a -o on -S on -s (S/../../7/00|L/../01/./01) -t -H -m root -M once'", # For rarely-updated, rarely-rebooted SOEs, apply what security updates we can into transient tmpfs COW. # This CANNOT apply kernel security updates (though it will download them). # This CANNOT make the upgrades persistent across reboots (they re-download each boot). # FIXME: Would it be cleaner to set Environment=NEEDRESTART_MODE=a in # apt-daily-upgrade.service and/or # unattended-upgrades.service, so # needrestart is noninteractive only when apt is noninteractive? "--include=unattended-upgrades needrestart", "--customize-hook=echo 'unattended-upgrades unattended-upgrades/enable_auto_updates boolean true' | chroot $1 debconf-set-selections", """--customize-hook=>$1/etc/needrestart/conf.d/unattended-needrestart.conf echo '$nrconf{restart} = "a";'""", # https://bugs.debian.org/894444 # Do an apt update & apt upgrade at boot time (as well as @daily). # The lack of /etc/machine-id causes these to be implicitly enabled. # FIXME: use dropin in /etc. "--customize-hook=>>$1/lib/systemd/system/apt-daily.service printf '%s\n' '[Install]' 'WantedBy=multi-user.target'", "--customize-hook=>>$1/lib/systemd/system/apt-daily-upgrade.service printf '%s\n' '[Install]' 'WantedBy=multi-user.target'", # FIXME: add support for this stuff (for the non-live final install this happens via ansible): # # unattended-upgrades # smartd # networkd (boot off ANY NIC, not EVERY NIC -- https://github.com/systemd/systemd/issues/9714) # refind (bootloader config) # misc safety nets # double-check that mmdebstrap's machine-id support works properly # Bare minimum to let me SSH in. # FIXME: make this configurable. # FIXME: trust a CA certificate instead -- see Zero Trust SSH, Jeremy Stott, LCA 2020 # WARNING: tinysshd does not support RSA, nor MaxStartups, nor sftp (unless you also install openssh-client, which is huge). # FIXME: double-check no host keys are baked into the image (openssh-server and dropbear do this). "--include=tinysshd rsync", "--essential-hook=install -dm700 $1/root/.ssh", '--essential-hook=echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIapAZ0E0353DaY6xBnasvu/DOvdWdKQ6RQURwq4l6Wu twb@cyber.com.au (Trent W. Buck)" >$1/root/.ssh/authorized_keys', # Bare minimum to let me log in locally. # DO NOT use this on production builds! "--essential-hook=chroot $1 passwd --delete root", # Configure language (not needed to boot). # Racism saves a **LOT** of space -- something like 2GB for Debian Live images. # FIXME: use live-config instead? "--include=locales localepurge", f"--essential-hook=echo locales locales/default_environment_locale select {args.locale} | chroot $1 debconf-set-selections", f"--essential-hook=echo locales locales/locales_to_be_generated multiselect {args.locale} UTF-8 | chroot $1 debconf-set-selections", # FIXME: https://bugs.debian.org/603700 "--customize-hook=chroot $1 sed -i /etc/locale.nopurge -e 's/^USE_DPKG/#ARGH#&/'", "--customize-hook=chroot $1 localepurge", "--customize-hook=chroot $1 sed -i /etc/locale.nopurge -e 's/^#ARGH#//'", # Removing documentation also saves a LOT of space. "--dpkgopt=path-exclude=/usr/share/doc/*", "--dpkgopt=path-exclude=/usr/share/info/*", "--dpkgopt=path-exclude=/usr/share/man/*", "--dpkgopt=path-exclude=/usr/share/omf/*", "--dpkgopt=path-exclude=/usr/share/help/*", "--dpkgopt=path-exclude=/usr/share/gnome/help/*", # Configure timezone (not needed to boot)` # FIXME: use live-config instead? "--include=tzdata", f"--essential-hook=echo tzdata tzdata/Areas select {args.timezone[0]} | chroot $1 debconf-set-selections", f"--essential-hook=echo tzdata tzdata/Zones/{args.timezone[0]} select {args.timezone[1]} | chroot $1 debconf-set-selections", # Do the **BARE MINIMUM** to make a USB key that can boot on X86_64 UEFI. # We use mtools so we do not ever need root privileges. # We can't use mkfs.vfat, as that needs kpartx or losetup (i.e. root). # We can't use mkfs.udf, as that needs mount (i.e. root). # We can't use "refind-install --usedefault" as that runs mount(8) (i.e. root). # We don't use genisoimage because # 1) ISO9660 must die; # 2) incomplete UDF 1.5+ support; # 3) resulting filesystem can't be tweaked after flashing (e.g. debian-live/site.dir/etc/systemd/network/up.network). # # We use refind because 1) I hate grub; and 2) I like refind. # If you want aarch64 or ia32 you need to install their BOOTxxx.EFI files. # If you want kernel+initrd on something other than FAT, you need refind/drivers_xxx/xxx_xxx.EFI. # # FIXME: with qemu in UEFI mode (OVMF), I get dumped into startup.nsh (UEFI REPL). # From there, I can manually type in "FS0:\EFI\BOOT\BOOTX64.EFI" to start refind, tho. # So WTF is its problem? Does it not support fallback bootloader? "--include=refind parted mtools", "--essential-hook=echo refind refind/install_to_esp boolean false | chroot $1 debconf-set-selections", "--customize-hook=echo refind refind/install_to_esp boolean true | chroot $1 debconf-set-selections", "--customize-hook=chroot $1 mkdir -p /boot/USB /boot/EFI/BOOT", "--customize-hook=chroot $1 cp /usr/share/refind/refind/refind_x64.efi /boot/EFI/BOOT/BOOTX64.EFI", "--customize-hook=chroot $1 cp /usr/share/refind/refind/refind.conf-sample /boot/EFI/BOOT/refind.conf", f"--customize-hook=chroot $1 truncate --size={filesystem_img_size} /boot/USB/filesystem.img", f"--customize-hook=chroot $1 parted --script --align=optimal /boot/USB/filesystem.img mklabel gpt mkpart {esp_label} {esp_offset}b 100% set 1 esp on", f"--customize-hook=chroot $1 mformat -i /boot/USB/filesystem.img@@{esp_offset} -F -v {esp_label}", f"--customize-hook=chroot $1 mmd -i /boot/USB/filesystem.img@@{esp_offset} ::{live_media_path}", f"""--customize-hook=echo '"Boot with default options" "boot=live live-media-path={live_media_path}"' >$1/boot/refind_linux.conf""", f"""--customize-hook=chroot $1 find /boot/ -xdev -mindepth 1 -maxdepth 1 -not -name filesystem.img -not -name USB -exec mcopy -vsbpm -i /boot/USB/filesystem.img@@{esp_offset} {{}} :: ';'""", # FIXME: copy-out doesn't handle sparseness, so is REALLY slow (about 50 seconds). # Therefore instead leave it in the squashfs, and extract it later. # f'--customize-hook=copy-out /boot/USB/filesystem.img /tmp/', # f'--customize-hook=chroot $1 rm /boot/USB/filesystem.img', "sid", td / "filesystem.squashfs", ] ) with args.output_file.open("wb") as f: subprocess.check_call( ["rdsquashfs", "--cat=boot/USB/filesystem.img", td / "filesystem.squashfs"], stdout=f, ) subprocess.check_call( [ "mcopy", "-i", f"{args.output_file}@@{esp_offset}", td / "filesystem.squashfs", f"::{live_media_path}/filesystem.squashfs", ] ) mmdebstrap/gpgvnoexpkeysig000077500000000000000000000040471477517524100164150ustar00rootroot00000000000000#!/bin/sh # # No copyright is claimed. This code is in the public domain; do with # it what you wish. # # Author: Johannes Schauer Marin Rodrigues # # This is a wrapper around gpgv as invoked by apt. It turns EXPKEYSIG results # from gpgv into GOODSIG results. This is necessary for apt to access very old # timestamps from snapshot.debian.org for which the GPG key is already expired: # # Get:1 http://snapshot.debian.org/archive/debian/20150106T000000Z unstable InRelease [242 kB] # Err:1 http://snapshot.debian.org/archive/debian/20150106T000000Z unstable InRelease # The following signatures were invalid: EXPKEYSIG 8B48AD6246925553 Debian Archive Automatic Signing Key (7.0/wheezy) # Reading package lists... # W: GPG error: http://snapshot.debian.org/archive/debian/20150106T000000Z unstable InRelease: The following signatures were invalid: EXPKEYSIG 8B48AD6246925553 Debian Archive Automatic Signing Key (7.0/wheezy) # E: The repository 'http://snapshot.debian.org/archive/debian/20150106T000000Z unstable InRelease' is not signed. # # To use this script, call apt with # # -o Apt::Key::gpgvcommand=/usr/libexec/mmdebstrap/gpgvnoexpkeysig # # Scripts doing similar things can be found here: # # * debuerreotype as /usr/share/debuerreotype/scripts/.gpgv-ignore-expiration.sh # * derivative census: salsa.d.o/deriv-team/census/-/blob/master/bin/fakegpgv set -eu find_gpgv_status_fd() { while [ "$#" -gt 0 ]; do if [ "$1" = '--status-fd' ]; then echo "$2" return 0 fi shift done # default fd is stdout echo 1 } GPGSTATUSFD="$(find_gpgv_status_fd "$@")" case $GPGSTATUSFD in '' | *[!0-9]*) echo "invalid --status-fd argument" >&2 exit 1 ;; esac if ! command -v gpgv >&2; then eval 'echo "[GNUPG:] ERROR gpgv executable not found" >&'"$GPGSTATUSFD" exit 1 fi # we need eval because we cannot redirect a variable fd eval 'exec gpgv "$@" '"$GPGSTATUSFD"'>&1 | sed "s/^\[GNUPG:\] EXPKEYSIG /[GNUPG:] GOODSIG /" >&'"$GPGSTATUSFD" mmdebstrap/hooks/000077500000000000000000000000001477517524100143545ustar00rootroot00000000000000mmdebstrap/hooks/busybox/000077500000000000000000000000001477517524100160475ustar00rootroot00000000000000mmdebstrap/hooks/busybox/extract00.sh000077500000000000000000000004661477517524100202260ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" # Run busybox using an absolute path so that this script also works in case # /proc is not mounted. Busybox uses /proc/self/exe to figure out the path # to its executable. chroot "$rootdir" /bin/busybox --install -s mmdebstrap/hooks/busybox/setup00.sh000077500000000000000000000003661477517524100177130ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" mkdir -p "$rootdir/bin" echo root:x:0:0:root:/root:/bin/sh > "$rootdir/etc/passwd" cat << END > "$rootdir/etc/group" root:x:0: mail:x:8: utmp:x:43: END mmdebstrap/hooks/copy-host-apt-sources-and-preferences/000077500000000000000000000000001477517524100236035ustar00rootroot00000000000000mmdebstrap/hooks/copy-host-apt-sources-and-preferences/customize00.pl000077500000000000000000000021661477517524100263320ustar00rootroot00000000000000#!/usr/bin/perl # # This script makes sure that all packages that are installed both locally as # well as inside the chroot have the same version. # # It is implemented in Perl because there are no associative arrays in POSIX # shell. use strict; use warnings; sub get_pkgs { my $root = shift; my %pkgs = (); open(my $fh, '-|', 'dpkg-query', "--root=$root", '--showformat', '${binary:Package}=${Version}\n', '--show') // die "cannot exec dpkg-query"; while (my $line = <$fh>) { my ($pkg, $ver) = split(/=/, $line, 2); $pkgs{$pkg} = $ver; } close $fh; if ($? != 0) { die "failed to run dpkg-query" } return %pkgs; } my %pkgs_local = get_pkgs('/'); my %pkgs_chroot = get_pkgs($ARGV[0]); my @diff = (); foreach my $pkg (keys %pkgs_chroot) { next unless exists $pkgs_local{$pkg}; if ($pkgs_local{$pkg} ne $pkgs_chroot{$pkg}) { push @diff, $pkg; } } if (scalar @diff > 0) { print STDERR "E: packages from the host and the chroot differ:\n"; foreach my $pkg (@diff) { print STDERR "E: $pkg $pkgs_local{$pkg} $pkgs_chroot{$pkg}\n"; } exit 1; } mmdebstrap/hooks/copy-host-apt-sources-and-preferences/setup00.sh000077500000000000000000000035571477517524100254540ustar00rootroot00000000000000#!/bin/sh # # This script makes sure that the apt sources.list and preferences from outside # the chroot also exist inside the chroot by *appending* them to any existing # files. If you do not want to keep the original content, add another setup # hook before this one which cleans up the files you don't want to keep. # # If instead of copying sources.list verbatim you want to mangle its contents, # consider using python-apt for that. An example can be found in the Debian # packaging of mmdebstrap in ./debian/tests/sourcesfilter set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi if [ -n "${MMDEBSTRAP_SUITE:-}" ]; then if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 1 ]; then echo "W: using a non-empty suite name $MMDEBSTRAP_SUITE does not make sense with this hook and might select the wrong Essential:yes package set" >&2 fi fi rootdir="$1" SOURCELIST="/etc/apt/sources.list" eval "$(apt-config shell SOURCELIST Dir::Etc::SourceList/f)" SOURCEPARTS="/etc/apt/sources.d/" eval "$(apt-config shell SOURCEPARTS Dir::Etc::SourceParts/d)" PREFERENCES="/etc/apt/preferences" eval "$(apt-config shell PREFERENCES Dir::Etc::Preferences/f)" PREFERENCESPARTS="/etc/apt/preferences.d/" eval "$(apt-config shell PREFERENCESPARTS Dir::Etc::PreferencesParts/d)" for f in "$SOURCELIST" \ "$SOURCEPARTS"/*.list \ "$SOURCEPARTS"/*.sources \ "$PREFERENCES" \ "$PREFERENCESPARTS"/*; do [ -e "$f" ] || continue mkdir --parents "$(dirname "$rootdir/$f")" if [ -e "$rootdir/$f" ]; then if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 2 ]; then echo "I: $f already exists in chroot, appending..." >&2 fi # Add extra newline between old content and new content. # This is required in case of deb822 files. echo >> "$rootdir/$f" fi cat "$f" >> "$rootdir/$f" if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then echo "D: contents of $f inside the chroot:" >&2 cat "$rootdir/$f" >&2 fi done mmdebstrap/hooks/eatmydata/000077500000000000000000000000001477517524100163255ustar00rootroot00000000000000mmdebstrap/hooks/eatmydata/README.txt000066400000000000000000000005131477517524100200220ustar00rootroot00000000000000Adding this directory with --hook-directory will result in mmdebstrap using dpkg inside an eatmydata wrapper script. This will result in spead-ups on systems where sync() takes some time. Using --dpkgopt=force-unsafe-io will have a lesser effect compared to eatmydata. See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=613428 mmdebstrap/hooks/eatmydata/customize.sh000077500000000000000000000013041477517524100207040ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" if [ -e "$rootdir/var/lib/dpkg/arch" ]; then chrootarch=$(head -1 "$rootdir/var/lib/dpkg/arch") else chrootarch=$(dpkg --print-architecture) fi libdir="/usr/lib/$(dpkg-architecture -a "$chrootarch" -q DEB_HOST_MULTIARCH)" # if eatmydata was actually installed properly, then we are not removing # anything here if ! chroot "$rootdir" dpkg-query --show eatmydata; then rm "$rootdir/usr/bin/eatmydata" fi if ! chroot "$rootdir" dpkg-query --show libeatmydata1; then rm "$rootdir$libdir"/libeatmydata.so* fi rm "$rootdir/usr/bin/dpkg" chroot "$rootdir" dpkg-divert --local --rename --remove /usr/bin/dpkg sync mmdebstrap/hooks/eatmydata/extract.sh000077500000000000000000000044621477517524100203440ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" if [ -e "$rootdir/var/lib/dpkg/arch" ]; then chrootarch=$(head -1 "$rootdir/var/lib/dpkg/arch") else chrootarch=$(dpkg --print-architecture) fi trusted= eval "$(apt-config shell trusted Dir::Etc::trusted/f)" trustedparts= eval "$(apt-config shell trustedparts Dir::Etc::trustedparts/d)" tmpfile=$(mktemp --tmpdir="$rootdir/tmp") cat << END > "$tmpfile" Apt::Architecture "$chrootarch"; Apt::Architectures "$chrootarch"; Dir "$rootdir"; Dir::Etc::Trusted "$trusted"; Dir::Etc::TrustedParts "$trustedparts"; END # we run "apt-get download --print-uris" in a temporary directory, to make sure # that the packages do not already exist in the current directory, or otherwise # nothing will be printed for them tmpdir=$(mktemp --directory --tmpdir="$rootdir/tmp") env --chdir="$tmpdir" APT_CONFIG="$tmpfile" apt-get download --print-uris eatmydata libeatmydata1 \ | sed -ne "s/^'\([^']\+\)'\s\+\(\S\+\)\s\+\([0-9]\+\)\s\+\(SHA256:[a-f0-9]\+\)$/\1 \2 \3 \4/p" \ | while read -r uri fname size hash; do echo "processing $fname" >&2 if [ -e "$tmpdir/$fname" ]; then echo "$tmpdir/$fname already exists" >&2 exit 1 fi [ -z "$hash" ] && hash="Checksum-FileSize:$size" env --chdir="$tmpdir" APT_CONFIG="$tmpfile" /usr/lib/apt/apt-helper download-file "$uri" "$fname" "$hash" case "$fname" in eatmydata_*_all.deb) mkdir -p "$rootdir/usr/bin" dpkg-deb --fsys-tarfile "$tmpdir/$fname" \ | tar --directory="$rootdir/usr/bin" --strip-components=3 --extract --verbose ./usr/bin/eatmydata ;; libeatmydata1_*_$chrootarch.deb) libdir="/usr/lib/$(dpkg-architecture -a "$chrootarch" -q DEB_HOST_MULTIARCH)" mkdir -p "$rootdir$libdir" dpkg-deb --fsys-tarfile "$tmpdir/$fname" \ | tar --directory="$rootdir$libdir" --strip-components=4 --extract --verbose --wildcards ".$libdir/libeatmydata.so*" ;; *) echo "unexpected filename: $fname" >&2 exit 1 ;; esac rm "$tmpdir/$fname" done rm "$tmpfile" rmdir "$tmpdir" mv "$rootdir/usr/bin/dpkg" "$rootdir/usr/bin/dpkg.distrib" cat << END > "$rootdir/usr/bin/dpkg" #!/bin/sh exec /usr/bin/eatmydata /usr/bin/dpkg.distrib "\$@" END chmod +x "$rootdir/usr/bin/dpkg" cat << END >> "$rootdir/var/lib/dpkg/diversions" /usr/bin/dpkg /usr/bin/dpkg.distrib : END mmdebstrap/hooks/file-mirror-automount/000077500000000000000000000000001477517524100206345ustar00rootroot00000000000000mmdebstrap/hooks/file-mirror-automount/customize00.sh000077500000000000000000000015441477517524100233610ustar00rootroot00000000000000#!/bin/sh # # shellcheck disable=SC2086 set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" if [ ! -e "$rootdir/run/mmdebstrap/file-mirror-automount" ]; then exit 0 fi xargsopts="--null --no-run-if-empty -I {} --max-args=1" case $MMDEBSTRAP_MODE in root|unshare) echo "unmounting the following mountpoints:" >&2 ;; *) echo "removing the following directories:" >&2 ;; esac < "$rootdir/run/mmdebstrap/file-mirror-automount" \ xargs $xargsopts echo " $rootdir/{}" case $MMDEBSTRAP_MODE in root|unshare) < "$rootdir/run/mmdebstrap/file-mirror-automount" \ xargs $xargsopts umount "$rootdir/{}" ;; *) < "$rootdir/run/mmdebstrap/file-mirror-automount" \ xargs $xargsopts rm -r "$rootdir/{}" ;; esac rm "$rootdir/run/mmdebstrap/file-mirror-automount" rmdir --ignore-fail-on-non-empty "$rootdir/run/mmdebstrap" mmdebstrap/hooks/file-mirror-automount/setup00.sh000077500000000000000000000041411477517524100224730ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi rootdir="$1" # process all configured apt repositories env APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get indextargets --no-release-info --format '$(REPO_URI)' \ | sed -ne 's/^file:\/\+//p' \ | sort -u \ | while read -r path; do mkdir -p "$rootdir/run/mmdebstrap" if [ ! -d "/$path" ]; then echo "W: /$path is not an existing directory" >&2 continue fi case $MMDEBSTRAP_MODE in root|unshare) echo "bind-mounting /$path into the chroot" >&2 mkdir -p "$rootdir/$path" mount -o ro,bind "/$path" "$rootdir/$path" ;; *) echo "copying /$path into the chroot" >&2 mkdir -p "$rootdir/$path" "$MMDEBSTRAP_ARGV0" --hook-helper "$rootdir" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" sync-in "/$path" "/$path" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK" ;; esac printf '/%s\0' "$path" >> "$rootdir/run/mmdebstrap/file-mirror-automount" done # process all files given via --include set -f # turn off pathname expansion IFS=',' # split by comma for pkg in $MMDEBSTRAP_INCLUDE; do set +f; unset IFS case $pkg in ./*|../*|/*) : ;; # we are interested in this case *) continue ;; # not a file esac # undo escaping pkg="$(printf '%s' "$pkg" | sed 's/%2C/,/g; s/%25/%/g')" # check for existance if [ ! -f "$pkg" ]; then echo "$pkg does not exist" >&2 continue fi # make path absolute pkg="$(realpath "$pkg")" case "$pkg" in /*) : ;; *) echo "path for $pkg is not absolute" >&2; continue;; esac mkdir -p "$rootdir/run/mmdebstrap" mkdir -p "$rootdir/$(dirname "$pkg")" case $MMDEBSTRAP_MODE in root|unshare) echo "bind-mounting $pkg into the chroot" >&2 touch "$rootdir/$pkg" mount -o bind "$pkg" "$rootdir/$pkg" ;; *) echo "copying $pkg into the chroot" >&2 "$MMDEBSTRAP_ARGV0" --hook-helper "$rootdir" "$MMDEBSTRAP_MODE" "$MMDEBSTRAP_HOOK" env "$MMDEBSTRAP_VERBOSITY" upload "$pkg" "$pkg" <&"$MMDEBSTRAP_HOOKSOCK" >&"$MMDEBSTRAP_HOOKSOCK" ;; esac printf '/%s\0' "$pkg" >> "$rootdir/run/mmdebstrap/file-mirror-automount" done set +f; unset IFS mmdebstrap/hooks/jessie-or-older/000077500000000000000000000000001477517524100173575ustar00rootroot00000000000000mmdebstrap/hooks/jessie-or-older/extract00.sh000077500000000000000000000004001477517524100215220ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" # not needed since dpkg 1.17.11 for f in available diversions cmethopt; do if [ ! -e "$TARGET/var/lib/dpkg/$f" ]; then touch "$TARGET/var/lib/dpkg/$f" fi done mmdebstrap/hooks/jessie-or-older/extract01.sh000077500000000000000000000021051477517524100215270ustar00rootroot00000000000000#!/bin/sh # # needed until init 1.33 which pre-depends on systemd-sysv # starting with init 1.34, init is not Essential:yes anymore # # jessie has init 1.22 set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" if [ -z "${MMDEBSTRAP_ESSENTIAL+x}" ]; then MMDEBSTRAP_ESSENTIAL= for f in "$TARGET/var/cache/apt/archives/"*.deb; do [ -f "$f" ] || continue f="${f#"$TARGET"}" MMDEBSTRAP_ESSENTIAL="$MMDEBSTRAP_ESSENTIAL $f" done fi fname_base_passwd= fname_base_files= fname_dpkg= for pkg in $MMDEBSTRAP_ESSENTIAL; do pkgname=$(dpkg-deb --show --showformat='${Package}' "$TARGET/$pkg") # shellcheck disable=SC2034 case $pkgname in base-passwd) fname_base_passwd=$pkg;; base-files) fname_base_files=$pkg;; dpkg) fname_dpkg=$pkg;; esac done for var in base_passwd base_files dpkg; do eval 'val=$fname_'"$var" [ -z "$val" ] && continue chroot "$TARGET" dpkg --install --force-depends "$val" done # shellcheck disable=SC2086 chroot "$TARGET" dpkg --unpack --force-depends $MMDEBSTRAP_ESSENTIAL chroot "$TARGET" dpkg --configure --pending mmdebstrap/hooks/maybe-jessie-or-older/000077500000000000000000000000001477517524100204525ustar00rootroot00000000000000mmdebstrap/hooks/maybe-jessie-or-older/extract00.sh000077500000000000000000000031241477517524100226230ustar00rootroot00000000000000#!/bin/sh set -eu # we need to check the version of dpkg # since at this point packages are just extracted but not installed, we cannot use dpkg-query # since we want to support chrootless, we cannot run dpkg --version inside the chroot # to avoid this hook depending on dpkg-dev being installed, we do not parse the extracted changelog with dpkg-parsechangelog # we also want to avoid parsing the changelog because /usr/share/doc might've been added to dpkg --path-exclude # instead, we just ask apt about the latest version of dpkg it knows of # this should only fail in situations where there are multiple versions of dpkg in different suites ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions dpkg 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '') if [ -z "$ver" ]; then echo "no package called dpkg can be installed -- not running jessie-or-older extract00 hook" >&2 exit 0 fi if dpkg --compare-versions "$ver" ge 1.17.11; then echo "dpkg version $ver is >= 1.17.11 -- not running jessie-or-older extract00 hook" >&2 exit 0 else echo "dpkg version $ver is << 1.17.11 -- running jessie-or-older extract00 hook" >&2 fi # resolve the script path using several methods in order: # 1. using dirname -- "$0" # 2. using ./hooks # 3. using /usr/share/mmdebstrap/hooks/ for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do if [ -x "$p/jessie-or-older/extract00.sh" ] && [ -x "$p/jessie-or-older/extract01.sh" ]; then "$p/jessie-or-older/extract00.sh" "$1" exit 0 fi done echo "cannot find jessie-or-older hook anywhere" >&2 exit 1 mmdebstrap/hooks/maybe-jessie-or-older/extract01.sh000077500000000000000000000051151477517524100226260ustar00rootroot00000000000000#!/bin/sh set -eu # The jessie-or-older extract01 hook has to be run up to the point where the # Essential:yes field was removed from the init package (with # init-system-helpers 1.34). Since the essential packages have only been # extracted but not installed, we cannot use dpkg-query to find out its # version. Since /usr/share/doc might be missing due to dpkg --path-exclude, we # also cannot check whether /usr/share/doc/init/copyright exists. There also # was a time (before init-system-helpers 1.20) where there was no init package # at all where we also want to apply this hook. So we just ask apt about the # candidate version for init-system-helpers. This should only fail in # situations where there are multiple versions of init-system-helpers in # different suites. ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions init-system-helpers 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '') if [ -z "$ver" ]; then # there is no package called init-system-helpers, so either: # - this is so old that init-system-helpers didn't exist yet # - we are in a future where init-system-helpers doesn't exist anymore # - something strange is going on # we should only call the hook in the first case ver=$(env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions base-files 2>/dev/null | sed -ne 's/^Version: \(.*\)$/\1/p' || printf '') if [ -z "$ver" ]; then echo "neither init-system-helpers nor base-files can be installed -- not running jessie-or-older extract01 hook" >&2 exit 0 fi # Jessie is Debian 8 if dpkg --compare-versions "$ver" ge 8; then echo "there is no init-system-helpers but base-files version $ver is >= 8 -- not running jessie-or-older extract01 hook" >&2 exit 0 else echo "there is no init-system-helpers but base-files version $ver is << 8 -- running jessie-or-older extract01 hook" >&2 fi else if dpkg --compare-versions "$ver" ge 1.34; then echo "init-system-helpers version $ver is >= 1.34 -- not running jessie-or-older extract01 hook" >&2 exit 0 else echo "init-system-helpers version $ver is << 1.34 -- running jessie-or-older extract01 hook" >&2 fi fi # resolve the script path using several methods in order: # 1. using dirname -- "$0" # 2. using ./hooks # 3. using /usr/share/mmdebstrap/hooks/ for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do if [ -x "$p/jessie-or-older/extract00.sh" ] && [ -x "$p/jessie-or-older/extract01.sh" ]; then "$p/jessie-or-older/extract01.sh" "$1" exit 0 fi done echo "cannot find jessie-or-older hook anywhere" >&2 exit 1 mmdebstrap/hooks/maybe-merged-usr/000077500000000000000000000000001477517524100175215ustar00rootroot00000000000000mmdebstrap/hooks/maybe-merged-usr/essential00.sh000077500000000000000000000023331477517524100222100ustar00rootroot00000000000000#!/bin/sh set -eu ver=$(dpkg-query --root="$1" -f '${db:Status-Status} ${Source} ${Version}' --show usr-is-merged 2>/dev/null || printf '') case "$ver" in '') echo "no package called usr-is-merged is installed -- not running merged-usr essential hook" >&2 exit 0 ;; 'installed mmdebstrap-dummy-usr-is-merged 1') echo "dummy usr-is-merged package installed -- running merged-usr essential hook" >&2 ;; 'installed usrmerge '*) echo "usr-is-merged package from src:usrmerge installed -- not running merged-usr essential hook" >&2 exit 0 ;; 'not-installed ') echo "usr-is-merged was not installed in a previous hook -- not running merged-usr essential hook" >&2 exit 0 ;; *) echo "unexpected situation for package usr-is-merged: $ver" >&2 exit 1 ;; esac # resolve the script path using several methods in order: # 1. using dirname -- "$0" # 2. using ./hooks # 3. using /usr/share/mmdebstrap/hooks/ for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then "$p/merged-usr/essential00.sh" "$1" exit 0 fi done echo "cannot find merged-usr hook anywhere" >&2 exit 1 mmdebstrap/hooks/maybe-merged-usr/extract00.sh000077500000000000000000000026621477517524100217000ustar00rootroot00000000000000#!/bin/sh set -eu env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then # if apt-cache exited successfully, then usr-is-merged exists either as # a real or virtual package if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then echo "usr-is-merged found -- running merged-usr extract hook" >&2 else # The usr-is-merged must be virtual, so assume that nothing # has to be done. This is the case with Debian Trixie or later # or with Ubuntu Lunar or later echo "usr-is-merged found but not real -- not running merged-usr extract hook" >&2 exit 0 fi else # if the usr-is-merged package cannot be installed with apt, do nothing echo "no package providing usr-is-merged found -- not running merged-usr extract hook" >&2 exit 0 fi # resolve the script path using several methods in order: # 1. using dirname -- "$0" # 2. using ./hooks # 3. using /usr/share/mmdebstrap/hooks/ for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then "$p/merged-usr/extract00.sh" "$1" exit 0 fi done echo "cannot find merged-usr hook anywhere" >&2 exit 1 mmdebstrap/hooks/maybe-merged-usr/setup00.sh000077500000000000000000000026521477517524100213650ustar00rootroot00000000000000#!/bin/sh set -eu env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-get update --error-on=any if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged > /dev/null 2>&1; then # if apt-cache exited successfully, then usr-is-merged exists either as # a real or virtual package if env --chdir="$1" APT_CONFIG="$MMDEBSTRAP_APT_CONFIG" apt-cache show --no-all-versions usr-is-merged 2>/dev/null | grep -q "Package: usr-is-merged"; then echo "usr-is-merged found -- running merged-usr setup hook" >&2 else # The usr-is-merged must be virtual, so assume that nothing # has to be done. This is the case with Debian Trixie or later # or with Ubuntu Lunar or later echo "usr-is-merged found but not real -- not running merged-usr setup hook" >&2 exit 0 fi else # if the usr-is-merged package cannot be installed with apt, do nothing echo "no package providing usr-is-merged found -- not running merged-usr setup hook" >&2 exit 0 fi # resolve the script path using several methods in order: # 1. using dirname -- "$0" # 2. using ./hooks # 3. using /usr/share/mmdebstrap/hooks/ for p in "$(dirname -- "$0")/.." ./hooks /usr/share/mmdebstrap/hooks; do if [ -x "$p/merged-usr/setup00.sh" ] && [ -x "$p/merged-usr/extract00.sh" ] && [ -x "$p/merged-usr/essential00.sh" ]; then "$p/merged-usr/setup00.sh" "$1" exit 0 fi done echo "cannot find merged-usr hook anywhere" >&2 exit 1 mmdebstrap/hooks/merged-usr/000077500000000000000000000000001477517524100164265ustar00rootroot00000000000000mmdebstrap/hooks/merged-usr/essential00.sh000077500000000000000000000021731477517524100211170ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" if [ "${MMDEBSTRAP_MODE:-}" = "chrootless" ]; then APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install \ -oDPkg::Chroot-Directory= \ -oDPkg::Options::=--force-not-root \ -oDPkg::Options::=--force-script-chrootless \ -oDPkg::Options::=--root="$TARGET" \ -oDPkg::Options::=--log="$TARGET/var/log/dpkg.log" \ usr-is-merged export DPKG_ROOT="$TARGET" dpkg-query --showformat '${db:Status-Status}\n' --show usr-is-merged | grep -q '^installed$' dpkg-query --showformat '${Source}\n' --show usr-is-merged | grep -q '^usrmerge$' dpkg --compare-versions "1" "lt" "$(dpkg-query --showformat '${Version}\n' --show usr-is-merged)" else APT_CONFIG=$MMDEBSTRAP_APT_CONFIG apt-get --yes install usr-is-merged chroot "$TARGET" dpkg-query --showformat '${db:Status-Status}\n' --show usr-is-merged | grep -q '^installed$' chroot "$TARGET" dpkg-query --showformat '${Source}\n' --show usr-is-merged | grep -q '^usrmerge$' dpkg --compare-versions "1" "lt" "$(chroot "$TARGET" dpkg-query --showformat '${Version}\n' --show usr-is-merged)" fi mmdebstrap/hooks/merged-usr/extract00.sh000077500000000000000000000053371477517524100206070ustar00rootroot00000000000000#!/bin/sh set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" # can_usrmerge_symlink() and can_usrmerge_symlink() are # Copyright 2023 Helmut Grohne # and part of the debootstrap source in /usr/share/debootstrap/functions # https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/96 # https://bugs.debian.org/104989 can_usrmerge_symlink() { # Absolute symlinks can be relocated without problems. test "${2#/}" = "$2" || return 0 while :; do if test "${2#/}" != "$2"; then # Handle double-slashes. set -- "$1" "${2#/}" elif test "${2#./}" != "$2"; then # Handle ./ inside a link target. set -- "$1" "${2#./}" elif test "$2" = ..; then # A parent directory symlink is ok if it does not # cross the top level directory. test "${1%/*/*}" != "$1" -a -n "${1%/*/*}" return $? elif test "${2#../}" != "$2"; then # Symbolic link crossing / cannot be moved safely. # This is prohibited by Debian Policy 10.5. test "${1%/*/*}" = "$1" -o -z "${1%/*/*}" && return 1 set -- "${1%/*}" "${2#../}" else # Consider the symlink ok if its target does not # contain a parent directory. When we fail here, # the link target is non-minimal and doesn't happen # in the archive. test "${2#*/../}" = "$2" return $? fi done } merge_usr_entry() { # shellcheck disable=SC3043 local entry canon canon="$TARGET/usr/${1#"$TARGET/"}" test -h "$canon" && error 1 USRMERGEFAIL "cannot move %s as its destination exists as a symlink" "${1#"$TARGET"}" if ! test -e "$canon"; then mv "$1" "$canon" return 0 fi test -d "$1" || error 1 USRMERGEFAIL "cannot move non-directory %s as its destination exists" "${1#"$TARGET"}" test -d "$canon" || error 1 USRMERGEFAIL "cannot move directory %s as its destination is not a directory" "${1#"$TARGET"}" for entry in "$1/"* "$1/."*; do # Some shells return . and .. on dot globs. test "${entry%/.}" != "${entry%/..}" && continue if test -h "$entry" && ! can_usrmerge_symlink "${entry#"$TARGET"}" "$(readlink "$entry")"; then error 1 USRMERGEFAIL "cannot move relative symlink crossing top-level directory" "${entry#"$TARGET"}" fi # Ignore glob match failures if test "${entry%'/*'}" != "${entry%'/.*'}" && ! test -e "$entry"; then continue fi merge_usr_entry "$entry" done rmdir "$1" } # This is list includes all possible multilib directories. It must be # updated when new multilib directories are being added. Hopefully, # all new architectures use multiarch instead, so we never get to # update this. for dir in bin lib lib32 lib64 libo32 libx32 sbin; do test -h "$TARGET/$dir" && continue test -e "$TARGET/$dir" || continue merge_usr_entry "$TARGET/$dir" ln -s "usr/$dir" "$TARGET/$dir" done mmdebstrap/hooks/merged-usr/setup00.sh000077500000000000000000000066111477517524100202710ustar00rootroot00000000000000#!/bin/sh # # mmdebstrap does have a --merged-usr option but only as a no-op for # debootstrap compatibility # # Using this hook script, you can emulate what debootstrap does to set up # merged /usr via directory symlinks, even using the exact same shell function # that debootstrap uses by running mmdebstrap with: # # --setup-hook=/usr/share/mmdebstrap/hooks/merged-usr/setup00.sh # # Alternatively, you can setup merged-/usr by installing the usrmerge package: # # --include=usrmerge # # mmdebstrap will not include this functionality via a --merged-usr option # because there are many reasons against implementing merged-/usr that way: # # https://wiki.debian.org/Teams/Dpkg/MergedUsr # https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F # https://lists.debian.org/20190219044924.GB21901@gaara.hadrons.org # https://lists.debian.org/YAkLOMIocggdprSQ@thunder.hadrons.org # https://lists.debian.org/20181223030614.GA8788@gaara.hadrons.org # # In addition, the merged-/usr-via-aliased-dirs approach violates an important # principle of component based software engineering one of the core design # ideas/goals of mmdebstrap: All the information to create a chroot of a Debian # based distribution should be included in its packages and their metadata. # Using directory symlinks as used by debootstrap contradicts this principle. # The information whether a distribution uses this approach to merged-/usr or # not is not anymore contained in its packages but in a tool from the outside. # # Example real world problem: I'm using debbisect to bisect Debian unstable # between 2015 and today. For which snapshot.d.o timestamp should a merged-/usr # chroot be created and for which ones not? # # The problem is not the idea of merged-/usr but the problem is the way how it # got implemented in debootstrap via directory symlinks. That way of rolling # out merged-/usr is bad from the dpkg point-of-view and completely opposite of # the vision with which in mind I wrote mmdebstrap. set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" # now install an empty "usr-is-merged" package to avoid installing the # usrmerge package on this system even after init-system-helpers starts # depending on "usrmerge | usr-is-merged". # # This package will not end up in the final chroot because the essential # hook replaces it with the actual usr-is-merged package from src:usrmerge. tmpdir=$(mktemp --directory --tmpdir="$TARGET/tmp") mkdir -p "$tmpdir/usr-is-merged/DEBIAN" cat << END > "$tmpdir/usr-is-merged/DEBIAN/control" Package: usr-is-merged Priority: optional Section: oldlibs Maintainer: Johannes Schauer Marin Rodrigues Architecture: all Multi-Arch: foreign Source: mmdebstrap-dummy-usr-is-merged Version: 1 Description: dummy package created by mmdebstrap merged-usr setup hook This package was generated and installed by the mmdebstrap merged-usr setup hook at /usr/share/mmdebstrap/hooks/merged-usr. . If this package is installed in the final chroot, then this is a bug in mmdebstrap. Please report: https://gitlab.mister-muffin.de/josch/mmdebstrap END dpkg-deb --build "$tmpdir/usr-is-merged" "$tmpdir/usr-is-merged.deb" dpkg --root="$TARGET" --log="$TARGET/var/log/dpkg.log" --install "$tmpdir/usr-is-merged.deb" rm "$tmpdir/usr-is-merged.deb" "$tmpdir/usr-is-merged/DEBIAN/control" rmdir "$tmpdir/usr-is-merged/DEBIAN" "$tmpdir/usr-is-merged" "$tmpdir" mmdebstrap/hooks/no-merged-usr/000077500000000000000000000000001477517524100170405ustar00rootroot00000000000000mmdebstrap/hooks/no-merged-usr/essential00.sh000077700000000000000000000000001477517524100264662../merged-usr/essential00.shustar00rootroot00000000000000mmdebstrap/hooks/no-merged-usr/setup00.sh000077500000000000000000000043171477517524100207040ustar00rootroot00000000000000#!/bin/sh # # mmdebstrap does have a --no-merged-usr option but only as a no-op for # debootstrap compatibility # # Using this hook script, you can emulate what debootstrap does to set up # a system without merged-/usr even after the essential init-system-helpers # package added a dependency on "usrmerge | usr-is-merged". By installing # a dummy usr-is-merged package, it avoids pulling in the dependencies of # the usrmerge package. set -eu if [ "${MMDEBSTRAP_VERBOSITY:-1}" -ge 3 ]; then set -x fi TARGET="$1" echo "Warning: starting with Debian 12 (Bookworm), systems without merged-/usr are not supported anymore" >&2 echo "Warning: starting with Debian 13 (Trixie), merged-/usr symlinks are shipped by packages in the essential-set making this hook ineffective" >&2 echo "this system will not be supported in the future" > "$TARGET/etc/unsupported-skip-usrmerge-conversion" # now install an empty "usr-is-merged" package to avoid installing the # usrmerge package on this system even after init-system-helpers starts # depending on "usrmerge | usr-is-merged". # # This package will not end up in the final chroot because the essential # hook replaces it with the actual usr-is-merged package from src:usrmerge. tmpdir=$(mktemp --directory --tmpdir="$TARGET/tmp") mkdir -p "$tmpdir/usr-is-merged/DEBIAN" cat << END > "$tmpdir/usr-is-merged/DEBIAN/control" Package: usr-is-merged Priority: optional Section: oldlibs Maintainer: Johannes Schauer Marin Rodrigues Architecture: all Multi-Arch: foreign Source: mmdebstrap-dummy-usr-is-merged Version: 1 Description: dummy package created by mmdebstrap no-merged-usr setup hook This package was generated and installed by the mmdebstrap no-merged-usr setup hook at /usr/share/mmdebstrap/hooks/no-merged-usr. . If this package is installed in the final chroot, then this is a bug in mmdebstrap. Please report: https://gitlab.mister-muffin.de/josch/mmdebstrap END dpkg-deb --build "$tmpdir/usr-is-merged" "$tmpdir/usr-is-merged.deb" dpkg --root="$TARGET" --log="$TARGET/var/log/dpkg.log" --install "$tmpdir/usr-is-merged.deb" rm "$tmpdir/usr-is-merged.deb" "$tmpdir/usr-is-merged/DEBIAN/control" rmdir "$tmpdir/usr-is-merged/DEBIAN" "$tmpdir/usr-is-merged" "$tmpdir" mmdebstrap/ldconfig.fakechroot000077500000000000000000000103711477517524100170720ustar00rootroot00000000000000#!/usr/bin/env python3 # # This script is in the public domain # # Author: Johannes Schauer Marin Rodrigues # # This is command substitution for ldconfig under fakechroot: # # export FAKECHROOT_CMD_SUBST=/sbin/ldconfig=/path/to/ldconfig.fakechroot # # Statically linked binaries cannot work with fakechroot and thus have to be # replaced by either /bin/true or a more clever solution like this one. The # ldconfig command supports the -r option which allows passing a chroot # directory for ldconfig to work in. This can be used to run ldconfig without # fakechroot but still let it create /etc/ld.so.cache inside the chroot. # # Since absolute symlinks are broken without fakechroot to translate them, # we read /etc/ld.so.conf and turn all absolute symlink shared libraries into # relative ones. At program exit, the original state is restored. import os import sys import subprocess import atexit import glob from pathlib import Path symlinks = [] def restore_symlinks(): for (link, target, atime, mtime) in symlinks: link.unlink() link.symlink_to(target) os.utime(link, times=None, ns=(atime, mtime), follow_symlinks=False) atexit.register(restore_symlinks) def get_libdirs(chroot, configs): res = [] for conf in configs: for line in (Path(conf)).read_text().splitlines(): line = line.strip() if not line: continue if line.startswith("#"): continue if line.startswith("include "): assert line.startswith("include /") res.extend( get_libdirs(chroot, chroot.glob(line.removeprefix("include /"))) ) continue assert line.startswith("/"), line line = line.lstrip("/") if not (chroot / Path(line)).is_dir(): continue for f in (chroot / Path(line)).iterdir(): if not f.is_symlink(): continue linktarget = f.readlink() # make sure that the linktarget is an absolute path inside the # chroot if not str(linktarget).startswith("/"): continue if chroot not in linktarget.parents: continue # store original link so that we can restore it later symlinks.append( (f, linktarget, f.lstat().st_atime_ns, f.lstat().st_mtime_ns) ) # replace absolute symlink by relative link relative = os.path.relpath(linktarget, f.parent) f.unlink() f.symlink_to(relative) return res def main(): if "FAKECHROOT_BASE_ORIG" not in os.environ: print("FAKECHROOT_BASE_ORIG is not set", file=sys.stderr) print( "must be executed under fakechroot using FAKECHROOT_CMD_SUBST", file=sys.stderr, ) sys.exit(1) chroot = Path(os.environ["FAKECHROOT_BASE_ORIG"]) # if chrootless mode is used from within a fakechroot chroot, then # FAKECHROOT_BASE_ORIG will point at the outer chroot. We want to use # the path from DPKG_ROOT inside of that instead if os.environ.get("DPKG_ROOT", "") not in ["", "/"]: chroot /= os.environ["DPKG_ROOT"].lstrip("/") if not (chroot / "sbin" / "ldconfig").exists(): sys.exit(0) (chroot / "var" / "cache" / "ldconfig").mkdir( mode=0o700, parents=True, exist_ok=True ) for d in get_libdirs(chroot, [chroot / "etc" / "ld.so.conf"]): make_relative(d) rootarg = chroot argv = sys.argv[1:] for arg in sys.argv[1:]: if arg == "-r": rootarg = None elif rootarg is None: argpath = Path(arg) if argpath.is_absolute(): rootarg = chroot / argpath.relative_to("/") else: rootarg = Path.cwd() / argpath if rootarg is None: rootarg = chroot # we add any additional arguments before "-r" such that any other "-r" # option will be overwritten by the one we set subprocess.check_call( [chroot / "sbin" / "ldconfig"] + sys.argv[1:] + ["-r", rootarg] ) if __name__ == "__main__": main() mmdebstrap/make_mirror.sh000077500000000000000000000460711477517524100161070ustar00rootroot00000000000000#!/bin/sh set -eu # This script fills either cache.A or cache.B with new content and then # atomically switches the cache symlink from one to the other at the end. # This way, at no point will the cache be in an non-working state, even # when this script got canceled at any point. # Working with two directories also automatically prunes old packages in # the local repository. deletecache() { dir="$1" echo "running deletecache $dir" >&2 if [ ! -e "$dir" ]; then return fi if [ ! -e "$dir/mmdebstrapcache" ]; then echo "$dir cannot be the mmdebstrap cache" >&2 return 1 fi # be very careful with removing the old directory # experimental is pulled in with USE_HOST_APT_CONFIG=yes on debci # when testing a package from experimental for dist in oldstable stable testing unstable experimental; do # deleting artifacts from test "debootstrap" for variant in minbase buildd -; do if [ -e "$dir/debian-$dist-$variant.tar" ]; then rm "$dir/debian-$dist-$variant.tar" else echo "does not exist: $dir/debian-$dist-$variant.tar" >&2 fi done # deleting artifacts from test "mmdebstrap" for variant in essential apt minbase buildd - standard; do for format in tar ext2 ext4 squashfs; do if [ -e "$dir/mmdebstrap-$dist-$variant.$format" ]; then # attempt to delete for all dists because DEFAULT_DIST might've been different the last time rm "$dir/mmdebstrap-$dist-$variant.$format" elif [ "$dist" = "$DEFAULT_DIST" ]; then # only warn about non-existance when it's expected to exist echo "does not exist: $dir/mmdebstrap-$dist-$variant.$format" >&2 fi done done if [ -e "$dir/debian/dists/$dist" ]; then rm --one-file-system --recursive "$dir/debian/dists/$dist" else echo "does not exist: $dir/debian/dists/$dist" >&2 fi case "$dist" in oldstable | stable) if [ -e "$dir/debian/dists/$dist-updates" ]; then rm --one-file-system --recursive "$dir/debian/dists/$dist-updates" else echo "does not exist: $dir/debian/dists/$dist-updates" >&2 fi ;; esac case "$dist" in oldstable | stable) if [ -e "$dir/debian-security/dists/$dist-security" ]; then rm --one-file-system --recursive "$dir/debian-security/dists/$dist-security" else echo "does not exist: $dir/debian-security/dists/$dist-security" >&2 fi ;; esac done for f in "$dir/debian-"*.ext4; do if [ -e "$f" ]; then rm --one-file-system "$f" fi done # on i386 and amd64, the intel-microcode and amd64-microcode packages # from non-free-firwame get pulled in because they are # priority:standard with USE_HOST_APT_CONFIG=yes for c in main non-free-firmware; do if [ -e "$dir/debian/pool/$c" ]; then rm --one-file-system --recursive "$dir/debian/pool/$c" else echo "does not exist: $dir/debian/pool/$c" >&2 fi done if [ -e "$dir/debian-security/pool/updates/main" ]; then rm --one-file-system --recursive "$dir/debian-security/pool/updates/main" else echo "does not exist: $dir/debian-security/pool/updates/main" >&2 fi for i in $(seq 1 6); do if [ ! -e "$dir/debian$i" ]; then continue fi rm "$dir/debian$i" done rm "$dir/mmdebstrapcache" # remove all symlinks find "$dir" -type l -delete # now the rest should only be empty directories if [ -e "$dir" ]; then find "$dir" -depth -print0 | xargs -0 --no-run-if-empty rmdir else echo "does not exist: $dir" >&2 fi } cleanup_newcachedir() { echo "running cleanup_newcachedir" deletecache "$newcachedir" } cleanupapt() { echo "running cleanupapt" >&2 if [ ! -e "$rootdir" ]; then return fi for f in \ "$rootdir/var/cache/apt/archives/"*.deb \ "$rootdir/var/cache/apt/archives/partial/"*.deb \ "$rootdir/var/cache/apt/"*.bin \ "$rootdir/var/lib/apt/lists/"* \ "$rootdir/var/lib/dpkg/status" \ "$rootdir/var/lib/dpkg/lock-frontend" \ "$rootdir/var/lib/dpkg/lock" \ "$rootdir/var/lib/apt/lists/lock" \ "$rootdir/etc/apt/apt.conf" \ "$rootdir/etc/apt/sources.list.d/"* \ "$rootdir/etc/apt/preferences.d/"* \ "$rootdir/etc/apt/sources.list" \ "$rootdir/var/cache/apt/archives/lock"; do if [ ! -e "$f" ]; then echo "does not exist: $f" >&2 continue fi if [ -d "$f" ]; then rmdir "$f" else rm "$f" fi done find "$rootdir" -depth -print0 | xargs -0 --no-run-if-empty rmdir } # note: this function uses brackets instead of curly braces, so that it's run # in its own process and we can handle traps independent from the outside update_cache() ( dist="$1" nativearch="$2" # use a subdirectory of $newcachedir so that we can use # hardlinks rootdir="$newcachedir/apt" mkdir -p "$rootdir" # we only set this trap here and overwrite the previous trap, because # the update_cache function is run as part of a pipe and thus in its # own process which will EXIT after it finished trap 'kill "$PROXYPID" || :;cleanupapt' EXIT INT TERM for p in /etc/apt/apt.conf.d /etc/apt/sources.list.d /etc/apt/preferences.d /var/cache/apt/archives /var/lib/apt/lists/partial /var/lib/dpkg; do mkdir -p "$rootdir/$p" done # read sources.list content from stdin cat >"$rootdir/etc/apt/sources.list" cat <"$rootdir/etc/apt/apt.conf" Apt::Architecture "$nativearch"; Apt::Architectures "$nativearch"; Dir::Etc "$rootdir/etc/apt"; Dir::State "$rootdir/var/lib/apt"; Dir::Cache "$rootdir/var/cache/apt"; Apt::Install-Recommends false; Apt::Get::Download-Only true; Acquire::Languages "none"; Dir::Etc::Trusted "/etc/apt/trusted.gpg"; Dir::Etc::TrustedParts "/etc/apt/trusted.gpg.d"; Acquire::http::Proxy "http://127.0.0.1:8080/"; END : >"$rootdir/var/lib/dpkg/status" if [ "$dist" = "$DEFAULT_DIST" ] && [ "$nativearch" = "$HOSTARCH" ] && [ "$USE_HOST_APT_CONFIG" = "yes" ]; then # we append sources and settings instead of overwriting after # an empty line for f in /etc/apt/sources.list /etc/apt/sources.list.d/*; do [ -e "$f" ] || continue [ -e "$rootdir/$f" ] && echo >>"$rootdir/$f" # Filter out file:// repositories as they are added # to each mmdebstrap call verbatim by # debian/tests/copy_host_apt_config # Also filter out all mirrors that are not of suite # $DEFAULT_DIST, except experimental if the suite # is unstable. This prevents packages from # unstable entering a testing mirror. if [ "$dist" = unstable ]; then grep -v ' file://' "$f" \ | grep -E " (unstable|experimental) " \ >>"$rootdir/$f" || : else grep -v ' file://' "$f" \ | grep " $DEFAULT_DIST " \ >>"$rootdir/$f" || : fi done for f in /etc/apt/preferences.d/*; do [ -e "$f" ] || continue [ -e "$rootdir/$f" ] && echo >>"$rootdir/$f" cat "$f" >>"$rootdir/$f" done fi echo "creating mirror for $dist" >&2 for f in /etc/apt/sources.list /etc/apt/sources.list.d/* /etc/apt/preferences.d/*; do [ -e "$rootdir/$f" ] || continue echo "contents of $f:" >&2 cat "$rootdir/$f" >&2 done APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get update --error-on=any pkgs=$(APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get indextargets \ --format '$(FILENAME)' 'Created-By: Packages' "Architecture: $nativearch" \ | xargs --delimiter='\n' /usr/lib/apt/apt-helper cat-file \ | grep-dctrl --no-field-names --show-field=Package --exact-match \ \( --field=Essential yes --or --field=Priority required \ --or --field=Priority important --or --field=Priority standard \ \)) pkgs="$pkgs build-essential busybox eatmydata fakechroot fakeroot" # we need usr-is-merged to simulate debootstrap behaviour for all dists # starting from Debian 12 (Bullseye) case "$dist" in oldstable) pkgs="$pkgs gpg" ;; stable) pkgs="$pkgs gpg usr-is-merged usrmerge" ;; testing | unstable) pkgs="$pkgs gpg-from-sq" ;; esac # shellcheck disable=SC2086 APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --yes install $pkgs \ || APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --yes install \ -oDebug::pkgProblemResolver=true -oDebug::pkgDepCache::Marker=1 \ -oDebug::pkgDepCache::AutoInstall=1 \ $pkgs rm "$rootdir/var/cache/apt/archives/lock" rmdir "$rootdir/var/cache/apt/archives/partial" APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get --option Dir::Etc::SourceList=/dev/null update APT_CONFIG="$rootdir/etc/apt/apt.conf" apt-get clean cleanupapt # this function is run in its own process, so we unset all traps before # returning trap "-" EXIT INT TERM ) check_proxy_running() { if timeout 1 bash -c 'exec 3<>/dev/tcp/127.0.0.1/8080 && printf "GET http://deb.debian.org/debian/dists/'"$DEFAULT_DIST"'/InRelease HTTP/1.1\nHost: deb.debian.org\n\n" >&3 && grep "Suite: '"$DEFAULT_DIST"'" <&3 >/dev/null' 2>/dev/null; then return 0 elif timeout 1 env http_proxy="http://127.0.0.1:8080/" wget --quiet -O - "http://deb.debian.org/debian/dists/$DEFAULT_DIST/InRelease" | grep "Suite: $DEFAULT_DIST" >/dev/null; then return 0 elif timeout 1 curl --proxy "http://127.0.0.1:8080/" --silent "http://deb.debian.org/debian/dists/$DEFAULT_DIST/InRelease" | grep "Suite: $DEFAULT_DIST" >/dev/null; then return 0 fi return 1 } if [ -e "./shared/cache.A" ] && [ -e "./shared/cache.B" ]; then echo "both ./shared/cache.A and ./shared/cache.B exist" >&2 echo "was a former run of the script aborted?" >&2 if [ -e ./shared/cache ]; then echo "cache symlink points to $(readlink ./shared/cache)" >&2 case "$(readlink ./shared/cache)" in cache.A) echo "removing ./shared/cache.B" >&2 rm -r ./shared/cache.B ;; cache.B) echo "removing ./shared/cache.A" >&2 rm -r ./shared/cache.A ;; *) echo "unexpected" >&2 exit 1 ;; esac else echo "./shared/cache doesn't exist" >&2 exit 1 fi fi if [ -e "./shared/cache.A" ]; then oldcache=cache.A newcache=cache.B else oldcache=cache.B newcache=cache.A fi oldcachedir="./shared/$oldcache" newcachedir="./shared/$newcache" oldmirrordir="$oldcachedir/debian" newmirrordir="$newcachedir/debian" mirror="http://deb.debian.org/debian" security_mirror="http://security.debian.org/debian-security" components=main : "${DEFAULT_DIST:=unstable}" : "${ONLY_DEFAULT_DIST:=no}" : "${ONLY_HOSTARCH:=no}" : "${HAVE_QEMU:=yes}" : "${RUN_MA_SAME_TESTS:=yes}" # by default, use the mmdebstrap executable in the current directory : "${CMD:=./mmdebstrap}" : "${USE_HOST_APT_CONFIG:=no}" : "${FORCE_UPDATE:=no}" if [ "$FORCE_UPDATE" != "yes" ] && [ -e "$oldmirrordir/dists/$DEFAULT_DIST/InRelease" ]; then http_code=$(curl --output /dev/null --silent --location --head --time-cond "$oldmirrordir/dists/$DEFAULT_DIST/InRelease" --write-out '%{http_code}' "$mirror/dists/$DEFAULT_DIST/InRelease") case "$http_code" in 200) ;; # need update 304) echo up-to-date exit 0 ;; *) echo "unexpected status: $http_code" exit 1 ;; esac fi ./caching_proxy.py "$oldcachedir" "$newcachedir" & PROXYPID=$! trap 'kill "$PROXYPID" || :' EXIT INT TERM for i in $(seq 10); do check_proxy_running && break sleep 1 done if [ ! -s "$newmirrordir/dists/$DEFAULT_DIST/InRelease" ]; then echo "failed to start proxy" >&2 kill $PROXYPID exit 1 fi trap 'kill "$PROXYPID" || :;cleanup_newcachedir' EXIT INT TERM mkdir -p "$newcachedir" touch "$newcachedir/mmdebstrapcache" HOSTARCH=$(dpkg --print-architecture) arches="$HOSTARCH" if [ "$HOSTARCH" = amd64 ]; then arches="$arches arm64 i386" elif [ "$HOSTARCH" = arm64 ]; then arches="$arches amd64 armhf" fi # we need the split_inline_sig() function # shellcheck disable=SC1091 . /usr/share/debootstrap/functions for dist in oldstable stable testing unstable; do for nativearch in $arches; do # non-host architectures are only downloaded for $DEFAULT_DIST if [ "$nativearch" != "$HOSTARCH" ] && [ "$DEFAULT_DIST" != "$dist" ]; then continue fi # if ONLY_DEFAULT_DIST is set, only download DEFAULT_DIST if [ "$ONLY_DEFAULT_DIST" = "yes" ] && [ "$DEFAULT_DIST" != "$dist" ]; then continue fi if [ "$ONLY_HOSTARCH" = "yes" ] && [ "$nativearch" != "$HOSTARCH" ]; then continue fi # we need a first pass without updates and security patches # because otherwise, old package versions needed by # debootstrap will not get included echo "deb [arch=$nativearch] $mirror $dist $components" | update_cache "$dist" "$nativearch" # we need to include the base mirror again or otherwise # packages like build-essential will be missing case "$dist" in oldstable | stable) cat <&2 continue fi rm "$f" done rmdir "$tmpdir" } SOURCE_DATE_EPOCH="$(date --date="$(grep-dctrl -s Date -n '' "$newmirrordir/dists/$DEFAULT_DIST/Release")" +%s)" export SOURCE_DATE_EPOCH if [ "$HAVE_QEMU" = "yes" ]; then # we use the caching proxy again when building the qemu image # - we can re-use the packages that were already downloaded earlier # - we make sure that the qemu image uses the same Release file even # if a mirror push happened between now and earlier # - we avoid polluting the mirror with the additional packages by # using --readonly ./caching_proxy.py --readonly "$oldcachedir" "$newcachedir" & PROXYPID=$! for i in $(seq 10); do check_proxy_running && break sleep 1 done if [ ! -s "$newmirrordir/dists/$DEFAULT_DIST/InRelease" ]; then echo "failed to start proxy" >&2 kill $PROXYPID exit 1 fi tmpdir="$(mktemp -d)" trap 'kill "$PROXYPID" || :;cleanuptmpdir; cleanup_newcachedir' EXIT INT TERM pkgs=perl-doc,systemd-sysv,perl,arch-test,fakechroot,fakeroot,mount,uidmap,qemu-user-binfmt,dpkg-dev,mini-httpd,libdevel-cover-perl,libtemplate-perl,debootstrap,procps,apt-cudf,aspcud,python3,libcap2-bin,gpg-from-sq,debootstrap,distro-info-data,iproute2,ubuntu-keyring,apt-utils,squashfs-tools-ng,genext2fs,linux-image-generic,passwd,e2fsprogs,uuid-runtime if [ ! -e ./mmdebstrap ]; then pkgs="$pkgs,mmdebstrap" fi arches=$HOSTARCH if [ "$RUN_MA_SAME_TESTS" = "yes" ]; then case "$HOSTARCH" in amd64) arches=amd64,arm64 pkgs="$pkgs,libfakechroot:arm64,libfakeroot:arm64" ;; arm64) arches=arm64,amd64 pkgs="$pkgs,libfakechroot:amd64,libfakeroot:amd64" ;; esac fi cat <"$tmpdir/mmdebstrap.service" [Unit] Description=mmdebstrap worker script [Service] Type=oneshot ExecStart=/worker.sh [Install] WantedBy=multi-user.target END # here is something crazy: # as we run mmdebstrap, the process ends up being run by different users with # different privileges (real or fake). But for being able to collect # Devel::Cover data, they must all share a single directory. The only way that # I found to make this work is to mount the database directory with a # filesystem that doesn't support ownership information at all and a umask that # gives read/write access to everybody. # https://github.com/pjcj/Devel--Cover/issues/223 cat <<'END' >"$tmpdir/worker.sh" #!/bin/sh echo 'root:root' | chpasswd mount -t 9p -o trans=virtio,access=any,msize=128k mmdebstrap /mnt # need to restart mini-httpd because we mounted different content into www-root systemctl restart mini-httpd ip link set enp0s1 down || : handler () { while IFS= read -r line || [ -n "$line" ]; do printf "%s %s: %s\n" "$(date -u -d "0 $(date +%s.%3N) seconds - $2 seconds" +"%T.%3N")" "$1" "$line" done } ( cd /mnt; if [ -e cover_db.img ]; then mkdir -p cover_db mount -o loop,umask=000 cover_db.img cover_db fi now=$(date +%s.%3N) ret=0 { { { { { sh -x ./test.sh 2>&1 1>&4 3>&- 4>&-; echo $? >&2; } | handler E "$now" >&3; } 4>&1 | handler O "$now" >&3; } 2>&1; } | { read xs; exit $xs; }; } 3>&1 || ret=$? echo $ret > /mnt/exitstatus.txt if [ -e cover_db.img ]; then df -h cover_db umount cover_db fi ) > /mnt/output.txt 2>&1 umount /mnt systemctl poweroff END chmod +x "$tmpdir/worker.sh" if [ -z ${DISK_SIZE+x} ]; then DISK_SIZE=10G fi # set PATH to pick up the correct mmdebstrap variant env PATH="$(dirname "$(realpath --canonicalize-existing "$CMD")"):$PATH" \ debvm-create --skip=usrmerge,systemdnetwork \ --size="$DISK_SIZE" --release="$DEFAULT_DIST" \ --output="$newcachedir/debian-$DEFAULT_DIST.ext4" -- \ --architectures="$arches" --include="$pkgs" \ --setup-hook='echo "Acquire::http::Proxy \"http://127.0.0.1:8080/\";" > "$1/etc/apt/apt.conf.d/00proxy"' \ --hook-dir=/usr/share/mmdebstrap/hooks/maybe-merged-usr \ --customize-hook='rm "$1/etc/apt/apt.conf.d/00proxy"' \ --customize-hook='mkdir -p "$1/etc/systemd/system/multi-user.target.wants"' \ --customize-hook='ln -s ../mmdebstrap.service "$1/etc/systemd/system/multi-user.target.wants/mmdebstrap.service"' \ --customize-hook='touch "$1/mmdebstrap-testenv"' \ --customize-hook='copy-in "'"$tmpdir"'/mmdebstrap.service" /etc/systemd/system/' \ --customize-hook='copy-in "'"$tmpdir"'/worker.sh" /' \ --customize-hook='echo 127.0.0.1 localhost > "$1/etc/hosts"' \ --customize-hook='printf "START=1\nDAEMON_OPTS=\"-h 127.0.0.1 -p 80 -u nobody -dd /mnt/cache -i /var/run/mini-httpd.pid -T UTF-8\"\n" > "$1/etc/default/mini-httpd"' \ --customize-hook='touch "$1/etc/systemd/system/tmp.mount"' \ "$mirror" kill $PROXYPID cleanuptmpdir trap "cleanup_newcachedir" EXIT INT TERM fi # delete possibly leftover symlink if [ -e ./shared/cache.tmp ]; then rm ./shared/cache.tmp fi # now atomically switch the symlink to point to the other directory ln -s $newcache ./shared/cache.tmp mv --no-target-directory ./shared/cache.tmp ./shared/cache deletecache "$oldcachedir" trap - EXIT INT TERM echo "$0 finished successfully" >&2 mmdebstrap/mmdebstrap000077500000000000000000014244421477517524100153300ustar00rootroot00000000000000#!/usr/bin/perl # # © 2018 - 2023 Johannes Schauer Marin Rodrigues # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to # deal in the Software without restriction, including without limitation the # rights to use, copy, modify, merge, publish, distribute, sublicense, and/or # sell copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. # # The software is provided "as is", without warranty of any kind, express or # implied, including but not limited to the warranties of merchantability, # fitness for a particular purpose and noninfringement. In no event shall the # authors or copyright holders be liable for any claim, damages or other # liability, whether in an action of contract, tort or otherwise, arising # from, out of or in connection with the software or the use or other dealings # in the software. use strict; use warnings; our $VERSION = '1.5.7'; use English; use Getopt::Long; use Pod::Usage; use File::Copy; use File::Path qw(make_path); use File::Temp qw(tempfile tempdir); use File::Basename; use File::Find; use Cwd qw(abs_path getcwd); require "syscall.ph"; ## no critic (Modules::RequireBarewordIncludes) require "sys/ioctl.ph"; ## no critic (Modules::RequireBarewordIncludes) use Fcntl qw(S_IFCHR S_IFBLK FD_CLOEXEC F_GETFD F_SETFD LOCK_EX O_RDONLY O_DIRECTORY); use List::Util qw(any none); use POSIX qw(SIGINT SIGHUP SIGPIPE SIGTERM SIG_BLOCK SIG_UNBLOCK strftime isatty WIFEXITED WEXITSTATUS WIFSIGNALED WTERMSIG); use Carp; use Term::ANSIColor; use Socket; use Time::HiRes; use Math::BigInt; use Text::ParseWords; use Digest::SHA; use version; ## no critic (InputOutput::RequireBriefOpen) # from sched.h # use typeglob constants because "use constant" has several drawback as # explained in the documentation for the Readonly CPAN module *CLONE_NEWNS = \0x20000; # mount namespace *CLONE_NEWUTS = \0x4000000; # utsname *CLONE_NEWIPC = \0x8000000; # ipc *CLONE_NEWUSER = \0x10000000; # user *CLONE_NEWPID = \0x20000000; # pid *CLONE_NEWNET = \0x40000000; # net *_LINUX_CAPABILITY_VERSION_3 = \0x20080522; *CAP_SYS_ADMIN = \21; *PR_CAPBSET_READ = \23; # from sys/mount.h *MS_BIND = \0x1000; *MS_REC = \0x4000; *MNT_DETACH = \2; # uuid_t NameSpace_DNS in rfc4122 *UUID_NS_DNS = \'6ba7b810-9dad-11d1-80b4-00c04fd430c8'; our ( $CLONE_NEWNS, $CLONE_NEWUTS, $CLONE_NEWIPC, $CLONE_NEWUSER, $CLONE_NEWPID, $CLONE_NEWNET, $_LINUX_CAPABILITY_VERSION_3, $CAP_SYS_ADMIN, $PR_CAPBSET_READ, $MS_BIND, $MS_REC, $MNT_DETACH, $UUID_NS_DNS ); #<<< # type codes: # 0 -> normal file # 1 -> hardlink # 2 -> symlink # 3 -> character special # 4 -> block special # 5 -> directory my @linuxdevfiles = ( # file name mode type link target major minor transl. ["./dev/", oct(755), '5', undef, undef, undef, undef], ["./dev/console", oct(666), '3', undef, 5, 1, undef], ["./dev/fd", oct(777), '2', '/proc/self/fd', undef, undef, undef], ["./dev/full", oct(666), '3', undef, 1, 7, undef], ["./dev/null", oct(666), '3', undef, 1, 3, undef], ["./dev/ptmx", oct(666), '3', undef, 5, 2, undef], ["./dev/pts/", oct(755), '5', undef, undef, undef, undef], ["./dev/random", oct(666), '3', undef, 1, 8, undef], ["./dev/shm/", oct(755), '5', undef, undef, undef, undef], ["./dev/stderr", oct(777), '2', '/proc/self/fd/2', undef, undef, undef], ["./dev/stdin", oct(777), '2', '/proc/self/fd/0', undef, undef, undef], ["./dev/stdout", oct(777), '2', '/proc/self/fd/1', undef, undef, undef], ["./dev/tty", oct(666), '3', undef, 5, 0, undef], ["./dev/urandom", oct(666), '3', undef, 1, 9, undef], ["./dev/zero", oct(666), '3', undef, 1, 5, undef], ); my @hurdfiles = ( # file name mode type link target major minor transl. ['./dev/', oct(755), '5', undef, undef, undef, undef], ['./dev/MAKEDEV', oct(755), '2', '/sbin/MAKEDEV', undef, undef, undef], ['./dev/cd0', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:cd0\0"], ['./dev/cd1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:cd1\0"], ['./dev/com0', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/com0\0device\0com0\0"], ['./dev/com1', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/com1\0device\0com1\0"], ['./dev/com2', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/com2\0device\0com2\0"], ['./dev/com3', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/com3\0device\0com3\0"], ['./dev/cons', oct(600), '0', undef, undef, undef, undef], ['./dev/console', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/console\0device\0console\0"], ['./dev/disk', oct(755), '2', 'rumpdisk', undef, undef, undef], ['./dev/eth0', oct(660), '0', undef, undef, undef, "/hurd/devnode\0-M\0/dev/net\0eth0\0"], ['./dev/eth1', oct(660), '0', undef, undef, undef, "/hurd/devnode\0-M\0/dev/net\0eth1\0"], ['./dev/eth2', oct(660), '0', undef, undef, undef, "/hurd/devnode\0-M\0/dev/net\0eth2\0"], ['./dev/eth3', oct(660), '0', undef, undef, undef, "/hurd/devnode\0-M\0/dev/net\0eth3\0"], ['./dev/fd', oct(666), '0', undef, undef, undef, "/hurd/magic\0--directory\0fd\0"], ['./dev/fd0', oct(640), '0', undef, undef, undef, "/hurd/storeio\0fd0\0"], ['./dev/fd1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0fd1\0"], ['./dev/full', oct(666), '0', undef, undef, undef, "/hurd/null\0--full\0"], ['./dev/hd0', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0\0"], ['./dev/hd0s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s1\0"], ['./dev/hd0s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s10\0"], ['./dev/hd0s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s11\0"], ['./dev/hd0s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s12\0"], ['./dev/hd0s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s13\0"], ['./dev/hd0s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s14\0"], ['./dev/hd0s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s15\0"], ['./dev/hd0s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s16\0"], ['./dev/hd0s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s2\0"], ['./dev/hd0s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s3\0"], ['./dev/hd0s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s4\0"], ['./dev/hd0s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s5\0"], ['./dev/hd0s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s6\0"], ['./dev/hd0s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s7\0"], ['./dev/hd0s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s8\0"], ['./dev/hd0s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd0s9\0"], ['./dev/hd1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1\0"], ['./dev/hd1s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s1\0"], ['./dev/hd1s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s10\0"], ['./dev/hd1s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s11\0"], ['./dev/hd1s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s12\0"], ['./dev/hd1s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s13\0"], ['./dev/hd1s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s14\0"], ['./dev/hd1s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s15\0"], ['./dev/hd1s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s16\0"], ['./dev/hd1s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s2\0"], ['./dev/hd1s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s3\0"], ['./dev/hd1s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s4\0"], ['./dev/hd1s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s5\0"], ['./dev/hd1s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s6\0"], ['./dev/hd1s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s7\0"], ['./dev/hd1s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s8\0"], ['./dev/hd1s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd1s9\0"], ['./dev/hd2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2\0"], ['./dev/hd2s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s1\0"], ['./dev/hd2s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s10\0"], ['./dev/hd2s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s11\0"], ['./dev/hd2s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s12\0"], ['./dev/hd2s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s13\0"], ['./dev/hd2s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s14\0"], ['./dev/hd2s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s15\0"], ['./dev/hd2s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s16\0"], ['./dev/hd2s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s2\0"], ['./dev/hd2s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s3\0"], ['./dev/hd2s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s4\0"], ['./dev/hd2s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s5\0"], ['./dev/hd2s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s6\0"], ['./dev/hd2s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s7\0"], ['./dev/hd2s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s8\0"], ['./dev/hd2s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd2s9\0"], ['./dev/hd3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3\0"], ['./dev/hd3s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s1\0"], ['./dev/hd3s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s10\0"], ['./dev/hd3s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s11\0"], ['./dev/hd3s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s12\0"], ['./dev/hd3s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s13\0"], ['./dev/hd3s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s14\0"], ['./dev/hd3s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s15\0"], ['./dev/hd3s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s16\0"], ['./dev/hd3s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s2\0"], ['./dev/hd3s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s3\0"], ['./dev/hd3s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s4\0"], ['./dev/hd3s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s5\0"], ['./dev/hd3s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s6\0"], ['./dev/hd3s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s7\0"], ['./dev/hd3s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s8\0"], ['./dev/hd3s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd3s9\0"], ['./dev/hd4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4\0"], ['./dev/hd4s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s1\0"], ['./dev/hd4s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s10\0"], ['./dev/hd4s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s11\0"], ['./dev/hd4s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s12\0"], ['./dev/hd4s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s13\0"], ['./dev/hd4s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s14\0"], ['./dev/hd4s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s15\0"], ['./dev/hd4s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s16\0"], ['./dev/hd4s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s2\0"], ['./dev/hd4s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s3\0"], ['./dev/hd4s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s4\0"], ['./dev/hd4s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s5\0"], ['./dev/hd4s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s6\0"], ['./dev/hd4s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s7\0"], ['./dev/hd4s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s8\0"], ['./dev/hd4s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd4s9\0"], ['./dev/hd5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5\0"], ['./dev/hd5s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s1\0"], ['./dev/hd5s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s10\0"], ['./dev/hd5s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s11\0"], ['./dev/hd5s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s12\0"], ['./dev/hd5s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s13\0"], ['./dev/hd5s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s14\0"], ['./dev/hd5s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s15\0"], ['./dev/hd5s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s16\0"], ['./dev/hd5s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s2\0"], ['./dev/hd5s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s3\0"], ['./dev/hd5s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s4\0"], ['./dev/hd5s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s5\0"], ['./dev/hd5s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s6\0"], ['./dev/hd5s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s7\0"], ['./dev/hd5s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s8\0"], ['./dev/hd5s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0hd5s9\0"], ['./dev/kbd', oct(644), '2', 'cons/kbd', undef, undef, undef], ['./dev/klog', oct(660), '0', undef, undef, undef, "/hurd/streamio\0kmsg\0"], ['./dev/loop0', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop1', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop2', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop3', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop4', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop5', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop6', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/loop7', oct(640), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/lpr0', oct(660), '0', undef, undef, undef, "/hurd/streamio\0lpr0\0"], ['./dev/lpr1', oct(660), '0', undef, undef, undef, "/hurd/streamio\0lpr1\0"], ['./dev/lpr2', oct(660), '0', undef, undef, undef, "/hurd/streamio\0lpr2\0"], ['./dev/mem', oct(660), '0', undef, undef, undef, "/hurd/storeio\0--no-cache\0mem\0"], ['./dev/mouse', oct(644), '2', 'cons/mouse', undef, undef, undef], ['./dev/net', oct(755), '2', 'netdde', undef, undef, undef], ['./dev/netdde', oct(660), '0', undef, undef, undef, "/hurd/netdde\0"], ['./dev/null', oct(666), '0', undef, undef, undef, "/hurd/null\0"], ['./dev/pseudo-root', oct(640), '4', undef, 0, 0, "/hurd/storeio\0pseudo-root\0"], ['./dev/ptyp0', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp0\0pty-master\0/dev/ttyp0\0"], ['./dev/ptyp1', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp1\0pty-master\0/dev/ttyp1\0"], ['./dev/ptyp2', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp2\0pty-master\0/dev/ttyp2\0"], ['./dev/ptyp3', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp3\0pty-master\0/dev/ttyp3\0"], ['./dev/ptyp4', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp4\0pty-master\0/dev/ttyp4\0"], ['./dev/ptyp5', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp5\0pty-master\0/dev/ttyp5\0"], ['./dev/ptyp6', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp6\0pty-master\0/dev/ttyp6\0"], ['./dev/ptyp7', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp7\0pty-master\0/dev/ttyp7\0"], ['./dev/ptyp8', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp8\0pty-master\0/dev/ttyp8\0"], ['./dev/ptyp9', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyp9\0pty-master\0/dev/ttyp9\0"], ['./dev/ptypa', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypa\0pty-master\0/dev/ttypa\0"], ['./dev/ptypb', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypb\0pty-master\0/dev/ttypb\0"], ['./dev/ptypc', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypc\0pty-master\0/dev/ttypc\0"], ['./dev/ptypd', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypd\0pty-master\0/dev/ttypd\0"], ['./dev/ptype', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptype\0pty-master\0/dev/ttype\0"], ['./dev/ptypf', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypf\0pty-master\0/dev/ttypf\0"], ['./dev/ptypg', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypg\0pty-master\0/dev/ttypg\0"], ['./dev/ptyph', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyph\0pty-master\0/dev/ttyph\0"], ['./dev/ptypi', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypi\0pty-master\0/dev/ttypi\0"], ['./dev/ptypj', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypj\0pty-master\0/dev/ttypj\0"], ['./dev/ptypk', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypk\0pty-master\0/dev/ttypk\0"], ['./dev/ptypl', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypl\0pty-master\0/dev/ttypl\0"], ['./dev/ptypm', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypm\0pty-master\0/dev/ttypm\0"], ['./dev/ptypn', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypn\0pty-master\0/dev/ttypn\0"], ['./dev/ptypo', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypo\0pty-master\0/dev/ttypo\0"], ['./dev/ptypp', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypp\0pty-master\0/dev/ttypp\0"], ['./dev/ptypq', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypq\0pty-master\0/dev/ttypq\0"], ['./dev/ptypr', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypr\0pty-master\0/dev/ttypr\0"], ['./dev/ptyps', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyps\0pty-master\0/dev/ttyps\0"], ['./dev/ptypt', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypt\0pty-master\0/dev/ttypt\0"], ['./dev/ptypu', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypu\0pty-master\0/dev/ttypu\0"], ['./dev/ptypv', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptypv\0pty-master\0/dev/ttypv\0"], ['./dev/ptyq0', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq0\0pty-master\0/dev/ttyq0\0"], ['./dev/ptyq1', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq1\0pty-master\0/dev/ttyq1\0"], ['./dev/ptyq2', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq2\0pty-master\0/dev/ttyq2\0"], ['./dev/ptyq3', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq3\0pty-master\0/dev/ttyq3\0"], ['./dev/ptyq4', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq4\0pty-master\0/dev/ttyq4\0"], ['./dev/ptyq5', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq5\0pty-master\0/dev/ttyq5\0"], ['./dev/ptyq6', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq6\0pty-master\0/dev/ttyq6\0"], ['./dev/ptyq7', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq7\0pty-master\0/dev/ttyq7\0"], ['./dev/ptyq8', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq8\0pty-master\0/dev/ttyq8\0"], ['./dev/ptyq9', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyq9\0pty-master\0/dev/ttyq9\0"], ['./dev/ptyqa', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqa\0pty-master\0/dev/ttyqa\0"], ['./dev/ptyqb', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqb\0pty-master\0/dev/ttyqb\0"], ['./dev/ptyqc', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqc\0pty-master\0/dev/ttyqc\0"], ['./dev/ptyqd', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqd\0pty-master\0/dev/ttyqd\0"], ['./dev/ptyqe', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqe\0pty-master\0/dev/ttyqe\0"], ['./dev/ptyqf', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqf\0pty-master\0/dev/ttyqf\0"], ['./dev/ptyqg', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqg\0pty-master\0/dev/ttyqg\0"], ['./dev/ptyqh', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqh\0pty-master\0/dev/ttyqh\0"], ['./dev/ptyqi', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqi\0pty-master\0/dev/ttyqi\0"], ['./dev/ptyqj', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqj\0pty-master\0/dev/ttyqj\0"], ['./dev/ptyqk', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqk\0pty-master\0/dev/ttyqk\0"], ['./dev/ptyql', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyql\0pty-master\0/dev/ttyql\0"], ['./dev/ptyqm', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqm\0pty-master\0/dev/ttyqm\0"], ['./dev/ptyqn', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqn\0pty-master\0/dev/ttyqn\0"], ['./dev/ptyqo', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqo\0pty-master\0/dev/ttyqo\0"], ['./dev/ptyqp', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqp\0pty-master\0/dev/ttyqp\0"], ['./dev/ptyqq', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqq\0pty-master\0/dev/ttyqq\0"], ['./dev/ptyqr', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqr\0pty-master\0/dev/ttyqr\0"], ['./dev/ptyqs', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqs\0pty-master\0/dev/ttyqs\0"], ['./dev/ptyqt', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqt\0pty-master\0/dev/ttyqt\0"], ['./dev/ptyqu', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqu\0pty-master\0/dev/ttyqu\0"], ['./dev/ptyqv', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ptyqv\0pty-master\0/dev/ttyqv\0"], ['./dev/random', oct(644), '0', undef, undef, undef, "/hurd/random\0--seed-file\0/var/lib/random-seed\0"], ['./dev/rumpdisk', oct(660), '0', undef, undef, undef, "/hurd/rumpdisk\0"], ['./dev/sd0', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0\0"], ['./dev/sd0s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s1\0"], ['./dev/sd0s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s10\0"], ['./dev/sd0s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s11\0"], ['./dev/sd0s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s12\0"], ['./dev/sd0s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s13\0"], ['./dev/sd0s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s14\0"], ['./dev/sd0s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s15\0"], ['./dev/sd0s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s16\0"], ['./dev/sd0s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s2\0"], ['./dev/sd0s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s3\0"], ['./dev/sd0s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s4\0"], ['./dev/sd0s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s5\0"], ['./dev/sd0s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s6\0"], ['./dev/sd0s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s7\0"], ['./dev/sd0s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s8\0"], ['./dev/sd0s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd0s9\0"], ['./dev/sd1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1\0"], ['./dev/sd1s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s1\0"], ['./dev/sd1s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s10\0"], ['./dev/sd1s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s11\0"], ['./dev/sd1s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s12\0"], ['./dev/sd1s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s13\0"], ['./dev/sd1s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s14\0"], ['./dev/sd1s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s15\0"], ['./dev/sd1s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s16\0"], ['./dev/sd1s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s2\0"], ['./dev/sd1s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s3\0"], ['./dev/sd1s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s4\0"], ['./dev/sd1s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s5\0"], ['./dev/sd1s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s6\0"], ['./dev/sd1s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s7\0"], ['./dev/sd1s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s8\0"], ['./dev/sd1s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd1s9\0"], ['./dev/sd2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2\0"], ['./dev/sd2s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s1\0"], ['./dev/sd2s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s10\0"], ['./dev/sd2s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s11\0"], ['./dev/sd2s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s12\0"], ['./dev/sd2s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s13\0"], ['./dev/sd2s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s14\0"], ['./dev/sd2s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s15\0"], ['./dev/sd2s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s16\0"], ['./dev/sd2s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s2\0"], ['./dev/sd2s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s3\0"], ['./dev/sd2s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s4\0"], ['./dev/sd2s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s5\0"], ['./dev/sd2s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s6\0"], ['./dev/sd2s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s7\0"], ['./dev/sd2s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s8\0"], ['./dev/sd2s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd2s9\0"], ['./dev/sd3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3\0"], ['./dev/sd3s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s1\0"], ['./dev/sd3s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s10\0"], ['./dev/sd3s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s11\0"], ['./dev/sd3s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s12\0"], ['./dev/sd3s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s13\0"], ['./dev/sd3s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s14\0"], ['./dev/sd3s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s15\0"], ['./dev/sd3s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s16\0"], ['./dev/sd3s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s2\0"], ['./dev/sd3s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s3\0"], ['./dev/sd3s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s4\0"], ['./dev/sd3s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s5\0"], ['./dev/sd3s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s6\0"], ['./dev/sd3s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s7\0"], ['./dev/sd3s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s8\0"], ['./dev/sd3s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd3s9\0"], ['./dev/sd4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4\0"], ['./dev/sd4s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s1\0"], ['./dev/sd4s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s10\0"], ['./dev/sd4s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s11\0"], ['./dev/sd4s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s12\0"], ['./dev/sd4s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s13\0"], ['./dev/sd4s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s14\0"], ['./dev/sd4s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s15\0"], ['./dev/sd4s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s16\0"], ['./dev/sd4s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s2\0"], ['./dev/sd4s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s3\0"], ['./dev/sd4s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s4\0"], ['./dev/sd4s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s5\0"], ['./dev/sd4s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s6\0"], ['./dev/sd4s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s7\0"], ['./dev/sd4s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s8\0"], ['./dev/sd4s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd4s9\0"], ['./dev/sd5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5\0"], ['./dev/sd5s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s1\0"], ['./dev/sd5s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s10\0"], ['./dev/sd5s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s11\0"], ['./dev/sd5s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s12\0"], ['./dev/sd5s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s13\0"], ['./dev/sd5s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s14\0"], ['./dev/sd5s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s15\0"], ['./dev/sd5s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s16\0"], ['./dev/sd5s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s2\0"], ['./dev/sd5s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s3\0"], ['./dev/sd5s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s4\0"], ['./dev/sd5s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s5\0"], ['./dev/sd5s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s6\0"], ['./dev/sd5s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s7\0"], ['./dev/sd5s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s8\0"], ['./dev/sd5s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0sd5s9\0"], ['./dev/shm', oct(644), '2', '/tmp', undef, undef, undef], ['./dev/stderr', oct(755), '2', 'fd/2', undef, undef, undef], ['./dev/stdin', oct(755), '2', 'fd/0', undef, undef, undef], ['./dev/stdout', oct(755), '2', 'fd/1', undef, undef, undef], ['./dev/time', oct(644), '0', undef, undef, undef, "/hurd/storeio\0--no-cache\0time\0"], ['./dev/tty', oct(666), '0', undef, undef, undef, "/hurd/magic\0tty\0"], ['./dev/tty1', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty1\0hurdio\0/dev/vcs/1/console\0"], ['./dev/tty2', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty2\0hurdio\0/dev/vcs/2/console\0"], ['./dev/tty3', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty3\0hurdio\0/dev/vcs/3/console\0"], ['./dev/tty4', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty4\0hurdio\0/dev/vcs/4/console\0"], ['./dev/tty5', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty5\0hurdio\0/dev/vcs/5/console\0"], ['./dev/tty6', oct(600), '0', undef, undef, undef, "/hurd/term\0/dev/tty6\0hurdio\0/dev/vcs/6/console\0"], ['./dev/ttyp0', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp0\0pty-slave\0/dev/ptyp0\0"], ['./dev/ttyp1', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp1\0pty-slave\0/dev/ptyp1\0"], ['./dev/ttyp2', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp2\0pty-slave\0/dev/ptyp2\0"], ['./dev/ttyp3', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp3\0pty-slave\0/dev/ptyp3\0"], ['./dev/ttyp4', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp4\0pty-slave\0/dev/ptyp4\0"], ['./dev/ttyp5', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp5\0pty-slave\0/dev/ptyp5\0"], ['./dev/ttyp6', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp6\0pty-slave\0/dev/ptyp6\0"], ['./dev/ttyp7', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp7\0pty-slave\0/dev/ptyp7\0"], ['./dev/ttyp8', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp8\0pty-slave\0/dev/ptyp8\0"], ['./dev/ttyp9', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyp9\0pty-slave\0/dev/ptyp9\0"], ['./dev/ttypa', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypa\0pty-slave\0/dev/ptypa\0"], ['./dev/ttypb', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypb\0pty-slave\0/dev/ptypb\0"], ['./dev/ttypc', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypc\0pty-slave\0/dev/ptypc\0"], ['./dev/ttypd', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypd\0pty-slave\0/dev/ptypd\0"], ['./dev/ttype', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttype\0pty-slave\0/dev/ptype\0"], ['./dev/ttypf', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypf\0pty-slave\0/dev/ptypf\0"], ['./dev/ttypg', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypg\0pty-slave\0/dev/ptypg\0"], ['./dev/ttyph', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyph\0pty-slave\0/dev/ptyph\0"], ['./dev/ttypi', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypi\0pty-slave\0/dev/ptypi\0"], ['./dev/ttypj', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypj\0pty-slave\0/dev/ptypj\0"], ['./dev/ttypk', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypk\0pty-slave\0/dev/ptypk\0"], ['./dev/ttypl', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypl\0pty-slave\0/dev/ptypl\0"], ['./dev/ttypm', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypm\0pty-slave\0/dev/ptypm\0"], ['./dev/ttypn', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypn\0pty-slave\0/dev/ptypn\0"], ['./dev/ttypo', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypo\0pty-slave\0/dev/ptypo\0"], ['./dev/ttypp', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypp\0pty-slave\0/dev/ptypp\0"], ['./dev/ttypq', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypq\0pty-slave\0/dev/ptypq\0"], ['./dev/ttypr', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypr\0pty-slave\0/dev/ptypr\0"], ['./dev/ttyps', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyps\0pty-slave\0/dev/ptyps\0"], ['./dev/ttypt', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypt\0pty-slave\0/dev/ptypt\0"], ['./dev/ttypu', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypu\0pty-slave\0/dev/ptypu\0"], ['./dev/ttypv', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttypv\0pty-slave\0/dev/ptypv\0"], ['./dev/ttyq0', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq0\0pty-slave\0/dev/ptyq0\0"], ['./dev/ttyq1', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq1\0pty-slave\0/dev/ptyq1\0"], ['./dev/ttyq2', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq2\0pty-slave\0/dev/ptyq2\0"], ['./dev/ttyq3', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq3\0pty-slave\0/dev/ptyq3\0"], ['./dev/ttyq4', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq4\0pty-slave\0/dev/ptyq4\0"], ['./dev/ttyq5', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq5\0pty-slave\0/dev/ptyq5\0"], ['./dev/ttyq6', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq6\0pty-slave\0/dev/ptyq6\0"], ['./dev/ttyq7', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq7\0pty-slave\0/dev/ptyq7\0"], ['./dev/ttyq8', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq8\0pty-slave\0/dev/ptyq8\0"], ['./dev/ttyq9', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyq9\0pty-slave\0/dev/ptyq9\0"], ['./dev/ttyqa', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqa\0pty-slave\0/dev/ptyqa\0"], ['./dev/ttyqb', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqb\0pty-slave\0/dev/ptyqb\0"], ['./dev/ttyqc', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqc\0pty-slave\0/dev/ptyqc\0"], ['./dev/ttyqd', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqd\0pty-slave\0/dev/ptyqd\0"], ['./dev/ttyqe', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqe\0pty-slave\0/dev/ptyqe\0"], ['./dev/ttyqf', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqf\0pty-slave\0/dev/ptyqf\0"], ['./dev/ttyqg', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqg\0pty-slave\0/dev/ptyqg\0"], ['./dev/ttyqh', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqh\0pty-slave\0/dev/ptyqh\0"], ['./dev/ttyqi', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqi\0pty-slave\0/dev/ptyqi\0"], ['./dev/ttyqj', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqj\0pty-slave\0/dev/ptyqj\0"], ['./dev/ttyqk', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqk\0pty-slave\0/dev/ptyqk\0"], ['./dev/ttyql', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyql\0pty-slave\0/dev/ptyql\0"], ['./dev/ttyqm', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqm\0pty-slave\0/dev/ptyqm\0"], ['./dev/ttyqn', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqn\0pty-slave\0/dev/ptyqn\0"], ['./dev/ttyqo', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqo\0pty-slave\0/dev/ptyqo\0"], ['./dev/ttyqp', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqp\0pty-slave\0/dev/ptyqp\0"], ['./dev/ttyqq', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqq\0pty-slave\0/dev/ptyqq\0"], ['./dev/ttyqr', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqr\0pty-slave\0/dev/ptyqr\0"], ['./dev/ttyqs', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqs\0pty-slave\0/dev/ptyqs\0"], ['./dev/ttyqt', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqt\0pty-slave\0/dev/ptyqt\0"], ['./dev/ttyqu', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqu\0pty-slave\0/dev/ptyqu\0"], ['./dev/ttyqv', oct(666), '0', undef, undef, undef, "/hurd/term\0/dev/ttyqv\0pty-slave\0/dev/ptyqv\0"], ['./dev/urandom', oct(755), '2', 'random', undef, undef, "/hurd/random\0--seed-file\0/var/lib/random-seed\0--fast\0"], ['./dev/vcs', oct(600), '0', undef, undef, undef, "/hurd/console\0"], ['./dev/wd0', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd0\0"], ['./dev/wd0s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd0\0"], ['./dev/wd0s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd0\0"], ['./dev/wd0s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd0\0"], ['./dev/wd0s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd0\0"], ['./dev/wd0s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd0\0"], ['./dev/wd0s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd0\0"], ['./dev/wd0s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd0\0"], ['./dev/wd0s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd0\0"], ['./dev/wd0s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd0\0"], ['./dev/wd0s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd0\0"], ['./dev/wd0s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd0\0"], ['./dev/wd0s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd0\0"], ['./dev/wd0s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd0\0"], ['./dev/wd0s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd0\0"], ['./dev/wd0s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd0\0"], ['./dev/wd0s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd0\0"], ['./dev/wd1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd1\0"], ['./dev/wd1s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd1\0"], ['./dev/wd1s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd1\0"], ['./dev/wd1s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd1\0"], ['./dev/wd1s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd1\0"], ['./dev/wd1s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd1\0"], ['./dev/wd1s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd1\0"], ['./dev/wd1s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd1\0"], ['./dev/wd1s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd1\0"], ['./dev/wd1s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd1\0"], ['./dev/wd1s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd1\0"], ['./dev/wd1s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd1\0"], ['./dev/wd1s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd1\0"], ['./dev/wd1s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd1\0"], ['./dev/wd1s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd1\0"], ['./dev/wd1s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd1\0"], ['./dev/wd1s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd1\0"], ['./dev/wd2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd2\0"], ['./dev/wd2s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd2\0"], ['./dev/wd2s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd2\0"], ['./dev/wd2s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd2\0"], ['./dev/wd2s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd2\0"], ['./dev/wd2s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd2\0"], ['./dev/wd2s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd2\0"], ['./dev/wd2s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd2\0"], ['./dev/wd2s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd2\0"], ['./dev/wd2s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd2\0"], ['./dev/wd2s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd2\0"], ['./dev/wd2s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd2\0"], ['./dev/wd2s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd2\0"], ['./dev/wd2s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd2\0"], ['./dev/wd2s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd2\0"], ['./dev/wd2s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd2\0"], ['./dev/wd2s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd2\0"], ['./dev/wd3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd3\0"], ['./dev/wd3s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd3\0"], ['./dev/wd3s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd3\0"], ['./dev/wd3s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd3\0"], ['./dev/wd3s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd3\0"], ['./dev/wd3s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd3\0"], ['./dev/wd3s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd3\0"], ['./dev/wd3s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd3\0"], ['./dev/wd3s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd3\0"], ['./dev/wd3s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd3\0"], ['./dev/wd3s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd3\0"], ['./dev/wd3s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd3\0"], ['./dev/wd3s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd3\0"], ['./dev/wd3s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd3\0"], ['./dev/wd3s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd3\0"], ['./dev/wd3s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd3\0"], ['./dev/wd3s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd3\0"], ['./dev/wd4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd4\0"], ['./dev/wd4s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd4\0"], ['./dev/wd4s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd4\0"], ['./dev/wd4s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd4\0"], ['./dev/wd4s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd4\0"], ['./dev/wd4s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd4\0"], ['./dev/wd4s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd4\0"], ['./dev/wd4s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd4\0"], ['./dev/wd4s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd4\0"], ['./dev/wd4s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd4\0"], ['./dev/wd4s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd4\0"], ['./dev/wd4s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd4\0"], ['./dev/wd4s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd4\0"], ['./dev/wd4s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd4\0"], ['./dev/wd4s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd4\0"], ['./dev/wd4s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd4\0"], ['./dev/wd4s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd4\0"], ['./dev/wd5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0@/dev/disk:wd5\0"], ['./dev/wd5s1', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:1:device:@/dev/disk:wd5\0"], ['./dev/wd5s10', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:10:device:@/dev/disk:wd5\0"], ['./dev/wd5s11', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:11:device:@/dev/disk:wd5\0"], ['./dev/wd5s12', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:12:device:@/dev/disk:wd5\0"], ['./dev/wd5s13', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:13:device:@/dev/disk:wd5\0"], ['./dev/wd5s14', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:14:device:@/dev/disk:wd5\0"], ['./dev/wd5s15', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:15:device:@/dev/disk:wd5\0"], ['./dev/wd5s16', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:16:device:@/dev/disk:wd5\0"], ['./dev/wd5s2', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:2:device:@/dev/disk:wd5\0"], ['./dev/wd5s3', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:3:device:@/dev/disk:wd5\0"], ['./dev/wd5s4', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:4:device:@/dev/disk:wd5\0"], ['./dev/wd5s5', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:5:device:@/dev/disk:wd5\0"], ['./dev/wd5s6', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:6:device:@/dev/disk:wd5\0"], ['./dev/wd5s7', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:7:device:@/dev/disk:wd5\0"], ['./dev/wd5s8', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:8:device:@/dev/disk:wd5\0"], ['./dev/wd5s9', oct(640), '0', undef, undef, undef, "/hurd/storeio\0-T\0typed\0part:9:device:@/dev/disk:wd5\0"], ['./dev/xconsole', oct(755), '2', '/run/xconsole', undef, undef, undef], ['./dev/zero', oct(666), '0', undef, undef, undef, "/bin/nullauth\0--\0/hurd/storeio\0-Tzero\0"], # file name mode type link tgt major minor transl. ['./servers/', oct(755), '5', undef, undef, undef, undef], ['./servers/acpi', oct(644), '0', undef, undef, undef, "/hurd/acpi\0"], ['./servers/bus/', oct(755), '5', undef, undef, undef, undef], ['./servers/bus/pci/', oct(755), '5', undef, undef, undef, "/hurd/pci-arbiter\0"], ['./servers/crash', oct(644), '2', 'crash-dump-core', undef, undef, undef], ['./servers/crash-dump-core', oct(644), '0', undef, undef, undef, "/hurd/crash\0--dump-core\0"], ['./servers/crash-kill', oct(644), '0', undef, undef, undef, "/hurd/crash\0--kill\0"], ['./servers/crash-suspend', oct(644), '0', undef, undef, undef, "/hurd/crash\0--suspend\0"], ['./servers/default-pager', oct(755), '0', undef, undef, undef, "/hurd/proxy-defpager\0"], ['./servers/exec', oct(644), '0', undef, undef, undef, "/hurd/exec\0"], ['./servers/password', oct(644), '0', undef, undef, undef, "/hurd/password\0"], ['./servers/shutdown', oct(644), '0', undef, undef, undef, "/hurd/shutdown\0"], ['./servers/socket/', oct(755), '5', undef, undef, undef, undef], ['./servers/socket/1', oct(644), '0', undef, undef, undef, "/hurd/pflocal\0"], ['./servers/socket/2', oct(644), '0', undef, undef, undef, "/hurd/pfinet\0-6\0/servers/socket/26\0"], ['./servers/socket/26', oct(644), '0', undef, undef, undef, "/hurd/pfinet\0-4\0/servers/socket/2\0"], ['./servers/socket/inet', oct(644), '2', "2", undef, undef, undef], ['./servers/socket/inet6', oct(644), '2', "26", undef, undef, undef], ['./servers/socket/local', oct(644), '2', "1", undef, undef, undef], ['./servers/startup', oct(644), '0', undef, undef, undef, undef] ); #>>> # verbosity levels: # 0 -> print nothing # 1 -> normal output and progress bars # 2 -> verbose output # 3 -> debug output my $verbosity_level = 1; my $is_covering = 0; { # make $@ local, so we don't print "Undefined subroutine called" # in other parts where we evaluate $@ local $@ = ''; $is_covering = !!(eval { Devel::Cover::get_coverage() }); } # the reason why Perl::Critic warns about this is, that it suspects that the # programmer wants to implement a test whether the terminal is interactive or # not, in which case, complex interactions with the magic *ARGV indeed make it # advisable to use IO::Interactive. In our case, we do not want to create an # interactivity check but just want to check whether STDERR is opened to a tty, # so our use of -t is fine and not "fragile and complicated" as is written in # the description of InputOutput::ProhibitInteractiveTest. Also see # https://github.com/Perl-Critic/Perl-Critic/issues/918 sub stderr_is_tty() { ## no critic (InputOutput::ProhibitInteractiveTest) if (-t STDERR) { return 1; } else { return 0; } } sub debug { if ($verbosity_level < 3) { return; } my $msg = shift; my ($package, $filename, $line) = caller; $msg = "D: $PID $line $msg"; if (stderr_is_tty()) { $msg = colored($msg, 'clear'); } print STDERR "$msg\n"; return; } sub info { if ($verbosity_level == 0) { return; } my $msg = shift; if ($verbosity_level >= 3) { my ($package, $filename, $line) = caller; $msg = "$PID $line $msg"; } $msg = "I: $msg"; if (stderr_is_tty()) { $msg = colored($msg, 'green'); } print STDERR "$msg\n"; return; } sub warning { if ($verbosity_level == 0) { return; } my $msg = shift; $msg = "W: $msg"; if (stderr_is_tty()) { $msg = colored($msg, 'bold yellow'); } print STDERR "$msg\n"; return; } sub error { # if error() is called with the string from a previous error() that was # caught inside an eval(), then the string will have a newline which we # are stripping here chomp(my $msg = shift); $msg = "E: $msg"; if (stderr_is_tty()) { $msg = colored($msg, 'bold red'); } if ($verbosity_level == 3) { croak $msg; # produces a backtrace } else { die "$msg\n"; } } # The encoding of dev_t is MMMM Mmmm mmmM MMmm, where M is a hex digit of # the major number and m is a hex digit of the minor number. sub major { my $rdev = shift; my $right = Math::BigInt->from_hex("0x00000000000fff00")->band($rdev)->brsft(8); my $left = Math::BigInt->from_hex("0xfffff00000000000")->band($rdev)->brsft(32); return $right->bior($left); } sub minor { my $rdev = shift; my $right = Math::BigInt->from_hex("0x00000000000000ff")->band($rdev); my $left = Math::BigInt->from_hex("0x00000ffffff00000")->band($rdev)->brsft(12); return $right->bior($left); } sub can_execute { my $tool = shift; my $verbose = shift // '--version'; my $pid = open my $fh, '-|' // return 0; if ($pid == 0) { open(STDERR, '>&', STDOUT) or die; exec {$tool} $tool, $verbose or die; } chomp( my $content = do { local $/; <$fh> } ); close $fh; if ($? != 0) { return 0; } if (length $content == 0) { return 0; } return 1; } # check whether a directory is mounted by comparing the device number of the # directory itself with its parent sub is_mountpoint { my $dir = shift; if (!-e $dir) { return 0; } my @a = stat "$dir/."; my @b = stat "$dir/.."; # if the device number is different, then the directory must be mounted if ($a[0] != $b[0]) { return 1; } # if the inode number is the same, then the directory must be mounted if ($a[1] == $b[1]) { return 1; } return 0; } # tar cannot figure out the decompression program when receiving data on # standard input, thus we do it ourselves. This is copied from tar's # src/suffix.c sub get_tar_compressor { my $filename = shift; if ($filename eq '-') { return; } elsif ($filename =~ /\.tar$/) { return; } elsif ($filename =~ /\.(gz|tgz|taz)$/) { return ['gzip']; } elsif ($filename =~ /\.(Z|taZ)$/) { return ['compress']; } elsif ($filename =~ /\.(bz2|tbz|tbz2|tz2)$/) { return ['bzip2']; } elsif ($filename =~ /\.lz$/) { return ['lzip']; } elsif ($filename =~ /\.(lzma|tlz)$/) { return ['lzma']; } elsif ($filename =~ /\.lzo$/) { return ['lzop']; } elsif ($filename =~ /\.lz4$/) { return ['lz4']; } elsif ($filename =~ /\.(xz|txz)$/) { return ['xz']; } elsif ($filename =~ /\.zst$/) { return ['zstd']; } return; } # avoid dependency on String::ShellQuote by implementing the mechanism # from python's shlex.quote function sub shellescape { my $string = shift; if (length $string == 0) { return "''"; } # search for occurrences of characters that are not safe # the 'a' regex modifier makes sure that \w only matches ASCII if ($string !~ m/[^\w@\%+=:,.\/-]/a) { return $string; } # wrap the string in single quotes and handle existing single quotes by # putting them outside of the single-quoted string $string =~ s/'/'"'"'/g; return "'$string'"; } sub create_v5_uuid { use bytes; my $ns_uuid = shift; my $name = shift; my $version = 0x50; # convert the namespace uuid to binary $ns_uuid =~ tr/-//d; $ns_uuid = pack 'H*', $ns_uuid; # concatenate namespace and name and take sha1 my $digest = Digest::SHA->new(1); $digest->add($ns_uuid); $digest->add($name); # only the first 16 bytes matter my $uuid = substr($digest->digest(), 0, 16); # set the version substr $uuid, 6, 1, chr(ord(substr($uuid, 6, 1)) & 0x0f | $version); substr $uuid, 8, 1, chr(ord(substr $uuid, 8, 1) & 0x3f | 0x80); # convert binary back to uuid formatting return join '-', map { unpack 'H*', $_ } map { substr $uuid, 0, $_, '' } (4, 2, 2, 2, 6); } sub test_unshare_userns { my $verbose = shift; local *maybe_error = sub { my $msg = shift; if ($verbose) { error $msg; } else { debug $msg; } }; if ($EFFECTIVE_USER_ID == 0) { maybe_error("cannot unshare user namespace when executing as root"); return 0; } # arguments to syscalls have to be stored in their own variable or # otherwise we will get "Modification of a read-only value attempted" my $unshare_flags = $CLONE_NEWUSER; # we spawn a new per process because if unshare succeeds, we would # otherwise have unshared the mmdebstrap process itself which we don't want my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { my $ret = syscall(&SYS_unshare, $unshare_flags); if ($ret == 0) { exit 0; } else { maybe_error("unshare syscall failed: $!"); exit 1; } } waitpid($pid, 0); if (($? >> 8) != 0) { return 0; } # if newuidmap and newgidmap exist, the exit status will be 1 when # executed without parameters system "newuidmap 2>/dev/null"; if (($? >> 8) != 1) { if (($? >> 8) == 127) { maybe_error("cannot find newuidmap"); } else { maybe_error("newuidmap returned unknown exit status: $?"); } return 0; } system "newgidmap 2>/dev/null"; if (($? >> 8) != 1) { if (($? >> 8) == 127) { maybe_error("cannot find newgidmap"); } else { maybe_error("newgidmap returned unknown exit status: $?"); } return 0; } my @idmap = read_subuid_subgid($verbose); if (scalar @idmap == 0) { maybe_error("failed to parse /etc/subuid and /etc/subgid"); return 0; } # too much can go wrong when doing the dance required to unsharing the user # namespace, so instead of adding more complexity to support maybe_error() # to a function that is already too complex, we use eval() eval { $pid = get_unshare_cmd( sub { if ($EFFECTIVE_USER_ID == 0) { exit 0; } else { exit 1; } }, \@idmap ); waitpid $pid, 0; if ($? != 0) { maybe_error("failed to unshare the user namespace"); return 0; } }; if ($@) { maybe_error($@); return 0; } return 1; } sub read_subuid_subgid { my $verbose = shift; my @result = (); my $username = getpwuid $REAL_USER_ID; my ($subid, $num_subid, $fh, $n); local *maybe_warn = sub { my $msg = shift; if ($verbose) { warning $msg; } else { debug $msg; } }; if (!-e "/etc/subuid") { maybe_warn("/etc/subuid doesn't exist"); return; } if (!-r "/etc/subuid") { maybe_warn("/etc/subuid is not readable"); return; } open $fh, "<", "/etc/subuid" or maybe_warn("cannot open /etc/subuid for reading: $!"); if (!$fh) { return; } while (my $line = <$fh>) { ($n, $subid, $num_subid) = split(/:/, $line, 3); last if ($n eq $username); } close $fh; if (!length $subid) { maybe_warn("/etc/subuid is empty"); return; } if ($n ne $username) { maybe_warn("no entry in /etc/subuid for $username"); return; } push @result, ["u", 0, $subid, $num_subid]; if (scalar(@result) < 1) { maybe_warn("/etc/subuid does not contain an entry for $username"); return; } if (scalar(@result) > 1) { maybe_warn("/etc/subuid contains multiple entries for $username"); return; } if (!-e "/etc/subgid") { maybe_warn("/etc/subgid doesn't exist"); return; } if (!-r "/etc/subgid") { maybe_warn("/etc/subgid is not readable"); return; } open $fh, "<", "/etc/subgid" or maybe_warn("cannot open /etc/subgid for reading: $!"); if (!$fh) { return; } while (my $line = <$fh>) { ($n, $subid, $num_subid) = split(/:/, $line, 3); last if ($n eq $username); } close $fh; if (!length $subid) { maybe_warn("/etc/subgid is empty"); return; } if ($n ne $username) { maybe_warn("no entry in /etc/subgid for $username"); return; } push @result, ["g", 0, $subid, $num_subid]; if (scalar(@result) < 2) { maybe_warn("/etc/subgid does not contain an entry for $username"); return; } if (scalar(@result) > 2) { maybe_warn("/etc/subgid contains multiple entries for $username"); return; } return @result; } # This function spawns two child processes forming the following process tree # # A # | # fork() # | \ # B C # | | # | fork() # | | \ # | D E # | | | # |unshare() # | close() # | | | # | | read() # | | newuidmap(D) # | | newgidmap(D) # | | / # | waitpid() # | | # | fork() # | | \ # | F G # | | | # | | exec() # | | / # | waitpid() # | / # waitpid() # # To better refer to each individual part, we give each process a new # identifier after calling fork(). Process A is the main process. After # executing fork() we call the parent and child B and C, respectively. This # first fork() is done because we do not want to modify A. B then remains # waiting for its child C to finish. C calls fork() again, splitting into # the parent D and its child E. In the parent D we call unshare() and close a # pipe shared by D and E to signal to E that D is done with calling unshare(). # E notices this by using read() and follows up with executing the tools # new[ug]idmap on D. E finishes and D continues with doing another fork(). # This is because when unsharing the PID namespace, we need a PID 1 to be kept # alive or otherwise any child processes cannot fork() anymore themselves. So # we keep F as PID 1 and finally call exec() in G. sub get_unshare_cmd { my $cmd = shift; my $idmap = shift; # unsharing the mount namespace (NEWNS) requires CAP_SYS_ADMIN my $unshare_flags = $CLONE_NEWNS | $CLONE_NEWPID | $CLONE_NEWUTS | $CLONE_NEWIPC; # we only need to add CLONE_NEWUSER if we are not yet root if ($EFFECTIVE_USER_ID != 0) { $unshare_flags |= $CLONE_NEWUSER; } if (0) { $unshare_flags |= $CLONE_NEWNET; } # fork a new process and let the child get unshare()ed # we don't want to unshare the parent process my $gcpid = fork() // error "fork() failed: $!"; if ($gcpid == 0) { # Create a pipe for the parent process to signal the child process that # it is done with calling unshare() so that the child can go ahead # setting up uid_map and gid_map. pipe my $rfh, my $wfh; # We have to do this dance with forking a process and then modifying # the parent from the child because: # - new[ug]idmap can only be called on a process id after that process # has unshared the user namespace # - a process looses its capabilities if it performs an execve() with # nonzero user ids see the capabilities(7) man page for details. # - a process that unshared the user namespace by default does not # have the privileges to call new[ug]idmap on itself # # this also works the other way around (the child setting up a user # namespace and being modified from the parent) but that way, the # parent would have to stay around until the child exited (so a pid # would be wasted). Additionally, that variant would require an # additional pipe to let the parent signal the child that it is done # with calling new[ug]idmap. The way it is done here, this signaling # can instead be done by wait()-ing for the exit of the child. my $ppid = $$; my $cpid = fork() // error "fork() failed: $!"; if ($cpid == 0) { # child # Close the writing descriptor at our end of the pipe so that we # see EOF when parent closes its descriptor. close $wfh; # Wait for the parent process to finish its unshare() call by # waiting for an EOF. 0 == sysread $rfh, my $c, 1 or error "read() did not receive EOF"; # the process is already root, so no need for newuidmap/newgidmap if ($EFFECTIVE_USER_ID == 0) { exit 0; } # The program's new[ug]idmap have to be used because they are # setuid root. These privileges are needed to map the ids from # /etc/sub[ug]id to the user namespace set up by the parent. # Without these privileges, only the id of the user itself can be # mapped into the new namespace. # # Since new[ug]idmap is setuid root we also don't need to write # "deny" to /proc/$$/setgroups beforehand (this is otherwise # required for unprivileged processes trying to write to # /proc/$$/gid_map since kernel version 3.19 for security reasons) # and therefore the parent process keeps its ability to change its # own group here. # # Since /proc/$ppid/[ug]id_map can only be written to once, # respectively, instead of making multiple calls to new[ug]idmap, # we assemble a command line that makes one call each. my $uidmapcmd = ""; my $gidmapcmd = ""; foreach (@{$idmap}) { my ($t, $hostid, $nsid, $range) = @{$_}; if ($t ne "u" and $t ne "g" and $t ne "b") { error "invalid idmap type: $t"; } if ($t eq "u" or $t eq "b") { $uidmapcmd .= " $hostid $nsid $range"; } if ($t eq "g" or $t eq "b") { $gidmapcmd .= " $hostid $nsid $range"; } } my $idmapcmd = ''; if ($uidmapcmd ne "") { 0 == system "newuidmap $ppid $uidmapcmd" or error "newuidmap $ppid $uidmapcmd failed: $!"; } if ($gidmapcmd ne "") { 0 == system "newgidmap $ppid $gidmapcmd" or error "newgidmap $ppid $gidmapcmd failed: $!"; } exit 0; } # parent # After fork()-ing, the parent immediately calls unshare... 0 == syscall &SYS_unshare, $unshare_flags or error "unshare() failed: $!"; # .. and then signals the child process that we are done with the # unshare() call by sending an EOF. close $wfh; # Wait for the child process to finish its setup by waiting for its # exit. $cpid == waitpid $cpid, 0 or error "waitpid() failed: $!"; my $exit = $? >> 8; if ($exit != 0) { error "child had a non-zero exit status: $exit"; } # Currently we are nobody (uid and gid are 65534). So we become root # user and group instead. # # We are using direct syscalls instead of setting $(, $), $< and $> # because then perl would do additional stuff which we don't need or # want here, like checking /proc/sys/kernel/ngroups_max (which might # not exist). It would also also call setgroups() in a way that makes # the root user be part of the group unknown. if ($EFFECTIVE_USER_ID != 0) { 0 == syscall &SYS_setgid, 0 or error "setgid failed: $!"; 0 == syscall &SYS_setuid, 0 or error "setuid failed: $!"; 0 == syscall &SYS_setgroups, 0, 0 or error "setgroups failed: $!"; } if (1) { # When the pid namespace is also unshared, then processes expect a # master pid to always be alive within the namespace. To achieve # this, we fork() here instead of exec() to always have one dummy # process running as pid 1 inside the namespace. This is also what # the unshare tool does when used with the --fork option. # # Otherwise, without a pid 1, new processes cannot be forked # anymore after pid 1 finished. my $cpid = fork() // error "fork() failed: $!"; if ($cpid != 0) { # The parent process will stay alive as pid 1 in this # namespace until the child finishes executing. This is # important because pid 1 must never die or otherwise nothing # new can be forked. $cpid == waitpid $cpid, 0 or error "waitpid() failed: $!"; exit($? >> 8); } } &{$cmd}(); exit 0; } # parent return $gcpid; } sub havemknod { my $root = shift; my $havemknod = 0; if (-e "$root/test-dev-null") { error "/test-dev-null already exists"; } TEST: { # we fork so that we can read STDERR my $pid = open my $fh, '-|' // error "failed to fork(): $!"; if ($pid == 0) { open(STDERR, '>&', STDOUT) or error "cannot open STDERR: $!"; # we use mknod(1) instead of the system call because creating the # right dev_t argument requires makedev(3) exec 'mknod', "$root/test-dev-null", 'c', '1', '3'; } chomp( my $content = do { local $/; <$fh> } ); close $fh; { last TEST unless $? == 0 and $content eq ''; last TEST unless -c "$root/test-dev-null"; last TEST unless open my $fh, '>', "$root/test-dev-null"; last TEST unless print $fh 'test'; } $havemknod = 1; } if (-e "$root/test-dev-null") { unlink "$root/test-dev-null" or error "cannot unlink /test-dev-null: $!"; } return $havemknod; } # inspired by /usr/share/perl/5.34/pod/perlfaq8.pod sub terminal_width { if (!stderr_is_tty()) { return -1; } if (!defined &TIOCGWINSZ) { return -1; } if (!-e "/dev/tty") { return -1; } my $tty_fh; if (!open($tty_fh, "+<", "/dev/tty")) { return -1; } my $winsize = ''; if (!ioctl($tty_fh, &TIOCGWINSZ, $winsize)) { return -1; } my (undef, $col, undef, undef) = unpack('S4', $winsize); return $col; } # Prints the current status, the percentage and a progress bar on STDERR if # it is an interactive tty and if verbosity is set to 1. # # * first 12 chars: status # * following 7 chars: percentage # * progress bar until 79 chars are filled sub print_progress { if ($verbosity_level != 1) { return; } if (!stderr_is_tty()) { return; } my $perc = shift; my $status = shift; my $len_status = 12; my $len_perc = 7; my $len_prog_min = 10; my $len_prog_max = 60; my $twidth = terminal_width(); if ($twidth <= $len_status) { return; } # \e[2K clears everything on the current line (i.e. the progress bar) print STDERR "\e[2K"; if ($perc eq "done") { print STDERR "done\n"; return; } if (defined $status) { printf STDERR "%*s", -$len_status, "$status:"; } else { print STDERR (" " x $len_status); } if ($twidth <= $len_status + $len_perc) { print STDERR "\r"; return; } if ($perc >= 100) { $perc = 100; } printf STDERR "%*.2f", $len_perc, $perc; if ($twidth <= $len_status + $len_perc + $len_prog_min) { print STDERR "\r"; return; } my $len_prog = $twidth - $len_perc - $len_status; if ($len_prog > $len_prog_max) { $len_prog = $len_prog_max; } my $num_x = int($perc * ($len_prog - 3) / 100); my $bar = '=' x $num_x; if ($num_x != ($len_prog - 3)) { $bar .= '>'; $bar .= ' ' x ($len_prog - $num_x - 4); } print STDERR " [$bar]\r"; return; } sub run_progress { my ($get_exec, $line_handler, $line_has_error, $chdir) = @_; pipe my $rfh, my $wfh; my $got_signal = 0; my $ignore = sub { info "run_progress() received signal $_[0]: waiting for child..."; }; debug("run_progress: exec " . (join ' ', ($get_exec->('${FD}')))); # delay signals so that we can fork and change behaviour of the signal # handler in parent and child without getting interrupted my $sigset = POSIX::SigSet->new(SIGINT, SIGHUP, SIGPIPE, SIGTERM); POSIX::sigprocmask(SIG_BLOCK, $sigset) or error "Can't block signals: $!"; my $pid1 = open(my $pipe, '-|') // error "failed to fork(): $!"; if ($pid1 == 0) { # child: default signal handlers local $SIG{'INT'} = 'DEFAULT'; local $SIG{'HUP'} = 'DEFAULT'; local $SIG{'PIPE'} = 'DEFAULT'; local $SIG{'TERM'} = 'DEFAULT'; # unblock all delayed signals (and possibly handle them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; close $rfh; # Unset the close-on-exec flag, so that the file descriptor does not # get closed when we exec my $flags = fcntl($wfh, F_GETFD, 0) or error "fcntl F_GETFD: $!"; fcntl($wfh, F_SETFD, $flags & ~FD_CLOEXEC) or error "fcntl F_SETFD: $!"; my $fd = fileno $wfh; # redirect stderr to stdout so that we can capture it open(STDERR, '>&', STDOUT) or error "cannot open STDOUT: $!"; my @execargs = $get_exec->($fd); # before apt 1.5, "apt-get update" attempted to chdir() into the # working directory. This will fail if the current working directory # is not accessible by the user (for example in unshare mode). See # Debian bug #860738 if (defined $chdir) { chdir $chdir or error "failed chdir() to $chdir: $!"; } eval { Devel::Cover::set_coverage("none") } if $is_covering; exec { $execargs[0] } @execargs or error 'cannot exec() ' . (join ' ', @execargs); } close $wfh; # spawn two processes: # parent will parse stdout to look for errors # child will parse $rfh for the progress meter my $pid2 = fork() // error "failed to fork(): $!"; if ($pid2 == 0) { # child: default signal handlers local $SIG{'INT'} = 'IGNORE'; local $SIG{'HUP'} = 'IGNORE'; local $SIG{'PIPE'} = 'IGNORE'; local $SIG{'TERM'} = 'IGNORE'; # unblock all delayed signals (and possibly handle them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; if ($verbosity_level != 1 || !stderr_is_tty()) { # no need to print any progress # we still need to consume everything from $rfh or otherwise apt # will block forever if there is too much output local $/; <$rfh>; close $rfh; exit 0; } my $progress = 0.0; my $status = undef; print_progress($progress); while (my $line = <$rfh>) { my ($newprogress, $newstatus) = $line_handler->($line); next unless $newprogress; # start a new line if the new progress value is less than the # previous one if ($newprogress < $progress) { print_progress("done"); } if (defined $newstatus) { $status = $newstatus; } print_progress($newprogress, $status); $progress = $newprogress; } print_progress("done"); exit 0; } # parent: ignore signals # by using "local", the original is automatically restored once the # function returns local $SIG{'INT'} = $ignore; local $SIG{'HUP'} = $ignore; local $SIG{'PIPE'} = $ignore; local $SIG{'TERM'} = $ignore; # unblock all delayed signals (and possibly handle them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; my $output = ''; my $has_error = 0; while (my $line = <$pipe>) { $has_error = $line_has_error->($line); if ($verbosity_level >= 2) { print STDERR $line; } else { # forward captured apt output $output .= $line; } } close($pipe); my $proc_exit = $?; waitpid $pid2, 0; $? == 0 or error "progress parsing failed"; if ($got_signal) { error "run_progress() received signal: $got_signal"; } # only print failure after progress output finished or otherwise it # might interfere with the remaining output if ($proc_exit != 0 or $has_error) { if ($verbosity_level >= 1) { print STDERR $output; } my $what = ''; if ($proc_exit != 0) { if (POSIX::WIFEXITED($proc_exit)) { my $exit = POSIX::WEXITSTATUS($proc_exit); $what .= "process exited with $exit and error in console output"; } elsif (POSIX::WIFSIGNALED($proc_exit)) { my $sig = POSIX::WTERMSIG($proc_exit); $what = "process was killed by signal $sig"; } else { $what = "process failed with unknown status code"; } if ($has_error) { $what .= " and error in console output"; } } else { $what .= "error in console output"; } error((join ' ', $get_exec->('<$fd>')) . " failed: $what"); } return; } sub run_dpkg_progress { my $options = shift; my @debs = @{ $options->{PKGS} // [] }; my $get_exec = sub { return @{ $options->{ARGV} }, "--status-fd=$_[0]", @debs; }; my $line_has_error = sub { return 0; }; my $num = 0; # each package has one install and one configure step, thus the total # number is twice the number of packages my $total = (scalar @debs) * 2; my $line_handler = sub { my $status = undef; if ($_[0] =~ /^processing: (install|configure): /) { if ($1 eq 'install') { $status = 'installing'; } elsif ($1 eq 'configure') { $status = 'configuring'; } else { error "unknown status: $1"; } $num += 1; } if ($total == 0) { return 0, $status; } else { return $num / $total * 100, $status; } }; run_progress $get_exec, $line_handler, $line_has_error; return; } sub run_apt_progress { my $options = shift; my @debs = @{ $options->{PKGS} // [] }; if ($verbosity_level >= 3) { my @apt_debug_opts = qw( -oDebug::pkgProblemResolver=true -oDebug::pkgDepCache::Marker=1 -oDebug::pkgDepCache::AutoInstall=1 ); push @{ $options->{ARGV} }, @apt_debug_opts; } my $get_exec = sub { my @prefix = (); my @opts = (); return ( @prefix, @{ $options->{ARGV} }, @opts, "-oAPT::Status-Fd=$_[0]", # prevent apt from messing up the terminal and allow dpkg to # receive SIGINT and quit immediately without waiting for # maintainer script to finish '-oDpkg::Use-Pty=false', @debs ); }; my $line_has_error = sub { return 0; }; if ($options->{FIND_APT_WARNINGS}) { $line_has_error = sub { # apt-get doesn't report a non-zero exit if the update failed. # Thus, we have to parse its output. See #778357, #776152, #696335 # and #745735 for the parsing bugs as well as #594813, #696335, # #776152, #778357 and #953726 for non-zero exit on transient # network errors. # # For example, we want to fail with the following warning: # W: Some index files failed to download. They have been ignored, # or old ones used instead. # But since this message is meant for human consumption it is not # guaranteed to be stable across different apt versions and may # change arbitrarily in the future. Thus, we error out on any W: # lines as well. The downside is, that apt also unconditionally # and by design prints a warning for unsigned repositories, even # if they were allowed with Acquire::AllowInsecureRepositories "1" # or with trusted=yes. # # A workaround was introduced by apt 2.1.16 with the --error-on=any # option to apt-get update. if ($_[0] =~ /^(W: |Err:)/) { return 1; } return 0; }; } my $line_handler = sub { if ($_[0] =~ /(pmstatus|dlstatus):[^:]+:(\d+\.\d+):.*/) { my $status = undef; if ($1 eq 'pmstatus') { $status = "installing"; } elsif ($1 eq 'dlstatus') { $status = "downloading"; } else { error "unknown status: $1"; } return $2, $status; } }; run_progress $get_exec, $line_handler, $line_has_error, $options->{CHDIR}; return; } sub run_apt_download_progress { my $options = shift; if ($options->{dryrun}) { info "simulate downloading packages with apt..."; } else { info "downloading packages with apt..."; } pipe my $rfh, my $wfh; my $pid = open my $fh, '-|' // error "fork() failed: $!"; if ($pid == 0) { close $wfh; # read until parent process closes $wfh my $content = do { local $/; <$rfh> }; close $rfh; # the parent is done -- pass what we read back to it print $content; exit 0; } close $rfh; # Unset the close-on-exec flag, so that the file descriptor does not # get closed when we exec my $flags = fcntl($wfh, F_GETFD, 0) or error "fcntl F_GETFD: $!"; fcntl($wfh, F_SETFD, $flags & ~FD_CLOEXEC) or error "fcntl F_SETFD: $!"; my $fd = fileno $wfh; # run_apt_progress() can raise an exception which would leave this function # without cleaning up the other thread we started, making mmdebstrap hang # in case run_apt_progress() fails -- so wrap this in eval() instead eval { # 2022-05-02, #debian-apt on OFTC, times in UTC+2 # 16:57 < josch> DonKult: how is -oDebug::pkgDpkgPm=1 # -oDir::Log=/dev/null a "fancy no-op"? # 11:52 < DonKult> josch: "fancy no-op" in sofar as it does nothing to # the system even through its not in a special mode # ala simulation or download-only. It does all the # things it normally does, except that it just prints # the dpkg calls instead of execv() them which in # practice amounts means it does nothing (the Dir::Log # just prevents libapt from creating the /var/log/apt # directories. As the code creates them even if no # logs will be placed there…). As said, midterm an apt # --print-install-packages or something would be nice # to avoid running everything. run_apt_progress({ ARGV => [ 'apt-get', '--yes', '-oDebug::pkgDpkgPm=1', '-oDir::Log=/dev/null', $options->{dryrun} ? '-oAPT::Get::Simulate=true' : ( "-oAPT::Keep-Fds::=$fd", "-oDPkg::Tools::options::'cat >&$fd'::InfoFD=$fd", "-oDpkg::Pre-Install-Pkgs::=cat >&$fd", # no need to lock the database if we are just downloading "-oDebug::NoLocking=1", # no need for pty magic if we write no log "-oDpkg::Use-Pty=0", # unset this or otherwise "cat >&$fd" will fail "-oDPkg::Chroot-Directory=", ), @{ $options->{APT_ARGV} }, ], }); }; my $err = ''; if ($@) { $err = "apt download failed: $@"; } # signal the child process that we are done close $wfh; # and then read from it what it got my @listofdebs = <$fh>; close $fh; if ($? != 0) { $err = "status child failed"; } if ($err) { error $err; } # remove trailing newlines chomp @listofdebs; return @listofdebs; } sub setup_mounts { my $options = shift; my @cleanup_tasks = (); eval { if (any { $_ eq $options->{mode} } ('root', 'unshare')) { 0 == system('mount', "--make-rprivate", "/") or warning("mount --make-rprivate / failed: $?"); # if more than essential should be installed, make the system look # more like a real one by creating or bind-mounting the device # nodes foreach my $file (@linuxdevfiles) { my ($fname, $mode, $type, $linkname, $devmajor, $devminor, undef) = @{$file}; next if $fname eq './dev/'; if ($type eq '0') { # normal file error "type 0 not implemented"; } elsif ($type eq '1') { # hardlink error "type 1 not implemented"; } elsif ($type eq '2') { # symlink if (!$options->{havemknod}) { # If we had mknod, then the symlink was already created # in the run_setup function. if (!-d "$options->{root}/dev") { warning( "skipping creation of $fname because the" . " /dev directory is missing in the target" ); next; } if (-e "$options->{root}/$fname") { warning( "skipping creation of $fname because it" . " already exists in the target"); next; } push @cleanup_tasks, sub { unlink "$options->{root}/$fname" or warning("cannot unlink ./dev/$fname: $!"); }; symlink $linkname, "$options->{root}/$fname" or warning "cannot create symlink $fname -> $linkname"; } } elsif ($type eq '3' or $type eq '4') { # character/block special if (any { $_ =~ '^chroot/mount(?:/dev)?$' } @{ $options->{skip} }) { info "skipping chroot/mount/dev as requested"; } elsif (!$options->{canmount}) { warning "skipping bind-mounting $fname"; } elsif (!$options->{havemknod}) { if (!-d "$options->{root}/dev") { warning( "skipping creation of $fname because the" . " /dev directory is missing in the target" ); next; } if ($fname eq "./dev/ptmx") { # We must not bind-mount ptmx from the outside or # otherwise posix_openpt() will fail. Instead # /dev/ptmx must refer to /dev/pts/ptmx either by # symlink or by bind-mounting. We choose a symlink. symlink '/dev/pts/ptmx', "$options->{root}/dev/ptmx" or error "cannot create /dev/pts/ptmx symlink"; push @cleanup_tasks, sub { unlink "$options->{root}/dev/ptmx" or warning "unlink /dev/ptmx"; }; next; } if (!-e "/$fname") { warning("skipping creation of $fname because" . " $fname does not exist" . " on the outside"); next; } if (!-c "/$fname") { warning("skipping creation of $fname because" . " $fname on the outside is not a" . " character special file"); next; } open my $fh, '>', "$options->{root}/$fname" or error "cannot open $options->{root}/$fname: $!"; close $fh; my @umountopts = (); if ($options->{mode} eq 'unshare') { push @umountopts, '--no-mtab'; } push @cleanup_tasks, sub { 0 == system('umount', @umountopts, "$options->{root}/$fname") or warning("umount $fname failed: $?"); unlink "$options->{root}/$fname" or warning("cannot unlink $fname: $!"); }; 0 == system('mount', '-o', 'bind', "/$fname", "$options->{root}/$fname") or error "mount $fname failed: $?"; } } elsif ($type eq '5') { # directory if (any { $_ =~ '^chroot/mount(?:/dev)?$' } @{ $options->{skip} }) { info "skipping chroot/mount/dev as requested"; } elsif (!$options->{canmount}) { warning "skipping bind-mounting $fname"; } else { if (!-d "$options->{root}/dev") { warning( "skipping creation of $fname because the" . " /dev directory is missing in the target" ); next; } if (!-e "/$fname" && $fname ne "./dev/pts/") { warning("skipping creation of $fname because" . " $fname does not exist" . " on the outside"); next; } if (!-d "/$fname" && $fname ne "./dev/pts/") { warning("skipping creation of $fname because" . " $fname on the outside is not a" . " directory"); next; } if (!$options->{havemknod}) { # If had mknod, then the directory to bind-mount # into was already created in the run_setup # function. push @cleanup_tasks, sub { rmdir "$options->{root}/$fname" or warning("cannot rmdir $fname: $!"); }; if (-e "$options->{root}/$fname") { if (!-d "$options->{root}/$fname") { error "$fname already exists but is not" . " a directory"; } } else { my $num_created = make_path "$options->{root}/$fname", { error => \my $err }; if ($err && @$err) { error( join "; ", ( map { "cannot create " . (join ": ", %{$_}) } @$err )); } elsif ($num_created == 0) { error( "cannot create $options->{root}" . "$fname"); } } chmod $mode, "$options->{root}/$fname" or error "cannot chmod $fname: $!"; } my @umountopts = (); if ($options->{mode} eq 'unshare') { push @umountopts, '--no-mtab'; } push @cleanup_tasks, sub { 0 == system('umount', @umountopts, "$options->{root}/$fname") or warning("umount $fname failed: $?"); }; if ($fname eq "./dev/pts/") { # We cannot just bind-mount /dev/pts from the host # as doing so will make posix_openpt() fail. # Instead, we need to mount a new devpts. # We need ptmxmode=666 because /dev/ptmx is a # symlink to /dev/pts/ptmx and without it # posix_openpt() will fail if we are not the root # user. See also: # kernel.o/doc/Documentation/filesystems/devpts.txt # salsa.d.o/debian/schroot/-/merge_requests/2 # https://bugs.debian.org/856877 # https://bugs.debian.org/817236 0 == system( 'mount', '-t', 'devpts', 'none', "$options->{root}/dev/pts", '-o', 'noexec,nosuid,uid=5,mode=620,ptmxmode=666' ) or error "mount /dev/pts failed"; } else { 0 == system('mount', '-o', 'bind', "/$fname", "$options->{root}/$fname") or error "mount $fname failed: $?"; } } } else { error "unsupported type: $type"; } } } elsif (any { $_ eq $options->{mode} } ('fakechroot', 'chrootless')) { # we cannot mount in fakechroot mode } else { error "unknown mode: $options->{mode}"; } # We can only mount /proc and /sys after extracting the essential # set because if we mount it before, then base-files will not be able # to extract those if ( (any { $_ eq $options->{mode} } ('root', 'unshare')) && (any { $_ =~ '^chroot/mount(?:/sys)?$' } @{ $options->{skip} })) { info "skipping chroot/mount/sys as requested"; } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !$options->{canmount}) { warning "skipping mount sysfs"; } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-d "$options->{root}/sys") { warning("skipping mounting of sysfs because the" . " /sys directory is missing in the target"); } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-e "/sys") { warning("skipping mounting /sys because" . " /sys does not exist on the outside"); } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-d "/sys") { warning("skipping mounting /sys because" . " /sys on the outside is not a directory"); } elsif ($options->{mode} eq 'root') { # we don't know whether we run in root mode inside an unshared # user namespace or as real root so we first try the real mount and # then fall back to mounting in a way that works in unshared mode if ( 0 == system( 'mount', '-t', 'sysfs', '-o', 'ro,nosuid,nodev,noexec', 'sys', "$options->{root}/sys" ) ) { push @cleanup_tasks, sub { 0 == system('umount', "$options->{root}/sys") or warning("umount /sys failed: $?"); }; } elsif ( 0 == system('mount', '-o', 'rbind', '/sys', "$options->{root}/sys")) { push @cleanup_tasks, sub { # since we cannot write to /etc/mtab we need --no-mtab # unmounting /sys only seems to be successful with --lazy 0 == system( 'umount', '--no-mtab', '--lazy', "$options->{root}/sys" ) or warning("umount /sys failed: $?"); }; } else { error "mount /sys failed: $?"; } } elsif ($options->{mode} eq 'unshare') { # naturally we have to clean up after ourselves in sudo mode where # we do a real mount. But we also need to unmount in unshare mode # because otherwise, even with the --one-file-system tar option, # the permissions of the mount source will be stored and not the # mount target (the directory) push @cleanup_tasks, sub { # since we cannot write to /etc/mtab we need --no-mtab # unmounting /sys only seems to be successful with --lazy 0 == system('umount', '--no-mtab', '--lazy', "$options->{root}/sys") or warning("umount /sys failed: $?"); }; # without the network namespace unshared, we cannot mount a new # sysfs. Since we need network, we just bind-mount. # # we have to rbind because just using bind results in "wrong fs # type, bad option, bad superblock" error 0 == system('mount', '-o', 'rbind', '/sys', "$options->{root}/sys") or error "mount /sys failed: $?"; } elsif (any { $_ eq $options->{mode} } ('fakechroot', 'chrootless')) { # we cannot mount in fakechroot mode } else { error "unknown mode: $options->{mode}"; } if ( (any { $_ eq $options->{mode} } ('root', 'unshare')) && (any { $_ =~ '^chroot/mount(?:/proc)?$' } @{ $options->{skip} }) ) { info "skipping chroot/mount/proc as requested"; } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !$options->{canmount}) { warning "skipping mount proc"; } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-d "$options->{root}/proc") { warning("skipping mounting of proc because the" . " /proc directory is missing in the target"); } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-e "/proc") { warning("skipping mounting /proc because" . " /proc does not exist on the outside"); } elsif ((any { $_ eq $options->{mode} } ('root', 'unshare')) && !-d "/proc") { warning("skipping mounting /proc because" . " /proc on the outside is not a directory"); } elsif (any { $_ eq $options->{mode} } ('root', 'unshare')) { # we don't know whether we run in root mode inside an unshared # user namespace or as real root so we first try the real mount and # then fall back to mounting in a way that works in unshared if ( $options->{mode} eq 'root' && 0 == system( 'mount', '-t', 'proc', '-o', 'ro', 'proc', "$options->{root}/proc" ) ) { push @cleanup_tasks, sub { # some maintainer scripts mount additional stuff into /proc # which we need to unmount beforehand if ( is_mountpoint( $options->{root} . "/proc/sys/fs/binfmt_misc" ) ) { 0 == system('umount', "$options->{root}/proc/sys/fs/binfmt_misc") or warning( "umount /proc/sys/fs/binfmt_misc failed: $?"); } 0 == system('umount', "$options->{root}/proc") or warning("umount /proc failed: $?"); }; } elsif ( 0 == system('mount', '-t', 'proc', 'proc', "$options->{root}/proc")) { push @cleanup_tasks, sub { # since we cannot write to /etc/mtab we need --no-mtab 0 == system('umount', '--no-mtab', "$options->{root}/proc") or warning("umount /proc failed: $?"); }; } elsif ( # if mounting proc failed, try bind-mounting it read-only as a # last resort 0 == system( 'mount', '-o', 'rbind', '/proc', "$options->{root}/proc" ) ) { warning("since mounting /proc normally failed, /proc is now " . "bind-mounted instead"); # to make sure that changes (like unmounting) to the # bind-mounted /proc do not affect the outside /proc, change # all the bind-mounts under /proc to be a slave mount. if ( 0 != system('mount', '--make-rslave', "$options->{root}/proc")) { warning("mount --make-rslave /proc failed"); } push @cleanup_tasks, sub { # since we cannot write to /etc/mtab we need --no-mtab 0 == system( 'umount', '--no-mtab', '--lazy', "$options->{root}/proc" ) or warning("umount /proc failed: $?"); }; } else { error "mount /proc failed: $?"; } } elsif (any { $_ eq $options->{mode} } ('fakechroot', 'chrootless')) { # we cannot mount in fakechroot mode } else { error "unknown mode: $options->{mode}"; } # prevent daemons from starting # the directory might not exist in custom variant, for example # # ideally, we should use update-alternatives but we cannot rely on it # existing inside the chroot # # See #911290 for more problems of this interface if (any { $_ eq 'chroot/policy-rc.d' } @{ $options->{skip} }) { info "skipping chroot/policy-rc.d as requested"; } else { push @cleanup_tasks, sub { if (-f "$options->{root}/usr/sbin/policy-rc.d") { unlink "$options->{root}/usr/sbin/policy-rc.d" or error "cannot unlink policy-rc.d: $!"; } }; if (-d "$options->{root}/usr/sbin/") { open my $fh, '>', "$options->{root}/usr/sbin/policy-rc.d" or error "cannot open policy-rc.d: $!"; print $fh "#!/bin/sh\n"; print $fh "exit 101\n"; close $fh; chmod 0755, "$options->{root}/usr/sbin/policy-rc.d" or error "cannot chmod policy-rc.d: $!"; } } # the file might not exist if it was removed in a hook if (any { $_ eq 'chroot/start-stop-daemon' } @{ $options->{skip} }) { info "skipping chroot/start-stop-daemon as requested"; } else { # $options->{root} must not be part of $ssdloc but must instead be # evaluated at the time the cleanup is run or otherwise, when # performing a pivot-root, the ssd location will still be prefixed # with the chroot path even though we changed root my $ssdloc; if (-f "$options->{root}/sbin/start-stop-daemon") { $ssdloc = "/sbin/start-stop-daemon"; } elsif (-f "$options->{root}/usr/sbin/start-stop-daemon") { $ssdloc = "/usr/sbin/start-stop-daemon"; } push @cleanup_tasks, sub { return unless length $ssdloc; if (-e "$options->{root}/$ssdloc.REAL") { move( "$options->{root}/$ssdloc.REAL", "$options->{root}/$ssdloc" ) or error "cannot move start-stop-daemon: $!"; } }; if (length $ssdloc) { if (-e "$options->{root}/$ssdloc.REAL") { error "$options->{root}/$ssdloc.REAL already exists"; } move( "$options->{root}/$ssdloc", "$options->{root}/$ssdloc.REAL" ) or error "cannot move start-stop-daemon: $!"; open my $fh, '>', "$options->{root}/$ssdloc" or error "cannot open start-stop-daemon: $!"; print $fh "#!/bin/sh\n"; print $fh "echo \"Warning: Fake start-stop-daemon called, doing" . " nothing\">&2\n"; close $fh; chmod 0755, "$options->{root}/$ssdloc" or error "cannot chmod start-stop-daemon: $!"; } } }; if ($@) { error "setup_mounts failed: $@"; } return @cleanup_tasks; } sub run_hooks { my $name = shift; my $options = shift; my $essential_pkgs = shift; if (scalar @{ $options->{"${name}_hook"} } == 0) { return; } if ($options->{dryrun}) { info "not running ${name}-hooks because of --dry-run"; return; } my @env_opts = (); # At this point TMPDIR is set to "$options->{root}/tmp". This is to have a # writable TMPDIR even in unshare mode. But if TMPDIR is still set when # running hooks, then every hook script calling chroot, will have to wrap # that into an "env --unset=TMPDIR". To avoid this, we unset TMPDIR here. # If the hook script needs a writable TMPDIR, then it can always use /tmp # inside the chroot. This is also why we do not set a new MMDEBSTRAP_TMPDIR # environment variable. if (length $ENV{TMPDIR}) { push @env_opts, '--unset=TMPDIR'; } # The APT_CONFIG variable, if set, will confuse any manual calls to # apt-get. If you want to use the same config used by mmdebstrap, the # original value is stored in MMDEBSTRAP_APT_CONFIG. if (length $ENV{APT_CONFIG}) { push @env_opts, '--unset=APT_CONFIG'; } if (length $ENV{APT_CONFIG}) { push @env_opts, "MMDEBSTRAP_APT_CONFIG=$ENV{APT_CONFIG}"; } # A hook script that wants to call mmdebstrap with --hook-helper needs to # know how mmdebstrap was executed push @env_opts, "MMDEBSTRAP_ARGV0=$PROGRAM_NAME"; # Storing the mode is important for hook scripts to potentially change # their behavior depending on the mode. It's also important for when the # hook wants to use the mmdebstrap --hook-helper. push @env_opts, "MMDEBSTRAP_MODE=$options->{mode}"; if (defined $options->{suite}) { push @env_opts, "MMDEBSTRAP_SUITE=$options->{suite}"; } push @env_opts, "MMDEBSTRAP_FORMAT=$options->{format}"; # Storing the hook name is important for hook scripts to potentially change # their behavior depending on the hook. It's also important for when the # hook wants to use the mmdebstrap --hook-helper. push @env_opts, "MMDEBSTRAP_HOOK=$name"; # This is the file descriptor of the socket that the mmdebstrap # --hook-helper can write to and read from to communicate with the outside. push @env_opts, ("MMDEBSTRAP_HOOKSOCK=" . fileno($options->{hooksock})); # Store the verbosity of mmdebstrap so that hooks can be just as verbose # as the mmdebstrap invocation that called them. push @env_opts, ("MMDEBSTRAP_VERBOSITY=" . $verbosity_level); # Store the packages given via --include in an environment variable so that # hooks can, for example, make .deb files available inside the chroot. { my @escaped_includes = @{ $options->{include} }; foreach my $incl (@escaped_includes) { # We have to encode commas so that values containing commas can # be stored in the list. Since we encode using percent-encoding # (urlencoding) we also have to encode the percent sign. $incl =~ s/%/%25/g; $incl =~ s/,/%2C/g; } push @env_opts, ("MMDEBSTRAP_INCLUDE=" . (join ",", @escaped_includes)); } # Give the extract hook access to the essential packages that are about to # be installed if ($name eq "extract" and scalar @{$essential_pkgs} > 0) { push @env_opts, ("MMDEBSTRAP_ESSENTIAL=" . (join " ", @{$essential_pkgs})); } if ($options->{mode} eq 'unshare') { push @env_opts, "container=mmdebstrap-unshare"; } # Unset the close-on-exec flag, so that the file descriptor does not # get closed when we exec my $flags = fcntl($options->{hooksock}, F_GETFD, 0) or error "fcntl F_GETFD: $!"; fcntl($options->{hooksock}, F_SETFD, $flags & ~FD_CLOEXEC) or error "fcntl F_SETFD: $!"; { foreach my $script (@{ $options->{"${name}_hook"} }) { my $type = $script->[0]; $script = $script->[1]; if ($type eq "pivoted") { info "running --chrooted-$name-hook in shell: sh -c " . "'$script'"; my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { # child my @cmdprefix = (); if ($options->{mode} eq 'fakechroot') { # we are calling the chroot executable instead of # chrooting the process so that fakechroot can handle # it @cmdprefix = ('chroot', $options->{root}); } elsif ($options->{mode} eq 'root') { # unsharing the mount namespace is not enough for # pivot_root to work as root (why?) unsharing the user # namespace as well (but without remapping) makes # pivot_root work (why??) but still makes later lazy # umounts fail (why???). Since pivot_root is mainly # useful for being able to run unshare mode inside # unshare mode, we fall back to just calling chroot() # until somebody has motivation and time to figure out # what is going on. chroot $options->{root} or error "failed to chroot(): $!"; $options->{root} = "/"; chdir "/" or error "failed chdir() to /: $!"; } elsif ($options->{mode} eq 'unshare') { 0 == syscall &SYS_unshare, $CLONE_NEWNS or error "unshare() failed: $!"; pivot_root($options->{root}); } else { error "unknown mode: $options->{mode}"; } 0 == system(@cmdprefix, 'env', @env_opts, 'sh', '-c', $script) or error "command failed: $script"; exit 0; } waitpid($pid, 0); $? == 0 or error "chrooted hook failed with exit code $?"; next; } # inode and device number of chroot before my ($dev_before, $ino_before, undef) = stat($options->{root}); if ( $script =~ /^( copy-in|copy-out |tar-in|tar-out |upload|download |sync-in|sync-out )\ /x ) { info "running special hook: $script"; if ((any { $_ eq $options->{variant} } ('extract', 'custom')) and $options->{mode} eq 'fakechroot' and $name ne 'setup') { info "the copy-in, copy-out, tar-in and tar-out commands" . " in fakechroot mode might fail in" . " extract and custom variants because there might be" . " no tar inside the chroot"; } my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { # whatever the script writes on stdout is sent to the # socket # whatever is written to the socket, send to stdin open(STDOUT, '>&', $options->{hooksock}) or error "cannot open STDOUT: $!"; open(STDIN, '<&', $options->{hooksock}) or error "cannot open STDIN: $!"; # Text::ParseWords::shellwords does for perl what shlex # does for python my @args = shellwords $script; hookhelper($options->{root}, $options->{mode}, $name, (join ',', @{ $options->{skip} }), $verbosity_level, @args); exit 0; } waitpid($pid, 0); $? == 0 or error "special hook failed with exit code $?"; } elsif (-x $script || $script !~ m/[^\w@\%+=:,.\/-]/a) { info "running --$name-hook directly: $script $options->{root}"; # execute it directly if it's an executable file # or if it there are no shell metacharacters # (the /a regex modifier makes \w match only ASCII) 0 == system('env', @env_opts, $script, $options->{root}) or error "command failed: $script"; } else { info "running --$name-hook in shell: sh -c '$script' exec" . " $options->{root}"; # otherwise, wrap everything in sh -c 0 == system('env', @env_opts, 'sh', '-c', $script, 'exec', $options->{root}) or error "command failed: $script"; } # If the chroot directory vanished, check if pivot_root was # performed. # # Running pivot_root is only really useful in the customize-hooks # because mmdebstrap uses apt from the outside to install packages # and that will fail after pivot_root because the process doesn't # have access to the system on the outside anymore. if (!-e $options->{root}) { my ($dev_root, $ino_root, undef) = stat("/"); if ($dev_before == $dev_root and $ino_before == $ino_root) { info "detected pivot_root, changing chroot directory to /"; # the old chroot directory is now / # the hook probably executed pivot_root $options->{root} = "/"; chdir "/" or error "failed chdir() to /: $!"; } else { error "chroot directory $options->{root} vanished"; } } } }; # Restore flags fcntl($options->{hooksock}, F_SETFD, $flags) or error "fcntl F_SETFD: $!"; return; } sub setup { my $options = shift; foreach my $key (sort keys %{$options}) { my $value = $options->{$key}; if (!defined $value) { next; } if (ref $value eq '') { debug "$key: $options->{$key}"; } elsif (ref $value eq 'ARRAY') { debug "$key: [" . (join ', ', @{$value}) . "]"; } elsif (ref $value eq 'GLOB') { debug "$key: GLOB"; } else { error "unknown type for key $key: " . (ref $value); } } if (-e $options->{apttrusted} && !-r $options->{apttrusted}) { # FIXME: obtain the keyring file with the permissions of the user # outside of the unshared namespace # FIXME: apt no longer sets Dir::Etc::trusted by default warning "cannot read $options->{apttrusted}"; } if (-e $options->{apttrustedparts} && !-r $options->{apttrustedparts}) { warning "cannot read $options->{apttrustedparts}"; } if (any { $_ eq 'setup' } @{ $options->{skip} }) { info "skipping setup as requested"; } else { run_setup($options); } run_hooks('setup', $options); # apt runs dpkg from inside the chroot and directly passes the filename to # dpkg. Hence, the included files on the outside must be present under the # same path on the inside. If they are not, dpkg cannot find them. if (scalar(grep { /^\// } @{ $options->{include} }) > 0) { my $ret = 0; foreach my $f (grep { /^\// } @{ $options->{include} }) { next if -e "$options->{root}/$f"; warning "path given via --include is not present inside the chroot: $f"; $ret = 1; } if ($ret != 0) { warning("apt runs chrooted dpkg which needs access to the " . "package paths given via --include inside the chroot."); warning "maybe try running mmdebstrap with " . "--hook-dir=/usr/share/mmdebstrap/hooks/file-mirror-automount"; } } if (any { $_ eq 'update' } @{ $options->{skip} }) { info "skipping update as requested"; } else { run_update($options); } (my $essential_pkgs, my $cached_debs) = run_download($options); # in theory, we don't have to extract the packages in chrootless mode # but we do it anyways because otherwise directory creation timestamps # will differ compared to non-chrootless and we want to create bit-by-bit # identical tar output # # FIXME: dpkg could be changed to produce the same results run_extract($options, $essential_pkgs); # setup mounts my @cleanup_tasks = (); my $cleanup = sub { my $signal = $_[0]; while (my $task = pop @cleanup_tasks) { $task->(); } if ($signal) { warning "pid $PID cought signal: $signal"; exit 1; } }; # we only need to setup the mounts if there is anything to do if ( $options->{variant} ne 'custom' or scalar @{ $options->{include} } > 0 or scalar @{ $options->{"extract_hook"} } > 0 or scalar @{ $options->{"essential_hook"} } > 0 or scalar @{ $options->{"customize_hook"} } > 0) { local $SIG{INT} = $cleanup; local $SIG{HUP} = $cleanup; local $SIG{PIPE} = $cleanup; local $SIG{TERM} = $cleanup; @cleanup_tasks = setup_mounts($options); } eval { my $chrootcmd = []; if ($options->{variant} ne 'extract') { if ($options->{mode} ne 'chrootless') { $chrootcmd = run_prepare($options); } } run_hooks('extract', $options, $essential_pkgs); if ($options->{variant} ne 'extract') { run_essential($options, $essential_pkgs, $chrootcmd, $cached_debs); run_hooks('essential', $options); run_install($options); run_hooks('customize', $options); } }; my $msg = $@; # Wait for (reap) potential zombies and otherwise long-running background # processes or otherwise they might hog resources like /dev/null which can # then not be unmounted resulting in their mountpoints (the regular files) # not being removable and then the removal of device nodes in run_cleanup # (if mmdebstrap is run with --skip=output/dev) will fail. if (any { $_ eq 'zombie-reaping' } @{ $options->{skip} }) { info "skipping zombie-reaping as requested"; } else { if (waitpid(-1, POSIX::WNOHANG) >= 0) { info "waiting for background processes to finish..."; } while ((my $child = waitpid(-1, 0)) > 0) { my $status = $? >> 8; info "PID $child exited with exit code $status"; } } $cleanup->(0); if ($msg) { error "setup failed: $msg"; } if (any { $_ eq 'cleanup' } @{ $options->{skip} }) { info "skipping cleanup as requested"; } else { run_cleanup($options); } return; } sub run_setup() { my $options = shift; { my @directories = ( '/etc/apt/apt.conf.d', '/etc/apt/sources.list.d', '/etc/apt/preferences.d', '/var/cache/apt', '/var/lib/apt/lists/partial', '/tmp' ); # we need /var/lib/dpkg in case we need to write to /var/lib/dpkg/arch push @directories, '/var/lib/dpkg'; # since we do not know the dpkg version inside the chroot at this # point, we can only omit it in chrootless mode if ($options->{mode} ne 'chrootless' or length $options->{dpkgopts} > 0) { push @directories, '/etc/dpkg/dpkg.cfg.d/'; } # if dpkg and apt operate from the outside we need some more # directories because dpkg and apt might not even be installed inside # the chroot. Thus, the following block is not strictly necessary in # chrootless mode. We unconditionally add it anyways, so that the # output with and without chrootless mode is equal. { push @directories, '/var/log/apt'; # since we do not know the dpkg version inside the chroot at this # point, we can only omit it in chrootless mode if ($options->{mode} ne 'chrootless') { push @directories, '/var/lib/dpkg/triggers', '/var/lib/dpkg/info', '/var/lib/dpkg/alternatives', '/var/lib/dpkg/updates'; } } foreach my $dir (@directories) { if (-e "$options->{root}/$dir") { if (!-d "$options->{root}/$dir") { error "$dir already exists but is not a directory"; } } else { my $num_created = make_path "$options->{root}/$dir", { error => \my $err }; if ($err && @$err) { error( join "; ", (map { "cannot create " . (join ": ", %{$_}) } @$err)); } elsif ($num_created == 0) { error "cannot create $options->{root}/$dir"; } } } # make sure /tmp is not 0755 like the rest chmod 01777, "$options->{root}/tmp" or error "cannot chmod /tmp: $!"; } # The TMPDIR set by the user or even /tmp might be inaccessible by the # unshared user. Thus, we place all temporary files in /tmp inside the new # rootfs. # # This will affect calls to tempfile() as well as runs of "apt-get update" # which will create temporary clearsigned.message.XXXXXX files to verify # signatures. # # Setting TMPDIR to inside the chroot is also necessary for when packages # are installed with apt from outside the chroot with # DPkg::Chroot-Directory { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{"TMPDIR"} = "$options->{root}/tmp"; } my ($conf, $tmpfile) = tempfile("mmdebstrap.apt.conf.XXXXXXXXXXXX", TMPDIR => 1) or error "cannot open apt.conf: $!"; print $conf "Apt::Architecture \"$options->{nativearch}\";\n"; # the host system might have configured additional architectures # force only the native architecture if (scalar @{ $options->{foreignarchs} } > 0) { print $conf "Apt::Architectures { \"$options->{nativearch}\"; "; foreach my $arch (@{ $options->{foreignarchs} }) { print $conf "\"$arch\"; "; } print $conf "};\n"; } else { print $conf "Apt::Architectures \"$options->{nativearch}\";\n"; } print $conf "Dir \"$options->{root}\";\n"; print $conf "DPkg::Chroot-Directory \"$options->{root}\";\n"; # not needed anymore for apt 1.3 and newer print $conf "Dir::State::Status \"$options->{root}/var/lib/dpkg/status\";\n"; # for authentication, use the keyrings from the host print $conf "Dir::Etc::Trusted \"$options->{apttrusted}\";\n"; print $conf "Dir::Etc::TrustedParts \"$options->{apttrustedparts}\";\n"; # apt considers itself essential. Thus, when generating an EDSP document # for an external solver, it will add the Essential:yes field to the apt # package stanza. This is unnecessary because we compile the set of # packages we consider essential ourselves and for the 'essential' variant # it would even be wrong to add apt. This workaround is only needed when # apt is used with an external solver but doesn't hurt otherwise and we # don't have a good way to figure out whether apt is using an external # solver or not short of parsing the --aptopt options. print $conf "pkgCacheGen::ForceEssential \",\";\n"; # Avoid running dpkg-preconfigure. This is not a problem unless in in # chrootless mode, which apt-extracttemplates does not and can not # support because it cannot know that this is a chrootless installation. # We always turn it off for equivalent behavior in all modes. Running # dpkg-preconfigure should not be needed as we also have set # DEBIAN_FRONTEND=noninteractive and DEBCONF_NONINTERACTIVE_SEEN=true and # should thus never see debconf prompts. See #1091442 for details. { open my $tmp, '>', "$options->{root}/etc/apt/apt.conf.d/99debconf" or error "cannot open /etc/apt/apt.conf.d/99debconf: $!"; print $tmp "#clear DPkg::Pre-Install-Pkgs;\n"; } close $conf; # We put certain configuration items in their own configuration file # because they have to be valid for apt invocation from outside as well as # from inside the chroot. # The config filename is chosen such that any settings in it will be # overridden by what the user specified with --aptopt. if (!-e "$options->{root}/etc/apt/apt.conf.d/00mmdebstrap") { open my $fh, '>', "$options->{root}/etc/apt/apt.conf.d/00mmdebstrap" or error "cannot open /etc/apt/apt.conf.d/00mmdebstrap: $!"; print $fh "Apt::Install-Recommends false;\n"; print $fh "Acquire::Languages \"none\";\n"; close $fh; } # apt-get update requires this if (!-e "$options->{root}/var/lib/dpkg/status") { open my $fh, '>', "$options->{root}/var/lib/dpkg/status" or error "failed to open(): $!"; close $fh; } # In theory, /var/lib/dpkg/arch is only useful if there are foreign # architectures configured or if the architecture of a chrootless chroot # is different from the native architecture outside the chroot. # We nevertheless always add /var/lib/dpkg/arch to make a chroot built the # normal way bit-by-bit identical to a foreign arch chroot built in # chrootless mode. if ((!-e "$options->{root}/var/lib/dpkg/arch")) { open my $fh, '>', "$options->{root}/var/lib/dpkg/arch" or error "cannot open /var/lib/dpkg/arch: $!"; print $fh "$options->{nativearch}\n"; foreach my $arch (@{ $options->{foreignarchs} }) { print $fh "$arch\n"; } close $fh; } if (length $options->{aptopts} > 0 and (!-e "$options->{root}/etc/apt/apt.conf.d/99mmdebstrap")) { open my $fh, '>', "$options->{root}/etc/apt/apt.conf.d/99mmdebstrap" or error "cannot open /etc/apt/apt.conf.d/99mmdebstrap: $!"; print $fh $options->{aptopts}; close $fh; if ($verbosity_level >= 3) { debug "content of /etc/apt/apt.conf.d/99mmdebstrap:"; copy("$options->{root}/etc/apt/apt.conf.d/99mmdebstrap", \*STDERR); } } if (length $options->{dpkgopts} > 0 and (!-e "$options->{root}/etc/dpkg/dpkg.cfg.d/99mmdebstrap")) { # FIXME: in chrootless mode, dpkg will only read the configuration # from the host -- see #808203 if ($options->{mode} eq 'chrootless') { warning('dpkg is unable to read an alternative configuration in' . 'chrootless mode -- see Debian bug #808203'); } open my $fh, '>', "$options->{root}/etc/dpkg/dpkg.cfg.d/99mmdebstrap" or error "cannot open /etc/dpkg/dpkg.cfg.d/99mmdebstrap: $!"; print $fh $options->{dpkgopts}; close $fh; if ($verbosity_level >= 3) { debug "content of /etc/dpkg/dpkg.cfg.d/99mmdebstrap:"; copy("$options->{root}/etc/dpkg/dpkg.cfg.d/99mmdebstrap", \*STDERR); } } if (!-e "$options->{root}/etc/fstab") { open my $fh, '>', "$options->{root}/etc/fstab" or error "cannot open fstab: $!"; print $fh "# UNCONFIGURED FSTAB FOR BASE SYSTEM\n"; close $fh; chmod 0644, "$options->{root}/etc/fstab" or error "cannot chmod fstab: $!"; } # write /etc/apt/sources.list and files in /etc/apt/sources.list.d/ if (scalar @{ $options->{sourceslists} } > 0) { my $firstentry = $options->{sourceslists}->[0]; # if the first sources.list entry is of one-line type and without # explicit filename, then write out an actual /etc/apt/sources.list # otherwise everything goes into /etc/apt/sources.list.d my $fname; if ($firstentry->{type} eq 'one-line' && !defined $firstentry->{fname}) { $fname = "$options->{root}/etc/apt/sources.list"; } else { $fname = "$options->{root}/etc/apt/sources.list.d/0000"; if (defined $firstentry->{fname}) { $fname .= $firstentry->{fname}; if ( $firstentry->{fname} !~ /\.list/ && $firstentry->{fname} !~ /\.sources/) { if ($firstentry->{type} eq 'one-line') { $fname .= '.list'; } elsif ($firstentry->{type} eq 'deb822') { $fname .= '.sources'; } else { error "invalid type: $firstentry->{type}"; } } } else { # if no filename is given, then this must be a deb822 file # because if it was a one-line type file, then it would've been # written to /etc/apt/sources.list $fname .= 'main.sources'; } } if (!-e $fname) { open my $fh, '>', "$fname" or error "cannot open $fname: $!"; print $fh $firstentry->{content}; close $fh; } # everything else goes into /etc/apt/sources.list.d/ for (my $i = 1 ; $i < scalar @{ $options->{sourceslists} } ; $i++) { my $entry = $options->{sourceslists}->[$i]; my $fname = "$options->{root}/etc/apt/sources.list.d/" . sprintf("%04d", $i); if (defined $entry->{fname}) { $fname .= $entry->{fname}; if ( $entry->{fname} !~ /\.list/ && $entry->{fname} !~ /\.sources/) { if ($entry->{type} eq 'one-line') { $fname .= '.list'; } elsif ($entry->{type} eq 'deb822') { $fname .= '.sources'; } else { error "invalid type: $entry->{type}"; } } } else { if ($entry->{type} eq 'one-line') { $fname .= 'main.list'; } elsif ($entry->{type} eq 'deb822') { $fname .= 'main.sources'; } else { error "invalid type: $entry->{type}"; } } if (!-e $fname) { open my $fh, '>', "$fname" or error "cannot open $fname: $!"; print $fh $entry->{content}; close $fh; } } } # allow network access from within foreach my $file ("/etc/resolv.conf", "/etc/hostname") { if (-e $file && !-e "$options->{root}/$file") { # this will create a new file with 644 permissions and copy # contents only even if $file was a symlink copy($file, "$options->{root}/$file") or error "cannot copy $file: $!"; # if the source was a regular file, preserve the permissions if (-f $file) { my $mode = (stat($file))[2]; $mode &= oct(7777); # mask off bits that aren't the mode chmod $mode, "$options->{root}/$file" or error "cannot chmod $file: $!"; } } elsif (-e $file && -e "$options->{root}/$file") { info "rootfs alreday contains $file"; } else { warning("Host system does not have a $file to copy into the" . " rootfs."); } } if ($options->{havemknod}) { foreach my $file (@linuxdevfiles) { my ($fname, $mode, $type, $linkname, $devmajor, $devminor, undef) = @{$file}; if ($type eq '0') { # normal file error "type 0 not implemented"; } elsif ($type eq '1') { # hardlink error "type 1 not implemented"; } elsif ($type eq '2') { # symlink if ( $options->{mode} eq 'fakechroot' and $linkname =~ /^\/proc/) { # there is no /proc in fakechroot mode next; } symlink $linkname, "$options->{root}/$fname" or error "cannot create symlink $fname"; next; # chmod cannot work on symlinks } elsif ($type eq '3') { # character special 0 == system('mknod', "$options->{root}/$fname", 'c', $devmajor, $devminor) or error "mknod failed: $?"; } elsif ($type eq '4') { # block special 0 == system('mknod', "$options->{root}/$fname", 'b', $devmajor, $devminor) or error "mknod failed: $?"; } elsif ($type eq '5') { # directory if (-e "$options->{root}/$fname") { if (!-d "$options->{root}/$fname") { error "$fname already exists but is not a directory"; } } else { my $num_created = make_path "$options->{root}/$fname", { error => \my $err }; if ($err && @$err) { error( join "; ", ( map { "cannot create " . (join ": ", %{$_}) } @$err )); } elsif ($num_created == 0) { error "cannot create $options->{root}/$fname"; } } } else { error "unsupported type: $type"; } chmod $mode, "$options->{root}/$fname" or error "cannot chmod $fname: $!"; } } # we tell apt about the configuration via a config file passed via the # APT_CONFIG environment variable instead of using the --option command # line arguments because configuration settings like Dir::Etc have already # been evaluated at the time that apt takes its command line arguments # into account. { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{"APT_CONFIG"} = "$tmpfile"; } # we have to make the config file world readable so that a possible # /usr/lib/apt/solvers/apt process which is run by the _apt user is also # able to read it chmod 0644, "$tmpfile" or error "cannot chmod $tmpfile: $!"; if ($verbosity_level >= 3) { 0 == system('apt-get', '--version') or error "apt-get --version failed: $?"; 0 == system('apt-config', 'dump') or error "apt-config failed: $?"; debug "content of $tmpfile:"; copy($tmpfile, \*STDERR); } if ($options->{mode} ne 'fakechroot') { # Apt dropping privileges to another user than root is not useful in # fakechroot mode because all users are faked and thus there is no real # privilege difference anyways. We could set APT::Sandbox::User "root" # in fakechroot mode but we don't because if we would, then # /var/cache/apt/archives/partial/ and /var/lib/apt/lists/partial/ # would not be owned by the _apt user if mmdebstrap was run in # fakechroot mode. # # when apt-get update is run by the root user, then apt will attempt to # drop privileges to the _apt user. This will fail if the _apt user # does not have permissions to read the root directory. In that case, # we have to disable apt sandboxing. This can for example happen in # root mode when the path of the chroot is not in a world-readable # location. my $partial = '/var/lib/apt/lists/partial'; my @testcmd = ( '/usr/lib/apt/apt-helper', 'drop-privs', '--', 'test', '-r', "$options->{root}$partial" ); my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { open(STDOUT, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; open(STDERR, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; exec { $testcmd[0] } @testcmd or error("cannot exec " . (join " ", @testcmd) . ": $!"); } waitpid $pid, 0; if ($? != 0) { warning "Download is performed unsandboxed as root as file" . " $options->{root}$partial couldn't be accessed by user _apt"; open my $fh, '>>', $tmpfile or error "cannot open $tmpfile for appending: $!"; print $fh "APT::Sandbox::User \"root\";\n"; close $fh; } } return; } sub run_update() { my $options = shift; my $aptopts = { ARGV => ['apt-get', 'update', '--error-on=any'], CHDIR => $options->{root}, }; # Maybe "apt-get update" was already run in the setup hook? If yes, skip # running it here. We are overly strict on purpose because better to run it # twice on accident than not at all. if ( !-d "$options->{root}/var/lib/apt/lists/auxfiles" || !-d "$options->{root}/var/lib/apt/lists/partial" || !-e "$options->{root}/var/lib/apt/lists/lock" || !-e "$options->{root}/var/cache/apt/pkgcache.bin" || !-e "$options->{root}/var/cache/apt/srcpkgcache.bin") { info "running apt-get update..."; run_apt_progress($aptopts); } else { info "skipping apt-get update because it was already run"; } # check if anything was downloaded at all { open my $fh, '-|', 'apt-get', 'indextargets' // error "failed to fork(): $!"; chomp( my $indextargets = do { local $/; <$fh> } ); close $fh; if ($indextargets eq '') { warning("apt-get indextargets output is empty"); if (scalar @{ $options->{sourceslists} } == 0) { warning "no known apt sources.list entry"; } for my $list (@{ $options->{sourceslists} }) { if (defined $list->{fname}) { info("Filename: $list->{fname}"); } info("Type: $list->{type}"); info("Content:"); for my $line (split "\n", $list->{content}) { info(" $line"); } } open(my $fh, '-|', 'apt-cache', 'policy') // error "failed to fork(): $!"; while (my $line = <$fh>) { chomp $line; info $line; } close $fh; my $msg = "apt-get update did not find any indices " . "for architecture '$options->{nativearch}' in "; if (length $options->{suite}) { $msg .= "suite '$options->{suite}'"; } else { $msg .= "the configured apt sources"; } error $msg; } } return; } sub run_download() { my $options = shift; # In the future we want to replace downloading packages with "apt-get # install" and installing them with dpkg by just installing the essential # packages with apt from the outside with DPkg::Chroot-Directory. # We are not doing that because then the preinst script of base-passwd will # not be called early enough and packages will fail to install because they # are missing /etc/passwd. my @cached_debs = (); my @dl_debs = (); if ( !$options->{dryrun} && ((none { $_ eq $options->{variant} } ('extract', 'custom')) || scalar @{ $options->{include} } != 0) && -d "$options->{root}/var/cache/apt/archives/" ) { my $apt_archives = "/var/cache/apt/archives/"; opendir my $dh, "$options->{root}/$apt_archives" or error "cannot read $apt_archives"; while (my $deb = readdir $dh) { if ($deb !~ /\.deb$/) { next; } if (!-f "$options->{root}/$apt_archives/$deb") { next; } push @cached_debs, $deb; } closedir $dh; } # To figure out the right package set for the apt variant we can use: # $ apt-get dist-upgrade -o dir::state::status=/dev/null # This is because that variants only contain essential packages and # apt and libapt treats apt as essential. If we want to install less # (essential variant) then we have to compute the package set ourselves. # Same if we want to install priority based variants. if (any { $_ eq $options->{variant} } ('extract', 'custom')) { if (scalar @{ $options->{include} } == 0) { info "nothing to download -- skipping..."; return ([], \@cached_debs); } my @apt_argv = ('install', @{ $options->{include} }); @dl_debs = run_apt_download_progress({ APT_ARGV => [@apt_argv], dryrun => $options->{dryrun}, }, ); } elsif (any { $_ eq $options->{variant} } ('essential', 'apt', 'standard', 'important', 'required', 'buildd')) { # 2021-06-07, #debian-apt on OFTC, times in UTC+2 # 17:27 < DonKult> (?essential includes 'apt' through) # 17:30 < josch> DonKult: no, because pkgCacheGen::ForceEssential ","; # 17:32 < DonKult> touché @dl_debs = run_apt_download_progress({ APT_ARGV => [ 'install', '?narrow(' . ( length($options->{suite}) ? '?or(?archive(^' . $options->{suite} . '$),?codename(^' . $options->{suite} . '$)),' : '' ) . '?architecture(' . $options->{nativearch} . '),?essential)' ], dryrun => $options->{dryrun}, }, ); } else { error "unknown variant: $options->{variant}"; } my @essential_pkgs; # strip the chroot directory from the filenames foreach my $deb (@dl_debs) { # if filename does not start with chroot directory then the user # might've used a file:// mirror and we check whether the path is # accessible inside the chroot if (rindex $deb, $options->{root}, 0) { if (!-e "$options->{root}/$deb") { error "package file $deb not accessible from chroot directory" . " -- use copy:// instead of file:// or a bind-mount. You" . " can also try using --hook-dir=/usr/share/mmdebstrap/" . "hooks/file-mirror-automount to automatically create" . " bind-mounts or copy the files as necessary."; } push @essential_pkgs, $deb; next; } # filename starts with chroot directory, strip it off # this is the normal case if (!-e $deb) { error "cannot find package file $deb"; } push @essential_pkgs, substr($deb, length($options->{root})); } return (\@essential_pkgs, \@cached_debs); } sub run_extract() { my $options = shift; my $essential_pkgs = shift; if ($options->{dryrun}) { info "skip extracting packages because of --dry-run"; return; } if (scalar @{$essential_pkgs} == 0) { info "nothing to extract -- skipping..."; return; } info "extracting archives..."; print_progress 0.0; my $counter = 0; my $total = scalar @{$essential_pkgs}; foreach my $deb (@{$essential_pkgs}) { $counter += 1; my $tarfilter; my @tarfilterargs; # if the path-excluded option was added to the dpkg config, # insert the tarfilter between dpkg-deb and tar if (-e "$options->{root}/etc/dpkg/dpkg.cfg.d/99mmdebstrap") { open(my $fh, '<', "$options->{root}/etc/dpkg/dpkg.cfg.d/99mmdebstrap") or error "cannot open /etc/dpkg/dpkg.cfg.d/99mmdebstrap: $!"; my @matches = grep { /^path-(?:exclude|include)=/ } <$fh>; close $fh; chop @matches; # remove trailing newline @tarfilterargs = map { "--" . $_ } @matches; } if (scalar @tarfilterargs > 0) { if (-x "./tarfilter") { $tarfilter = "./tarfilter"; } else { $tarfilter = "mmtarfilter"; } } my $dpkg_writer; my $tar_reader; my $filter_reader; my $filter_writer; if (scalar @tarfilterargs > 0) { pipe $filter_reader, $dpkg_writer or error "pipe failed: $!"; pipe $tar_reader, $filter_writer or error "pipe failed: $!"; } else { pipe $tar_reader, $dpkg_writer or error "pipe failed: $!"; } # not using dpkg-deb --extract as that would replace the # merged-usr symlinks with plain directories # even after switching from pre-merging to post-merging, dpkg-deb # will ignore filter rules from dpkg.cfg.d # https://bugs.debian.org/989602 # not using dpkg --unpack because that would try running preinst # maintainer scripts my $pid1 = fork() // error "fork() failed: $!"; if ($pid1 == 0) { open(STDOUT, '>&', $dpkg_writer) or error "cannot open STDOUT: $!"; close($tar_reader) or error "cannot close tar_reader: $!"; if (scalar @tarfilterargs > 0) { close($filter_reader) or error "cannot close filter_reader: $!"; close($filter_writer) or error "cannot close filter_writer: $!"; } debug("running dpkg-deb --fsys-tarfile $options->{root}/$deb"); eval { Devel::Cover::set_coverage("none") } if $is_covering; exec 'dpkg-deb', '--fsys-tarfile', "$options->{root}/$deb"; } my $pid2; if (scalar @tarfilterargs > 0) { $pid2 = fork() // error "fork() failed: $!"; if ($pid2 == 0) { open(STDIN, '<&', $filter_reader) or error "cannot open STDIN: $!"; open(STDOUT, '>&', $filter_writer) or error "cannot open STDOUT: $!"; close($dpkg_writer) or error "cannot close dpkg_writer: $!"; close($tar_reader) or error "cannot close tar_reader: $!"; debug("running $tarfilter " . (join " ", @tarfilterargs)); eval { Devel::Cover::set_coverage("none") } if $is_covering; exec $tarfilter, @tarfilterargs; } } my $pid3 = fork() // error "fork() failed: $!"; if ($pid3 == 0) { open(STDIN, '<&', $tar_reader) or error "cannot open STDIN: $!"; close($dpkg_writer) or error "cannot close dpkg_writer: $!"; if (scalar @tarfilterargs > 0) { close($filter_reader) or error "cannot close filter_reader: $!"; close($filter_writer) or error "cannot close filter_writer: $!"; } debug( "running tar -C $options->{root}" . " --keep-directory-symlink --extract --file -"); eval { Devel::Cover::set_coverage("none") } if $is_covering; exec 'tar', '-C', $options->{root}, '--keep-directory-symlink', '--extract', '--file', '-'; } close($dpkg_writer) or error "cannot close dpkg_writer: $!"; close($tar_reader) or error "cannot close tar_reader: $!"; if (scalar @tarfilterargs > 0) { close($filter_reader) or error "cannot close filter_reader: $!"; close($filter_writer) or error "cannot close filter_writer: $!"; } waitpid($pid1, 0); $? == 0 or error "dpkg-deb --fsys-tarfile failed: $?"; if (scalar @tarfilterargs > 0) { waitpid($pid2, 0); $? == 0 or error "tarfilter failed: $?"; } waitpid($pid3, 0); $? == 0 or error "tar --extract failed: $?"; print_progress($counter / $total * 100, "extracting"); } print_progress "done"; return; } sub run_prepare { my $options = shift; if ($options->{mode} eq 'fakechroot') { # this borrows from and extends # /etc/fakechroot/debootstrap.env and # /etc/fakechroot/chroot.env { my %subst = ( chroot => "/usr/sbin/chroot.fakechroot", mkfifo => "/bin/true", ldconfig => (getcwd() . '/ldconfig.fakechroot'), ldd => "/usr/bin/ldd.fakechroot", ischroot => "/bin/true" ); if (!-x $subst{ldconfig}) { $subst{ldconfig} = '/usr/libexec/mmdebstrap/ldconfig.fakechroot'; } my %mergedusrmap = ( "/bin" => "/usr/bin", "/sbin" => "/usr/sbin", "/usr/bin/" => "/bin", "/usr/sbin" => "/sbin" ); my %fakechrootsubst; foreach my $d (split ':', $ENV{PATH}) { foreach my $k (sort %subst) { my $mapped_path = $mergedusrmap{$d} // $d; next if !-e "$d/$k" && !-e "$mapped_path/$k"; $fakechrootsubst{"$d/$k=$subst{$k}"} = 1; $fakechrootsubst{"$mapped_path/$k=$subst{$k}"} = 1; } } if (defined $ENV{FAKECHROOT_CMD_SUBST} && $ENV{FAKECHROOT_CMD_SUBST} ne "") { foreach my $e (split /:/, $ENV{FAKECHROOT_CMD_SUBST}) { $fakechrootsubst{$e} = 1; } } ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{FAKECHROOT_CMD_SUBST} = join ':', (sort keys %fakechrootsubst); } if (defined $ENV{FAKECHROOT_EXCLUDE_PATH} && $ENV{FAKECHROOT_EXCLUDE_PATH} ne "") { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{FAKECHROOT_EXCLUDE_PATH} = "$ENV{FAKECHROOT_EXCLUDE_PATH}:/dev:/proc:/sys"; } else { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{FAKECHROOT_EXCLUDE_PATH} = '/dev:/proc:/sys'; } # workaround for long unix socket path if FAKECHROOT_BASE # exceeds the limit of 108 bytes { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{FAKECHROOT_AF_UNIX_PATH} = "/tmp"; } { my @ldlibpath = (); if (defined $ENV{LD_LIBRARY_PATH} && $ENV{LD_LIBRARY_PATH} ne "") { push @ldlibpath, (split /:/, $ENV{LD_LIBRARY_PATH}); } # FIXME: workaround allowing installation of systemd should # live in fakechroot, see #917920 push @ldlibpath, "$options->{root}/lib/systemd"; my $parse_ld_so_conf; $parse_ld_so_conf = sub { foreach my $conf (@_) { next if !-r $conf; open my $fh, '<', "$conf" or error "can't read $conf: $!"; while (my $line = <$fh>) { chomp $line; if ($line eq "") { next; } if ($line =~ /^#/) { next; } if ($line =~ /include (.*)/) { $parse_ld_so_conf->(glob("$options->{root}/$1")); next; } if (!-d "$options->{root}/$line") { next; } push @ldlibpath, "$options->{root}/$line"; } close $fh; } }; if (-e "$options->{root}/etc/ld.so.conf") { $parse_ld_so_conf->("$options->{root}/etc/ld.so.conf"); } ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{LD_LIBRARY_PATH} = join ':', @ldlibpath; } } # make sure that APT_CONFIG and TMPDIR are not set when executing # anything inside the chroot my @chrootcmd = ('env', '--unset=APT_CONFIG', '--unset=TMPDIR'); if (any { $_ eq $options->{mode} } ('root', 'unshare', 'fakechroot')) { push @chrootcmd, ('chroot', $options->{root}); } else { error "unknown mode: $options->{mode}"; } # foreign architecture setup for fakechroot mode if (defined $options->{qemu} && $options->{mode} eq 'fakechroot') { # Make sure that the fakeroot and fakechroot shared libraries exist for # the right architecture open my $fh, '-|', 'dpkg-architecture', '-a', $options->{nativearch}, '-qDEB_HOST_MULTIARCH' // error "failed to fork(): $!"; chomp( my $deb_host_multiarch = do { local $/; <$fh> } ); close $fh; if (($? != 0) or (!$deb_host_multiarch)) { error "dpkg-architecture failed: $?"; } my $fakechrootdir = "/usr/lib/$deb_host_multiarch/fakechroot"; if (!-e "$fakechrootdir/libfakechroot.so") { error "$fakechrootdir/libfakechroot.so doesn't exist." . " Install libfakechroot:$options->{nativearch}" . " outside the chroot"; } my $fakerootdir = "/usr/lib/$deb_host_multiarch/libfakeroot"; if (!-e "$fakerootdir/libfakeroot-sysv.so") { error "$fakerootdir/libfakeroot-sysv.so doesn't exist." . " Install libfakeroot:$options->{nativearch}" . " outside the chroot"; } # The rest of this block sets environment variables, so we have to add # the "no critic" statement to stop perlcritic from complaining about # setting global variables ## no critic (Variables::RequireLocalizedPunctuationVars) # fakechroot only fills LD_LIBRARY_PATH with the directories of the # host's architecture. We append the directories of the chroot # architecture. $ENV{LD_LIBRARY_PATH} = "$ENV{LD_LIBRARY_PATH}:$fakechrootdir:$fakerootdir"; # The binfmt support on the outside is used, so qemu needs to know # where it has to look for shared libraries if (defined $ENV{QEMU_LD_PREFIX} && $ENV{QEMU_LD_PREFIX} ne "") { $ENV{QEMU_LD_PREFIX} = "$ENV{QEMU_LD_PREFIX}:$options->{root}"; } else { $ENV{QEMU_LD_PREFIX} = $options->{root}; } } # some versions of coreutils use the renameat2 system call in mv. # This breaks certain versions of fakechroot. Here we do # a sanity check and warn the user in case things might break. if ($options->{mode} eq 'fakechroot' and -e "$options->{root}/bin/mv") { mkdir "$options->{root}/000-move-me" or error "cannot create directory: $!"; my $ret = system @chrootcmd, '/bin/mv', '/000-move-me', '/001-delete-me'; if ($ret != 0) { info "the /bin/mv binary inside the chroot doesn't" . " work under fakechroot"; info "with certain versions of coreutils and glibc," . " this is due to missing support for renameat2 in" . " fakechroot"; info "see https://github.com/dex4er/fakechroot/issues/60"; info "expect package post installation scripts not to work"; rmdir "$options->{root}/000-move-me" or error "cannot rmdir: $!"; } else { rmdir "$options->{root}/001-delete-me" or error "cannot rmdir: $!"; } } return \@chrootcmd; } sub run_essential() { my $options = shift; my $essential_pkgs = shift; my $chrootcmd = shift; my $cached_debs = shift; if (scalar @{$essential_pkgs} == 0) { info "no essential packages -- skipping..."; return; } if ($options->{mode} eq 'chrootless') { if ($options->{dryrun}) { info "simulate installing essential packages..."; } else { info "installing essential packages..."; } # FIXME: the dpkg config from the host is parsed before the command # line arguments are parsed and might break this mode # Example: if the host has --path-exclude set, then this will also # affect the chroot. See #808203 my @chrootless_opts = ( '-oDPkg::Chroot-Directory=', '-oDPkg::Options::=--force-not-root', '-oDPkg::Options::=--force-script-chrootless', '-oDPkg::Options::=--root=' . $options->{root}, '-oDPkg::Options::=--log=' . "$options->{root}/var/log/dpkg.log", $options->{dryrun} ? '-oAPT::Get::Simulate=true' : (), ); if (defined $options->{qemu}) { # The binfmt support on the outside is used, so qemu needs to know # where it has to look for shared libraries if (defined $ENV{QEMU_LD_PREFIX} && $ENV{QEMU_LD_PREFIX} ne "") { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{QEMU_LD_PREFIX} = "$ENV{QEMU_LD_PREFIX}:$options->{root}"; } else { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{QEMU_LD_PREFIX} = $options->{root}; } } # we don't use apt because that will not run the base-passwd preinst # early enough #run_apt_progress({ # ARGV => ['apt-get', '--yes', @chrootless_opts, 'install'], # PKGS => [map { "$options->{root}/$_" } @{$essential_pkgs}], #}); run_dpkg_progress({ ARGV => [ 'dpkg', '--force-not-root', '--force-script-chrootless', "--root=$options->{root}", "--log=$options->{root}/var/log/dpkg.log", '--install', '--force-depends' ], PKGS => [map { "$options->{root}/$_" } @{$essential_pkgs}] }); } elsif (any { $_ eq $options->{mode} } ('root', 'unshare', 'fakechroot')) { # install the extracted packages properly # we need --force-depends because dpkg does not take Pre-Depends # into account and thus doesn't install them in the right order # And the --predep-package option is broken: #539133 # # We could use apt from outside the chroot using DPkg::Chroot-Directory # but then the preinst script of base-passwd will not be called early # enough and packages will fail to install because they are missing # /etc/passwd. Also, with plain dpkg the essential variant can finish # within 9 seconds. If we use apt instead, it becomes 12 seconds. We # prefer speed here. if ($options->{dryrun}) { info "simulate installing essential packages..."; } else { info "installing essential packages..."; run_dpkg_progress({ ARGV => [@{$chrootcmd}, 'dpkg', '--install', '--force-depends'], PKGS => $essential_pkgs, }); } } else { error "unknown mode: $options->{mode}"; } if (any { $_ eq 'essential/unlink' } @{ $options->{skip} }) { info "skipping essential/unlink as requested"; } else { foreach my $deb (@{$essential_pkgs}) { # do not unlink those packages that were in /var/cache/apt/archive # before the download phase next if any { "/var/cache/apt/archives/$_" eq $deb } @{$cached_debs}; # do not unlink those packages that were not in # /var/cache/apt/archive (for example because they were provided by # a file:// mirror) next if $deb !~ /\/var\/cache\/apt\/archives\//; unlink "$options->{root}/$deb" or error "cannot unlink $deb: $!"; } } return; } sub run_install() { my $options = shift; my @pkgs_to_install = (@{ $options->{include} }); if ($options->{variant} eq 'extract') { error "must not be called with variant extract"; } if (none { $_ eq $options->{variant} } ('custom', 'essential')) { push @pkgs_to_install, 'apt'; } if ($options->{variant} eq 'buildd') { push @pkgs_to_install, 'build-essential'; } if (any { $_ eq $options->{variant} } ('required', 'important', 'standard')) { # Many of the priority:required packages are also essential:yes. We # make sure not to select those here to avoid useless "xxx is already # the newest version" messages. my $priority; if (any { $_ eq $options->{variant} } ('required')) { $priority = '?and(?priority(required),?not(?essential))'; } elsif ($options->{variant} eq 'important') { $priority = '?and(?or(?priority(required),?priority(important)),' . '?not(?essential))'; } elsif ($options->{variant} eq 'standard') { $priority = '?and(?or(~prequired,~pimportant,~pstandard),' . '?not(?essential))'; } push @pkgs_to_install, ( "?narrow(" . ( length($options->{suite}) ? '?or(?archive(^' . $options->{suite} . '$),?codename(^' . $options->{suite} . '$)),' : '' ) . "?architecture($options->{nativearch})," . "$priority)" ); } if ($options->{mode} eq 'chrootless') { if (scalar @pkgs_to_install > 0) { my @chrootless_opts = ( '-oDPkg::Chroot-Directory=', '-oDPkg::Options::=--force-not-root', '-oDPkg::Options::=--force-script-chrootless', '-oDPkg::Options::=--root=' . $options->{root}, '-oDPkg::Options::=--log=' . "$options->{root}/var/log/dpkg.log", $options->{dryrun} ? '-oAPT::Get::Simulate=true' : (), ); run_apt_progress({ ARGV => ['apt-get', '--yes', @chrootless_opts, 'install'], PKGS => [@pkgs_to_install], }); } } elsif (any { $_ eq $options->{mode} } ('root', 'unshare', 'fakechroot')) { if ($options->{variant} ne 'custom' and scalar @pkgs_to_install > 0) { # Advantage of running apt on the outside instead of inside the # chroot: # # - we can build chroots without apt (for example from buildinfo # files) # # - we do not need to install additional packages like # apt-transport-* or ca-certificates inside the chroot # # - we do not not need additional key material inside the chroot # # - we can make use of file:// and copy:// # # - we can use EDSP solvers without installing apt-utils or other # solvers inside the chroot # # The DPkg::Install::Recursive::force=true workaround can be # dropped after this issue is fixed: # https://salsa.debian.org/apt-team/apt/-/merge_requests/189 # # We could also move the dpkg call to the outside and run dpkg with # --root but this would only make sense in situations where there # is no dpkg inside the chroot. if (!$options->{dryrun}) { info "installing remaining packages inside the chroot..."; run_apt_progress({ ARGV => [ 'apt-get', '-o', 'Dir::Bin::dpkg=env', '-o', 'DPkg::Options::=--unset=TMPDIR', '-o', 'DPkg::Options::=dpkg', $options->{mode} eq 'fakechroot' ? ('-o', 'DPkg::Install::Recursive::force=true') : (), '--yes', 'install' ], PKGS => [@pkgs_to_install], }); } else { info "simulate installing remaining packages inside the" . " chroot..."; run_apt_progress({ ARGV => [ 'apt-get', '--yes', '-oAPT::Get::Simulate=true', 'install' ], PKGS => [@pkgs_to_install], }); } } } else { error "unknown mode: $options->{mode}"; } return; } sub run_cleanup() { my $options = shift; if (any { $_ eq 'cleanup/apt' } @{ $options->{skip} }) { info "skipping cleanup/apt as requested"; } else { if ( none { $_ eq 'cleanup/apt/lists' } @{ $options->{skip} } and none { $_ eq 'cleanup/apt/cache' } @{ $options->{skip} }) { info "cleaning package lists and apt cache..."; } if (any { $_ eq 'cleanup/apt/lists' } @{ $options->{skip} }) { info "skipping cleanup/apt/lists as requested"; } else { if (any { $_ eq 'cleanup/apt/cache' } @{ $options->{skip} }) { info "cleaning package lists..."; } run_apt_progress({ ARGV => [ 'apt-get', '--option', 'Dir::Etc::SourceList=/dev/null', '--option', 'Dir::Etc::SourceParts=/dev/null', 'update' ], CHDIR => $options->{root}, }); } if (any { $_ eq 'cleanup/apt/cache' } @{ $options->{skip} }) { info "skipping cleanup/apt/cache as requested"; } else { if (any { $_ eq 'cleanup/apt/lists' } @{ $options->{skip} }) { info "cleaning apt cache..."; } run_apt_progress( { ARGV => ['apt-get', 'clean'], CHDIR => $options->{root} }); } # apt since 1.6 creates the auxfiles directory. If apt inside the # chroot is older than that, then it will not know how to clean it. if (-e "$options->{root}/var/lib/apt/lists/auxfiles") { 0 == system( 'rm', '--interactive=never', '--recursive', '--preserve-root', '--one-file-system', "$options->{root}/var/lib/apt/lists/auxfiles" ) or error "rm failed: $?"; } } if (any { $_ eq 'cleanup/mmdebstrap' } @{ $options->{skip} }) { info "skipping cleanup/mmdebstrap as requested"; } else { # clean up temporary configuration file unlink "$options->{root}/etc/apt/apt.conf.d/00mmdebstrap" or warning "failed to unlink /etc/apt/apt.conf.d/00mmdebstrap: $!"; unlink "$options->{root}/etc/apt/apt.conf.d/99debconf" or warning "failed to unlink /etc/apt/apt.conf.d/99debconf: $!"; if (defined $ENV{APT_CONFIG} && -e $ENV{APT_CONFIG}) { unlink $ENV{APT_CONFIG} or error "failed to unlink $ENV{APT_CONFIG}: $!"; } } if (any { $_ eq 'cleanup/reproducible' } @{ $options->{skip} }) { info "skipping cleanup/reproducible as requested"; } else { # clean up certain files to make output reproducible foreach my $fname ( '/var/log/dpkg.log', '/var/log/apt/history.log', '/var/log/apt/term.log', '/var/log/alternatives.log', '/var/cache/ldconfig/aux-cache', '/var/log/apt/eipp.log.xz', '/var/lib/dbus/machine-id' ) { my $path = "$options->{root}$fname"; if (!-e $path) { next; } unlink $path or error "cannot unlink $path: $!"; } if (-e "$options->{root}/etc/machine-id") { # from machine-id(5): # For operating system images which are created once and used on # multiple machines, for example for containers or in the cloud, # /etc/machine-id should be an empty file in the generic file # system image. An ID will be generated during boot and saved to # this file if possible. Having an empty file in place is useful # because it allows a temporary file to be bind-mounted over the # real file, in case the image is used read-only. if (any { $_ eq 'cleanup/reproducible/machine-id' } @{ $options->{skip} }) { info "skipping cleanup/reproducible/machine-id as requested"; } else { unlink "$options->{root}/etc/machine-id" or error "cannot unlink /etc/machine-id: $!"; open my $fh, '>', "$options->{root}/etc/machine-id" or error "failed to open(): $!"; close $fh; } } } if (any { $_ eq 'cleanup/run' } @{ $options->{skip} }) { info "skipping cleanup/run as requested"; } else { # remove any possible leftovers in /run if (-d "$options->{root}/run") { opendir(my $dh, "$options->{root}/run") or error "Can't opendir($options->{root}/run): $!"; while (my $entry = readdir $dh) { # skip the "." and ".." entries next if $entry eq "."; next if $entry eq ".."; # skip deleting /run/lock as /var/lock is a symlink to it # according to Debian policy §9.1.4 next if $entry eq "lock"; debug "deleting files in /run: $entry"; 0 == system( 'rm', '--interactive=never', '--recursive', '--preserve-root', '--one-file-system', "$options->{root}/run/$entry" ) or error "rm failed: $?"; } closedir($dh); } } if (any { $_ eq 'cleanup/tmp' } @{ $options->{skip} }) { info "skipping cleanup/tmp as requested"; } else { # remove any possible leftovers in /tmp if (-d "$options->{root}/tmp") { opendir(my $dh, "$options->{root}/tmp") or error "Can't opendir($options->{root}/tmp): $!"; while (my $entry = readdir $dh) { # skip the "." and ".." entries next if $entry eq "."; next if $entry eq ".."; debug "deleting files in /tmp: $entry"; 0 == system( 'rm', '--interactive=never', '--recursive', '--preserve-root', '--one-file-system', "$options->{root}/tmp/$entry" ) or error "rm failed: $?"; } closedir($dh); } } if (any { $_ eq 'cleanup/dev' } @{ $options->{skip} }) { info "skipping cleanup/dev as requested"; } else { # By default, tar is run with --exclude=./dev because we create the # ./dev entries ourselves using @devfiles. But if --skip=output/dev is # used, --exclude=./dev is not passed so that the chroot includes ./dev # as created by base-files. But if mknod was available (for example # when running as root) then ./dev will also include the @devfiles # entries created by run_setup() and thus the resulting tarball will # include things inside ./dev despite the user having supplied # --skip=output/dev. So if --skip=output/dev was passed and if a # tarball is to be created, we need to make sure to clean up the # ./dev entries that were created in run_setup(). This is not done # when creating a directory because in that case we want to do the # same as debootstrap and create a directory including device nodes. if ($options->{format} ne 'directory' && any { $_ eq 'output/dev' } @{ $options->{skip} }) { foreach my $file (@linuxdevfiles) { my ($fname, $mode, $type, $linkname, $devmajor, $devminor, undef) = @{$file}; if (!-e "$options->{root}/$fname") { next; } # do not remove ./dev itself if ($fname eq "./dev/") { next; } if ($type == 0) { # normal file error "type 0 not implemented"; } elsif ($type == 1) { # hardlink error "type 1 not implemented"; } elsif (any { $_ eq $type } (2, 3, 4)) { # symlink, char, block unlink "$options->{root}/$fname" or error "failed to unlink $fname: $!"; } elsif ($type == 5) { # directory rmdir "$options->{root}/$fname" or error "failed to unlink $fname: $!"; } else { error "unsupported type: $type"; } } } } return; } # messages from process inside unshared namespace to the outside # openw -- open file for writing # untar -- extract tar into directory # write -- write data to last opened file or tar process # close -- finish file writing or tar extraction # adios -- last message and tear-down # messages from process outside unshared namespace to the inside # okthx -- success sub checkokthx { my $fh = shift; my $ret = read($fh, my $buf, 2 + 5) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } my ($len, $msg) = unpack("nA5", $buf); if ($msg ne "okthx") { error "expected okthx but got: $msg"; } if ($len != 0) { error "expected no payload but got $len bytes"; } return; } # resolve a path inside a chroot sub chrooted_realpath { my $root = shift; my $src = shift; my $result = $root; my $prefix; # relative paths are relative to the root of the chroot # remove prefixed slashes $src =~ s{^/+}{}; my $loop = 0; while (length $src) { if ($loop > 25) { error "too many levels of symbolic links"; } # Get the first directory component. ($prefix, $src) = split m{/+}, $src, 2; # Resolve the first directory component. if ($prefix eq ".") { # Ignore, stay at the same directory. } elsif ($prefix eq "..") { # Go up one directory. $result =~ s{(.*)/[^/]*}{$1}; # but not further than the root if ($result !~ m/^\Q$root\E/) { $result = $root; } } elsif (-l "$result/$prefix") { my $dst = readlink "$result/$prefix"; if ($dst =~ s{^/+}{}) { # Absolute pathname, reset result back to $root. $result = $root; } $src = length $src ? "$dst/$src" : $dst; $loop++; } else { # Otherwise append the prefix. $result = "$result/$prefix"; } } return $result; } sub pivot_root { my $root = shift; my $target = "/mnt"; my $put_old = "tmp"; 0 == syscall &SYS_mount, $root, $target, 0, $MS_REC | $MS_BIND, 0 or error "mount failed: $!"; chdir "/mnt" or error "failed chdir() to /mnt: $!"; 0 == syscall &SYS_pivot_root, my $new_root = ".", $put_old or error "pivot_root failed: $!"; chroot "." or error "failed to chroot() to .: $!"; 0 == syscall &SYS_umount2, $put_old, $MNT_DETACH or error "umount2 failed: $!"; 0 == syscall &SYS_umount2, my $sys = "sys", $MNT_DETACH or error "umount2 failed: $!"; return; } sub hookhelper { my ($root, $mode, $hook, $skipopt, $verbosity, $command, @args) = @_; $verbosity_level = $verbosity; my @skipopts = (); if (length $skipopt) { for my $skip (split /[,\s]+/, $skipopt) { # strip leading and trailing whitespace $skip =~ s/^\s+|\s+$//g; # skip if the remainder is an empty string if ($skip eq '') { next; } push @skipopts, $skip; } } # we put everything in an eval block because that way we can easily handle # errors without goto labels or much code duplication: the error handler # has to send an "error" message to the other side eval { my @cmdprefix = (); my @tarcmd = ( 'tar', '--numeric-owner', '--xattrs', '--format=pax', '--pax-option=exthdr.name=%d/PaxHeaders/%f,' . 'delete=atime,delete=ctime' ); if ($hook eq 'setup') { } elsif (any { $_ eq $hook } ('extract', 'essential', 'customize')) { if ($mode eq 'fakechroot') { # Fakechroot requires tar to run inside the chroot or # otherwise absolute symlinks will include the path to the # root directory push @cmdprefix, 'chroot', $root; } elsif (any { $_ eq $mode } ('root', 'chrootless', 'unshare')) { # not chrooting in this case } else { error "unknown mode: $mode"; } } else { error "unknown hook: $hook"; } if (any { $_ eq $command } ('copy-in', 'tar-in', 'upload', 'sync-in')) { if (scalar @args < 2) { error "$command needs at least one path on the" . " outside and the output path inside the chroot"; } my $outpath = pop @args; foreach my $file (@args) { # the right argument for tar's --directory argument depends on # whether tar is called from inside the chroot or from the # outside my $directory; if ($hook eq 'setup') { # tar runs outside, so acquire the correct path $directory = chrooted_realpath $root, $outpath; } elsif (any { $_ eq $hook } ('extract', 'essential', 'customize')) { if ($mode eq 'fakechroot') { # tar will run inside the chroot $directory = $outpath; } elsif (any { $_ eq $mode } ('root', 'chrootless', 'unshare')) { $directory = chrooted_realpath $root, $outpath; } else { error "unknown mode: $mode"; } } else { error "unknown hook: $hook"; } # if chrooted_realpath was used and if fakechroot # was used (absolute symlinks will be broken) we can # check and potentially fail early if the target does not exist if ($mode ne 'fakechroot') { my $dirtocheck = $directory; if ($command eq 'upload') { # check the parent directory instead $dirtocheck =~ s/(.*)\/[^\/]*/$1/; } if (!-e $dirtocheck) { error "path does not exist: $dirtocheck"; } if (!-d $dirtocheck) { error "path is not a directory: $dirtocheck"; } } my $fh; if ($command eq 'upload') { # open the requested file for writing open $fh, '|-', @cmdprefix, 'sh', '-c', 'cat > "$1"', 'exec', $directory // error "failed to fork(): $!"; } elsif (any { $_ eq $command } ('copy-in', 'tar-in', 'sync-in')) { # open a tar process that extracts the tarfile that we # supply it with on stdin to the output directory inside # the chroot my @cmd = ( @cmdprefix, @tarcmd, '--xattrs-include=*', '--directory', $directory, '--extract', '--file', '-' ); # go via mmtarfilter if copy-in/mknod, tar-in/mknod or # sync-in/mknod were part of the skip options if (any { $_ eq "$command/mknod" } @skipopts) { info "skipping $command/mknod as requested"; my $tarfilter = "mmtarfilter"; if (-x "./tarfilter") { $tarfilter = "./tarfilter"; } pipe my $filter_reader, $fh or error "pipe failed: $!"; pipe my $tar_reader, my $filter_writer or error "pipe failed: $!"; my $pid1 = fork() // error "fork() failed: $!"; if ($pid1 == 0) { open(STDIN, '<&', $filter_reader) or error "cannot open STDIN: $!"; open(STDOUT, '>&', $filter_writer) or error "cannot open STDOUT: $!"; close($tar_reader) or error "cannot close tar_reader: $!"; debug( "helper: running $tarfilter --type-exclude=3 " . "--type-exclude=4"); eval { Devel::Cover::set_coverage("none") } if $is_covering; exec $tarfilter, '--type-exclude=3', '--type-exclude=4'; } my $pid2 = fork() // error "fork() failed: $!"; if ($pid2 == 0) { open(STDIN, '<&', $tar_reader) or error "cannot open STDIN: $!"; close($filter_writer) or error "cannot close filter_writer: $!"; debug("helper: running " . (join " ", @cmd)); eval { Devel::Cover::set_coverage("none") } if $is_covering; exec { $cmd[0] } @cmd; } } else { debug("helper: running " . (join " ", @cmd)); open($fh, '|-', @cmd) // error "failed to fork(): $!"; } } else { error "unknown command: $command"; } if ($command eq 'copy-in') { # instruct the parent process to create a tarball of the # requested path outside the chroot debug "helper: sending mktar"; print STDOUT (pack("n", length $file) . "mktar" . $file); } elsif ($command eq 'sync-in') { # instruct the parent process to create a tarball of the # content of the requested path outside the chroot debug "helper: sending mktac"; print STDOUT (pack("n", length $file) . "mktac" . $file); } elsif (any { $_ eq $command } ('upload', 'tar-in')) { # instruct parent process to open a tarball of the # requested path outside the chroot for reading debug "helper: sending openr"; print STDOUT (pack("n", length $file) . "openr" . $file); } else { error "unknown command: $command"; } STDOUT->flush(); debug "helper: waiting for okthx"; checkokthx \*STDIN; # handle "write" messages from the parent process and feed # their payload into the tar process until a "close" message # is encountered while (1) { # receive the next message my $ret = read(STDIN, my $buf, 2 + 5) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } my ($len, $msg) = unpack("nA5", $buf); debug "helper: received message: $msg"; if ($msg eq "close") { # finish the loop if ($len != 0) { error "expected no payload but got $len bytes"; } debug "helper: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); last; } elsif ($msg ne "write") { error "expected write but got: $msg"; } # read the payload my $content; { my $ret = read(STDIN, $content, $len) // error "error cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # write the payload to the tar process print $fh $content or error "cannot write to tar process: $!"; debug "helper: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); } close $fh; if ($command ne 'upload' and $? != 0) { error "tar failed"; } } } elsif (any { $_ eq $command } ('copy-out', 'tar-out', 'download', 'sync-out')) { if (scalar @args < 2) { error "$command needs at least one path inside the chroot and" . " the output path on the outside"; } my $outpath = pop @args; foreach my $file (@args) { # the right argument for tar's --directory argument depends on # whether tar is called from inside the chroot or from the # outside my $directory; if ($hook eq 'setup') { # tar runs outside, so acquire the correct path $directory = chrooted_realpath $root, $file; } elsif (any { $_ eq $hook } ('extract', 'essential', 'customize')) { if ($mode eq 'fakechroot') { # tar will run inside the chroot $directory = $file; } elsif (any { $_ eq $mode } ('root', 'chrootless', 'unshare')) { $directory = chrooted_realpath $root, $file; } else { error "unknown mode: $mode"; } } else { error "unknown hook: $hook"; } # if chrooted_realpath was used and if fakechroot # was used (absolute symlinks will be broken) we can # check and potentially fail early if the source does not exist if ($mode ne 'fakechroot') { if (!-e $directory) { error "path does not exist: $directory"; } if ($command eq 'download') { if (!-f $directory) { error "path is not a file: $directory"; } } } my $fh; if ($command eq 'download') { # open the requested file for reading open $fh, '-|', @cmdprefix, 'sh', '-c', 'cat "$1"', 'exec', $directory // error "failed to fork(): $!"; } elsif ($command eq 'sync-out') { # Open a tar process that creates a tarfile of everything # inside the requested directory inside the chroot and # writes it to stdout. my @cmd = ( @cmdprefix, @tarcmd, '--directory', $directory, '--create', '--file', '-', '.' ); debug("helper: running " . (join " ", @cmd)); open($fh, '-|', @cmd) // error "failed to fork(): $!"; } elsif (any { $_ eq $command } ('copy-out', 'tar-out')) { # Open a tar process that creates a tarfile of the # requested directory inside the chroot and writes it to # stdout. To emulate the behaviour of cp, change to the # dirname of the requested path first. my @cmd = ( @cmdprefix, @tarcmd, '--directory', dirname($directory), '--create', '--file', '-', basename($directory)); debug("helper: running " . (join " ", @cmd)); open($fh, '-|', @cmd) // error "failed to fork(): $!"; } else { error "unknown command: $command"; } if (any { $_ eq $command } ('copy-out', 'sync-out')) { # instruct the parent process to extract a tarball to a # certain path outside the chroot debug "helper: sending untar"; print STDOUT ( pack("n", length $outpath) . "untar" . $outpath); } elsif (any { $_ eq $command } ('download', 'tar-out')) { # instruct parent process to open a tarball of the # requested path outside the chroot for writing debug "helper: sending openw"; print STDOUT ( pack("n", length $outpath) . "openw" . $outpath); } else { error "unknown command: $command"; } STDOUT->flush(); debug "helper: waiting for okthx"; checkokthx \*STDIN; # read from the tar process and send as payload to the parent # process while (1) { # read from tar my $ret = read($fh, my $cont, 4096) // error "cannot read from pipe: $!"; if ($ret == 0) { last; } debug "helper: sending write"; # send to parent print STDOUT pack("n", $ret) . "write" . $cont; STDOUT->flush(); debug "helper: waiting for okthx"; checkokthx \*STDIN; if ($ret < 4096) { last; } } # signal to the parent process that we are done debug "helper: sending close"; print STDOUT pack("n", 0) . "close"; STDOUT->flush(); debug "helper: waiting for okthx"; checkokthx \*STDIN; close $fh; if ($? != 0) { error "$command failed"; } } } else { error "unknown command: $command"; } }; if ($@) { # inform the other side that something went wrong print STDOUT (pack("n", 0) . "error"); STDOUT->flush(); error "hookhelper failed: $@"; } return; } sub hooklistener { $verbosity_level = shift; # we put everything in an eval block because that way we can easily handle # errors without goto labels or much code duplication: the error handler # has to send an "error" message to the other side eval { while (1) { # get the next message my $msg = "error"; my $len = -1; { debug "listener: reading next command"; my $ret = read(STDIN, my $buf, 2 + 5) // error "cannot read from socket: $!"; debug "listener: finished reading command"; if ($ret == 0) { error "received eof on socket"; } ($len, $msg) = unpack("nA5", $buf); } if ($msg eq "adios") { debug "listener: received message: adios"; # setup finished, so we break out of the loop if ($len != 0) { error "expected no payload but got $len bytes"; } last; } elsif ($msg eq "openr") { # handle the openr message debug "listener: received message: openr"; my $infile; { my $ret = read(STDIN, $infile, $len) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # make sure that the requested path exists outside the chroot if (!-e $infile) { error "$infile does not exist"; } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); open my $fh, '<', $infile or error "failed to open $infile for reading: $!"; # read from the file and send as payload to the child process while (1) { # read from file my $ret = read($fh, my $cont, 4096) // error "cannot read from pipe: $!"; if ($ret == 0) { last; } debug "listener: sending write"; # send to child print STDOUT pack("n", $ret) . "write" . $cont; STDOUT->flush(); debug "listener: waiting for okthx"; checkokthx \*STDIN; if ($ret < 4096) { last; } } # signal to the child process that we are done debug "listener: sending close"; print STDOUT pack("n", 0) . "close"; STDOUT->flush(); debug "listener: waiting for okthx"; checkokthx \*STDIN; close $fh; } elsif ($msg eq "openw") { debug "listener: received message: openw"; # payload is the output directory my $outfile; { my $ret = read(STDIN, $outfile, $len) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # make sure that the directory exists my $outdir = dirname($outfile); if (-e $outdir) { if (!-d $outdir) { error "$outdir already exists but is not a directory"; } } else { my $num_created = make_path $outdir, { error => \my $err }; if ($err && @$err) { error( join "; ", ( map { "cannot create " . (join ": ", %{$_}) } @$err )); } elsif ($num_created == 0) { error "cannot create $outdir"; } } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); # now we expect one or more "write" messages containing the # tarball to write open my $fh, '>', $outfile or error "failed to open $outfile for writing: $!"; # handle "write" messages from the child process and feed # their payload into the file handle until a "close" message # is encountered while (1) { # receive the next message my $ret = read(STDIN, my $buf, 2 + 5) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } my ($len, $msg) = unpack("nA5", $buf); debug "listener: received message: $msg"; if ($msg eq "close") { # finish the loop if ($len != 0) { error "expected no payload but got $len bytes"; } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); last; } elsif ($msg ne "write") { # we should not receive this message at this point error "expected write but got: $msg"; } # read the payload my $content; { my $ret = read(STDIN, $content, $len) // error "error cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # write the payload to the file handle print $fh $content or error "cannot write to file handle: $!"; debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); } close $fh; } elsif (any { $_ eq $msg } ('mktar', 'mktac')) { # handle the mktar message debug "listener: received message: $msg"; my $indir; { my $ret = read(STDIN, $indir, $len) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # make sure that the requested path exists outside the chroot if (!-e $indir) { error "$indir does not exist"; } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); # Open a tar process creating a tarfile of the instructed # path. To emulate the behaviour of cp, change to the # dirname of the requested path first. my @cmd = ( 'tar', '--numeric-owner', '--xattrs', '--format=pax', '--pax-option=exthdr.name=%d/PaxHeaders/%f,' . 'delete=atime,delete=ctime', '--directory', $msg eq 'mktar' ? dirname($indir) : $indir, '--create', '--file', '-', $msg eq 'mktar' ? basename($indir) : '.' ); debug("listener: running " . (join " ", @cmd)); open(my $fh, '-|', @cmd) // error "failed to fork(): $!"; # read from the tar process and send as payload to the child # process while (1) { # read from tar my $ret = read($fh, my $cont, 4096) // error "cannot read from pipe: $!"; if ($ret == 0) { last; } debug "listener: sending write ($ret bytes)"; # send to child print STDOUT pack("n", $ret) . "write" . $cont; STDOUT->flush(); debug "listener: waiting for okthx"; checkokthx \*STDIN; if ($ret < 4096) { last; } } # signal to the child process that we are done debug "listener: sending close"; print STDOUT pack("n", 0) . "close"; STDOUT->flush(); debug "listener: waiting for okthx"; checkokthx \*STDIN; close $fh; if ($? != 0) { error "tar failed"; } } elsif ($msg eq "untar") { debug "listener: received message: untar"; # payload is the output directory my $outdir; { my $ret = read(STDIN, $outdir, $len) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # make sure that the directory exists if (-e $outdir) { if (!-d $outdir) { error "$outdir already exists but is not a directory"; } } else { my $num_created = make_path $outdir, { error => \my $err }; if ($err && @$err) { error( join "; ", ( map { "cannot create " . (join ": ", %{$_}) } @$err )); } elsif ($num_created == 0) { error "cannot create $outdir"; } } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); # now we expect one or more "write" messages containing the # tarball to unpack open my $fh, '|-', 'tar', '--numeric-owner', '--xattrs', '--xattrs-include=*', '--directory', $outdir, '--extract', '--file', '-' // error "failed to fork(): $!"; # handle "write" messages from the child process and feed # their payload into the tar process until a "close" message # is encountered while (1) { # receive the next message my $ret = read(STDIN, my $buf, 2 + 5) // error "cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } my ($len, $msg) = unpack("nA5", $buf); debug "listener: received message: $msg"; if ($msg eq "close") { # finish the loop if ($len != 0) { error "expected no payload but got $len bytes"; } debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); last; } elsif ($msg ne "write") { # we should not receive this message at this point error "expected write but got: $msg"; } # read the payload my $content; { my $ret = read(STDIN, $content, $len) // error "error cannot read from socket: $!"; if ($ret == 0) { error "received eof on socket"; } } # write the payload to the tar process print $fh $content or error "cannot write to tar process: $!"; debug "listener: sending okthx"; print STDOUT (pack("n", 0) . "okthx") or error "cannot write to socket: $!"; STDOUT->flush(); } close $fh; if ($? != 0) { error "tar failed"; } } elsif ($msg eq "error") { error "received error on socket"; } else { error "unknown message: $msg"; } } }; if ($@) { warning("hooklistener errored out: $@"); # inform the other side that something went wrong print STDOUT (pack("n", 0) . "error") or error "cannot write to socket: $!"; STDOUT->flush(); } return; } # parse files of the format found in /usr/share/distro-info/ and return two # lists: the first contains codenames of end-of-life distros and the second # list contains codenames of currently active distros sub parse_distro_info { my $file = shift; my @eol = (); my @current = (); my $today = POSIX::strftime "%Y-%m-%d", localtime; open my $fh, '<', $file or error "cannot open $file: $!"; my $i = 0; while (my $line = <$fh>) { chomp($line); $i++; my @cells = split /,/, $line; if (scalar @cells < 4) { error "cannot parse line $i of $file"; } if ( $i == 1 and ( scalar @cells < 6 or $cells[0] ne 'version' or $cells[1] ne 'codename' or $cells[2] ne 'series' or $cells[3] ne 'created' or $cells[4] ne 'release' or $cells[5] ne 'eol') ) { error "cannot find correct header in $file"; } if ($i == 1) { next; } if (scalar @cells == 6) { if ($cells[5] !~ m/^\d\d\d\d-\d\d-\d\d$/) { error "invalid eof date format in $file:$i: $cells[5]"; } # since the date format is iso8601, we can use lexicographic string # comparison to compare dates if ($cells[5] lt $today) { push @eol, $cells[2]; } else { push @current, $cells[2]; } } else { push @current, $cells[2]; } } close $fh; return ([@eol], [@current]); } sub get_suite_by_vendor { my %suite_by_vendor = ( 'debian' => {}, 'ubuntu' => {}, 'tanglu' => {}, 'kali' => {}, 'elxr' => {}, ); # pre-fill with some known values foreach my $suite ( 'potato', 'woody', 'sarge', 'etch', 'lenny', 'squeeze', 'wheezy', 'jessie' ) { $suite_by_vendor{'debian'}->{$suite} = 1; } foreach my $suite ( 'unstable', 'stable', 'oldstable', 'stretch', 'buster', 'bullseye', 'bookworm', 'trixie' ) { $suite_by_vendor{'debian'}->{$suite} = 0; } foreach my $suite ('aequorea', 'bartholomea', 'chromodoris', 'dasyatis') { $suite_by_vendor{'tanglu'}->{$suite} = 0; } foreach my $suite ('kali-dev', 'kali-rolling', 'kali-bleeding-edge') { $suite_by_vendor{'kali'}->{$suite} = 0; } foreach my $suite ('aria') { $suite_by_vendor{'elxr'}->{$suite} = 0; } foreach my $suite ('trusty', 'xenial', 'zesty', 'artful', 'bionic', 'cosmic') { $suite_by_vendor{'ubuntu'}->{$suite} = 0; } # if the Debian package distro-info-data is installed, then we can use it, # to get better data about new distros or EOL distros if (-e '/usr/share/distro-info/debian.csv') { my ($eol, $current) = parse_distro_info('/usr/share/distro-info/debian.csv'); foreach my $suite (@{$eol}) { $suite_by_vendor{'debian'}->{$suite} = 1; } foreach my $suite (@{$current}) { $suite_by_vendor{'debian'}->{$suite} = 0; } } if (-e '/usr/share/distro-info/ubuntu.csv') { my ($eol, $current) = parse_distro_info('/usr/share/distro-info/ubuntu.csv'); foreach my $suite (@{$eol}, @{$current}) { $suite_by_vendor{'ubuntu'}->{$suite} = 0; } } # if debootstrap is installed we infer distro names from the symlink # targets of the scripts in /usr/share/debootstrap/scripts/ my $debootstrap_scripts = '/usr/share/debootstrap/scripts/'; if (-d $debootstrap_scripts) { opendir(my $dh, $debootstrap_scripts) or error "Can't opendir($debootstrap_scripts): $!"; while (my $suite = readdir $dh) { # this is only a heuristic -- don't overwrite anything but instead # just update anything that was missing if (!-l "$debootstrap_scripts/$suite") { next; } my $target = readlink "$debootstrap_scripts/$suite"; if ($target eq "sid" and not exists $suite_by_vendor{'debian'}->{$suite}) { $suite_by_vendor{'debian'}->{$suite} = 0; } elsif ($target eq "gutsy" and not exists $suite_by_vendor{'ubuntu'}->{$suite}) { $suite_by_vendor{'ubuntu'}->{$suite} = 0; } elsif ($target eq "aequorea" and not exists $suite_by_vendor{'tanglu'}->{$suite}) { $suite_by_vendor{'tanglu'}->{$suite} = 0; } elsif ($target eq "kali" and not exists $suite_by_vendor{'kali'}->{$suite}) { $suite_by_vendor{'kali'}->{$suite} = 0; } elsif ($target eq "elxr" and not exists $suite_by_vendor{'elxr'}->{$suite}) { $suite_by_vendor{'elxr'}->{$suite} = 0; } } closedir($dh); } return %suite_by_vendor; } # try to guess the right keyring path for the given suite sub get_keyring_by_suite { my $query = shift; my $suite_by_vendor = shift; my $debianvendor; my $ubuntuvendor; # make $@ local, so we don't print "Can't locate Dpkg/Vendor/Debian.pm" # in other parts where we evaluate $@ local $@ = ''; eval { require Dpkg::Vendor::Debian; require Dpkg::Vendor::Ubuntu; $debianvendor = Dpkg::Vendor::Debian->new(); $ubuntuvendor = Dpkg::Vendor::Ubuntu->new(); }; my $keyring_by_vendor = sub { my $vendor = shift; my $eol = shift; if ($vendor eq 'debian') { if ($eol) { if (defined $debianvendor) { return $debianvendor->run_hook( 'archive-keyrings-historic'); } else { return '/usr/share/keyrings/debian-archive-removed-keys.gpg'; } } else { if (defined $debianvendor) { return $debianvendor->run_hook('archive-keyrings'); } else { return '/usr/share/keyrings/debian-archive-keyring.gpg'; } } } elsif ($vendor eq 'ubuntu') { if (defined $ubuntuvendor) { return $ubuntuvendor->run_hook('archive-keyrings'); } else { return '/usr/share/keyrings/ubuntu-archive-keyring.gpg'; } } elsif ($vendor eq 'tanglu') { return '/usr/share/keyrings/tanglu-archive-keyring.gpg'; } elsif ($vendor eq 'kali') { return '/usr/share/keyrings/kali-archive-keyring.gpg'; } elsif ($vendor eq 'elxr') { return '/usr/share/keyrings/elxr-archive-keyring.gpg'; } else { error "unknown vendor: $vendor"; } }; my %keyrings = (); foreach my $vendor (keys %{$suite_by_vendor}) { foreach my $suite (keys %{ $suite_by_vendor->{$vendor} }) { my $keyring = $keyring_by_vendor->( $vendor, $suite_by_vendor->{$vendor}->{$suite}); debug "suite $suite with keyring $keyring"; $keyrings{$suite} = $keyring; } } if (exists $keyrings{$query}) { return $keyrings{$query}; } else { return; } } sub get_sourceslist_by_suite { my $suite = shift; my $arch = shift; my $signedby = shift; my $compstr = shift; my $suite_by_vendor = shift; if (!$suite) { error "get_sourceslist_by_suite cannot operate with an empty suite"; } my @debstable = keys %{ $suite_by_vendor->{'debian'} }; my @ubuntustable = keys %{ $suite_by_vendor->{'ubuntu'} }; my @tanglustable = keys %{ $suite_by_vendor->{'tanglu'} }; my @kali = keys %{ $suite_by_vendor->{'kali'} }; my @elxr = keys %{ $suite_by_vendor->{'elxr'} }; my $mirror = 'http://deb.debian.org/debian'; my $secmirror = 'http://security.debian.org/debian-security'; if (any { $_ eq $suite } @ubuntustable) { if (any { $_ eq $arch } ('amd64', 'i386')) { $mirror = 'http://archive.ubuntu.com/ubuntu'; $secmirror = 'http://security.ubuntu.com/ubuntu'; } else { $mirror = 'http://ports.ubuntu.com/ubuntu-ports'; $secmirror = 'http://ports.ubuntu.com/ubuntu-ports'; } if (-e '/usr/share/debootstrap/scripts/gutsy') { # try running the debootstrap script but ignore errors my $script = 'set -eu; default_mirror() { echo $1; }; mirror_style() { :; }; download_style() { :; }; finddebs_style() { :; }; variants() { :; }; keyring() { :; }; doing_variant() { false; }; info() { fmt="$2"; shift; shift; printf "I: $fmt\n" "$@" >&2; }; . /usr/share/debootstrap/scripts/gutsy;'; open my $fh, '-|', 'env', "ARCH=$arch", "SUITE=$suite", 'sh', '-c', $script // last; chomp( my $output = do { local $/; <$fh> } ); close $fh; if ($? == 0 && $output ne '') { $mirror = $output; } } } elsif (any { $_ eq $suite } @tanglustable) { $mirror = 'http://archive.tanglu.org/tanglu'; } elsif (any { $_ eq $suite } @kali) { $mirror = 'https://http.kali.org/kali'; } elsif (any { $_ eq $suite } @elxr) { $mirror = 'https://mirror.elxr.dev'; } my $sourceslist = ''; $sourceslist .= "deb$signedby $mirror $suite $compstr\n"; if (any { $_ eq $suite } @ubuntustable) { $sourceslist .= "deb$signedby $mirror $suite-updates $compstr\n"; $sourceslist .= "deb$signedby $secmirror $suite-security $compstr\n"; } elsif (any { $_ eq $suite } @tanglustable) { $sourceslist .= "deb$signedby $secmirror $suite-updates $compstr\n"; } elsif (any { $_ eq $suite } @debstable and none { $_ eq $suite } ('testing', 'unstable', 'sid')) { $sourceslist .= "deb$signedby $mirror $suite-updates $compstr\n"; # the security mirror changes, starting with bullseye # https://lists.debian.org/87r26wqr2a.fsf@43-1.org my $bullseye_or_later = 0; if (any { $_ eq $suite } ('oldstable', 'stable', 'bullseye', 'bookworm', 'trixie')) { $bullseye_or_later = 1; } my $distro_info = '/usr/share/distro-info/debian.csv'; # make $@ local, so we don't print "Can't locate Debian/DistroInfo.pm" # in other parts where we evaluate $@ local $@ = ''; eval { require Debian::DistroInfo; }; if (!$@) { debug "libdistro-info-perl is installed"; my $debinfo = DebianDistroInfo->new(); if ($debinfo->version($suite, 0) >= 11) { $bullseye_or_later = 1; } } elsif (-f $distro_info) { debug "distro-info-data is installed"; open my $fh, '<', $distro_info or error "cannot open $distro_info: $!"; my $i = 0; my $matching_version; my @releases; my $today = POSIX::strftime "%Y-%m-%d", localtime; while (my $line = <$fh>) { chomp($line); $i++; my @cells = split /,/, $line; if (scalar @cells < 4) { error "cannot parse line $i of $distro_info"; } if ( $i == 1 and ( scalar @cells < 6 or $cells[0] ne 'version' or $cells[1] ne 'codename' or $cells[2] ne 'series' or $cells[3] ne 'created' or $cells[4] ne 'release' or $cells[5] ne 'eol') ) { error "cannot find correct header in $distro_info"; } if ($i == 1) { next; } if ( scalar @cells > 4 and $cells[4] =~ m/^\d\d\d\d-\d\d-\d\d$/ and $cells[4] lt $today) { push @releases, $cells[0]; } if (lc $cells[1] eq $suite or lc $cells[2] eq $suite) { $matching_version = $cells[0]; last; } } close $fh; if (defined $matching_version and $matching_version >= 11) { $bullseye_or_later = 1; } if ($suite eq "stable" and $releases[-1] >= 11) { $bullseye_or_later = 1; } } else { debug "neither libdistro-info-perl nor distro-info-data installed"; } if ($bullseye_or_later) { # starting from bullseye use $sourceslist .= "deb$signedby $secmirror $suite-security" . " $compstr\n"; } else { $sourceslist .= "deb$signedby $secmirror $suite/updates" . " $compstr\n"; } } return $sourceslist; } sub guess_sources_format { my $content = shift; my $is_deb822 = 0; my $is_oneline = 0; for my $line (split "\n", $content) { if ($line =~ /^deb(-src)? /) { $is_oneline = 1; last; } if ($line =~ /^[^#:\s]+:/) { $is_deb822 = 1; last; } } if ($is_deb822) { return 'deb822'; } if ($is_oneline) { return 'one-line'; } return; } sub approx_disk_usage { my $directory = shift; my $block_size = shift; info "approximating disk usage..."; # the "du" utility reports different results depending on the underlying # filesystem, see https://bugs.debian.org/650077 for a discussion # # we use code similar to the one used by dpkg-gencontrol instead # # Regular files are measured in number of $block_size byte blocks. All # other entries are assumed to take one block of space. # # We ignore /dev because depending on the mode, the directory might be # populated or not and we want consistent disk usage results independent # of the mode. my $installed_size = 0; my %hardlink; my $scan_installed_size = sub { if ($File::Find::name eq "$directory/dev") { # add all entries of @devfiles once $installed_size += scalar @linuxdevfiles; return; } elsif ($File::Find::name =~ /^$directory\/dev\//) { # ignore everything below /dev return; } lstat or error "cannot stat $File::Find::name"; if (-f _ or -l _) { my ($dev, $ino, $nlink) = (lstat _)[0, 1, 3]; return if exists $hardlink{"$dev:$ino"}; # Track hardlinks to avoid repeated additions. $hardlink{"$dev:$ino"} = 1 if $nlink > 1; # add file size in $block_size byte blocks, rounded up $installed_size += int(((-s _) + $block_size) / $block_size); } else { # all other entries are assumed to only take up one block $installed_size += 1; } }; # We use no_chdir because otherwise the unshared user has to have read # permissions for the current working directory when producing an ext2 # image. See https://bugs.debian.org/1005857 find({ wanted => $scan_installed_size, no_chdir => 1 }, $directory); # the above is only a heuristic and especially ext4 will consume quite a # few more blocks than the heuristic above is going to compute return int($installed_size * 1.2); } sub main() { my $before = Time::HiRes::time; umask 022; if (scalar @ARGV >= 7 && $ARGV[0] eq "--hook-helper") { shift @ARGV; # shift off "--hook-helper" hookhelper(@ARGV); exit 0; } # this is the counterpart to --hook-helper and will receive and carry # out its instructions if (scalar @ARGV == 2 && $ARGV[0] eq "--hook-listener") { hooklistener($ARGV[1]); exit 0; } # this is like: # lxc-usernsexec -- lxc-unshare -s 'MOUNT|PID|UTSNAME|IPC' ... # but without needing lxc if (scalar @ARGV >= 1 && $ARGV[0] eq "--unshare-helper") { if ($EFFECTIVE_USER_ID != 0) { test_unshare_userns(1); } my @idmap = (); if ($EFFECTIVE_USER_ID != 0) { @idmap = read_subuid_subgid 1; } my $pid = get_unshare_cmd( sub { 0 == system @ARGV[1 .. $#ARGV] or error "system failed: $?"; }, \@idmap ); waitpid $pid, 0; $? == 0 or error "unshared command failed"; exit 0; } my $mtime = time; if (exists $ENV{SOURCE_DATE_EPOCH}) { $mtime = $ENV{SOURCE_DATE_EPOCH} + 0; } { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{DEBIAN_FRONTEND} = 'noninteractive'; $ENV{DEBCONF_NONINTERACTIVE_SEEN} = 'true'; $ENV{LC_ALL} = 'C.UTF-8'; $ENV{LANGUAGE} = 'C.UTF-8'; $ENV{LANG} = 'C.UTF-8'; } # copy ARGV because getopt modifies it my @ARGVORIG = @ARGV; # obtain the correct defaults for the keyring locations that apt knows # about my $apttrusted = `eval \$(apt-config shell v Dir::Etc::trusted/f); printf %s \$v`; my $apttrustedparts = `eval \$(apt-config shell v Dir::Etc::trustedparts/d); printf %s \$v`; chomp(my $hostarch = `dpkg --print-architecture`); my $options = { components => ["main"], variant => "important", include => [], architectures => [$hostarch], mode => 'auto', format => 'auto', dpkgopts => '', aptopts => '', apttrusted => $apttrusted, apttrustedparts => $apttrustedparts, noop => [], setup_hook => [], extract_hook => [], essential_hook => [], customize_hook => [], dryrun => 0, skip => [], }; my $logfile = undef; Getopt::Long::Configure('default', 'bundling', 'auto_abbrev', 'ignore_case_always'); GetOptions( 'h|help' => sub { pod2usage(-exitval => 0, -verbose => 1) }, 'man' => sub { pod2usage(-exitval => 0, -verbose => 2) }, 'version' => sub { print STDOUT "mmdebstrap $VERSION\n"; exit 0; }, 'components=s@' => \$options->{components}, 'variant=s' => \$options->{variant}, 'include=s' => sub { my ($opt_name, $opt_value) = @_; my $sanitize_path = sub { my $pkg = shift; $pkg = abs_path($pkg) // error "cannot resolve absolute path of $pkg: $!"; if ($pkg !~ /^\//) { error "absolute path of $pkg doesn't start with a slash"; } if (!-f $pkg) { error "$pkg is not an existing file"; } if (!-r $pkg) { error "$pkg is not readable"; } return $pkg; }; if ($opt_value =~ /^[?~!(]/) { # Treat option as a single apt pattern and don't split by comma # or whitespace -- append it verbatim. push @{ $options->{include} }, $opt_value; } elsif ($opt_value =~ /^\.?\.?\//) { # Treat option as a single path name and don't split by comma # or whitespace -- append the normalized path. push @{ $options->{include} }, &{$sanitize_path}($opt_value); } else { for my $pkg (split /[,\s]+/, $opt_value) { # strip leading and trailing whitespace $pkg =~ s/^\s+|\s+$//g; # skip if the remainder is an empty string if ($pkg eq '') { next; } # Make paths canonical absolute paths, resolve symlinks # and check if it's an existing file. if ($pkg =~ /^\.?\.?\//) { $pkg = &{$sanitize_path}($pkg); } push @{ $options->{include} }, $pkg; } } # We are not sorting or otherwise normalizing the order of # arguments to apt because package order matters for "apt install" # since https://salsa.debian.org/apt-team/apt/-/merge_requests/256 }, 'architectures=s@' => \$options->{architectures}, 'mode=s' => \$options->{mode}, 'dpkgopt=s' => sub { my ($opt_name, $opt_value) = @_; if (-r $opt_value) { open my $fh, '<', $opt_value or error "failed to open $opt_value: $!"; $options->{dpkgopts} .= do { local $/; <$fh> }; if ($options->{dpkgopts} !~ /\n$/) { print $fh "\n"; } close $fh; } else { $options->{dpkgopts} .= $opt_value; if ($opt_value !~ /\n$/) { $options->{dpkgopts} .= "\n"; } } }, 'aptopt=s' => sub { my ($opt_name, $opt_value) = @_; if (-r $opt_value) { open my $fh, '<', $opt_value or error "failed to open $opt_value: $!"; $options->{aptopts} .= do { local $/; <$fh> }; if ($options->{aptopts} !~ /\n$/) { print $fh "\n"; } close $fh; } else { $options->{aptopts} .= $opt_value; if ($opt_value !~ /;$/) { $options->{aptopts} .= ';'; } if ($opt_value !~ /\n$/) { $options->{aptopts} .= "\n"; } } }, 'keyring=s' => sub { my ($opt_name, $opt_value) = @_; if ($opt_value =~ /"/) { error "--keyring: apt cannot handle paths with double quotes:" . " $opt_value"; } if (!-e $opt_value) { error "keyring \"$opt_value\" does not exist"; } my $abs_path = abs_path($opt_value); if (!defined $abs_path) { error "unable to get absolute path of --keyring: $opt_value"; } # since abs_path resolved all symlinks for us, we can now test # what the actual target actually is if (-d $abs_path) { $options->{apttrustedparts} = $abs_path; } else { $options->{apttrusted} = $abs_path; } }, 's|silent' => sub { $verbosity_level = 0; }, 'q|quiet' => sub { $verbosity_level = 0; }, 'v|verbose' => sub { $verbosity_level = 2; }, 'd|debug' => sub { $verbosity_level = 3; }, 'format=s' => \$options->{format}, 'logfile=s' => \$logfile, # no-op options so that mmdebstrap can be used with # sbuild-createchroot --debootstrap=mmdebstrap 'resolve-deps' => sub { push @{ $options->{noop} }, 'resolve-deps'; }, 'merged-usr' => sub { push @{ $options->{noop} }, 'merged-usr'; }, 'no-merged-usr' => sub { push @{ $options->{noop} }, 'no-merged-usr'; }, 'force-check-gpg' => sub { push @{ $options->{noop} }, 'force-check-gpg'; }, 'setup-hook=s' => sub { push @{ $options->{setup_hook} }, ["normal", $_[1]]; }, 'extract-hook=s' => sub { push @{ $options->{extract_hook} }, ["normal", $_[1]]; }, 'chrooted-extract-hook=s' => sub { push @{ $options->{extract_hook} }, ["pivoted", $_[1]]; }, 'essential-hook=s' => sub { push @{ $options->{essential_hook} }, ["normal", $_[1]]; }, 'chrooted-essential-hook=s' => sub { push @{ $options->{essential_hook} }, ["pivoted", $_[1]]; }, 'customize-hook=s' => sub { push @{ $options->{customize_hook} }, ["normal", $_[1]]; }, 'chrooted-customize-hook=s' => sub { push @{ $options->{customize_hook} }, ["pivoted", $_[1]]; }, 'hook-directory=s' => sub { my ($opt_name, $opt_value) = @_; if (!-e $opt_value) { error "hook directory \"$opt_value\" does not exist"; } my $abs_path = abs_path($opt_value); if (!defined $abs_path) { error( "unable to get absolute path of " . "--hook-directory: $opt_value"); } # since abs_path resolved all symlinks for us, we can now test # what the actual target actually is if (!-d $opt_value) { error "hook directory \"$opt_value\" is not a directory"; } # gather all files starting with special prefixes into the # respective keys of a hash my %scripts; my $count = 0; opendir(my $dh, $opt_value) or error "Can't opendir($opt_value): $!"; while (my $entry = readdir $dh) { # skip the "." and ".." entries next if $entry eq "."; next if $entry eq ".."; my $found = 0; foreach my $hook ('setup', 'extract', 'essential', 'customize') { if ($entry =~ m/^\Q$hook\E/) { if (-x "$opt_value/$entry") { push @{ $scripts{$hook} }, "$opt_value/$entry"; $count += 1; $found = 1; } else { warning("$opt_value/$entry is named like a " . "hook but not executable"); } } } if (!$found && -x "$opt_value/$entry") { warning("$opt_value/$entry: is executable " . "but not prefixed with a hook name"); } } closedir($dh); if ($count == 0) { warning "No executable hook scripts found in $opt_value"; return; } # add the sorted list associated with each key to the respective # list of hooks foreach my $hook (keys %scripts) { push @{ $options->{"${hook}_hook"} }, (map { ["normal", $_] } (sort @{ $scripts{$hook} })); } }, # Sometimes --simulate fails even though non-simulate succeeds because # in simulate mode, apt cannot rely on dpkg to figure out tricky # dependency situations and will give up instead when it cannot find # a solution. # # 2020-02-06, #debian-apt on OFTC, times in UTC+1 # 12:52 < DonKult> [...] It works in non-simulation because simulate is # more picky. If you wanna know why simulate complains # here prepare for long suffering in dependency hell. 'simulate' => \$options->{dryrun}, 'dry-run' => \$options->{dryrun}, 'skip=s' => sub { my ($opt_name, $opt_value) = @_; for my $skip (split /[,\s]+/, $opt_value) { # strip leading and trailing whitespace $skip =~ s/^\s+|\s+$//g; # skip if the remainder is an empty string if ($skip eq '') { next; } push @{ $options->{skip} }, $skip; } }) or pod2usage(-exitval => 2, -verbose => 0); if (defined($logfile)) { open(STDERR, '>', $logfile) or error "cannot open $logfile: $!"; } foreach my $arg (@{ $options->{noop} }) { info "the option --$arg is a no-op. It only exists for compatibility" . " with some debootstrap wrappers."; } if ($options->{dryrun}) { foreach my $hook ('setup', 'extract', 'essential', 'customize') { if (scalar @{ $options->{"${hook}_hook"} } > 0) { warning "In dry-run mode, --$hook-hook options have no effect"; } if ($options->{mode} eq 'chrootless') { foreach my $script (@{ $options->{"${hook}_hook"} }) { if ($script->[0] eq "pivoted") { error "--chrooted-$hook-hook are illegal in " . "chrootless mode"; } } } } } my @valid_variants = ( 'extract', 'custom', 'essential', 'apt', 'required', 'minbase', 'buildd', 'important', 'debootstrap', '-', 'standard' ); if (none { $_ eq $options->{variant} } @valid_variants) { error "invalid variant. Choose from " . (join ', ', @valid_variants); } # debootstrap and - are an alias for important if (any { $_ eq $options->{variant} } ('-', 'debootstrap')) { $options->{variant} = 'important'; } # minbase is an alias for required if ($options->{variant} eq 'minbase') { $options->{variant} = 'required'; } # fakeroot is an alias for fakechroot if ($options->{mode} eq 'fakeroot') { $options->{mode} = 'fakechroot'; } # sudo is an alias for root if ($options->{mode} eq 'sudo') { $options->{mode} = 'root'; } my @valid_modes = ('auto', 'root', 'unshare', 'fakechroot', 'chrootless'); if (none { $_ eq $options->{mode} } @valid_modes) { error "invalid mode. Choose from " . (join ', ', @valid_modes); } # sqfs is an alias for squashfs if ($options->{format} eq 'sqfs') { $options->{format} = 'squashfs'; } # dir is an alias for directory if ($options->{format} eq 'dir') { $options->{format} = 'directory'; } my @valid_formats = ('auto', 'directory', 'tar', 'squashfs', 'ext2', 'ext4', 'null'); if (none { $_ eq $options->{format} } @valid_formats) { error "invalid format. Choose from " . (join ', ', @valid_formats); } # setting PATH for chroot, ldconfig, start-stop-daemon... my $defaultpath = `eval \$(apt-config shell v DPkg::Path); printf %s \$v`; if (length $ENV{PATH}) { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{PATH} = "$ENV{PATH}:$defaultpath"; } else { ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{PATH} = $defaultpath; } foreach my $tool ( 'dpkg', 'dpkg-deb', 'apt-get', 'apt-cache', 'apt-config', 'tar', 'rm', 'find', 'env' ) { if (!can_execute $tool) { error "cannot find $tool"; } } { my $dpkgversion = version->new(0); my $pid = open my $fh, '-|' // error "failed to fork(): $!"; if ($pid == 0) { # redirect stderr to /dev/null to hide error messages from dpkg # versions before 1.20.0 open(STDERR, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; exec 'dpkg', '--robot', '--version'; } chomp( my $content = do { local $/; <$fh> } ); close $fh; # the --robot option was introduced in 1.20.0 but until 1.20.2 the # output contained a string after the version, separated by a # whitespace -- since then, it's only the version if ($? == 0 and $content =~ /^([0-9.]+).*$/) { # dpkg is new enough for the --robot option $dpkgversion = version->new($1); } if ($dpkgversion < "1.20.0") { error "need dpkg >= 1.20.0 but have $dpkgversion"; } } { my $aptversion = version->new(0); my $pid = open my $fh, '-|', 'apt-get', '--version' // error "failed to fork(): $!"; chomp( my $content = do { local $/; <$fh> } ); close $fh; if ( $? == 0 and $content =~ /^apt (\d+\.\d+\.\d+)\S* \(\S+\)$/am) { $aptversion = version->new($1); } if ($aptversion < "2.3.14") { error "need apt >= 2.3.14 but have $aptversion"; } } my $check_fakechroot_running = sub { # test if we are inside fakechroot already # We fork a child process because setting FAKECHROOT_DETECT seems to # be an irreversible operation for fakechroot. my $pid = open my $rfh, '-|' // error "failed to fork(): $!"; if ($pid == 0) { # with the FAKECHROOT_DETECT environment variable set, any program # execution will be replaced with the output "fakeroot [version]" local $ENV{FAKECHROOT_DETECT} = 0; exec 'echo', 'If fakechroot is running, this will not be printed'; } my $content = do { local $/; <$rfh> }; waitpid $pid, 0; my $result = 0; if ($? == 0 and $content =~ /^fakechroot [0-9.]+$/) { $result = 1; } return $result; }; # figure out the mode to use or test whether the chosen mode is legal if ($options->{mode} eq 'auto') { if (&{$check_fakechroot_running}()) { # if mmdebstrap is executed inside fakechroot, then we assume the # user expects fakechroot mode $options->{mode} = 'fakechroot'; } elsif ($EFFECTIVE_USER_ID == 0) { # if mmdebstrap is executed as root, we assume the user wants root # mode $options->{mode} = 'root'; } elsif (test_unshare_userns(0)) { # if we are not root, unshare mode is our best option if # test_unshare_userns() succeeds $options->{mode} = 'unshare'; } elsif (can_execute 'fakechroot') { # the next fallback is fakechroot # exec ourselves again but within fakechroot my @prefix = (); if ($is_covering) { @prefix = ($EXECUTABLE_NAME, '-MDevel::Cover=-silent,-nogcov'); } exec 'fakechroot', 'fakeroot', @prefix, $PROGRAM_NAME, @ARGVORIG; } else { error( "unable to pick chroot mode automatically (use --mode for " . "manual selection)"); } info "automatically chosen mode: $options->{mode}"; } elsif ($options->{mode} eq 'root') { if ($EFFECTIVE_USER_ID != 0) { error "need to be root"; } } elsif ($options->{mode} eq 'fakechroot') { if (&{$check_fakechroot_running}()) { # fakechroot is already running } elsif (!can_execute 'fakechroot') { error "need working fakechroot binary"; } else { # exec ourselves again but within fakechroot my @prefix = (); if ($is_covering) { @prefix = ($EXECUTABLE_NAME, '-MDevel::Cover=-silent,-nogcov'); } exec 'fakechroot', 'fakeroot', @prefix, $PROGRAM_NAME, @ARGVORIG; } } elsif ($options->{mode} eq 'unshare') { # For unshare mode to work we either need to already be the root user # and then we do not have to unshare the user namespace anymore but we # need to be able to unshare the mount namespace... # # We need to call unshare with "--propagation unchanged" or otherwise # we get 'cannot change root filesystem propagation' when running # mmdebstrap inside a chroot for which the root of the chroot is not # its own mount point. if ($EFFECTIVE_USER_ID == 0 && 0 != system 'unshare --mount --propagation unchanged -- true') { error "unable to unshare the mount namespace"; } # ...or we are not root and then we need to be able to unshare the user # namespace. if ($EFFECTIVE_USER_ID != 0) { test_unshare_userns(1); } } elsif ($options->{mode} eq 'chrootless') { if (any { $_ eq 'check/chrootless' } @{ $options->{skip} }) { info "skipping check/chrootless as requested"; } else { my $ischroot = 0 == system 'ischroot'; if ( $EFFECTIVE_USER_ID == 0 && !exists $ENV{FAKEROOTKEY} && !$ischroot) { error "running chrootless mode as root without fakeroot might " . "damage the host system if not run inside a chroot"; } } } else { error "unknown mode: $options->{mode}"; } $options->{canmount} = 1; if ($options->{mode} eq 'root') { # It's possible to be root but not be able to mount anything. # This is for example the case when running under docker. # Mounting needs CAP_SYS_ADMIN which might not be available. # # We test for CAP_SYS_ADMIN using the capget syscall. # We cannot use cap_get_proc from sys/capability.h because Perl. # We don't use capsh because we don't want to depend on libcap2-bin my $hdrp = pack( "Li", # __u32 followed by int $_LINUX_CAPABILITY_VERSION_3, # available since Linux 2.6.26 0 # caps of this process ); my $datap = pack("LLLLLL", 0, 0, 0, 0, 0, 0); # six __u32 0 == syscall &SYS_capget, $hdrp, $datap or error "capget failed: $!"; my ($effective, undef) = unpack "LLLLLL", $datap; if ((($effective >> $CAP_SYS_ADMIN) & 1) != 1) { warning "cannot mount because CAP_SYS_ADMIN is not in the effective set"; $options->{canmount} = 0; } if (0 == syscall &SYS_prctl, $PR_CAPBSET_READ, $CAP_SYS_ADMIN) { warning "cannot mount because CAP_SYS_ADMIN is not in the bounding set"; $options->{canmount} = 0; } # To test whether we can use mount without actually trying to mount # something we try unsharing the mount namespace. If this is allowed, # then we are also allowed to mount. # # We need to call unshare with "--propagation unchanged" or otherwise # we get 'cannot change root filesystem propagation' when running # mmdebstrap inside a chroot for which the root of the chroot is not # its own mount point. if (0 != system 'unshare --mount --propagation unchanged -- true') { # if we cannot unshare the mount namespace as root, then we also # cannot mount warning "cannot mount because unshare --mount failed"; $options->{canmount} = 0; } } if (any { $_ eq $options->{mode} } ('root', 'unshare')) { if (!can_execute 'mount') { warning "cannot execute mount"; $options->{canmount} = 0; } } # we can only possibly mount in root and unshare mode if (none { $_ eq $options->{mode} } ('root', 'unshare')) { $options->{canmount} = 0; } my @architectures = (); foreach my $archs (@{ $options->{architectures} }) { foreach my $arch (split /[,\s]+/, $archs) { # strip leading and trailing whitespace $arch =~ s/^\s+|\s+$//g; # skip if the remainder is an empty string if ($arch eq '') { next; } # do not append component if it's already in the list if (any { $_ eq $arch } @architectures) { next; } push @architectures, $arch; } } $options->{nativearch} = $hostarch; $options->{foreignarchs} = []; if (scalar @architectures == 0) { warning "empty architecture list: falling back to native architecture" . " $hostarch"; } elsif (scalar @architectures == 1) { $options->{nativearch} = $architectures[0]; } else { $options->{nativearch} = $architectures[0]; push @{ $options->{foreignarchs} }, @architectures[1 .. $#architectures]; } debug "Native architecture (outside): $hostarch"; debug "Native architecture (inside): $options->{nativearch}"; debug("Foreign architectures (inside): " . (join ', ', @{ $options->{foreignarchs} })); { # FIXME: autogenerate this list my $deb2qemu = { alpha => 'alpha', amd64 => 'x86_64', arm => 'arm', arm64 => 'aarch64', armel => 'arm', armhf => 'arm', hppa => 'hppa', i386 => 'i386', loong64 => 'loongarch64', m68k => 'm68k', mips => 'mips', mips64 => 'mips64', mips64el => 'mips64el', mipsel => 'mipsel', powerpc => 'ppc', ppc64 => 'ppc64', ppc64el => 'ppc64le', riscv64 => 'riscv64', s390x => 's390x', sh4 => 'sh4', sparc => 'sparc', sparc64 => 'sparc64', }; if (any { $_ eq 'check/qemu' } @{ $options->{skip} }) { info "skipping check/qemu as requested"; } elsif ($options->{mode} eq "chrootless") { info "skipping emulation check in chrootless mode"; } elsif ($options->{variant} eq "extract") { info "skipping emulation check for extract variant"; } elsif ($hostarch ne $options->{nativearch}) { if (!can_execute 'arch-test') { error "install arch-test for foreign architecture support"; } my $withemu = 0; my $noemu = 0; { my $pid = open my $fh, '-|' // error "failed to fork(): $!"; if ($pid == 0) { { ## no critic (TestingAndDebugging::ProhibitNoWarnings) # don't print a warning if the following fails no warnings; exec 'arch-test', $options->{nativearch}; } # if exec didn't work (for example because the arch-test # program is missing) prepare for the worst and assume that # the architecture cannot be executed print "$options->{nativearch}: not supported on this" . " machine/kernel\n"; exit 1; } chomp( my $content = do { local $/; <$fh> } ); close $fh; if ($? == 0 and $content eq "$options->{nativearch}: ok") { $withemu = 1; } } { my $pid = open my $fh, '-|' // error "failed to fork(): $!"; if ($pid == 0) { { ## no critic (TestingAndDebugging::ProhibitNoWarnings) # don't print a warning if the following fails no warnings; exec 'arch-test', '-n', $options->{nativearch}; } # if exec didn't work (for example because the arch-test # program is missing) prepare for the worst and assume that # the architecture cannot be executed print "$options->{nativearch}: not supported on this" . " machine/kernel\n"; exit 1; } chomp( my $content = do { local $/; <$fh> } ); close $fh; if ($? == 0 and $content eq "$options->{nativearch}: ok") { $noemu = 1; } } # four different outcomes, depending on whether arch-test # succeeded with or without emulation # # withemu | noemu | # --------+-------+----------------- # 0 | 0 | test why emu doesn't work and quit # 0 | 1 | should never happen # 1 | 0 | use qemu emulation # 1 | 1 | don't use qemu emulation if ($withemu == 0 and $noemu == 0) { { open my $fh, '<', '/proc/filesystems' or error "failed to open /proc/filesystems: $!"; unless (grep { /^nodev\tbinfmt_misc$/ } (<$fh>)) { warning "binfmt_misc not found in /proc/filesystems --" . " is the module loaded?"; } close $fh; } { open my $fh, '<', '/proc/mounts' or error "failed to open /proc/mounts: $!"; unless ( grep { /^binfmt_misc\s+ \/proc\/sys\/fs\/binfmt_misc\s+ binfmt_misc\s+/x } (<$fh>) ) { warning "binfmt_misc not found in /proc/mounts -- not" . " mounted?"; } close $fh; } { if (!exists $deb2qemu->{ $options->{nativearch} }) { warning "no mapping from $options->{nativearch} to" . " qemu-user binary"; } elsif (!can_execute 'update-binfmts') { warning "cannot find update-binfmts"; } else { my $binfmt_identifier = 'qemu-' . $deb2qemu->{ $options->{nativearch} }; open my $fh, '-|', 'update-binfmts', '--display', $binfmt_identifier // error "failed to fork(): $!"; chomp( my $binfmts = do { local $/; <$fh> } ); close $fh; if ($? != 0 || $binfmts eq '') { warning "$binfmt_identifier is not a supported" . " binfmt name"; } } } error "$options->{nativearch} can neither be executed natively" . " nor via qemu user emulation with binfmt_misc"; } elsif ($withemu == 0 and $noemu == 1) { error "arch-test succeeded without emu but not with emu"; } elsif ($withemu == 1 and $noemu == 0) { info "$options->{nativearch} cannot be executed natively, but" . " transparently using qemu-user binfmt emulation"; if (!exists $deb2qemu->{ $options->{nativearch} }) { error "no mapping from $options->{nativearch} to qemu-user" . " binary"; } $options->{qemu} = $deb2qemu->{ $options->{nativearch} }; } elsif ($withemu == 1 and $noemu == 1) { info "$options->{nativearch} is different from $hostarch but" . " can be executed natively"; } else { error "logic error"; } } else { info "chroot architecture $options->{nativearch} is equal to the" . " host's architecture"; } } if (defined $options->{qemu} && $options->{mode} eq 'fakechroot') { if (!can_execute 'dpkg-architecture') { error "cannot find dpkg-architecture"; } } { $options->{suite} = undef; if (scalar @ARGV > 0) { $options->{suite} = shift @ARGV; if (scalar @ARGV > 0) { $options->{target} = shift @ARGV; } else { $options->{target} = '-'; } } else { info "No SUITE specified, expecting sources.list on standard input"; $options->{target} = '-'; } my $sourceslists = []; if (!defined $options->{suite}) { # If no suite was specified, then the whole sources.list has to # come from standard input info "reading sources.list from standard input..."; my $content = do { local $/; ## no critic (InputOutput::ProhibitExplicitStdin) ; }; if ($content eq "") { warning "sources.list from standard input is empty"; } else { my $type = guess_sources_format($content); if (!defined $type || ($type ne "deb822" and $type ne "one-line")) { error "cannot determine sources.list format"; } push @{$sourceslists}, { type => $type, fname => undef, content => $content, }; } } else { my @components = (); foreach my $comp (@{ $options->{components} }) { my @comps = split /[,\s]+/, $comp; foreach my $c (@comps) { # strip leading and trailing whitespace $c =~ s/^\s+|\s+$//g; # skip if the remainder is an empty string if ($c eq "") { next; } # do not append component if it's already in the list if (any { $_ eq $c } @components) { next; } push @components, $c; } } my $compstr = join " ", @components; # From the suite name we can maybe infer which key we need. If we # can infer this information, then we need to check whether the # currently running apt actually trusts this key or not. If it # doesn't, then we need to add a signed-by line to the sources.list # entry. my $signedby = ''; my %suite_by_vendor = get_suite_by_vendor(); my $gpgproc = sub { my $keyring = get_keyring_by_suite($options->{suite}, \%suite_by_vendor); if (!defined $keyring) { debug "get_keyring_by_suite() cannot find keyring"; return ''; } # we can only check if we need the signed-by entry if we u # automatically chosen keyring exists if (!defined $keyring || !-e $keyring) { debug "found keyring does not exist"; return ''; } # we can only check key material if gpg is installed my $gpghome = tempdir( "mmdebstrap.gpghome.XXXXXXXXXXXX", TMPDIR => 1, CLEANUP => 1 ); my @gpgcmd = ( 'gpg', '--quiet', '--ignore-time-conflict', '--no-options', '--no-default-keyring', '--homedir', $gpghome, '--no-auto-check-trustdb', ); my ($ret, $message); { my $fh; { # change warning handler to prevent message # Can't exec "gpg": No such file or directory local $SIG{__WARN__} = sub { $message = shift; }; $ret = open $fh, '-|', @gpgcmd, '--version'; } # we only want to check if the gpg command exists close $fh; } if ($? != 0 || !defined $ret || defined $message) { warning "gpg --version failed: cannot infer signed-by value"; return ''; } # initialize gpg trustdb with empty one { 0 == system(@gpgcmd, '--check-trustdb') or error "gpg failed to initialize trustdb:: $?"; } if (!-d $options->{apttrustedparts}) { warning "$options->{apttrustedparts} doesn't exist"; return ''; } # find all the fingerprints of the keys apt currently # knows about my @keyrings = (); opendir my $dh, $options->{apttrustedparts} or error "cannot read $options->{apttrustedparts}"; while (my $filename = readdir $dh) { if ($filename !~ /\.(asc|gpg)$/) { next; } $filename = "$options->{apttrustedparts}/$filename"; # skip empty keyrings -s "$filename" || next; push @keyrings, $filename; } closedir $dh; if (-s $options->{apttrusted}) { push @keyrings, $options->{apttrusted}; } my @aptfingerprints = (); if (scalar @keyrings == 0) { debug "no keyring is trusted by apt"; return " [signed-by=\"$keyring\"]"; } info "finding correct signed-by value..."; my $progress = 0.0; print_progress($progress); for (my $i = 0 ; $i < scalar @keyrings ; $i++) { my $k = $keyrings[$i]; open(my $fh, '-|', @gpgcmd, '--with-colons', '--show-keys', $k) // error "failed to fork(): $!"; while (my $line = <$fh>) { if ($line !~ /^fpr:::::::::([^:]+):/) { next; } push @aptfingerprints, $1; } close $fh; if ($? != 0) { warning("gpg failed to read $k"); } print_progress($i / (scalar @keyrings) * 100.0, undef); } print_progress("done"); if (scalar @aptfingerprints == 0) { debug "no fingerprints found"; return " [signed-by=\"$keyring\"]"; } # check if all fingerprints from the keyring that we guessed # are known by apt and only add signed-by option if that's not # the case my @suitefingerprints = (); { open(my $fh, '-|', @gpgcmd, '--with-colons', '--show-keys', $keyring) // error "failed to fork(): $!"; while (my $line = <$fh>) { if ($line !~ /^fpr:::::::::([^:]+):/) { next; } # if this fingerprint is not known by apt, then we need #to add the signed-by option if (none { $_ eq $1 } @aptfingerprints) { debug "fingerprint $1 is not trusted by apt"; return " [signed-by=\"$keyring\"]"; } } close $fh; if ($? != 0) { warning "gpg failed -- cannot infer signed-by value"; } } return ''; }; if (any { $_ eq 'check/signed-by' } @{ $options->{skip} }) { info "skipping check/signed-by as requested"; } else { $signedby = $gpgproc->(); } if (scalar @ARGV > 0) { for my $arg (@ARGV) { if ($arg eq '-') { info 'reading sources.list from standard input...'; my $content = do { local $/; ## no critic (InputOutput::ProhibitExplicitStdin) ; }; if ($content eq "") { warning "sources.list from standard input is empty"; } else { my $type = guess_sources_format($content); if (!defined $type || ($type ne 'deb822' and $type ne 'one-line')) { error "cannot determine sources.list format"; } # if last entry is of same type and without filename, # then append if ( scalar @{$sourceslists} > 0 && $sourceslists->[-1]{type} eq $type && !defined $sourceslists->[-1]{fname}) { $sourceslists->[-1]{content} .= ($type eq 'one-line' ? "\n" : "\n\n") . $content; } else { push @{$sourceslists}, { type => $type, fname => undef, content => $content, }; } } } elsif ($arg =~ /^deb(-src)? /) { my $content = "$arg\n"; # if last entry is of same type and without filename, # then append if ( scalar @{$sourceslists} > 0 && $sourceslists->[-1]{type} eq 'one-line' && !defined $sourceslists->[-1]{fname}) { $sourceslists->[-1]{content} .= "\n" . $content; } else { push @{$sourceslists}, { type => 'one-line', fname => undef, content => $content, }; } } elsif ($arg =~ /:\/\//) { if (!$options->{suite}) { error( "cannot create sources.list entry for URI" . "with empty suite"); } my $content = join ' ', ( "deb$signedby", $arg, $options->{suite}, "$compstr\n" ); # if last entry is of same type and without filename, # then append if ( scalar @{$sourceslists} > 0 && $sourceslists->[-1]{type} eq 'one-line' && !defined $sourceslists->[-1]{fname}) { $sourceslists->[-1]{content} .= "\n" . $content; } else { push @{$sourceslists}, { type => 'one-line', fname => undef, content => $content, }; } } elsif (-f $arg) { my $content = ''; open my $fh, '<', $arg or error "cannot open $arg: $!"; while (my $line = <$fh>) { $content .= $line; } close $fh; if ($content eq "") { warning "$arg is empty"; } else { my $type = undef; if ($arg =~ /\.list$/) { $type = 'one-line'; } elsif ($arg =~ /\.sources$/) { $type = 'deb822'; } else { $type = guess_sources_format($content); } if (!defined $type || ($type ne 'deb822' and $type ne 'one-line')) { error "cannot determine sources.list format"; } push @{$sourceslists}, { type => $type, fname => basename($arg), content => $content, }; } } elsif ($arg eq '') { # empty } else { error "invalid mirror: $arg"; } } } elsif (!length($options->{suite})) { warning "cannot guess apt sources.list entry with empty suite name"; } else { my $sourceslist = get_sourceslist_by_suite($options->{suite}, $options->{nativearch}, $signedby, $compstr, \%suite_by_vendor); push @{$sourceslists}, { type => 'one-line', fname => undef, content => $sourceslist, }; } } if (scalar @{$sourceslists} == 0) { warning "empty apt sources.list"; } debug("sources list entries:"); for my $list (@{$sourceslists}) { if (defined $list->{fname}) { debug("fname: $list->{fname}"); } debug("type: $list->{type}"); debug("content:"); for my $line (split "\n", $list->{content}) { debug(" $line"); } } $options->{sourceslists} = $sourceslists; } if ($options->{target} eq '-') { if (POSIX::isatty STDOUT) { error "stdout is an interactive tty"; } } else { if (!-e dirname($options->{target})) { error "path leading to $options->{target} does not exist"; } if (!-d dirname($options->{target})) { error "parent of $options->{target} is not a directory"; } my $abs_path = abs_path($options->{target}); if (!defined $abs_path) { error "unable to get absolute path of target directory" . " $options->{target}"; } $options->{target} = $abs_path; } if ($options->{target} eq '/') { error "refusing to use the filesystem root as output directory"; } my $tar_compressor = get_tar_compressor($options->{target}); # figure out the right format if ($options->{format} eq 'auto') { # (stat(...))[6] is the device identifier which contains the major and # minor numbers for character special files # major 1 and minor 3 is /dev/null on Linux if ( $options->{target} eq '/dev/null' and $OSNAME eq 'linux' and -c '/dev/null' and major((stat("/dev/null"))[6]) == 1 and minor((stat("/dev/null"))[6]) == 3) { $options->{format} = 'null'; } elsif ($options->{target} eq '-' and $OSNAME eq 'linux' and major((stat(STDOUT))[6]) == 1 and minor((stat(STDOUT))[6]) == 3) { # by checking the major and minor number of the STDOUT fd we also # can detect redirections to /dev/null and choose the null format # accordingly $options->{format} = 'null'; } elsif ($options->{target} ne '-' and -d $options->{target}) { $options->{format} = 'directory'; } elsif ( defined $tar_compressor or $options->{target} =~ /\.tar$/ or $options->{target} eq '-' or -p $options->{target} # named pipe (fifo) or -c $options->{target} # character special like /dev/null ) { $options->{format} = 'tar'; # check if the compressor is installed if (defined $tar_compressor) { my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { open(STDOUT, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; open(STDIN, '<', '/dev/null') or error "cannot open /dev/null for reading: $!"; exec { $tar_compressor->[0] } @{$tar_compressor} or error("cannot exec " . (join " ", @{$tar_compressor}) . ": $!"); } waitpid $pid, 0; if ($? != 0) { error("failed to start " . (join " ", @{$tar_compressor})); } } } elsif ($options->{target} =~ /\.(squashfs|sqfs)$/) { $options->{format} = 'squashfs'; # check if tar2sqfs is installed my $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { open(STDOUT, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; open(STDIN, '<', '/dev/null') or error "cannot open /dev/null for reading: $!"; exec('tar2sqfs', '--version') or error("cannot exec tar2sqfs --version: $!"); } waitpid $pid, 0; if ($? != 0) { error("failed to start tar2sqfs --version"); } } elsif ($options->{target} =~ /\.ext2$/) { $options->{format} = 'ext2'; # check if the installed version of genext2fs supports tarballs on # stdin (undef, my $filename) = tempfile( "mmdebstrap.ext2.XXXXXXXXXXXX", OPEN => 0, TMPDIR => 1 ); open my $fh, '|-', 'genext2fs', '-B', '1024', '-b', '8', '-N', '11', '-a', '-', $filename // error "failed to fork(): $!"; # write 10240 null-bytes to genext2fs -- this represents an empty # tar archive print $fh ("\0" x 10240) or error "cannot write to genext2fs process"; close $fh; my $exitstatus = $?; unlink $filename // die "cannot unlink $filename"; if ($exitstatus != 0) { error "genext2fs failed with exit status: $exitstatus"; } } elsif ($options->{target} =~ /\.ext4$/) { $options->{format} = 'ext4'; # check if the installed version of e2fsprogs supports tarballs on # stdin (undef, my $filename) = tempfile( "mmdebstrap.ext4.XXXXXXXXXXXX", OPEN => 0, TMPDIR => 1 ); # creating file to suppress message "Creating regular file ..." { open my $fh, '>', $filename; } open my $fh, '|-', 'mke2fs', '-q', '-F', '-o', 'Linux', '-T', 'ext4', '-b', '4096', '-d', '-', $filename, '16384' // error "failed to fork(): $!"; # write 10240 null-bytes to mke2fs -- this represents an empty # tar archive print $fh ("\0" x 10240) or error "cannot write to mke2fs process"; close $fh; my $exitstatus = $?; unlink $filename // die "cannot unlink $filename"; if ($exitstatus != 0) { error "mke2fs failed with exit status: $exitstatus"; } } else { $options->{format} = 'directory'; } info "automatically chosen format: $options->{format}"; } if ( $options->{target} eq '-' and $options->{format} ne 'tar' and $options->{format} ne 'null') { error "the $options->{format} format is unable to write to stdout"; } if ($options->{format} eq 'null' and none { $_ eq $options->{target} } ('-', '/dev/null')) { info "ignoring target $options->{target} with null format"; } my $blocksize = -1; if ($options->{format} eq 'ext2') { if (!can_execute 'genext2fs') { error "need genext2fs for ext2 format"; } $blocksize = 1024; } elsif ($options->{format} eq 'ext4') { if (!can_execute 'mke2fs', '-V') { error "need mke2fs for ext4 format"; } require DynaLoader; my $libarchive = DynaLoader::dl_load_file("libarchive.so.13", 0) or error "need libarchive for ext4 format"; $blocksize = 4096; } elsif ($options->{format} eq 'squashfs') { if (!can_execute 'tar2sqfs') { error "need tar2sqfs binary from the squashfs-tools-ng package"; } $blocksize = 1048576; } my $rootdir_handle; if (any { $_ eq $options->{format} } ('tar', 'squashfs', 'ext2', 'ext4', 'null')) { if ($options->{format} ne 'null') { #<<< perltidy 20220613 formatting if (any { $_ eq $options->{variant} } ('extract', 'custom') and $options->{mode} eq 'fakechroot') { info "creating a tarball, squashfs, ext2 or ext4 image in" . " fakechroot mode might fail in extract and" . " custom variants because there might be no tar inside the" . " chroot"; } #>>> # try to fail early if target tarball or squashfs image cannot be # opened for writing if ($options->{target} ne '-') { if ($options->{dryrun}) { if (-e $options->{target}) { info "not overwriting $options->{target} because in" . " dry-run mode"; } } else { open my $fh, '>', $options->{target} or error "cannot open $options->{target} for writing: $!"; close $fh; } } } # since the output is a tarball, we create the rootfs in a temporary # directory $options->{root} = tempdir('mmdebstrap.XXXXXXXXXX', TMPDIR => 1); info "using $options->{root} as tempdir"; # add an flock on the temporary directory to prevent cleanup by systemd # see section Age in tmpfiles.d(5) sysopen($rootdir_handle, $options->{root}, O_RDONLY | O_DIRECTORY) or error "Failed to sysopen $options->{root}: $!\n"; flock($rootdir_handle, LOCK_EX) or error "Unable to flock $options->{root}: $!\n"; # in unshare and root mode, other users than the current user need to # access the rootfs, most prominently, the _apt user. Thus, make the # temporary directory world readable. #<<< perltidy 20220613 formatting if ( any { $_ eq $options->{mode} } ('unshare', 'root') or ($EFFECTIVE_USER_ID == 0 and $options->{mode} eq 'chrootless') ) { chmod 0755, $options->{root} or error "cannot chmod root: $!"; } #>>> } elsif ($options->{format} eq 'directory') { # user does not seem to have specified a tarball as output, thus work # directly in the supplied directory $options->{root} = $options->{target}; if (-e $options->{root}) { if (!-d $options->{root}) { error "$options->{root} exists and is not a directory"; } if (any { $_ eq 'check/empty' } @{ $options->{skip} }) { info "skipping check/empty as requested"; } else { # check if the directory is empty or contains nothing more than # an empty lost+found directory. The latter exists on freshly # created ext3 and ext4 partitions. # rationale for requiring an empty directory: # https://bugs.debian.org/833525 opendir(my $dh, $options->{root}) or error "Can't opendir($options->{root}): $!"; while (my $entry = readdir $dh) { # skip the "." and ".." entries next if $entry eq "."; next if $entry eq ".."; # if the entry is a directory named "lost+found" then skip # it, if it's empty if ($entry eq "lost+found" and -d "$options->{root}/$entry") { opendir(my $dh2, "$options->{root}/$entry"); # Attempt reading the directory thrice. If the third # time succeeds, then it has more entries than just "." # and ".." and must thus not be empty. readdir $dh2; readdir $dh2; # rationale for requiring an empty directory: # https://bugs.debian.org/833525 if (readdir $dh2) { error "$options->{root} contains a non-empty" . " lost+found directory"; } closedir($dh2); } else { error "$options->{root} is not empty"; } } closedir($dh); } } else { my $num_created = make_path "$options->{root}", { error => \my $err }; if ($err && @$err) { error(join "; ", (map { "cannot create " . (join ": ", %{$_}) } @$err)); } elsif ($num_created == 0) { error "cannot create $options->{root}"; } } } else { error "unknown format: $options->{format}"; } # check for double quotes because apt doesn't allow to escape them and # thus paths with double quotes are invalid in the apt config if ($options->{root} =~ /"/) { error "apt cannot handle paths with double quotes"; } my @idmap; # for unshare mode the rootfs directory has to have appropriate # permissions if ($EFFECTIVE_USER_ID != 0 and $options->{mode} eq 'unshare') { @idmap = read_subuid_subgid 1; # sanity check if ( scalar(@idmap) != 2 || $idmap[0][0] ne 'u' || $idmap[1][0] ne 'g' || !length $idmap[0][2] || !length $idmap[1][2]) { error "invalid idmap"; } my $outer_gid = $REAL_GROUP_ID + 0; my $pid = get_unshare_cmd( sub { chown 1, 1, $options->{root} }, [ ['u', '0', $REAL_USER_ID, '1'], ['g', '0', $outer_gid, '1'], ['u', '1', $idmap[0][2], '1'], ['g', '1', $idmap[1][2], '1']]); waitpid $pid, 0; $? == 0 or error "chown failed"; } # check if .deb files given by --include are readable by the unshared user if ($options->{mode} eq 'unshare' and scalar(grep { /^\// } @{ $options->{include} }) > 0) { my $pid = get_unshare_cmd( sub { my $ret = 0; foreach my $f (grep { /^\// } @{ $options->{include} }) { # open the file for real because -r will report the file as # readable even though open will fail (in contrast to the # coreutils test utility, perl doesn't use faccessat) my $res = open(my $fh, '<', $f); if (!$res) { warning "unshared user cannot access $f for reading"; $ret = 1; } else { close $fh; } } exit $ret; }, \@idmap ); waitpid $pid, 0; if ($? != 0) { warning("apt on the outside is run as the unshared user and " . "needs read access to packages outside the chroot given " . "via --include"); } } # figure out whether we have mknod $options->{havemknod} = 0; if ($options->{mode} eq 'unshare') { my $pid = get_unshare_cmd( sub { $options->{havemknod} = havemknod($options->{root}); }, \@idmap ); waitpid $pid, 0; $? == 0 or error "havemknod failed"; } elsif (any { $_ eq $options->{mode} } ('root', 'fakechroot', 'chrootless')) { $options->{havemknod} = havemknod($options->{root}); } else { error "unknown mode: $options->{mode}"; } # If a tarball is to be created, we always (except if --skip=output/dev is # passed) craft the /dev entries ourselves. # Why do we put /dev entries in the final tarball? # - because debootstrap does it # - because schroot (#856877) and pbuilder rely on it and we care about # Debian buildds (using schroot) and reproducible builds infra (using # pbuilder) # If both the above assertion change, we can stop creating /dev entries as # well. my $devtar = ''; if (any { $_ eq $options->{format} } ('tar', 'squashfs', 'ext2', 'ext4')) { my @entries = (); my @paxentries; if ($options->{nativearch} eq "hurd-i386") { @paxentries = @hurdfiles; } else { @paxentries = @linuxdevfiles; } foreach my $paxentry (@paxentries) { my ($name, $mode, $type, $linktarget, $major, $minor, $content) = @{$paxentry}; if (defined $content) { $content = "SCHILY.xattr.gnu.translator=$content"; # In the beginning the decimal length of the field is recorded. # But the length includes the length of the number itself. # Luckily we only need to support decimal numbers with two digits. if ((length $content) + 4 > 99 || (length $content) + 4 < 10) { exit 1; # not supported yet } my $len = (length $content) + 4; my $dirname = dirname $name; my $basename = basename $name; push @entries, [ "$dirname/PaxHeaders/$basename", 0, 'x', undef, undef, undef, "$len $content\n" ]; } push @entries, [$name, $mode, $type, $linktarget, $major, $minor, undef]; } foreach my $file (@entries) { my ($fname, $mode, $type, $linktarget, $devmajor, $devminor, $content) = @{$file}; if (length "$fname" > 100) { error "tar entry cannot exceed 100 characters"; } if ($type eq '3' and any { $_ eq 'output/mknod' } @{ $options->{skip} }) { info "skipping output/mknod as requested for $fname"; next; } my $size = defined $content ? length $content : 0; my $etime = $type eq 'x' ? 0 : $mtime; my $entry = pack( # name mode uid gid size mtime type linktarget 'a100 a8 a8 a8 a12 a12 A8 a1 a100 ' # magic version username groupname major minor prefix . 'a6 a2 a32 a32 a8 a8 a155 x12', $fname, sprintf('%07o', $mode), sprintf('%07o', 0), # uid sprintf('%07o', 0), # gid sprintf('%011o', $size), # size sprintf('%011o', $etime), '', # checksum $type, # type $linktarget // '', # linktarget "ustar", # magic "00", # version '', # username '', # groupname defined($devmajor) ? sprintf('%07o', $devmajor) : '', defined($devminor) ? sprintf('%07o', $devminor) : '', '', # prefix ); # compute and insert checksum substr($entry, 148, 7) = sprintf("%06o\0", unpack("%16C*", $entry)); $devtar .= $entry; if (length $content) { $devtar .= (pack 'a512', $content); } } } elsif (any { $_ eq $options->{format} } ('directory', 'null')) { # nothing to do } else { error "unknown format: $options->{format}"; } my $exitstatus = 0; my @taropts = ( '--sort=name', "--mtime=\@$mtime", '--clamp-mtime', '--numeric-owner', '--one-file-system', '--format=pax', # If POSIXLY_CORRECT were set in the environment, the ustar header # name would include the process id. This forces tar to always use # %d/PaxHeaders/%f even if the POSIXLY_CORRECT environment variable is # set. atime and ctime are deleted for improved reproducibility. '--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime', '-c', '--exclude=./lost+found' ); # only exclude ./dev if device nodes are written out (the default) if (none { $_ eq 'output/dev' } @{ $options->{skip} }) { push @taropts, '--exclude=./dev', '--exclude=./servers', '--exclude=./servers/*'; } # tar2sqfs and genext2fs do not support extended attributes if ($options->{format} eq "squashfs") { # tar2sqfs supports user.*, trusted.* and security.* but not system.* # https://bugs.debian.org/988100 # lib/sqfs/xattr/xattr.c of https://github.com/AgentD/squashfs-tools-ng # https://github.com/AgentD/squashfs-tools-ng/issues/83 # https://github.com/AgentD/squashfs-tools-ng/issues/25 warning("tar2sqfs does not support extended attributes" . " from the 'system' namespace"); push @taropts, '--xattrs', '--xattrs-exclude=system.*'; } elsif ($options->{format} eq "ext2") { warning "genext2fs does not support extended attributes"; warning "ext2 does not support sub-second precision timestamps"; warning "ext2 does not support timestamps beyond 2038 January 18"; warning "ext2 inode size of 128 prevents removing these limitations"; } else { push @taropts, '--xattrs'; } # disable signals so that we can fork and change behaviour of the signal # handler in the parent and child without getting interrupted my $sigset = POSIX::SigSet->new(SIGINT, SIGHUP, SIGPIPE, SIGTERM); POSIX::sigprocmask(SIG_BLOCK, $sigset) or error "Can't block signals: $!"; # a pipe to transfer the final tarball from the child to the parent pipe my $rfh, my $wfh; # instead of two pipe calls, creating four file handles, we use socketpair socketpair my $childsock, my $parentsock, AF_UNIX, SOCK_STREAM, PF_UNSPEC or error "socketpair failed: $!"; $options->{hooksock} = $childsock; # for communicating the required number of blocks, we don't need # bidirectional communication, so a pipe() is enough # we don't communicate this via the hook communication because # a) this would abuse the functionality exclusively for hooks # b) it puts code writing the protocol outside of the helper/listener # c) the forked listener process cannot communicate to its parent pipe my $nblkreader, my $nblkwriter or error "pipe failed: $!"; my $worker = sub { # child local $SIG{'INT'} = 'DEFAULT'; local $SIG{'HUP'} = 'DEFAULT'; local $SIG{'PIPE'} = 'DEFAULT'; local $SIG{'TERM'} = 'DEFAULT'; # unblock all delayed signals (and possibly handle them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; close $rfh; close $parentsock; open(STDOUT, '>&', STDERR) or error "cannot open STDOUT: $!"; setup($options); print $childsock (pack('n', 0) . 'adios'); $childsock->flush(); close $childsock; close $nblkreader; if (!$options->{dryrun} && any { $_ eq $options->{format} } ('ext2', 'ext4')) { my $numblocks = approx_disk_usage($options->{root}, $blocksize); print $nblkwriter "$numblocks\n"; $nblkwriter->flush(); } close $nblkwriter; if ($options->{dryrun}) { info "simulate creating tarball..."; } elsif (any { $_ eq $options->{format} } ('tar', 'squashfs', 'ext2', 'ext4')) { info "creating tarball..."; # redirect tar output to the writing end of the pipe so # that the parent process can capture the output open(STDOUT, '>&', $wfh) or error "cannot open STDOUT: $!"; # Add ./dev as the first entries of the tar file. # We cannot add them after calling tar, because there is no # way to prevent tar from writing NULL entries at the end. if (any { $_ eq 'output/dev' } @{ $options->{skip} }) { info "skipping output/dev as requested"; } else { print $devtar; } if ($options->{mode} eq 'unshare') { # pack everything except ./dev 0 == system('tar', @taropts, '-C', $options->{root}, '.') or error "tar failed: $?"; } elsif ($options->{mode} eq 'fakechroot') { # By default, FAKECHROOT_EXCLUDE_PATH includes /proc and /sys # which means that the resulting tarball will contain the # permission and ownership information of /proc and /sys from # the outside, which we want to avoid. ## no critic (Variables::RequireLocalizedPunctuationVars) $ENV{FAKECHROOT_EXCLUDE_PATH} = "/dev"; # Fakechroot requires tar to run inside the chroot or otherwise # absolute symlinks will include the path to the root directory 0 == system('chroot', $options->{root}, 'tar', @taropts, '-C', '/', '.') or error "tar failed: $?"; } elsif (any { $_ eq $options->{mode} } ('root', 'chrootless')) { # If the chroot directory is not owned by the root user, then # we assume that no measure was taken to fake root permissions. # Since the final tarball should contain entries with root # ownership, we instruct tar to do so. my @owneropts = (); if ((stat $options->{root})[4] != 0) { push @owneropts, '--owner=0', '--group=0', '--numeric-owner'; } 0 == system('tar', @taropts, @owneropts, '-C', $options->{root}, '.') or error "tar failed: $?"; } else { error "unknown mode: $options->{mode}"; } info "done"; } elsif (any { $_ eq $options->{format} } ('directory', 'null')) { # nothing to do } else { error "unknown format: $options->{format}"; } exit 0; }; my $pid; if ($options->{mode} eq 'unshare') { $pid = get_unshare_cmd($worker, \@idmap); } elsif (any { $_ eq $options->{mode} } ('root', 'fakechroot', 'chrootless')) { $pid = fork() // error "fork() failed: $!"; if ($pid == 0) { $worker->(); } } else { error "unknown mode: $options->{mode}"; } # parent my $got_signal = 0; my $waiting_for = "setup"; my $ignore = sub { $got_signal = shift; info "main() received signal $got_signal: waiting for $waiting_for..."; }; local $SIG{'INT'} = $ignore; local $SIG{'HUP'} = $ignore; local $SIG{'PIPE'} = $ignore; local $SIG{'TERM'} = $ignore; # unblock all delayed signals (and possibly handle them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; close $wfh; close $childsock; debug "starting to listen for hooks"; # handle special hook commands via parentsock my $lpid = fork() // error "fork() failed: $!"; if ($lpid == 0) { # whatever the script writes on stdout is sent to the # socket # whatever is written to the socket, send to stdin open(STDOUT, '>&', $parentsock) or error "cannot open STDOUT: $!"; open(STDIN, '<&', $parentsock) or error "cannot open STDIN: $!"; hooklistener($verbosity_level); exit 0; } waitpid($lpid, 0); if ($? != 0) { # we cannot die here because that would leave the other thread # running without a parent warning "listening on child socket failed: $@"; $exitstatus = 1; } debug "finish to listen for hooks"; close $parentsock; my $numblocks = 0; close $nblkwriter; if (!$options->{dryrun} && any { $_ eq $options->{format} } ('ext2', 'ext4')) { $numblocks = <$nblkreader>; if (defined $numblocks) { chomp $numblocks; } else { # This can happen if the setup process died early and thus closes # the pipe from the other and. The EOF is turned into undef. # we cannot die here because that would skip the cleanup task warning "failed to read required number of blocks"; $exitstatus = 1; $numblocks = -1; } } close $nblkreader; if ($options->{dryrun}) { # nothing to do } elsif (any { $_ eq $options->{format} } ('directory', 'null')) { # nothing to do } elsif ((any { $_ eq $options->{format} } ('ext2', 'ext4')) && $numblocks <= 0) { # nothing to do because of invalid $numblocks } elsif (any { $_ eq $options->{format} } ('tar', 'squashfs', 'ext2', 'ext4')) { # we use eval() so that error() doesn't take this process down and # thus leaves the setup() process without a parent eval { if ($options->{target} eq '-') { if (!copy($rfh, *STDOUT)) { error "cannot copy to standard output: $!"; } } else { #<<< perltidy 20220613 formatting if (any { $_ eq $options->{format} } ('squashfs', 'ext2', 'ext4') or defined $tar_compressor) { #>>> my @argv = (); if ($options->{format} eq 'squashfs') { push @argv, 'tar2sqfs', '--quiet', '--no-skip', '--force', '--exportable', '--compressor', 'xz', '--block-size', $blocksize, $options->{target}; } elsif ($options->{format} eq 'ext2') { if ($numblocks <= 0) { error "invalid number of blocks: $numblocks"; } push @argv, 'genext2fs', '-B', 1024, '-b', $numblocks, '-i', '16384', '-a', '-', $options->{target}; } elsif ($options->{format} eq 'ext4') { if ($numblocks <= 0) { error "invalid number of blocks: $numblocks"; } push @argv, 'mke2fs', '-q', '-F', '-o', 'Linux', '-T', 'ext4'; if (exists $ENV{SOURCE_DATE_EPOCH}) { # if SOURCE_DATE_EPOCH was set, make the image # reproducible by setting a fixed uuid and # hash_seed my $uuid = create_v5_uuid( create_v5_uuid( $UUID_NS_DNS, "mister-muffin.de" ), $mtime ); push @argv, '-U', $uuid, '-E', "hash_seed=$uuid"; } push @argv, '-b', $blocksize, '-d', '-', $options->{target}, $numblocks; } elsif ($options->{format} eq 'tar') { push @argv, @{$tar_compressor}; } else { error "unknown format: $options->{format}"; } POSIX::sigprocmask(SIG_BLOCK, $sigset) or error "Can't block signals: $!"; my $cpid = fork() // error "fork() failed: $!"; if ($cpid == 0) { # child: default signal handlers local $SIG{'INT'} = 'DEFAULT'; local $SIG{'HUP'} = 'DEFAULT'; local $SIG{'PIPE'} = 'DEFAULT'; local $SIG{'TERM'} = 'DEFAULT'; # unblock all delayed signals (and possibly handle # them) POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; # redirect stdout to file or /dev/null if (any { $_ eq $options->{format} } ('squashfs', 'ext2', 'ext4')) { open(STDOUT, '>', '/dev/null') or error "cannot open /dev/null for writing: $!"; } elsif ($options->{format} eq 'tar') { open(STDOUT, '>', $options->{target}) or error "cannot open $options->{target} for writing: $!"; } else { error "unknown format: $options->{format}"; } open(STDIN, '<&', $rfh) or error "cannot open file handle for reading: $!"; eval { Devel::Cover::set_coverage("none") } if $is_covering; exec { $argv[0] } @argv or error("cannot exec " . (join " ", @argv) . ": $!"); } POSIX::sigprocmask(SIG_UNBLOCK, $sigset) or error "Can't unblock signals: $!"; waitpid $cpid, 0; if ($? != 0) { error("failed to run " . (join " ", @argv)); } } else { # somehow, when running under qemu, writing to a virtio # device will not result in a ENOSPC but just stall forever if (!copy($rfh, $options->{target})) { error "cannot copy to $options->{target}: $!"; } } } }; if ($@) { # we cannot die here because that would leave the other thread # running without a parent # We send SIGHUP to all our processes (including eventually # running tar and this process itself) to reliably tear down # all running child processes. The main process is not affected # because we are ignoring SIGHUP. # # FIXME: this codepath becomes dangerous in case mmdebstrap is not # run in its own process group. When run from the terminal, the # shell creates a new process group as part of its job control, so # sending SIGHUP to all processes in our own process group should # not be dangerous. But for example, on debci, lxc will run in the # same process group as mmdebstrap and sending SIGHUP to the whole # process group will also kill lxc. Creating a new process group # for $pid will break things because only the foreground job is # allowed to read from the terminal. If a background job does it, # i will be suspended with SIGTTIN. Even though apt could be told # to not read from the terminal by opening STDIN from /dev/null, # this would make --chrooted-customize-hook=bash impossible. # Making the $pid process group the foreground job will destroy all # the signal handling we have set up for when the user presses # ctrl+c in a terminal. Even if we fix the signal handling we now # find ourselves in the opposite situation: the $pid process must # now clean up the former main process tree reliably. And we cannot # create a new process group for everything all-in-one because that # would also destroy CTRL+C handling from the terminal. warning "creating tarball failed: $@"; my $pgroup = getpgrp(); warning "sending SIGHUP to all processes in process group $pgroup"; kill HUP => -$pgroup; $exitstatus = 1; } } else { error "unknown format: $options->{format}"; } close($rfh); waitpid $pid, 0; if ($? != 0) { $exitstatus = 1; } # change signal handler message $waiting_for = "cleanup"; if (any { $_ eq $options->{format} } ('directory')) { # nothing to do } elsif (any { $_ eq $options->{format} } ('tar', 'squashfs', 'ext2', 'ext4', 'null')) { if (!-e $options->{root}) { error "$options->{root} does not exist"; } info "removing tempdir $options->{root}..."; if ($options->{mode} eq 'unshare') { # We don't have permissions to remove the directory outside # the unshared namespace, so we remove it here. # Since this is still inside the unshared namespace, there is # no risk of removing anything important. $pid = get_unshare_cmd( sub { # change CWD to chroot directory because find tries to # chdir to the current directory which might not be # accessible by the unshared user: # find: Failed to restore initial working directory 0 == system('env', "--chdir=$options->{root}", 'find', $options->{root}, '-mount', '-mindepth', '1', '-delete') or error "rm failed: $?"; # ignore failure in case the unshared user doesn't have the # required permissions -- we attempt again later if # necessary rmdir "$options->{root}"; }, \@idmap ); waitpid $pid, 0; $? == 0 or error "remove_tree failed"; # in unshare mode, the toplevel directory might've been created in # a directory that the unshared user cannot change and thus cannot # delete. We attempt its removal again outside as the normal user. if (-e $options->{root}) { rmdir "$options->{root}" or error "cannot rmdir $options->{root}: $!"; } } elsif (any { $_ eq $options->{mode} } ('root', 'fakechroot', 'chrootless')) { # without unshare, we use the system's rm to recursively remove the # temporary directory just to make sure that we do not accidentally # remove more than we should by using --one-file-system. 0 == system('rm', '--interactive=never', '--recursive', '--preserve-root', '--one-file-system', $options->{root}) or error "rm failed: $?"; } else { error "unknown mode: $options->{mode}"; } } else { error "unknown format: $options->{format}"; } if ($got_signal) { $exitstatus = 1; } if ($exitstatus == 0) { my $duration = Time::HiRes::time - $before; info "success in " . (sprintf "%.04f", $duration) . " seconds"; exit 0; } error "mmdebstrap failed to run"; return 1; } main(); __END__ =head1 NAME mmdebstrap - multi-mirror Debian chroot creation =head1 SYNOPSIS B [B] [I [I [I...]]] =head1 DESCRIPTION B creates a Debian chroot of I into I from one or more Is. It is meant as an alternative to the debootstrap tool (see section B). In contrast to debootstrap it uses apt to resolve dependencies and is thus able to use more than one mirror and resolve more complex dependency relationships. See section B for an overview of how B works internally. The I option may either be a valid release code name (eg, sid, bookworm, trixie) or a symbolic name (eg, unstable, testing, stable, oldstable). Any suite name that works with apt on the given mirror will work. The I option is optional if no I and no I option is provided. If I is missing, then the information of the desired suite has to come from standard input as part of a valid apt sources.list file or be set up via hooks. The value of the I argument will be used to determine which apt index to use for finding out the set of C packages and/or the set of packages with the right priority for the selected variant. This functionality can be disabled by choosing the empty string for I. See the section B for more information. The I option may either be the path to a directory, the path to a tarball filename, the path to a squashfs image, the path to an ext2 or ext4 image, a FIFO, a character special device, or C<->. The I option is optional if no I option is provided. If I is missing or if I is C<->, an uncompressed tarball will be sent to standard output. Without the B<--format> option, I will be used to choose the format. See the section B for more information. The I option may either be provided as a URI, in apt one-line format, as a path to a file in apt's one-line or deb822-format, or C<->. If no I option is provided, then L is used as the default. If I does not refer to "unstable" or "testing", then I-updates and I-security mirrors are automatically added. If a I option starts with "deb " or "deb-src " then it is used as a one-line format entry for apt's sources.list inside the chroot. If a I option contains a "://" then it is interpreted as a mirror URI and the apt line inside the chroot is assembled as "deb [arch=A] B C D" where A is the host's native architecture, B is the I, C is the given I and D is the components given via B<--components> (defaults to "main"). If a I option happens to be an existing file, then its contents are written into the chroot's sources.list (if the first I is a file in one-line format) or into the chroot's sources.list.d directory, named with the extension .list or .sources, depending on whether the file is in one-line or deb822 format, respectively. If I is C<-> then standard input is pasted into the chroot's sources.list. More than one mirror can be specified and are appended to the chroot's sources.list in the given order. If you specify a https or tor I and you want the chroot to be able to update itself, don't forget to also install the ca-certificates package, the apt-transport-https package for apt versions less than 1.5 and/or the apt-transport-tor package using the B<--include> option, as necessary. All status output is printed to standard error unless B<--logfile> is used to redirect it to a file or B<--quiet> or B<--silent> is used to suppress any output on standard error. Help and version information will be printed to standard error with the B<--help> and B<--version> options, respectively. Otherwise, an uncompressed tarball might be sent to standard output if I is C<-> or if no I was specified. =head1 OPTIONS Options are case insensitive. Short options may be bundled. Long options require a double dash and may be abbreviated to uniqueness. Options can be placed anywhere on the command line, even before or mixed with the I, I, and I arguments. A double dash C<--> can be used to stop interpreting command line arguments as options to allow I, I and I arguments that start with a single or double dash. Option order only matters for options that can be passed multiple times as documented below. =over 8 =item B<-h,--help> Print synopsis and options of this man page and exit. =item B<--man> Show the full man page as generated from Perl POD in a pager. This requires the perldoc program from the perl-doc package. This is the same as running: pod2man /usr/bin/mmdebstrap | man -l - =item B<--version> Print the B version and exit. =item B<--variant>=I Choose which package set to install. Valid variant Is are B, B, B, B, B, B, B, B, B, B<->, and B. The default variant is B. See the section B for more information. =item B<--mode>=I Choose how to perform the chroot operation and create a filesystem with ownership information different from the current user. Valid mode Is are B, B, B, B, B, B and B. The default mode is B. See the section B for more information. =item B<--format>=I Choose the output format. Valid format Is are B, B, B, B, B, B and B. The default format is B. See the section B for more information. =item B<--aptopt>=I