piuparts-1.1.1/0000755000000000000000000000000013607334164010256 5ustar piuparts-1.1.1/CONTRIBUTING0000644000000000000000000000254513363064533012115 0ustar Contributing code to this project --------------------------------- It's helpful to track fixes or new features via wishlist bugs against the 'piuparts' package, eg with the 'reportbug' tool ('devscripts' package). The BTS will ensure the developers' mailing list piuparts-devel@alioth-lists.debian.net is notified. Patches can be submitted by mail (git format-patch, see below) or as requests to pull from a publicly-visible git repository. In either case, please make a topic branch based on the 'develop' branch. You can send patches or requests to the development list, or to the tracking bug: @bugs.debian.org. One possible workflow: git clone https://salsa.debian.org/debian/piuparts.git git checkout origin/develop -b git commit -a git format-patch -M origin/develop reportbug piuparts Does and don'ts --------------- Please do *not* push to the develop or master branch directly, unless explicitly asked for! Please always *do* include a debian/changelog entry with your patches. Please always *do* test your patches and state that you tested them. Contributing bugs to other projects ----------------------------------- Another very useful type of contributions are filing bug reports based on piuparts runs. This is described in https://piuparts.debian.org/bug_howto.html piuparts-1.1.1/ChangeLog0000644000000000000000000002552013342501370012023 0ustar 2009-12-18 Holger Levsen * This changelog is obsolete since several years. See debian/changelog instead. 2005-11-12 Lars Wirzenius * Further detailed changes to be seen in the bzr commit message log, and summarized in the NEWS file for each release. 2005-10-17 Lars Wirzenius * Version 0.12. * /var/log/faillog ignored by default. * Documented that upgrade testing to experimental doesn't work. 2005-10-14 Lars Wirzenius * Version 0.11. * Summary of changes: - Checks for missing package names on command line immediately, not after creating chroot. - apt-get in the chroot gets configured to allow unauthenticated repositories. - Tweaks to list of files ignored by default. 2005-09-15 Lars Wirzenius * Version 0.10. 2005-09-15 Lars Wirzenius * piuparts.py: When creating policy-rc.d, make it executable. * piuparts.docbook: Added missing "if". Thanks, Jonas Meurer. * run-piuparts.py: Wrote. 2005-09-09 Lars Wirzenius * Version 0.9. 2005-09-08 Lars Wirzenius * piuparts.py: Added --resolv-deps to debootstrap call, so that it works with testing/sid. 2005-09-05 Lars Wirzenius * piuparts.py, piuparts.docbook: Added -k (--keep-tmpdir) option. * piuparts.py: Added some more default ignores. * piuparts.py: Remember which packages own which files before removing packages. 2005-08-31 Lars Wirzenius * piuparts.py: Made panic() work. Added feature to report which packages own the new/modifed/removed files. Requested in Debian bug 324248 by Anibal Monsalve Salazar. 2005-08-30 Lars Wirzenius * piuparts.py: Use Python 2.3's logging module instead of NIH code. Also, log everything identically to stdout and log file, since the old behavior was confusing to many people, as observed on IRC and at Debconf5. This also obsoletes -v (--verbose). * piuparts.docbook: Document that -v (--verbose) is obsolete. 2005-08-30 Lars Wirzenius * piuparts.docbook: It's , not , and you need to pay attention to warning messages, you imbecile. 2005-08-29 Lars Wirzenius * Version 0.8. 2005-08-29 Lars Wirzenius * piuparts.py: Accept matches for any part of a filename, instead of requiring that they match everything. This is less surprising to users and they can still use ^ and $ to anchor a match to the ends, if they need it. * Makefile: Ignore the fdmount temp files that cause upgrades to etch/sid fail at the moments, and re-enable upgrade tests. * piuparts.py, piuparts.docbook: Added option -t and changed the default location for temporary files and directorys (including the chroot) to be $TMPDIR (or /tmp if not set). 2005-08-25 Lars Wirzenius * Makefile: Upgrade testing from sarge to etch fails for me, because of something to do with fdutils, so I have disabled it for now. * piuparts.py: Only use the first mirror found in sources.list, which is anyway what the manual page claims we are doing. This should reduce the problems people have been having with piuparts creating broken source.lists in the chroot when people have more than a canonical Debian repository listed. * piuparts.py: Remove temporary files upon error exit. * piuparts.docbook: Use code names instead of stable/testing/unstable in the example. 2005-07-15 Lars Wirzenius * Version 0.7. 2005-07-15 Lars Wirzenius * Makefile: Moved binary to sbin since only root can usefully run it. 2005-07-13 Lars Wirzenius * piuparts.py: Create a /usr/sbin/policy-rc.d inside the chroot when it is created. Suggested by Frank Lichtenheld. * piuparts.py: Fixed a message from "FAIL:" to "PASS:". Oops. * Makefile: Commented out an explicit mirror definition so it should now use sources.list defaults. 2005-07-10 Lars Wirzenius * piuparts.py: Suggested by Frank Lichtenheld: don't run dpkg and do other work if there are no files (cleans up log file, saves a couple of seconds). * piuparts.py, piuparts.docbook: Implemented and documented a limit on the size of outputs of command piuparts runs. 2005-07-08 Lars Wirzenius * piuparts.py: When a command has been run and it failed, show the output it gave. Removed extra dashes from argument to getopt. Changed references to "dit" (the old name for the program) to "piuparts". Changed some log file lines a bit easier to grep for, and made the start of a log easier to see. Bugs reported and wishes expressed by Frank Lichtenheld. 2005-07-08 Lars Wirzenius * piuparts.docbook: Added notes that piuparts needs to be run as root and that options must come before other command line arguments. 2005-07-05 Lars Wirzenius * Version 0.6. 2005-07-05 Lars Wirzenius * piuparts.py: Bugfix: when removing packages from the chroot, don't remove packages being depended on before the packages doing the depending. That won't work. Any fool would know it, except me. * piuparts.py: Implemented saving of meta data for the upgrade testing between Debian releases. 2005-07-04 Lars Wirzenius * piuparts.py: Made it possible to specify components to use for a mirror. * piuparts.docbook: Documented the user visible changes. 2005-06-27 Lars Wirzenius * piuparts.py: Remove and purge dependencies before the packages that were installed, so that problems with postrm scripts using non-essential packages upon purge become evident. 2005-06-27 Lars Wirzenius * piuparts.py, piuparts.docbook: Added -n / --no-ignores option. 2005-06-27 Lars Wirzenius * piuparts.py: Made mirrors mentioned in /etc/apt/sources.list be the default mirrors. Also, --mirror can be used multiple times now. * piuparts.docbook: Documented this. * Makefile: log to tmp3.log instead of twice to tmp2.log. 2005-06-27 Lars Wirzenius * Makefile: Use -f with gzip so it doesn't complain when installing new version on top of old. 2005-06-27 Lars Wirzenius * piuparts.py: Added /etc/ld.so.conf to list of ignored files and changed exim related ignore patterns (adding exim4, for example). * piuparts.py: When purging, use --purge and not --remove to dpkg. * piuparts.py: Report version and command line arguments at startup. * piuparts.py: When calling install_purge_test, report package list when command line args are package names, instead of empty list. 2005-06-23 Lars Wirzenius * Version 0.5. 2005-06-23 Lars Wirzenius * piuparts.py: Check symlink targets as well. 2005-06-23 Lars Wirzenius * piuparts.docbook: Added --ignore-regexp to manual page. 2005-06-23 Lars Wirzenius * TODO: Added. * Makefile: Added test case for upgrading between distros. * piuparts.docbook: Rewrote DESCRIPTION and EXAMPLES sections. * piuparts.py: Added /var/cache/man/index.db and /var/log/dpkg.log to ignored files. Added regexps for more versatile ignoring and -I option. * piuparts.py: Disabled comparison of mtimes of files for now. * piuparts.py: File listings are now sorted. * piuparts.py: Added upgrade tests between Debian releases. 2005-06-20 Lars Wirzenius * piuparts.py: Refactored things more than a bit in preparation for new functionality. 2005-06-19 Lars Wirzenius * Version 0.4. 2005-06-19 Lars Wirzenius * piuparts.docbook, piuparts.py: Added -p option. * piuparts.docbook: Updated a description of what the program does. 2005-06-19 Lars Wirzenius * piuparts.docbook: Documented upgrade testing. 2005-06-19 Lars Wirzenius * piuparts.py: Added a simple form of upgrade testing. 2005-06-19 Lars Wirzenius * piuparts.py: Implemented suggestion from Frank Lichtenheld to do package purginging in two steps: first remove, then purge. This helps find postrm scripts that use things they aren't allowed to use. * README: Added naming credit to Tollef Fog Heen. 2005-06-18 Lars Wirzenius * Version 0.3. 2005-06-18 Lars Wirzenius * piuparts.py: Added /var/cache/debconf and /var/cache/debconf/passwords.dat to the list of filenames that are automatically ignored. * piuparts.py: After unpacking a base tarball, run apt-get update and clean on it, to be sure everything is up to date. 2005-06-18 Lars Wirzenius * piuparts.py: Added option -a (--apt) that lets the user install the packages to be tested via apt-get, instead of by specifying package files. * piuparts.docbook: Documented -a (--apt). 2005-06-17 Lars Wirzenius * piuparts.docbook: Added a note that piuparts is meant for people making Debian packages before they upload them. 2005-06-17 Lars Wirzenius * piuparts.py, piuparts.docbook: Added -V (--version) option. 2005-06-17 Lars Wirzenius * piuparts.py, README: Removed notice about python-apt / apt_pkg since it wasn't actually used. * piuparts.py: Changed things so that we us os.walk and os.stat to find filesystem object meta data, instead of creating a tarball and then scanning that. This makes things quite a bit faster: with a pre-generate base tarball (from pbuilder, for example), a simple installation test can now happen in less than 7 seconds wall clock time. 2005-06-14 Lars Wirzenius * Version 0.2. * Second public release. The first one was just a mention on an IRC channel, so it doesn't really count, and I didn't feel like doing change log entries before now. This version only does simple installation and removal testing, but should work at least for simple cases. piuparts-1.1.1/Makefile0000644000000000000000000002324513607334164011724 0ustar prefix = /usr/local sbindir = $(prefix)/sbin sharedir = $(prefix)/share mandir = $(sharedir)/man man1dir = $(mandir)/man1 man8dir = $(mandir)/man8 libdir = $(prefix)/lib docdir = $(prefix)/share/doc/piuparts site3 = $(libdir)/python3/dist-packages htdocsdir = $(sharedir)/piuparts/htdocs etcdir = $(prefix)/etc distribution=${shell dpkg-parsechangelog | sed -n 's/^Distribution: *//p'} ifeq ($(distribution),UNRELEASED) version := ${shell echo "`dpkg-parsechangelog | sed -n 's/^Version: *//p'`~`date +%Y%m%d%H%M`~`git describe --dirty`"} else version := ${shell dpkg-parsechangelog | sed -n 's/^Version: *//p'} endif # generate several scripts, conffiles, ... from templates (*.in, *.py) # by substituting placeholders SCRIPTS_TEMPLATES = $(wildcard *.in master-bin/*.in slave-bin/*.in conf/*.in) SCRIPTS_PYTHON_BINARY = $(wildcard *.py master-bin/*.py slave-bin/*.py) SCRIPTS_GENERATED = $(SCRIPTS_TEMPLATES:.in=) $(SCRIPTS_PYTHON_BINARY:.py=) DOCS_GENERATED = piuparts.1 piuparts_slave_run.8 piuparts_slave_join.8 piuparts_slave_stop.8 docs/build define placeholder_substitution sed -r \ -e 's/__PIUPARTS_VERSION__/$(version)/g' \ -e 's%@libdir@%$(libdir)%g' \ -e 's%@sharedir@%$(sharedir)%g' \ -e 's%@sbindir@%$(sbindir)%g' \ $< > $@ endef %: %.in Makefile $(placeholder_substitution) %: %.py Makefile $(placeholder_substitution) all: build python_scripts = $(wildcard *.py piupartslib/*.py master-bin/*.py slave-bin/*.py) python-syntax-check: @set -e -x; $(foreach py,$(python_scripts),python3 -m py_compile $(py);) $(RM) $(python_scripts:=c) build: build-stamp build-master-stamp build-slave: build-stamp build-master: build-stamp build-master-stamp build-stamp: $(SCRIPTS_GENERATED) $(DOCS_GENERATED) Makefile $(MAKE) -C instances $(MAKE) python-syntax-check touch $@ build-master-stamp: (cd helpers/debiman-piuparts-distill && go build) touch $@ build-doc: $(DOCS_GENERATED) docs/build: docs/build python3 -m sphinx docs/ docs/build/ piuparts.1: docs/piuparts/piuparts.1.txt python3 -m sphinx -b man -c docs/piuparts/ docs/piuparts/ ./ piuparts_slave_run.8: docs/piuparts_slave_run/piuparts_slave_run.8.txt python3 -m sphinx -b man -c docs/piuparts_slave_run/ docs/piuparts_slave_run/ ./ piuparts_slave_join.8: docs/piuparts_slave_join/piuparts_slave_join.8.txt python3 -m sphinx -b man -c docs/piuparts_slave_join/ docs/piuparts_slave_join/ ./ piuparts_slave_stop.8: docs/piuparts_slave_stop/piuparts_slave_stop.8.txt python3 -m sphinx -b man -c docs/piuparts_slave_stop/ docs/piuparts_slave_stop/ ./ install-doc: build-stamp # txt install -d $(DESTDIR)$(docdir)/ install -m 0644 docs/README.txt docs/README_server.txt $(DESTDIR)$(docdir)/ # html install -d $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/*.html $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/searchindex.js $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/objects.inv $(DESTDIR)$(docdir)/html/ install -d $(DESTDIR)$(docdir)/html/_static/ install -m 0644 docs/build/_static/* $(DESTDIR)$(docdir)/html/_static install -d $(DESTDIR)$(docdir)/html/piuparts/ install -m 0644 docs/build/piuparts/index.html $(DESTDIR)$(docdir)/html/piuparts/ install -m 0644 docs/build/piuparts/piuparts.1.html $(DESTDIR)$(docdir)/html/piuparts/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -m 0644 docs/build/piuparts_slave_run/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -m 0644 docs/build/piuparts_slave_run/piuparts_slave_run.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -m 0644 docs/build/piuparts_slave_join/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -m 0644 docs/build/piuparts_slave_join/piuparts_slave_join.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ install -m 0644 docs/build/piuparts_slave_stop/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ install -m 0644 docs/build/piuparts_slave_stop/piuparts_slave_stop.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ install -d $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/*.html $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/searchindex.js $(DESTDIR)$(docdir)/html/ install -m 0644 docs/build/objects.inv $(DESTDIR)$(docdir)/html/ install -d $(DESTDIR)$(docdir)/html/_static/ install -m 0644 docs/build/_static/* $(DESTDIR)$(docdir)/html/_static install -d $(DESTDIR)$(docdir)/html/piuparts/ install -m 0644 docs/build/piuparts/index.html $(DESTDIR)$(docdir)/html/piuparts/ install -m 0644 docs/build/piuparts/piuparts.1.html $(DESTDIR)$(docdir)/html/piuparts/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -m 0644 docs/build/piuparts_slave_run/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -m 0644 docs/build/piuparts_slave_run/piuparts_slave_run.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -m 0644 docs/build/piuparts_slave_join/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -m 0644 docs/build/piuparts_slave_join/piuparts_slave_join.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/ install -d $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ install -m 0644 docs/build/piuparts_slave_stop/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ install -m 0644 docs/build/piuparts_slave_stop/piuparts_slave_stop.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/ # manpages install -d $(DESTDIR)$(man1dir) install -m 0644 piuparts.1 $(DESTDIR)$(man1dir)/ gzip -9fn $(DESTDIR)$(man1dir)/piuparts.1 install -d $(DESTDIR)$(man8dir) install -m 0644 piuparts_slave_run.8 piuparts_slave_join.8 piuparts_slave_stop.8 $(DESTDIR)$(man8dir)/ gzip -9fn $(DESTDIR)$(man8dir)/piuparts_slave_run.8 gzip -9fn $(DESTDIR)$(man8dir)/piuparts_slave_join.8 gzip -9fn $(DESTDIR)$(man8dir)/piuparts_slave_stop.8 install-conf: build-stamp install -d $(DESTDIR)$(etcdir)/piuparts install -m 0644 conf/piuparts.conf.sample $(DESTDIR)$(etcdir)/piuparts/piuparts.conf install -m 0644 conf/distros.conf $(DESTDIR)$(etcdir)/piuparts/ install -d $(DESTDIR)$(etcdir)/apache2/conf-available install -m 0644 conf/piuparts-master.conf $(DESTDIR)$(etcdir)/apache2/conf-available/ install-conf-4-running-from-git: build-stamp install -d $(DESTDIR)$(etcdir)/piuparts install -m 0644 conf/crontab-master $(DESTDIR)$(etcdir)/piuparts/ install -m 0644 conf/crontab-slave $(DESTDIR)$(etcdir)/piuparts/ install -m 0644 conf/distros.conf $(DESTDIR)$(etcdir)/piuparts/ install -m 0644 instances/piuparts.conf.* $(DESTDIR)$(etcdir)/piuparts/ install -d $(DESTDIR)$(sharedir)/piuparts/slave install -m 0755 update-piuparts-slave-setup $(DESTDIR)$(sharedir)/piuparts/slave/ install -d $(DESTDIR)$(sharedir)/piuparts/master install -m 0755 update-piuparts-master-setup $(DESTDIR)$(sharedir)/piuparts/master/ install-common: build-stamp install -d $(DESTDIR)$(site3)/piupartslib install -m 0644 piupartslib/*.py $(DESTDIR)$(site3)/piupartslib/ install -d $(DESTDIR)$(sharedir)/piuparts/lib install -m 0644 lib/*.sh $(DESTDIR)$(sharedir)/piuparts/lib/ install-master: build-master-stamp install-common install -d $(DESTDIR)$(libdir)/piuparts/ install -m 0755 helpers/debiman-piuparts-distill/debiman-piuparts-distill $(DESTDIR)$(libdir)/piuparts/ install -d $(DESTDIR)$(sharedir)/piuparts install -m 0755 piuparts-master piuparts-master-backend piuparts-report piuparts-analyze $(DESTDIR)$(sharedir)/piuparts/ # do not install the templates (*.in, *.py, *.) nor __pycache__ install -d $(DESTDIR)$(sharedir)/piuparts/master install -m 0755 $(filter-out %.in %.py %__pycache__,$(wildcard master-bin/*)) $(DESTDIR)$(sharedir)/piuparts/master/ install -d $(DESTDIR)$(sharedir)/piuparts/known_problems install -m 0644 known_problems/*.conf $(DESTDIR)$(sharedir)/piuparts/known_problems/ install -d $(DESTDIR)$(htdocsdir) install -m 0644 htdocs/*.* $(DESTDIR)$(htdocsdir)/ install -d $(DESTDIR)$(htdocsdir)/images install -m 0644 htdocs/images/*.* $(DESTDIR)$(htdocsdir)/images/ install -d $(DESTDIR)$(htdocsdir)/templates/mail install -m 0644 bug-templates/*.mail $(DESTDIR)$(htdocsdir)/templates/mail/ #install -d $(DESTDIR)$(etcdir)/piuparts/known_problems #install -m 0644 known_problems/*.conf $(DESTDIR)$(etcdir)/piuparts/known_problems/ install-slave: install-common install -d $(DESTDIR)$(sbindir) install -m 0755 piuparts $(DESTDIR)$(sbindir)/ install -d $(DESTDIR)$(sharedir)/piuparts install -m 0755 piuparts-slave $(DESTDIR)$(sharedir)/piuparts/ # do not install the templates (*.in, *.py) nor __pycache__ install -d $(DESTDIR)$(sharedir)/piuparts/slave install -m 0755 $(filter-out %.in %.py %__pycache__,$(wildcard slave-bin/*)) $(DESTDIR)$(sharedir)/piuparts/slave/ install -d $(DESTDIR)$(etcdir)/piuparts @set -e -x ; \ for d in $$(ls custom-scripts) ; do \ install -d $(DESTDIR)$(etcdir)/piuparts/$$d ; \ install -m 0755 custom-scripts/$$d/* $(DESTDIR)$(etcdir)/piuparts/$$d/ ; done install: install-master install-slave check: python3 -m nose --verbose clean: rm -f build-stamp rm -f build-master-stamp rm -fr $(DOCS_GENERATED) rm -fr .doctrees/ find . -iname '*.pyc' -type f -delete find . -iname __pycache__ -type d -delete rm -f $(SCRIPTS_GENERATED) $(RM) helpers/debiman-piuparts-distill/debiman-piuparts-distill $(MAKE) -C instances clean # for maintainer convenience only check-whitespace: grep -r --exclude-dir .git --exclude '*.pyc' --exclude '*.png' --exclude '*.ico' -E '\s+$$' . || echo "no trailing whitespace found" grep -r --exclude-dir .git --exclude '*.pyc' --exclude '*.png' --exclude '*.ico' -P ' \t' . || echo "no space-tab combo found" # for maintainer convenience only tg-deps: tg summary --graphviz | dot -T png -o deps.png xli deps.png & piuparts-1.1.1/NEWS0000644000000000000000000001465513266762112010767 0ustar NEWS file for piuparts ====================== piuparts is a tool for testing that .deb packages can be installed, upgraded, and removed without trouble. See the README file and the manual page for more information. This file summarizes the major changes, particularly user visible changes, for each release. Detailed change information can be found in bzr commit messages. This file is _deprecated_ now, see debian/NEWS.Debian instead! Version 0.20, September 22, 2006 -------------------------------- When running external commands, use subprocess.Popen so that there is no need for tricky (and therefore buggy) quoting of shell command line arguments. Version 0.19, September 8, 2006 ------------------------------- When reporting a bad symlink, show the target. Version 0.18, September 7, 2006 ------------------------------- New features Piuparts now checks for symlinks whose target does not exist. Option parsing code has been rewritten, and --help now works better. The chroot is now minimized before used: all unnecessary packages are purged. /dev/MAKEDEV, /etc/nologin, /usr/doc/cpio, /var/spool/cron added to default ignores. A version number may now begin with a + character. There was a package that did that and piuparts crashed. Version 0.17 ------------ Bug fixes The configuration files of piuparts-master/slave are now documented in the README. The Python profiler is no longer used. It used to be, but that was a leftover from development (also known as failure to read diffs before committing). When testing upgrades between distributions, piuparts now makes sure that the packages being tested are upgraded, even if it means removing an old version of a dependency. New features Piuparts now checks for processes running inside the chroot after it has installed or purged a package. Because it uses policy-rc.d to prevent any services from starting, no processes should run inside the chroot after an installation or a purge has completed. This check then finds packages that don't use invoke-rc.d to start new processes. A number of new default ignores have been added: /etc/modprobe.d, compiled versions of debconf's Python modules, papercut, ssl certificates. /proc is now mounted (and unmounted) inside the chroot. Version 0.16 ------------ Bug fixes The temporary directory for the chroot was not being removed in all cases when running apt-get failed. This has now been fixed. New features Added piuparts-analyze.py, which looks at new logs of failed tests and compares them to logs of failed tests for earlier versions of the same packages, and if so, moves them automatically around. This saves a bit of manual works. Thanks to Goswin Brederlow for the idea. When piuparts creates a chroot from a tarball (option -b), it now upgrades it before using it. A number of new entries to the default ignores list. Log files are now created with permissions that are 0666 modified with the process umask. piuparts-report.py has been optimized somewhat. Version 0.15 ------------ Bug fixes The dependency parser in piupartslib now understands < and > (they're deprecated but one or two packges still use them). It also now allows underscores in package names because of the type-handling package. Small fixes to the manual page. New features and significant user visible changes piuparts-master now understands Provides: headers. A number of new entries to the default ignores list. New option --keep-sources-list from John Wright. Version 0.14 ------------ Bug fixes Specifications for ignoring some directories were buggy and have now been fixed: /var/spool/news, /var/lib/cvs. When testing a .deb file given on the command line, if any of its dependencies were missing, the package itself would be removed instead of piuparts reporting an error. This has been fixed. The check for whether a package is marked untestable for piuparts-master now works. New features and significant user visible changes New program piuparts-report.py produces some "statistics" about packages and their status with regard to testing with piuparts-slave. The chroot is always set up for piuparts, even if it is unpacked from a tarball. This reduces problems with out-of-date chroots and with using the pbuilder base.tgz tarball. Now ignored by default: /var/lib/firebird, /var/spool/news, /var/lib/rbldns, /home/ftp. Version 0.13 ------------ Bug fixes The configuration for apt-get (in the chroot) to allow un-authenticated sources now actually works. There used to be a typo. New features and other user visible changes The old run-piuparts.py script has been replaced by a distributed system, consisting of piuparts-master.py and piuparts-slave.py, plus a reporting script piuparts-report.py. Since these are not useful for most users, they're not installed on $PATH, but in /usr/share/piuparts instead. The slave part also runs upgrade tests between Debian releases, which run-piuparts.py didn't. Some additional files are ignored by default when comparing the state of the chroot before and after package installation. piuparts-1.1.1/TODO0000644000000000000000000001616413463366315010761 0ustar Things to do for piuparts ========================= Please also check the BTS, especially the bugs with a severity higher than wishlist! And see the bottom of this file for wishlist bugs someone once expressed… Some stuff we might tackle during the bullseye development cycle: - switch to python3 (that's actually a must) - README_server.txt: rewrite style a bit more. its super easy to setup now! - ==== piuparts.debian.org specific configuration ^^^^^^^ scripts should be the headline, not piu.d.o - move more bits from README_pejacevic.txt to README_server.txt - split out README_protocol and README_piuparts.conf? -> in piuparts.conf manpage maybe? - support multiple architectures: #762308 - piuparts-report should have a list of available arch and list packages only available on untested archs in a new state "(depends-)not-available-on-tested-archs" - master should (per default) only schedule packages which are not available on the master arch to slaves of different archs -> "schedule-evenly-to-slaves = no" - more stats and graphs: - new section stats page: - packages processed per day and section, master writes submissions.txt since 0.45 for all sections. - generate simple diagrams: number of source + binary packages in all single distros: lenny, squeeze, wheezy, jessie, stretch, buster, sid. - graph about piuparts stati for all sections combined? (possible ignore successful) - master should create the master and backup directories, if they don't exist. If master does that remove that sentence from README_server.txt again, same with slave and tmp. - if it weren't for 'slave-bin/slave_cleanup', the slave would only need rights to run "sudo piuparts" but nothing else. If we can clean this up, the sudoers.d should recommend sudo (lsof|kill|umount) for admins. - if there were real schroot support, piuparts could be used without sudo. (#708663) - use network namespaces to disable network during the tests: - < weasel> says: unshare --uts --ipc --net -- sh -c 'ip addr add 127.0.0.1/8 dev lo && ip link set dev lo up && stuff' and points to https://anonscm.debian.org/gitweb/?p=mirror/dsa-puppet.git;a=blob;f=modules/porterbox/files/dd-schroot-cmd#l104 - problem might be access to the mirror, either (bind mounted) nfs access will still work in the chroots or do as its done on the porterboxes: apt-get install -d , unshare apt-get install foo - add a sample config with all possible keys set to some useful value (like /usr/share/doc/apt/examples/configure-index.gz) - generate piuparts.1.txt automatically from piuparts.py - see this blog post for a nice howto: http://andialbrecht.wordpress.com/2009/03/17/creating-a-man-page-with-distutils-and-optparse/ - though this seems pretty complicated... maybe rather grep for parser.add_option and help= in piuparts.py ?! - requires merging all the additional infomation in piuparts.1.txt into piuarts.py - parsing piuparts --help output may be easier than parsing piuparts.py - elso: examples are duplicated in piuparts.1.txt and README.txt - check the logfiles (especially pass/) for - "Exception in thread" - java stacktraces - "Can't locate .* in @INC" - we should probably have an install test with --enable-recommends and without --warn-on-others to avoid adding artificial barriers where package subsets are configured (wheezy2jessie-rcmd is *not* the solution for this) - report actually ignored files/patterns ROT13 encoded to be able to spot and reschedule such tests - p-r: in the section summary page, report the piuparts flags being used - p-s: report age of the basetgz being used - solve the discrepancies between "dist-upgrade" (apt) and "distupgrade" (piuparts), adopting apt's naming Ideas for later: - maybe compress all logfiles - install from git/Makefile: remove the need for /etc/piuparts - accept a PIUPARTS_CONF environment variable everywhere to point to a different piuparts.conf - write reportbug-like wrapper for mass bug filing (start simple, make it more sophisticated later). - rework known_problems: - use a number prefix for sorting - add title information - piuparts-report: "discover" the available known_problems, don't hardcode the list - drop _issue/_error duplication, have flags inside to indicate whether to generate _issues.tpl (pass/) and/or _error.tpl (fail/ bugged/ affected/) - rework known problems to a python-friendlier format - the templates used by piuparts-report.py should all be taken from /etc/piuparts/templates/ and not be included in the python source - a redirect of http://piuparts.d.o/foo to http://p.d.o/source/f/foo.html would be nice More ideas for even later: - make it possible to call aptitude (or similar) instead of apt-get and allow to override the commandline arguments. - "decorate" (strike-through) bug links generated by piuparts-analyze to indicate resolved state (take package version into account!) - report: - write stats about the reasons for failures, as its done with shell scripts now (piuparts-analyze.py is an existing "fragment".) - RSS feeds of logs - do more fancy R graphs, eg. also per state - link (and target) to piuparts.d.o configuration is static to pejacevic. should refer to the actual hosts configuration if running somewhere else - not sure if it's a sensible thing to to, but provide a way to turn off debugging output for piuparts.py - see http://docs.python.org/library/logging.html - automated testing of piuparts using an archive of known broken packages: - create archive of broken packages to provide test cases for piuparts testing. - create emacspeak-broken-dpkg-preconfigure package for broken repo. (then later put more broken packages in there and use that for testing piuparts) (Closed) wishlist bugs are listed here so it's easier to find information about them should one wants to implement support for those. Usually wishlist bugs are closed not because the feature is unwanted, but because noone is working on them… see http://blog.liw.fi/posts/wishlist-bugs for a longer rationale. piuparts: - #664727 - support cowbuilder in addition to pbuilder - #798306 - piuparts should support [trusted=yes] in "file deb src" sources.list lines - #588042 - add an option to override the log file instead of append to it - #354842 - new option: remove /usr/local - #558819 - new option: check that daemon stop scripts don't fail even if the daemon doesn't run - #660896 - new option: use etckeeper and show how modified files have changed - #403648 - new test: Try unpack, then remove? - #561444 - new test: install, remove configuration files, re-install, check that configuration files don't re-appear - #561445 - new test: install, mangle configuration file, re-install, check that manglement is still there - #665016 - new test: more checks on alternatives - #810323 - new test: deconfiguring and configuring a package again - #658731 - add pattern for owned files that are always an error - #682741 - add a second set of ignored files and patterns for "partial purge" tests piuparts-report: - #685813 - add a single report of all failures (from all sections) ordered by age piuparts-1.1.1/bug-templates/0000755000000000000000000000000013605426771013033 5ustar piuparts-1.1.1/bug-templates/broken_symlink.mail0000644000000000000000000000044713266771660016735 0ustar To: submit@bugs.debian.org Subject: broken symlink: Package: Version: Severity: normal User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package ships (or creates) a broken symlink. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/copyright_file_missing_after_upgrade.mail0000644000000000000000000000227313403304304023312 0ustar To: submit@bugs.debian.org Subject: copyright file missing after upgrade (policy 12.5) Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, a test with piuparts revealed that your package misses the copyright file after an upgrade, which is a violation of Policy 12.5: https://www.debian.org/doc/debian-policy/ch-docs.html#copyright-information After the upgrade /usr/share/doc/$PACKAGE/ is just an empty directory. This was observed on the following upgrade paths: From the attached log (scroll to the bottom...): Additional info may be available here: https://wiki.debian.org/MissingCopyrightFile Note that dpkg intentionally does not replace directories with symlinks and vice versa, you need the maintainer scripts to do this. See in particular the end of point 4 in https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-unpack-phase-of-installation-or-upgrade It is recommended to use the dpkg-maintscript-helper commands 'dir_to_symlink' and 'symlink_to_dir' (available since dpkg 1.17.14) to perform the conversion, ideally using d/$PACKAGE.maintscript. See dpkg-maintscript-helper(1) and dh_installdeb(1) for details. cheers, piuparts-1.1.1/bug-templates/creates_system_user_in_home.mail0000644000000000000000000000176513266762112021471 0ustar To: submit@bugs.debian.org Subject: creates system user in /home Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package creates a system user with a home directory in /home/$SYSUSER, which is a policy violation. The FHS (which is part of the policy) states: "/home : User home directories (optional)" "/home is a fairly standard concept, but it is clearly a site-specific filesystem. The setup will differ from host to host. Therefore, no program should rely on this location." https://www.debian.org/doc/packaging-manuals/fhs/fhs-2.3.html#HOMEUSERHOMEDIRECTORIES System users are usually placed in /var/lib. Possible problems that can arise in case /home is a remote file system: * the local root user may not be allowed to write to /home * $SYSUSER may have different UIDs on different hosts For that particular piuparts test /home has been made unusable. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/cronjob_exits_with_error_after_package_removal.mail0000644000000000000000000000054213266762112025370 0ustar To: submit@bugs.debian.org Subject: cronjob exits with error after package removal Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your packages cronjob exits with error after the package has been removed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/cronjob_produces_output_after_package_removal.mail0000644000000000000000000000054013266762112025232 0ustar To: submit@bugs.debian.org Subject: cronjob produces output after package removal Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your packages cronjob produces output after the package has been removed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/doesnt_use_invoke-rc.d.mail0000644000000000000000000000115113351442132020227 0ustar To: submit@bugs.debian.org Subject: doesn't use invoke-rc.d Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package starts processes where it shouldn't. This is very probably due to not using invoke-rc.d as mandated by policy 9.3.3.2. This is seriously disturbing! ;-) See https://www.debian.org/doc/debian-policy/ch-opersys.html#interfacing-with-init-systems and /usr/share/doc/sysv-rc/README.invoke-rc.d.gz as well as /usr/share/doc/sysv-rc/README.policy-rc.d.gz From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install.mail0000644000000000000000000000060113266762112017215 0ustar To: submit@bugs.debian.org Subject: fails to install: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install. As per definition of the release team this makes the package too buggy for a release, thus the severity. From the attached log (scroll to the bottom...): cheers, ././@LongLink0000644000000000000000000000014600000000000007774 Lustar piuparts-1.1.1/bug-templates/fails_to_install_due_to_incorrect_dependencies_in_init.d_LSB_header.mailpiuparts-1.1.1/bug-templates/fails_to_install_due_to_incorrect_dependencies_in_init.d_LSB_header.mai0000644000000000000000000000104513266762112031124 0ustar To: submit@bugs.debian.org Subject: fails to install due to incorrect dependencies in init.d LSB header Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts User: initscripts-ng-devel@lists.alioth.debian.org Usertags: incorrect-dependency Hi, during a test with piuparts I noticed your package failed to install due to incorrect dependencies in the init.d LSB header. Some Debian notes are available from at https://wiki.debian.org/LSBInitScripts From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install_due_to_insserv_rejecting_the_script_header.mail0000644000000000000000000000066413266762112030124 0ustar To: submit@bugs.debian.org Subject: fails to install due to insserv rejecting the script header Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install due to insserv rejecting the script header. Some notes are available from at https://wiki.debian.org/LSBInitScripts From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install_purge_install.mail0000644000000000000000000000126313266762112022152 0ustar To: submit@bugs.debian.org Subject: fails to install, purge, and install again Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install, remove+purge, and install again. Like a plain failure on initial install this makes the package too buggy for a release, thus the severity. This is often a problem with the home directory of a system user: the user is not deleted during purge (which is fine), but the home directory is removed. Since the user already exists on the next installation, adduser won't recreate the home. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install_remove_distupgrade_install.mail0000644000000000000000000000131613266762112024717 0ustar To: submit@bugs.debian.org Subject: fails to install, remove, distupgrade, and install again Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install (in 'CODENAME1'), remove (but not purge), distupgrade to 'CODENAME2', and install again. Before the second installation the package is in config-files-remaining state. The configuration is remaining from the last version that was successfully configured - which is from the previous release. Like a plain failure on initial install this makes the package too buggy for a release, thus the severity. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install_remove_install.mail0000644000000000000000000000125713266762112022330 0ustar To: submit@bugs.debian.org Subject: fails to install, remove, and install again Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install, remove (but not purge), and install again. Before the second installation the package is in config-files-remaining state. The configuration is remaining from the last version that was successfully configured - which is the same version that is going to be installed again. Like a plain failure on initial install this makes the package too buggy for a release, thus the severity. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_install_trying_to_overwrite_other_packages_files.mail0000644000000000000000000000104413351442132027634 0ustar To: submit@bugs.debian.org Subject: fails to install, trying to overwrite other packages files Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to install because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_purge.mail0000644000000000000000000000063513416657733016711 0ustar To: submit@bugs.debian.org Subject: fails to purge: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to purge. According to policy 7.2 you cannot rely on the depends being available during purge, only the essential packages are available for sure. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_purge_-_command_deluser_adduser_in_postrm_not_found.mail0000644000000000000000000000217113416657733030217 0ustar To: submit@bugs.debian.org Subject: fails to purge - command (deluser|adduser) in postrm not found Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to purge due to a command not found. According to policy 7.2 you cannot rely on the depends being available during purge, only the essential packages are available for sure. The fix should be easy: your package is using adduser or deluser from the adduser package, which is only priority important. Using useradd or userdel from the passwd package (priority required) should fix this problem. There is ongoing discussion how to handle system users on package removal, see https://bugs.debian.org/621833 Consensus seems to be not to remove system users (to avoid reusing UIDs which could grant access to the wrong files) but to "lock" them (where "locking"/"unlocking" is not yet precisely defined). Until that has been decided it should be sufficient to have the postrm script ignore any errors from deluser: deluser ... || true From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_purge_-_command_in_postrm_not_found.mail0000644000000000000000000000072513416657733024770 0ustar To: submit@bugs.debian.org Subject: fails to purge - command in postrm not found Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to purge due to a command not found. According to policy 7.2 you cannot rely on the depends being available during purge, only the essential packages are available for sure. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_purge_-_command_ucf_in_postrm_not_found.mail0000644000000000000000000000114113416657733025616 0ustar To: submit@bugs.debian.org Subject: fails to purge - command ucf in postrm not found Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to purge due to a command not found. According to policy 7.2 you cannot rely on the depends being available during purge, only the essential packages are available for sure. Please see the manpages ucf(1), ucfr(1) and the example maintainer scripts under /usr/share/doc/ucf/examples/ for correct usage of ucf. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_purge_due_to_incorrect_dependencies_in_init.d_LSB_header.mail0000644000000000000000000000104113416657733030761 0ustar To: submit@bugs.debian.org Subject: fails to purge due to incorrect dependencies in init.d LSB header Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts User: initscripts-ng-devel@lists.alioth.debian.org Usertags: incorrect-dependency Hi, during a test with piuparts I noticed your package failed to purge due to incorrect dependencies in the init.d LSB header. Some Debian notes are available from at https://wiki.debian.org/LSBInitScripts From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_reinstall.mail0000644000000000000000000000051013266771660017553 0ustar To: submit@bugs.debian.org Subject: fails to reinstall: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed to reinstall the same version that was already installed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_remove.mail0000644000000000000000000000042413266762112017047 0ustar To: submit@bugs.debian.org Subject: fails to remove: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to remove. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_-_trying_to_overwrite.mail0000644000000000000000000000157713351442132023623 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'DISTRO1' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'DISTRO1'. It installed fine in 'DISTRO1', then the upgrade to 'DISTRO2' fails because it tries to overwrite files that are owned by other packages without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces This problem is exposed during the upgrade of the 'EXPOSER' package, not by upgrading only the buggy package itself. In order to have piuparts automatically track this problem, I'll mark this bug as Affects/Found in the exposing package(s), too. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_jessie.mail0000644000000000000000000000056613266762112021575 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'jessie': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'jessie'. It installed fine in 'jessie', then the upgrade to 'stretch' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_jessie222testing.mail0000644000000000000000000000111713266771660023422 0ustar To: submit@bugs.debian.org Subject: fails to upgrade jessie -> stretch -> buster Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'jessie' to 'stretch' to 'buster'. It installed fine in 'jessie', and upgraded to 'stretch' successfully, but then the upgrade to 'buster' failed. In case the package was not part of an intermediate stable release, the version from the preceding stable release was kept installed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_jessie_-_trying_to_overwrite.mail0000644000000000000000000000116313351442132026177 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'jessie' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'jessie'. It installed fine in 'jessie', then the upgrade to 'stretch' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_lenny.mail0000644000000000000000000000056313266762112021435 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'lenny': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'lenny'. It installed fine in 'lenny', then the upgrade to 'squeeze' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_lenny222testing.mail0000644000000000000000000000126313266771660023267 0ustar To: submit@bugs.debian.org Subject: fails to upgrade lenny -> squeeze -> wheezy -> jessie -> stretch -> buster Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'lenny' to 'squeeze' to 'wheezy' to 'jessie' to 'stretch' to 'buster'. It installed fine in 'lenny', and upgraded to 'squeeze', 'wheezy', 'jessie', and 'stretch' successfully, but then the upgrade to 'buster' failed. In case the package was not part of an intermediate stable release, the version from the preceding stable release was kept installed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_lenny_to_squeeze_to_wheezy.mail0000644000000000000000000000070513266762112025773 0ustar To: submit@bugs.debian.org Subject: fails to upgrade lenny -> squeeze -> wheezy Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'lenny' to 'squeeze' to 'wheezy'. It installed fine in 'lenny', and upgraded to 'squeeze' successfully, but then the upgrade to 'wheezy' failed. From the attached log (scroll to the bottom...): cheers, ././@LongLink0000644000000000000000000000015100000000000007770 Lustar piuparts-1.1.1/bug-templates/fails_to_upgrade_from_lenny_to_squeeze_to_wheezy_-_trying_to_overwrite.mailpiuparts-1.1.1/bug-templates/fails_to_upgrade_from_lenny_to_squeeze_to_wheezy_-_trying_to_overwrite.0000644000000000000000000000130313351442132031533 0ustar To: submit@bugs.debian.org Subject: fails to upgrade lenny -> squeeze -> wheezy - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'lenny' to 'squeeze' to 'wheezy'. It installed fine in 'lenny', and upgraded to 'squeeze' successfully, but then the upgrade to 'wheezy' failed because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_sid_-_trying_to_overwrite.mail0000644000000000000000000000120113351442132025465 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'sid' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'sid' to 'experimental'. It installed fine in 'sid', then the upgrade to 'experimental' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_squeeze.mail0000644000000000000000000000057013266762112021767 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'squeeze': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'squeeze'. It installed fine in 'squeeze', then the upgrade to 'wheezy' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_squeeze222testing.mail0000644000000000000000000000122313266771660023617 0ustar To: submit@bugs.debian.org Subject: fails to upgrade squeeze -> wheezy -> jessie -> stretch -> buster Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'squeeze' to 'wheezy' to 'jessie' to stretch to buster.. It installed fine in 'squeeze', and upgraded to 'wheezy', 'jessie', and 'stretch' successfully, but then the upgrade to 'buster' failed. In case the package was not part of an intermediate stable release, the version from the preceding stable release was kept installed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_squeeze_-_trying_to_overwrite.mail0000644000000000000000000000116513351442132026400 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'squeeze' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'squeeze'. It installed fine in 'squeeze', then the upgrade to 'wheezy' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_stable_to_sid_-_trying_to_overwrite.mail0000644000000000000000000000137113351442132027531 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'stable' to 'sid' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'stable'. It installed fine in 'stable', then the upgrade to 'sid' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces This test intentionally skipped 'testing' to find file overwrite problems before packages migrate from 'unstable' to 'testing'. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_stretch.mail0000644000000000000000000000057013266771660021772 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'stretch': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'stretch'. It installed fine in 'stretch', then the upgrade to 'buster' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_stretch_-_trying_to_overwrite.mail0000644000000000000000000000116513351442132026373 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'stretch' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'stretch'. It installed fine in 'stretch', then the upgrade to 'buster' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_testing.mail0000644000000000000000000000056513266771660021777 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'testing': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'testing'. It installed fine in 'testing', then the upgrade to 'sid' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_testing_-_trying_to_overwrite.mail0000644000000000000000000000116213351442132026371 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'testing' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'testing'. It installed fine in 'testing', then the upgrade to 'sid' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_wheezy.mail0000644000000000000000000000056513266762112021625 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'wheezy': Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'wheezy'. It installed fine in 'wheezy', then the upgrade to 'jessie' fails. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_wheezy222testing.mail0000644000000000000000000000116213266771660023453 0ustar To: submit@bugs.debian.org Subject: fails to upgrade wheezy -> jessie -> stretch -> buster Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'wheezy' to 'jessie' to 'stretch' to 'buster'. It installed fine in 'wheezy', and upgraded to 'jessie' and 'stretch' successfully, but then the upgrade to 'buster' failed. In case the package was not part of an intermediate stable release, the version from the preceding stable release was kept installed. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/fails_to_upgrade_from_wheezy_-_trying_to_overwrite.mail0000644000000000000000000000116213351442132026227 0ustar To: submit@bugs.debian.org Subject: fails to upgrade from 'wheezy' - trying to overwrite ... Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to upgrade from 'wheezy'. It installed fine in 'wheezy', then the upgrade to 'jessie' fails because it tries to overwrite other packages files without declaring a Breaks+Replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/installs_over_existing_symlink.mail0000644000000000000000000000270613351442132022233 0ustar To: submit@bugs.debian.org Subject: directory vs. symlink conflict: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package installs files over an existing symlink shipped or created by another package. Your package ships: but package CONFLICTOR ships: Installing something over existing symlinks is considered bad practice. See e.g. https://lists.debian.org/87ehlevcrf.fsf@windlord.stanford.edu It may break in subtle ways and dpkg cannot detect this as a problem. * Your package might silently overwrite files installed at the symlink destination by other packages. * If the package shipping the symlink decides to make the link point somewhere else (or turn it into a real directory), the files owned by your package "will be lost" somewhere in the filesystem. * Depending on installation order the problematic path will be created either as a symlink or a directory: the package installed first will "win" and all others have "lost". Note that dpkg intentionally does not replace directories with symlinks and vice versa, see in particular the end of point 4 in https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-unpack-phase-of-installation-or-upgrade (Note: Adding Pre-Depends is *not* a solution.) Please move the files shipped in your package to the "real" location. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/leaves_alternatives_after_purge.mail0000644000000000000000000000315613351442132022312 0ustar To: submit@bugs.debian.org Subject: leaves alternatives after purge: Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging The leftover files are actually alternatives that were installed by the package but have not been properly removed. While there is ongoing discussion how to remove alternatives correctly (see https://bugs.debian.org/71621 for details) the following strategy should work for regular cases: * 'postinst configure' always installs the alternative * 'prerm remove' removes the alternative * 'postrm remove' and 'postrm disappear' remove the alternative In all other cases a maintainer script is invoked (e.g. upgrade, deconfigure) the alternatives are not modified to preserve user configuration. Removing the alternative in 'prerm remove' avoids having a dangling link once the actual file gets removed, but 'prerm remove' is not called in all cases (e.g. unpacked but not configured packages or disappearing packages) so the postrm must remove the alternative again (update-alternatives gracefully handles removal of non-existing alternatives). Note that the arguments for adding and removing alternatives differ, for removal it's 'update-alternatives --remove '. Filing this as important as having a piuparts clean archive is a release goal since lenny. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/leaves_diversions_after_upgrade.mail0000644000000000000000000000115613266762112022311 0ustar To: submit@bugs.debian.org Subject: leaves diversion after upgrade from squeeze Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package fails to remove some diversions after upgrading from squeeze and removing the package afterwards. Filing this as important as having a piuparts clean archive is a release goal since lenny. From the attached log (scroll to the bottom...): The test did the following: setup minimal squeeze chroot install $package/squeeze distupgrade wheezy remove $package purge $package cheers, piuparts-1.1.1/bug-templates/logrotate_exits_with_error_after_package_removal.mail0000644000000000000000000000124013416657733025741 0ustar To: submit@bugs.debian.org Subject: logrotate exits with error after package removal Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package's logrotate configuration causes logrotate to exit with an error after the package has been removed (*) or when logrote is run but no logfile exists. Usually the solution is to specify 'missingok' in the logrotate configuration. *) logrotate configuration files remain installed and executed after a package has been removed, they only get removed when the package is purged. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/maintainer_script_unknown_argument.mail0000644000000000000000000000102613351442132023055 0ustar To: submit@bugs.debian.org Subject: foobar called with unknown argument `foobar' Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed the maintainer scripts of your package don't support all the ways they are going to be called. See policy 6.5 at https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#summary-of-ways-maintainer-scripts-are-called From the attached log (scroll to the bottom...): Testing reinstallation in sid: Cheers, piuparts-1.1.1/bug-templates/modifies_conffiles.mail0000644000000000000000000000320513351442133017512 0ustar To: submit@bugs.debian.org Subject: modifies conffiles (policy 10.7.3): Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package modifies conffiles. This is forbidden by the policy, see https://www.debian.org/doc/debian-policy/ch-files.html#configuration-files 10.7.3: "[...] The easy way to achieve this behavior is to make the configuration file a conffile. [...] This implies that the default version will be part of the package distribution, and must not be modified by the maintainer scripts during installation (or at any other time)." Note that once a package ships a modified version of that conffile, dpkg will prompt the user for an action how to handle the upgrade of this modified conffile (that was not modified by the user). Further in 10.7.3: "[...] must not ask unnecessary questions (particularly during upgrades) [...]" If a configuration file is customized by a maintainer script after having asked some debconf questions, it may not be marked as a conffile. Instead a template could be installed in /usr/share and used by the postinst script to fill in the custom values and create (or update) the configuration file (preserving any user modifications!). This file must be removed during postrm purge. ucf(1) may help with these tasks. See also https://wiki.debian.org/DpkgConffileHandling In https://lists.debian.org/debian-devel/2012/09/msg00412.html and followups it has been agreed that these bugs are to be filed with severity serious. debsums reports modification of the following files, from the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/modifying_files_from_another_package.mail0000644000000000000000000000063613266762112023264 0ustar To: submit@bugs.debian.org Subject: modifying files from another package: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package modifies files from another package in /usr. This is so wrong, I'm not even bothered to look up the part of policy this violates ;-P From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/obsolete_conffiles.mail0000644000000000000000000000362613605426771017552 0ustar To: submit@bugs.debian.org Subject: obsolete conffiles (policy 10.7.3): Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package has obsolete conffiles. To see the policy statement, https://www.debian.org/doc/debian-policy/ch-files.html#behavior 10.7.3: "[...] Obsolete configuration files without local changes should be removed by the package during upgrade.[11]" [11] The dpkg-maintscript-helper tool, available from the dpkg package, can help for this task. Quotes from manpage: " [...] If the conffile has not been shipped for several versions, and you are now modifying the maintainer scripts to clean up the obsolete file, prior-version should be based on the version of the package that you are now preparing, not the first version of the package that lacked the conffile. This applies to all other actions in the same way. For example, for a conffile removed in version 2.0-1 of a package, prior-version should be set to 2.0-1~. This will cause the conffile to be removed even if the user rebuilt the previous version 1.0-1 as 1.0-1local1. Or a package switching a path from a symlink (shipped in version 1.0-1) to a directory (shipped in version 2.0-1), but only performing the actual switch in the maintainer scripts in version 3.0-1, should set prior-version to 3.0-1~. [...] " The debian/deb_package.maintscript file should look like this: rm_conffile /path/to/the/obsolete_conffile X.X-X~ Please read the manpage for details. Attached you will find a compressed log file. To find the line in the log file look for "OBSOLETE CONFFILE". Here are the line(s) in package_version_log_file: OBSOLETE CONFFILE /path/to/file REGISTERED BY srptools See also https://wiki.debian.org/DpkgConffileHandling There may be false positives, e.g. if a conffile was converted to a maintainer script managed configuration file. cheers, piuparts-1.1.1/bug-templates/owned_after_purge_policy_6.8_+_10.7.3.mail0000644000000000000000000000106213351442133022353 0ustar To: submit@bugs.debian.org Subject: owned files after purge (policy 6.8 + 10.7.3): Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left owned files on the system after purge, which is a violation of policy 6.8 and 10.7.3: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging https://www.debian.org/doc/debian-policy/ch-files.html#behavior From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/owned_and_unowned_files_after_purge_policy_6.8_+_10.7.3.mail0000644000000000000000000000111213351442133026112 0ustar To: submit@bugs.debian.org Subject: owned and unowned files after purge (policy 6.8 + 10.7.3): Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left owned and unowned files on the system after purge, which is a violation of policy 6.8 and 10.7.3: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging https://www.debian.org/doc/debian-policy/ch-files.html#behavior From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/package_removed_processes_still_running.mail0000644000000000000000000000077513266762112024054 0ustar To: submit@bugs.debian.org Subject: package removed, processes still running Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left processes running after the package has been removed and/or purged. In https://lists.debian.org/debian-devel/2009/08/msg00182.html and followups it has been agreed that these bugs are to be filed with severity serious. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/partial-upgrade-file-overwrite.mail0000644000000000000000000000312113351442133021702 0ustar To: submit@bugs.debian.org Subject: FOO: missing Breaks+Replaces: BAR (<< VBAR) Package: FOO Version: VFOO Severity: serious User: treinen@debian.org Usertags: edos-file-overwrite Architecture: amd64 Distribution: squeeze->wheezy (partial) upgrade Hi, automatic installation tests of packages that share a file and at the same time do not conflict by their package dependency relationships has detected the following problem: This is a serious bug as it makes installation/upgrade fail, and violates sections 7.6.1 and 10.1 of the policy. As this problem can be demonstrated during partial upgrades from squeeze to wheezy (but not within squeeze or wheezy itself), this indicates a missing or insufficiently versioned Replaces+Breaks relationship. But since this particular upgrade ordering is not forbidden by any dependency relationship, it is possible that apt (or $PACKAGE_MANAGER) will use this erroneus path on squeeze->wheezy upgrades. Here is a list of files that are known to be shared by both packages (according to the Contents files for squeeze and wheezy on amd64, which may be slightly out of sync): The following relationships are currently defined: Package: Conflicts: n/a Breaks: n/a Replaces: n/a The following relationships should be added for a clean takeover of these files (https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces): Package: Breaks: Replaces: Cheers, PS: for more information about the detection of file overwrite errors of this kind see https://qa.debian.org/dose/file-overwrites.html piuparts-1.1.1/bug-templates/postinst_uses_usr_share_doc.mail0000644000000000000000000000141013351442133021501 0ustar To: submit@bugs.debian.org Subject: postinst uses /usr/share/doc content (Policy 12.3): Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, a test with piuparts revealed that your package uses files from /usr/share/doc in its maintainer scripts which is a violation of Policy 12.3: "Packages must not require the existence of any files in /usr/share/doc/ in order to function." https://www.debian.org/doc/debian-policy/ch-docs.html#additional-documentation These files must be moved to /usr/share/$PACKAGE and may be symlinked from /usr/share/doc/$PACKAGE. This piuparts test prevents the installation of (most) files into /usr/share/doc with 'dpkg --path-exclude=...'. From the attached log (scroll to the bottom...): Cheers, ././@LongLink0000644000000000000000000000015200000000000007771 Lustar piuparts-1.1.1/bug-templates/prompting_due_to_modified_conffiles_which_were_not_modified_by_the_user.mailpiuparts-1.1.1/bug-templates/prompting_due_to_modified_conffiles_which_were_not_modified_by_the_user0000644000000000000000000000225613351442133031531 0ustar To: submit@bugs.debian.org Subject: prompting due to modified conffiles which were not modified by the user: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package failed the piuparts upgrade test because dpkg detected a conffile as being modified and then prompted the user for an action. As there is no user input, this fails. But this is not the real problem, the real problem is that this prompt shows up in the first place, as there was nobody modifying this conffile at all, the package has just been installed and upgraded... This is a violation of policy 10.7.3, see https://www.debian.org/doc/debian-policy/ch-files.html#behavior, which says "[These scripts handling conffiles] must not ask unnecessary questions (particularly during upgrades), and must otherwise be good citizens." https://wiki.debian.org/DpkgConffileHandling should help with figuring out how to do this properly. In https://lists.debian.org/debian-devel/2009/08/msg00675.html and followups it has been agreed that these bugs are to be filed with severity serious. From the attached log (scroll to the bottom...): cheers, ././@LongLink0000644000000000000000000000015700000000000007776 Lustar piuparts-1.1.1/bug-templates/prompts_user_without_following_Debian_Configuration_Management_Specification.mailpiuparts-1.1.1/bug-templates/prompts_user_without_following_Debian_Configuration_Management_Specific0000644000000000000000000000312513351442133031442 0ustar To: submit@bugs.debian.org Subject: prompts user without following Debian Configuration Management Specification Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package prompts the user badly. Prompting in maintainer scripts must be done by communicating through a program such as debconf which conforms to the Debian Configuration Management Specification, version 2 or higher. Refer to Debian Policy Manual section 3.9.1 (Prompting in maintainer scripts) for details. Quoting from https://www.debian.org/doc/debian-policy/ch-binary.html#prompting-in-maintainer-scripts ------------------------------------------------------------------------------ [ 3.9.1 Prompting in maintainer scripts ] Package maintainer scripts may prompt the user if necessary. Prompting must be done by communicating through a program, such as debconf, which conforms to the Debian Configuration Management Specification, version 2 or higher. Packages which are essential, or which are dependencies of essential packages, may fall back on another prompting method if no such interface is available when they are executed. The Debian Configuration Management Specification is included in the debconf_specification files in the debian-policy package. ------------------------------------------------------------------------------ From the attached log (scroll to the bottom...): And as it's not possible to set two usertags at the same time, this also should to be done: User: lintian-maint@debian.org Usertags: read-in-maintainer-script cheers, piuparts-1.1.1/bug-templates/removal_makes_files_disappear.mail0000644000000000000000000000220613351442133021722 0ustar To: submit@bugs.debian.org Subject: removal of $OFFENDER makes files disappear from $VICTIM Package: $OFFENDER Affects: $VICTIM Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts replaces-without-breaks Hi, during a test with piuparts and DOSE tools I noticed your package causes removal of files that also belong to another package. This is caused by using Replaces without corresponding Breaks. The installation sequence to reproduce this problem is apt-get install $VICTIM # (1) apt-get install $OFFENDER apt-get remove $OFFENDER # (2) The list of installed files at points (1) and (2) should be identical, but the following files have disappeared: This is a serious bug violating policy 7.6, see https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces and also see the footnote that describes this incorrect behavior: https://www.debian.org/doc/debian-policy/ch-relationships.html#id13 The $OFFENDER package has the following relationships with $VICTIM: Conflicts: Breaks: Replaces: From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/removes_files_installed_by_other_package.mail0000644000000000000000000000222113351442133024127 0ustar To: submit@bugs.debian.org Subject: removes files that were installed by another package: Package: Affects: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package removes files that were installed by another package. The removed files were already present before the package was installed, they may have been shipped or created by a dependency. This could be a violation of policy 10.7.4 ("Sharing configuration files"), see https://www.debian.org/doc/debian-policy/ch-files.html#sharing-configuration-files or policy chapter 6 ("Package maintainer scripts..."), see https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html or policy 7.6 ("Overwriting files..."), see https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces If a directory is used by several packages, all should ship it as part of the package (possibly empty, using $package.dirs to create it), and no package should mkdir/rmdir it in the maintainer scripts as dpkg will take care of this. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/unhandled_symlink_to_directory_conversion.mail0000644000000000000000000000324313403304304024423 0ustar To: submit@bugs.debian.org Subject: unhandled symlink to directory conversion: /usr/share/doc/PACKAGE Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, an upgrade test with piuparts revealed that your package installs files over existing symlinks and possibly overwrites files owned by other packages. This usually means an old version of the package shipped a symlink but that was later replaced by a real (and non-empty) directory. This kind of overwriting another package's files cannot be detected by dpkg. This was observed on the following upgrade paths: For /usr/share/doc/PACKAGE this may not be problematic as long as both packages are installed, ship byte-for-byte identical files and are upgraded in lockstep. But once one of the involved packages gets removed, the other one will lose its documentation files, too, including the copyright file, which is a violation of Policy 12.5: https://www.debian.org/doc/debian-policy/ch-docs.html#copyright-information For other overwritten locations anything interesting may happen. Note that dpkg intentionally does not replace directories with symlinks and vice versa, you need the maintainer scripts to do this. See in particular the end of point 4 in https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-unpack-phase-of-installation-or-upgrade It is recommended to use the dpkg-maintscript-helper commands 'dir_to_symlink' and 'symlink_to_dir' (available since dpkg 1.17.14) to perform the conversion, ideally using d/$PACKAGE.maintscript. See dpkg-maintscript-helper(1) and dh_installdeb(1) for details. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/uninstallable_in_sid.mail0000644000000000000000000000044713266762112020061 0ustar To: submit@bugs.debian.org Subject: uninstallable in sid Package: Version: Severity: grave Justification: renders package unusable Tags: sid User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package is no longer installable in sid: Cheers, piuparts-1.1.1/bug-templates/unowned_directories_in_usr_local_after_purge_policy_6.8_and_9.1.2.mail0000644000000000000000000000125113351442133030374 0ustar To: submit@bugs.debian.org Subject: unowned directories in /usr/local after purge (policy 6.8 and 9.1.2) Package: Version: Severity: normal User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging From the attached log (scroll to the bottom...): Leaving directories in /usr/local after purge is also a violation of a should-directive in https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs cheers, piuparts-1.1.1/bug-templates/unowned_directory_after_purge.mail0000644000000000000000000000154713351442133022020 0ustar To: submit@bugs.debian.org Subject: unowned directory after purge: Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned directories on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging Filing this as important as having a piuparts clean archive is a release goal since lenny. The maintainer scripts create (and later remove) a file in that directory. Manual directory removal may be not appropriate as this directory is shared between several packages. If the package would ship this as an empty directory, dpkg would take care of the creation and removal (if it's empty). From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/unowned_files_after_purge_policy_6.8_and_10.8.mail0000644000000000000000000000111713351442133024351 0ustar To: submit@bugs.debian.org Subject: unowned files after purge (policy 6.8, 10.8): Package: Version: Severity: important User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8 (or 10.8): https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging Filing this as important as having a piuparts clean archive is a release goal since lenny. From the attached log (scroll to the bottom...): cheers, piuparts-1.1.1/bug-templates/unowned_files_after_purge_policy_6.8_violating_FHS_policy_9.1_too.mail0000644000000000000000000000125613351442133030470 0ustar To: submit@bugs.debian.org Subject: unowned files after purge (policy 6.8) violating FHS (policy 9.1) too: Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging From the attached log (scroll to the bottom...): As putting files into /usr/local is also a violation of https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs I'm setting the severity to serious. cheers, piuparts-1.1.1/bug-templates/unowned_files_in_usr_local_after_purge_policy_6.8.mail0000644000000000000000000000123513351442133025613 0ustar To: submit@bugs.debian.org Subject: unowned files in /usr/local after purge (policy 6.8): Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging From the attached log (scroll to the bottom...): As putting files into /usr/local is also a violation of https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs I'm setting the severity to serious. cheers, piuparts-1.1.1/bug-templates/unowned_ls-R_file_in_usr_local_after_purge.mail0000644000000000000000000000262313351442133024355 0ustar To: submit@bugs.debian.org Subject: unowned file /usr/local/share/texmf/ls-R after purge (policy 6.8, 9.1.2) Package: Version: Severity: serious User: debian-qa@lists.debian.org Usertags: piuparts Hi, during a test with piuparts I noticed your package left unowned files on the system after purge, which is a violation of policy 6.8: https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging As putting files into /usr/local is also a violation of https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs I'm setting the severity to serious. From the attached log (scroll to the bottom...): 0m53.8s ERROR: FAIL: Package purging left files on system: /usr/local/share/texmf/ not owned /usr/local/share/texmf/ls-R not owned This problem is usually caused by running mktexlsr (or texhash) without path arguments from a maintainer script. The recommended solution is to switch to use dh_installtex and have this generate most (or perhaps even all) of the maintainer scripts content. Otherwise run mktexlsr with the tree as argument where the package installs its files, which is usually mktexlsr /usr/share/texmf Please have a look at the Debian-TeX-Policy (in the tex-common package) for the current practice of handling TeX packages. For further TeX packaging related questions contact debian-tex-maint@lists.debian.org cheers, piuparts-1.1.1/bug-templates/usertag_piuparts.mail0000644000000000000000000000017013266762112017271 0ustar To: control@bugs.debian.org Subject: usertag piuparts user debian-qa@lists.debian.org usertag 123456 + piuparts thanks piuparts-1.1.1/conf/0000755000000000000000000000000013605426771011207 5ustar piuparts-1.1.1/conf/crontab-master.in0000644000000000000000000000227613605426771014467 0ustar # m h dom mon dow (0|7=sun,1=mon) command # set PATH, variable substitution is not supported (PATH=$PATH:/my/bin) PATH=@sharedir@/piuparts/master:@sharedir@/piuparts:/usr/bin:/bin PYTHONPATH=@libdir@/python3/dist-packages # # cleanup $HTDOCS/daily.lock # @reboot master_cleanup # # generate reports twice or three times a day # (dinstall runs 1|7|13|19:52, so this is long after mirror pushes...) # 0 4-23/12 * * * master_shell_runner piuparts-analyze 0 2-23/3 * * * master_shell_runner detect_well_known_errors 0 0-23/3 * * * master_shell_runner piuparts-report 15 23 * * * generate_daily_report # # reschedule old logs twice a day # 0 3,15 * * * reschedule_oldest_logs # # monitor for problems several times day # - these may result in packages being retested # - if that's not the case, run them from within generate_daily_report # 30 2-23/3 * * * detect_network_issues 45 2-23/3 * * * detect_piuparts_issues 0 10,22 * * * detect_archive_issues 30 22 * * * report_untestable_packages 0 23 * * * report_stale_reserved_packages # # misc # 0 2 * * * prepare_backup 55 23 * * * gather_bts_stats 42 23 * * * rotate_master_logs # # data export for manpages.d.o # 5 5 * * * distill_alternatives_log piuparts-1.1.1/conf/crontab-slave.in0000644000000000000000000000112213605426771014273 0ustar # m h dom mon dow (0|7=sun,1=mon) command # set PATH, variable substitution is not supported (PATH=$PATH:/my/bin) PATH=@sharedir@/piuparts/slave:/usr/bin:/bin # # start slave_run automatically after reboots # #@reboot slave_cleanup ; slave_run #42 * * * * slave_cleanup ; ! slave_run >/dev/null 2>&1 || echo "piuparts-slave started by cron" # # monitor slave session every hour # 16 * * * * detect_slave_problems # # monitor for problems every six hours # 23 */6 * * * detect_leftover_processes # # monitor for cruft in the temporary directory once a day # 0 0 * * * detect_tmp_cruft piuparts-1.1.1/conf/distros.conf0000644000000000000000000001100513552054171013532 0ustar # # Omitted fields will be set to defaults and completely "missing" # [distribution] entries will automatically be generated as follows: # # [] # uri = # distribution = # components = # depends = # candidates = # target-release = # # These are the standard fields for sources.list entries: # uri, distribution, components # # A non-empty "target-release" will add a -t argument to apt-get: # apt-get -t ... # # The "depends" entry can be used to refer to the "parent distribution" # (e.g. stable) of a "partial distribution" (e.g. stable-backports). # These are resolved recursively and will be made available in the # sources.list file. # # The "candidates" entry can be used to build a (virtually) merged # Packages file from one or more partial distribution (and maybe a full # parent distribution). This is used for selecting (package,version) # tuples to be tested. No recursive lookup. [etch] uri = http://archive.debian.org/debian [lenny] uri = http://archive.debian.org/debian [squeeze] uri = http://archive.debian.org/debian [squeeze-lts] uri = http://archive.debian.org/debian depends = squeeze target-release = squeeze-lts [squeeze-backports] uri = http://archive.debian.org/debian-backports depends = squeeze target-release = squeeze-backports [squeeze-backports-sloppy] uri = http://archive.debian.org/debian-backports depends = squeeze squeeze-backports target-release = squeeze-backports-sloppy [wheezy] uri = http://archive.debian.org/debian [wheezy/updates] uri = http://archive.debian.org/debian-security depends = wheezy [wheezy-lts] uri = None depends = wheezy wheezy/updates candidates = wheezy wheezy/updates [wheezy-backports] uri = http://archive.debian.org/debian depends = wheezy wheezy/updates target-release = wheezy-backports [wheezy-backports-sloppy] uri = http://archive.debian.org/debian depends = wheezy wheezy-backports target-release = wheezy-backports-sloppy [jessie/updates] uri = http://security.debian.org depends = jessie [jessie-lts] uri = None depends = jessie jessie/updates candidates = jessie jessie/updates [jessie-backports] uri = http://archive.debian.org/debian depends = jessie jessie/updates target-release = jessie-backports [jessie-backports-sloppy] uri = http://archive.debian.org/debian depends = jessie jessie/updates jessie-backports target-release = jessie-backports-sloppy [stretch/updates] uri = http://security.debian.org depends = stretch [stretch-updates] depends = stretch [stretch-proposed-updates] depends = stretch [stretch-debug] uri = http://deb.debian.org/debian-debug depends = stretch # this is a full distro representing an upcoming (old-)stable point release [stretch-next] uri = None depends = stretch stretch/updates stretch-updates stretch-proposed-updates candidates = stretch stretch/updates stretch-proposed-updates [stretch-backports] depends = stretch stretch/updates target-release = stretch-backports [stretch-backports-sloppy] depends = stretch stretch/updates stretch-backports target-release = stretch-backports-sloppy [buster/updates] uri = http://security.debian.org depends = buster [buster-updates] depends = buster [buster-proposed-updates] depends = buster [buster-debug] uri = http://deb.debian.org/debian-debug depends = buster # this is a full distro representing an upcoming (old-)stable point release [buster-next] uri = None depends = buster buster/updates buster-updates buster-proposed-updates candidates = buster buster/updates buster-proposed-updates [buster-backports] depends = buster buster/updates target-release = buster-backports [buster-backports-sloppy] depends = buster buster/updates buster-backports target-release = buster-backports-sloppy [bullseye-proposed-updates] depends = bullseye [sid-debug] uri = http://deb.debian.org/debian-debug depends = sid [experimental] depends = sid target-release = experimental [experimental-debug] uri = http://deb.debian.org/debian-debug depends = experimental target-release = experimental # alias [oldoldstable] distribution = jessie # alias [oldstable] distribution = stretch # alias [stable] distribution = buster # alias [testing] distribution = bullseye # It's also possible to have "virtual" entries by setting uri to # the string "None". This allows e.g. to combine several partial # distributions. Such virtual distros can be used for standalone # piuparts runs, but not in master-slave setup (due to a missing # dependency on a full distro). [stretch-current] uri = None depends = stretch/updates stretch-updates piuparts-1.1.1/conf/piuparts-master.conf0000644000000000000000000000037113266771660015222 0ustar Alias /piuparts /var/lib/piuparts/htdocs Require all granted Options indexes IndexOptions FancyIndexing NameWidth=* AddType text/plain .log AddDefaultCharset utf-8 # vim:set syn=apache: piuparts-1.1.1/conf/piuparts-slave.sudoers0000644000000000000000000000051713266762112015572 0ustar # # copy this file to /etc/sudoers.d/piuparts-slave # # The piuparts slave needs to handle chroots. piupartss ALL = NOPASSWD: /usr/sbin/piuparts *, \ /bin/umount /srv/piuparts.debian.org/tmp/tmp*, \ /usr/bin/test -f /srv/piuparts.debian.org/tmp/tmp*, \ /usr/bin/rm -rf --one-file-system /srv/piuparts.debian.org/tmp/tmp* piuparts-1.1.1/conf/piuparts.conf.sample0000644000000000000000000000214413266771660015211 0ustar # # This is the configuration file for piuparts running in master-slave mode. # Usually it's placed in /etc/piuparts/piuparts.conf # # You MUST make sure that master-host, master-user, master-directory, and # mirror are set correctly. # [global] sections = sid mirror = http://deb.debian.org/debian master-host = localhost master-user = piupartsm piuparts-command = sudo piuparts --scriptsdir /etc/piuparts/scripts master-directory = /var/lib/piuparts/master slave-directory = /var/lib/piuparts/slave basetgz-directory = /var/cache/piuparts/basetgz output-directory = /var/lib/piuparts/htdocs tmpdir = /var/cache/piuparts/tmp doc-root = /piuparts/ idle-sleep = 300 max-tgz-age = 604800 max-reserved = 50 expire-old-days = 120 reschedule-old-days = 90 reschedule-old-count = 150 expire-fail-days = 45 reschedule-fail-days = 30 reschedule-fail-count = 25 [sid] precedence = 1 description = "Debian sid / main" piuparts-flags = --no-symlinks distro = sid # area = main # arch = amd64 upgrade-test-distros = debug = no ## another example: ## [s-p-u-i386] ## distro = stable-proposed-updates ## # area = main ## arch = i386 piuparts-1.1.1/custom-scripts/0000755000000000000000000000000013436204472013253 5ustar piuparts-1.1.1/custom-scripts/scripts/0000755000000000000000000000000013553616457014754 5ustar piuparts-1.1.1/custom-scripts/scripts/is_testable_uninstallable0000755000000000000000000002727513553616457022132 0ustar #!/bin/sh set -e # Return a non-zero error code to have piuparts skip testing an # uninstallable (or otherwise untestable) package. # Packages in main don't qualify for "external services out of our # control". They only qualify for "utterly broken" once on archive.d.o. # There are very few exceptions to this rule: # - usrmerge, which is one-way. # - localepurge, which seems hard to undo without a complete --reinstall. skip_test() { echo "Skipping piuparts test: $*" exit 1 } fail_test() { echo "Failing piuparts test: $*" exit 2 } skip_always() { skip_test "$@" } skip_distupgrade() { if [ "$PIUPARTS_TEST" = "distupgrade" ]; then skip_test "$@" fi } # use where a later version might fix the bug and enable dist-upgrade testing skip_distupgrade_from() { from_ver=$1 shift pkg=${PIUPARTS_OBJECTS%%=*} cand_ver=$(apt-cache policy "$pkg" | awk '{if ($1 == "Candidate:") {print $2}}') if [ "$from_ver" = "$cand_ver" ]; then skip_distupgrade "$@" fi } # * For uninstallable *broken* packages we usually run the normal # installation+purge tests (which will fail and get the package marked # as bugged) and only skip the distupgrade tests since there is no point # in upgrading something that could not be installed as a start. # Automatically marking the upgrade logs as bugged would also be more # difficult, since the bug is in some *old* version. # * For packages requiring *external resources* not available in piuparts # we usually skip the failing initial install, too, since that is # *not a bug* in the package. if [ "${PIUPARTS_DISTRIBUTION}" = "lenny" ]; then case ${PIUPARTS_OBJECTS%%=*} in blootbot) skip_distupgrade "#502753: blootbot: fails to install: prompts without using debconf" ;; distributed-net) skip_distupgrade "#655922: distributed-net: noninteractive install: postinst loops infinitely" ;; dnscache-run) skip_distupgrade "#664848: dnscache-run: breaks dns resolution in chroot" ;; libdbd-oracle-perl|\ ora2pg) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient-basic" ;; libpils0|\ libpils-dev|\ libstonith0|\ libstonith-dev) skip_distupgrade "#702055: ${PIUPARTS_OBJECTS%%=*}: transitional package uninstallable in lenny" ;; mhc) skip_distupgrade "#696231: mhc: fails to install: emacs-package-install failure" ;; multex-bin) skip_distupgrade "#668791: multex-bin: fails to install: The format file of 'multex' is NOT built successfully." ;; naist-jdic|\ naist-jdic-utf8|\ chasen) skip_distupgrade "#929614: naist-jdic: 0.4.3-4 (lenny) was overwritten by 1:0.4.3-4 (jessie) on archive.debian.org" ;; python-milter|\ spf-milter-python) skip_always "python-milter contains virus samples that may get blocked by the firewall" ;; runit-run) skip_distupgrade "#542157: runit-run: fails to install" ;; sun-java5-doc|\ sun-java6-doc) skip_distupgrade "#709967: ${PIUPARTS_OBJECTS%%=*}: fails to install: download location no longer available" ;; trn) skip_distupgrade "#688563: trn: fails to install: prompts without using debconf" ;; ttf-root-installer) skip_distupgrade "#678807: ttf-root-installer: fails to install: ftp:// download location disappeared" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "squeeze" ]; then case ${PIUPARTS_OBJECTS%%=*} in distributed-net) skip_distupgrade "#655922: distributed-net: noninteractive install: postinst loops infinitely" ;; firmware-b43-lpphy-installer) skip_distupgrade "#623703: firmware-b43-lpphy-installer: download location no longer available" ;; fossology|\ fossology-agents|\ fossology-agents-single|\ fossology-scheduler|\ fossology-web|\ fossology-web-single) # removed after squeeze skip_distupgrade "#595593: fossology: fails to install" ;; libdbd-oracle-perl|\ ora2pg) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient11.2-basic" ;; mini-buildd-bld) skip_distupgrade "#632955: time limit exceeded during install" fail_test "#632955: time limit exceeded during install" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "wheezy" ]; then case ${PIUPARTS_OBJECTS%%=*} in firefox-esr-dev|\ iceowl-l10n-bn-bd|\ iceowl-l10n-pa-in|\ iceowl-l10n-ta-lk|\ iceweasel-l10n-ak|\ iceweasel-l10n-be|\ iceweasel-l10n-csb|\ iceweasel-l10n-ku|\ iceweasel-l10n-lg|\ iceweasel-l10n-nso|\ iceweasel-l10n-ta-lk|\ iceweasel-l10n-zu|\ lightning-l10n-bn-bd|\ lightning-l10n-pa-in|\ lightning-l10n-ta-lk|\ thunderbird-dev|\ thunderbird-l10n-bn-bd|\ thunderbird-l10n-pa-in|\ thunderbird-l10n-ta-lk) skip_distupgrade "#928226: uninstallable cruft package" ;; firmware-b43-lpphy-installer) skip_distupgrade "#623703: firmware-b43-lpphy-installer: download location no longer available" ;; geoip-database-contrib) skip_distupgrade "#885441: geoip-database-contrib: download location no longer available" ;; libdbd-oracle-perl|\ ora2pg) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient11.2-basic" ;; nautilus-dropbox) skip_distupgrade "#770402: nautilus-dropbox: downloads awfully slow, hangs during installation" ;; xul-ext-compactheader) skip_distupgrade "#837387: xul-ext-compactheader: uninstallable in wheezy with icedove (>= 1:45)" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "jessie" ]; then case ${PIUPARTS_OBJECTS%%=*} in enigmail) skip_distupgrade_from "2:1.9.9-1~deb8u1" "#909000: enigmail 2.0 needed after thunderbird 60 upload" ;; firefox-esr-dev|\ icedove-dev|\ icedove-l10n-bn-bd|\ icedove-l10n-pa-in|\ icedove-l10n-ta-lk|\ iceowl-l10n-bn-bd|\ iceowl-l10n-pa-in|\ iceowl-l10n-ta-lk|\ iceweasel-dev|\ lightning-l10n-bn-bd|\ lightning-l10n-pa-in|\ lightning-l10n-ta-lk|\ thunderbird-dev|\ thunderbird-l10n-bn-bd|\ thunderbird-l10n-pa-in|\ thunderbird-l10n-ta-lk) skip_distupgrade "#928226: uninstallable cruft package" ;; fusionforge-plugin-mediawiki) skip_distupgrade "#929139: fusionforge-plugin-mediawiki: uninstallable cruft package" ;; geoip-database-contrib) skip_distupgrade "#885441: geoip-database-contrib: download location no longer available" ;; steam) skip_distupgrade_from "1.0.0.49-1" "#772598: steam: cannot preseed licence acceptance" ;; susv4) skip_distupgrade_from "7.20161013~deb8u1" "#900882: susv4: download location no longer available" ;; xul-ext-compactheader) skip_distupgrade_from "2.0.8-1" "#837387: xul-ext-compactheader: uninstallable in jessie with icedove (>= 1:45)" ;; xul-ext-sogo-connector) skip_distupgrade_from "31.0.0-1" "#909313: xul-ext-sogo-connector: not compatible with thunderbird 60" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "stretch" ]; then case ${PIUPARTS_OBJECTS%%=*} in geoip-database-contrib) skip_distupgrade "#885441: geoip-database-contrib: download location no longer available" ;; libdbd-oracle-perl|\ ora2pg) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; llvm-3.7-examples) skip_distupgrade "#929138: llvm-3.7-examples: uninstallable on !arm*" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "stretch-debug" ]; then case ${PIUPARTS_OBJECTS%%=*} in libdbd-oracle-perl-dbgsym) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "buster" ]; then case ${PIUPARTS_OBJECTS%%=*} in libdbd-oracle-perl) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "buster-debug" ]; then case ${PIUPARTS_OBJECTS%%=*} in libdbd-oracle-perl-dbgsym) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "sid" ]; then case ${PIUPARTS_OBJECTS%%=*} in libdbd-oracle-perl) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; esac fi if [ "${PIUPARTS_DISTRIBUTION}" = "sid-debug" ]; then case ${PIUPARTS_OBJECTS%%=*} in libdbd-oracle-perl-dbgsym) skip_always "libdbd-oracle-perl/contrib depends on third-party oracle-instantclient12.1-basic" ;; esac fi if [ "$(uname)" = "Linux" ]; then case ${PIUPARTS_OBJECTS%%=*} in console-setup-freebsd) skip_always "non-Linux package" ;; esac fi case ${PIUPARTS_OBJECTS%%=*}@${PIUPARTS_DISTRIBUTION} in acgvision-agent@squeeze) # removed after squeeze skip_distupgrade "#688231: modifies conffiles: /etc/acgvision/agent.ini" ;; arden@jessie) if [ "$(dpkg --print-architecture)" = "i386" ]; then skip_distupgrade "arch:all package no longer installable in stretch/i386" fi ;; bugzilla3@lenny) # fixed in squeeze, removed after squeeze, excluded from squeeze-lts skip_distupgrade "was excluded from squeeze-lts" ;; chef-solr@wheezy|\ chef-expander@wheezy|\ chef-server-api@wheezy) skip_distupgrade "#681573: chef-solr: fails to install if policy-rc.d forbids to start services" ;; cpushare@wheezy) # removed after wheezy skip_distupgrade "#689885: Ships a folder in /var/run or /var/lock" ;; dict-stardic@lenny) # removed after lenny skip_distupgrade "#523770: must use invoke-rc.d" ;; dms-core@jessie|\ dms@jessie|\ dms-dr@jessie|\ dms-wsgi@jessie) # fixed after jessie, not in stable after jessie skip_distupgrade '#852519: dms-core: fails to install: NZST FATAL: configuration file "/etc/postgresql/9.4/dms/postgresql.conf" contains errors' ;; docbookwiki@lenny) # removed after lenny skip_distupgrade "unconditionally uses mysql-client during purge" ;; drupal6-trans-ru@squeeze) # removed after squeeze skip_distupgrade "#689252: installs files in /usr/share/drupal6/profiles which is a symlink in the drupal package" ;; dynare-matlab@squeeze) skip_distupgrade "requires a matlab installation" ;; dynare-matlab@*|\ matlab-gdf@*|\ matlab-support@*) skip_always "requires a matlab installation" ;; firebird2.0-classic@lenny) # removed after lenny skip_distupgrade "#574220: package purge (after dependencies removal) fails" ;; gclcvs@squeeze) # removed after squeeze skip_distupgrade "modifies conffiles: /etc/default/gclcvs" ;; geordi@lenny) # removed after lenny skip_distupgrade "modifies conffiles: /etc/geordi/irc-config" ;; ldap2dns@lenny) skip_distupgrade "#671671: fails to install" ;; libpam-rsa@lenny) # removed after lenny skip_distupgrade "modifies conffiles: /etc/security/pam_rsa.conf" ;; localepurge@*) skip_distupgrade "requires a full --reinstall to restore disappeared files" ;; mailreader@lenny) # removed after lenny skip_distupgrade "modifies conffiles: /var/lib/mailreader/ads/debian/ad.cfg, /etc/mailreader/{main,server}.cfg" ;; nessusd@lenny|\ harden-remoteaudit@lenny) # removed after lenny skip_distupgrade "nessusd: modifies conffiles: /etc/nessus/nessusd.conf" ;; openvas-server@squeeze|\ harden-remoteaudit@squeeze|\ openvas-server-dev@squeeze) # removed after squeeze skip_distupgrade "#688379: openvas-server: modifies conffiles: /etc/openvas/openvasd.conf" ;; pdbv@lenny) # removed after lenny skip_distupgrade "modifies conffiles: /etc/pdbv/pdbvrc" ;; poker-web@squeeze) # removed after squeeze skip_distupgrade "#698533: modifies a shipped file: /usr/share/poker-web/currency_one.php" ;; python-poker-stats@squeeze) # removed after squeeze skip_distupgrade "#608646: cronjob exits with error after package removal" ;; sdic-eijiro@lenny) skip_distupgrade "requires a dictionary CD" ;; sdic-eijiro@*) skip_always "requires a dictionary CD" ;; usrmerge@*) skip_always "performs piuparts-unfriendy filesystem layout changes" ;; esac piuparts-1.1.1/custom-scripts/scripts/post_chroot_unpack_allow_unauthenticated0000755000000000000000000000246013463366315025246 0ustar #!/bin/sh if [ "$PIUPARTS_DISTRIBUTION" = "lenny" ]; then echo "Creating /etc/apt/apt.conf.d/unauthenticated-lenny ..." # apt/lenny does not like comments ... tee /etc/apt/apt.conf.d/unauthenticated-lenny </dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } # # Remove obsolete base packages from the reference chroot. # # These packages are part of a minimal chroot in release R, but no # longer exist in release R+1. # Package dependencies will cause removal of the obsolete packages # during a subset of upgrade paths. Since these packages cannot be # reinstalled in release R+1 ensure they are always gone from the # final reference chroot. # # Only while creating the reference chroot. test "$PIUPARTS_PHASE" = "" || exit 0 PURGE= mark_for_purge() { pkg="$1" # skip if the package is not installed is_installed "$pkg" || return 0 case ${PIUPARTS_OBJECTS%%=*} in $pkg) # keep it while testing it ;; *) PURGE="$PURGE $pkg" esac } if [ "$PIUPARTS_DISTRIBUTION" = "wheezy" ] ; then mark_for_purge libdb4.8 # gcc-4.4-base is part of the minimal squeeze chroot # but it no longer exists in wheezy mark_for_purge gcc-4.4-base fi if [ "$PIUPARTS_DISTRIBUTION" = "jessie" ] ; then mark_for_purge libdb5.1 fi if [ "$PIUPARTS_DISTRIBUTION" = "stretch" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "stretch-next" ] ; then mark_for_purge libprocps3 mark_for_purge sysvinit # pulled in during dist-upgrade of the reference chroot with --install-recommends # can cause problems if the tested upgrade path installs a different pinentry-* # because apt may fail to properly switch during removal (seen on e.g. # plasma-desktop, plasma-netbook, kdeutils, print-manager, science-chemistry) mark_for_purge pinentry-curses fi if [ "$PIUPARTS_DISTRIBUTION" = "buster" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "buster-next" ] ; then mark_for_purge libcryptsetup4 mark_for_purge libprocps6 fi if [ -n "$PURGE" ]; then echo "Removing packages from base system:$PURGE" apt-get -y remove --purge $PURGE fi piuparts-1.1.1/custom-scripts/scripts/post_distupgrade_exceptions0000755000000000000000000001054113553616457022524 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } # E: Could not perform immediate configuration on ... rm -fv /etc/apt/apt.conf.d/piuparts-disable-immediate-configure if [ "$PIUPARTS_DISTRIBUTION" = "squeeze" ]; then case ${PIUPARTS_OBJECTS%%=*} in linpopup) # package removed after lenny log_debug for file in /var/lib/linpopup/messages.dat do test ! -f "$file" || chmod -c o-w "$file" done ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "wheezy" ]; then # https://bugs.debian.org/687611 if [ -f /usr/share/keyrings/debian-archive-removed-keys.gpg~ ]; then echo "FIXING /usr/share/keyrings/debian-archive-removed-keys.gpg~" mv -v /usr/share/keyrings/debian-archive-removed-keys.gpg~ /usr/share/keyrings/debian-archive-removed-keys.gpg fi case ${PIUPARTS_OBJECTS%%=*} in kismet|\ tshark|\ wireshark|\ wireshark-common|\ wireshark-dbg|\ libcap2-bin) # libcap2-bin/wheezy is part of the minimal chroot and recommends libpam-cap # a conffile moved from libcap2-bin/squeeze to libpam-cap/wheezy log_debug apt-get install -yf libpam-cap ;; ogre-doc-nonfree) # #773059 - ogre-doc: unhandled symlink to directory conversion: /usr/share/doc/PACKAGE # package removed after lenny log_debug apt-get install -yf ogre-1.8-doc ;; phpgacl) # #682825 # package not in wheezy log_debug for dir in /usr/share/phpgacl/admin/templates_c do test ! -d "$dir" || chmod -c o-w "$dir" done ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "jessie" ]; then # base-files only upgrades pristine /etc/nsswitch.conf if ! grep -q ^gshadow: /etc/nsswitch.conf ; then echo "Adding gshadow line to /etc/nsswitch.conf" sed -i '/^shadow:/a gshadow: files' /etc/nsswitch.conf fi fi if [ "$PIUPARTS_DISTRIBUTION" = "stretch" ]; then # debianutils in jessie shipped /usr/share/man/sl/*/ with mode 0775 find /usr/share/man/sl -type d -perm /020 -exec chmod -c g-w {} + # fakeroot:i386 in jessie shipped /usr/share/man/**/ with mode 0775 (#826318) if is_installed fakeroot ; then find /usr/share/man -type d -perm /020 -exec chmod -c g-w {} + fi # git:i386 in jessie shipped /usr/share/locale/**/ with mode 0775 if is_installed git ; then find /usr/share/locale -type d -perm /020 -exec chmod -c g-w {} + fi fi if [ "$PIUPARTS_DISTRIBUTION_PREV" = "stretch" ]; then case ${PIUPARTS_OBJECTS%%=*} in design-desktop*) #850948: needrestart/stretch may hang during upgrade log_debug dpkg-divert --remove --rename /usr/lib/needrestart/apt-pinvoke ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "buster" ]; then # libc-bin only upgrades pristine /etc/nsswitch.conf if grep -q '^passwd:.*compat' /etc/nsswitch.conf ; then echo "Switching from compat to files in /etc/nsswitch.conf" sed -r -i '/^(passwd|group|shadow):/ s/compat/files/' /etc/nsswitch.conf fi # upgrading ca-certificates disables obsolete certificates and appends new certificates # normalize and sort the list to match fresh installations if [ -f "/etc/ca-certificates.conf" ]; then sed -rn '/^#/p' /etc/ca-certificates.conf > /etc/ca-certificates.conf.normalized sed -r '/^[#!]/d' /etc/ca-certificates.conf | sort >> /etc/ca-certificates.conf.normalized if ! cmp -s /etc/ca-certificates.conf.normalized /etc/ca-certificates.conf ; then echo "Normalized /etc/ca-certificates.conf" cp /etc/ca-certificates.conf.normalized /etc/ca-certificates.conf fi rm -f /etc/ca-certificates.conf.normalized fi # dpkg does not properly clean up directories getting empty and no longer shipped for dir in /etc/dbus-1/system.d /etc/dbus-1 do if [ -d "$dir" ]; then rmdir --ignore-fail-on-non-empty "$dir" test -d "$dir" || echo "removed empty directory '$dir'" fi done # policykit-1 in buster changes the permissions 0755 -> 0700 if [ -d /var/lib/polkit-1 ]; then chmod -c go-rx /var/lib/polkit-1 fi #920760, wontfix, libpam-modules: does not ensure that pam-auth-update gets called after the package was configured if ! grep -q mkhomedir /var/lib/pam/seen ; then echo "Running pam-auth-update..." pam-auth-update fi fi piuparts-1.1.1/custom-scripts/scripts/post_distupgrade_force-unsafe-io0000777000000000000000000000000013266762112030477 2post_setup_force-unsafe-ioustar piuparts-1.1.1/custom-scripts/scripts/post_distupgrade_squeeze-backports0000755000000000000000000000062213266762112024000 0ustar #!/bin/sh set -e test "$PIUPARTS_DISTRIBUTION" = "squeeze-backports" || exit 0 # apt-get -t squeeze-backports dist-upgrade may pull in too many # packages that are not co-installable in squeeze-backports # so maintain a list of packages in the sid base system # that are in squeeze-backports, too, and don't cause problems PKGS="" PKGS="$PKGS insserv" apt-get -y -t squeeze-backports install $PKGS piuparts-1.1.1/custom-scripts/scripts/post_distupgrade_zz_backports0000777000000000000000000000000013266771660027771 2post_setup_zz_backportsustar piuparts-1.1.1/custom-scripts/scripts/post_install_exceptions0000755000000000000000000000227313266771660021661 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } case ${PIUPARTS_OBJECTS%%=*} in apt-listbugs) log_debug # when installed apt-listbugs is run on installations / upgrades # and will cause them to fail due to prompting # if packages being installed currently have RC bugs. # so disable it here. dpkg-divert --rename /usr/sbin/apt-listbugs ln -svf /bin/true /usr/sbin/apt-listbugs ;; esac if is_installed dkms; then # remember for post_purge_exceptions touch /var/run/piuparts-stamp-dkms-installed # if dkms built a kernel module, there might be a # leftover 'sleep' process from progress reporting sleep 3 fi for pgconf in /etc/postgresql/*/main/postgresql.conf do test -f "$pgconf" && test -r "$pgconf" || continue port=$(awk '/^port/ { print $3 }' "$pgconf") if [ "-n" $port ] && [ "$port" != "5432" ] then echo "Non-default port in $pgconf" fi done piuparts-1.1.1/custom-scripts/scripts/post_purge_exceptions0000755000000000000000000000316713266771660021340 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in amd64-libs|amd64-libs-dev) # leaves a superfluous empty line after purge log_debug sed -i '3{/^$/d}' /etc/ld.so.conf ;; localepurge) case ${PIUPARTS_DISTRIBUTION} in lenny*|squeeze*) ;; *) # reinstall packages where files might have been dropped log_debug EXTRA="" apt-get -u --reinstall --fix-missing install $(dpkg -S LC_MESSAGES 'man/??/man' | cut -d: -f1 | tr ', ' '\n' | sort -u) $EXTRA ;; esac ;; logcheck|\ checksecurity) case "$PIUPARTS_DISTRIBUTION" in stretch) if [ -f /etc/logcheck/ignore.d.server/libsasl2-modules ]; then # logcheck chowns/chmods the /etc/logcheck tree log_debug chgrp -v root /etc/logcheck /etc/logcheck/ignore.d.server /etc/logcheck/ignore.d.server/libsasl2-modules chmod -v g-s,o+rx /etc/logcheck/ignore.d.server/ fi ;; esac ;; sasl2-bin|\ libapache2-mod-authn-sasl) case "$PIUPARTS_DISTRIBUTION" in stretch) # unowned directory in jessie may disappear after removal of owning packages in stretch if [ ! -d /usr/lib/sasl2 ]; then log_debug mkdir -v -p /usr/lib/sasl2 fi ;; esac ;; virtualbox-ext-pack) #802143: virtualbox-ext-pack leaves VBoxSVC processes running # these should terminate within 5 seconds if not used log_debug sleep 10 ;; esac if [ -f /var/run/piuparts-stamp-dkms-installed ]; then rm /var/run/piuparts-stamp-dkms-installed # if dkms built a kernel module, there might be a # leftover 'sleep' process from progress reporting sleep 3 fi piuparts-1.1.1/custom-scripts/scripts/post_remove_exceptions0000755000000000000000000000162213553616457021506 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } # cleanup from pre_remove_exceptions rm -fv /etc/apt/apt.conf.d/piuparts-allow-remove-essential case ${PIUPARTS_OBJECTS%%=*} in asclassic) case "$PIUPARTS_DISTRIBUTION" in lenny) # purging works in lenny ;; *) log_debug # postrm purge unconditionally removes some files that may be gone already sed -i 's/rm system.steprc menudefs.hook/rm -f system.steprc menudefs.hook/' /var/lib/dpkg/info/asclassic.postrm ;; esac ;; ibod|\ isdnactivecards|\ isdneurofile|\ isdnlog|\ isdnutils|\ isdnutils-xtools|\ isdnvbox|\ isdnvboxserver|\ INVALID) case "$PIUPARTS_DISTRIBUTION" in lenny|squeeze) #431855: isdnutils-base fails to purge with "There are still files in /etc/isdn/ that you may want to remove manually." log_debug rm -fv /etc/isdn/* esac ;; esac piuparts-1.1.1/custom-scripts/scripts/post_setup_dummy_users0000755000000000000000000000067613266771660021553 0ustar #!/bin/sh set -e sys_ids=$(seq 100 149) usr_ids=$(seq 1000 1010) case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; dpsyco*) # may kill processes of these users on the host if they exist in the chroot usr_ids= ;; esac for id in $sys_ids $usr_ids do getent group $id > /dev/null || groupadd --system -g $id dummy$id getent passwd $id > /dev/null || useradd --system -u $id -g 100 dummy$id done piuparts-1.1.1/custom-scripts/scripts/post_setup_forbid_home0000755000000000000000000000072413363064533021436 0ustar #!/bin/sh set -e case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; esac case $PIUPARTS_DISTRIBUTION in lenny|squeeze) exit 0 ;; esac if [ -d /home ]; then echo "Disabling /home" mv /home /home.distrib if dpkg-divert --help | grep -q -e --no-rename ; then dpkg-divert --no-rename /home else dpkg-divert /home fi echo "This is a dummy file to prevent creating directories in /home" > /home chmod 000 /home fi piuparts-1.1.1/custom-scripts/scripts/post_setup_force-unsafe-io0000755000000000000000000000066113266771660022153 0ustar #!/bin/sh set -e case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; esac if [ ! -f /etc/dpkg/dpkg.cfg.d/piuparts-force-unsafe-io ] then if dpkg --force-help | grep -q unsafe-io then echo "Enabling dpkg --force-unsafe-io." echo force-unsafe-io > /etc/dpkg/dpkg.cfg.d/piuparts-force-unsafe-io fi fi if [ ! -h /bin/sync ] then dpkg-divert --rename /bin/sync ln -sv true /bin/sync fi piuparts-1.1.1/custom-scripts/scripts/post_setup_minimize0000755000000000000000000000035513552054171020777 0ustar #!/bin/sh set -e # only while creating the tarball if [ "${PIUPARTS_OBJECTS%%=*}" = "TARBALL" ] then case $PIUPARTS_DISTRIBUTION in jessie) dpkg --purge gcc-4.8-base ;; stretch) dpkg --purge lsb-base tzdata ;; esac fi piuparts-1.1.1/custom-scripts/scripts/post_setup_squeeze-backports0000777000000000000000000000000013266762112031651 2post_distupgrade_squeeze-backportsustar piuparts-1.1.1/custom-scripts/scripts/post_setup_zz_backports0000755000000000000000000000117013266771660021700 0ustar #!/bin/sh set -e is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } PKGS="" if [ "$PIUPARTS_DISTRIBUTION" = "jessie-backports" ]; then # downgrading them from jessie-backports to jessie during removal # is problematic due to triggers ! is_installed systemd || PKGS="${PKGS} systemd" ! is_installed udev || PKGS="${PKGS} udev" fi if [ -n "$PKGS" ]; then apt-get -y -t "$PIUPARTS_DISTRIBUTION" install $PKGS fi piuparts-1.1.1/custom-scripts/scripts/post_test_exceptions0000755000000000000000000000173613266771660021175 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } # clean up from pre_test_exceptions case ${PIUPARTS_OBJECTS%%=*} in corekeeper) log_debug rm -fv /etc/sysctl.d/corekeeper.conf /etc/sysctl.d/corekeeper.conf.distrib dpkg-divert --remove --rename /etc/sysctl.d/corekeeper.conf rmdir /etc/sysctl.d 2>/dev/null || true ;; fai-nfsroot) log_debug rm -fv /.THIS_IS_THE_FAI_NFSROOT ;; freedombox-setup) log_debug if [ -e /etc/motd.orig ]; then if [ -L /etc/motd ]; then rm -v /etc/motd fi if [ ! -e /etc/motd ]; then mv -v /etc/motd.orig /etc/motd fi fi ;; ltsp-client-core|\ education-thin-client|\ ltsp-client) log_debug rm -fv /etc/ltsp_chroot ;; reseed) log_debug test ! -L /usr/sbin/reseed || rm /usr/sbin/reseed dpkg-divert --remove --rename /usr/sbin/reseed ;; roger-router-cli|\ ffgtk|\ roger-router) log_debug dpkg-divert --remove --rename /usr/sbin/lpadmin ;; esac piuparts-1.1.1/custom-scripts/scripts/pre_distupgrade_allow_unauthenticated0000755000000000000000000000431413553616457024530 0ustar #!/bin/sh set -e if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "squeeze" ]; then rm -fv /etc/apt/apt.conf.d/unauthenticated-lenny fi if [ -e /etc/apt/apt.conf.d/unauthenticated-lenny ]; then echo "FAIL: /etc/apt/apt.conf.d/unauthenticated-lenny exists" exit 1 fi ############################################################################ if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "wheezy" ]; then rm -fv /etc/apt/apt.conf.d/unauthenticated-squeeze fi if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "squeeze" ]; then echo "Creating /etc/apt/apt.conf.d/unauthenticated-squeeze ..." # apt/lenny does not like comments ... tee /etc/apt/apt.conf.d/unauthenticated-squeeze </dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } dbconfig_enable_purge() { log_debug echo "$1 $1${2+/$2}/dbconfig-remove boolean true" | debconf-set-selections echo "$1 $1${2+/$2}/purge boolean true" | debconf-set-selections } if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "squeeze" ]; then case ${PIUPARTS_OBJECTS%%=*} in crm114) #562946 log_debug echo "crm114 crm114/forceupgrade boolean true" | debconf-set-selections ;; slapd|\ slapd-dbg) # squeeze postinst wants to restore a backup from lenny that was never created log_debug dpkg --purge slapd slapd-dbg ;; esac # squeeze does not properly upgrade adduser.conf, so generate a new one if [ -f /etc/adduser.conf ]; then md5="$(md5sum /etc/adduser.conf | awk '{ print $1 }')" # 5b577c9cb18e4852fc7e45830d230ec1: adduser/lenny pristine # 28928669e36f1ab616dfda39af3d79a7: adduser/lenny + dpsyco-lib/lenny if [ "$md5" = "5b577c9cb18e4852fc7e45830d230ec1" ] || \ [ "$md5" = "28928669e36f1ab616dfda39af3d79a7" ] then rm -fv /etc/adduser.conf fi fi # prevent disappearance of /etc/udev/rules.d/ if is_installed libticables3; then log_debug touch /etc/udev/rules.d/.piuparts.dummy fi # isc-dhcp-server.postinst unconditionally overwrites conffile /etc/dhcp/dhcpd.conf if is_installed dhcp3-server; then log_debug rm -fv /etc/dhcp3/dhcpd.conf fi fi if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "wheezy" ]; then case ${PIUPARTS_OBJECTS%%=*} in cnews) # dpkg 1.16 does not like the bad cnews version number cr.g7-40.4 # cnews was removed after lenny log_debug dpkg --purge cnews ;; lcl-utils|\ lcl) # WORKSAROUND #696075: lcl-utils: directory vs. symlink mess after squeeze => wheezy upgrade: /etc/lazarus # the renaming was performed in the postinst of lcl-utils-0.9.30.4, # which was not the owner of the old /etc/lazarus, # therefore dpkg does never forget about the old conffiles log_debug if [ -d /etc/lazarus ] && [ ! -L /etc/lazarus ]; then mv -v /etc/lazarus /etc/lazarus.bak fi ;; nfsbooted) # sysv-rc fails migration to dependency based boot sequence if nfsbooted is installed # nfsbooted was removed after lenny log_debug dpkg --purge nfsbooted ;; pdns-backend-mysql) # WORKSAROUND #698562: upgrade from squeeze => wheezy fails if database has not been created before the upgrade log_debug dpkg --purge pdns-backend-mysql ;; pdns-backend-pgsql) # WORKSAROUND #698562: upgrade from squeeze => wheezy fails if database has not been created before the upgrade log_debug dpkg --purge pdns-backend-pgsql ;; esac # E: Could not perform immediate configuration on ... #645713, #677805, #706641 case ${PIUPARTS_OBJECTS%%=*} in bug-buddy|\ gnome|\ gnome-accessibility|\ gnome-core|\ gnome-dbg|\ gnome-desktop-environment|\ libgd-text-perl|\ open-cobol|\ python-sugar-toolkit-0.86|\ speechd-el|\ sucrose-0.86|\ sugar-browse-activity-0.86|\ sugar-chat-activity-0.86|\ sugar-read-activity-0.86) log_debug echo "Disabling APT::Immediate-Configure" echo 'APT::Immediate-Configure "No";' >> /etc/apt/apt.conf.d/piuparts-disable-immediate-configure ;; esac # WORKSAROUND #655969: lirc: prompting due to modified conffiles which where not modified by the user: /etc/lirc/hardware.conf if [ -f /etc/lirc/hardware.conf ]; then log_debug sed -i '/^DRIVER=/s/.*/DRIVER="UNCONFIGURED"/' /etc/lirc/hardware.conf fi fi if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "jessie" ]; then case ${PIUPARTS_OBJECTS%%=*} in *cyrus*) # something was not working properly during the upgrade from 2.2 to 2.4 if is_installed cyrus-common && [ ! -d /var/lib/cyrus/db ]; then echo "FIXING missing /var/lib/cyrus/db" cyrus-makedirs --cleansquat fi ;; debian-edu-archive-keyring|\ debian-ports-archive-keyring|\ emdebian-archive-keyring|\ emdebian-grip-server) # https://bugs.debian.org/687611 log_debug for keyring in /etc/apt/trusted.gpg.d/*.gpg~ do if [ -f "${keyring}" ]; then mv -v "${keyring}" "${keyring%~}" fi done ;; ekeyd-uds) # ekeyd-uds was removed after squeeze, maintainer scripts are incompatible with udev/jessie log_debug dpkg --purge ekeyd-uds ;; otrs2) #707075: cannot automatically upgrade from wheezy to jessie due to mysql storage engine change if is_installed mysql-server; then log_debug dbconfig_enable_purge otrs2 dpkg --purge otrs2 fi ;; esac #825650: libreoffice-common: prompting due to modified conffiles which were not modified by the user: /etc/libreoffice/sofficerc # migration did not work properly while upgrading from squeeze to wheezy, # gets messed up further while upgrading from wheezy to jessie, # eventually leading to problems while upgrading from jessie to stretch if is_installed openoffice.org-common; then if [ -f /etc/openoffice/sofficerc ] && [ -f /etc/libreoffice/sofficerc ]; then log_debug for file in sofficerc soffice.sh psprint.conf do if [ -f /etc/openoffice/$file ] && [ -f /etc/libreoffice/$file ]; then rm -v /etc/openoffice/$file fi done fi fi fi if [ "$PIUPARTS_DISTRIBUTION" = "stretch" ]; then case ${PIUPARTS_OBJECTS%%=*} in design-desktop*) #850948: needrestart/stretch may hang during upgrade log_debug dpkg-divert --rename /usr/lib/needrestart/apt-pinvoke ;; esac fi piuparts-1.1.1/custom-scripts/scripts/pre_distupgrade_foreign_architecture_i3860000755000000000000000000000135213266762112025077 0ustar #!/bin/sh set -e case "$PIUPARTS_DISTRIBUTION" in squeeze*) ;; *) exit 0 ;; esac case "$PIUPARTS_DISTRIBUTION_NEXT" in lenny*|squeeze*) exit 0 ;; esac test "$(dpkg --print-architecture)" = "amd64" || exit 0 go= case ${PIUPARTS_OBJECTS%%=*} in ia32-libs|ia32-libs-gtk) go=yes ;; lib32nss-mdns) go=yes ;; *) dpkg-query -s "ia32-libs" >/dev/null 2>&1 || exit 0 ;; esac case ${PIUPARTS_OBJECTS} in *=None) go=yes ;; esac case ${PIUPARTS_OBJECTS%%=*} in *wine*) go=yes ;; education-thin-client-server|education-workstation) go=yes ;; playonlinux) go=yes ;; esac test -n "$go" || exit 0 echo "Enabling foreign architecture i386 for $PIUPARTS_OBJECTS" apt-get update apt-get -y install apt dpkg dpkg --add-architecture i386 piuparts-1.1.1/custom-scripts/scripts/pre_distupgrade_zz_database-server0000755000000000000000000000662313553616457023745 0ustar #!/bin/sh set -e # Skip while creating the reference chroot. if [ "$PIUPARTS_PHASE" = "" ]; then exit 0 fi is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } retry_configure_pending() { echo "" echo "Upgrade didn't go smooth, trying dpkg --configure --pending" dpkg --configure --pending } CANDIDATES= CANDIDATES="$CANDIDATES default-mysql-server" #CANDIDATES="$CANDIDATES mysql-server" CANDIDATES="$CANDIDATES postgresql" case "$PIUPARTS_DISTRIBUTION_NEXT" in buster) : # no mysql-server in buster ;; *) CANDIDATES="$CANDIDATES mysql-server" ;; esac case "$PIUPARTS_DISTRIBUTION_NEXT" in wheezy) CANDIDATES="$CANDIDATES mysql-server-5.5" CANDIDATES="$CANDIDATES postgresql-9.1" ;; jessie) CANDIDATES="$CANDIDATES mysql-server-5.5" CANDIDATES="$CANDIDATES mariadb-server-10.0" CANDIDATES="$CANDIDATES postgresql-9.4" ;; stretch) CANDIDATES="$CANDIDATES mariadb-server-10.1" CANDIDATES="$CANDIDATES postgresql-9.6" ;; buster) CANDIDATES="$CANDIDATES mariadb-server-10.3" CANDIDATES="$CANDIDATES postgresql-11" ;; bullseye|sid) CANDIDATES="$CANDIDATES mariadb-server-10.3" CANDIDATES="$CANDIDATES postgresql-11" ;; esac PACKAGES= RETRY="false" # early upgrade runs into even more trouble for some packages ... case ${PIUPARTS_OBJECTS%%=*} in med-practice|audiolink) CANDIDATES="" ;; redmine) CANDIDATES="" ;; esac if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "stretch" ]; then case ${PIUPARTS_OBJECTS%%=*} in bacula-director-mysql|\ bacula-director-mysql-dbg) # (new) transitively recommended package mariadb-server-x.y gets # configured (and the server started) too late (after ourselves) RETRY="retry_configure_pending" ;; esac fi for pkg in $CANDIDATES do if is_installed "$pkg" then PACKAGES="$PACKAGES $pkg" fi done test ! -x /usr/bin/pg_lsclusters || pg_lsclusters did_apt_get_update= apt_get_update() { if [ -z "$did_apt_get_update" ] then apt-get update did_apt_get_update=true fi } if [ -n "$PACKAGES" ] then echo "Upgrading early: ${PACKAGES# }" apt_get_update apt-get -y install $PACKAGES || $RETRY fi upgrade_pg_cluster() { local from=$1 local to=$2 if ! is_installed postgresql-${from} then return 0 fi if ! is_installed postgresql-${to} then echo "" echo "Installing: postgresql-${to}" apt_get_update apt-get -y install postgresql-${to} fi if is_installed postgresql-${from} && is_installed postgresql-${to} then if [ "$3" = "--restart-first" ]; then echo "" echo "Restarting PostgreSQL (#882409)..." invoke-rc.d postgresql restart fi echo "" echo "Upgrading PostgreSQL Cluster from ${from} to ${to}" pg_dropcluster ${to} main --stop pg_upgradecluster -v ${to} ${from} main pg_dropcluster ${from} main fi } case "${PIUPARTS_DISTRIBUTION}:${PIUPARTS_DISTRIBUTION_NEXT}" in lenny:squeeze) upgrade_pg_cluster 8.3 8.4 ;; squeeze*:wheezy) upgrade_pg_cluster 8.4 9.1 ;; wheezy*:jessie) upgrade_pg_cluster 9.1 9.4 --restart-first ;; jessie*:stretch) upgrade_pg_cluster 9.4 9.6 ;; stretch*:buster) upgrade_pg_cluster 9.6 11 ;; buster*:bullseye) upgrade_pg_cluster 9.6 11 ;; buster:sid) upgrade_pg_cluster 9.6 11 ;; esac test ! -x /usr/bin/pg_lsclusters || pg_lsclusters piuparts-1.1.1/custom-scripts/scripts/pre_distupgrade_zz_upgrade_early0000755000000000000000000000157513266771660023520 0ustar #!/bin/sh set -e # Skip while creating the reference chroot. if [ "$PIUPARTS_PHASE" = "" ]; then exit 0 fi log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } # packages to upgrade early EARLY= if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "squeeze" ]; then if dpkg-query -s "octave3.0" >/dev/null 2>&1 then #696377 EARLY="$EARLY libblas3gf liblapack3gf" fi case ${PIUPARTS_OBJECTS%%=*} in libapt-rpm-pkg-dev) # libapt-rpm-pkg-dev no longer exists in squeeze # causing some packages to be "kept back" EARLY="$EARLY libreadline5-dev" ;; python-pydoctor) # work around #696376: "/usr/sbin/update-python-modules: /usr/bin/python: bad interpreter: No such file or directory" EARLY="$EARLY python-pydoctor" ;; esac fi if [ -n "$EARLY" ]; then log_debug echo "Upgrading early: ${EARLY# }" apt-get update apt-get -y install $EARLY fi piuparts-1.1.1/custom-scripts/scripts/pre_install_database-server0000755000000000000000000001776213553616457022363 0ustar #!/bin/sh set -e # Install a database server for packages that require one during postinst # but only recommend or suggest (or maybe not even this) the server # package to allow for remote db servers. MYSQL= POSTGRESQL= SQLITE3= FUSIONFORGE= CLIENT= INSTALL= case ${PIUPARTS_OBJECTS%%=*} in acidbase) MYSQL=yes ;; auth2db) MYSQL=yes ; CLIENT=yes ;; auth2db-common) MYSQL=yes ; CLIENT=yes ;; auth2db-frontend) MYSQL=yes ; CLIENT=yes ;; b2evolution) MYSQL=yes ;; bacula) POSTGRESQL=yes ;; bacula-director) POSTGRESQL=yes ;; bacula-director-dbgsym) POSTGRESQL=yes ;; bacula-director-mysql) MYSQL=yes ;; bacula-director-mysql-dbg) MYSQL=yes ;; bacula-director-pgsql) POSTGRESQL=yes ;; bacula-director-pgsql-dbg) POSTGRESQL=yes ;; bacula-server) POSTGRESQL=yes ;; bandwidthd-pgsql) POSTGRESQL=yes ;; bareos) POSTGRESQL=yes ;; bareos-database-common) POSTGRESQL=yes ;; bareos-database-common-dbgsym) POSTGRESQL=yes ;; bareos-database-mysql) POSTGRESQL=yes ; MYSQL=yes ;; bareos-database-postgresql) POSTGRESQL=yes ;; bareos-database-sqlite3) POSTGRESQL=yes ; SQLITE3=yes ;; bareos-database-tools) POSTGRESQL=yes ;; bareos-database-tools-dbgsym) POSTGRESQL=yes ;; bareos-director) POSTGRESQL=yes ;; bareos-director-dbgsym) POSTGRESQL=yes ;; bley) POSTGRESQL=yes ;; blootbot) MYSQL=yes ;; buddycloud-server) POSTGRESQL=yes ;; bugzilla3) MYSQL=yes ;; cacti) MYSQL=yes ;; cacti-cactid) MYSQL=yes ;; cacti-spine) MYSQL=yes ;; cacti-spine-dbgsym) MYSQL=yes ;; chado-utils) POSTGRESQL=yes ;; convirt) MYSQL=yes ;; docbookwiki) MYSQL=yes ;; dotclear) MYSQL=yes ;; dotlrn) POSTGRESQL=yes ;; drupal6) MYSQL=yes ;; drupal6-mod-*) MYSQL=yes ;; drupal6-thm-*) MYSQL=yes ;; drupal6-trans-ru) MYSQL=yes ;; drupal7) MYSQL=yes ;; drupal7-mod-*) MYSQL=yes ;; dtc-postfix-dovecot) MYSQL=yes ;; frontaccounting) MYSQL=yes ;; fossology-agents) POSTGRESQL=yes ;; fusionforge-plugin-*) FUSIONFORGE=yes ;; fusionforge-standard) POSTGRESQL=yes ;; fusioninventory-for-glpi) MYSQL=yes ;; gforge-db-postgresql) POSTGRESQL=yes ;; gforge-lists-mailman) POSTGRESQL=yes ;; gforge-plugin-extratabs) POSTGRESQL=yes ;; gforge-plugin-mediawiki) POSTGRESQL=yes ;; gforge-plugin-scmarch) POSTGRESQL=yes ;; gforge-plugin-scmcvs) POSTGRESQL=yes ;; gforge-plugin-scmdarcs) POSTGRESQL=yes ;; gforge-plugin-scmgit) POSTGRESQL=yes ;; gforge-plugin-scmhg) POSTGRESQL=yes ;; gforge-web-apache2) POSTGRESQL=yes ;; gforge-web-apache2-vhosts) POSTGRESQL=yes ;; glance) SQLITE3=yes ;; glance-api) SQLITE3=yes ;; glance-common) SQLITE3=yes ;; glance-registry) SQLITE3=yes ;; glpi) MYSQL=yes ;; gnuhealth-server) POSTGRESQL=yes ;; grr) MYSQL=yes ;; heat-api) SQLITE3=yes ;; heat-api-cfn) SQLITE3=yes ;; heat-api-cloudwatch) SQLITE3=yes ;; heat-common) SQLITE3=yes ;; heat-engine) SQLITE3=yes ;; icinga-idoutils) POSTGRESQL=yes ;; icinga-idoutils-dbgsym) POSTGRESQL=yes ;; icinga-phpapi) MYSQL=yes ;; icinga-web) POSTGRESQL=yes ;; icinga-web-config-icinga) POSTGRESQL=yes ;; icinga-web-config-icinga2-ido-mysql) MYSQL=yes ;; icinga-web-config-icinga2-ido-pgsql) POSTGRESQL=yes ;; icinga-web-pnp) POSTGRESQL=yes ;; icinga2-ido-mysql) MYSQL=yes ;; icinga2-ido-mysql-dbgsym) MYSQL=yes ;; icinga2-ido-pgsql) POSTGRESQL=yes ;; icinga2-ido-pgsql-dbgsym) POSTGRESQL=yes ;; jclicmoodle) POSTGRESQL=yes ;; jffnms) MYSQL=yes ;; kopano-core) MYSQL=yes ;; kopano-server) MYSQL=yes ;; kopano-server-dbgsym) MYSQL=yes ;; letodms) MYSQL=yes ;; letodms-webdav) MYSQL=yes ;; libchado-perl) POSTGRESQL=yes ;; libdspam7-dbg) POSTGRESQL=yes ;; libdspam7-drv-mysql) MYSQL=yes ;; libdspam7-drv-pgsql) POSTGRESQL=yes ;; mantis) MYSQL=yes ;; med-bio-dev) POSTGRESQL=yes ;; moodle) POSTGRESQL=yes ; MYSQL=yes ;; moodle-book) POSTGRESQL=yes ;; moodle-debian-edu-*) POSTGRESQL=yes ;; movabletype-opensource) SQLITE3=yes ;; movim) POSTGRESQL=yes ;; mtop) MYSQL=yes ;; mythtv-database) MYSQL=yes ;; nagvis) MYSQL=yes ;; ndoutils-common) MYSQL=yes ;; ndoutils-nagios3-mysql) MYSQL=yes ;; neutron-common) SQLITE3=yes ;; neutron-server) SQLITE3=yes ;; neutron-*-agent) SQLITE3=yes ;; nginx-naxsi-ui) MYSQL=yes ;; obs-api) MYSQL=yes ;; ocsinventory-reports) MYSQL=yes ;; ocsinventory-server) MYSQL=yes ;; openacs) POSTGRESQL=yes ;; opendmarc) MYSQL=yes ;; opendmarc-dbgsym) MYSQL=yes ;; openstack-dashboard) SQLITE3=yes ;; openstack-dashboard-apache) SQLITE3=yes ;; otrs) POSTGRESQL=yes ;; otrs2) POSTGRESQL=yes case "$PIUPARTS_DISTRIBUTION" in lenny*|squeeze*) MYSQL=yes ;; #707075 esac ;; pdns-backend-mysql) MYSQL=yes ;; pdns-backend-pgsql) POSTGRESQL=yes ;; phpbb3) MYSQL=yes ;; phpbb3-l10n) MYSQL=yes ;; phpgacl) MYSQL=yes ;; phpmyadmin) MYSQL=yes ;; phpwiki) MYSQL=yes ;; pinba-engine-mysql) MYSQL=yes ;; pinba-engine-mysql-5.5) MYSQL=yes ;; pinba-engine-mysql-5.6) MYSQL=yes ;; piwigo) MYSQL=yes ;; pnopaste) MYSQL=yes ;; poker-web) MYSQL=yes ;; postfix-policyd) MYSQL=yes ;; postfixadmin) MYSQL=yes ;; prelude-manager) MYSQL=yes ;; prelude-manager-dbgsym) MYSQL=yes ;; prewikka) MYSQL=yes ;; puppetdb) POSTGRESQL=yes ;; pybit-web) POSTGRESQL=yes ;; python-django-horizon) SQLITE3=yes ;; python-quantum) SQLITE3=yes ;; python-quantumclient) SQLITE3=yes ;; quantum-common) SQLITE3=yes ;; quantum-plugin-*) SQLITE3=yes ;; quantum-server) SQLITE3=yes ;; quantum-*-agent) SQLITE3=yes ;; redmine) MYSQL=yes ;; redmine-mysql) MYSQL=yes ;; redmine-pgsql) POSTGRESQL=yes ;; redmine-plugin-botsfilter) MYSQL=yes ;; redmine-plugin-recaptcha) MYSQL=yes ;; roundcube) MYSQL=yes ;; roundcube-core) MYSQL=yes ;; roundcube-plugins) MYSQL=yes ;; roundcube-plugins-extra) MYSQL=yes ;; rsyslog-mysql) MYSQL=yes ;; rsyslog-mysql-dbgsym) MYSQL=yes ;; rsyslog-pgsql) POSTGRESQL=yes ;; rsyslog-pgsql-dbgsym) POSTGRESQL=yes ;; scuttle) MYSQL=yes ;; semanticscuttle) MYSQL=yes ;; serendipity) MYSQL=yes ;; simba) MYSQL=yes ;; spotweb) MYSQL=yes ;; sshproxy-backend-mysql) MYSQL=yes ;; steam) MYSQL=yes ;; sympa) MYSQL=yes ;; tango-accesscontrol) MYSQL=yes ;; tango-accesscontrol-dbg) MYSQL=yes ;; tango-accesscontrol-dbgsym) MYSQL=yes ;; tango-db) MYSQL=yes ;; tango-db-dbg) MYSQL=yes ;; tango-db-dbgsym) MYSQL=yes ;; textpattern) MYSQL=yes ;; torrentflux) MYSQL=yes ;; tt-rss) MYSQL=yes ;; tuskar) SQLITE3=yes ;; tuskar-api) SQLITE3=yes ;; tuskar-common) SQLITE3=yes ;; tuskar-manager) SQLITE3=yes ;; typo3-dummy) MYSQL=yes ;; ukolovnik) MYSQL=yes ;; webcalendar) MYSQL=yes ;; webissues-server) MYSQL=yes ;; websimba) MYSQL=yes ;; wims-moodle) POSTGRESQL=yes ;; zabbix-frontend-php) MYSQL=yes ;; zabbix-proxy-mysql) MYSQL=yes ;; zabbix-proxy-pgsql) POSTGRESQL=yes ;; zabbix-server-mysql) MYSQL=yes ;; zabbix-server-pgsql) POSTGRESQL=yes ;; esac if [ "$FUSIONFORGE" = "yes" ]; then case "$PIUPARTS_DISTRIBUTION" in wheezy*|jessie|jessie/updates) POSTGRESQL=yes ;; *) INSTALL="$INSTALL fusionforge-db-local" ;; esac fi if [ "$MYSQL" = "yes" ]; then case "$PIUPARTS_DISTRIBUTION" in lenny*|squeeze*|wheezy*|jessie*) INSTALL="$INSTALL mysql-server ${CLIENT:+mysql-client}" ;; *) INSTALL="$INSTALL default-mysql-server ${CLIENT:+default-mysql-client}" ;; esac fi if [ "$POSTGRESQL" = "yes" ]; then INSTALL="$INSTALL postgresql" fi if [ "$SQLITE3" = "yes" ]; then INSTALL="$INSTALL sqlite3" fi if [ -n "$INSTALL" ]; then echo "Installing ${INSTALL# }..." apt-get -y install $INSTALL fi for pgconf in /etc/postgresql/*/main/postgresql.conf do test -f "$pgconf" && test -r "$pgconf" || continue port=$(awk '/^port/ { print $3 }' "$pgconf") if [ "-n" $port ] && [ "$port" != "5432" ] then echo "Non-default port in $pgconf" fi done exit 0 piuparts-1.1.1/custom-scripts/scripts/pre_install_exceptions0000755000000000000000000001543713553616457021471 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in file-rc) case "$PIUPARTS_DISTRIBUTION" in lenny) # force installation and removal of essential package sysv-rc log_debug echo 'Yes, do as I say!' | apt-get -y --force-yes install file-rc ;; squeeze*|stretch*) # force installation and removal of essential package sysv-rc log_debug yes 'Yes, do as I say!' | apt-get -y --force-yes install file-rc ;; esac ;; sysvinit-core) # this will remove systemd-sysv log_debug apt-get -y install sysvinit-core ;; live-config-upstart|\ netscript-2.4-upstart|\ upstart) case "$PIUPARTS_DISTRIBUTION" in squeeze*|wheezy*) # force installation and removal of essential package sysvinit log_debug yes 'Yes, do as I say!' | apt-get -y --force-yes install upstart ;; esac ;; upstart-dconf-bridge|\ upstart-monitor) # switch init to upstart before installation apt-get install upstart ;; systemd-sysv) case "$PIUPARTS_DISTRIBUTION" in wheezy*) # force installation and removal of essential package sysvinit log_debug yes 'Yes, do as I say!' | apt-get -y --force-yes install systemd-sysv ;; esac ;; esac if [ "$PIUPARTS_DISTRIBUTION" = "jessie-backports" ]; then case ${PIUPARTS_OBJECTS%%=*} in libudev1) log_debug apt-get -y -t $PIUPARTS_DISTRIBUTION install udev systemd ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "wheezy-backports" ]; then case ${PIUPARTS_OBJECTS%%=*} in libreoffice-style-*) log_debug apt-get -y -t $PIUPARTS_DISTRIBUTION install ${PIUPARTS_OBJECTS%%=*} libreoffice-common- ;; esac fi # # the remaining exceptions are only for the initial package installation # if [ "$PIUPARTS_PHASE" != "install" ]; then exit 0 fi case ${PIUPARTS_OBJECTS%%=*} in auctex) # do not run the (long) latex macro indexing task in the background (#867020) log_debug echo "auctex auctex/doauto select Foreground" | debconf-set-selections ;; bugzilla3) # checksetup.pl goes into infinite loop asking for them log_debug echo "bugzilla3 bugzilla3/bugzilla_admin_name string bz@local.host" | debconf-set-selections echo "bugzilla3 bugzilla3/bugzilla_admin_real_name string Bz" | debconf-set-selections echo "bugzilla3 bugzilla3/bugzilla_admin_pwd password bzbzbzbz" | debconf-set-selections ;; msttcorefonts|\ ttf-mscorefonts-installer) # Recommends: ca-certificates log_debug apt-get -y install ca-certificates ;; samhain) # work around #749602 log_debug mkdir -p /var/state/samhain/ touch /var/state/samhain/samhain_file ;; esac if [ "$PIUPARTS_DISTRIBUTION" = "jessie" ]; then case ${PIUPARTS_OBJECTS%%=*} in libjs-protoaculous) #919917: yui-compressor: insufficient java dependency in jessie log_debug apt-get -y install default-jre-headless ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "wheezy" ]; then case ${PIUPARTS_OBJECTS%%=*} in ttf-root-installer) #774163: missing Depends: ca-certificates log_debug apt-get -y install ca-certificates ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "squeeze" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "squeeze-backports" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "lenny" ]; then case ${PIUPARTS_OBJECTS%%=*} in autopkgtest-xenlvm|\ clvm|\ cman|\ collectd|\ collectd-dbg|\ collectd-utils|\ dtc-xen|\ ganeti|\ ganeti2|\ gfs-tools|\ gfs2-tools|\ libcollectdclient0|\ libcollectdclient-dev|\ liblinux-lvm-perl|\ libsys-virt-perl|\ libvirt0|\ libvirt0-dbg|\ libvirt-dev|\ libvirt-ocaml|\ libvirt-ocaml-dev|\ libvirt-ruby|\ libvirt-ruby1.8|\ mozilla-virt-viewer|\ munin-libvirt-plugins|\ mylvmbackup|\ python-libvirt|\ redhat-cluster-suite|\ rgmanager|\ virtinst|\ virt-top|\ virt-viewer|\ xenwatch|\ lvm2) # work around lvm2 bug https://bugs.debian.org/603036 which is squeeze-ignore log_debug apt-get -y install udev ;; jhove|\ libbytecode-java|\ libcodemodel-java|\ libjavassist-java|\ libjdic-java|\ libjgroups-java|\ libjibx-java|\ libpj-java|\ libswingx-java) # these packages depend on sun-java* | java2-runtime # install default-jre to satisfy the java2-runtime dependency log_debug apt-get -y install default-jre ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "squeeze" ]; then case ${PIUPARTS_OBJECTS%%=*} in bootcd-ia64) if [ "$PIUPARTS_TEST" = "distupgrade" ] && [ "$(uname -m)" = "x86_64" ]; then #622690: bootcd-ia64 has a Pre-Depends/Depends cycle that apt cannot resolve log_debug apt-get -y install bootcd-i386 fi ;; mini-buildd-rep) #656746: mini-buildd-rep: fails to install: missing dependency on lvm2 log_debug apt-get -y install udev lvm2 ;; esac fi if [ "$PIUPARTS_DISTRIBUTION" = "lenny" ]; then # install undeclared dependencies case ${PIUPARTS_OBJECTS%%=*} in clamav-getfiles) #603082 log_debug apt-get -y install curl ;; cyrus-*-2.2|sa-learn-cyrus|dtc-cyrus) #694254 db4.2-util exists in lenny only and is needed for upgrades to wheezy log_debug apt-get -y install db4.2-util ;; debian-edu-artwork-usplash) # postinst fails without update-initramfs log_debug apt-get -y install initramfs-tools ;; gforge-shell-postgresql|gforge-plugin-scmcvs|gforge-plugin-scmsvn) #604218 log_debug apt-get -y install libcap2 ;; otrs2) #561889 log_debug apt-get -y install libapache2-mod-perl2 ;; sdic-gene95) #478592 log_debug apt-get -y install bzip2 wget ;; tftpd-hpa) #522780 log_debug apt-get -y install update-inetd ;; ttf-beteckna) #502707 log_debug apt-get -y install defoma ;; esac # prefer inn over cnews (bad version number cr.g7-40.4) case ${PIUPARTS_OBJECTS%%=*} in newsx|post-faq) log_debug apt-get -y install inn ;; esac # work around postinst failures case ${PIUPARTS_OBJECTS%%=*} in ion3) # annoying debconf prompt log_debug echo "ion3 ion3/acknowledge-maybe-outdated boolean true" | debconf-set-selections echo "ion3 ion3/acknowledge-outdated boolean true" | debconf-set-selections ;; ipppd|ibod|isdnutils) #542156 if [ ! -e /dev/MAKEDEV ]; then log_debug ln -sfv /sbin/MAKEDEV /dev/MAKEDEV fi ;; esac # allow lenny kernel image installation in chroot for flavor in amd64 openvz-amd64 vserver-amd64 xen-amd64 486 686 686-bigmem openvz-686 vserver-686 vserver-686-bigmem xen-686 do echo "linux-image-2.6.26-2-$flavor linux-image-2.6.26-2-$flavor/preinst/bootloader-initrd-2.6.26-2-$flavor boolean false" echo "linux-image-2.6.26-2-$flavor linux-image-2.6.26-2-$flavor/preinst/lilo-initrd-2.6.26-2-$flavor boolean false" done | debconf-set-selections # deterministic /bin/sh on upgrades echo "dash dash/sh boolean true" | debconf-set-selections fi piuparts-1.1.1/custom-scripts/scripts/pre_install_extras0000755000000000000000000000465713266771660020617 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts extra for package $PIUPARTS_OBJECTS" } # # these extras are only for the initial package installation # if [ "$PIUPARTS_PHASE" != "install" ]; then exit 0 fi case ${PIUPARTS_OBJECTS%%=*} in *dkms|\ ktap|\ zfs-*|\ zfsutils-linux) # Install kernel headers, so that dkms tries to build a module HEADERS=linux-headers FLAVOR=unknown FLAVOR_i386=686-pae case $PIUPARTS_DISTRIBUTION in lenny*|squeeze*) HEADERS=linux-headers-2.6 FLAVOR_i386=686 ;; esac case $(dpkg --print-architecture) in amd64) FLAVOR=amd64 ;; i386) FLAVOR=$FLAVOR_i386 ;; esac case ${PIUPARTS_OBJECTS} in *=None) # do not test discontinued packages - they won't support newer kernels ;; *) log_debug apt-get -y install $HEADERS-$FLAVOR ;; esac ;; firmware-ipw2x00) log_debug #echo "firmware-ipw2x00 firmware-ipw2x00/license/accepted boolean true" | debconf-set-selections ;; firmware-ivtv) log_debug #echo "firmware-ivtv firmware-ivtv/license/accepted boolean true" | debconf-set-selections ;; fonts-mathematica) log_debug #echo "fonts-mathematica mathematica-fonts/accept_license boolean true" | debconf-set-selections ;; mathematica-fonts|ttf-mathematica4.1) log_debug apt-get -y install wget #echo "mathematica-fonts mathematica-fonts/accept_license boolean true" | debconf-set-selections if [ "$PIUPARTS_DISTRIBUTION" = "lenny" ]; then log_debug #echo "ttf-mathematica4.1 ttf-mathematica4.1/accept_license boolean true" | debconf-set-selections fi ;; steam|steam-devices|steamcmd) log_debug #echo "steam steam/question select I AGREE" | debconf-set-selections ;; zekr-quran-translations-en) log_debug #echo "zekr-quran-translations-en quran-translations-en/accepted-Zekr-Quran-Trans boolean true" | debconf-set-selections ;; esac if [ "$PIUPARTS_DISTRIBUTION" = "squeeze" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "lenny" ]; then case ${PIUPARTS_OBJECTS%%=*} in *sun-java5-*|*sun-java6-*) log_debug #echo "sun-java6-jre shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections ;; jajuk|\ liblucene2-java|\ libworldwind-java|\ omegat|\ robocode|\ solr-common|\ solr-tomcat5.5|\ weirdx|\ weka|\ worldwind) # these packages depend on sun-java* only log_debug #echo "sun-java6-jre shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections ;; esac fi piuparts-1.1.1/custom-scripts/scripts/pre_install_foreign_architecture0000755000000000000000000000072313266771660023472 0ustar #!/bin/sh set -e test "$PIUPARTS_PHASE" = "install" || exit 0 case "$PIUPARTS_DISTRIBUTION" in lenny*|squeeze*|wheezy*) exit 0 ;; esac FOREIGN= case ${PIUPARTS_OBJECTS%%=*} in nvidia-driver|\ nvidia-legacy-*xx-driver|\ lib*cuda1) FOREIGN="i386" ;; esac if [ -n "$FOREIGN" ] && [ "$FOREIGN" != "$(dpkg --print-architecture)" ]; then echo "Enabling foreign architecture $FOREIGN for $PIUPARTS_OBJECTS" dpkg --add-architecture $FOREIGN apt-get update fi piuparts-1.1.1/custom-scripts/scripts/pre_install_foreign_architecture_i3860000755000000000000000000000076013266762112024234 0ustar #!/bin/sh set -e test "$PIUPARTS_PHASE" = "install" || exit 0 case "$PIUPARTS_DISTRIBUTION" in lenny*|squeeze*) exit 0 ;; esac test "$(dpkg --print-architecture)" = "amd64" || exit 0 case ${PIUPARTS_OBJECTS%%=*} in ia32-libs|ia32-libs-gtk) ;; libwine-unstable|libwine-*-unstable|wine|wine-unstable) ;; boinc-nvidia-cuda|teamspeak-client) ;; lib32nss-mdns) ;; *) exit 0 ;; esac echo "Enabling foreign architecture i386 for $PIUPARTS_OBJECTS" dpkg --add-architecture i386 apt-get update piuparts-1.1.1/custom-scripts/scripts/pre_remove_40_find_missing_md5sums0000755000000000000000000000317213363064533023550 0ustar #!/bin/sh set -e is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } # skip the md5sum check if /usr/share/doc is pruned test ! -f /etc/dpkg/dpkg.cfg.d/piuparts-path-exclude || exit 0 for pkg in ${PIUPARTS_OBJECTS%%=*} do # skip check if the package is not installed is_installed "$pkg" || continue md5file="/var/lib/dpkg/info/$pkg.md5sums" test -f "$md5file" || md5file="/var/lib/dpkg/info/$pkg:$(dpkg --print-architecture).md5sums" if [ ! -f "$md5file" ]; then echo "MD5SUM FILE NOT FOUND FOR $pkg" continue fi f1=/var/run/f1.$$ sed -r 's%^[0-9a-f]{32} %/%' "$md5file" | sort >$f1 f2=/var/run/f2.$$ >$f2 dpkg -L "$pkg" | sed 's,\\,\\\\,g' | sort | \ while read f ; do if [ "$f" != "${f#package diverts others to: }" ]; then : # skip diversion information elif [ "$f" != "${f#locally diverted to: }" ]; then : # skip diversion information elif [ -d "$f" ]; then : # ignore directories elif [ "$f" = "/home" ]; then : # ignore directory shipped by base-files that is # turned into a file by our post_setup_forbid_home script elif [ -L "$f" ]; then : # ignore links elif [ -z "${f%%/etc/*}" ]; then : # ignore files in /etc - probably conffiles elif [ ! -e "$f" ]; then echo "${pkg}: MISSING OBJECT $f" else echo "$f" >> $f2 fi done comm -13 $f1 $f2 | sed "s/^/${pkg}: FILE WITHOUT MD5SUM /" comm -23 $f1 $f2 | sed "s/^/${pkg}: MD5SUM WITHOUT FILE /" rm -f $f1 $f2 done piuparts-1.1.1/custom-scripts/scripts/pre_remove_40_find_obsolete_conffiles0000755000000000000000000000115013266771660024270 0ustar #!/bin/sh set -e for pkg in ${PIUPARTS_OBJECTS%%=*} do dpkg-query -W -f='${Conffiles}\n' $pkg | \ grep ' obsolete$' | \ while read file md5expected obs do info="OBSOLETE CONFFILE $file REGISTERED BY $pkg" query=$(dpkg-query -S "$file" || echo "???: $file") owner=${query%: ${file}} if [ "${owner%:*}" != "$pkg" ]; then info="${info} OWNER CHANGED TO $owner" fi if [ ! -f "$file" ]; then info="${info} (MISSING)" else md5=$(md5sum "$file" | awk '{ print $1 }') if [ "$md5expected" != "$md5" ]; then info="${info} (MODIFIED)" fi fi echo "$info" done done piuparts-1.1.1/custom-scripts/scripts/pre_remove_40_find_unowned_lib_links0000755000000000000000000000247413463366315024137 0ustar #!/bin/sh set -e for prefix in / /usr/ do for lib in "${prefix}lib" "${prefix}lib32" "${prefix}lib64" "${prefix}libx32" do test -d "$lib" || continue test -L "$lib" && continue for libdir in "$lib" "$lib"/*-gnu* do test -d "$libdir" || continue for f in "$libdir"/* do test -d "$f" && continue test -L "$f" || continue rl=$(readlink "$f") test -n "${rl##/etc/alternatives/*}" || continue owned=0 dpkg-query -S "$f" >/dev/null 2>&1 && owned=1 if [ "$owned" = "0" ] && [ -L "/lib" ] then dpkg-query -S "${f#/usr}" >/dev/null 2>&1 && owned=1 fi if [ "$owned" = "0" ] then case "$f" in /lib/ld-lsb.so.?|\ /lib/ld-lsb-*.so.?) # created by lsb-core continue ;; /usr/lib/libglide.so.2|\ /usr/lib/libglide3.so.3) # created by maintainer script continue ;; /usr/lib/libfmpich.so.12|\ /usr/lib/libmpi_cxx.so.20|\ /usr/lib/libmpich.so.12|\ /usr/lib/libmpichcxx.so.12|\ /usr/lib/libmpichf90.so.12|\ /usr/lib/libmpichfort.so.12) if [ -f /etc/alternatives/libmpi.so ] then # managed by mpi-default-dev via alternatives # to /usr/lib//lib${impl}.so # ldconfig adds the "real" soname links continue fi ;; esac echo "UNOWNED SYMLINK $f -> $rl" fi done done done done piuparts-1.1.1/custom-scripts/scripts/pre_remove_50_find_bad_permissions0000755000000000000000000000377313553616457023624 0ustar #!/bin/sh set -e case "$PIUPARTS_DISTRIBUTION" in lenny) case ${PIUPARTS_OBJECTS%%=*} in linpopup) # package removed after lenny for file in /var/lib/linpopup/messages.dat do test ! -f "$file" || chmod -c o-w "$file" done ;; esac ;; esac case "$PIUPARTS_DISTRIBUTION" in lenny|squeeze*) case ${PIUPARTS_OBJECTS%%=*} in citadel-server|citadel-dbg|citadel-mta|citadel-suite|bcron-run|capisuite|debbugs|raccess4vbox3|smartlist|sxid) #WORKSAROUND #684964: citadel-server: world writable config file: /etc/citadel/netconfigs/7 for file in /etc/citadel/netconfigs/7 /etc/citadel/refcount_adjustments.dat /etc/citadel/citadel.control do test ! -f "$file" || chmod -c o-w "$file" done ;; esac ;; esac case "$PIUPARTS_DISTRIBUTION" in wheezy*) case ${PIUPARTS_OBJECTS%%=*} in citadel-server|citadel-dbg|citadel-mta|citadel-suite|bcron|bcron-run|capisuite|debbugs|exmh|nmh|raccess4vbox3|smartlist|xlbiff) #WORKSAROUND #684964: citadel-server: world writable config file: /etc/citadel/netconfigs/7 for file in /etc/citadel/netconfigs/7 /etc/citadel/refcount_adjustments.dat /var/lib/citadel/data/refcount_adjustments.dat do test ! -f "$file" || chmod -c o-w "$file" done ;; ilisp) #WORKSAROUND #682826: ilisp: creates world writable directory /usr/lib/ilisp # package was not in wheezy for dir in /usr/lib/ilisp do test ! -d "$dir" || chmod -c o-w "$dir" done ;; esac ;; esac case ${PIUPARTS_OBJECTS%%=*} in gpe-tetris|gpe) #WORKSAROUND #684178: gpe-tetris: creates world writable directory /var/games/gpe # package removed after wheezy for file in /var/games/gpe/gpe-tetris.dat do test ! -f "$file" || chmod -c o-w "$file" done for dir in /var/games/gpe do test ! -d "$dir" || chmod -c o-w "$dir" done ;; esac # find world writables without sticky bit BADPERMS=$(find / -mount ! -type l ! -type c ! -type p ! -type s -perm -o+w ! -perm -1000) if [ -n "$BADPERMS" ]; then echo "ERROR: BAD PERMISSIONS" ls -lad $BADPERMS exit 1 fi piuparts-1.1.1/custom-scripts/scripts/pre_remove_50_find_missing_copyright0000755000000000000000000000252213266771660024172 0ustar #!/bin/sh set -e is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } failed= for pkg in ${PIUPARTS_OBJECTS%%=*} do # ignore failures for some old packages with many rdepends ignore= case "${pkg}_${PIUPARTS_DISTRIBUTION}" in gij_lenny) ignore=1 ;; gnumeric-common_lenny) ignore=1 ;; #554201 libuim6_lenny) ignore=1 ;; #554204 libuim-data_lenny) ignore=1 ;; #554200 mozilla-plugin-vlc_lenny) ignore=1 ;; #687657 postgresql-8.3-plsh_lenny) ignore=1 ;; # removed after lenny vlc_lenny) ignore=1 ;; #687657 cdd-common_squeeze) ignore=1 ;; #692946 libfbclient2_squeeze) ignore=1 ;; #692948 libcucul0_wheezy) ignore=1 ;; # removed esac # skip check if the package is not installed is_installed "$pkg" || continue docdir="/usr/share/doc/${pkg%:*}" copyright="$docdir/copyright" if [ ! -f "$copyright" ] then if [ -n "$ignore" ]; then echo "ignoring failure of $pkg on $PIUPARTS_DISTRIBUTION" else failed="$failed $copyright" fi echo "MISSING COPYRIGHT FILE: $copyright" echo "# ls -lad $docdir" ls -lad "$docdir" || true echo "# ls -la $docdir/" ls -la "$docdir/" || true fi done piuparts-1.1.1/custom-scripts/scripts/pre_remove_exceptions0000755000000000000000000000430413553616457021307 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in apt-listbugs) # cleanup from post_install_exceptions log_debug rm -f /usr/sbin/apt-listbugs dpkg-divert --remove --rename /usr/sbin/apt-listbugs ;; file-rc) # removal won't work if sysv-rc isn't reinstalled case "$PIUPARTS_DISTRIBUTION" in lenny) log_debug echo 'Yes, do as I say!' | apt-get -y --force-yes install sysv-rc ;; squeeze*) log_debug yes 'Yes, do as I say!' | apt-get -y --force-yes install sysv-rc ;; esac ;; freebsd-buildutils) case "$PIUPARTS_DISTRIBUTION" in lenny|squeeze) # apt considers original-awk as essential log_debug apt-get -y remove freebsd-buildutils echo 'Yes, do as I say!' | apt-get -y --force-yes remove original-awk esac ;; original-awk) case "$PIUPARTS_DISTRIBUTION" in lenny|squeeze) # apt considers original-awk as essential log_debug echo 'Yes, do as I say!' | apt-get -y --force-yes remove original-awk esac ;; esac case "$PIUPARTS_DISTRIBUTION" in jessie*) # prevent failure of 'apt-get --reinstall install systemd-sysv systemd' # due to spurious dependency cycle (#237470) case ${PIUPARTS_OBJECTS%%=*} in file-rc|\ live-config|\ live-config-upstart|\ upstart) log_debug apt-get -y install systemd-sysv ;; esac ;; esac case "$PIUPARTS_DISTRIBUTION" in lenny*|squeeze*|wheezy*|jessie*) # --allow-remove-essential was added in apt 1.1 (stretch) ;; *) case ${PIUPARTS_OBJECTS%%=*} in init|\ education-thin-client|\ ltsp-client|\ ltsp-client-core|\ ltsp-client-core-dbgsym) log_debug # requires removal of essential packages if [ ! -f /etc/apt/apt.conf.d/piuparts-allow-remove-essential ] then echo "Enabling APT::Get::allow-remove-essential" echo 'APT::Get::allow-remove-essential "true";' >> /etc/apt/apt.conf.d/piuparts-allow-remove-essential fi ;; esac ;; esac # Allow removal of the kernel running on the host from the chroot. UNAME_R="$(uname -r)" echo "linux-image-$UNAME_R linux-image-$UNAME_R/prerm/removing-running-kernel-$UNAME_R boolean false" | debconf-set-selections piuparts-1.1.1/custom-scripts/scripts/pre_test_exceptions0000755000000000000000000000241313266771660020767 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in corekeeper) # installing corekeeper reconfigures /proc/sys/kernel/core_pattern log_debug echo "Disabling /etc/sysctl.d/corekeeper.conf" dpkg-divert --rename /etc/sysctl.d/corekeeper.conf mkdir -p /etc/sysctl.d echo "# disabled" > /etc/sysctl.d/corekeeper.conf ;; fai-nfsroot) # fai-nfsroot refuses installation unless this file exists log_debug echo "Touching /.THIS_IS_THE_FAI_NFSROOT" touch /.THIS_IS_THE_FAI_NFSROOT ;; freedombox-setup) # replaces /etc/motd by a symlink log_debug if [ -e /etc/motd ] && [ ! -L /etc/motd ] && [ ! -e /etc/motd.orig ]; then mv -v /etc/motd /etc/motd.orig fi ;; ltsp-client-core|\ education-thin-client|\ ltsp-client) # ltsp-client-core refuses installation unless this file exists log_debug echo "Touching /etc/ltsp_chroot" touch /etc/ltsp_chroot ;; reseed) # postinst runs reseed which accesses the net log_debug dpkg-divert --rename /usr/sbin/reseed ln -s /bin/true /usr/sbin/reseed ;; roger-router-cli|\ ffgtk|\ roger-router) # roger-router-cli.postinst tries to add a printer with lpadmin log_debug dpkg-divert --rename /usr/sbin/lpadmin ;; esac piuparts-1.1.1/custom-scripts/scripts/pre_test_root_password0000755000000000000000000000034113266762112021501 0ustar #!/bin/sh set -e # sudo refuses removal if no root password is set, so set one # do this unconditionally, as there are quite some packages depending on sudo # (and since its harmless and fast) yes "yes" 2>/dev/null | passwd piuparts-1.1.1/custom-scripts/scripts-apt-first/0000755000000000000000000000000013266762112016652 5ustar piuparts-1.1.1/custom-scripts/scripts-apt-first/pre_distupgrade_zz_apt_first0000755000000000000000000000007013266762112024554 0ustar #!/bin/sh set -e apt-get update apt-get -y install apt piuparts-1.1.1/custom-scripts/scripts-broken-symlinks/0000755000000000000000000000000013553616457020101 5ustar piuparts-1.1.1/custom-scripts/scripts-broken-symlinks/post_install_extras_unbreak_symlinks0000755000000000000000000004242713553616457027601 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } EXTRA= # if package $1 is installed, $2... should be installed as well install_extra_package() { local p q p=$1 shift if is_installed "$p" ; then for q in "$@" do if ! is_installed "$q" ; then EXTRA="$EXTRA $q" fi done fi } # /etc/xdg/menus/cinnamon-applications-merged -> applications-merged install_extra_package cinnamon-common extra-xdg-menus # /usr/bin/db_sql -> db5.3_sql # #686875 install_extra_package db-util db5.3-sql-util # /usr/bin/pykdeuic4-3.5 -> ../lib/python3/dist-packages/PyQt4/uic/pykdeuic4.py # python-kde4-dev: Depends: python-kde4 | python3-pykde4 install_extra_package python-kde4-dev python3-pykde4 # /usr/bin/vmware-user -> vmware-user-suid-wrapper # open-vm-tools Suggests: open-vm-tools-desktop install_extra_package open-vm-tools open-vm-tools-desktop # /usr/include/libreoffice/sal/typesizes.h -> ../..//libreoffice/sal/typesizes.h # /usr/lib/libreoffice/sdk/index.html -> ../../../share/doc/libreoffice/sdk/index.html # libreoffice-dev: Suggests: libreoffice-dev-doc install_extra_package libreoffice-dev-common libreoffice-dev libreoffice-dev-doc # /usr/lib/android-sdk/tools/proguard/lib/proguardgui.jar -> ../../../../../share/java/proguardgui.jar # /usr/lib/android-sdk/tools/proguard/bin/proguardgui -> ../../../../../bin/proguardgui # #856410 install_extra_package android-sdk proguard-gui # /usr/lib/codeaster/outils/xmgrace -> ../../../bin/xmgrace # /usr/lib/codeaster/outils/eficasQt -> ../../../bin/eficasQt # /usr/lib/codeaster/outils/eficas -> ../../../bin/eficas # /usr/lib/codeaster/outils/gmsh -> ../../../bin/gmsh # code-aster-gui: Suggests: eficas, gmsh, grace install_extra_package code-aster-gui eficas gmsh grace # /usr/lib/doublecmd/doc -> ../../share/doublecmd/doc # doublecmd-common: Suggests: doublecmd-help-en | doublecmd-help install_extra_package doublecmd-common doublecmd-help-en # /usr/lib/firefox/dictionaries -> ../../share/hunspell install_extra_package firefox hunspell-en-us # /usr/lib/python2.7/dist-packages/Bio/Entrez/DTDs/mathml2.dtd -> ../../../../../../share/xml/schema/w3c/mathml/dtd/mathml2.dtd # python-biopython: Suggests: w3-dtd-mathml install_extra_package python-biopython w3-dtd-mathml # /usr/lib/python2.7/dist-packages/sas/sasview/doc -> ../../../../../share/doc/sasview/html # python-sasview: Suggests: sasview-doc install_extra_package python-sasview sasview-doc # /usr/lib/python2.7/dist-packages/wx.pth -> ../../wx/python/wx.pth # python-wxversion, alternative set up by python-wxgtk3.0, #857757 install_extra_package python-wxversion python-wxgtk3.0 # /usr/lib/qflow/bin/netgen -> ../../../bin/netgen-lvs # qflow: Suggests: netgen-lvs install_extra_package qflow netgen-lvs # /usr/lib/ruby/vendor_ruby/psych.jar -> ../../../share/java/psych.jar # ruby-psych: Suggests: libpsych-java install_extra_package ruby-psych libpsych-java # /usr/lib/slepcdir/slepc3.10/-complex/share/slepc/datafiles -> ../../../../../../share/doc/libslepc3.10-dev-examples/datafiles # libslepc-complex3.10-dev: Suggests: libslepc3.10-dev-examples install_extra_package libslepc-complex3.10-dev libslepc3.10-dev-examples slepc3.10-doc # /usr/lib/slepcdir/slepc3.10/-real/share/slepc/datafiles -> ../../../../../../share/doc/libslepc3.10-dev-examples/datafiles # libslepc-real3.10-dev: Suggests: libslepc3.10-dev-examples install_extra_package libslepc-real3.10-dev libslepc3.10-dev-examples slepc3.10-doc # /usr/lib/wml/exec/wml_aux_weblint -> ../../../bin/weblint # /usr/lib/wml/exec/wml_aux_freetable -> ../../../bin/freetable # wml: Suggests: freetable, weblint-perl install_extra_package wml freetable weblint-perl # /usr/lib//dbus-1.0/debug-build/share/dbus-1/system.d -> ../../../../../../share/dbus-1/system.d install_extra_package dbus-1-dbg systemd # /usr/lib//firebird/3.0/fbtrace.conf -> /etc/firebird/3.0/fbtrace.conf # /usr/lib//firebird/3.0/databases.conf -> /etc/firebird/3.0/databases.conf # firebird3.0-server-core: Suggests: firebird3.0-server install_extra_package firebird3.0-server-core firebird3.0-server # /usr/lib//qt-default/qtchooser/default.conf -> ../../../../share/qtchooser/qt4-.conf install_extra_package libqtcore4 qtchooser # /usr/share/bacula-common/dbc_database.conf -> /etc/dbconfig-common/bacula-director-pgsql.conf install_extra_package bacula-common-pgsql bacula-director-pgsql install_extra_package bacula-common-mysql bacula-director-mysql install_extra_package bacula-common-sqlite3 bacula-director-sqlite3 # /usr/share/bug/distccmon-gnome/presubj -> ../../doc/distcc/reporting-bugs.txt # distccmon-gnome: Suggests: distcc install_extra_package distccmon-gnome distcc # /usr/share/citadel-webcit/static/icons -> ../../icons/gnome/24x24/mimetypes # citadel-webcit: Suggests: gnome-icon-theme install_extra_package citadel-webcit gnome-icon-theme # /usr/share/doc/apache2-data/changelog.Debian.gz -> ../apache2/changelog.Debian.gz # apache2: Depends: apache2-data install_extra_package apache2-data apache2 # /usr/share/doc/binutils-doc/ld -> ../binutils/ld # /usr/share/doc/binutils-doc/gprof -> ../binutils/gprof # /usr/share/doc/binutils-doc/gas -> ../binutils/gas # /usr/share/doc/binutils-doc/bfd -> ../binutils/bfd # binutils-doc: Suggests: binutils install_extra_package binutils-doc binutils # /usr/share/doc/exim4-dev/changelog.gz -> ../exim4-base/changelog.gz # /usr/share/doc/exim4-dev/README.Debian.gz -> ../exim4-base/README.Debian.gz install_extra_package exim4-dev exim4-base # /usr/share/doc/gimp-help-common/html -> ../../gimp/2.0/help install_extra_package gimp-help-common gimp-help-en # /usr/share/doc/keyboard-configuration/xorg.lst -> ../../X11/xkb/rules/xorg.lst install_extra_package keyboard-configuration xkb-data # /usr/share/doc/libboost1.62-doc/HTML/boost -> ../../../../include/boost # libboost1.XX-doc: Suggests: libboost1.XX-dev install_extra_package libboost1.62-doc libboost1.62-dev # /usr/share/doc/libembperl-perl/examples/forms/js/prototype.js -> ../../../../../javascript/prototype/prototype.js # libembperl-perl: Suggests: libjs-prototype install_extra_package libembperl-perl libjs-prototype # /usr/share/doc/libgraphicsmagick1-dev/images -> ../graphicsmagick/www/images # libgraphicsmagick1-dev: Suggests: graphicsmagick install_extra_package libgraphicsmagick1-dev graphicsmagick # /usr/share/doc/libjs-mocha/example/chai.js -> ../../../javascript/chai/chai.js # libjs-mocha: Suggests: libjs-chai install_extra_package libjs-mocha libjs-chai # /usr/share/doc/libjs-sprintf-js/examples/angular.min.js -> ../../../javascript/angular.js/angular.min.js # libjs-sprintf-js: Suggests: libjs-angularjs install_extra_package libjs-sprintf-js libjs-angularjs # /usr/share/doc/libpetsc3.10-dev-examples/docs -> ../petsc3.10-doc/docs (libpetsc3.10-dev-examples) # /usr/share/doc/libpetsc3.10-dev-examples/include -> ../petsc3.10-doc/include (libpetsc3.10-dev-examples) # libpetsc3.10-dev-examples: Suggests: petsc3.10-doc install_extra_package libpetsc3.10-dev-examples petsc3.10-doc # /usr/share/doc/libslepc3.10-dev-examples/examples/docs -> ../../slepc3.10-doc/docs # libslepc3.10-dev-examples: Suggests: slepc3.10-doc install_extra_package libslepc3.10-dev-examples slepc3.10-doc # /usr/share/doc/libvirt-doc/changelog.gz -> ../libvirt0/changelog.gz install_extra_package libvirt-doc libvirt0 # /usr/share/doc/mkvtoolnix/README.source -> ../quilt/README.source install_extra_package mkvtoolnix quilt nullmailer # /usr/share/doc/node-setimmediate/examples/chai.js -> ../../../javascript/chai/chai.js # /usr/share/doc/node-setimmediate/examples/mocha.css -> ../../../javascript/mocha/mocha.css # /usr/share/doc/node-setimmediate/examples/mocha.js -> ../../../javascript/mocha/mocha.js # node-setimmediate: Suggests: libjs-mocha, libjs-chai install_extra_package node-setimmediate libjs-chai libjs-mocha # /usr/share/doc/node-yajsml/examples/public/javascripts/lib/underscore.js -> ../../../../../../javascript/underscore/underscrore.js # node-yajsml: Suggests: libjs-underscore install_extra_package node-yajsml libjs-underscore # /usr/share/doc/python-ly/html -> ../python-ly-doc/html # python-ly: Suggests: python-ly-doc install_extra_package python-ly python-ly-doc install_extra_package python3-ly python-ly-doc # /usr/share/doc/python-statsmodels-doc/examples -> ../python-statsmodels/examples # python-statsmodels-doc: Suggests: python-statsmodels install_extra_package python-statsmodels-doc python-statsmodels # /usr/share/doc/scribus/COPYING -> copyright # /usr/share/scribus/dicts -> ../hyphen # scribus-data: Recommends: scribus (= ${binary:Version}) install_extra_package scribus-data scribus hyphen-en-us # /usr/share/doc/xen-utils-common/html -> ../xen/html # xen-utils-common: Suggests: xen-doc install_extra_package xen-utils-common xen-doc # /usr/share/eclipse/dropins/eclox/eclipse/plugins/org.gna.eclox.help_0.10.1/html/doxygen-doc -> ../../../../../../../doc/doxygen/html # doxygen: Suggests: doxygen-doc install_extra_package eclipse-eclox doxygen-doc # /usr/share/games/neverball/ttf/wqy-microhei.ttc -> ../../../fonts/truetype/wqy/wqy-microhei.ttc # neverball-common: Suggests: fonts-wqy-microhei install_extra_package neverball-common fonts-wqy-microhei # /usr/share/gnome/help/gnumeric/C/images -> ../../../../xml/docbook/stylesheet/nwalsh/images # gnumeric-doc: Suggests: docbook-xsl install_extra_package gnumeric-doc docbook-xsl # /usr/share/gtkmorph/pixmaps -> ../xmorph/pixmaps # /usr/share/man/man1/morph.1.gz -> xmorph.1.gz install_extra_package libmorph gtkmorph xmorph # /usr/share/icons/default.kde4 -> oxygen install_extra_package kde-runtime-data oxygen-icon-theme # /usr/share/jcadencii/WineMinimum.bundle/Contents/SharedSupport/bin/wine -> ../../../../../../bin/wine # jcadencii: Suggests: wine install_extra_package jcadencii wine # /usr/share/live/build/bootloaders/???linux/*.c32 -> /usr/lib/syslinux/modules/bios/*.c32 # /usr/share/live/build/bootloaders/pxelinux/pxelinux.0 -> /usr/lib/PXELINUX/pxelinux.0 # /usr/share/live/build/bootloaders/isolinux/isolinux.bin -> /usr/lib/ISOLINUX/isolinux.bin # live-build: the required packages will be installed in the live image (#857151) # open-infrastructure-system-build: alternate implementation of live-build install_extra_package live-build syslinux-common pxelinux isolinux install_extra_package open-infrastructure-system-build syslinux-common pxelinux isolinux # /usr/share/lyx/thes -> ../mythes install_extra_package lyx-common mythes-en-us # /usr/share/man/man3/libwildmagic-common.3.gz -> libwildmagic.3.gz # libwildmagic-common: Suggests: libwildmagic5 install_extra_package libwildmagic-common libwildmagic5 # /usr/share/nginx/modules -> ../../lib/nginx/modules install_extra_package nginx-common libnginx-mod-http-echo # /usr/share/org-mode/scripts/ditaa.jar -> ../../ditaa/ditaa.jar # org-mode: Suggests: ditaa install_extra_package org-mode ditaa # /usr/share/pyshared/MoinMoin/words -> /etc/moin/words # /etc/moin/words -> /usr/share/dict/words install_extra_package python-moinmoin wamerican # /usr/share/quassel/scripts/inxi -> ../../../bin/inxi # quassel-data: Suggests: inxi install_extra_package quassel-data inxi # /usr/share/quilt/compat/sendmail -> ../../../sbin/sendmail # quilt: Suggests: default-mta | mail-transport-agent install_extra_package quilt nullmailer # /usr/share/syslinux/themes/debian-wheezy/???linux/*.c32 install_extra_package syslinux-themes-debian-wheezy syslinux-common # /usr/share/tuxpaint/fonts/locale/zh_TW.ttf -> ../../../fonts/truetype/arphic/uming.ttc # /usr/share/tuxpaint/fonts/locale/zh_CN.ttf -> ../../../fonts/truetype/arphic-gbsn00lp/gbsn00lp.ttf # /usr/share/tuxpaint/fonts/locale/th.ttf -> ../../../fonts/opentype/tlwg/Garuda-Bold.otf # /usr/share/tuxpaint/fonts/locale/te.ttf -> ../../../fonts/truetype/fonts-telu-extra/Pothana2000.ttf # /usr/share/tuxpaint/fonts/locale/ko.ttf -> ../../../fonts/truetype/nanum/NanumGothic.ttf # /usr/share/tuxpaint/fonts/locale/gu.ttf -> ../../../fonts/truetype/lohit-gujarati/Lohit-Gujarati.ttf # /usr/share/tuxpaint/fonts/locale/ar.ttf -> ../../../fonts/truetype/fonts-arabeyes/ae_Nice.ttf # tuxpaint-data: Suggests: fonts-arabeyes, fonts-arphic-gbsn00lp, fonts-arphic-uming, fonts-nanum, fonts-gujr, fonts-taml, fonts-telu, fonts-tlwg-garuda-otf install_extra_package tuxpaint-data fonts-arabeyes fonts-arphic-gbsn00lp fonts-arphic-uming fonts-nanum fonts-gujr fonts-taml fonts-telu fonts-tlwg-garuda-otf # /var/lib/gridengine/lib/lx-* -> /usr/lib/gridengine install_extra_package gridengine-common gridengine-client db-util db5.3-sql-util if [ -n "$EXTRA" ]; then log_debug echo "Installing extra packages: ${EXTRA# }" apt-get -y install $EXTRA fi # if package $1 is installed, $2... should exist create_extra_file() { local p f p=$1 shift if is_installed "$p" ; then for f in "$@" do if [ ! -e $f ]; then echo "Creating empty target $f" touch $f fi done fi } # if package $1 is installed, $2... should exist create_extra_dir() { local p d p=$1 shift if is_installed "$p" ; then for d in "$@" do if [ ! -e $d ]; then echo "Creating empty directory $d" mkdir -p $d fi done fi } # /etc/issue.d/cockpit.issue -> /run/cockpit/motd # /etc/motd.d/cockpit -> /run/cockpit/motd create_extra_dir cockpit-ws /run/cockpit create_extra_file cockpit-ws /run/cockpit/motd # /etc/issue.linuxlogo -> /var/run/linuxlogo/issue.linuxlogo # /etc/issue.linuxlogo.ascii -> /var/run/linuxlogo/issue.linuxlogo.ascii create_extra_dir linuxlogo /var/run/linuxlogo create_extra_file linuxlogo /var/run/linuxlogo/issue.linuxlogo /var/run/linuxlogo/issue.linuxlogo.ascii # /etc/lprng/printcap -> ../printcap create_extra_file lprng /etc/printcap # /etc/openvswitch/.conf.db.~lock~ -> /var/lib/openvswitch/.conf.db.~lock~ # /etc/openvswitch/conf.db -> /var/lib/openvswitch/conf.db create_extra_file openvswitch-switch /var/lib/openvswitch/.conf.db.~lock~ /var/lib/openvswitch/conf.db # /etc/vdr/plugins/xineliboutput/config -> /var/lib/vdr/plugins/xineliboutput/config create_extra_file vdr-plugin-xineliboutput /var/lib/vdr/plugins/xineliboutput/config # /etc/vdr/remote.conf -> /var/lib/vdr/remote.conf # /etc/vdr/setup.conf -> /var/lib/vdr/setup.conf # /var/lib/vdr/commands.conf -> ../../cache/vdr/commands.conf # /var/lib/vdr/reccmds.conf -> ../../cache/vdr/reccmds.conf create_extra_file vdr /var/cache/vdr/commands.conf /var/cache/vdr/reccmds.conf /var/lib/vdr/remote.conf /var/lib/vdr/setup.conf # /usr/lib/mesa-diverted//libGLX_indirect.so.0 -> libGLX_mesa.so.0 create_extra_file glx-diversions /usr/lib/mesa-diverted/i386-linux-gnu/libGLX_mesa.so.0 create_extra_file glx-diversions /usr/lib/mesa-diverted/x86_64-linux-gnu/libGLX_mesa.so.0 # /usr/lib/plan/holiday -> /etc/plan/holiday create_extra_file plan /etc/plan/holiday # /usr/sbin/ganeti-masterd -> ../share/ganeti/default/ganeti-masterd # #857289 create_extra_file ganeti /usr/share/ganeti/default/ganeti-masterd # /usr/share/epiphany-browser/mime-types-permissions.xml -> /etc/gnome/epiphany/mime-types-permissions.xml create_extra_file epiphany-browser-data /etc/gnome/epiphany/mime-types-permissions.xml # /usr/share/games/xwelltris/welltris.scores -> /var/games/xwelltris/welltris.scores create_extra_file xwelltris /var/games/xwelltris/welltris.scores # /usr/share/gitlab-shell/config.yml -> /etc/gitlab-shell/config.yml create_extra_file gitlab-shell /etc/gitlab-shell/config.yml # /usr/share/man/man1/rust-gdb.1.gz -> gdb.1.gz # /usr/share/man/man1/i686-w64-mingw32-gdb.1.gz -> gdb.1.gz # /usr/share/man/man1/x86_64-w64-mingw32-gdb.1.gz -> gdb.1.gz # gdb-doc is in non-free create_extra_file rust-gdb /usr/share/man/man1/gdb.1.gz create_extra_file gdb-mingw-w64 /usr/share/man/man1/gdb.1.gz # /usr/share/notion/debian-menu.lua -> /var/lib/notion/debian-menu.lua # /usr/share/notion/debian-menu-i18n.lua -> /var/lib/notion/debian-menu-i18n.lua create_extra_file notion /var/lib/notion/debian-menu.lua /var/lib/notion/debian-menu-i18n.lua # /usr/share/otrs/Kernel/Config/GenericAgent.pm -> /etc/otrs/Kernel/Config/GenericAgent.pm # /var/lib/otrs/run -> /run/otrs create_extra_file otrs2 /etc/otrs/Kernel/Config/GenericAgent.pm create_extra_dir otrs2 /run/otrs # /usr/share/slrn/newsgroups.dsc -> /var/lib/slrn/newsgroups.dsc create_extra_file slrn /var/lib/slrn/newsgroups.dsc # /usr/share/texlive/texmf-dist/tex/latex/pdfx/sRGB_IEC61966-2-1_black_scaled.icc -> ../../../../../color/icc/sRGB_IEC61966-2-1_black_scaled.icc # sRGB_IEC61966-2-1_black_scaled.icc is provided by icc-profiles in non-free (#880363) create_extra_file texlive-latex-extra /usr/share/color/icc/sRGB_IEC61966-2-1_black_scaled.icc # /usr/share/uim/loader.scm -> /var/lib/uim/loader.scm # /usr/share/uim/installed-modules.scm -> /var/lib/uim/installed-modules.scm create_extra_file libuim-data /var/lib/uim/loader.scm /var/lib/uim/installed-modules.scm # /var/cache/vdr/dvd -> /dev/dvd create_extra_file vdr-plugin-dvd /dev/dvd # /usr/share/wordpress/wp-content/plugins/civicrm/civicrm/civicrm-wordpress.settings.php -> /etc/civicrm/civicrm-wordpress.settings.php create_extra_file wordpress-civicrm /etc/civicrm/civicrm-wordpress.settings.php piuparts-1.1.1/custom-scripts/scripts-broken-symlinks/post_setup_fake-essential_unbreak_symlinks0000755000000000000000000000052313266771660030646 0ustar #!/bin/sh set -e case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; stone) # openssl may be used during purge to compute the hash for a # certificate, otherwise files in /etc/ssl/certs can't be removed. echo "*** Adding fake essential packages ***" apt-get install -yf openssl ;; esac piuparts-1.1.1/custom-scripts/scripts-debug-packagemanager/0000755000000000000000000000000013266762112020753 5ustar piuparts-1.1.1/custom-scripts/scripts-debug-packagemanager/pre_distupgrade_debug_packagemanager0000755000000000000000000000034013266762112030233 0ustar #!/bin/sh set -e if [ ! -f /etc/apt/apt.conf.d/piuparts-debug-packagemanager ] then echo "Enabling Debug::pkgPackageManager" echo 'Debug::pkgPackageManager "true";' >> /etc/apt/apt.conf.d/piuparts-debug-packagemanager fi piuparts-1.1.1/custom-scripts/scripts-debug-problemresolver/0000755000000000000000000000000013266762112021247 5ustar piuparts-1.1.1/custom-scripts/scripts-debug-problemresolver/pre_distupgrade_debug_problemresolver0000755000000000000000000000052013266762112031023 0ustar #!/bin/sh set -e if [ ! -f /etc/apt/apt.conf.d/piuparts-debug-problemresolver ] then echo "Enabling Debug::pkgProblemResolver" echo 'Debug::pkgProblemResolver "true";' >> /etc/apt/apt.conf.d/piuparts-debug-problemresolver echo 'Debug::pkgProblemResolver::ShowScores "true";' >> /etc/apt/apt.conf.d/piuparts-debug-problemresolver fi piuparts-1.1.1/custom-scripts/scripts-debug-problemresolver/pre_install_debug_problemresolver0000755000000000000000000000057513266762112030170 0ustar #!/bin/sh set -e test "$PIUPARTS_TEST" = "install" || exit 0 if [ ! -f /etc/apt/apt.conf.d/piuparts-debug-problemresolver ] then echo "Enabling Debug::pkgProblemResolver" echo 'Debug::pkgProblemResolver "true";' >> /etc/apt/apt.conf.d/piuparts-debug-problemresolver echo 'Debug::pkgProblemResolver::ShowScores "true";' >> /etc/apt/apt.conf.d/piuparts-debug-problemresolver fi piuparts-1.1.1/custom-scripts/scripts-debug-purge/0000755000000000000000000000000013266771660017157 5ustar piuparts-1.1.1/custom-scripts/scripts-debug-purge/post_remove_postrm_set-x0000755000000000000000000000076213266771660024200 0ustar #!/bin/sh set -e set_minus_x() { local script=$1 if [ -f $script ]; then if head -n 1 $script | grep -qE '/bin/(ba)?sh' ; then echo "DEBUG PURGE: enabling 'set -x' in ${script##*/}" sed -i '2 i set -x' $script else echo "Unsupported script type in $script:" head -n 1 $script fi fi } arch=$(dpkg --print-architecture) for target in ${PIUPARTS_OBJECTS%%=*} do pkg=${target} set_minus_x /var/lib/dpkg/info/$pkg.postrm set_minus_x /var/lib/dpkg/info/$pkg:$arch.postrm done piuparts-1.1.1/custom-scripts/scripts-debug-remove/0000755000000000000000000000000013266771660017332 5ustar piuparts-1.1.1/custom-scripts/scripts-debug-remove/pre_remove_prerm_postrm_set-x0000755000000000000000000000111713266771660025354 0ustar #!/bin/sh set -e set_minus_x() { local script=$1 if [ -f $script ]; then if head -n 1 $script | grep -qE '/bin/(ba)?sh' ; then echo "DEBUG REMOVE: enabling 'set -x' in ${script##*/}" sed -i '2 i set -x' $script else echo "Unsupported script type in $script:" head -n 1 $script fi fi } arch=$(dpkg --print-architecture) for target in ${PIUPARTS_OBJECTS%%=*} do pkg=${target} set_minus_x /var/lib/dpkg/info/$pkg.prerm set_minus_x /var/lib/dpkg/info/$pkg.postrm set_minus_x /var/lib/dpkg/info/$pkg:$arch.prerm set_minus_x /var/lib/dpkg/info/$pkg:$arch.postrm done piuparts-1.1.1/custom-scripts/scripts-debug-triggers/0000755000000000000000000000000013553616457017664 5ustar piuparts-1.1.1/custom-scripts/scripts-debug-triggers/pre_distupgrade_debug_triggers0000755000000000000000000000027613553616457026054 0ustar #!/bin/sh set -e if [ ! -f /etc/dpkg/dpkg.cfg.d/piuparts-debug-triggers ] then echo "Enabling dpkg trigger debugging" echo 'debug=70441' > /etc/dpkg/dpkg.cfg.d/piuparts-debug-triggers fi piuparts-1.1.1/custom-scripts/scripts-debug-triggers/pre_install_debug_triggers0000755000000000000000000000035313553616457025203 0ustar #!/bin/sh set -e test "$PIUPARTS_TEST" = "install" || exit 0 if [ ! -f /etc/dpkg/dpkg.cfg.d/piuparts-debug-triggers ] then echo "Enabling dpkg trigger debugging" echo 'debug=70441' > /etc/dpkg/dpkg.cfg.d/piuparts-debug-triggers fi piuparts-1.1.1/custom-scripts/scripts-jessie/0000755000000000000000000000000013266771660016233 5ustar piuparts-1.1.1/custom-scripts/scripts-jessie/post_setup_jessie-fake-essential0000755000000000000000000000133513266771660024623 0ustar #!/bin/sh set -e # The following issues won't be fixed in jessie: # - unconditional use of ucf during postrm purge # so add these packages to the "fake" essential set. USED_DURING_PURGE="ucf" # For purging configuration from /var/lib/systemd/ INITSYSTEMHELPERS="init-system-helpers" case ${PIUPARTS_DISTRIBUTION} in lenny*|squeeze*|wheezy*) # package does not exist INITSYSTEMHELPERS="" ;; esac case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # don't install fake essential packages while creating the tarball exit 0 ;; ucf|init-system-helpers) # allow testing of the fake essential packages exit 0 ;; esac echo "*** Adding fake essential packages ***" apt-get install -yf $USED_DURING_PURGE $INITSYSTEMHELPERS piuparts-1.1.1/custom-scripts/scripts-leftovers/0000755000000000000000000000000013463366315016756 5ustar piuparts-1.1.1/custom-scripts/scripts-leftovers/post_purge_manual_cleanup0000755000000000000000000000102313266762112024127 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } remove_ssl_cert() { for c in /etc/ssl/certs/*.0 ; do if [ -L "$c" ] && [ "$(readlink "$c")" = "$1" ]; then rm -fv "$c" fi done rm -fv /etc/ssl/certs/$1 rmdir --ignore-fail-on-non-empty /etc/ssl/certs } case ${PIUPARTS_OBJECTS%%=*} in uw-imapd) log_debug remove_ssl_cert imapd.pem ;; ipopd) log_debug remove_ssl_cert ipop3d.pem ;; smartlist) log_debug rm -fv /etc/aliases.????-??-??.??:??:?? ;; esac piuparts-1.1.1/custom-scripts/scripts-leftovers/post_remove_cleanup0000755000000000000000000000043713266762112022755 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in dovecot-core) # #330519 - does not remove certificates on purge log_debug rm -fv /etc/dovecot/dovecot.pem /etc/dovecot/private/dovecot.pem ;; esac piuparts-1.1.1/custom-scripts/scripts-leftovers/post_setup_fake-essential0000755000000000000000000000157013266771660024073 0ustar #!/bin/sh set -e # If dbconfig-common was already purged, packages that have used # dbconfig-common in postinst will leave configuration files in # /etc/dbconfig-common DBCONFIG="dbconfig-common" # For purging configuration from /var/lib/systemd/ INITSYSTEMHELPERS="init-system-helpers" case ${PIUPARTS_DISTRIBUTION} in lenny*|squeeze*|wheezy*) # package does not exist INITSYSTEMHELPERS="" ;; esac # openssl may be used during purge to compute the hash for a # certificate, otherwise files in /etc/ssl/certs can't be removed. OPENSSL="" case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; dbconfig-common|init-system-helpers) # allow testing of the fake essential packages exit 0 ;; stone) OPENSSL="openssl" ;; esac echo "*** Adding fake essential packages ***" apt-get install -yf $DBCONFIG $INITSYSTEMHELPERS $OPENSSL piuparts-1.1.1/custom-scripts/scripts-leftovers/pre_remove_preseed_cleanup0000755000000000000000000000651113463366315024270 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } dbconfig_enable_purge() { log_debug echo "Enabling dbconfig purge for $1." echo "$1 $1${2+/$2}/dbconfig-remove boolean true" | debconf-set-selections echo "$1 $1${2+/$2}/purge boolean true" | debconf-set-selections } # # enable extended purge mode that is available in some packages # but disabled by default because it might remove valuable user or # application data # case ${PIUPARTS_OBJECTS%%=*} in cvsd) log_debug echo "cvsd cvsd/remove_chroot boolean true" | debconf-set-selections ;; docbookwiki) log_debug echo "docbookwiki docbookwiki/purge_books boolean true" | debconf-set-selections ;; grub-pc) log_debug echo "grub-pc grub-pc/postrm_purge_boot_grub boolean true" | debconf-set-selections ;; ifetch-tools) log_debug echo "ifetch-tools ifetch-tools/purge boolean true" | debconf-set-selections ;; mlmmj) #668752 log_debug echo "mlmmj mlmmj/remove-on-purge boolean true" | debconf-set-selections ;; pluxml) log_debug echo "pluxml pluxml/system/purgedata boolean true" | debconf-set-selections ;; sympa) #673972 log_debug echo "sympa sympa/remove_spool boolean true" | debconf-set-selections ;; #################################################################### ironic-common) dbconfig_enable_purge ironic-common ;; movabletype-opensource) dbconfig_enable_purge movabletype-opensource ;; nova-common) dbconfig_enable_purge nova-common ;; otrs2) dbconfig_enable_purge otrs2 ;; pdns-backend-sqlite) dbconfig_enable_purge pdns-backend-sqlite ;; pdns-backend-sqlite3) dbconfig_enable_purge pdns-backend-sqlite3 ;; redmine) dbconfig_enable_purge redmine instances/default ;; request-tracker3.8) dbconfig_enable_purge request-tracker3.8 ;; request-tracker4) dbconfig_enable_purge request-tracker4 ;; esac if [ -d /var/lib/mysql ]; then log_debug echo "Enabling MySQL database purge." echo "mysql-server-5.1 mysql-server-5.1/postrm_remove_databases boolean true" | debconf-set-selections echo "mysql-server-5.5 mysql-server-5.5/postrm_remove_databases boolean true" | debconf-set-selections echo "mysql-server-5.6 mysql-server-5.6/postrm_remove_databases boolean true" | debconf-set-selections echo "mysql-server-5.7 mysql-server-5.7/postrm_remove_databases boolean true" | debconf-set-selections # wrong template name is used in jessie echo "mariadb-server-10.0 mysql-server-10.0/postrm_remove_databases boolean true" | debconf-set-selections echo "mariadb-server-10.0 mariadb-server-10.0/postrm_remove_databases boolean true" | debconf-set-selections echo "mariadb-server-10.1 mariadb-server-10.1/postrm_remove_databases boolean true" | debconf-set-selections echo "mariadb-server-10.3 mariadb-server-10.3/postrm_remove_databases boolean true" | debconf-set-selections echo "percona-xtradb-cluster-server-5.5 mysql-server-5.1/postrm_remove_databases boolean true" | debconf-set-selections fi if [ -d /var/lib/ldap ]; then log_debug echo "Enabling LDAP database purge." echo "slapd slapd/purge_database boolean true" | debconf-set-selections fi if [ -d /var/lib/cyrus ] || [ -d /var/spool/cyrus ] || [ -d /var/spool/sieve ]; then log_debug echo "Enabling Cyrus spool purge." echo "cyrus-common cyrus-common/removespools boolean true" | debconf-set-selections fi exit 0 piuparts-1.1.1/custom-scripts/scripts-lenny/0000755000000000000000000000000013363064533016066 5ustar piuparts-1.1.1/custom-scripts/scripts-lenny/post_remove_exceptions_lenny0000755000000000000000000000076513363064533024034 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case "$PIUPARTS_DISTRIBUTION" in lenny) : ;; *) exit 0 ;; esac case ${PIUPARTS_OBJECTS%%=*} in apt-xapian-index) # wait for update-apt-xapian-index in the background to finish log_debug sleep 30 ;; postgis) #560409: postgis can't be purged if postgresql is no longer installed # (fails due to a missing directory) log_debug mkdir -p -v /usr/lib/postgresql/8.3/lib ;; esac piuparts-1.1.1/custom-scripts/scripts-lenny/post_setup_lenny-fake-essential0000755000000000000000000000211113363064533024312 0ustar #!/bin/sh set -e # The following issues won't be fixed in lenny: # - unconditional use of deluser during postrm purge # - unconditional use of ucf during postrm purge # so add these packages to the "fake" essential set. USED_DURING_PURGE="adduser ucf" case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # don't install fake essential packages while creating the tarball exit 0 ;; adduser|ucf) # allow testing of the fake essential packages exit 0 ;; console-log|systraq) USED_DURING_PURGE="$USED_DURING_PURGE perl-modules" ;; docbookwiki) USED_DURING_PURGE="$USED_DURING_PURGE mysql-client" ;; prelude-manager) #660455 USED_DURING_PURGE="$USED_DURING_PURGE dbconfig-common" ;; drupal6|moodle|moodle-book|moodle-debian-edu-theme|scuttle) USED_DURING_PURGE="$USED_DURING_PURGE wwwconfig-common" ;; octave-audio|octave-symbolic|octave-vrml) USED_DURING_PURGE="$USED_DURING_PURGE octave3.0" ;; ttf-beteckna) #502707 USED_DURING_PURGE="$USED_DURING_PURGE defoma" ;; esac echo "*** Adding fake essential packages ***" apt-get install -yf $USED_DURING_PURGE piuparts-1.1.1/custom-scripts/scripts-log-alternatives/0000755000000000000000000000000013266771660020231 5ustar piuparts-1.1.1/custom-scripts/scripts-log-alternatives/post_install_log_alternatives0000755000000000000000000000041513266771660026314 0ustar #!/bin/sh set -e [ ! -e /usr/bin/update-alternatives.orig ] && exit 0 rm /usr/bin/update-alternatives dpkg-divert --rename --remove /usr/bin/update-alternatives if [ -f /tmp/log-alternatives.log ]; then cat /tmp/log-alternatives.log rm /tmp/log-alternatives.log fi piuparts-1.1.1/custom-scripts/scripts-log-alternatives/pre_install_log_alternatives0000755000000000000000000000100213266771660026106 0ustar #!/bin/sh set -e # Do nothing if the script already ran. # The pre_install step can be run multiple times. [ -e /usr/bin/update-alternatives.orig ] && exit 0 dpkg-divert --divert /usr/bin/update-alternatives.orig --rename /usr/bin/update-alternatives cat >/usr/bin/update-alternatives <<'EOT' #!/bin/sh echo "LOG-ALTERNATIVES: dpkg=${DPKG_MAINTSCRIPT_PACKAGE}: piuparts=${PIUPARTS_OBJECTS}: $0 $@" >>/tmp/log-alternatives.log exec /usr/bin/update-alternatives.orig "$@" EOT chmod +x /usr/bin/update-alternatives piuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/0000755000000000000000000000000013553616457021033 5ustar ././@LongLink0000644000000000000000000000015600000000000007775 Lustar piuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_distupgrade_base_cleanup_multi-distro-upgradepiuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_distupgrade_base_cleanup_multi-distr0000755000000000000000000000043713463366315031376 0ustar #!/bin/sh set -e # Only while creating the reference chroot. test "$PIUPARTS_PHASE" = "" || exit 0 if [ "$PIUPARTS_DISTRIBUTION" = "buster" ] || \ [ "$PIUPARTS_DISTRIBUTION" = "buster-next" ] ; then if [ -d /etc/init ]; then rmdir --ignore-fail-on-non-empty -v /etc/init fi fi ././@LongLink0000644000000000000000000000015400000000000007773 Lustar piuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_distupgrade_exceptions_multi-distro-upgradepiuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_distupgrade_exceptions_multi-distro-0000755000000000000000000000156313463366315031373 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } is_installed() { local pkg="$1" dpkg-query -s "$pkg" >/dev/null 2>&1 || return 1 local status="$(dpkg-query -W -f '${Status}' $pkg)" test "$status" != "unknown ok not-installed" || return 1 test "$status" != "deinstall ok config-files" || return 1 return 0 } # Skip while creating the reference chroot. test "$PIUPARTS_PHASE" != "" || exit 0 if [ "$PIUPARTS_DISTRIBUTION" = "jessie" ]; then if [ -h /etc/alternatives/javaws ] && ! is_installed openjdk-7-jre ; then #775072 openjdk-7-jre: cleanup broken alternatives left by openjdk-6-jre: javaws, pluginappletviewer log_debug update-alternatives --remove javaws /usr/lib/jvm/java-6-openjdk/jre/bin/javaws update-alternatives --remove pluginappletviewer /usr/lib/jvm/java-6-openjdk/jre/bin/pluginappletviewer fi fi ././@LongLink0000644000000000000000000000015200000000000007771 Lustar piuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_setup_fake-essential_multi-distro-upgradepiuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/post_setup_fake-essential_multi-distro-up0000755000000000000000000000170513463366315031275 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS} in mediamate=None) # removed after lenny #574228: mediamate: package purge (after dependencies removal) fails log_debug apt-get -y install wwwconfig-common ;; moodle-book=None|\ moodle-debian-edu-theme=None|\ moodle=None) # fixed after squeeze, not in stable after squeeze #678027: moodle: fails to purge - command ucf in postrm not found log_debug apt-get -y install ucf ;; rt3.6-rtfm=None) # removed after lenny # fails to purge: rt3.6-rtfm.postrm: ucf: not found log_debug apt-get -y install ucf ;; rt3.8-rtfm=None) # removed after squeeze # fails to purge: rt3.8-rtfm.postrm: ucf: not found log_debug apt-get -y install ucf ;; torrentflux=None) # removed after wheezy #677061: torrentflux: fails to purge - command ucf in postrm not found log_debug apt-get -y install ucf ;; esac ././@LongLink0000644000000000000000000000014600000000000007774 Lustar piuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/pre_remove_exceptions_multi-distro-upgradepiuparts-1.1.1/custom-scripts/scripts-multi-distro-upgrade/pre_remove_exceptions_multi-distro-upgrad0000755000000000000000000000460013553616457031361 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS} in autotrust=None) # removed after squeeze # logrotate config references /var/log/autotrust/*.log log_debug touch /var/log/autotrust/dummy.log ;; chef-solr=None) # removed after wheezy # logrotate config references /var/log/chef/solr.log log_debug touch /var/log/chef/solr.log ;; chef-server-api=None) # removed after wheezy # logrotate config references /var/log/chef/server.log /var/log/chef/merb*log log_debug touch /var/log/chef/server.log touch /var/log/chef/merb-dummy.log ;; dancer-ircd=None) # removed after wheezy # logrotate config references /var/log/dancer-ircd/ircd.log log_debug touch /var/log/dancer-ircd/ircd.log ;; dsyslog=None) # removed after jessie # logrotate config references /var/log/messages log_debug touch /var/log/messages ;; fossology-scheduler-single=None) # removed after squeeze # logrotate config references /var/log/fossology/*.log log_debug chown root:root /var/log/fossology touch /var/log/fossology/dummy.log ;; fspd=None) # removed after lenny # logrotate config references /var/log/fspd/fspd.log, /var/log/fspd/xferlog log_debug touch /var/log/fspd/fspd.log touch /var/log/fspd/xferlog ;; gidentd=None) # removed after lenny # logrotate config references /var/log/gidentd.log log_debug touch /var/log/gidentd.log ;; interchange=None) # removed after wheezy # logrotate config references /var/log/interchange/debug.log, /var/log/interchange/error.log, /var/log/interchange/makecat.log log_debug chown root:root /var/log/interchange touch /var/log/interchange/debug.log touch /var/log/interchange/error.log touch /var/log/interchange/makecat.log ;; mserv=None) # removed after lenny # logrotate config references /var/log/mserv/mserv.log log_debug touch /var/log/mserv/mserv.log ;; nws=None) # removed after lenny # logrotate config references /var/log/nws/*.err, /var/log/nws/*.log log_debug touch /var/log/nws/dummy.err touch /var/log/nws/dummy.log ;; qwik=None) # removed after lenny # logrotate config references /var/log/qwik/*log log_debug touch /var/log/qwik/dummy.log ;; smtpfeed=None) # removed after lenny # logrotate config references /var/log/smtpfeed.log log_debug touch /var/log/smtpfeed.log ;; esac piuparts-1.1.1/custom-scripts/scripts-no-usr-share-doc/0000755000000000000000000000000013266771660020037 5ustar piuparts-1.1.1/custom-scripts/scripts-no-usr-share-doc/post_setup_disable_usr_share_doc0000755000000000000000000000257013266771660026561 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # skip while creating the tarball exit 0 ;; esac # clear out /usr/share/doc # but keep the directories (and symlinks) as well as the copyright files cat << EOF > /etc/dpkg/dpkg.cfg.d/piuparts-path-exclude path-exclude=/usr/share/doc/*/* path-include=/usr/share/doc/*/copyright EOF # switching init systems forth and back will clean out /usr/share/doc # reinstalling the affected packages beforehand makes the files disappear # before the snapshot of the reference system is created CANDIDATES="systemd systemd-sysv sysv-rc" CANDIDATES=$(dpkg-query -W $CANDIDATES | awk '{ if ($2) { print $1 } }') if [ -n "$CANDIDATES" ]; then echo "Reinstalling $(echo $CANDIDATES)..." # workaround apt bug #770291 - do it one by one, not all at once for package in $CANDIDATES do apt-get -u --reinstall install $package done fi case ${PIUPARTS_OBJECTS%%=*} in localepurge) case ${PIUPARTS_DISTRIBUTION} in lenny*|squeeze*) ;; *) # reinstall packages that will be reinstalled after purge # to not record their /usr/share/doc content that is about to disappear log_debug EXTRA="" apt-get -u --reinstall --fix-missing install $(dpkg -S LC_MESSAGES 'man/??/man' | cut -d: -f1 | tr ', ' '\n' | sort -u) $EXTRA ;; esac ;; esac piuparts-1.1.1/custom-scripts/scripts-squeeze/0000755000000000000000000000000013463366315016426 5ustar piuparts-1.1.1/custom-scripts/scripts-squeeze/post_distupgrade_squeeze-fake-essential0000755000000000000000000000025413266771660026373 0ustar #!/bin/sh set -e test "$PIUPARTS_DISTRIBUTION" = "squeeze" || exit 0 # work around #601961: apt: wrongly thinks install-info is essential apt-get -y install install-info piuparts-1.1.1/custom-scripts/scripts-squeeze/post_remove_exceptions_squeeze0000755000000000000000000000061013266762112024711 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } case "$PIUPARTS_DISTRIBUTION" in squeeze) : ;; *) exit 0 ;; esac case ${PIUPARTS_OBJECTS%%=*} in libblkid-dev) #775350 - unhandled symlink to directory conversion if [ ! -f /usr/share/doc/libblkid1/copyright ]; then log_debug apt-get install --reinstall libblkid1 fi ;; esac piuparts-1.1.1/custom-scripts/scripts-squeeze/post_setup_squeeze-fake-essential0000755000000000000000000000216713463366315025221 0ustar #!/bin/sh set -e # The following issues won't be fixed in squeeze: # - unconditional use of deluser during postrm purge # - unconditional use of ucf during postrm purge # so add these packages to the "fake" essential set. USED_DURING_PURGE="adduser ucf" FAILS_TO_REMOVE= case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # don't install fake essential packages while creating the tarball exit 0 ;; adduser|ucf) # allow testing of the fake essential packages exit 0 ;; mediamate) # removed after lenny #574228: mediamate: package purge (after dependencies removal) fails USED_DURING_PURGE="$USED_DURING_PURGE wwwconfig-common" ;; phpbb3) USED_DURING_PURGE="$USED_DURING_PURGE dbconfig-common" ;; prelude-manager) #660455 USED_DURING_PURGE="$USED_DURING_PURGE dbconfig-common" ;; esac # work around #601961: apt: wrongly thinks install-info is essential case ${PIUPARTS_DISTRIBUTION} in squeeze|squeeze-lts) # there is no install-info in lenny FAILS_TO_REMOVE="$FAILS_TO_REMOVE install-info" ;; esac echo "*** Adding fake essential packages ***" apt-get install -yf $USED_DURING_PURGE $FAILS_TO_REMOVE piuparts-1.1.1/custom-scripts/scripts-sysvinit/0000755000000000000000000000000013266762112016631 5ustar piuparts-1.1.1/custom-scripts/scripts-sysvinit/pre_distupgrade_zz_sysvinit0000755000000000000000000000017713266762112024440 0ustar #!/bin/sh set -e if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "jessie" ]; then apt-get update apt-get -y install sysvinit-core fi piuparts-1.1.1/custom-scripts/scripts-unused-examples/0000755000000000000000000000000013266771660020070 5ustar piuparts-1.1.1/custom-scripts/scripts-unused-examples/post_chroot_unpack_key_setup0000755000000000000000000000056413266771660026017 0ustar #!/bin/sh # we rely on gnupg and wget being available, # make sure to use "--include=gnupg,wget" in your debootstrap cmdline echo "Setting up https://example.com/internal_key.asc for apt-get usage." wget -O - 'https://example.com/internal_key.asc' | apt-key add - echo "Running apt-get update to have a verified and working Debian repository available." apt-get update piuparts-1.1.1/custom-scripts/scripts-wheezy/0000755000000000000000000000000013463366315016260 5ustar piuparts-1.1.1/custom-scripts/scripts-wheezy/post_setup_wheezy-fake-essential0000755000000000000000000000105113266771660024700 0ustar #!/bin/sh set -e # The following issues won't be fixed in wheezy: # - unconditional use of deluser during postrm purge # - unconditional use of ucf during postrm purge # so add these packages to the "fake" essential set. USED_DURING_PURGE="adduser ucf" case ${PIUPARTS_OBJECTS%%=*} in TARBALL) # don't install fake essential packages while creating the tarball exit 0 ;; adduser|ucf) # allow testing of the fake essential packages exit 0 ;; esac echo "*** Adding fake essential packages ***" apt-get install -yf $USED_DURING_PURGE piuparts-1.1.1/custom-scripts/scripts-wheezy/pre_distupgrade_wheezy0000755000000000000000000000407013463366315022763 0ustar #!/bin/sh set -e log_debug() { echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS" } # Work around bug where apt/squeeze prefers to 'keep back' some # upgradable packages instead of removing some obsolete ones. FORCEREMOVE= if [ "$PIUPARTS_DISTRIBUTION_NEXT" = "wheezy" ] then case ${PIUPARTS_OBJECTS%%=*} in gnustep|\ gnustep-games|\ gnustep-back0.18-cairo|\ gnustep-back0.18-art|\ gnustep-back-dbg|\ libgnustep-base1.20-dbg|\ addressmanager.app|\ biococoa.app|\ easydiff.app|\ gnumail.app|\ gnumail.app-dbg|\ mines.app|\ projectmanager.app|\ INVALID) FORCEREMOVE="libobjc2" ;; libahven17.0|\ libalog0.3-full|\ libalog1-full-dev|\ libalog-full-dbg|\ libapq3.0|\ libapq-postgresql3.0|\ libaws2.7|\ libaws2.7-dev|\ libaws-dbg|\ libflorist2009|\ libflorist-dbg|\ libgmpada1|\ libgmpada1-dbg|\ libgnomeada2.14.2|\ libgnomeada2.14.2-dbg|\ libgnomeada2.14.2-dev|\ libgtkada2.14.2|\ libgtkada2.14.2-dbg|\ libgtkada2.14.2-dev|\ libgtkada-gl2.14.2|\ libgtkada-glade2.14.2|\ libgtkada2-bin|\ liblog4ada0|\ liblog4ada-dbg|\ libnarval1.10.1|\ libnarval1-dev|\ libnarval-dbg|\ libpcscada0.6|\ libplplot-ada|\ libplplot-dev|\ libpolyorb2|\ libtemplates-parser11.5|\ libtexttools4|\ libtexttools4-dbg|\ libtexttools-doc|\ libxmlada3.2|\ libxmlezout0|\ libxmlezout-dbg|\ cl-plplot|\ gprbuild|\ narval-doc|\ narval-generic-actors|\ narval-servers|\ narval-tests-actors|\ narval-utils|\ polyorb-servers|\ INVALID) FORCEREMOVE="gnat-4.4-base" ;; dolfin-bin|\ dolfin-dev|\ dolfin-doc|\ petsc-dev|\ python-dolfin|\ INVALID) # only affects i386 FORCEREMOVE="libjpeg62-dev" ;; science-physics) # #706111 FORCEREMOVE="tessa" ;; firestarter|\ gnome-netstatus-applet|\ gnome-utils|\ INVALID) FORCEREMOVE="gconf2" ;; esac fi if [ -n "$FORCEREMOVE" ]; then if dpkg-query -s $FORCEREMOVE >/dev/null 2>&1 ; then log_debug echo "Forcibly removing $FORCEREMOVE for smoother upgrade" dpkg -r --force-depends $FORCEREMOVE fi fi piuparts-1.1.1/debian/0000755000000000000000000000000013617514262011500 5ustar piuparts-1.1.1/debian/changelog0000644000000000000000000050442413617514262013363 0ustar piuparts (1.1.1build1) focal; urgency=medium * No-change rebuild with fixed binutils on arm64. -- Matthias Klose Sat, 08 Feb 2020 11:17:06 +0000 piuparts (1.1.1) unstable; urgency=medium * Team upload to unstuck python-debianbts migration. [ Nis Martensen ] * autopkgtest: replace broken stubs with actual tests. * Makefile: delete __pycache__ dirs in make clean. * piupartslib/dwke: do not error out on non-utf8 logfiles. -- Holger Levsen Tue, 14 Jan 2020 13:48:49 +0100 piuparts (1.1.0) unstable; urgency=medium * Team upload. [ Herbert Parentes Fortes Neto ] * piuparts.py: use Popen with universal_newlines=True. Py2, Py3 compatible. * New mail template: add bug-templates/obsolete_conffiles.mail. [ Thomas Goirand ] * Switch to package to Python 3: - Replaced python- with python3- in debian/control. - Removed all python2 (>= 2.7) by a ${python3:Depends}. - Replaced python2 by python3 in debian/rules. - Replaced dh_python2 stuff by dh_python3 in debian/rules. - Patched piupartslib/conf.py, piupartslib/dependencyparser.py, piupartslib/packagesdb.py, piupartslib/pkgsummary.py, tests/unittests.py so that they are Python 3 compatible. - Removed python-lzma (build-)depends as it's in standard in Py 3. - Add python3-six as (build-)depends. - Fix Makefile to use python3 and not python2. - d/piuparts-common.install: switch to python 3. - d/rules: remove all __pycache__ folder before proceeding to install. - Add --shebang=/usr/bin/python3 when calling dh_python3. - Do not use -N piuparts-master -N piuparts-slave when calling dh_python3 for the 2nd time, so that it correctly adds the interpreter as depends. - Lintian overrides "python-script-but-no-python-dep": that's ok, because piuparts-common already has the dependency. [ Bastian Venthur ] * piuparts-analyze.py: updates for python-debianbts 2.10.0 deprecating some methodcalls, update depends in d/control accordingly. Closes: #919170. [ Nis Martensen ] * piuparts, piuparts-master-backend, piuparts-report, detect_well_known_errors, piupartslib/(dependencyparser|packagesdb): several fixes and improvements related to porting to python3. [ James Clarke ] * piuparts: Fix diff_meta_data reporting every file as new and removed. [ Mattia Rizzolo ] * piuparts-slave.py: port to python3. [ David Prévot ] * Use the same favicon as the one from www.d.o. [ Holger Levsen ] * Bump version number to 1.1.x due to the switch to python3. * piuparts.conf-template.pejacevic: - also test bullseye and bullseye-rcmd. Closes: #933061. - increase precedence values for oldstable and older. * docs/README_server.txt, docs/piuparts/piuparts.1.txt and piuparts.py: update examples for buster being stable and bullseye being the new testing. * Use /usr/bin/python3 everywhere. Closes: #937293. * d/control: - bump standards version to 4.4.1, no changes needed. - bump debhelper-compat to 12. * d/tests: Add simple smoke-test. * htdocs/news.tpl: - mention new suites bullseye and bullseye-rcmd being tested. - mention piuparts.debian.org is running the python3 branch. - add farewell note. * instances/Makefile: set default mirror to deb.debian.org. * d/rules: set GOCACHE to a safely writable directory during build as the cache is mandatory with Go 1.12. * d/control: - bump standards version to 4.4.1, no changes needed. - bump debhelper-compat to 12. - remove myself from uploaders. It was fun! Policy is your friend. Trust the Policy. Love the Policy. Obey the Policy. * instances/*, docs/README_pejacevic.txt and htdocs/index: stop mentioning piu-slave-bm-a.debian.org. -- Holger Levsen Wed, 08 Jan 2020 20:50:42 +0100 piuparts (1.0.1) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.conf, distros.conf: Update for buster release. * piuparts.py: - Add '--max-command-output-size' option to allow overriding the default of 8 MB for debugging runs. - Run pre_remove scripts before computing packages to be removed. - Fix spurious failure to remove packages with names ending with '+'. * piupartslib/packagesdb.py: - Remove stale .kpr files after receiving new logs. * piuparts-slave.py: - Generate separate tarball names for --merged-usr chroots. - Re-exec on SIGUSR1, picking up updated code and new config sections. * master-bin/detect_piuparts_issues: - Clean up stale temporary and empty files. * master-bin/rotate_master_logs: Delete master logs older than 90 days. * slave-bin/slave_reexec: New, send SIGUSR1 to all running slaves. * custom-scripts: Add and update several exceptions. * piuparts.conf.anbe: Add some more example sections. [ Holger Levsen ] * generate_daily_report: improve statistics. -- Holger Levsen Tue, 18 Jun 2019 14:48:59 +0200 piuparts (1.0.0) unstable; urgency=medium [ David Steele ] * piuparts-reports.py: Revert "Don't add results from outdated binaries to summary.json" from 0.94. The reverted commit is 62a6b3c438f42480ce3a08996ed4994de1561591. The patch was added to avoid incorporating summary.json reporting results for binary packages which are no longer part of the source package. It stopped Piuparts from blocking the source package from testing migration. It has a bug related to "upgrade test" sections, which cause a large number of packages to incorrectly be reported in the "waiting" state. [ Andreas Beckmann ] * piuparts.py: - Add '--merged-usr' support when using debootstrap to create the chroot. - Add '--distupgrade-to-testdebs-from DIR' as a shorthand for '--bindmount DIR --testdebs-repo DIR --distupgrade-to-testdebs'. * distros.conf: - wheezy* has been archived/removed. - jessie* has been partially archived/removed. - Add buster-backports. - Add bullseye-proposed-updates. * piuparts.conf: - Add section [sid-merged-usr], using --merged-usr. (Closes: #848968) - Use --warn-on-debsums-errors for tests ending in buster. - Rename [stretch2proposed] to [stretch2next] and [buster2proposed] to [buster2next]. * piupartslib/packagesdb.py: - Keep recycled pass logs while waiting for new test results, continuing to show them as "successfully-tested" instead of "waiting-to-be-tested". - Add (hardcoded) list of missing dependencies that resolve to "ignore-does-not-exist". Used for packages to be skipped via "is_installable_*" scripts to exclude them from "dependency-does-not-exist" state. * piuparts-slave.py: - New option "slave-flush-interval" to flush logs more frequently to the master. Set to 30 minutes. - New option "chroot-meta-directory" to share reference target chroot metadata between slave instances. * piuparts-report.py: - Omit states with 0 packages from section summaries. * scripts/{post_chroot_unpack,pre_distupgrade}_allow_unauthenticated: - Disable Check-Valid-Until for wheezy/updates and jessie-backports. * scripts/post_distupgrade_exceptions: - Handle /var/lib/pam/seen stretch -> buster upgrade by running pam-auth-update if needed. Workaround for #920760. * scripts/pre_remove_40_find_unowned_lib_links: - Handle --merged-usr environments. * master-bin/reschedule_piuparts_tests: Do not act on 'bugged/*.log' and 'affected/*.log' by default. Add --bugged, --affected flags. [ Holger Levsen ] * Declare this release to be version 1.0.0, finally. May the force be with you. * htdocs/news.tpl: - add news about sid-merged-usr being tested. - add news about the 1.0.0 release. * piuparts_slave_join.8.txt: update revdate to date of last change. -- Holger Levsen Sat, 04 May 2019 21:37:32 +0200 piuparts (0.98) unstable; urgency=medium [ Holger Levsen ] * piuparts-reports.py: add link to states graph for each suite. * instances/piuparts.conf-template.pejacevic: increase sid-nodoc's precedence. * Refactor reschedule_piuparts_tests to use new global configuration variable 'testing-suite'. * generate_daily_report: show failures in sid, $TESTING and testing2sid at the top of the report. * master-bin/prepare_backup: ignore empty submissions.txt for $TESTING-pu and $TESTING-security. * Update TODO for bullseye. [ Andreas Beckmann ] * piuparts.py: - If packages were downgraded to restore the reference chroot state reinstall the downgraded-to version again to recover potentially lost files in case of bad interactions of Breaks/Replaces with downgrading. * distros.conf: Add support for buster partial distros. * piuparts.conf: - Use --warn-on-others for [sid-strict] only. - Add sections [buster-security], [buster-pu] and [buster2proposed]. - Add sections [stretch2Xbuster] and [stretch2Xbuster-rcmd]. - Add section [sid-broken-symlinks], failing on broken symlinks * piuparts-slave.py: - Catch IOError while creating tarballs, will be retried later. * scripts/is_testable_uninstallable: - Refactor handling of known uninstallable packages. * scripts-multi-distro-upgrade/: New custom scripts for distupgrade tests spanning a series of distros. * Add metapackages piuparts-{master,slave}-from-git-deps to ease running piuparts-master and piuparts-slave instances from git. -- Holger Levsen Fri, 01 Mar 2019 11:27:47 +0100 piuparts (0.97) unstable; urgency=medium [ Andreas Beckmann ] * Ignore broken /var/lib/mender symlink. (Closes: #915680) * scripts/pre_distupgrade_zz_database-server: Also act on dist-upgrades within a release, e.g. stable -> stable-proposed-updates. * piuparts-slave.py: - Support prepending a setarch command to the piuparts invocation. * piuparts-report.py: - Report [cruft] in logfile lists. [ David Steele ] * instances/piuparts.conf-template.pejacevic: - report results of stable2sid to DDPO. - swap stable2sid and stable22sid priorities, to promote the one reported to DDPO. [ Holger Levsen ] * slave_bin: on stop, show what's keeping the slaves busy, to identify packages with a very long installation time. * Makefile: include @sbindir@ in placeholder_substitution. * instances/piuparts.conf-template.pejacevic: lower priorities of oldstable suites. [ Herbert Fortes ] * piuparts.py, piuparts-slave.py, piupartslib/pkgsummary.py and piuparts-analyze.py: add '()' to print. Py2 and Py3 compatible. * piupartslib/dwke.py: - class Problem: - Class attr instead of instance attr Save some memory - get_file_dict function: - dictionary comprehension. "More readable and often faster at the same time". - create_problem_list function: - list.sort() instead of sorted(list). sorted() returns a *new* sorted list. This is not necessary here. list.sort() sorts the list in place. Save some memory - make_kprs function: do not concat str, simplify 'if' statement. -- Holger Levsen Mon, 11 Feb 2019 12:30:10 +0100 piuparts (0.96) unstable; urgency=medium [ Andreas Beckmann ] * bug-templates: Raise severity of "fails to purge" errors to serious. (see #918379) * distros.conf: Add jessie-lts as a full distro. * piuparts.conf: - Use jessie-lts as upgrade target and rename section [jessie2proposed] => [jessie2lts] - Add sections [wheezy2jessie-lts] and [jessie-lts2stretch]. - [wheezy2bpo2jessie]: Use jessie-lts as final upgrade target. - Disable sections [wheezy-pu], [jessie-pu]: not used for lts releases. * scripts/post_distupgrade_exceptions: - Handle /etc/ca-certificates.conf stretch -> buster upgrade by normalizing the file to match fresh installations. (Closes: #889852) * scripts-broken-symlinks: Unbreak more symlinks. [ Holger Levsen ] * htdocs/index.tpl: stop referring people to the #debian-qa IRC channel, we really prefer mail nowadays. * instances/piuparts.conf-template.pejacevic: - include sid-strict into json outputs. - improve sid-strict description. Closes: #918379 * master-bin/detect_piuparts_issues: ignore files which vanish during run. * Bump standards version to 4.3.0, no changes needed. * d/copyright: update years. -- Holger Levsen Sun, 13 Jan 2019 16:56:06 +0100 piuparts (0.95) unstable; urgency=medium [ Holger Levsen ] * piuparts.py: add /var/lib/debian-security-support/ to self.ignored_files. (Closes: #749317) * Reduce number of mails per day, from 34 down to 1. (Closes: #912591) - lib/write_log.sh: new helper to publish logs in htdocs/logs. - master-bin/master_shell_runner: new script to run python scripts to redirect logs to htdocs/logs. - master-bin/generate_daily_report: - redirect logs of called scripts to htdocs/logs. - include listing of htdocs/logs/$TODAY into daily report. - publish dailyreport.txt in htdocs/logs/$TODAY instead of htdocs/. - master-bin/detect_archive_issues, detect_piuparts_issues, detect_network_issues, reschedule_oldest_logs, rotate_master_logs, report_untestable_packages and report_stale_reserved_packages: redirect logs to htdocs/logs. - conf/crontab-master.in: run detect_well_known_errors, piuparts-report and piuparts-analyze through the new master_shell_runner script. - master-bin/rotate_master_logs: compress htdocs/logs older than a week. - piuparts-report.py: add navigation link to https://piuparts.d.o/logs/ - htdocs/news.tpl: explain the result of these changes. * d/control: use canonical URL for Vcs-Git, thanks lintian. * Add d/piuparts.lintian-overrides to override uses-dpkg-database-directly warnings, this cannot be avoided without breaking compatibility with ancient releases. * bug-templates/logrotate_exits_with_error_after_package_removal: clarify possibles causes of this problem as well as the solution. [ Andreas Beckmann ] * Clarify errors from installing objects over existing symlinks. * bug-templates: Do not mention ${misc:Pre-Depends} as symlink_to_dir/dir_to_symlink are supported by dpkg in oldstable. -- Holger Levsen Sun, 09 Dec 2018 22:27:41 +0100 piuparts (0.94) unstable; urgency=medium [ Holger Levsen ] * master-bin/detect_well_known_errors.py: suppress output if nothing new is found in a section, or section is busy or has been recently processed. * master-bin/generate_daily_report: include number of failures in daily mail. * slave-bin/detect_slave_problems: increase amount of time (from 30 to 60m) a slave has to be inactive before complaining. [ Andreas Beckmann ] * piuparts.py: - Logrotate files can only be placed directly in /etc/logrotate.d/. * scripts/post_distupgrade_exceptions: - Handle /etc/nsswitch.conf stretch -> buster upgrade if modifications from libnss-myhostname and friends are present. [ Mathieu Parent ] * piuparts.py: Use mknod instead of touch to create missing /dev/ptmx mountpoint, working around findutils bug #912180 (Closes: #911334) [ Ivo De Decker ] * piuparts-reports.py: Don't add results from outdated binaries to summary.json. -- Holger Levsen Thu, 08 Nov 2018 12:09:57 +0100 piuparts (0.93) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Report the package that owns a broken symlink. - --allow-database now also enables firebird3.0. * piuparts.conf: Disable rescheduling tests that end in wheezy. * piuparts.conf.anbe: Add some more example sections. * piupartslib/packagesdb.py: - Add methods Package.source(), Package.source_version(). - Remove method PackagesDB.get_source(). * piuparts-report.py: - Postpone deletion of old logs after the html pages possibly referencing them have been updated, avoiding error 404 while processing a section. - Generate missing/outdated .kpr files. - Simplify known problem report generation. - Mark logs of packages built from an outdated source version as [cruft]. * scripts/{post_chroot_unpack,pre_distupgrade}_allow_unauthenticated: New scripts to handle installation from archived releases with expired signing keys (lenny, squeeze). Drop the --no-check-valid-until from *.conf since that is active for all steps in a distupgrade test while the new scripts only enable it for the distros that actually need it. * scripts/post_setup_forbid_home: Use dpkg-divert --no-rename (dpkg 1.19.1). * scripts/pre_distupgrade_zz_database-server: sid/buster has postgresql 11. * scripts/post_distupgrade_base_cleanup: Remove libcryptsetup4, libprocps6 from the buster reference chroot. * Overhaul some exceptions. * Add bug template for logrotate errors after package removal. [ Holger Levsen ] * piuparts-report.py: - Add new page: overview.html - Split index.html into about (index.html) and news.html. - Shuffle some entries on index.html, explain that piuparts.debian.org exists since 2005 (even though that FQDN is younger). - Move suite "by uploaders" and "by source packages" links further up in navigation menu. Improve structure and headings in navigation menu. - Link mail templates for filings from navigation menu. - Drop references to "PTS" and use "tracker.d.o" instead. - Add link to piuparts source package pages from known problem pages. - Minor cleanups. * htdocs/bug_howto.tpl: some improvements, it's been some years. * htdocs/news.tpl: confirm #582630 has been fixed. * Use the new debhelper-compat(=11) notation and drop d/compat. * CONTRIBUTING: explain never to directly push to the develop nor master branch. Explain to always include a debian/changelog entry with patches. Explain to always test patches and to explain that testing has happened. -- Holger Levsen Sun, 21 Oct 2018 13:24:09 +0200 piuparts (0.92) unstable; urgency=medium [ Simon McVittie ] * piuparts.py: - Add to ignored_files: + /var/lib/dpkg/lock-frontend (Closes: #909182) [ Holger Levsen ] * Update several URLs pointing to https://www.debian.org/doc/debian-policy to match the new URLs since debian-policy 4.1.0. * instances/piuparts.conf-template.pejacevic: reschedule already tested packages in sid more often: set reschedule-old-count = 666 and reschedule-old-days = 66. * Update TODO. -- Holger Levsen Sat, 22 Sep 2018 15:29:49 +0200 piuparts (0.91) unstable; urgency=medium * Don't use --skip-logrotatefiles-test when testing sid and buster, see #582630. (Closes: #604807) Keep it for stretch, jessie, wheezy, squeeze and lenny based tests. * piuparts.py: - Add to ignored_files: + /var/lib/sreview (Closes: #905500) - Add alias '--ignore-regexp' for '--ignore-regex' as the former was documented since 2005 while the latter was working since then. Thanks to Nicolas Braud-Santoni for reporting. (Closes: #906683) * Bump standards version to 4.2.1, no changes needed. -- Holger Levsen Fri, 07 Sep 2018 19:35:35 +0200 piuparts (0.90) unstable; urgency=medium [ Michael Stapelberg ] * debiman-piuparts-distill: also consider --install, fixes https://github.com/Debian/debiman/issues/106 [ Holger Levsen ] * Bump standards version to 4.2.0, no changes needed. -- Holger Levsen Mon, 13 Aug 2018 13:37:03 +0200 piuparts (0.89) unstable; urgency=medium * scripts/pre_install_exceptions: force cleanup of sympa's spool directories. (Closes: #673972) * d/control: - move Built-Using: to piuparts-master binary package. - add Xs-Go-Import-Path: header. -- Holger Levsen Tue, 12 Jun 2018 14:44:37 +0000 piuparts (0.88) unstable; urgency=medium * Recycle failed packages in sid after 2 days and force rescheduling them 2 days later. * d/control: - shorten Vcs: headers. - drop versioned Pre-Depends on dpkg which is even satisfied in oldoldstable. - drop Breaks and Replaces on versions of piuparts which are older than oldoldstable. - add "Built-Using: ${misc:Built-Using}" which is needed because we ship golang binaries, thanks lintian. -- Holger Levsen Fri, 08 Jun 2018 13:17:44 +0000 piuparts (0.87) unstable; urgency=medium [ Philipp Hahn ] * piuparts.py: use `apt-cache show --no-all-versions` to prevent generating a "piuparts-depends-dummy" package with multiple dependencies on the same package with different versions. (Closes: #898606) [ Holger Levsen ] * Update all occurrences of piuparts-devel@lists.alioth.debian.org with piuparts-devel@alioth-lists.debian.net. * Drop X-Python-Version header from debian/control. -- Holger Levsen Sat, 02 Jun 2018 11:36:36 +0000 piuparts (0.86) unstable; urgency=medium * debian/control: switch packaging to salsa.debian.org. Thanks to the alioth admins for providing such a nice service so long! * Update several occurrences of anonscm.debian.org and git.debian.org with salsa.debian.org URLs. * index.tpl: add news entry about the salsa migration. * debian/copyright: update years of my contributions. -- Holger Levsen Sat, 28 Apr 2018 14:57:34 -0300 piuparts (0.85) unstable; urgency=medium [ Agustin Henze ] * piuparts.py: - Add docker support, new param is introduced `--docker-image`. (Closes: #893731) - Deprecate --keep-tmpdir in favor of --keep-env (Closes: #894232) - Create ${ENV}/dev/ptmx path if it doesn't exist (Closes: #895380) - Create ${ENV}/dev/null if it doesn't exist ASAP (Closes: #895379) * Add `docker.io` as suggested package. * Reworded documentation for `--keep-env` and `--schroot` parameters. [ Holger Levsen ] * dwke.py: change logging to make relevant information more visible. * detect_*_issues.in: improve output. * report_untestable_packages.in: improve output. * piuparts.1.txt: improve wording for docker related changes and add explaination about the limitations of docker support. * Bump standards version to 4.1.4, no changes needed. -- Holger Levsen Sat, 14 Apr 2018 17:39:13 -0300 piuparts (0.84) unstable; urgency=medium * debian/control: - Add "Rules-Requires-Root: no" to support building as non-root. (I've also confirmed that the build output is bit by bit identical with and without this.) - Bump standards version to 4.1.3, no changes needed. * piuparts-report.py: credit Andreas properly. * Bump debian/compat to 11 and build-depend on debhelper >= 11~. -- Holger Levsen Fri, 16 Feb 2018 12:34:20 +0000 piuparts (0.83) unstable; urgency=medium [ Andreas Beckmann ] * distros.conf: - Add experimental-debug. * piuparts.conf.anbe: Add some more example sections. * piupartslib/packagesdb.py: - Add new package data accessors name(), version(), test_versions(). - Use test_versions() in all places referring to a piuparts test. - Set test_versions while loading versions from the final target distro. - Use the actual package versions for resolving the "outdated" state. * piuparts-analyze.py: - Merge report_newly_bugged_packages and process all sections. * piuparts-report.py, detect_well_known_errors.py, piuparts-analyze.py: - Use global locks to allow only one running instance of each script. - Acquire section locks to prevent concurrent processing of a section, including piuparts-master access. - Postpone busy sections and retry them later. - Run as three separate cronjobs. * conf/crontab-master: - Run detect_well_known_errors every three hours to avoid accumulating a large backlog of new logfiles. - Run piuparts-analyze twice per day. - Try piuparts-report every three hours s.t. we start 2-3 runs per day even with running times above 8-12 hours. - Run detect_{piuparts,network}_issues a short time before trying piuparts-report to exclude (most of) these logfiles from the report. * master-bin/report_newly_bugged_packages: Remove, merged into piuparts-analyze. * scripts/pre_distupgrade_zz_database-server: wheezy->jessie: Restart postgresql before upgrading the cluster, working around #882409. * Update policy references, the permalinks changed with the new toolchain. * Add bug template for failing to reinstall a package. * Adjust known_problems patterns for dpkg 1.19 message changes. [ Holger Levsen ] * dwke.py/-analyze.py/-report.py: quietly exit if another process is still running. * master-bin/detect_well_known_errors.py: compress output. * Bump standards version to 4.1.2, no changes needed. -- Holger Levsen Fri, 15 Dec 2017 11:12:13 +0000 piuparts (0.82) unstable; urgency=medium * Move xmlto from Build-Depends-Indep to Build-Depends. -- Holger Levsen Sat, 07 Oct 2017 17:30:19 +0200 piuparts (0.81) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /var/log/tallylog (Closes: #877592) - Run post_chroot_unpack scripts before initial apt-get update. * piuparts-analyze.py: - Add PiupartsBTS, a caching wrapper around debianbts. * General python code and style cleanup. Consistently use 'with open(...) as ...:' for scoped open files. * scripts/{pre,post}_test_exceptions: Add exception to prevent testing corekeeper from changing the core file settings in the running kernel. * scripts/pre_distupgrade_zz_database-server: sid/buster has postgresql 10. * Separate scripts-lenny/ from scripts-squeeze/. * scripts-squeeze/post_{setup,distupgrade}_squeeze-fake-essential: - Treat install-info as fake-essential for all tests ending in squeeze. * scripts-broken-symlinks/post_install_extras_unbreak_symlinks: Unbreak more symlinks. (Closes: #857151) * master-bin/detect_piuparts_issues: Detect another way of 'apt-cache show' failing. (Closes: #877417) * master-bin/distill_alternatives_log: Parse piuparts.conf to find the sections that use scripts-log-alternatives. * master-bin/generate_daily_report: Most parts are now separate cronjobs. * Add bug template for stable -> sid file overwrite errors. * Use copyright notices without FSF street address. [ Simon McVittie ] * Consider the /usr merge to be acceptable variation. Files that the dpkg database considers to be in /bin, /sbin, /lib/**, /lib*/** are now allowed to be physically in /usr/bin, etc. due to symlinks in the root directory, as created by usrmerge or debootstrap --merged-usr. (Closes: #860433) [ Holger Levsen ] * Fix syntax errors in git snapshot running on piuparts.debian.org, thanks to Boud Roukema and Philipp Huebner for the reports. (Closes: #875605, #875664) * Drop code to support using autopkgtest as a test backend. This code was merged in 2008 from Ubuntu, but it's unclear (and unlikely) that it still works. Also it never has really been supported. (See #815084) * Bump standards version to 4.1.1, no changes needed. * Update TODO. [ Michael Stapelberg ] * piuparts-distill: correctly handle errors. -- Holger Levsen Thu, 05 Oct 2017 15:56:25 +0200 piuparts (0.80) unstable; urgency=medium [ Michael Stapelberg ] * debiman-piuparts-distill: - increase maximum token size. - bugfix: wait until all results are collected. - don’t print all results. * distill_alternatives_log: also process buster. [ Holger Levsen ] * Move asciidoc to from Build-Depends-Indep to Build-Depends. (Newly needed because d697f2991 changed piuparts-master to an arch:any package.) Thanks to Adrian Bunk. (Closes: #873825) -- Holger Levsen Thu, 31 Aug 2017 17:38:11 +0200 piuparts (0.79) unstable; urgency=medium [ Holger Levsen ] * slave-run: properly delete all screenlogs from previous runs. * Bump standards version to 4.1.0, change priority from extra to optional. * Depend on debhelper >= 10.2.5~ and bump debian/compat to 10. [ Andreas Beckmann ] * piuparts.py: - Do not ignore errors while purging packages. - Add post_test_* custom scripts. These are run exactly once at the end of each test (before comparing with the reference chroot state). - is_testable_* custom scripts can use return value '2' to make a test fail without actually installing the package. Use for (unfixable) packages that would fail anyway, but are resource hogs or otherwise break piuparts. - Silence diff_meta_data() for "others" for --warn-on-others. - Store the initial package selections in the --save-end-meta meta-data and ignore meta-data from --end-meta if these selections don't match, e.g. due to different fake-essential package sets. - Let debsums always ignore obsolete conffiles. - Run debootstrap with --no-merged-usr. * distros.conf: - Add wheezy-lts as a full distro. * piuparts.conf: - Use relative expiration ages. - Use wheezy-lts as upgrade target and rename sections: + [wheezy2proposed] => [wheezy2lts] + [squeeze2wheezy-proposed] => [squeeze2wheezy-lts] - [squeeze2bpo2wheezy]: Use wheezy-lts as final upgrade target. * piuparts.conf.anbe: Add some more example sections. * Move some exceptions to scripts/{pre,post}_test_exceptions. * scripts/post_distupgrade_base_cleanup: Remove pinentry-curses from the stretch (--install-recommends) reference chroot. * scripts/is_testable_uninstallable: - squeeze: Skip and fail install of mini-buildd-{bld,rep}. - jessie->stretch: Skip distupgrade of arden:i386. * scripts/pre_distupgrade_zz_database-server: Improve some upgrade paths. * scripts/post_purge_exceptions: - Restore /usr/lib/sasl2/ disappearing on some jessie->stretch upgrades. * Add/update exceptions for some ancient packages. * master-bin/reschedule_oldest_logs: Expiration age values starting with "+" are relative to the rescheduling age. * master-bin/detect_network_issues: Detect appstream metadata errors. * master-bin/rotate_master_logs: New daily cronjob to rotate and compress (with one day of delay) master logfiles larger than 1 MB. * slave_run: Ensure screen is run with SHELL=/bin/bash. * Makefile: Split build and install targets between master and slave. * Bump debsums dependency to (>= 2.2.2~) for --ignore-obsolete. * Depend on mount, no longer essential. [ Michael Stapelberg ] * helpers/debiman-piuparts-distill: New helper for parsing the output generated from scripts-log-alternatives. (Closes: #863089) * master-bin/distill_alternatives_log: Run debiman-piuparts-distill on the sections that use scripts-log-alternatives. Runs as a daily cronjob. -- Holger Levsen Wed, 30 Aug 2017 15:31:34 +0200 piuparts (0.78) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Refactor internal handling of reference chroot state. - Do not run custom scripts before/after installing fake essential packages. - Only warn on broken symlinks found between installing the dependencies of the package to be tested and installing the package itself. - Keep track of md5sums of the list of available packages in the chroot to recognize mirror pushes during distupgrade tests and outdated --end-meta meta-data. - Mount a new devpts instance on /dev/pts and /dev/ptmx inside the chroot for better pty separation from the host. Mount the current tty onto /dev/console in the chroot. Inspired by pbuilder. - Arguments to the --ignore/--ignore-regexp options (and internal ignored_files/ignored_patterns settings) that are prefixed with a ':' will verbosely log the ignored files if matched. - New option --upgrade-before-dist-upgrade for 2-stage upgrades: 'apt-get upgrade && apt-get dist-upgrade'. - Do not ignore errors while removing packages. - Run panic handlers before aborting due to an unhandled exception. * piuparts.conf: - Use more aliases for mapping between releases and codenames to reduce the amount of changes needed on "moving targets" (like stable22sid) after a stable Debian release. - Set keyring for archived distros to debian-archive-removed-keys.gpg. - Add new suites: [jessie2Xstretch], [jessie2Xstretch-rcmd] for testing two-stage upgrades (apt-get update && apt-get dist-upgrade) from jessie to stretch (without and with --install-recommends). - Enable [stretch2proposed]. (Closes: #864933) - Use --warn-on-debsums-errors for tests ending in stretch. - Rotate aliases for testing=buster and stable=stretch. - Add new suites: [buster], [buster-rcmd], [stretch2buster], [stretch2buster-rcmd] for testing buster. - Add new suites: [stretch2bpo], [stretch2bpo2buster] for testing stretch-backports. - New (per-section) conffile setting: 'exclude-known-problems' can be set to a list of known problem iconffile names that should *not* be used for this section. * piuparts.conf.anbe: Add some more example sections. * distros.conf: Add support for buster-debug, buster-proposed-updates. * piupartslib/packagesdb.py: - Add new state "outdated" to ignore packages if a dependency distro already has a newer version. (Closes: #856846) - Resolve packages in "outdated" state after dependency databases are initialized. * piuparts-slave.py: - Run piuparts on dummy package "TARBALL" (but skip tests) to create tarballs. - Add "chroot-meta-auto" config option to automatically maintain cached target chroot meta data (using --end-meta and --save-end-meta) for distupgrade tests instead of generating it on-the-fly. - Flush pending logfiles upon startup. * piuparts-analyze.py: - Process newest logfiles first, in case of timeouts. - Drop report_packages_with_many_logs(), obsoleted by archiving old logs. - Remove support for magic "$DISTRO/None" versions. * piuparts-report.py: - Plot all states in the graphs. - Support new states in plots (varying number of columns in counts.txt). - Reorder stacking in the bts_stats plot. - Defer rewriting unmodified maintainer and source summaries, refresh them once per month. * custom-scripts: Check for dummy package "TARBALL" to detect tarball creation phase. * scripts/is_testable_uninstallable: - Add exceptions to skip distupgrades of many buggy ancient packages. (Closes: #688250, #696081) * scripts/post_setup_dummy_users: Skip regular users when testing dpsyco*. * scripts/pre_distupgrade_exceptions: - Disable APT::Immediate-Configure on some squeeze->wheezy upgrade paths. - Add exceptions for old buggy packages. * scripts/post_{setup,distupgrade}_zz_backports: Avoid downgrading systemd/udev from jessie-backports to jessie during removal phase. (Closes: #856845) * scripts/post_distupgrade_exceptions: Add stretch->buster workarounds: - Remove empty unowned /etc/dbus-1/, /etc/dbus-1/system.d/. - Change permissions from 0755 to 0700 on /var/lib/polkit-1/. * scripts-broken-symlinks/post_install_extras_unbreak_symlinks: New script for "fixing" some broken symlinks (e.g. by installing selected suggested packages) in tests using --fail-on-broken-symlinks. * scripts-debug-triggers/: New scripts for enabling dpkg trigger debugging. * conf/crontab-*: Set PATH and run scripts without absolute path. * master-bin/detect_archive_issues: - Look at failed logs older than 24 hours (48 hours previously). - Mark logfiles for immediate recycling upon first appearance. - Run twice per day, moving new logs to untestable within 24-36 hours. * master-bin/detect_network_issues: - Report the issues found in each logfile. - Detect alternate wording of gpg verification error. - Detect master and slave disagreement on the "current" version of a package due to using different mirrors and ongoing mirror push. * master-bin/detect_piuparts_issues: - Report the issues found in each logfile. - Recognize more problems caused by a full filesystem. - Recognize more failures when connecting to databases. * master-bin/detect_well_known_errors: Ignore known_problems conffiles listed in 'exclude-known-problems' in piuparts.conf. * Add known_problems/trigger_cycle_error.conf to report trigger cycles and other trigger problems. * Add bug templates for failing to upgrade from stretch to buster. * Update bug templates for long grown upgrades ending in buster. [ Holger Levsen ] * update TODO. Work on buster has begun. \o/ * generate_daily_report.in: only add runtime to external script output if not already included. * master-bin/report_untestable_packages.in: - also report runtime. - only act on sections with precedence < 100. * master-bin/detect_network_issues.in: - dynamically report detected issues. - move check for outdated reference chroot to detect_piuparts_issues. * master-bin/detect_piuparts_issues.in: - dynamically report detected issues. * instances/piuparts.conf-template.pejacevic: - Add new suites to be tested: - stable2sid, upgrading stretch -> sid (skipping testing/buster) to find possible upgrade problems before a package migrates to testing - stable22sid, upgrading stretch -> buster -> sid - oldstable222sid, upgrading from jessie -> stretch -> buster -> sid (Closes: #859620) - Adjust precedences for most of the tested suites, set precedence to 100 for suites which don't change anymore. - Set idle-sleep to 3600 seconds, the results pages are only updated twice a day anyway. * Update README_server.txt. * htdocs/index.tpl: announce new suites. -- Holger Levsen Mon, 31 Jul 2017 13:32:50 -0400 piuparts (0.77) unstable; urgency=medium [ Andreas Beckmann ] * distros.conf: - Add aliases for oldstable and oldoldstable. - Update aliases for stretch being released as stable. * piuparts.py: - Add to ignored_files: + /etc/rc.local + /etc/X11/Xwrapper.config (Closes: #859929) + /var/lib/apt/daily_lock * piuparts-analyze.py: - Do not ignore Ctrl-C. * scripts/post_distupgrade_base_cleanup: Remove (transitional) sysvinit from stretch. (Closes: #860819) * scripts/pre_distupgrade_zz_database-server: Do not migrate manually from mysql-server to default-mysql-server (aka mariadb-server), there is now a transitional mysql-server package in stretch. [ Holger Levsen ] * piuparts-report.py: - update navigation to point to piuparts.conf-template.pejacevic. -- Holger Levsen Sat, 03 Jun 2017 23:08:19 +0200 piuparts (0.76) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /usr/share/texmf/ls-R - Remove from ignored_files: - /usr/share/keyrings/debian-archive-removed-keys.gpg~ - Do not require --scriptsdir for proper operation. (Closes: #855038) - Make install_over_symlink tests fail by default, add --warn-on-install-over-symlink option to demote this to a warning. - Add --install-suggests option. - Fall back to generating the reference chroot state on-the-fly if the --end-meta file is missing. - Make logged command lines shell-safe with pipes.quote(). * piuparts.conf: - Use --warn-on-install-over-symlink for tests ending before stretch. - Enable scripts-log-alternatives for [sid] and [stretch]. (See: #850917) - Add new suite: oldstable22testing, upgrading wheezy -> jessie -> stretch. * piuparts.conf.anbe: Add some more example sections. * distros.conf: Add support for sid-debug, stretch-debug. * piuparts-slave.py: - Replace literal ESC (0x1b) chars with string '[ESC]' for easier copy+paste from logfiles. - Make logged command lines shell-safe with pipes.quote(). * piuparts-analyze.py: - Also process untestable/*.log. * instances/*: Generate the config files piuparts.conf.pejacevic, piuparts.conf.piu-slave-bm-a, piuparts.conf.piu-slave-ubc-01 (new) and piuparts.conf.piu-slave-1und1-01 (unused) from a common template to keep the settings in sync, since they should only differ in mirror and arch. * Clean up leftover /etc/piuparts/scripts/post_setup_experimental. (Closes: #854317) * scripts/post_setup_minimize: - Exclude gcc-4.8-base from jessie chroot. * scripts/pre_install_database-server: Add/fix some database requirements. * scripts/pre_distupgrade_zz_database-server: Switch from mysql-server to default-mysql-server (aka mariadb-server) for selected packages on upgrades to stretch, otherwise they will lose their database server. Try 'dpkg --configure --pending' after failed upgrades in case the database server was started too late - after a client tried to access it. * scripts/pre_remove_40_find_missing_md5sums: Correctly escape backslashes in filenames. * scripts/pre_remove_exceptions: Handle removal of more packages pulling in 'init'. * Add/update exceptions for ancient packages in scripts/is_testable_uninstallable, scripts/pre_install_exceptions, and scripts/post_distupgrade_exceptions. * master-bin/detect_piuparts_issues: Recognize more cases of full disks. * master-bin/detect_network_issues: Recognize unauthenticated packages. * Add known_problems/problems_and_no_force_issue.conf to recognize non-fatal failures when attempting to remove essential packages. * Add bug template for failing to upgrade from testing to sid. * Add bug template for shipping broken symlinks. [ Holger Levsen ] * piuparts.py: - Apply patch from Maxime Lareo to allow usage of --keep-sources-list without --basetgz. (Closes: #841157) - Add to ignored_patterns: + /srv/.* (Closes: #848186) - Add my copyright note according to git blame. * Add python (>= 2.7) to the depends of piuparts-master and piuparts-slave as dh-python has been changed to only add python depends if it finds python code in PATH. Thanks lintian. * Replace httpredir.debian.org with deb.debian.org everywhere: with deb.d.o apt will use SRV records to go directly to the mirror, with httpredir.d.o there will be http redirects for every file. * slave_run: Fix bug which prevented a different number of slaves to be started than defined in the configuration. Also add a check to ensure that this optional argument is an integer and within the defined maximum number of slaves. * slave_stop: New script to stop all piuparts-slaves on a host. * slave_stop.8: Add new manpage for this new script, also update slave_run.8 to mention slave_stop.8. * Update README_pejacevic.txt and README_server.txt. * Drop instances/piuparts.conf.lamarr and …/piuparts.conf.goldwasser. * htdocs/index.tpl: - Announce 2nd piuparts-slave, explain this is a three host setup now, mention that pejacevic is a four core system and other news. - Make "contribute" a link. * archive_old_logs: use xargs -P $NUM_CPU and -n 1 to compress old logs in parallel. * update-piuparts-(master|slave)-setup: touch $PIUPARTS_TMPDIR/.nobackup to exclude it from backups made by DSA. * conf/crontab-master.in: run piuparts-report twice a day instead of trying four times a day, which hasn't worked out for a long time. * generate_daily_report.in: measure runtime of this script and the scripts started by it. * detect_(network|piuparts|archive)_issues.in: measure runtimes too. * reschedule_oldest_logs.in: measure runtime. [ Michael Stapelberg ] * Add support for logging update-alternatives calls using optional custom-scripts/scripts-log-alternatives/ (Closes: #850917) -- Holger Levsen Sun, 12 Mar 2017 17:55:07 +0100 piuparts (0.75) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/network/ + /etc/php/7.0/ and selected subdirs + /var/lib/apache2/ - Run panic handlers in reverse order. - Add --shell-on-error option to simplify debugging of piuparts failures. Starts an interactive shell in the chroot if an error occurred. - Unmount /proc/sys/fs/binfmt_misc only if mounted. - Query the apt version in the chroot. (Closes: #851517) - Use apt_pkg.version_compare(). * piuparts.conf: - Add new suite: stretch-rcmd (test with --install-recommends). - Add new suite: jessie2stretch-rcmd (test with --install-recommends). - Add new suite: stretch-security. - Add new suite: stretch-pu (test packages in stretch-proposed-updates). - Add new suite: stretch2proposed. - Reduce delay before rescheduling untestable packages. * piuparts-analyze.py: - Adjust debianbts syntax to be compatible with both jessie and stretch. - Improve exception handling. * Handle more database setup issues. * Add/adjust some exceptions for ancient packages. * scripts/pre_install_extras: Add examples how to preseed license acceptance for non-free and downloader packages. * scripts/pre_distupgrade_zz_database-server: Run pg_upgradecluster during dist-upgrades. * scripts/post_distupgrade_hack_debsums: Limit to squeeze->wheezy and merge into scripts/post_distupgrade_exceptions. * scripts/{pre,post}_remove_exceptions: Support removal of essential packages with apt-get from stretch onwards. * known_problems/piuparts-depends-dummy_error.conf: Recognize the failure from 'apt-get install piuparts-depends-dummy.deb'. * known_problems/problems_and_no_force_error.conf: Recognize the apt 1.1 message on attempted removal of essential packages. * master-bin/detect_piuparts_issues: Recognize more cases of full disks. -- Holger Levsen Wed, 25 Jan 2017 13:19:30 +0100 piuparts (0.74) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/modprobe.d/ - Add new class of is_testable_* custom scripts: A return value of '1' makes the test succeed without actually installing the package. Use for packages that cannot be tested with piuparts by design or are broken in an unfixable way. (Closes: #682739) - Ensure /etc/resolv.conf in the chroot has permissions 0644 to match a possible recreation of that file by resolvconf. (Closes: #660350) - Increase output limit to 8 MB. (Closes: #794331) - Increase single command time limit to 60 minutes. * piuparts.conf: - New section [sid-strict], testing with --install-remove-install and failing on leftover files. (Closes: #604406) - Only warn on leftover files for [sid] and use the same settings for both [sid] and [stretch], since these are now used by britney to block testing migration in case of regressions. * piuparts.conf.anbe: Add some more example sections. * piuparts-slave.py: - Increase test timeout to 90 minutes. * piuparts-analyze.py: - Also consider bugs filed against source packages. * piuparts-reports.py: - Mark arch:all packages with an :all suffix in package lists. - Report skipped tests. * Makefile: Add check-whitespace target looking for trailing whitespace etc. * scripts/pre_install_database-server: Use default-mysql-* from stretch onwards. * scripts/post-{install,purge}-exceptions: Wait for dkms progress indicator to terminate. * scripts/post_distupgrade_exceptions: Work around some jessie->stretch issues. * scripts/is_testable_uninstallable: New custom script to exclude some uninstallable (or similarly broken) packages from being tested. * scripts/post_setup_minimize: New custom script to minimize chroots. - Remove tzdata and lsb-base from sid and stretch. * scripts/post_setup_dummy_users: New custom script to create dummy groups and users that block ids that are likely being used by the host system. * scripts/post_{setup,distupgrade}_experimental: Removed, obsolete since restoring the initial package selection supports downgrades (0.64). * custom-scripts/scripts-debug-{remove,purge}/: Improved debugging of maintainer scripts during package removal. * Overhaul some more custom scripts. * known_problems/debsums_mismatch_*.conf: Report missing symlinks and directories. * known_problems/needs_rebuild_issue.conf: Report "called emacs-package-install as a new-style add-on, but has no compat file" * known_problems/test_was_skipped_issue.conf: New known problem for skipped tests. * master-bin/detect_network_issues: Detect "E: The repository '.*' does no longer have a Release file." * master-bin/detect_piuparts_issues: Detect 'invoke-rc.d: initscript mysql, action "restart" failed.' * master-bin/reschedule_piuparts_tests: New experimental helper script for rescheduling piuparts tests based on detected known problems. [ Holger Levsen ] * Apply patch by Antonio Ospite to workaround #847788 by force-unmounting /proc/sys/fs/binfmt_misc - some java packages trigger update-binfmts, which mounts /proc/sys/fs/binfmt_misc but never unmounts it. (Closes: #704037) - Thanks Antonio! * Add build-depends to dh-python, thanks lintian. * Bump copyright years. [ Sean Whitton ] * piuparts.py: use `apt-get install ./foo.deb` instead of `dpkg -i ./foo.deb && apt-get install -yf` for more reliable dependency resolution. (Closes: #825487) -- Holger Levsen Sun, 15 Jan 2017 11:59:00 +0100 piuparts (0.73) unstable; urgency=medium * Add new option, --hard-link, and do *not* use it by default. (Closes: #841025) -- Holger Levsen Tue, 06 Dec 2016 18:14:19 +0100 piuparts (0.72) unstable; urgency=medium [ Holger Levsen ] * instances/piuparts.conf.pejacevic: test jessie2bpo and jessie2bpo2stretch. * piuparts.py: - Fix devpts mounting to use the correct gid=5,mode=620 mount options. Patch taken from Adam Conrad. (Closes: #829146) - explain (as a code comment) why /root/.rnd shouldn't be ignored. (Closes: #750099) - Add /var/log/apt/eipp.log.xz to ignored files. (Closes: #830527) * custom-scripts/scripts/pre_remove_exceptions: force removal of usrmerge package. (Closes: #798374) [ Stefano Rivera ] * Don't test the current Debian release status, tracking that is distro-info-data's problem. (Closes: #827411) -- Holger Levsen Wed, 03 Aug 2016 11:24:56 +0200 piuparts (0.71) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Define namedtuple FileInfo globally s.t. it can be pickled. - New option --no-check-valid-until to set apt option Acquire::Check-Valid-Until=false for testing archived releases. - Clear all locale settings from the environment before running commands in the chroot. * piuparts.conf: - Use --no-check-valid-until for squeeze. - New section [wheezy-security] (also used by wheezy LTS). - New section [jessie-security]. * distros.conf: squeeze-* has been archived/removed. * piuparts-analyze.py: - Skip logfile on HTTPError while querying the BTS. * piuparts-reports.py: - Skip section on HTTPError while downloading Packages. * post_setup_forbid_home: Adjust for basefiles now shipping /home/ * Add/adjust some exceptions. [ Holger Levsen ] * Bump standards version to 3.9.8, no changes needed. -- Holger Levsen Thu, 05 May 2016 14:03:30 +0200 piuparts (0.70) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/sysctl.d/99-sysctl.conf (dangling symlink to ../sysctl.conf) + /var/log/btmp - Canonicalize mountpoint inside the chroot to avoid mounting over symlinks and possibly outside the chroot. - --install-remove-install runs 'apt-get install --reinstall $pkg', too. * post_distupgrade_base_cleanup: Remove libprocps3 in stretch. * pre_remove_40_find_unowned_lib_links: Handle another mpi alternative. * Add bug template for maintainer scripts incorrectly complaining about being called with unknown arguments. [ Holger Levsen ] * conf/distros.conf and instances/piuparts.conf.pejacevic: move squeeze to archive.debian.org, stop testing squeeze only suites - but keep testing upgrades to wheezy suites. * Fix typo in piuparts_slave_join.8.txt, thanks lintian. -- Holger Levsen Tue, 08 Mar 2016 13:00:22 +0100 piuparts (0.69) unstable; urgency=medium * piuparts.py: add to ignored_files: - /var/lib/pkcs11proxyd/ (Closes: #810703) * piuparts.1.txt: Fix typos, thanks to Дилян Палаузов and lintian. * debian/control: - Bump standards version to 3.9.7, no changes needed. - Use more secure https URL for Vcs-Git: header, thanks lintian. - Use /git/ not /cgit/ in Vcs-* headers. * debian/copyright: bump years for my contributions to 2016. -- Holger Levsen Sat, 13 Feb 2016 12:46:10 +0100 piuparts (0.68) unstable; urgency=medium * Dedicated to the memory of Ian Murdock. Thank you very much for starting and shaping Debian, Ian! The world would be very different today without your work and you will never be forgotten. [ Andreas Beckmann ] * piuparts.py: - Simplify mounting/unmounting into the chroot. - Mount a 64 MB tmpfs onto /dev/shm in the chroot. (Closes: #793487) [ Sven Haardiek ] * Add option "--single-packages" to test every package specified via path or package name individually. (Closes: #810237) [ Michael Prokop ] * piuparts.py: - Execute dist-upgrade also when using --existing-chroot option. (Closes: #798266) -- Holger Levsen Tue, 02 Feb 2016 10:33:40 +0100 piuparts (0.67) unstable; urgency=medium [ Holger Levsen ] * Add wheezy-pu to the suites being tested. (Closes: #800094) [ Andreas Beckmann ] * piuparts.py: - Ensure /etc/mtab exists in the chroot. - Record and compare file owner and group names instead of uid/gid. * Add support for adequate tag ldd-failure. (Closes: #793108) * scripts/post_distupgrade_exceptions: - Fix the /etc/nsswitch.conf wheezy -> jessie upgrade handling. * Update custom scripts handling databases. * Add bug template for wheezy -> jessie -> stretch upgrades. -- Holger Levsen Sat, 17 Oct 2015 16:32:06 +0200 piuparts (0.66) unstable; urgency=medium * Add python-lzma to build-depends, as the tests need it now. -- Holger Levsen Wed, 26 Aug 2015 00:59:42 +0200 piuparts (0.65) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Create sources.list entries for --testdebs-repo with '[ trusted=yes ]' to avoid needing --do-not-verify-signatures which would act globally. - Add --fake-essential-packages option to include packages in the base chroot that are not removed after the test. These are available during purge and for checking against mistreatment. - Increase permitted command output volume from 3 MB to 4 MB (for daptup). - Only ignore 'dpkg -i' failures that can be fixed by 'apt-get -f install'. (Closes: #794628) - Set Architecture of piuparts-depends-dummy to match the package to be tested. * piuparts.conf.anbe: Add some more example sections. * piupartslib/packagesdb.py: - Put package in 'dependency-does-not-exist' state if a pass/ log exists but a dependency has been removed from the archive since then. - Prepare improvements for handling foreign arch dependencies. * piupartslib/open_packages_url(): Support xz-compressed and uncompressed Packages files. * piuparts-master-backend.py: - Catch URLError, log it and abort without backtrace. * piuparts-common: Add Depends: python-lzma. * scripts/pre_remove_40_find_unowned_lib_links: Ignore some symlinks caused by mpi-default-dev and libglide2/libglide3. * Add/adjust some exceptions for ancient packages. -- Holger Levsen Tue, 25 Aug 2015 08:23:51 +0000 piuparts (0.64) unstable; urgency=medium [ Holger Levsen ] * Add FancyIndexing to piuparts-master.conf and enable apache syntax highlighting in vim. (Thanks DSA!) * Use httpredir.debian.org instead of http.debian.net. (Closes: #783617) * Add support for adequate tag missing-pkgconfig-dependency. (Closes: #783528) * Use revdate: in the asciidoc headers of manpages instead of DATE in the body to achieve reproducible builds and set the revdate: to the last modification date of each manpage. [ Andreas Beckmann ] * piuparts.py: - Use 'dpkg-query -W' instead of 'dpkg --get-selections' to record architecture and version information for installed packages, too. - Restore the selections to the package versions from the reference chroot in scenarios like sid+experimental where more than one version could be available. Use --force-yes since this may involve downgrades. * piuparts.conf: - New global setting: basetgz-sections, used by piuparts-slave only. - Switch to separate tarball sections. * piuparts-slave.py: - Add support for special sections that only create/refresh reference basetgz tarballs without testing packages. The basetgz can be shared between multiple sections without being affected by their flags. -- Holger Levsen Fri, 12 Jun 2015 13:42:52 +0200 piuparts (0.63) unstable; urgency=medium [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/clamav/ + /etc/lighttpd/ + /etc/lighttpd/conf-available/ - Increase verbosity when reporting things installed over directory symlinks: report all symlinks and their targets. - Reinstall base system packages missing upon removal with --no-install-recommends. - Mount /dev/pts into the chroot. Wanted by recent apt. - Fix usage of schroot with chroot names without ':'. (Closes: #773874) - Resolve distro aliases before setting PIUPARTS_DISTRIBUTION. - Fix running debootstrap if --do-not-verify-signatures is given. (Closes: #781673) * piuparts.conf: - Enable --warn-on-debsums-errors for all jessie tests. - Add new suite: jessie-pu (test only packages in jessie-proposed-updates). - Add new suite: jessie-rcmd (test with --install-recommends). - Add new suite: wheezy2jessie-rcmd (test with --install-recommends). - Add new suite: stretch. - Add new suite: jessie2stretch. * piuparts.conf.anbe: Add many more example sections. * distros.conf: Reactivate 'testing' and 'stable' aliases. * piupartslib/packagesdb.py: - Consider logfile age in addition to the time waiting for recycling. * piuparts-slave.py: - Handle Ctrl-C while downloading Packages files and flush/unreserve, too. - Catch another exception that may happen while talking to the master. * detect_archive_issues: Move archive_issues.txt from htdocs/ to master/. * detect_piuparts_issues: Detect transient apt-cache failures. * generate_daily_report: Remove obsolete .html files (e.g. summaries for removed source packages or obsolete maintainer addresses) after 30 days. * reschedule_oldest_logs: - Fix gradual oldest-first recycling if nearly all logs are candidates. - Recycle/expire pass/ logs only if the section is (recycle-)idle. * scripts-apt-first/pre_distupgrade_zz_apt_first: New custom script for upgrading apt first and having the new apt perform the remaining upgrade. * scripts-sysvinit/pre_distupgrade_zz_sysvinit: New custom script for doing wheezy->jessie upgrade tests while keeping sysvinit as init if possible. * scripts/pre_remove_40_find_unowned_lib_links: Ignore /lib/ld-lsb.so.[123] links created by lsb-core. * scripts-no-usr-share-doc: Add exception for localepurge. * post_distupgrade_base_cleanup: New custom script for generalized libdbX.Y exceptions. * Update exceptions for squeeze-lts. * Add/update exceptions for many packages. * Add/update some bug templates. * known_problems/{initdscript_lsb_header_issue,insserv_error}.conf: Report "service.*already provided", too. [ Holger Levsen ] * piuparts.conf.pejacevic: add new suite: jessie2proposed. * piuparts.conf.piu-slave-1und1-01: merge changes from pejacevic config but replace amd64 with i386. * crontab-master.in: update webpages every six hours. * piuparts-reports.py: - Link to the new "Debian Package Tracker" (tracker.debian.org) instead to the old "Package Tracker System" (PTS). - Switch links to lintian.debian.org to https. * crontab-slave.in: try to start slave every hour to make sure it's always running. * Use "autopep8 --max-line-length=160" to re-format all .py files, then also use "-a" on a 2nd run. * test_config.py: update for Jessie being stable and Stretch being the new testing suite. * debian/copyright: Remove space from W3C FreeUse licence name, thanks lintian. * Update piuparts.py, piuparts.1.txt and README_server.txt due to the fact that jessie is the new stable and testing is stretch now. -- Holger Levsen Mon, 27 Apr 2015 16:46:29 +0200 piuparts (0.62) unstable; urgency=medium * Default to http://http.debian.net/debian instead of the deprecated http://cdn.debian.net service as default mirror. (Closes: #774471) -- Holger Levsen Tue, 06 Jan 2015 10:36:39 +0100 piuparts (0.61) unstable; urgency=medium [ Michael Prokop ] * Support new scripts directive 'post_chroot_unpack', useful for adding apt-keys for custom apt repositories. (Closes: #767485) * Add custom-scripts/scripts-unused-examples/post_chroot_unpack_key_setup. [ Holger Levsen ] * piuparts.py: Add "/var/cache/apt/archives/partial/" to ignored_files. (Closes: #767489) * piuparts.py and piuparts-report: Detect four more (in)adequate issues: - broken-binfmt-detector - broken-binfmt-interpreter - missing-alternative - program-name-collision (Closes: #767499) * distros.conf: Remove unneeded URI for squeeze-lts, it just uses the default repository. * instances/pejacevic+slaves: Do not reschedule passed squeeze logs anymore. [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/modules-load.d/modules.conf (dangling symlink to ../modules) - Place an up-to-date copy of the host's /etc/resolv.conf (with comments stripped and whitespace normalized) in the chroot. * piuparts.conf: - Add flags-start-jessie, flags-end-jessie global variables. - Use --scriptsdir .../scripts-jessie for tests ending in jessie. - Enable --warn-on-debsums-errors for wheezy2jessie upgrades. * piuparts.conf.anbe: Add many more example sections. * distros.conf: - Add *updates, *backports, *proposed for jessie. - Switch 'testing' alias to 'stretch', but deactivate it for stable. - Add 'stable' alias pointing to 'jessie', deactivated as well. * piupartslib/dwke.py: - Fix parsing of known_problems/*.conf that have EXCLUDE_PATTERN. * piuparts-slave.py: - Avoid a busy loop by suspending a section for 5 minutes if basetgz creation is in progress or failed. * Add "AddDefaultCharset utf-8" to the apache2 configuration snipplet example, so that logfiles are displayed as UTF-8. * master-bin/detect_piuparts_issues: Another partial write error from tar. * pre_install_foreign_architecture: New script to enable foreign architectures for testing cross toolchain packages. * pre_remove_40_find_unowned_lib_links, unowned_lib_symlink_*.conf: New script and reports for unowned symlinks in [/usr]/lib[/]. * post_distupgrade_exceptions: Handle libdb5.1 removal in jessie. * scripts-jessie/post_setup_jessie-fake-essential: New custom script. - Make ucf fake-essential. (ucf may be used unconditionally during purge.) - Make init-system-helpers fake-essential. (Systemd service link cleanup.) * Rework the file-rc/sysvinit/upstart/systemd specific exceptions. * Remove debugging output for resolvconf. * Update custom scripts handling databases, experimental, ... and some more. * Update bug templates. -- Holger Levsen Wed, 03 Dec 2014 19:55:26 +0100 piuparts (0.60) unstable; urgency=medium [ Leo Iannacone ] * piuparts.py: when using the --existing-chroot option, instead of copying files, use hard links if the original chroot and destination directory are on the same device. (Closes: #754878) [ Iain Lane ] * Use python-distro-info to get the Debian and Ubuntu devel releases. * tests: Remove the Ubuntu devel release test. [ Holger Levsen ] * Update README_server.txt and README_pejacevic.txt after setting up a piuparts master/slave system from packages myself. * Drop conf/piuparts-master.sudoers from debian/piuparts-master.examples and include all commands piupartss needs to run as root in conf/piuparts-slave.sudoers. * Update instances/piuparts.conf* to match what is in use for piuparts.debian.org today. Add two new systems, goldwasser and lamarr, as examples taken from piuparts development. * Add "AddType text/plain .log" to the apache2 configuration snipplet example, so that logfiles are displayed in the browser. * Bump standards version to 3.9.6, no changes needed. * Replace all occurrences of old gitweb Alioth URLs with cgit ones. -- Holger Levsen Sun, 26 Oct 2014 20:07:24 +0100 piuparts (0.59) unstable; urgency=low [ Holger Levsen ] * piuparts.py: - mount selinuxfs read-only and on new location when possible. Thanks to Laurent Bigonville for the patch! (Closes: #682068) - Add to ignored_files: + /etc/sub{g,u}id[-] introduced by shadow 1:4.2-1. * piuparts-report.py: - create https URLs in source_summary_url(). This should probably be made configurable. - fix log directory creation. (Closes: #746722) - fix spelling of "ditto". (Closes: #746724) - create graph of rc and non-rc bugs filed (and fixed) since the beginning of piuparts testing and display it on piuparts.d.o welcome page. - move counts.txt from htdocs to master directory. * master-bin/gather_bts_stats: differate between rc and non-rc bugs. * Makefile: - Don't install crontabs and sudoers file anymore, except when installing from git. - Also cleanup python bytecode from tests. * Packaging: - install cron files for piupartsm and piupartss users in /usr/share/doc/piuparts-(master|slave)/examples/ (Closes: #746721) - split sudoers.d into master and slave and provide them in /examples/ too. - install piuparts.conf.sample in /usr/share/doc/piuparts-master/examples/ - install slave_run and slave_join in /usr/bin/ (Closes: #746718) - piuparts-slave.postinst: do not attempt to add the slave user's ssh public key to the master user's authorized key. The slave package should maybe recommend the master package, but definitly not pre-depend on it, which would be a requirement to make this work. So document this instead. * Updates various pieces of documentation, rename README_1st to README. Explicitly state what will break when using cdebootstrap with pbuilder. (Closes: #709348) * Update various custom-scripts (usually to workaround problems in other packages). * Switch all links to bugs.debian.org, www.debian.org, wiki.debian.org, packages.debian.org, anonscm.debian.org and lists.debian.org, db.debian.org, qa.debian.org, lists.alioth.debian.org and udd.debian.org to https. [ David Steele ] * dependencyparser.py - Be more lenient about the multi-arch modifiers accepted and stripped by the parser. * piuparts-report.py: - Fix indentation of generate_global_summary() - Refactor sections_by_precedence() out of generate_global_summary(). - Restore (a copy of) the counts.txt file to the web view, in
/counts.txt. - Also add a link to counts.txt to the section page. * Remove oldstable distribution results from the JSON summary. - Also remove stable distribution results. -- Holger Levsen Mon, 14 Jul 2014 13:14:19 +0000 piuparts (0.58) unstable; urgency=low [ Holger Levsen ] * debian/copyright: update years. * piuparts.py: - ignore multiarch definition in_default_debian_mirrors() (Closes: #711370) - Add to ignored_files: + /etc/machine-id (Closes: #619244) * custom-scripts: add debugging output for resolvconf. (See #660350) * Adapt and rename conf/piuparts.apache to piuparts-master.conf as per apache2.4 transition. (Closes: #712384) * piuparts.1.txt: explain that --do-not-verify-signatures also causes APT::Get::AllowUnauthenticated to be set accordingly in the chroots. * Switch ubuntu default to 'trusty'. (Closes: #744848). Then switch it to 'utopic'... * piuparts-report: Add link to ci.debian.net to navigation menu. * Replace http://piuparts.debian.org with https everywhere. Thanks to DSA for all their support running piuparts.debian.org and for everything else! [ Andreas Beckmann ] * piupartslib/packagesdb.py: - Add missing PackagesDB.get_version(package_name) method, needed by piuparts-report. - Add PackagesDB.get_source(package_name) method, for easy lookup of source package names. * piuparts-report.py: - Improve integration of known problem reports generation. * master-bin/detect_well_known_errors.py: - Classify unclassified failures as unclassified_failures.conf. [ David Steele ] * piuparts-report.py: Store test summary.json output at the section and global level, using a new piupartslib/pksummary.py module. The new Config.get_std_distro() provides a standard default distro name for summaries. (Closes: #740386) * Add python-distro-info to piuparts-common's depends and to build-depends as it's used in the tests too. -- Holger Levsen Thu, 01 May 2014 00:23:42 +0200 piuparts (0.57) unstable; urgency=low [ Ivo De Decker ] * piuparts-report.py: Use source version from final_distro. (Closes: #733900) [ Matthias Schmitz ] * Rearrange old unittests.py, use nosetests instead: - Move unittests.py to tests/ subdirectory, - Split out dependencyparser test to an extra file, - Add configuration for nosetests, - Use nosetests in Makefile target "check". * piuparts.py: - Add unittest for DefaultsFactory / Defaults. - Use pymox to mock the piuparts.guess_flavor() function as it would run lsb_release in a subprocess. - Move broken_symlink unittest out of piuparts.py into tests/test_piuparts.py so nosetests can run this test and detect coverage. * Add tests/README documenting the naming of the test classes and functions. * Re-enable unittests during package build - Don't create coverage report during pkg build. - Add to Build-Depends: python-debian, python-apt, python-nose, python-debianbts, python-yaml, and python-mox3. [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/group.org + /etc/passwd.org + /etc/shadow.org - Create piuparts-depends-dummy.deb with gzip compression for installability with older dpkg versions. - Exclude virtual packages provided by the real package (but not the dummy package) from the Conflicts of piuparts-depends-dummy.deb. - Check the status of the packages after installing the supplied .debs. Based on an initial patch by Julien Cristau. (Closes: #718544, #736743) * piupartslib/packagesdb.py: - load_alternate_versions_from_packages_urls(): New method to load alternate version numbers from a different set of URLs (e.g. final_distro). - get_best_package_state(): New method that considers successfully tested providers of a virtual package if the real package of the same name failed. Used for dependency resolution. - No longer special-case packages as essential-required and test them like normal packages (no-op test plus adequate run). (Closes: #735907) - Improve ordering of packages to be tested/recycled. * piupartslib/dwke.py: - Factored out classes Problem and FailureManager and some helpers from detect_well_known_errors.py * piuparts-master-backend.py: - Improve master.log verbosity. * piuparts-report.py: - Mark essential-required as obsolete and hide it. - Integrate the reporting part from detect_well_known_errors.py. - Archive outdated logs even if the current version is untested. * master-bin/detect_well_known_errors.py: - Only update .kpr files, the reporting part was integrated into piuparts-report. * scripts/post_{setup,distupgrade}_force-unsafe-io: New custom script to automatically enable dpkg --force-unsafe-io once that is supported. Also divert /bin/sync and replace it with a link to /bin/true. Syncing a throwaway chroot is useless, but affects on the host's performance. * scripts/pre_install_extras: New custom script to install a linux-headers package before testing a *-dkms package, s.t. dkms tries to build a kernel module. * known_problems/module_build_error_issue.conf: New known problem for dkms module build failures. * known_problems/piuparts-depends-dummy_{error,issue}.conf: New known problem for failing to install and fixup piuparts-depends-dummy.deb. * Add more piuparts exceptions. * Handle lib32nss-mdns like ia32-libs[-gtk]. [ Holger Levsen ] * piuparts.py: - Report adequate version used. -- Holger Levsen Fri, 14 Feb 2014 13:37:23 +0100 piuparts (0.56) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /etc/apt/apt.conf.d/01autoremove-kernels - Use temporary files while creating basetgz and rename afterwards. - Remove the package to be tested separately before removing its Depends and Recommends. - --allow-database now also enables firebird2.5-super. * piuparts.conf: - New per-section settings: + components * piupartslib/__init__.py: - open_packages_url(): Try Packages.gz if Packages.bz2 is not available. (Closes: #711157) - Add new class DecompressedStream. - open_packages_url(): Use DecompressedStream to transparently decompress a Packages.{bz2,gz} file while downloading and reading the content without requiring to store the complete file (compressed and/or decompressed) in memory. * piupartslib/packagesdb.py: - PackagesFile: Allow restricting to a set of package names. * piuparts-slave.py: - Use locking to prevent concurrent basetgz creation. (Closes: #708287) - Use the "components" setting to restrict the archive areas being used in the generated sources.list. - Reduce memory footprint by discarding unneeded Packages file parts. * piuparts-report.py: - Report URLs of all Packages files used for a section. - Avoid reporting duplicate dependencies after stripping versioning. - Fix some invalid HTML constructs. - Sort maintainer summary by state: failing packages on top. * Improve exception handling. * known_problems: Upgrade adequate issue 'missing-copyright-file' from boring to normal and make pre_remove_50_find_missing_copyright no longer generate fatal errors. * Add support for new adequate tags missing-symbol-version-information and symbol-size-mismatch. * reschedule_oldest_logs: Report number of logfiles in recycle/. * Add more piuparts exceptions to work around various failures in multiple packages. Add a known_problem report for packages using exceptions. * bug-templates: Recommend to use the new dpkg-maintscript-helper commands dir_to_symlink/symlink_to_dir that were added in dpkg 1.17.2. (Closes: #720185) [ Holger Levsen ] * piuparts-report.py: use descriptive link targets for (in)adequate issues instead of mere tag names, thanks to Jakub Wilk for the wordings. * Bump standards version to 3.9.5, no changes needed. * Bump compat level to 9, thus build-depend on debhelper > 9.20120909~. -- Holger Levsen Mon, 16 Dec 2013 12:59:14 +0100 piuparts (0.55) unstable; urgency=low [ Ivo De Decker ] * piuparts-report.py: Add sources.yaml and section.yaml exports. [ Holger Levsen ] * Add file CONTRIBUTING, thanks to Vincent McIntyre for it! (Closes: #721348) * piuparts-report: Add "contact us" link. * piuparts.py: Explain at the very beginning of the generated log that contact information is part of the FAQ. [ Andreas Beckmann ] * Add bug templates for wheezy. * scripts-leftovers: Improve cleanup in /var/lib/systemd with fake-essential init-system-helpers. (Closes: #719869) -- Holger Levsen Thu, 03 Oct 2013 15:31:38 +0200 piuparts (0.54) unstable; urgency=low [ Holger Levsen ] * known_problems: fix some adequate patterns. * piuparts-master: recommend apache2 | httpd. * README_server.txt: be more verbose about configuring sudo. (Closes: #711155) * piuparts.py and piuparts-reports.py: add "incompatible-licenses" (detected since adequate 0.7) and "ldd" to inadequate_tags and known_problems. [ David Steele ] * piupartslib/packagesdb.py - The dependency calculation for rdeps is reworked to match that used elsewhere, to allow package reserve() to drive waiting-for-dep to drive to 0 more quickly. (Closes: #705977) - Reworked rdep metrics (block_count, rrdep_count, waiting_count) to calculate on-demand. Access methods are moved from Package to PackagesDB. - Add a metric for the worst-case reverse-dependency chain length. - Weight the reserved packages returned by this additional factor. [ Andreas Beckmann ] * piuparts.py: add to ignored_files: /etc/hosts -- Holger Levsen Tue, 30 Jul 2013 11:05:32 +0200 piuparts (0.53) unstable; urgency=low [ Andreas Beckmann ] * *.py: Reindent and reformat, slowly moving towards PEP 8 style. * piuparts.py: - Do not run adequate checks on possibly disappearing packages. - Add --allow-database option that enables starting MySQL and PostgreSQL database servers in the chroot. (Closes: #662633) - Relax the check for running processes if --allow-database is given. * distros.conf: - Add "candidates" key to merge multiple Packages files into one virtual distro. - Define wheezy-proposed and squeeze-proposed. * piupartslib/conf.py: - Add get_{packages,sources}_urls() that resolve "candidates". * piupartslib/packagesdb.py: - Add load_packages_urls() methods. * piuparts-{master-backend,slave,report}.py: - Switch to load_*_urls(get_*_urls()) for loading Packages/Sources files, adding support for arbitrarily composed virtual distros. * Makefile: - Stop installing python modules in $libdir/python2.6/dist-packages. * debian/control: - Bump Build-Depends: python (>= 2.7). - piuparts-slave: Add Depends: sudo. (Closes: #710350) * New known problem: Packages with missing or incomplete .md5sums files. * piuparts-master.deb: Ship some piuparts.conf examples. * detect_piuparts_issues: Detect problems starting MySQL or PostgreSQL. * pre_install_database-server: New script to install a database server for packages that require one during postinst but only recommend or suggest (or maybe not even this) the server package to allow for remote db servers. * pre_distupgrade_zz_database-server: New script to upgrade (running) database servers before all other packages. This ensures the database server is running again before the dependent packages (that may declare 'insufficient' dependencies on a 'local' database server) are upgraded. [ David Steele ] * Update python Requires to 2.7 (for argparse) * Edit piuparts_slave_* man pages for more information and format. * Specify the manpage doctype for AsciiDoc man sources. [ Holger Levsen ] * piuparts.1.txt, README.txt and README_pejacevic.txt: proof-read and cleaned up. * Move master-slave related documentation from README.txt to README_server.txt and leave a pointer in the former. * Rename README.txt to README_1st.txt. * piuparts-master.deb: - Ship README_server.txt. - Recommend devscripts and libsoap-lite-perl (for gather_bts_stats). * piuparts-report.py: Add a link to README_server.html. * master-bin/gather_bts_stats: write bts_stats.txt to master-directory. * conf/crontab-master: run gather_bts_stats daily. * master-bin/prepare_backup: backup bts_stats.txt and do not backup sources.txt. * piuparts.py: categorize adequate results using inadequate_tags, boring_tags, ignored_tags and unknown. Show adequate exit code if non-zero. * Add known_problems/*inadequate*.conf for all adequate tags we care about. * htdocs/index.tpl: Improve description about piuparts.debian.org. (Closes: #700848) [ Andrew Starr-Bochicchio ] * piuparts.py: - Move default keyring to distro defaults. (Closes: #710346) - When distro defaults are explicitly specified, use the distro default mirrors (Closes: #710347). -- Holger Levsen Sun, 09 Jun 2013 12:57:52 +0200 piuparts (0.52) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Really disable PDiffs. (Closes: #708241) * piuparts-master.in: - Use timeout to prevent master from getting stuck. (Closes: #701618) * piuparts-slave.py: - Report upgrade tests involving downgrades as untestable. - Always pass --arch to piuparts. * piuparts-analyze.py: - Consider archived bugs, too. (Closes: #669966) * piuparts-report.py: - Switch from python-rpy to python-rpy2. (Closes: #705975) - rpy2: Use a noninteractive grDevice. * Makefile: - Switch default target from 'install' to 'build'. * scripts-no-usr-share-doc/post_setup_disable_usr_share_doc: New custom script to instruct dpkg to skip installing most files in /usr/share/doc. This catches maintainer scripts relying on /usr/share/doc content. * New bug template: postinst_uses_usr_share_doc. * piuparts-master.deb: - Switch Depends: python-rpy to Depends: python-rpy2. [ Holger Levsen ] * piuparts.py: - Run adequate (if installed) after installation test. New options --no-adequate and --fail-if-inadequate. (Closes: #703902) - Workaround #709372 and ignore broken symlinks found by adequate. - Do not print a "Guessed: debian" line on every run. It's the default anyway. (Closes: #703058) * piuparts-report: - Add templates for known_problems/inadequate_issue and inadequate_error. - Add link to the piuparts source at the top of the navigation menu on the left. - Add links to Debian QA group and jenkins.debian.net at the bottom of that navigation menu. * debian/control: Add adequate to piuparts' recommends. * piuparts-slave.postrm: Also remove /var/cache/piuparts/tmp on purge. * conf/crontab-master: Run generate_daily_report twice a day. * piuparts-slave.deb: Add piuparts_slave_run.8 and piuparts_slave_join.8 manpages. -- Holger Levsen Mon, 27 May 2013 18:10:43 +0200 piuparts (0.51) unstable; urgency=low [ Andreas Beckmann ] * Mark all python code as utf-8 encoded. * piuparts.py: - In install-purge tests support installing extra packages before the package to be tested via --extra-old-packages. - Add --arch option to run the test for the requested instead of the default architecture. - Disable using PDiffs in order to save CPU instead of bandwidth to a local mirror/proxy. (Closes: #708241) * piuparts.conf: - New global settings: + slave-count - Use installation-dependent defaults for known-problem-directory. * distros.conf: - Update backports setup after integration into the regular archive. - Enable stable security updates when installing stable backports. - Set testing = jessie and add jessie-proposed-updates. * piupartslib/conf.py: - Raise MissingSection if the requested section is not in piuparts.conf. - Improve handling of virtual distros. * piupartslib/packagesdb.py: - get_package(): Add flag resolve_virtual, disabled by default. * Rename piuparts-master.py to piuparts-master-backend.py * piuparts-master: New. - Wrapper shell script to simplify calling the master from the slave. - Get the 'master-directory' from piuparts.conf and chdir there. - Get the (global) 'log-file' from piuparts.conf and handle the error logging. No longer uses the section specific logfile name for errors. * Allow the master-command to be set in ~piupartsm/.ssh/authorized_keys to restrict ssh keys to only run piuparts-master. * piuparts-master-backend.py: - Raise CommandSyntaxError on unknown commands. - Add "section" command: + Make section selection part of the master-slave protocol. + Allow switching sections without reconnecting. + No longer accept section argument on the command line. + Switch the logfile after switching sections. * piuparts-slave.py: - Fix parsing 'idle' return value from master. - Let the piuparts-master (wrapper script) handle chdir and error logging. - Skip sections that don't exist in piuparts.conf and continue to run. - Pass the section to piuparts-master via the new "section" command instead of using a command line argument. - Keep the connection to the master open while switching sections. This reduces the number of ssh connections attempted while checking idle sections. Close the connection before processing packages, before going to sleep, and after communication errors. - Support empty master-command if the command is set in master's .ssh/authorized_keys file. - Change default basetgz name to {DISTRO}_{ARCH}.tar.gz. * piuparts-report.py: - Skip sections that don't exist in piuparts.conf. - "Missing" binary packages don't influence the overall test result of a source package. These are either udebs or for other architectures. (Closes: #699160) - Report the correct version for binary packages. * archive_old_logs: - Compress the archived logs. * detect_well_known_errors: - Skip sections that don't exist in piuparts.conf. * piuparts-master.deb: - Add Depends: xz-utils. * piuparts-slave.deb: - Restrict the ssh key added to master's authorized_keys to only run piuparts-master. * slave_run: Support starting multiple concurrent slaves. (Closes: #678725) * Update and add new exceptions for buggy packages. * scripts-wheezy/post_setup_wheezy-fake-essential: New custom script to suppress some purge failures in wheezy. * post_{setup,distupgrade}_squeeze-backports: New custom scripts to improve support for squeeze-backports. * New known problem: Database setup failure (issue). [ David Steele ] * detect_well_known_errors: - Replace the bash script with an equivalent Python script. - Sort known error and issue packages by reverse dependency count, separating 'fail' from everything else. (Closes: #698526) - Add a PTS link to issue and error summary entries. - Replace the known_problem COMMAND with PATTERN and EXCLUDE_PATTERN, and replace grep shell calls with python re modules calls, for a 10x speedup. - Validate field names, and only use valid known problem conf files. - Minor HTML syntax fix. - Minor template integration. [ Holger Levsen ] * piuparts.py: - Add to ignored_files: + /var/log/fontconfig.log (Closes: #703810) - Thanks to Hideki Yamane. - switch default Ubuntu distribution to "saucy". * Bump Standards-Version to 3.9.4. * Use /srv/piuparts.debian.org/ on piatti.debian.org as per DSA policy. * Provide instance configuration for pejacevic.d.o and piu-slave-bm-a.d.o. * Move README_piatti.txt and README_pejacevic.txt and update accordingly. * Provide two new helper scripts for running from git: update-piuparts-master-setup and update-piuparts-slave-setup, drop update-piuparts-setup. * Drop instances/forward.* files. * slave-bin/slave_run: - Run slave_cleanup before starting the slave. * slave-bin/detect_slave_problems: remove check limiting this script to be run on piatti only. * master-bin/prepare_backup: also backup sources.txt. -- Holger Levsen Wed, 15 May 2013 11:31:31 +0200 piuparts (0.50) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files: + /var/lib/dpkg/arch + /usr/lib/python2.[67]/dist-packages/python-support.pth broken symlinks caused by python-support (see #635493 and #385775, thanks David Steele) - Call dpkg-deb with --nocheck to allow bad version numbers. - Pass unqualified package names to apt-cache show for lenny support. - Support plain URLs or local paths as --testdebs-repo argument and prepend/append "deb", "file://", and "./" as needed. - Improve the "dirname part contains a symlink" test and report overwritten files owned by other packages as failures. - Add --proxy=URL option. - piuparts.deb: Add Depends: piuparts-common for using distros.conf and piupartslib. - Use distros.conf to generate mirror configuration. This enables more complex scenarios involving partial distributions and dist-upgrades. - Use apt-get -t if set in distros.conf. * piuparts.conf: - New global settings: + proxy + slave-directory + known-problem-directory (for known_problem definitions) + backup-directory (for the master script prepare_backup) + PYTHONPATH (to find the python modules in non-default path setups) + slave-load-max - New per-section settings: + depends-sections + basetgz-directory - Obsolete settings: upgrade-test-chroot-tgz. - Setting tmpdir is now mandatory for master/slave operation. * distros.conf: - New configuration file to describe full distributions (e.g. sid, squeeze) and partial distributions (e.g. squeeze/security, squeeze-backports, experimental) along with their dependencies (e.g. base distribution) and non-default mirrors. (Closes: #699028) - Allows setting a target-release (for apt-get -t ...). (Closes: #687995) - Shipped in package piuparts-common. * piupartslib/conf.py: - Add methods: get_distros(), get_start_distro(). - Add class DistroConfig for reading distros.conf. * piupartslib/packagesdb.py: - Add Package.waiting_count() and friends, populated in calc_rrdep_count. - Optimize reserve() and skip unavailable candidates. - Add method: set_dependency_databases(). - Lookup missing packages, states, ... in the dependency_databases. * piuparts-master.py: - Sort packages to be tested by importance, i.e. descending waiting_count. - Use piuparts.conf global proxy setting as http_proxy. - Use distros.conf to compute URLs. - Load depends-sections for package lookup in partial distros. * piuparts-slave.py: - Tarball recreation can be disabled by setting max-tgz-age to 0. - Relax package existence checks to allow distupgrading to backports. - Fix SIGHUP handling while recycling. - Flush and unreserve all sections on interrupt. - Add support for running piuparts under /usr/bin/timeout. - Check system load frequently and enter sleep mode if threshold (piuparts.conf global setting slave-load-max) is exceeded. Operation will be resumed after load drops below 'slave-load-max - 1.0'. Disabled by default. - Use piuparts.conf global proxy setting as http_proxy and pass it to piuparts via --proxy. - Use distros.conf to compute URLs. - Use the upgrade-test-distros setting to select between install/purge test (if unset) and dist-upgrade test (if there are at least 2 distros). Drop support for running both tests for a section. * piuparts-analyze.py: - Add support for magic "$DISTRO/None" versions. * piuparts-report.py: - Call r.dev_off() after generating a plot. (Closes: #657799) - Use piuparts.conf global proxy setting as http_proxy. - Use distros.conf to compute URLs. - Generate/prepend/append a default description. - Load depends-sections for package lookup in partial distros. * Makefile: - Add DESTDIR support. - Overhaul. Add dependencies between build and install targets. - Add support for placeholder substitution in scripts and config. - Perform syntax check on *.py before installation. * debian/rules: Set 'prefix' and 'DESTDIR' for build and install. * lib/read_config.sh: Factored out the piuparts.conf reading function that is used by all master/slave shell scripts. Shipped in piuparts.common. - Add support for RFC822 line continuation (wrap + indent) and comments. * master-bin/slave-bin: Switch to sourcing read_config.sh. * conf/crontab*: Substitute @sharedir@ at build time. * known_problems: Install to @sharedir@/piuparts/known_problems/. * Replace all remaining references to $HOME/bin and ~/bin with @sharedir@. * prepare_backup: - Get paths from piuparts.conf. - Prepare a copy of submissions.txt, too. * master-bin/slave-bin: Use PYTHONPATH from piuparts.conf. * piuparts-slave.deb: - Use /var/cache/piuparts/tmp as tmpdir. - Use /var/cache/piuparts/basetgz as basetgz-directory. - Populate ~piupartss/.ssh/known_hosts with localhost's hostkey. - Restrict the ssh key added to master's authorized_keys to only run piuparts-master. * slave_{run,join}: Rename the screen session to "piuparts_slave_screen". * slave_cleanup: Use pgrep to find running piuparts-slave processes. * slave_{run,join}: Use screen -ls to find running screen sessions. * slave_join: Use 'script /dev/null' hack if needed to get a writable tty. This avoids running the piuparts-slave screen session as root. * slave_run: Get paths etc. from piuparts.conf. * slave_run: Ensure the tmpdir exists, this could be on tmpfs. * detect_leftover_processes: Move statefile to slave-directory. * detect_stale_mounts: Merge into detect_tmp_cruft. * detect_tmp_cruft: Move to slave-bin and run from slave crontab. * detect_slave_problems: - Move to slave-bin and run from slave crontab. - Use idle-sleep setting from piuparts.conf. - Move statefile to slave-directory. * master: Get rid of the monitor-slave directory. * pre_install_exceptions: Handle bootcd-ia64 installation in squeeze. * post_distupgrade_exceptions: Handle libdb4.8 removal in wheezy. * pre_remove_50_find_bad_permissions: Add some exceptions. * pre_remove_50_find_missing_copyright: Add some exceptions. * post_{setup,distupgrade}_experimental: New custom scripts to improve support for experimental. * Add several exceptions for lenny and lenny2squeeze tests. * New script to enable Debug::pkgPackageManager during upgrades: scripts-debug-packagemanager/pre_distupgrade_debug_packagemanager * New known problem: Packages that need to be rebuild or updated for newer tools, e.g. due to obsolete install-info or tex-common usage. * New bug template: unhandled_symlink_to_directory_conversion. * New bug template: installs_over_existing_symlink. [ Holger Levsen ] * Mention http://piuparts.debian.org in README.txt prominently. * Mention github clone in htdocs/index.tpl. [ David Steele ] * Replace the Tango icon symlinks with copies. * Make the Tango weather icons required for master. -- Holger Levsen Fri, 15 Mar 2013 15:36:12 -0700 piuparts (0.49) unstable; urgency=low [ Andreas Beckmann ] * Add support for dist-upgrade tests of "disappearing" packages: - master/report: Get candidate packages from distro in the "distro" setting in piuparts.conf and (target) versions to test from last distro entry in "upgrade-test-distros". In case a package no longer exists, use a pseudo-version "None". (Closes: #656438) - slave: Support pseudo-version "None". - piuparts: Do not try to install unavailable packages after dist-upgrade. * piuparts.py: - Run custom scripts only once if dependencies are installed via a metapackage. - Rename --no-debsums option to --warn-on-debsums-errors. - Add --extra-repo= option for adding extra package sources, e.g. backports, security or local repositories. The argument must be a valid line (including whitespace) that is added to sources.list verbatim. The option may be repeated to add more lines. - Qualify to the package names extracted from .debs with the version. - Add --testdebs-repo= option. Provide the package(s) to be tested (and additional dependencies that are not yet in the archive) in a repository to enable dependency resolution with apt-get. Use for complex install and upgrade scenarios. (Closes: #676694) - Add --distupgrade-to-testdebs option. During distupgrade tests override packages from the target distro with the packages from the testdebs repository. This allows doing distupgrade tests before uploading. - Enable upgrade tests if both --apt and --testdebs-repo are given. * piupartslib/conf.py: - Add get_final_distro() method. - Add optional distro argument to get_mirror() method. * piupartslib/packagesdb.py: - Speed up LogDB by fetching existing files with os.listdir(). * piuparts-slave.py: - Disable X forwarding while connecting to master. - Move checks for package existence to test_package(). * piuparts-analyze.py: - Sort bugs and try the newest bug first. * piuparts-report.py: - Fix URLs to piatti's config. - Hardlink the logfiles to the htdocs tree (with copy as fallback). * post_setup_squeeze-fake-essential: Restrict fake-essential install-info to the 4 packages that actually need this. * New custom script: pre_remove_40_find_obsolete_conffiles, report obsolete conffiles after package upgrades. Suggested by Michael Biebl. * pre_remove_50_find_missing_copyright: Skip check for packages that are not installed or have only config files remaining. * post_remove_cleanup: Cleanup dovecot certificates. * Add support for installing and upgrading to the multiarch ia32-libs* in wheezy/sid on amd64. (Closes: #679780) * Merge known problem analyses excessive_output and runtime_exceeded into new known problem resource_violation. * New known problem: Leaving obsolete conffiles after upgrade. * New known problem: Modifying conffiles or other shipped files (issue). * New bug template: partial-upgrade-file-overwrite. [ Holger Levsen ] * README.txt: - reformat with shorter line lengths (Closes: #603873). - fix some typos. -- Holger Levsen Sat, 12 Jan 2013 12:12:45 +0100 piuparts (0.48) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Ignore *.dpkg-* conffile backups found in --scriptsdir directories. - Report attributes (for reference and current file) in case of file modification. - Report paths of installed files that contain symlinks (for better debugging of the problems caused by such a setup). ( test $(dirname $x) != $(readlink -f $(dirname $x)) ) - Add debsums, processes, and broken-symlinks check after the first purge in --install-purge-install mode. - Add --install-recommends option, disabled by default. - Mark temporary directories with a .piuparts.tmpdir file. - Use rm -rf --one-file-system to delete chroots. * piuparts.conf: - New per section settings: expire-old-days, expire-fail-days (defaulting to 0 which means expiration is disabled). - Split and rename piuparts-cmd into piuparts-command ([global] setting that should include all common flags) and piuparts-flags (for additional section-specific flags). - Rename the tempdir setting to tmpdir, this will be used for the --tmpdir argument of piuparts, too. * piuparts-master.py: - Enable recycling before initializing the real package db. - Enable logging by default and set default log-file to $SECTION/master.log. Use "/dev/null" to disable logging. * piuparts-slave.py: - Retry sections with higher precedence more often. - Construct the piuparts command line from the new config options piuparts-command, piuparts-flags, and tmpdir. * piuparts-report.py: - Don't hardcode section names, always show all known_problem reports. * detect_piuparts_issues: Catch the chroot running out of space. * reschedule_oldest_logs: Implement logfile expiration. Delete logfiles that are marked for recycling and are older than expire-{old,fail}-days. These packages have not been retested because of some dependency problem making the package currently untestable. Deleting the log will cause the package to be moved to dependency-failed-testing or similar state. expire-*-days needs to be greater than reschedule-*-days to enable this. * slave_cleanup: Only remove directories containing a .piuparts.tmpdir file. * New bug template: copyright_file_missing_after_upgrade. * New known problem: Installing something over existing symlinks. -- Holger Levsen Fri, 30 Nov 2012 12:12:09 +0100 piuparts (0.47) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files/ignored_patterns: + /var/log/dbconfig-common/dbc.log - Print a final message before terminating in the panic() handler. - Add support for running debsums before package removal, enabled by default. May be disabled via --no-debsums. (Closes: #687538) - Add debsums as dependency. * piuparts-slave.py: - Detect incomplete piuparts output and ensure such tests fail. * piuparts-report.py: - Rewrite find_files_with_suffix(), handle disappearing files. * New custom script pre_remove_50_find_missing_copyright: check for missing /usr/share/doc/$package/copyright files. (Closes: #683685) * Rename custom script post_install_find_bad_permissions to pre_remove_50_find_bad_permissions to avoid early failures during upgrade tests that may be fixed in a later version. * Remove post_purge_dbconfig_cleanup custom script. * reschedule_oldest_logs: Refactored. - Fix duplicate variable name usage that sometimes prevented recycling. - Support a [global] auto-reschedule=no setting. * detect_well_known_errors: Unclassified logs may still be bugged. * detect_piuparts_issues: Detect incomplete logfiles. * New bug template: modifies_conffiles. * New known problem: Modifying conffiles or other shipped files. * New custom script post_distupgrade_hack_debsums to work around #687611 which makes all debsum checks fail due to modification of /usr/share/keyrings/debian-archive-removed-keys.gpg during upgrades. -- Holger Levsen Fri, 28 Sep 2012 13:24:47 +0200 piuparts (0.46) unstable; urgency=low [ Holger Levsen ] * piuparts-report: drop in-code index.tpl stub. * slave-join: fix bug preventing detection that the slave is not running. * piuparts: call schroot with session: argument, thanks to Stephan Sürken for the patch. (Closes: #686360) [ Andreas Beckmann ] * piuparts.py: - Add to ignored_files/ignored_patterns: + /var/lib/cdebconf/{questions,templates}.dat-old + /var/lib/onioncat/ + /var/lib/vmm/(./.*)? (Closes: #682184) - Fix some leftover files and directories getting incorrectly accounted to the package being tested instead of the dependencies in --warn-on-others mode. - Implement --install-purge-install option: test purging with all dependencies still installed; re-install after purge. - Install the dependencies and snapshot the chroot before the --install-purge-install test. Check that the chroot after purge matches the snapshot. (Closes: #668713) * piupartslib/dependencyparser.py: - Package names may now have a ":any" qualification in some cases. Extend the parser to recognize and discard this suffix. * piupartslib/packagesdb.py: - Implement logfile recycling mode that avoids getting a large backlog of untested packages due to uncoordinated log deletion. Delay log deletion and give preference to packages (or versions) that have not yet been tested. (Closes: #670150) + logfiles to be recycled should be hardlinked from pass/, fail/, ... to recycle/ + recycling can be enabled if recycle/ is not empty + treat packages with logfile in recycle/ as initially "unknown" and compute state ("waiting[-for-dependency]-to-be-tested") + delete old logfile while reserving such a package * piuparts-master.py: - Accept and discard duplicate log submissions, preventing the slave from retrying infinitely. - Delay loading the Packages file which is only needed for "status" and "reserve" commands, but not for submitting logs. - Add "recycle" command to enter logfile recycling mode (needs to be issued before "status" or "reserve"). - Remember idle state by creating stamp files. - Add "idle" command to check a remembered idle status before trying to "reserve" packages (avoid package state computation). Idle status (i.e. no packages available for "reserve") will be remembered for an hour, but any log modification (submission, deletion, marking for recycling) will clear the idle status. * piuparts-slave.py: - Fix stopping the currently running test (Ctrl-C Ctrl-C). - Handle master communication exceptions while sending logs or reserving packages: skip the section but keep the slave running. - Flush finished logs from sections that have processing postponed due to precedence settings. - Skip connection to master if we have reserved but no submittable logs. - Submit finished logs after completing all reserved logs in a section. - Send finished logs to master if interrupted with a single Ctrl-C. - Try to unreserve all reserved logs after interrupt. - Add SIGHUP handler that triggers flushing finished logs. Flushing (all sections) will be done after the current test has finished. - Enter logfile recycling mode if no section has packages left for regular processing. Recycle logfiles until a section becomes available for regular processing. If no logfiles could be recycled, retry after an hour. - Issue "idle" command before "reserve" and skip status computation and reserving if master is idle. * Simplify running piuparts from GIT. * Reorganize layout in the GIT repository to reduce path nesting and length. * Makefile: - Separate build and install targets. - Adjust for new layout, cleanup, simplify. * conf/piuparts.apache: Set DefaultType text/plain (for the logfiles). * reschedule_oldest_logs: Hardlink old logfiles to recycle/ instead of deleting them. Cleanup obsolete rescheduling requests. * New helper script: master-bin/reclassify_bugged. * New custom script post_setup_forbid_home: replace /home with a file to prevent creating home directories there. (Closes: #677332) * New custom script post_install_find_bad_permissions: look for world writable directories without sticky bit set. -- Holger Levsen Mon, 03 Sep 2012 15:28:04 +0200 piuparts (0.45) unstable; urgency=low [ Holger Levsen ] * Merge piatti.git into piuparts.git, so that piatti.git has become obsolete now. Update documentation accordingly. * Further split packaging into piuparts-master, piuparts-slave and piuparts-common binary packages (Closes: #585861) and cleanup dependencies. * Update piuparts.NEWS about master+slave packages. * Switch to debhelper8 packaging. * debian/copyright: use versioned URI of the machine-readable copyright file, thanks lintian. * Move /etc/sudoers.d/piuparts to piuparts-slave package. * piuparts-master: depend on tango-icon-theme and drop sunny.png and weather-severe-alert.png and use links instead. * Reviewed and merged/cherry-picked all of the following changes. [ David Steele ] * Remove debian/postinst as it was only dealing with lenny time area configuration files. * Added server configuration files for apache, sudo, and cron. * Use dh_install, in preparation for multiple package build. * Added packaging to create the piuparts-server package out of the old piatti directory tree (Closes: #652934). - Create and configure piupartss and piupartsm users. - Coordinate ssh keys between master and slave users. - Create working directory tree for each under /var/lib/piuparts * packagesdb.py: - Calculate recursive reverse dependency count, and store in PackagesDB. - Calculate the count of packages being blocked by a failed package. * piuparts-report.py: - Release Section object when section processing is complete. (Closes: #673919) - Display reverse dependency counts and block counts to failure summaries. - Sort the failed-testing and cannot-be-tested summaries by block count. (Closes: #674498) - Replace O(n^2) search in remove_old_logs() with a hash piuparts-report run time improved 20% on mature environment. - Another search fix in create_package_summaries (find_links_to_logs) yielding a 10x speedup for piuparts-report runs. (Closes: #674672) - Cleanup/wrap some long source lines. - Cleanup/wrap some long html lines. - Allow the html root to be changed from / (piuparts.conf: e.g. [global] doc-root = /piuparts/). [ Andreas Beckmann ] * piuparts.py: - Implement --install-remove-install option to test installation in config-files-remaining state. (Closes: #359154) - Report leftover symlinks with target. - Report leftover directories with a trailing slash. - Allow ignore entries to specifically match directories. - Ignore patterns need to match the full filename, not a substring. - Cleanup and regroup ignore lists. + Remove many obsolete entries not needed for squeeze or later. + Group ignore entries into piuparts state, system state, package management, logfiles, home directories, application data, and HACKS. - Add to ignored_files/ignored_patterns: + Default system logfiles as created by syslogd in /var/log/ + Common empty directories in /etc not removed due to dpkg bug #316521 + cdebconf status files + /boot/grub/ + /etc/X11/default-display-manager + /etc/aliases.db + /etc/apt/trusted.gpg.d/.*.gpg~ + /etc/network/interfaces + /etc/news/whoami + /usr/share/keyrings/debian-archive-removed-keys.gpg~ + /var/crash/ + /var/lib/citadel/(data/.*)? + /var/lib/gozerbot/ + /var/lib/nagios/ (Closes: #668756) + /var/lib/openvswitch/(pki/.*)? + /var/spool/powerdns/ (Closes: #531134, #531135) - Implement a timeout for commands being run in the chroot. - Set time limit to 30 minutes per command. (Closes: #654423) - Terminate all processes running inside the chroot before removing the chroot; also in case piuparts aborts due to an error. - Continue killing running processes as long as new processes get spawned in the chroot. - Perform chroot cleanup after receiving Ctrl-C/SIGINT, too. - Let Chroot register/de-register the cleanup panic handler. - Remove temp_tgz on error exit. - Remove metapackage build directory on error exit. (Closes: #663702) - Don't remove eatmydata when minimizing a chroot. (Closes: #658962) - Add support for version-qualified package arguments (--apt foo=1.2-3). - Switch fallback mirror to http://cdn.debian.net/debian. * piuparts.conf: - Defaults for all [section] settings can be set in the [global] section. - Change master-command to not include the section name so that it can be set in the [global] section. The section name will be given as an argument to this command. - Make doc-root default to "/". - New per section key: area (set to one of main, contrib, non-free). - New defaults: mirror=http://cdn.debian.net/debian, area=main. - Compute URLs from mirror, distro, area, arch. - Removed deprecated keys: known_circular_depends, packages-url, sources-url. * master/db/report: Add a new category /affected/ for failed logs where the bug is in a dependency, but only exposed by the package being tested. - In the BTS these are reported as (Closes: #657740) Package: buggy-package Version: buggy-version Affects: package-under-test Found: package-under-test/tested-version - Reschedule /affected/ logs like /fail/ logs. * piupartslib/conf.py: - Add support for getting default values from a special section. - Add methods to get mirror, distro, area, arch from config file or defaults. - Add methods to compute URLs to Packages and Sources files. * piupartslib/packagesdb.py: - Do not consider a package as "untestable" if untestable/ contains a logfile from an obsolete version of that package. - Simplify handling of package and dependency states. - Do not inherit 'dependency-does-not-exist' state (which may be fixed in the package) but propagate it to 'dependency-cannot-be-tested'. - Remove 'no-dependency-from-alternatives-exists' state and use 'dependency-does-not-exist' instead. - Remove 'unknown-preferred-alternative' state as it interferes with circular dependencies involving alternatives and virtual packages. - Fix state resolution of package dependencies containing alternatives, virtual packages and circular dependencies. - Remove 'circular-dependency' state. - Stop using static known_circular_depends. - Compute dependency cycles and test packages on such cycles if all non-circular dependencies are satisfied. - Consider any combination of two error states for the blocking count. - Always run create_subdirs() during initialization. * piuparts-master.py: - Remove known_circular_depends handling. - Reduce logfile verboseness: do not include received logs. - Always chdir to master_directory, do not rely on being run from there. - Record timestamps of submitted logs in submissions.txt. * piuparts-slave.py: - Randomize waiting time (between 60 and 180 seconds) if master is busy. - Sleep until the next section can be tried, but at least 1 minute. - Simplify and merge Section.setup() into Section.__init__(). - Generate master command: use global setting and append section. - Test the 'dpkg' package for creating/updating a base_tgz. - Don't update the tarball for disabled sections. - Rewrite starting piuparts. - Rewrite stopping piuparts for more reliable cleanup. - Introduce Ctrl-C/SIGINT handling: press Ctrl-C once to exit after the current piuparts test has finished, press Ctrl-C again to cleanly terminate the currently running test, press Ctrl-C again to abort any ongoing cleanup. - Pass a version qualified package name (foo=1.2-3) to piuparts to ensure the correct package version gets tested. - Increase verboseness for untestable packages. - Remove idle.stamp functionality. * piuparts-report.py: - Remove known_circular_depends handling. - Exclude obsolete states from generated report. - Establish packagesdb working directory in Section. - Handle logs in /affected/ directories. - detect_well_known_errors: Add bug links if bugs are known. - New known problem: "packages have been kept back". - Report rdeps and blocking counts in all error state reports. - Fix another race with logfiles disappearing while copying. * piuparts-analyze.py: - Don't report further bugs/versions if we found a match. - Classify logs with bugs filed into /bugged/ or /affected/. - Write .bug files with links to the BTS. * detect_archive_issues: Only consider logs older than one day for marking as "untestable" to allow piuparts-analyze to handle the log first. * Replace all references to lenny and squeeze with squeeze and wheezy. * Add/update several bug reporting templates. * Add custom scripts to aid debugging common problems in maintainer scripts. * Add custom script for squeeze to ensure adduser, ucf, and install-info are not purged. * Add custom scripts to enable/perform additional cleanup procedures to reduce the "FAIL: Package purging left files on system" errors if --warn-on-leftovers-after-purge is not used. (Closes: #668752) * Ship custom scripts in the piuparts package. * Comment sudoers and crontabs shipped in the master/slave packages, requiring explicit editing after installation to activate them. * Bump debhelper and dpkg dependencies for using dpkg-maintscript-helper. * Remove obsolete /etc/piuparts/piuparts.conf from old piuparts package. * Ship /etc/piuparts/piuparts.conf in the piuparts-slave package. * Create missing home directories if the piupartsm/piupartss users remain from a previously purged installation. [ Leo Iannacone ] * piuparts.py: add --existing-chroot option, specify a directory as source for the chroot, instead of building a new one with debootstrap or decompressing an archived one. (Closes: #658044) [ Mika Pflüger ] * piuparts.py: Remove obsolete functions shellquote, apt_get_knows, and append_to_file. [ Mehdi Abaakouk ] * piuparts.py: Add support for schroot. (Closes: #530733) -- Holger Levsen Thu, 21 Jun 2012 22:17:26 +0200 piuparts (0.44) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Add to self.ignored_patterns: /var/lib/ucf/.* - Increase output limit to 3 MB (for dist-upgrading daptup). - Do not miss Pre-Depends in --warn-on-others mode. - Remove packages and reinstall missing ones in one apt-get call. - Run 'apt-get dist-upgrade' (instead of upgrade) during chroot setup. * piupartslib/packagesdb.py: - Only consider the newest version of a package as test candidate. (Packages files may contain more than version of the same package.) * piuparts-master.py: - Implement simple section locking to allow concurrent master instances but allow only one instance per section. * piuparts-slave.py: - Do not leak the logfile file descriptor to piuparts. - Skip disabled sections (max-reserved = 0). - Do not abort if connection to master failed, just skip the section. - Do not abort if Packages file cannot be fetched, just skip the section. - Implement simple section locking to allow concurrent slave instances. - Sections wait before retry 15 minutes after error and one hour if idle. - Create idle.stamp files to remember idle sections for one hour and notify concurrent slaves about idle state. - Skip misconfigured sections that have neither 'distro' nor 'upgrade-test-distros' in their config. (Closes: #668863) * piuparts-report.py: - Add link to the list of untestable logs. - Add more known problem reports: forgotten alternatives, warnings in passed logs: leftover files and failed maintainer scripts. - Do not abort if logfiles disappear during copying. * Makefile: Install documentation with mode 0644 instead of 0755. [ Holger Levsen ] * piuparts-report: don't create output for disabled sections. (Bye bye Lenny!) * Bump Standards-Version to 3.9.3, no changes necessary. -- Holger Levsen Thu, 17 May 2012 14:36:53 +0200 piuparts (0.43) unstable; urgency=low [ Andreas Beckmann ] * piuparts.py: - Add to self.ignored_patterns: /run/.* - Unset DISPLAY variable before running commands in the chroot. - Read possibly remaining buffered output after a command terminated. - Run apt-cache policy on the packages to be installed. - Simplify keeping the reference chroot state: move tree info, selections and diversions into a dict, can be extended easily. - Fix --*end-meta documentation and actually enable these options. - Cleanup options and settings. * piupartslib/packagesdb.py: - Restrict circular-dependency state to packages on the circle. * piuparts-slave.py: - Fix chdir in/out of the slave dir to work with relative paths. * piuparts-report.py: - Reorder the list of known problem analyses and add a few new ones. - Correctly split + strip comma-separated lists. * piuparts-analyze.py: - Also consider bugs that 'affects' a package. - Treat bugs without 'found' version as older than anything. * Makefile: Support version number substitution in piuparts-*.py, too. * README.txt: Document piuparts.conf settings used by piatti.git scripts. [ Holger Levsen ] * piuparts.1.txt: correctly reverse the description of --save-end-meta and --end-meta. * piuparts-report.py: - add support for more static pages and add bug_howto.tpl, explaining how to file meaningful bugs piuparts-analyze understands. This page also links to the templates used for bug filing. * piuparts-slave.py: make status message more useful by including the section. -- Holger Levsen Sun, 22 Jan 2012 13:03:59 +0100 piuparts (0.42) unstable; urgency=low [ Holger Levsen ] * piuparts.py: - add to self.ignored_files: /etc/blkid.tab (Closes: #638831) - add to self.ignored_patterns: /var/lib/apt/lists/.* - apply patch by Gregor Herrmann to fix --minimize. (Closes: #648423) * Remove Debian.NEWS entry about source in git. (Closes: #640121) * piuparts.py, piuparts-report.py, ChangeLog: Expand tabs to spaces. * Remove whitespaces from whitespace-only lines. * piuparts-report: - create maintainer subdirs if they don't exist. - detect tests being terminated due to excessive output. * Add git to Build-Depends-Indep: as git describe is used to generate part of the version string for development builds. * Add debian/.gitignore [ Mika Pflüger ] * piuparts-analyze.py: - Rewrite to use python-debianbts to analyze if bugs are filed already. - The BTS only tracks source versions, so remove binNMU-part from package versions when comparing with versions from the BTS. - Reduce noise in the output by only printing one action/advise per package. - Fix extraction of package versions from bug reports. Thanks to Andreas Beckmann for catching and solving the error. * debian/control: Add python-apt and python-debianbts to piuparts depends. [ Scott Schaefer ] * debian/copyright: Make it compliant with DEP-5. * piuparts-slave.py: - Replace deprecated os.popen2 with subprocess.Popen. (Closes: #640646) - Add some more logging. - Kill children (hard-coded value, 45 minutes) to ensure test doesn't run "forever" (Closes: #640647, #387428) * piuparty.py: - Kill leftover processes (via SIGTERM, then if that fails, via SIGKILL). (Closes: #522918) - Test for incorrect diversion handling: (Closes: #588313) a) Existing diversions removed/modified, and/or b) Installed diversions not removed by purge. * piupartslib/packagesdb.py: Modify several functions in PackagesDB class to use python 'set' type, instead of a list. This permitted replacing inefficient combination of unique() function and random.shuffle() with python set.pop() method. Since python prohibits storing non-hashable object in a set, minor modifications to piuparts-report and to piuparts-master required. (Closes: #640648) [ Andreas Beckmann ] * *.py: Add vim modeline. * piuparts.py: - Add unittests for misbehaving symlinks. - Fix resolving absolute symlinks of intermediate directory components, i.e. /var/run -> /run while checking /etc/motd -> /var/run/motd. Solves about 30000 false positives of 'Broken symlinks: /etc/motd -> /var/run/motd'. (Closes: #648784) - When running commands in the chroot, redirect stdin from /dev/null. - Stop using Popen.communicate() as it may run out of memory. - Terminate commands producing more than 2 MB of output. (Closes: #649335) - Create /etc/dpkg/dpkg.cfg.d/ if missing inside the chroot (e.g. on lenny). (Closes: #647752) - Remove logrotate and its dependencies only once. - Only run 'apt-get update' after updating the sources.list. - Only run 'apt-get clean' before creating tarballs or saving meta data. - Do the same checks for running processes and broken symlinks in all tests. (Closes: #648708) - Create piupart's apt config in the chroot as /etc/apt.conf.d/piuparts instead of /etc/apt.conf in order to allow overriding the settings from custom scripts by just dropping new config bits in e.g. /etc/apt/apt.conf.d/piuparts-foobar. - Integrate diversion checks with Chroot.check_result(). - Use 'apt-get remove' to remove all packages at once with proper dependency ordering. (Closes: #648162) - Purge all packages at once instead of doing it one-by-one. - Make restore_selections() reinstall missing packages. (Closes: #648707) - Set more environment variables to let custom scripts know where and when they are being run: PIUPARTS_TEST, PIUPARTS_PHASE, PIUPARTS_DISTRIBUTION{,_PREV,_NEXT}. See the README for details. (Closes: #589498) - Add missing post_install_* custom scripts to install_packages_by_name(). (Closes: #628077) - Remove pre_upgrade_* custom scripts, they can be replaced with pre_install_* scripts that check for PIUPARTS_PHASE=upgrade. - Add pre_test_* custom scripts. These are run exactly once at the beginning of each test (after recording the initial chroot state). - Allow multiple --scriptsdir options, the contents will be merged. - Exclude /tmp/scripts when creating a tarball. - Use --one-file-system when creating a tarball to exclude bindmounts etc. - Restore base system from temp_tgz instead of running debootstrap again. (Closes: #648193) - Do not fail upgrade/distupgrade tests for a set of packages where not all packages exist in the start distribution. This happens e.g. when testing .changes files and packages were split/added. Only install the old packages that actually exist according to 'apt-cache show'. - Add --extra-old-packages option to intall additional old packages that are not in the set of new packages to be tested. Useful for testing upgrades with Conflicts/Replaces relationships, e.g. in package renames or merges. - Use consistent variable names for package lists. (Closes: #648177) - Compute the changes in restore_selections(). - Check for settings.scriptsdir inside run_scripts(). - Consistently use chroot.relative() to build filenames inside the chroot. * piupartslib/packagesdb.py: - Handle 'unknown-preferred-alternative' state correctly. - Add 'does-not-exist' state for dependency resolution to distinguish this from 'unknown' state so that the latter only indicates 'unresolvable' or 'not yet resolved'. - Handle virtual packages separately from real packages. - Use get_package_state() internally which 'resolves' (purely) virtual packages by default (can be disabled). * piuparts-master.py: - Add a 'status' command that reports package counts for all states. * piuparts-slave.py: - Fix triggering tarball recreation. - Check tarball age regularily. - Log tarball creation in *.tgz.log. - Request and print package counts from master. - Reload section config every time a section is being run. - Add precedence attribute to allow prioritizing different sections and to suspend processing of low priority ones while there are packages waiting in more important sections. * piuparts-report.py: - state-*.html: Sort package lists by name, display state of all alternative dependencies and packages providing a virtual dependency. - source/?/*.html: Sort binary packages by name. - maintainer/?/*.html: Sort source packages by name. - Update list of error states to be highlighted. - Archive logs of packages that were removed from the distribution. - Speedup generating maintainer summaries. * Makefile: Use 'git describe' to get an exact stamp for development versions. [ Dave Steele ] * piuparts-slave.py: make Section.run() report the number of packages processed and use this to decide whether a slave should sleep. (Closes: #649967) [ Stefano Rivera ] * piuparts.py: - Properly install and remove logrotate. (Closes: #638832) - Use eatmydata by default, add option --no-eatmydata. (This was discussed in #633033.) -- Holger Levsen Fri, 23 Dec 2011 10:51:28 +0100 piuparts (0.41) unstable; urgency=low * piupartslib/packagesdb.py: Apply patch by Scott Schaefer to also consider alternative depends: the algorithm used to select the "best" alternative is: 1) Prefer first alternative in state "essential-required" 2) If no "essential-required" alternatives, prefer first alternative in state "successfully-tested" 3) Otherwise, prefer first alternative in state "waiting-to-be-tested" IF NO REMAINING alternatives are in one of the "unknown/fail" states For this two new states have been introduced: a) "unknown-preferred-alternative": equivalent of "unknown", this defers calculation of this package's state, since one or more of its alternative depends are "unknown" (or "unknown-preferred-alternative"), and no alternative is either "essential-required" or "successfully-tested". The alternatives will be re-tested on subsequest pass. b) "no-dependency-from-alternatives-exists": if none of the alternatives can be found in the archive. (Closes: #526045) * piuparts-report.py: - correct a typo from the patch for #523950. - make_stats_graph: draw the two new states as well. - point to source code in git instead of svn. * piuparts.py: - use proxy settings either from apt configuration or http_proxy environment variable, the latter overwriting the former (if present) - Thanks to Scott Schaefer for the patch. (Closes: #632046) - new option "--no-install-purge-test" to only do upgrade tests - Thanks to Andreas Beckmann for the patch (Closes: #588482) - run dpkg with --force-unsafe-io by default and introduce new option "--dpkg-noforce-unsafe-io" to disable this feature. (Closes: #633033) Thanks to Scott once more! - new option: "--fail-on-broken-symlinks". Remove option "--warn-symlinks" as this is the default now. * piuparts-slave: - check if chroot-tgz is older than max-tgz-age (defaulting to 30 days) and recreate it, if it is. Keep backup and put in back in place when debootstrapping a new chroot-tgz fails. Retry after min-tgz-retry-delay seconds. Thanks to Scott Schaefer for the patch. (Closes: #632924) - document new configuration options max-tgz-age and min-tgz-retry-delay in README.txt. - fix bug in piuparts-slave which prevented running a single section by including section name as command-line argument - thanks again to Scott and see 632924 too. * debian/control: - require python >= 2.6.6-3~, get rid of python-support build-depends. - add X-Python-Version: pseudo-header * debian/rules: use dh_python2 instead of python-support to build the package * Makefile: build for python 2.6 and 2.7 instead of 2.5 and 2.6. * Source code is now stored in git. https://anonscm.debian.org/gitweb/?p=piuparts/piuparts.git git clone git+ssh://git.debian.org/git/piuparts/piuparts.git Thanks to Thomas Koch for setting those up and keeping history! * The configuration and scripts used on piatti.debian.org have also been moved to git, the URLs are the same, just s#piuparts#piatti#. * Add entry about git to NEWS. * Update Vcs-*-Headers in control to reflect move to git. -- Holger Levsen Wed, 24 Aug 2011 13:47:07 +0200 piuparts (0.40) unstable; urgency=low * piuparts.py: - make "natty" the default Ubuntu distribution. - add to self.ignored_files: - /var/log/bootstrap.log * debian/control: depend on ${python:Depends} instead of just python. Thanks to Jakub Wilk. (Closes: #614617) * *.py: do not use Python built-in function names as variable name. Thanks to Carl Chenet for a first patch and Scott Schaefer for the applied one. (Closes: #523950) * Bump Standards-Version to 3.9.2, no changes necessary. * debian/rules: add empty build-arch and build-indep to appease lintian. * Makefile: drop workaround for #559305 in lenny. -- Holger Levsen Sat, 25 Jun 2011 21:20:31 +0000 piuparts (0.39) unstable; urgency=low [ Holger Levsen ] * The vintage of 2010 release! * piuparts-report.py: - report packages which failed - due to broken maintainer scripts, - due to output from cron after removal, - due to not enough force being, - due to a problem with pre-depends, - due to files having been modified after purge, - due to files having disappeared after purge and - due to problems configuring a database. - only report about (un)owned files and directories and symlink issues in sid - that's hardcoded as it's not practical to track them anywhere else anyway. And at least for broken symlinks this will also stay like this for a long time. - visual improvements in the output. - ignore udebs and binaries from other archs too. - graph is generated using all the available data points. * piupartslib/packagesdb.py: - automatically break circular dependencies when there are only circular dependencies left, using a configurable list of circular depdencies in piuparts.conf (Closes: #526046) The list of circular depends is taken from a script written by Robert Lemmen available at http://debian.semistable.com/debgraph.out.html - obviously it would be better to merge this into piuparts directly. Note that the dependency resovler is still buggy, the state waiting-for-dependency-to-be-tested still contains some packages with failed depends (due to them being dependent on packages with circular depends). This bug has no effect other than that (so the state waiting-to-be-tested is calculated correctly). This bug is also no regression. - get rid of the longtime unused states "fixed" and "fix-not-yet-tested" (also in piuparts-master and piuparts-report). - forget reservations of untestable packages. * piuparts-slave.py: - fix crash when going into sleep when idle. - add support for only doing upgrade tests (without the basic test in the upgrade target distro). - honor mirror configuration option also for upgrade-tests. - also do upgrade tests if the version of a package being tested (the one in the upgraded distro) is not available in the distro being upgraded from. (This is a very short test, but a successfully one.) - make configuration setting "distro" to not default to any distro. * piuparts.py: - new option: --log-level to specify the verbosity of piuparts output. Thanks to Fabrice Coutadeur for the patch! (Closes: #567190) - new option: --warn-on-leftovers-after-purge to not fail if a package leaves files behind after purge. (Closes: #566599) - add to self.ignored_files: - /var/lib/apt/lists/partial/.delete-me-later - /var/log/alternatives.log (Closes: #591966) - /var/log/apt/history.log (Closes: #594964) - /usr/share/fonts/X11/misc/fonts.alias (Closes: #576321) - add to self.ignored_patterns: - /etc/init.d/.depend.* - /var/mail(/.*)? - this used to be just /var/mail/.* - /var/lib//update-rc.d(/.*)? (Closes: #605127) - create temporary DEBIAN directory for the piuparts-depends-dummy package with 0755 perms regardless of the umask of the calling shell. (Closes: #573904) - fix --do-not-verify-signatures option (Closes: #574936) - thanks to Cleto Martin Angelina for the patch. - make Chroot.diff_meta_data() special case and ignore files matching (/etc/rc.\.d/)[SK][0-9]{2}(.*)$ so that piuparts doesn't complain about initscripts renamed by insserv. Thanks to by Andreas Beckmann for the patch! (Closes: #586793) - remove logrotate and depended packages after the test. (Closes: #602409) This is a fix for the incomplete patch for #566597. Again, this is a hardcoded list. :-( - new custom script: pre_remove_ (Closes: #539146) - set environment variable PIUPARTS_OBJECTS in custom scripts to a space separated list of packages / changes files being tested. - do not call apt-get with --no-remove when installing packages. (Closes: #603453) - apply patch by Andres Mejia to fix parsing of the --force-confdef option and also to configure apt to use that option. Thanks, Andres. (Closes: #605475) - don't panic if package is not known by apt-get, this probably just means a package not yet in the archive is being tested. (Closes: #566672) Thanks to Cleto Martin Angelina for the patch! - fix parsing of the options --end-meta and --save-end-meta. - supply help texts for --end-meta and --save-end-meta options, also add comments to install_and_upgrade_between_distros() to make the code easier to understand and hopefully improve in future. (Closes: #560050). - add logging to install_and_upgrade_between_distros() to point people to read the functions source code if they wonder why the log (for distro upgrade tests) looks like it looks. * piuparts.1.txt: - update the pointer to custom-scripts.txt to point to README.txt, as those two files have been merged. Thanks to Gregor Hermann for spotting this. (Closes: #574504) - indicate that it's possible to check several packages or .changes files at once. - update several options descritpions. * README.txt - better documentation of custom scripts. - reformat to achieve consistent width. * Add ${misc:Depends} to Depends in debian/control. * Remove versioned dependencies on debootstrap as the version available in Etch is sufficient. Drop build-depends on dpkg-dev as it's in build-essential. * Build-depend on debhelper>=7 and use dh_prep instead of dh_clean -k, bump compat level to 7. * Bump Standards-Version to 3.9.1, no changes necessary. * Remove Ian Jackson from uploaders - thanks for your work, Ian! * Makefile: support python 2.5 and 2.6 instead of 2.4 and 2.5, adjust debian/control accordingly. * debian/preinst: update code snipplet rm_conffile() from https://wiki.debian.org/DpkgConffileHandling [ Evgeni Golov ] * piuparts.py: - Check for remaining logrotate files which produce output after the package was removed. (Closes: #566597) [ John Wright ] * Use debian module when available, else debian_bundle module. (Closes: #586443) * Use built-in set type, available since python-2.4. -- Holger Levsen Tue, 04 Jan 2011 14:12:30 +0100 piuparts (0.38) unstable; urgency=low * piuparts.py: - Add support for using LVM snapshots. Thanks to Patrick Schoenfeld for the patch. (Closes: #559449) - Add support for warning on broken symlinks. Thanks to Carl Chenet for the patch. (Closes: #440151) - Make --help usuable without needing to be run as root. (Closes: #547379) - --skip-minimize is now the default. This is because debootstrap is called with --variant=minbase instead of --resolv-deps now and because if a base.tgz or an lvm snapshot is specified, piuparts should not touch it. (Closes: #539142, #559048) Introduce a new option, --minimize, to minimize a chroot with debfoster. (This is the old default behaviour. In future versions the --minimize option might be removed again.) - Add two new options: --keyring to specify the keyring to use with debootstrap (defaults to /usr/share/keyrings/debian-archive-keyring.gpg) and --do-not-verify-signatures to not use --keyring with debootstrap. (Closes: #545907) In the chroots, APT::Get::AllowUnauthenticated is set accordingly in /etc/apt/apt.conf. - Add new option, --pedantic-purge-test, to tell piuparts to be pedantic when checking if a purged package leaves files behind. If this option is not set, files left in /tmp are ignored. (Closes: #528266) - Add new option, --dpkg-force-confdef, to make dpkg use --force-confdev, which lets dpkg always choose the default action when a modified conffile is found. This options will make piuparts ignore errors it was designed to report and therefore should only be used to hide problems in dependent packages. (Closes: #466118) - Add another type of custom-scripts, post_setup_, which are executed after the chroot was setup. (Closes: #466043) - Create policy-rc.d in 2nd chroot when using -b. (Closes: #466049) Thanks again to Patrick Schoenfeld for his help. * piuparts-report: report broken symlinks. * piuparts-slave: - gracefully deal with upgrade-test-distros and upgrade-test-chroot-tgz not being set in the configuration. Thanks to Carl Chenet and Patrick Schoenfeld for the patch. (Closes: #519192) - new default value for idle-sleep, 300 instead of 10 seconds. Also the slave will now only sleep if there is nothing to do for any section. * Makefile: workaround 559305 by calling a2x twice. * Update debian/NEWS. * Replace all references to sarge and etch with ones to lenny and squeeze. -- Holger Levsen Thu, 17 Dec 2009 14:10:27 +0000 piuparts (0.37) unstable; urgency=low * piuparts-report.py: report packages with update-rc.d warnings and those failing due to insserv errors. (Closes: #546540) * Fix typo in NEWS.Debian, thanks to Justin Rye for spotting it. (Closes: #547439) * piuparts.py: - added to self.ignored_patterns: - /var/cache/man(/.*)? (Closes: #550953) - /var/lib/insserv/run.*.log - /var/lib/mercurial-server(/.*)? - removed from self.ignored_files: - /var/cache/man/index.db * fix typo in -i option in manpage (Closes: #555202), thanks to James Vega for spotting it. * Make "lucid" the default Ubuntu distribution (Closes: #559047), thanks to Krzysztof Klimonda. * fix FTBFS by adding "-r /etc/asciidoc" to the a2x call in Makefile, thanks to Michael Vogt. (Closes: #559299) -- Holger Levsen Thu, 03 Dec 2009 13:25:43 +0000 piuparts (0.36) unstable; urgency=low [ Holger Levsen ] * The "For me. For you. For everybody." release. * Break backwards compatibility of the configuration files for master-slave- mode. Merge those three into one: /etc/piuparts/piuparts/piuparts.conf. Introduce a new global section in piuparts.conf which is shared among the other sections there. * piuparts.py: - add check whether scriptsdir exits, to fail gracefully if not. - copy scriptsdir to chroot also when doing upgrade tests. - added to self.ignored_files: - /etc/shadow and /etc/shadow- (just as /etc/passwd* and /etc/group*) - /var/games (see #524461) - /etc/apt/trusted.gpg and /etc/apt/trusted.gpg~ (just as /etc/apt/trustdb.gpg and other backup files) - added to self.ignored_patterns: - /var/lib/dpkg/triggers/* - /var/lib/ldap(/.*)? (see #527898) - /var/backups/.* - modified self.ignored_patters: - ignore everything in /tmp, not just in /tmp/scripts (see #528266) - configure apt in chroots to not install recommends and suggests. - add support for scanning for packages in changes files, thanks to Andres Mejia for the patch. (Closes: #352940) - change some methods from using 'args' to 'package_list'. This more accurately represents what is being passed into these methods now. - add an optional parameter to panic() method to specify what exit status to use. Also thanks to Andres. - add advice how to read the logfile to the top of the logfiles produced. - add "FAIL: " to logging output for seven more failure types, so that it becomes easier to group failure causes. - allow piuparts to be stopped with control-C. Thanks to Carl Chenet for the patch. (Closes: #523958) - fail gracefully if piuparts is run as non-root user. Thanks to Ignace Mouzannar for the patch. (Closes: #538273) * piupartslib/packagesdb.py: - change the test whether a package is testable to check whether the package is of priority "required", and not whether it's "Essential". - rename status "essential-required-important" to "essential-required" as important packages can be tested like all the others. - handle binNMUs where the Source header in the Packages file includes the Version and the Source package name in one line. - fix get_packages_in_state() to only return unique packages and not also the provided ones as exact copies. * Install scripts for master-slave mode without .py extension. * piuparts-master.py: remove code to write statistics, that is done only in piuparts-report.py now. * piuparts-slave.py: - support looping trough multiple sections in piuparts.conf via new config value "sections". Thanks to Frank Ploss for writing this patch with me and helping me understand the code much better! - allow the user to stop the slave with control-C. - call piuparts with --mirror when mirror is set in piuparts.conf. * Re-add piuparts-reports.py which got removed/lost between 0.14 and 0.20 without changelog entry. * piuparts-report.py: - support sections in configuration file. - support looping trough multiple sections in piuparts.conf via new config value "sections". - generate counts.txt (per section) with raw numbers of packages per state, include these stats in the debug output (so it gets included in the mails send by cron.) - introduce new setting "sources-url" for piuparts.conf. - generate sources.txt (per section) with a summary status per source package (for the PTS to display a source packages piuparts state) - generate html status pages for all source packages (to link from the PTS) with links to state explaination and available logfiles, handle udebs. - provide links to logfiles in statistics page. - provide links to source packages pages from state pages and back, as well as links to the dependencies state. - draw graphs of package states over time (if pypthon-rpy and gs are installed) - thanks to Thomas Viehmann and Dirk Eddelbüttel for inspiration and support. - create maintainer centric pages for each email address found in Maintainers or Uploaders. - improve layout, generate navigation for all pages. - use icons to provide a quick overview of a source packages status. (Thanks to tango.freedesktop.org for the icons!) - include index.tpl (if it exists) from output-directory into the generated index page, so one can add news to the index page without editing piuparts-report.py. - generate pages for known issues, which are detected by bash helper script, run by cron on piuparts.debian.org and available from svn/piuparts/piatti/home/piupartsm/bin/detect_well_known_errors * Always use alphabetic time zone abbreviation in timestamps. * Makefile: - add "~$date" to versionstring if building an unreleased version, thus adding dpkg-dev to build-depends. - split install target into install-doc, install-conf and install, to aid development and deployment from trunk. * debian/control: - depend on python (>>2.4), make dependency to python-debian unversioned. - add build-dependencies on debhelper, asciidoc and xmlto. - dependency gs has been renamed to ghostcript - remove build-dependencies on docbook2x and docbook-xml. - suggest python-rpy and gs to improve piuparts-report output. - set Homepage: to http://piuparts.debian.org - remove Lucas Nussbaum, Ana Guerrero, John Wright and Bill Allombert from uploaders - thanks for your work! - Replace Ians email address with one that doesn't give an errorcode when sending mail to it - bump Standards-Version to 3.8.3, no changes necessary. * Rewrite debian/rules from scratch using debhelper. * Merge README, how-to-use-piuparts.txt and custom-scripts.txt into README.txt, convert it to asciidoc and build pdf and html versions of it. * Restructure and update README.txt to reflect the configuration changes in master-slave mode. * Add debian/NEWS file. * Rewrite piuparts manpage in asciidoc. * Build and install html version of the piuparts manpage. * Update debian/copyright to reflect that piuparts is licenced under GPL2+. * Update FSF address in all files referencing the GPL. * Remove unused file piuparts.css. [ Lars Wirzenius ] * Removed Lars Wirzenius as uploader. -- Holger Levsen Tue, 08 Sep 2009 14:35:42 +0200 piuparts (0.35) unstable; urgency=medium * Fix recursive failure which occured when selinux-utils was installed but not enabled. Thanks to Peter De Wachter for the patch. (Closes: #519017) * Output path to temp directory if -k is used. (Closes: #466112) -- Holger Levsen Tue, 10 Mar 2009 00:43:22 +0100 piuparts (0.34) unstable; urgency=low [ Holger Levsen ] * Mount /selinux on systems where selinux is enabled. Thanks to Filippo Giunchedi for the patch! (CLoses: 507171) * Remove wrong advice in debian/copyright which irritated lintian. [ Filippo Giunchedi ] * Check for any output when running cron files in addition to exit code -- Holger Levsen Fri, 27 Feb 2009 12:34:31 +0100 piuparts (0.33) unstable; urgency=low * Added --bindmount option, thanks to Aníbal Monsalve Salaza for the patch. (Closes: #496186) -- Holger Levsen Sat, 08 Nov 2008 17:07:22 +0000 piuparts (0.32) unstable; urgency=low [ John Wright ] * Fix a typo in how-to-use-piuparts.txt. [ Holger Levsen ] * Replace all references to sarge and etch with etch and lenny. (Closes: #466111) * Update README to reflect that piuparts runs fine in etch. * Rename the post_upgrade custom script to post_distupgrade and introduce pre_distupgrade. * Bumped standards version, no changes needed. -- Holger Levsen Fri, 17 Oct 2008 10:28:14 +0000 piuparts (0.31) unstable; urgency=low [ Lars Wirzenius ] * piuparts.docbook: Added a few words of description of the tarball that -b expects. (Closes: 466048) [ Lucas Nussbaum ] * Added a --debfoster-options option, to allow the user to override debfoster's default options and test with more packages installed in the chroot. (Closes: #458922) * Mention piuparts -s in in how-to-use-piuparts.txt. * Finally fixes the check for broken symlinks. Thanks go to Tobias Grimm for the patch. (Closes: #468157, 478587) * Ignore /var/cache/ldconfig/aux-cache. * Added myself to uploaders. * Keep /proc mounted. Switch to calling tar directly in pack_into_tgz(). Minor refactorings. Thanks go to Tobias Grimm for the patch. (Closes: #478577) * Move piuparts to priority: extra, since it depends on debootstrap which is extra. (Closes: #477634) [ Luk Claes ] * Added -w to lsof call to suppress warnings. (Closes: #466102). [ Holger Levsen ] * Add a copyright statement to debian/copyright. * Add support for post_upgrade custom script. * Minor fixes in how-to-use-piuparts.txt. * piuparts.docbook: Correct the path to dtd, it's /usr/share/xml/docbook/schema/dtd/4.1.2/docbookx.dtd and add a build-dependency for docbook-xml -- Holger Levsen Sun, 11 May 2008 22:17:52 +0200 piuparts (0.30) unstable; urgency=low * piuparts.py: - Implement Chroot.create_temp_tgz_file() (since it's used in the VirtServ subclass) - Fix a typo -- chroot.create_temp_tgz() was being called instead of chroot.create_temp_tgz_file() (Closes: #465416) -- John Wright Thu, 14 Feb 2008 20:32:35 -0700 piuparts (0.29) unstable; urgency=low [ Ana Beatriz Guerrero Lopez ] * Updated uploaders, remove Amaya and Alastair. [ Lars Wirzenius ] * piuparts.py: Call "apt-get update" before calling Chroot.minimize, so that we can find the debfoster package on Ubuntu. * debian/control: Fixed "Uploaders:" to have my preferred e-mail address (liw@iki.fi) instead of my Debian one. * piuparts.py: Added -D option to set Debian flavor, plus two sets of default settings depending on the flavor, one for Debian itself, and another for Ubuntu. The settings choose default mirror and distribution set (sid vs gutsy). This will allow an Ubuntu version of the package to set defaults at package building time. * piuparts.py: Report target of broken symlink correctly, instead of saying "True". * piuparts.py: Use lsb-release to guess the Debian flavor. * debian/control: Added dependency on lsb-release. * piuparts-master.py: Make the master write summary of total pass/fail packages, plus status per binary package. This is for having the Debian PTS and the Ubuntu developer weather report show summaries of piuparts results. * integraged changes from Ubuntu by Ian Jackson to support autopkgtest. * wrapped long lines in debian/control. [ Holger Levsen ] * Added myself to uploaders. * Added Homepage: header to debian/control and changed the homepage to be https://wiki.debian.org/piuparts. * Use Vcs-* headers in debian/control instead of XS-Vcs-*. * Bumped policy version to 3.7.3. * Updated download location in debian/copyright. [ John Wright ] * Change the --warn-on-others implementation to create a dummy metapackage with the needed dependencies and install it in the chroot, rather than using "apt-get install package package-". The previous implementation had issues when the package was already installed (and especially if the package had essential packages in its reverse dependencies). This has the pleasant side-effect of making --warn-on-others work both with and without --apt. (Closes: #458929) * Fix a bug in check_for_broken_symlinks(): the targets for the broken symlinks (used for logging) were being read from files on the real root filesystem, rather than in the chroot. -- Lars Wirzenius Fri, 01 Feb 2008 16:38:38 +0200 piuparts (0.28) unstable; urgency=low * Fix an UnboundLocalError introduced in the --warn-on-others fix that would occur if run without the --apt option. (Closes: #443641) -- John Wright Sat, 22 Sep 2007 23:32:17 -0600 piuparts (0.27) unstable; urgency=low * Add support for testing multiple distributions and architectures to piuparts-master.py and piuparts-slave.py. Please see the README file, piuparts-master.conf.sample and piuparts-slave.conf.sample for more details. (Closes: #349365) * Fix the --warn-on-others option. Now, dependencies are installed before the packages we wish to test, and an inventory is taken then, so that we can know which errors were caused by the packages explicitly specified on the command-line. Currently, this requires --apt, and doesn't work quite as advertised if there are circular dependencies with the packages you wish to test (see the man page for more details). (Closes: #440310) * debian/control: - Update my email address in the Uploaders field -- John Wright Sun, 16 Sep 2007 22:28:14 -0600 piuparts (0.26) unstable; urgency=low * Update list of ignored files. (Closes: #439592) * In the cron test, not executable files are not run. (Closes: #440141) -- Ana Beatriz Guerrero Lopez Fri, 31 Aug 2007 15:44:36 +0200 piuparts (0.25) unstable; urgency=low * Add the option to run custom scripts inside the piuparts chroot. Scripts are stored in a directory and give it as argument with the option --scriptsdir=/dir/with/the/scripts This feature is still experimental :) * Add custom-scripts.txt with information about the custom scripts. -- Ana Beatriz Guerrero Lopez Tue, 28 Aug 2007 14:39:32 +0200 piuparts (0.24) unstable; urgency=low * Add /var/lib/apt/extended_states to ignored_files. Thanks Anibal! (Closes: #434980) * Add quick howto about how to use piuparts under docs. * Add test that checks the output from the cron files left in the system after removing a package. This includes the option --skip-cronfiles-test, to allow skipping this test. -- Ana Beatriz Guerrero Lopez Mon, 20 Aug 2007 10:27:29 +0200 piuparts (0.23) unstable; urgency=low * Cosmetic change, now file owner and file are shown in the same line. * Add option --list-installed-files, that list files added/removed/modified to the chroot after the package's installation. (Closes: #431821) * Add option --no-upgrade-test, that allows skip testing upgrade from an existing version in the archive. (Closes: #349933) -- Ana Beatriz Guerrero Lopez Tue, 10 Jul 2007 19:47:21 +0200 piuparts (0.22) unstable; urgency=low [ John Wright ] * debian/control: - Add XS-Vcs-Svn and XS-Vcs-Browser fields [ Ana Beatriz Guerrero Lopez ] * Rename piuparts.py.in back to piuparts.py. * Add option --skip-minimize that allows skip the minimize chroot step with debfoster. * Remove m4 substitution, and use sed instead. Drop Build-Depends on m4. (Closes: #431248) * Drop support for python 2.3 and add support for python 2.5. * debian/rules: - Remove execution of checkversion in package build, not longer needed. * debian/control: - Add myself to uploaders. - Add (future) piuparts website. -- Ana Beatriz Guerrero Lopez Wed, 04 Jul 2007 21:03:44 +0200 piuparts (0.21) unstable; urgency=low * Convert to Debian-native packaging style. From now on, changes to piuparts will be tracked here. (Closes: #389610) * Makefile: - Give docbook2x-man a --encoding=utf-8 argument. It was failing to build. - Generate piuparts.py from piuparts.py.in, filling in the VERSION variable with the version from debian/changelog * debian/control: - Fix the name of the maintainer (we're the "piuparts developers team", not Lustre) - Correct my email address in the uploaders field - Add Build-Depends on m4 * piuparts.py.in: - Don't call shellquote() on arguments, since we're not using the shell. (Closes: #386839) - Add a --warn-on-others option. See the man page for details. * piuparts.docbook: - Document the --warn-on-others option. -- John Wright Thu, 21 Jun 2007 00:27:22 +0100 piuparts (0.20-3) unstable; urgency=low * New Maintainer(s): piuparts team. Closes: #390754. -- Alastair McKinstry Mon, 23 Oct 2006 16:02:19 +0100 piuparts (0.20-2) unstable; urgency=low * Orphaning. -- Lars Wirzenius Mon, 2 Oct 2006 23:43:00 +0300 piuparts (0.20-1) unstable; urgency=low * New upstream version. Fixed Debian bugs: - Symlink correctness checking was broken when symlinks pointed at symlinks. (Closes: #387796) - fails if a deb filename contains a "strange" char. (Closes: #386839) -- Lars Wirzenius Fri, 22 Sep 2006 12:58:24 +0300 piuparts (0.19-1) unstable; urgency=low * New upstream version. No Debian bugs fixed. -- Lars Wirzenius Fri, 8 Sep 2006 20:28:31 +0300 piuparts (0.18-1) unstable; urgency=low * New upstream version. No Debian bugs fixed. * debian/control: Updated for new Python policy. * debian/prerm, debian/postrm: Written. They compile/remove byte code files. -- Lars Wirzenius Thu, 7 Sep 2006 20:42:03 +0300 piuparts (0.17-1) unstable; urgency=low * debian/control: Added dependency on lsof, since that is now used to check that there are no processes running inside the chroot. * New upstream version, fixes the following open bugs in Debian: - missing words in piuparts(1) (Closes: 362551) - default ignore pattern for papercut files added (Closes: #355401) -- Lars Wirzenius Sun, 25 Jun 2006 19:35:19 +0300 piuparts (0.16-1) unstable; urgency=low * New upstream version. Fixes the following bugs reported against the Debian package: - TODO: keep track of bugs to show expected failures and unexpected successes (Closes: #353215) - It would be nice if the chroot were updated before the snapshot (Closes: #356678) * debian/control: Made the dependency on debootstrap be versioned. Closes: #355875. -- Lars Wirzenius Fri, 22 Mar 2006 22:40:00 +0200 piuparts (0.15-1) unstable; urgency=low * New upstream version. Includes fixes for Debian bugs: - "Specify packages to use for testing" (Closes: #354811) - "Misleading documentation of -v" (Closes: #352941) -- Lars Wirzenius Wed, 01 Mar 2006 20:53:00 +0200 piuparts (0.14-1) unstable; urgency=low * New upstream version. Includes fix for: - "piuparts should flag as an error a failed installation of a dependency" (Closes: #343324) -- Lars Wirzenius Sun, 18 Dec 2005 23:22:00 +0200 piuparts (0.13-1) unstable; urgency=low * New upstream version. * debian/changelog: Minor tweaking to the description. -- Lars Wirzenius Sat, 12 Nov 2005 00:53:04 +0200 piuparts (0.12-1) unstable; urgency=low * New upstream version. Fixes bug in Debian: - dist-upgrade to experimental does not work (now documented as a feature). Closes: #330749. -- Lars Wirzenius Mon, 17 Oct 2005 21:03:12 +0300 piuparts (0.11-1) unstable; urgency=low * New upstream version. Fixes bugs in Debian: - Checks whether there are packages on the command line before creating a chroot. Closes: #322441. - apt-get configured to allow unauthenticated repositories. Closes: #326705. -- Lars Wirzenius Fri, 14 Oct 2005 01:16:25 +0300 piuparts (0.10-1) unstable; urgency=low * New upstream version. -- Lars Wirzenius Thu, 15 Sep 2005 23:31:30 +0300 piuparts (0.9-1) unstable; urgency=low * New upstream version, fixes bugs reported against Debian: - Upstream version number in Debian package and in upstream sources now match. Closes: #326058. - Files reported by piuparts are now associated with their packages, when possible. Closes: #324248. * debian/rules: Added "checkversion" target to make sure we no longer do stupid things like #326058, at least in the Debian package. -- Lars Wirzenius Fri, 09 Sep 2005 00:34:36 +0300 piuparts (0.8-1) unstable; urgency=low * New upstream version, fixes bugs: - Example in manual page now works (Closes: #319990) - Temporary directories are now removed on failure (Closes: #323653) - Parsing /etc/apt/sources.list now better (Closes: #319989) - Temporary directorys are now put in $TMPDIR or /etc (Closes: #322440) * debian/control: Added build-depends on docbook2x. Closes: #318693 * debian/changelog: Added note about upstream source repository. -- Lars Wirzenius Mon, 29 Aug 2005 23:03:32 +0300 piuparts (0.7-1) unstable; urgency=low * New upstream version. * This is the first package that will be uploaded to the Debian archive. -- Lars Wirzenius Fri, 15 Jul 2005 13:09:00 +0300 piuparts (0.6-1) unstable; urgency=low * First release of the Debian package. -- Lars Wirzenius Tue, 5 Jul 2005 20:08:00 +0300 piuparts-1.1.1/debian/clean0000644000000000000000000000001213605426771012502 0ustar .gocache/ piuparts-1.1.1/debian/control0000644000000000000000000001305413605426771013112 0ustar Source: piuparts Maintainer: piuparts developers team Section: devel Priority: optional Standards-Version: 4.4.1 Rules-Requires-Root: no Build-Depends: debhelper-compat (= 12), dh-python, python3-all, python3-apt, python3-debian, python3-debianbts (>= 2.10.0), python3-distro-info, python3-mox3, python3-nose, python3-six, python3-sphinx, python3-yaml, golang-any, Build-Depends-Indep: git, Homepage: https://piuparts.debian.org Vcs-Git: https://salsa.debian.org/debian/piuparts.git Vcs-Browser: https://salsa.debian.org/debian/piuparts Xs-Go-Import-Path: salsa.debian.org/debian/piuparts Package: piuparts Architecture: all Depends: piuparts-common (= ${binary:Version}), # keep this list in sync with piuparts-slave-from-git-deps debootstrap (>= 1.0.89~), debsums (>= 2.2.2~), libjs-sphinxdoc, lsb-release, lsof, mount, python3-debian, ${python3:Depends}, ${misc:Depends}, Recommends: adequate Suggests: schroot, docker.io, Description: .deb package installation, upgrading, and removal testing tool piuparts tests that .deb packages (as used by Debian) handle installation, upgrading, and removal correctly. It does this by creating a minimal Debian installation in a chroot, and installing, upgrading, and removing packages in that environment, and comparing the state of the directory tree before and after. piuparts reports any files that have been added, removed, or modified during this process. . piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive. Package: piuparts-master Architecture: any Built-Using: ${misc:Built-Using} Depends: piuparts-common (>= ${source:Version}), piuparts-common (<< ${source:Version}+), # keep this list in sync with piuparts-master-from-git-deps adduser, openssh-server, python3-debianbts (>= 2.10.0), python3-setproctitle, tango-icon-theme, xz-utils, python3-yaml, ${python3:Depends}, ${shlibs:Depends}, ${misc:Depends}, Recommends: apache2 | httpd, ghostscript, python3-rpy2, r-base-dev, r-recommended, devscripts, libsoap-lite-perl, Description: piuparts master components piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive. A master/slave piuparts installation is used for testing complete archives as it is done on https://piuparts.debian.org/. . This package provides the piuparts-master, which needs one or more slaves to operate. It also contains piuparts-report to generate web pages with the testing results as well as documentation. Package: piuparts-master-from-git-deps Architecture: all Depends: git, # this list is synced from piuparts-master adduser, openssh-server, python3-debianbts (>= 2.10.0), python3-setproctitle, tango-icon-theme, xz-utils, python3-yaml, # selected packages from piuparts-master Recommends apache2 | httpd, python3-rpy2, # this list is synced from piuparts-common python3-apt, python3-distro-info, ${misc:Depends}, ${python3:Depends}, Description: dependencies for running piuparts master from git piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive. A master/slave piuparts installation is used for testing complete archives as it is done on https://piuparts.debian.org/. . This metapackage provides all the dependencies needed for running a piuparts-master instance (including piuparts-report etc.) from git. Package: piuparts-slave Architecture: all Depends: piuparts (= ${binary:Version}), piuparts-common (= ${binary:Version}), # keep this list in sync with piuparts-slave-from-git-deps adduser, openssh-client, screen, sudo, ${misc:Depends}, ${python3:Depends}, Suggests: schroot Description: piuparts slave components piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive. A master/slave piuparts installation is used for testing complete archives as it is done on https://piuparts.debian.org/. . This package provides the piuparts slave components, which need a piuparts master to operate. See the documentation in the piuparts-master package. Package: piuparts-slave-from-git-deps Architecture: all Depends: git, # this list is synced from piuparts-slave adduser, openssh-client, screen, sudo, # this list is synced from piuparts debootstrap (>= 1.0.89~), debsums (>= 2.2.2~), libjs-sphinxdoc, lsb-release, lsof, mount, python3-debian, # this list is synced from piuparts-common python3-apt, python3-distro-info, ${misc:Depends}, ${python3:Depends}, Description: dependencies for running piuparts slave from git piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive. A master/slave piuparts installation is used for testing complete archives as it is done on https://piuparts.debian.org/. . This metapackage provides all the dependencies needed for running a piuparts-slave instance from git. Package: piuparts-common Architecture: all Depends: # keep this list in sync with piuparts-{master,slave}-from-git-deps python3-apt, python3-distro-info, python3-six, python3-urllib3, ${misc:Depends}, ${python3:Depends}, Description: common piuparts components piupartslib, common library used by piuparts-master, -report, -analyze and -slave. . See https://piuparts.debian.org or the piuparts package to learn more about piuparts. piuparts-1.1.1/debian/copyright0000644000000000000000000000426213604710767013443 0ustar Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ Upstream-Name: piuparts Upstream-Contact: Holger Levsen Source: https://salsa.debian.org/debian/piuparts.git Copyright: © 2005-2008 Lars Wirzenius © 2008-2019 Holger Levsen © 2011-2019 Andreas Beckmann Comment: Original author is also Lars Wirzenius Upstream is bunch of lunatics who don't make release tarballs publicly available. The Debian .orig.tar.gz is what would be released, though. Files: * Copyright: © 2005-2008 Lars Wirzenius © 2008-2019 Holger Levsen © 2011-2019 Andreas Beckmann License: GPL-2+ This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. . This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. . You should have received a copy of the GNU General Public License along with this program. If not, see . On Debian systems, the full text of the GNU General Public License version 2 can be found in the file '/usr/share/common-licenses/GPL-2'. Files: htdocs/images/debian.png htdocs/images/openlogo-nd-50.png Copyright: 1999 Software in the Public Interest License: Free This logo or a modified version may be used by anyone to refer to the Debian project, but does not indicate endorsement by the project. . Note: we would appreciate that you make the image a link to https://www.debian.org/ if you use it on a web page. Files: htdocs/images/valid-html401.png htdocs/images/w3c-valid-css.png Copyright: 2009 W3C License: FreeUse In most cases, W3C logos and icons may be used without requesting permission from W3C. When the logo usage policy requires that you obtain express written permission, please send your request to the W3C Communications Team (w3t-pr@w3.org) piuparts-1.1.1/debian/piuparts-common.install0000644000000000000000000000010513605426771016225 0ustar etc/piuparts/distros.conf usr/lib/python*/* usr/share/piuparts/lib/* piuparts-1.1.1/debian/piuparts-common.pyinstall0000644000000000000000000000022613266762112016575 0ustar piupartslib/__init__.py piupartslib piupartslib/conf.py piupartslib piupartslib/dependencyparser.py piupartslib piupartslib/packagesdb.py piupartslib piuparts-1.1.1/debian/piuparts-master.dirs0000644000000000000000000000002113266762112015513 0ustar var/lib/piuparts piuparts-1.1.1/debian/piuparts-master.docs0000644000000000000000000000006113605426771015513 0ustar docs/README_server.txt docs/README_pejacevic.txt piuparts-1.1.1/debian/piuparts-master.examples0000644000000000000000000000015513266762112016400 0ustar instances/piuparts.conf.anbe instances/piuparts.conf.pejacevic conf/piuparts.conf.sample conf/crontab-master piuparts-1.1.1/debian/piuparts-master.install0000644000000000000000000000055113266771660016240 0ustar etc/apache2/conf-available/piuparts-master.conf #etc/piuparts/known_problems/ usr/share/piuparts/htdocs/* usr/share/piuparts/master/* usr/share/piuparts/known_problems/* usr/share/piuparts/piuparts-analyze usr/share/piuparts/piuparts-master usr/share/piuparts/piuparts-master-backend usr/share/piuparts/piuparts-report usr/lib/piuparts/debiman-piuparts-distill piuparts-1.1.1/debian/piuparts-master.lintian-overrides0000644000000000000000000000077713605426771020237 0ustar # It's ok, because it depends on piuparts-common that has the dependency. piuparts-master: python-script-but-no-python-dep usr/share/piuparts/master/detect_well_known_errors #!/usr/bin/python3 piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-analyze #!/usr/bin/python3 piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-master-backend #!/usr/bin/python3 piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-report #!/usr/bin/python3 piuparts-1.1.1/debian/piuparts-master.maintscript0000644000000000000000000000014713266762112017120 0ustar mv_conffile /etc/apache2/conf.d/piuparts.apache /etc/apache2/conf-available/piuparts-master.conf 0.58~ piuparts-1.1.1/debian/piuparts-master.postinst0000644000000000000000000000325313266762112016447 0ustar #!/bin/sh set -e pgroup=piuparts muser=piupartsm scriptroot=/usr/share/piuparts userhome=/var/lib/piuparts if [ "$1" = "configure" ] ; then addgroup --system --quiet $pgroup for user in $muser do adduser --system --quiet --home $userhome/$user --ingroup $pgroup \ --shell /bin/sh $user if ! [ -d $userhome/$user ]; then mkdir -m 0755 $userhome/$user chown $user:$pgroup $userhome/$user fi touch $userhome/$user/.profile chown $user:$pgroup $userhome/$user/.profile if [ $user = "piupartsm" ] ; then role="master" else role="slave" fi if ! grep -q $scriptroot/$role $userhome/$user/.profile ; then echo PATH=\$PATH:$scriptroot:$scriptroot/$role \ >>$userhome/$user/.profile fi if [ ! -d $userhome/$user/.ssh ] ; then install -d -o $user -g $pgroup -m 0755 $userhome/$user/.ssh fi done install -d -o $muser -g $pgroup -m 0755 $userhome/master install -d -o $muser -g $pgroup -m 0755 $userhome/backup install -d -o $muser -g $pgroup -m 0755 $userhome/htdocs cp -rf $scriptroot/htdocs/* $userhome/htdocs/ for icon in weather-severe-alert.png sunny.png do if [ -h $userhome/htdocs/images/$icon ] ; then rm -f $userhome/htdocs/images/$icon fi if [ -e /usr/share/icons/Tango/24x24/status/$icon ] ; then cp -f /usr/share/icons/Tango/24x24/status/$icon $userhome/htdocs/images/$icon fi done if dpkg --compare-versions "$2" lt-nl "0.50" ; then rm -rf $userhome/master/monitor-slave fi fi #DEBHELPER# piuparts-1.1.1/debian/piuparts-master.postrm0000644000000000000000000000075513266762112016114 0ustar #!/bin/sh set -e user=piupartsm if [ "$1" = "purge" ] ; then rm -rf /var/lib/piuparts/$user if [ -d /var/lib/piuparts/htdocs ] ; then rm -rf /var/lib/piuparts/htdocs fi if [ -d /var/lib/piuparts/backup ] ; then rm -rf /var/lib/piuparts/backup fi if [ -d /var/lib/piuparts/master ] ; then rm -rf /var/lib/piuparts/master fi rmdir /var/lib/piuparts 2>/dev/null || true rmdir /etc/piuparts 2>/dev/null || true fi #DEBHELPER# piuparts-1.1.1/debian/piuparts-slave.dirs0000644000000000000000000000010313266762112015333 0ustar var/lib/piuparts var/cache/piuparts/basetgz var/cache/piuparts/tmp piuparts-1.1.1/debian/piuparts-slave.examples0000644000000000000000000000005713266762112016220 0ustar conf/crontab-slave conf/piuparts-slave.sudoers piuparts-1.1.1/debian/piuparts-slave.install0000644000000000000000000000013013266762112016040 0ustar etc/piuparts/piuparts.conf usr/share/piuparts/piuparts-slave usr/share/piuparts/slave/* piuparts-1.1.1/debian/piuparts-slave.links0000644000000000000000000000030413266771660015525 0ustar /usr/share/piuparts/slave/slave_run /usr/bin/piuparts_slave_run /usr/share/piuparts/slave/slave_join /usr/bin/piuparts_slave_join /usr/share/piuparts/slave/slave_stop /usr/bin/piuparts_slave_stop piuparts-1.1.1/debian/piuparts-slave.lintian-overrides0000644000000000000000000000025713605426771020047 0ustar # It's ok, because it depends on piuparts-common that has the dependency. piuparts-slave: python-script-but-no-python-dep usr/share/piuparts/piuparts-slave #!/usr/bin/python3 piuparts-1.1.1/debian/piuparts-slave.manpages0000644000000000000000000000024413266771660016203 0ustar debian/tmp/usr/share/man/man8/piuparts_slave_run.8.gz debian/tmp/usr/share/man/man8/piuparts_slave_join.8.gz debian/tmp/usr/share/man/man8/piuparts_slave_stop.8.gz piuparts-1.1.1/debian/piuparts-slave.postinst0000644000000000000000000000312513266762112016264 0ustar #!/bin/sh set -e pgroup=piuparts suser=piupartss scriptroot=/usr/share/piuparts userhome=/var/lib/piuparts if [ "$1" = "configure" ] ; then addgroup --system --quiet $pgroup for user in $suser do adduser --system --quiet --home $userhome/$user --ingroup $pgroup \ --shell /bin/sh $user if ! [ -d $userhome/$user ]; then mkdir -m 0755 $userhome/$user chown $user:$pgroup $userhome/$user fi touch $userhome/$user/.profile chown $user:$pgroup $userhome/$user/.profile if [ $user = "piupartsm" ] ; then role="master" else role="slave" fi if ! grep -q $scriptroot/$role $userhome/$user/.profile ; then echo PATH=\$PATH:$scriptroot:$scriptroot/$role \ >>$userhome/$user/.profile fi if [ ! -d $userhome/$user/.ssh ] ; then install -d -o $user -g $pgroup -m 0755 $userhome/$user/.ssh fi done if [ ! -e $userhome/$suser/.ssh/id_rsa ] ; then ssh-keygen -q -f $userhome/$suser/.ssh/id_rsa -N "" chown $suser:$pgroup $userhome/$suser/.ssh/id_rsa \ $userhome/$suser/.ssh/id_rsa.pub fi if [ ! -e $userhome/$suser/.ssh/known_hosts ] ; then ssh-keyscan localhost > $userhome/$suser/.ssh/known_hosts 2>/dev/null chown $suser:$pgroup $userhome/$suser/.ssh/known_hosts fi install -d -o $suser -g $pgroup -m 0755 $userhome/slave install -d -o $suser -g $pgroup -m 0755 $userhome/slave/0 chown $suser:$pgroup /var/cache/piuparts/basetgz fi #DEBHELPER# piuparts-1.1.1/debian/piuparts-slave.postrm0000644000000000000000000000112013266762112015716 0ustar #!/bin/sh set -e user=piupartss if [ "$1" = "purge" ] ; then rm -rf /var/lib/piuparts/$user if [ -d /var/lib/piuparts/slave ] ; then rm -rf /var/lib/piuparts/slave fi rmdir /var/lib/piuparts 2>/dev/null || true rmdir /etc/piuparts 2>/dev/null || true rm -rf /var/cache/piuparts/basetgz/*.log rm -rf /var/cache/piuparts/basetgz/*.tar.gz # rm -rf is safe if it's not mounted anymore mount | grep -q /var/cache/piuparts/tmp || rm -rf --one-file-system /var/cache/piuparts/tmp rmdir /var/cache/piuparts 2>/dev/null || true fi #DEBHELPER# piuparts-1.1.1/debian/piuparts.NEWS0000644000000000000000000000402313266762112014043 0ustar piuparts (0.45) unstable; urgency=low For those interested in running their own local instance of https://piuparts.debian.org there are now piuparts-master and piuparts-slave packages for ease of installation. This is useful for both developing piuparts as well as doing QA on the Debian archive or other .deb archives. See README_server.txt and README_piatti.txt in the piuparts-master package for more information. -- Holger Levsen Mon, 18 Jun 2012 09:26:27 +0200 piuparts (0.38) unstable; urgency=low New default behaviours in piuparts: * --skip-minimize is now the default. This is because debootstrap is called with --variant=minbase instead of --resolv-deps now and because if a base.tgz or an lvm snapshot is specified, piuparts should not touch it. (Closes: #539142) Introduce a new option, --minimize, to minimize a chroot with debfoster. (This is the old default behaviour. In future versions the --minimize option might be removed again.) Add two new options: --keyring to specify the keyring to use with debootstrap (defaults to /usr/share/keyrings/debian-archive-keyring.gpg) and --do-not-verify-signatures to not use --keyring with debootstrap. (Closes: #545907) In the chroots, APT::Get::AllowUnauthenticated is set accordingly in /etc/apt/apt.conf. Add another type of custom-scripts, post_setup_, which are executed after the chroot was setup. (Closes: #466043) -- Holger Levsen Thu, 17 Dec 2009 14:10:27 +0000 piuparts (0.36) unstable; urgency=low piuparts 0.36 introduces many changes to master-slave mode, please read debian/changelog carefully. Backwards compatibility of the configuration files for master-slave-mode has been broken, as the three config files got merged into one: /etc/piuparts/piuparts.conf piuparts-reports has been re-added and improved. The documentation files have been merged and converted to asciidoc. -- Holger Levsen Tue, 10 Mar 2009 15:23:59 +0100 piuparts-1.1.1/debian/piuparts.doc-base0000644000000000000000000000041613605426771014753 0ustar Document: piuparts Title: Piuparts Manual Author: Holger Abstract: This manual describes what Piuparts is, and how it can be used. Section: System/Package Management Format: HTML Index: /usr/share/doc/piuparts/html/index.html Files: /usr/share/doc/piuparts/html/*.html piuparts-1.1.1/debian/piuparts.docs0000644000000000000000000000007013605426771014222 0ustar docs/README.txt debian/tmp/usr/share/doc/piuparts/html/ piuparts-1.1.1/debian/piuparts.install0000644000000000000000000000005213266762112014733 0ustar /usr/sbin/piuparts /etc/piuparts/scripts* piuparts-1.1.1/debian/piuparts.links0000644000000000000000000000062313605426771014416 0ustar /usr/share/javascript/sphinxdoc/1.0/jquery.js usr/share/doc/piuparts/html/_static/jquery.js /usr/share/javascript/sphinxdoc/1.0/underscore.js usr/share/doc/piuparts/html/_static/underscore.js /usr/share/javascript/sphinxdoc/1.0/doctools.js usr/share/doc/piuparts/html/_static/doctools.js /usr/share/javascript/sphinxdoc/1.0/searchtools.js usr/share/doc/piuparts/html/_static/searchtools.js piuparts-1.1.1/debian/piuparts.lintian-overrides0000644000000000000000000000072113416657733016737 0ustar # it is impossible not to do this without losing compatibility with ancient releases uses-dpkg-database-directly etc/piuparts/scripts-debug-purge/post_remove_postrm_set-x uses-dpkg-database-directly etc/piuparts/scripts-debug-remove/pre_remove_prerm_postrm_set-x uses-dpkg-database-directly etc/piuparts/scripts/post_remove_exceptions uses-dpkg-database-directly etc/piuparts/scripts/pre_remove_40_find_missing_md5sums uses-dpkg-database-directly usr/sbin/piuparts piuparts-1.1.1/debian/piuparts.maintscript0000644000000000000000000000123613266771660015637 0ustar rm_conffile /etc/piuparts/piuparts.conf 0.45~ rm_conffile /etc/piuparts/scripts-leftovers/post_purge_dbconfig_cleanup 0.47~ mv_conffile /etc/piuparts/scripts/post_install_find_bad_permissions /etc/piuparts/scripts/pre_remove_50_find_bad_permissions 0.47~ rm_conffile /etc/piuparts/scripts/post_remove_extras 0.61~ rm_conffile /etc/piuparts/scripts/post_distupgrade_experimental 0.74~ rm_conffile /etc/piuparts/scripts-debug-prerm/pre_remove_prerm_set-x 0.74~ rm_conffile /etc/piuparts/scripts/post_distupgrade_hack_debsums 0.75~ rm_conffile /etc/piuparts/scripts/post_setup_experimental 0.76~ rm_conffile /etc/piuparts/scripts-squeeze/post_remove_exceptions_lenny 0.81~ piuparts-1.1.1/debian/piuparts.manpages0000644000000000000000000000005513266762112015063 0ustar debian/tmp/usr/share/man/man1/piuparts.1.gz piuparts-1.1.1/debian/piuparts.postrm0000644000000000000000000000023513266762112014614 0ustar #!/bin/sh set -e if [ "$1" = "purge" ] ; then # may be left over from a previous version rmdir /etc/piuparts 2>/dev/null || true fi #DEBHELPER# piuparts-1.1.1/debian/rules0000755000000000000000000000076213607334164012565 0ustar #!/usr/bin/make -f # Uncomment this to turn on verbose mode. #export DH_VERBOSE=1 export GOCACHE := $(CURDIR)/.gocache %: dh $@ --with python3 override_dh_auto_build: $(MAKE) prefix=/usr build build-doc override_dh_auto_install: $(MAKE) DESTDIR=$(CURDIR)/debian/tmp prefix=/usr etcdir=/etc install install-doc install-conf override_dh_python3: dh_python3 -p piuparts-master -p piuparts-slave /usr/share/piuparts --shebang=/usr/bin/python3 dh_python3 -N piuparts-master -N piuparts-slave piuparts-1.1.1/debian/source/0000755000000000000000000000000013266762112012777 5ustar piuparts-1.1.1/debian/source/format0000644000000000000000000000000413266762112014204 0ustar 1.0 piuparts-1.1.1/debian/source/options0000644000000000000000000000002213266762112014407 0ustar tar-ignore = .git piuparts-1.1.1/debian/tests/0000755000000000000000000000000013607334164012642 5ustar piuparts-1.1.1/debian/tests/control0000644000000000000000000000006613605426771014253 0ustar Tests: smoke-test Depends: @ Restrictions: needs-root piuparts-1.1.1/debian/tests/smoke-test0000755000000000000000000000102113607334164014655 0ustar #!/bin/sh set -e echo running $0 test_this() { echo echo "running $@" $@ } test_this piuparts --version WORKDIR=$(mktemp -d) trap "rm -rf $WORKDIR" 0 INT QUIT ABRT PIPE TERM cd $WORKDIR mkdir -p t/DEBIAN t/usr t/etc cat >t/DEBIAN/control < Priority: optional Architecture: all Installed-Size: 0 Description: Auto Package Test Dummy Extremely simple binary package for piuparts testing EOF dpkg-deb -b t test_this piuparts t.deb piuparts-1.1.1/docs/0000755000000000000000000000000013605426771011212 5ustar piuparts-1.1.1/docs/README.txt0000644000000000000000000002317713605426771012722 0ustar .. raw:: html .. role:: blue .. _top1: README ====== Author: Lars Wirzenius Email: After reading this README you probably also want to have a look at the piuparts manpage, to learn about the available options. But read this document first! :blue:`Introduction` ^^^^^^^^^^^^^^^^^^^^ piuparts is a tool for testing that .deb packages can be installed, upgraded, and removed without problems. The name, a variant of something suggested by Tollef Fog Heen, is short for "package installation, upgrading, and removal testing suite". piuparts is licensed under the GNU General Public License, version 2, or (at your option) any later version. https://piuparts.debian.org has been testing the Debian archive since the Lenny release in 2009, though responsible maintainers run piuparts locally before uploading packages to the archive. :blue:`How to use piuparts in 5 minutes` :blue:`Basic Usage` ^^^^^^^^^^^^^^^^^^^ Testing your packages with piuparts is as easy as typing at the console prompt::: piuparts sm_0.6-1_i386.deb Note that in order to work, piuparts has to be executed as user root, so you need to be logged as root or use 'sudo'. This will create a sid chroot with debootstrap, where it'll test your package. If you want to test your package in another release, for example, testing, you can do so with::: # piuparts ./sm_0.6-1_i386.deb -d testing By default, this will read the first mirror from your '/etc/apt/sources.list' file. If you want to specify a different mirror you can do it with the option '-m'::: # piuparts ./sm_0.6-1_i386.deb -m http://ftp.de.debian.org/debian It's possible to use -d more than once. For example, to do a first installation in stable, then upgrade to testing, then upgrade to unstable and then upgrade to the local package use this::: # piuparts -d stable -d testing -d unstable ./sm_0.6-1_i386.deb :ref:`top ` :blue:`Some tips` ^^^^^^^^^^^^^^^^^ piuparts also has a manpage, where all available options are explained. If you use piuparts on a regular basis, waiting for it to create a chroot every time takes too much time, even if you are using a local mirror or a caching tool such as approx. Piuparts has the option of using a tarball as the contents of the initial chroot, instead of building a new one with debootstrap. A easy way to use this option is use a tarball created with pbuilder. If you are not a pbuilder user, you can create this tarball with the command (again, as root)::: # pbuilder --create then you only have to remember to update this tarball with::: # pbuilder --update To run piuparts using this tarball::: # piuparts -p ./sm_0.6-1_i386.deb If you want to use your own pre-made tarball::: # piuparts --basetgz=/path/to/my/tarball.tgz ./sm_0.6-1_i386.deb Piuparts also has the option of using a tarball as the contents of the initial chroot, instead of building a new one with pbuilder. You can save a tarball for later use with the '-s' ('*-*-save') piuparts option. Some people like this, others prefer to only have to maintain one tarball. Read the piuparts manpage about the '-p', '-b' and '-s' options While pbuilder itself supports using cdebootstrap, this is not fully supported by piuparts: You will need to use debootstrap or use the '*-*-warn-on-debsums-errors' option for piuparts and then you will still see spurious warnings in the log. :ref:`top ` :blue:`Piuparts tests` ^^^^^^^^^^^^^^^^^^^^^^ By default, piuparts does two tests: . Installation and purging test. . Installation, upgrade and purging tests. The first test installs the package in a minimal chroot, removes it and purges it. The second test installs the current version in the archive of the given packages, then upgrades to the new version (deb files given to piuparts in the input), removes and purges. If you only want to perfom the first test, you can use the option: '*-*-no-upgrade-test' :ref:`top ` :blue:`Testing packages in the config-files-remaining state` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The *-*-install-remove-install option modifies the three piuparts tests in order to test package installation while config files from a previous installation are remaining, but the package itself was removed inbetween. This exercises different code paths in the maintainer scripts. . Installation and purging test: install, remove, install again and purge. . Installation, upgrade and purging test: install the old version, remove, install the new version and purge. . Distupgrade test: install the version from the first distribution, remove, distupgrade to the last distribution, install the new version. :ref:`top ` :blue:`Analyzing piuparts results` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When piuparts finishes all the tests satisfactorily, you will get these lines as final output::: 0m39.5s INFO: PASS: All tests. 0m39.5s INFO: piuparts run ends. Anyway, it is a good idea to read the whole log in order to discover possible problems that did not stop the piuparts execution. If you do not get those lines, piuparts has failed during a test. The latest lines should give you a pointer to the problem with your package. :ref:`top ` :blue:`Custom scripts with piuparts` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can specify several custom scripts to be run inside piuparts. You have to store them in a directory and give it as argument to piuparts: '*-*-scriptsdir=/dir/with/the/scripts' This option can be given multiple times. The scripts from all directories will be merged together (and later ones may overwrite earlier scripts with the same filename). By default this is *not* set to anything. Have a look at '/etc/piuparts/scripts*' to learn which scripts and script directories are shipped by the package. The script prefix determines in which step it is executed. You can run several scripts in every step, they are run in alphabetical order. The scripts need to be executable and are run *inside* the piuparts chroot and can only be shell scripts. If you want to run Python or Perl scripts, you have to install Python or Perl. The chroot where piuparts is run is minimized and does not include Perl. The variable PIUPARTS_OBJECTS is set to the packages currently being tested (seperated by spaces, if applicable) or the .changes file(s) being used. So when running in master-slave mode, it will be set to the (one) package being tested at a time. Depending on the current test, the variable PIUPARTS_TEST is set to . 'install' (installation and purging test), . 'upgrade' (installation, upgrade and purging tests) or . 'distupgrade'. During the 'upgrade' and 'distupgrade' tests, the variable PIUPARTS_PHASE is set to one of the following values: . 'install' while initially installing the packages from the repository, . 'upgrade' when upgrading to the .debs, . 'distupgrade' while reinstalling the packages after 'apt-get dist-upgrade' to ensure they were not removed accidently During the 'install' test, the PIUPARTS_PHASE variable is set to 'install'. The current distribution is available in the variable PIUPARTS_DISTRIBUTION. The following prefixes for scripts are recognized: 'post_chroot_unpack' - after the chroot has been unpacked/debootrapped. Before the chroot gets updated/dist-upgraded initially. 'post_setup\_' - after the *setup* of the chroot is finished. Before metadata of the chroot is recorded for later comparison. 'pre_test\_' - at the beginning of each test. After metadata of the chroot was recorded for later comparison. 'is_testable\_' - before *installing* your package. If this script returns a non-zero return value, the installation of the package will be skipped. With a return value of 1 the test will be reported as successful, but with a return value if 2 it will be reported as failed. Use this to flag packages that cannot be be tested with piuparts by design (e.g. usrmerge), require not publicly available external ressources (e.g. some downloader packages) or are broken beyond repair (e.g. buggy packages in archived releases). Use the return value of 2 for seriously broken packages that can break piuparts. 'pre_install\_' - before *installing* your package. Depending on the test, this may be run multiple times. The PIUPARTS_TEST and PIUPARTS_PHASE variables can be used to distinguish the cases. 'post_install\_' - after *installing* your package and its dependencies. Depending on the test, this may be run multiple times. The PIUPARTS_TEST and PIUPARTS_PHASE variables can be used to distinguish the cases. 'pre_remove\_' - before *removing* your package. Depending on the test, this may be run multiple times. 'post_remove\_' - after *removing* your package. Depending on the test, this may be run multiple times. 'post_purge\_' - after *purging* your package. Depending on the test, this may be run multiple times. 'post_test\_' - at the end of each test. Right before performing final checks and comparing the chroot with the reference chroot metadata. 'pre_distupgrade\_' - before *upgrading* the chroot to the *next distribution*. The next distribution is available in the variable PIUPARTS_DISTRIBUTION_NEXT. 'post_distupgrade\_' - after *upgrading* the chroot to the *next distribution*. The previous distribution is available in the variable PIUPARTS_DISTRIBUTION_PREV. :ref:`top ` :blue:`Example custom scripts` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: text '$ cat post_install_numbers' #!/bin/bash number=`dpkg -l | wc -l` echo "There are $number packages installed." exit 0 '$ cat post_setup_package' #!/bin/sh echo "$PIUPARTS_OBJECTS will now get tested." exit 0 :ref:`top ` :blue:`Distributed testing` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is described in README_server.txt. piuparts-1.1.1/docs/README_pejacevic.txt0000644000000000000000000001542513605426771014730 0ustar .. raw:: html .. role:: blue .. _top3: README_perjacevic ================= :blue:`Notes about the piuparts installation on pejacevic.debian.org and it's slave(s)` This document describes the setup for https://piuparts.debian.org - it's used for reference for the Debian System Administrators (DSA) as well as a guide for other setting up a similar system, with the piuparts source code installed from git. For regular installations we recommend to use the piuparts-master and piuparts-slaves packages as described in /usr/share/doc/piuparts-master/README_server.txt :blue:`Installation` ^^^^^^^^^^^^^^^^^^^^ piuparts.debian.org is a setup running on two systems: * pejacevic.debian.org, running the piuparts-master instance and an apache webserver to display the results. * piu-slave-ubc-01.debian.org, running four piuparts-slave nodes to run the actual tests. :blue:`piuparts installation from source` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ * basically, apt-get build-dep piuparts - in reality both systems get their package configuration from git.debian.org/git/mirror/debian.org.git * pejacevic runs a webserver as well (see below for apache configuration) * Copy 'https://salsa.debian.org/debian/piuparts/blob/develop/update-piuparts-master-setup' and 'https://salsa.debian.org/debian/piuparts/blob/develop/update-piuparts-slave-setup' to the hosts which should be master and slave. (It's possible and has been done for a long time to run them on the same host.( Run the scripts as the piupartsm and piupartss users and clone that git repository into '/srv/piuparts.debian.org/src' in the first place. Then checkout the develop branch. * Ideally provide '/srv/piuparts.debian.org/tmp' on (a sufficiently large) tmpfs. * `sudo ln -s /srv/piuparts.debian.org/etc/piuparts /etc/piuparts` * See below for further user setup instructions. :ref:`top ` :blue:`User setup` ^^^^^^^^^^^^^^^^^^ On pejacevic the piuparts-master user piupartsm needs to be created, on piu-slave-ubc-01 a piupartss user is needed for the slave. Both are members of the group piuparts and '/srv/piuparts.debian.org' needs to be chmod 2775 and chown piuparts(sm):piuparts. :ref:`top ` :blue:`'~/bashrc' for piupartsm and piuparts` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Do this for the piupartsm user on pejacevic and piupartss on the slave(s)::: piupartsm@pejacevic$ cat >> ~/.bashrc <<-EOF # added manually for piuparts umask 0002 export PATH="~/bin:\$PATH" EOF :ref:`top ` :blue:`set up ssh pubkey authentification` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Then create an SSH keypair for piupartss and put it into '/etc/ssh/userkeys/piupartsm' on pejacevic, so the piupartss user can login with ssh and run only piuparts-master. Restrict it like this::: $ cat /etc/ssh/userkeys/piupartsm command="/srv/piuparts.debian.org/share/piuparts/piuparts-master",from="2001:41c8:1000:21::21:7,5.153.231.7",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa ... :ref:`top ` :blue:`Setup sudo for the slave(s)` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is actually done by DSA: '/etc/sudoers' for piu-slave-ubc-01: .. code-block:: text # The piuparts slave needs to handle chroots. piupartss ALL = NOPASSWD: /usr/sbin/piuparts *, \ /bin/umount /srv/piuparts.debian.org/tmp/tmp*, \ /usr/bin/test -f /srv/piuparts.debian.org/tmp/tmp*, \ /usr/bin/rm -rf --one-file-system /srv/piuparts.debian.org/tmp/tmp* :ref:`top ` :blue:`Apache configuration` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Any other webserver will do but apache is used on pejacevic (and maintained by DSA)::: ServerName piuparts.debian.org ServerAdmin debian-admin@debian.org ErrorLog /var/log/apache2/piuparts.debian.org-error.log CustomLog /var/log/apache2/piuparts.debian.org-access.log combined DocumentRoot /srv/piuparts.debian.org/htdocs AddType text/plain .log AddDefaultCharset utf-8 HostnameLookups Off UseCanonicalName Off ServerSignature On UserDir disabled :ref:`top ` :blue:`Running piuparts` ^^^^^^^^^^^^^^^^^^^^^^^^ Updating the piuparts installation Updating the master, pejacevic.debian.org::: holger@pejacevic~$ sudo su - piupartsm update-piuparts-master-setup develop origin Updating the slave(s), for example on piu-slave-ubc-01.debian.org::: holger@piu-slave-ubc-01~$ sudo su - piupartss update-piuparts-slave-setup develop origin :ref:`top ` :blue:`Running piuparts` ^^^^^^^^^^^^^^^^^^^^^^^^ When running piuparts in master/slave mode, the master is never run by itself, instead it is always started by the slave(s). :ref:`top ` :blue:`Starting and stopping the slaves` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Run the following script under *your* user account to start four instances of piuparts-slave on pejacevic, piuparts-master will be started automatically by the slaves.:: holger@piu-slave-ubc-01:~$ sudo -u piupartss -i slave_run There are several cronjobs installed via '~piupartsm/crontab' and '~piupartss/crontab') to monitor both master and slave as well as the hosts they are running on. It's possible to kill a slave any time by pressing Ctrl-C. Pressing Ctrl-C once will wait for the current test to finish, pressing twice will abort the currently running test (which will be redone). Clean termination may take some time and can be aborted by a third Ctrl-C, but that may leave temporary directories and processes around. See the 'piuparts_slave_run (8)' manpage for more information on 'slave_run'. :ref:`top ` :blue:`Joining an existing slave session` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Run the following script under *your* user account::: holger@pejacevic:~$ sudo -u piupartss -i slave_join See the 'piuparts_slave_join (8)' manpage for more information on 'slave_join'. :ref:`top ` :blue:`Generating reports for the website` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 'piuparts-report' is run daily at midnight and at noon from '~piupartsm/crontab' on pejacevic. :ref:`top ` :blue:`Cronjobs to aid problem spotting` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Some cronjobs to aid problem spotting reside in '~piupartsm/bin/' and are run daily by '~piupartsm/crontab'. - 'detect_network_issues' should detect failed piuparts runs due to network issues on the host. - 'detect_stale_mounts' should detect stale mountpoints (usually of /proc) from failed piuparts runs. More checks should be added as we become aware of them. :ref:`top ` :blue:`Authors` ^^^^^^^^^^^^^^^ Last updated: February 2017 Holger Levsen piuparts-1.1.1/docs/README_server.txt0000644000000000000000000006372713605426771014315 0ustar .. raw:: html .. role:: blue .. _top: README_server ------------- Authors: Lars Wirzenius, Holger Levsen and Andreas Beckmann Email: :blue:`piuparts runs itself and other stuff as root` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ WARNING: Please note that running piuparts on unknown packages is somewhat risky, to say the least. There are security implications that you want to consider. It's best to do it on machines that you don't mind wiping clean at a moment's notice, and preferably so that they don't have direct network access. You have been warned. :blue:`piuparts in master/slave mode` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ As part of the quality assurance efforts of Debian, piuparts is run on the Debian package archive. This requires a lot of processing power, and so the work can be distributed over several hosts. There is one central machine, the master, and any number of slave machines. Each piuparts-slave instance connects to the master, via ssh, and runs the piuparts-master program to report results of packages it has tested already, and to get more work. To set this up for yourself, the following steps should suffice: :blue:`Setting up the master` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ . Pick a machine for running the piuparts master. It cannot be a chroot, but basically any real (or properly virtualized) Debian system is good enough. . Install the package 'piuparts-master' on it. . Create an account for the master, if you install the piuparts-master package it will automatically create a 'piupartsm' user for you. . Configure '/etc/piuparts/piuparts.conf' appropriately. . Create the master and backup directories as defined in that 'piuparts.conf' and make sure master owns them. . To generate the web reports, configure your webserver as needed. If you want to use the supplied 'conf-available/piuparts-master.conf' for apache2, you will need to do two things: a.) enable it and b.) link the htdocs directory defined in 'piuparts.conf' to '/var/lib/piuparts/htdocs' (thats the DocumentRoot as defined in 'conf-available/piuparts-master.conf'). :ref:`top ` :blue:`Setting up the slave(s)` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ . Pick one or more machines for running one or several piuparts slaves. You can use the machine which is running the master also for running a slave. It's also perfectly ok to run several slaves on a multi-core machine which has lots of IO available. . Install the package 'piuparts-slave' on it. . Configure '/etc/piuparts/piuparts.conf' appropriately - if master and slave share the machine, they also share the config file. If you want to run more than one slave on a machine, set the slave-count parameter as desired. By default one slave will be run. . Create the slave and tmp directories as defined in that 'piuparts.conf' and make sure the slave can read and write there. . Create an account for the slave. This must be different from the master account. The piuparts-slave package will create a 'piupartss' user on installation. Whether you run one or many slaves, they run with the same user. . Create an ssh keypair for the slave. No passphrase. If you installed the piuparts-slave package this was done automatically and the public key can be found in '/var/lib/piuparts/piupartss/.ssh/id_rsa.pub' . Copy the slave's public key to the master's '.ssh/authorized_keys', for an installation from packages this will be '/var/lib/piuparts/piupartsm/.ssh/authorized_keys'. The key should be restricted to only allow running 'piuparts-master' by prefixing it with 'command="/usr/share/piuparts/piuparts-master",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ' . Configure sudo to allow the slave account to run several commands as root as root without password. See the example provided in '/usr/share/doc/piuparts-slave/examples/' to learn which. . Run '/usr/bin/piuparts_slave_run' and 'piuparts_slave_join' to actually let the slave(s) run and to join their sessions. . Run '/usr/bin/piuparts_slave_stop' to stop all piuparts-slaves on a host. . The logs go into the master account, into subdirectories. :ref:`top ` :blue:`Tuning the setup` ^^^^^^^^^^^^^^^^^^^^^^^^ The piuparts-server package installs a piuparts server along the lines of https://piuparts.debian.org/. Custome '/etc/piuparts/piuparts.conf' according to your needs, most probably you will want to re-define the 'sections' to be tested (e.g. 'sid') and also maybe use a different Debian mirror. Note that the server can place a significant load on the repository. Consider setting up a local mirror, or a caching proxy for http and apt-get, to reduce the load. Running multiple slaves on a fast host can easily saturate a 100 MBit link. Logs are stored under '/var/lib/piuparts' by default. They are stored there because they are basically the result of piuparts running. There are maintenance cron jobs defined in /usr/share/doc/piuparts-(master|slave)/examples/. In particular, piuparts-report will create static html pages, defaulting to http://localhost/piuparts to be served by any webserver. :ref:`top ` :blue:`Setup from GIT` ^^^^^^^^^^^^^^^^^^^^^^ https://piuparts.debian.org has been set up directly from GIT, this is described in '/usr/share/doc/piuparts-master/README_pejacevic.txt'. :ref:`top ` :blue:`Distributed piuparts testing protocol` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The slave machine and the piuparts-master program communicate using a simplistic line based protocol. SSH takes care of authentication, so there is nothing in the protocol for that. The protocol is transaction based: the slave gives a command, the master program responds. Commands and responses can be simple (a single line) or long (a status line plus additional data lines). Simple commands and responses are of the following format: 'keyword arg1 arg2 arg3 ... argN' The keyword is a command or status code ("ok"), and it and the arguments are separated by spaces. An argument may not contain a space. A long command or response is deduced from the context: certain commands always include additional data, and certain commands always get a long response, if successful (error responses are always simple). The first line of a long command or response is the same as for a simple one, the additional lines are prefixed with a space, and followed by a line containing only a period. A sample session (">>" indicates what the slave sends, "<<" what the master responds with):: << hello >> section sid << ok >> pass liwc 1.2.3-4 >> The piuparts >> log file comes >> here >> . << ok >> reserve << ok vorbisgain 2.3-4 Here the slave first reports a successful test of package liwc, version 1.2.3-4, and sends the piuparts log file for it. Then it reserves a new package to test and the master gives it vorbisgain, version 2.3-4. The communication always starts with the master saying "hello". The slave shall not speak until the master has spoken. Commands and responses in this protocol:: Command: section Success: ok Failure: error Failure: busy Slave asks master to select the given section. This must be the very first command sent by the slave, but may be repeated later on to switch between sections. It will return "error" if the section is unknown and "busy" if it is currently processed by another master instance. If the section command fails, no other commands than "section" will be allowed until one succeeds.:: Command: recycle Success: ok Failure: error Slave asks master to enable logfile recycling mode. In this mode logfiles that have been marked for rechecking will be deleted and reissued in subsequent "reserve" commands. The "recycle" command must be issued before the first "reserve" (or "status") command. It will return "error" if no more logfiles are marked for rechecking or the command is issued too late.:: Command: idle Success: ok Slave asks master whether it remembers having no packages available at a previous "reserve" command. Returns 0 (not known to be idle or timeout expired) or the number of seconds until the master wants to recompute the package state. This command should be given after "recycle" and logfile submission, but before "reserve" or "status" commands. If the slave closes the connection without issuing a "reserve" or "status" command, the expensive Packages file parsing and status computation will be skipped.:: Command: reserve Success: ok Failure: error Slave asks master to reserve a package (a particular version of it) for the slave to test. The slave may reserve any number of packages to test. If the transaction fails, there are no more packages to test, and the slave should disconnect, wait some time and try again.:: Command: unreserve Success: ok Slave informs master it cannot test the desired version of a package and the package should be rescheduled by the master.:: Command: pass log file contents . Success: ok Slave reports that it has tested a particular version of a package and that the package passed all tests. Master records this and stores the log file somewhere suitable.:: Command: fail log file contents . Success: ok Same as "pass", but package failed one or more tests.:: Command: untestable log file contents . Success: ok Slave informs master it cannot test the desired version of a package (perhaps it went away from the mirror?).:: Command: status Success: ok = =... Slave asks master to report the number of packages in all different states. The "status" command should only be issued after all logs have been transmitted ("pass", "fail", and "untestable" commands). In all cases, if the master cannot respond with "ok" (e.g., because of a disk error storing a log file), it aborts and the connection fails. The slave may only assume the command has succeeded if the master responds with "ok". The master may likewise abort, without an error message, if the slave sends garbage, or sends too much data. :ref:`top ` :blue:`piuparts.conf configuration file` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ piuparts-master, piuparts-slave and piuparts-report share the configuration file '/etc/piuparts/piuparts.conf'. The syntax is defined by the Python ConfigParser class, and is, briefly, like this::: [master] foo = bar :ref:`top ` :blue:`Global Configuration` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ These settings have to be placed in the [global] section and are used for all further sections. .. code-block:: text * "sections" defaults to sid and defines which sections should be processed in master-slave mode. Each section defined here has to have a section with the section specific settings explained below. The first section defined should always be sid, because the data from first section a package is in is used for the source package html report. * "basetgz-sections" is an additional list of sections that are only used to maintain the basetgz tarballs and will therefore be ignored by all scripts except piuparts-slave. This list is empty by default. * "master-host" is the host where the master exists. The slave will give this host to ssh. This option is mandatory. * "master-user" is the username of the master. The slave will log in using this username. This option is mandatory. * "master-directory" is the directory where the master keeps its files. Can be relative to the master's home directory. * "slave-directory" is the directory where the slave keeps its files. Can be relative to the slave's home directory. * "slave-count" is the number of concurrent slaves to start. Default: "1". * "slave-flush-interval" is an interval (in seconds) of processing a section which has more queued work after that the slave will connect to the master and flush pending logfiles before resuming the section. Default: "0", i.e. disabled. * "output-directory" is the directory where piuparts-report places the logfiles, generated html files, charts, ... that can be served by a webserver. * "backup-directory" is the directory where the prepare_backup script will place copies of the history data needed to generate the plots. This directory should be included in system backups while the logfiles and html pages in 'master-directory' and 'output-directory' (several GB of data) are regeneratable with some effort and can be excluded from backups. By default this is undefined meaning that no backups of the history data will be made. * "web-host" is the domain name for the reporting web server. Default: "piuparts.debian.org". * "doc-root" is the location where the webserver will serve the piuparts report from. Default: "/". * "slave-load-max" specifies the system load limit when piuparts-slave will enter sleep mode. Operation will be resumed after load drops below 'slave-load-max - 1.0'. Floating point value. Defaults to 0 (= disabled). * "proxy" sets the http_proxy that will be used for fetching Packages files etc. (by master/slave/report) and .debs etc. (by piuparts). This will override a http_proxy setting in the environment. By default (no value being set) the http_proxy variable from the environment will be used (and no proxy will be used if this is not set). It is highly recommended to use a proxy running on localhost (e.g. installing squid and using a setting of "http://localhost:3128") due to the high bandwidth consumption of piuparts and repeated downloading of the same files. :ref:`top ` :blue:`section specific configuration` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The section specific settings will be reloaded each time a section is being run. All these keys can be specified in the [global] section, too, and will serve as defaults for all other sections (overriding the builtin defaults). .. code-block:: text * "master-command" is the command to run on master-host to start the master. Better then setting it here is actually setting it in '~piupartsm/.ssh/authorized_keys' to limit ssh access to that single command. The key should be restricted to only allow running 'piuparts-master' by prefixing it with 'command="/usr/share/piuparts/piuparts-master",no-pty,no-port-forwarding'. * "idle-sleep" is the length of time the slave should wait before querying the master again if the master didn't have any new packages to test. In seconds, so a value of 300 would mean five minutes, and that seems to be a good value for a repo that gets updated frequently. The default is 300 seconds. * "max-tgz-age" is used to specify the maximum age (in seconds) after which basesystem tarballs will be recreated. If recreation fails, the old tarball will be used again. The default is 2592000 seconds, which is 30 days. A value of 0 disables recreation. * "min-tgz-retry-delay" is used to specify the minimum time (in seconds) between attempts to recreate a tarball which was created more than "max-tgz-age" seconds ago. The default is 21600 seconds, which is 6h. * "log-file" is the name of a file to where the master should write its log messages. In the default configuration file it is "$SECTION/master.log". To disable logging, set it to "/dev/null". The global "log-file" setting (defaulting to master-error.log) is used for logging stderr output from piuparts-master. This logfile will be placed in the 'master-directory' and has the PID appended. * "setarch" contains arguments for an optional 'setarch' command that can be prepended to the 'piuparts-command'. Recommended value for running 'i386' tests on an 'amd64' host is 'linux32'. * "piuparts-command" is the command the slave uses to start piuparts. It should include 'sudo' if necessary so that piuparts runs with sufficient priviledges to do its testing (and that means root priviledges). This command should be given in the [global] section and include all flags that are common for all sections. * "piuparts-flags" are appended to "piuparts-command" and should contain the section-specific flags. * "tmpdir" is the scratch area where piuparts will create the chroots. Note: the filesystem where this is located must not be mounted with the nodev or nosuid options. This is a mandatory setting with no default. The scripts that are monitoring this directory for leftover mountpoints and chroots only evaluate the [global] setting. * "description" is a synopsis of the test used in the report. A default description will be generated if this is not set or will be prepended (appended) if the description starts (ends) with '+'. * "mirror" tells the slave which mirror it is to use. The slave gives this to piuparts when it runs it. The URLs for Packages and Sources files will be generated from this setting, too. Default (for fetching Packages/Sources): "http://deb.debian.org/debian". * "distro" is the distribution the slave should tell piuparts to use for basic install/purge testing. It is also possible to use a "partial" distribution as defined in distros.conf. No default. If 'upgrade-test-distros' is set, this selects the distribution that will be used for getting the packages to be tested. Defaults to the last entry in 'upgrade-test-distros', but other useful settings are the first entry (to test upgrades of "disappearing" packages) or the restricted set in a partial distribution (e.g. stable to backports to testing). The special keyword "None" is used to denote that no packages are to be tested, but only the basetgz tarball will be created and refreshed regularily (for the distribution given in 'upgrade-test-distros'). This reference basetgz can be shared between several sections without being affected by their flags. * "area" is the archive area used to get the list of packages to be tested. The Packages file for this area will be loaded. The default is "main" and the possible values depend on the vendor, for Debian these are main, contrib, non-free. * "components" sets the archive areas that will be available when testing the packages selected via the "area" setting. These will be enabled in the generated sources.list. Defaults to "", which means all components will be available. A useful setting is "main" together with area = main to avoid using packages outside main. Testing packages from a 'partial' area like contrib or non-free usually requires additional or all components to be available. * "arch" is the architecture to use. Default: dpkg *-*-print-architecture. * "chroot-tgz" is the name of the file the slave should use for the tarball containing the base chroot. The default name is generated automatically from the "distro" or "upgrade-test-distros" setting. If the tarball doesn't exist, the slave creates it. * "basetgz-directory" is the directory where "chroot-tgz" (or the automatically selected default name) is located. The default is '.'. * "chroot-meta-auto" (global, section) is a file in "chroot-meta-directory" (falling back to the section directory) where the slave will store cached chroot meta data for the reference target chroot in distupgrade tests. This speeds up distupgrade tests since it avoids doing an empty upgrade test to generate this data on-the-fly as part of each test. Cached data will be valid for 6 hours unless a mismatch in the package versions available in the chroot is detected earlier. This is not set (and therefore not enabled) by default. * "chroot-meta-directory" is the directory where "chroot-meta-auto" stores the reference target chroot metadata. * "upgrade-test-distros" is the space delimited list of distributions the slave should use for testing upgrades between distributions (i.e., Debian versions). Using "partial" distributions as defined in distros.conf is possible. Currently, "buster bullseye sid" is a good choice. Setting this switches from doing install/purge tests to dist-upgrade tests. Not set by default. * "max-reserved" is the maximum number of packages the slave will reserve at once. It should be large enough that the host that runs master is not unduly stressed by frequent ssh logins and running master (both of which take quite a bit of CPU cycles), yet at the same time it should not be so large that one slave grabs so many packages all other slaves just sit idle. The number obviously depends on the speed of the slave. A good value seems to be enough to let the slave test packages for about an hour before reporting results and reserving more. For a contemporary AMD64 machine with a reasonably fast disk subsystem the value 50 seems to work fine. To disable a section set this to 0. * "keep-sources-list" controls whether the slave runs piuparts with the '*-*-keep-sources-list' option. This option does not apply to upgrade tests. The value should be "yes" or "no", with the default being "no". Use this option for dists that you need a custom sources.list for, such as "stable-proposed-updates". * "precedence" controls the order the sections are being processed by the slave. Sections with a larger precedence value will be run only if all sections with a smaller precedence value are idle, i.e. master does not have any packages that this slave could test. Sections with the same precedence value will be processed round-robin until they are all idle (or a more important section has packages to be tested). The default is 1. * "depends-sections" lists additional sections that will be searched for dependencies that are not available in the current section if that describes a partial distro. * "known-problem-directory" is the path to the directory containing definitions of known problems. Default: "${prefix}/share/piuparts/known_problems" * "debug" tells the slave whether to log debug level messages. The value should be "yes" or "no", with the default being "no". piuparts itself currently always produces debug output and there is no way to disable that. * "PYTHONPATH" (global) sets the search path to the piupartslib python modules if they are not installed in their default location in /usr. * "reschedule-untestable-days" (global) sets the rescheduling delay for untestable packages (e.g. due to unsatisfied dependencies). This is handled by the 'report_untestable_packages' script and the default is "7" days. * "reschedule-old-days" (global, section) and the following five settings define the rescheduling scheme that it performed by the 'reschedule_oldest_logs' script. Passed/failed logs that are older than reschedule-(old|fail)-days will be marked for rechecking (limited to reschedule-(old|fail)-count). Only packages that are actually testable will be reissued by piuparts-master (and the "old" log will be deleted at that time). Logs that are marked for recycling but have not been rechecked due to missing/failing dependencies will be deleted anyway if they are older than expire-(old|fail)-days. * "reschedule-old-count" (global, section) is the maximum number of passed logs that will be marked for recycling. Set to 0 to disable rescheduling passed logs. * "expire-old-days" (global, section) can be set to a value larger than 'reschedule-old-days' to delete logs older than the setting that are marked for recycling but haven't been rechecked due to failing or missing dependencies. A value starting with "+" is relative to 'reschedule-old-days'. Disabled by default ("0"). * "reschedule-fail-days" (global, section) sets the minimum age of failing logs (fail/*.log or affected/*.log) before they will be rechecked. * "reschedule-fail-count" (global, section) is the maximum number of failed logs that will be marked for recycling. Set to 0 to disable rescheduling failed logs. * "expire-fail-days" (global, section) can be set to a value larger than 'reschedule-fail-days' to delete logs older than the setting that are marked for recycling but haven't been rechecked due to failing or missing dependencies. A value starting with "+" is relative to 'reschedule-fail-days'. Disabled by default ("0"). * "auto-reschedule" (section) can be set to "no" to disable rescheduling of passed and failed packages. To disable only rescheduling one of passed or failed logs, set the corresponding -count variable to zero. * "json-sections" is a space-separated list of the section/distribution names which receive test results for this section. The results, by package, are stored with this name/these names in the section and global test summary.json files. If "json-sections" is undefined, or defined as "default", piuparts will assign the section to one of "unstable", "testing", "stable", "oldstable", "experimental", or "unknown". If "json-sections" is "none", the summary will not be created. The "json-sections" name "overall" is reserved. * "exclude-known-problems" (global, section) can be set to a list of known_problems conffile names (including the .conf suffix) that should not be used for this section. Useful if a known problem triggers on (nearly) all logfiles. Some of the configuration items are not required, but it is best to set them all to be sure what the configuration actually is. :ref:`top ` :blue:`piuparts.debian.org specific configuration` In addition to some of the above settings the following configuration settings are used by the scripts in '~piuparts?/bin/' used to run piuparts.debian.org. They are all optional, default values are set in the scripts. .. code-block:: text * "urlbase" (global) is the base url of the webserver serving this piuparts instance. Used to provide links to logfiles in email reports. It defaults to "https://piuparts.debian.org". :ref:`top ` :blue:`Running piuparts-report as it is done for piuparts.debian.org` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you want to run piuparts-report (which is only+very useful if you run piuparts in master-slave mode), you need to 'apt-get install python-rpy r-recommended r-base-dev'. For more information see "https://salsa.debian.org/debian/piuparts/blob/develop/README_pejacevic.txt". To generate the report on the master host run::: piupartsm@pejacevic:~$ /usr/share/piuparts/master/generate_daily_report piuparts-1.1.1/docs/conf.py0000644000000000000000000001177313605426771012522 0ustar # -*- coding: utf-8 -*- # # Configuration file for the Sphinx documentation builder. # # This file does only contain a selection of the most common options. For a # full list see the documentation: # http://www.sphinx-doc.org/en/master/config # -- Path setup -------------------------------------------------------------- # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- Project information ----------------------------------------------------- project = 'piuparts' copyright = '2019, Lars Wirzenius' author = 'Lars Wirzenius' # The short X.Y version version = '' # The full version, including alpha/beta/rc tags release = '0.98' # -- General configuration --------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ ] # Add any paths that contain templates here, relative to this directory. templates_path = ['ntemplates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] source_suffix = '.txt' # The master toctree document. master_doc = 'index' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. exclude_patterns = [] # The name of the Pygments (syntax highlighting) style to use. pygments_style = None # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'alabaster' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". #html_static_path = ['nstatic'] # Custom sidebar templates, must be a dictionary that maps document names # to template names. # # The default sidebars (for documents that don't match any pattern) are # defined by theme itself. Builtin themes are using these templates by # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', # 'searchbox.html']``. # # html_sidebars = {} # -- Options for HTMLHelp output --------------------------------------------- # Output file base name for HTML help builder. htmlhelp_basename = 'piupartsdoc' # -- Options for LaTeX output ------------------------------------------------ latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # # 'preamble': '', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'piuparts.tex', 'piuparts Documentation', 'Lars Wirzenius', 'manual'), ] # -- Options for manual page output ------------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'piuparts', 'piuparts Documentation', [author], 1) ] # -- Options for Texinfo output ---------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'piuparts', 'piuparts Documentation', author, 'piuparts', 'One line description of project.', 'Miscellaneous'), ] # -- Options for Epub output ------------------------------------------------- # Bibliographic Dublin Core info. epub_title = project # The unique identifier of the text. This can be a ISBN number # or the project homepage. # # epub_identifier = '' # A unique identification for the text. # # epub_uid = '' # A list of files that should not be packed into the epub file. epub_exclude_files = ['search.html'] piuparts-1.1.1/docs/index.txt0000644000000000000000000000061613605426771013065 0ustar Welcome to piuparts's documentation! ==================================== .. toctree:: :maxdepth: 4 README README_server README_pejacevic piuparts piuparts slave join piuparts slave run piuparts slave stop license Search ====== * :ref:`search` piuparts-1.1.1/docs/license.txt0000644000000000000000000000144313605426771013377 0ustar License GPL-2+ ============== This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. . This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. . You should have received a copy of the GNU General Public License along with this program. If not, see . On Debian systems, the full text of the GNU General Public License version 2 can be found in the file '/usr/share/common-licenses/GPL-2'. piuparts-1.1.1/docs/piuparts/0000755000000000000000000000000013605426771013061 5ustar piuparts-1.1.1/docs/piuparts/conf.py0000644000000000000000000000333013605426771014357 0ustar # -*- coding: utf-8 -*- # # Configuration file for the Sphinx documentation builder. # # This file does only contain a selection of the most common options. For a # full list see the documentation: # http://www.sphinx-doc.org/en/master/config # -- Path setup -------------------------------------------------------------- # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- Project information ----------------------------------------------------- project = 'Piuparts' copyright = '' author = '' # The short X.Y version version = '' # The full version, including alpha/beta/rc tags release = '0.98' # source_suffix = ['.rst', '.md'] source_suffix = '.txt' # The master toctree document. master_doc = 'index' # The name of the Pygments (syntax highlighting) style to use. pygments_style = None # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'alabaster' # -- Options for HTMLHelp output --------------------------------------------- # Output file base name for HTML help builder. htmlhelp_basename = 'piupartsdoc' # -- Options for manual page output ------------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'piuparts', '.deb installation, upgrade, and removal testing suite\n', '', 1) ] piuparts-1.1.1/docs/piuparts/index.txt0000644000000000000000000000010213605426771014722 0ustar Manpage ======= .. toctree:: :maxdepth: 1 piuparts.1 piuparts-1.1.1/docs/piuparts/piuparts.1.txt0000644000000000000000000004734313605426771015643 0ustar .. only:: html piuparts(1) =========== :doctype: manpage :revdate: 2017-03-06 SYNOPSIS -------- *piuparts* ['-apvV'] ['-d' _distro_] ['-i' _filename_] ['-I' _regexp_] ['-l' _logfile_] ['-m' _url_] ['--bindmount' _dir_] [_package_]... [_changes_file_]... DESCRIPTION ----------- *piuparts* tests that Debian packages handle installation, upgrading, and removal correctly. It does this by creating a minimal Debian installation in a chroot, and installing, upgrading, and removing packages in that environment, and comparing the state of the directory tree before and after. *piuparts* reports any files that have been added, removed, or modified during this process. *piuparts* is meant as a quality assurance tool for people who create Debian packages to test them before they upload them to the Debian package archive. By default, piuparts can do three different tests: . A simple install-purge test within one Debian distribution (chosen with the '-d' option, unstable by default). It sets up the chroot with the desired distribution, then installs and purges the packages, and reports problems. . A simple install-upgrade-purge test within one Debian distribution. This test is like the install-purge test, but it installs the packages first via *apt-get* and then from the package files given on the command line. If the command line has package names (option '--apt' used), or no tested package is known to *apt-get* (new packages), this test is skipped, otherwise it is performed automatically. . An upgrade test between Debian releases. This test is enabled by using the '-d' option multiple times and disables the other two tests. It sets up the chroot with the first distribution named, then upgrades it to each successive one, and then remembers the directory tree state at the end. After this, it starts over with the chroot of the first distribution, installs the desired packages (via *apt-get*), and does the successive upgrading (via *apt-get dist-upgrade*). Then, if package files (and not just package names) were given on the command line, it installs them. Finally, it reports problems against the state of the directory tree at the last distribution compared with the state without the packages having been installed. This test can be quite slow to execute. + Note that this does not work with experimental, because *apt-get* does not automatically upgrade to packages in experimental. To test a particular package or group of packages in experimental, use the second test. Command line arguments are the paths to package files (e.g., *piuparts_1.0-1_all.deb*), paths to changes files (e.g., *piuparts_1.0-1_i386.changes*), or names of packages, if the '--apt' option is given. When processing changes files, by default, all packages in a changes file will be processed together with all individual packages given on the command line. Then each package given on the command line is processed in a single group. If the '--single-changes-list' is used, the packages in all changes files are processed together along with any individual packages that were given on the command line. To avoid this behaviour, it is possible to specify '--single-packages'. *piuparts* outputs to the standard output some log messages to show what is going on. If a log file is used, the messages go there as well. *piuparts* needs to be run as root. OPTIONS ------- Options must come before the other command line arguments. *-a*, *--apt*:: The package arguments on the command line are to be treated as package names and installed via *apt-get* instead of being names of package files, to be installed via *dpkg -i*. *-*-allow-database*:: Allow starting MySQL and PostgreSQL database servers in the chroot for packages requiring database access in their maintainer scripts. Do not use this option if there is already a database server running on the system running piuparts (or piuparts-slave)! In master-slave setups with multiple slaves running on one host collisions may occur, these will be detected by 'detect_piuparts_issues' and the affected packages will be tested again. *-*-arch*='arch':: Create chroot and run tests for (non-default) architecture 'arch'. The default is the output from 'dpkg --print-architecture'. *-b* 'tarball', *-*-basetgz*='tarball':: Use tarball as the contents of the initial chroot, instead of building a new one with debootstrap. The tarball can be created with the '-s' option, or you can use one that *pbuilder* has created (see '-p'). If you create one manually, make sure the root of the chroot is the root of the tarball. *-*-bindmount*='dir':: Bind-mount a directory inside the chroot. *-d* 'name', *-*-distribution*='name':: Which Debian distribution to use: a code name (for example buster, bullseye or sid) or experimental. The default is sid (=unstable). *-D* 'flavor', *-*-defaults*='flavor':: Use default settings suitable for a particular flavor of Debian: either debian or ubuntu. The default is debian. *-*-do-not-verify-signatures*:: Do not verify signatures from the Release files when running debootstrap. Also set APT::Get::AllowUnauthenticated accordingly in /etc/apt/apt.conf in the chroots. *-*-dpkg-force-confdef*:: Make dpkg use --force-confdef, which lets dpkg always choose the default action when a modified conffile is found. This option will make piuparts ignore errors it was designed to report and therefore should only be used to hide problems in depending packages. 'This option shall normally not be used.' (See #466118.) *-*-dpkg-noforce-unsafe-io*:: Prevent running dpkg with --force-unsafe-io. --force-unsafe-io causes dpkg to skip certain file system syncs known to cause substantial performance degradation on some filesystems. Thus, including this option reverts to safe but slower behavior. The --dpkg-noforce-unsafe-io is required for running tests on distributions older than squeeze. *-*-no-eatmydata*:: Prevent use of eatmydata. The --no-eatmydata option is required for running tests on squeeze or older distributions. *-*-extra-old-packages*='pkg1[,pkg2]...':: Install additional old packages before upgrading. Allows testing package renames/merges where the old package is no longer available in the new distribution and the new one utilizes Conflicts/Replaces. The argument is a comma separated list of package names and the option can be given multiple times. For install/purge tests these packages will be installed before the package that is to be tested. *-e* 'dirname', *-*-existing-chroot*='dirname':: Use the specified directory as source for the new chroot, instead of building a new one with debootstrap. This is similar to '--basetgz', but the contents are not archived. See also the --hard-link option. *-*-distupgrade-to-testdebs*:: Use the "testdebs" repository to override the packages in the distupgrade target distribution. This allows one to test complex upgrade paths before the packages enter the archive. *-*-extra-repo*='deb-line':: Provide an additional (unparsed) line to be appended to sources.list, e.g.:: 'deb ' or 'deb file:// ./' Useful for e.g. backports, security or local repositories that cannot be handled by '--mirror'. May be repeated to add more than one line. *-*-fake-essential-packages*='pkg1[,pkg2]...':: Install additional packages in the base chroot that are not removed after the test. These are available during purge and for checking against mistreatment. Takes a comma separated list of package names and can be given multiple times. *-*-hard-link*:: When the --existing-chroot option is used, and the source directory is on the same filesystem, hard-link files instead of copying them. This is faster, but any modifications to files will be reflected in the originals. *-i* 'filename', *-*-ignore*='filename':: Add a filename to the list of filenames to be ignored when comparing changes before and after installation. By default, piuparts ignores files that always change during a package installation and uninstallation, such as *dpkg* status files. The filename should be relative to the root of the chroot (e.g., *var/lib/dpkg/status*). Filenames prefixed with a ':' will be logged verbosely if found. This option can be used as many times as necessary. *-I* 'regexp', *-*-ignore-regexp*='regexp':: Add a regular expression pattern to the list of patterns for filenames to be ignored when comparing changes before and after installation. Patterns prefixed with a ':' will log verbosely all matching files. This option can be used as many times as necessary. *-*-install-purge-install*:: Purge package after installation and reinstall. All dependencies are available during purge. *-*-install-recommends*:: Enable installation of Recommends. *-*-install-suggests*:: Enable installation of Suggests. *-*-install-remove-install*:: Remove package after installation and reinstall. For testing installation in config-files-remaining state. *-k*, *-*-keep-env*:: Depending on which option is passed, keep the environment used for testing after the program ends:: * By default it doesn't remove the temporary directory for the chroot, * or if --schroot is used, the schroot session is not terminated, * or if --docker-image is used, the container created is not destroyed. *-K*, *-*-keyring*='filename':: Use FILE as the keyring to use with debootstrap when creating chroots. *-*-keep-sources-list*:: Don't modify the chroot's etc/apt/sources.list. *-*-list-installed-files*:: List the files added to the chroot after the installation of the package and after the installation of the package dependencies. *-*-lvm-volume*='lvm-volume':: Use the specified lvm-volume as source for the chroot, instead of building a new one with debootstrap. This creates a snapshot of the given LVM volume and mounts it to the chroot path. *-*-lvm-snapshot-size*='snapshot-size':: Use the specified snapshot-size as snapshot size when creating a new LVM snapshot (default: 1G) *-l* 'filename', *-*-log-file*='filename':: Append log file to _filename_ in addition to the standard output. *-*-log-level*='level':: Display messages from loglevel LEVEL, possible values are: error, info, dump, debug. The default is dump. *-*-max-command-output-size*='size':: Set the maximum permitted command output to _size_ (in MB) for debugging runs exceeding the default of 8 MB. *-*-merged-usr*:: When using debootstrap to create the chroot, use the '--merged-usr' option to create a chroot with /bin, /lib, /sbin being symlinks to their /usr counterparts. *-m* 'url', *-*-mirror*='url':: Which Debian mirror to use. The default is the first mirror named in */etc/apt/sources.list* or *http://deb.debian.org/debian* if none is found. This option may be used multiple times to use multiple mirrors. Only the first mirror is used with *debootstrap*. The 'components' that are used for a mirror can also be set with this option: a space separated list within the same argument (so you need to quote the entire argument in the shell). If no components are given explicitly, the usual Debian components are used (main, contrib, and non-free). For the mirrors read from */etc/apt/sources.list*, the components are read from the same place. Note that file: addresses works if the directories are made accessible from within the chroot with '--bindmount'. *-*-no-adequate*:: Don't run adequate after installation. The default is to run adequate, provided it is installed. *-*-no-check-valid-until*:: Set apt option Acquire::Check-Valid-Until=false in the chroot to ignore the expiration of Release files. This is needed for testing archived releases. *-*-no-diversions*:: Don't check for broken diversions. *-n*, *-*-no-ignores*:: Forget all built-in and other ignores that have been set so far. Any '-i' or '-I' arguments that come after this one will be obeyed, but none of the ones that come before. *-N*, *-*-no-symlinks*:: Don't check for broken symlinks. *-*-fail-if-inadequate*:: Fail on inadequate results from running adequate. The default is to just issue those errors as warnings. *-*-fail-on-broken-symlinks*:: Fail on broken symlinks. The default is to just issue those errors as warnings. *-*-no-upgrade-test*:: Skip testing upgrade from an existing version in the archive. *-*-no-install-purge-test*:: Skip the install and purge test. *-p, *-*-pbuilder*:: Use */var/cache/pbuilder/base.tgz* as the base tarball. This is a shorthand so that you don't need to use '-b' for it. *-*-pedantic-purge-test*:: Be pedantic when checking if a purged package leaves files behind. If this option is not set, files left in */tmp* are ignored.") *-*-proxy*='URL':: Use the proxy at URL to access the Debian mirror(s). Takes precedence over the 'http_proxy' environment variable. Using a local proxy is recommended because piuparts may use large amounts of bandwidth to repeatedly download the same files. *-s* 'filename', *-*-save*='filename':: Save the chroot, after it has been set up, as a tarball into *filename*. It can then be used with '-b'. *-B* 'FILE', *-*-end-meta*='FILE':: Load chroot package selection and file meta data from FILE. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests. *-S* 'FILE', *-*-save-end-meta*='FILE':: Save chroot package selection and file meta data in FILE for later use. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests. *-*-scriptsdir*='DIR':: Directory where are custom scripts are placed. By default, this is not set. For more information about this, read README_server.txt *-*-schroot*='SCHROOT-NAME':: Use schroot session named SCHROOT-NAME for the testing environment, instead of building a new one with debootstrap. *-*-docker-image*='DOCKER-IMAGE':: Use a container created from the docker image DOCKER-IMAGE for the testing environment, instead of building a new one with debootstrap. This only supports overlay2 for now and it uses the `MergedDir` layer where piuparts can access, add, edit and remove files easily. *-*-single-changes-list*:: When processing changes files, piuparts will process the packages in each individual changes file separately. This option will set piuparts to scan the packages of all changes files together along with any individual package files that may have been given on the command line. *-*-single-packages*:: Process every package file or package name individually, thus piuparts process runs multiple times. This option can be useful with conflicting packages. *-*-shell-on-error*:: Start an interactive shell in the chroot after an error occurred. This should help debugging failures directly inside the piuparts test environment. The chroot cleanup will continue after the shell terminates. Note: This does not work if the piuparts command is prefixed with 'timeout', which is usually the case in command lines directly copied from logfiles from a master-slave setup. Removing the 'timeout' part is sufficient. *-*-skip-minimize*:: Allow skip minimize chroot step. This is useful when you want to test several packages with piuparts. You can prepare a tarball already minimized and skip this step in all the tests. This is the default now. *-*-minimize*:: Minimize the chroot with debfoster. This used to be the default until #539142 was fixed. *-*-skip-cronfiles-test*:: Skip testing the output from the cron files left in the system after remove a package. *-*-skip-logrotatefiles-test*:: Skip testing the output from the logrotate files left in the system after remove a package. *-*-testdebs-repo*='deb-line':: Provide an additional line to be appended to sources.list, e.g.:: 'deb [ trusted=yes ] ' or 'deb [ trusted=yes ] file:// ./' If only an URL or local path is given as argument, "deb [ trusted=yes ]", "file://", and "./" will be prepended/appended as needed. The "testdebs" repository provides the packages to be tested (and some additional dependencies, if needed, e.g. all packages built from the same source package as the (binary) package being tested) and can be used for testing complex installation and upgrade scenarios involving dependencies that are not yet in the archive. This repository will be available only for installing the target packages. Dependency resolution will be done by apt-get. The packages to be tested can be passed as .debs or as package names (with '--apt'). The "trusted=yes" option causes this (and only this) repository to be trustworthy even if the Packages file is not signed, such that a (globally acting) '--do-not-verify-signatures' will not be needed. *-t directory*, *-*-tmpdir*='directory':: Use directory as the place where temporary files and directories are created. The default is the environment variable *TMPDIR*, or */tmp* if not set. Note: the temporary directory must *not* be mounted with the _nodev_ or _nosuid_ mount option. *-*-upgrade-before-dist-upgrade*:: Perform two-stage upgrades: 'apt-get upgrade && apt-get dist-upgrade'. *-v*, *-*-verbose*:: This option no longer has any meaning, but it is still accepted for backwards compatibility. *-V*, *-*-version*:: Write out the version number of the program. *-*-warn-on-debsums-errors*:: Print a warning rather than failing if debsums reports modified files. *-*-warn-on-leftovers-after-purge*:: Print a warning rather than failing if files are left behind after purge. *-*-warn-on-others*:: Print a warning rather than failing if files are left behind, modified, or removed by a package that was not given on the command-line. This way, you can basically isolate the purge test to your own packages. If a package that is brought in as a dependency doesn't purge cleanly, the test will not fail because of it (but a warning message will be printed). Behavior with multiple packages given on the command-line could be problematic, particularly if the dependency tree of one package in the list includes another in the list. Therefore, it is recommended to use this option with one package at a time. EXAMPLES -------- Assume that you have just built a new version of your Debian package, to be uploaded to Debian unstable. It is in *../foo_1.0-2_i386.deb* and you would like to know whether it installs and uninstalls properly. Here's what you would do:: ---- piuparts ../foo_1.0-2_i386.deb ---- If the package exists in the Debian archive already, the above command also tests that it upgrades properly. To do the same test, but using a particular mirror, and only the main component, you would do this:: ---- piuparts -m 'http://gytha/debian main' ../foo_1.0-2_i386.deb ---- If you want to do the same as above but for your changes files, pass in your changes files when running piuparts, and piuparts will process each package in the changes files as though you had passed all those packages on the command line to piuparts yourself. For example:: ---- piuparts ../foo_1.0-2_i386.changes piuparts -m 'http://gytha/debian main' ../foo_1.0-2_i386.changes ---- If you want to test that a package installs properly in the stable (currently buster) Debian release, then can be upgraded to the testing (currently bullseye) and unstable (sid) versions, and then uninstalled without problems, you would give the following command:: ---- piuparts -a -d buster -d bullseye -d sid foo ---- ENVIRONMENT ----------- *TMPDIR* Location for temporary files and directories. If not set, use */tmp*. See also the '-t' ('--tmpdir') option. NOTES ----- Output of commands run by piuparts is limited to three megabytes. To change this limit, the source code needs to be edited. Commands exceeding this limit will be aborted. SEE ALSO -------- *pbuilder* (1), *debootstrap* (8) AUTHOR ------ Lars Wirzenius (liw@iki.fi) and others piuparts-1.1.1/docs/piuparts_slave_join/0000755000000000000000000000000013605426771015272 5ustar piuparts-1.1.1/docs/piuparts_slave_join/conf.py0000644000000000000000000000320613605426771016572 0ustar # -*- coding: utf-8 -*- # # Configuration file for the Sphinx documentation builder. # # This file does only contain a selection of the most common options. For a # full list see the documentation: # http://www.sphinx-doc.org/en/master/config # -- Path setup -------------------------------------------------------------- # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- Project information ----------------------------------------------------- project = 'Piuparts' copyright = '' author = '' # The short X.Y version version = '' # The full version, including alpha/beta/rc tags release = '0.98' # source_suffix = ['.rst', '.md'] source_suffix = '.txt' # The master toctree document. master_doc = 'index' # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'alabaster' # -- Options for HTMLHelp output --------------------------------------------- # Output file base name for HTML help builder. htmlhelp_basename = 'piupartsdoc' # -- Options for manual page output ------------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'piuparts_slave_join', 'joins a running piuparts-slave session in screen\n', '', 8) ] piuparts-1.1.1/docs/piuparts_slave_join/index.txt0000644000000000000000000000014313605426771017140 0ustar Manpage ======= .. toctree:: :maxdepth: 1 piuparts slave join piuparts-1.1.1/docs/piuparts_slave_join/piuparts_slave_join.8.txt0000644000000000000000000000167413605426771022271 0ustar .. only:: html piuparts_slave_join(8) ====================== :doctype: manpage :revdate: 2016-03-06 SYNOPSIS -------- *piuparts_slave_join* DESCRIPTION ----------- *piuparts_slave_join* joins a running piuparts-slave session in screen. OPTIONS ------- There are no options to this command. ENVIRONMENT ----------- Running piuparts in master-slave mode requires configuration in */etc/piuparts*. NOTES ----- Make sure to also read */usr/share/doc/piuparts-master/README_server.txt*. See *screen* (1) for full instructions on how to navigate between displays. In short:: *Ctrl-A *:: Navigates to display . Slaves are found at screens 1 through 'slave-count', defined in the piuparts configuration file. *Ctrl-A d*:: Detach from *screen*. The session can be re-established with *piuparts-slave-join*. SEE ALSO -------- *piuparts* (1), *piuparts_slave_run* (8), *screen* (1) AUTHOR ------ Holger Levsen (holger@layer-acht.org) piuparts-1.1.1/docs/piuparts_slave_run/0000755000000000000000000000000013605426771015137 5ustar piuparts-1.1.1/docs/piuparts_slave_run/conf.py0000644000000000000000000000320313605426771016434 0ustar # -*- coding: utf-8 -*- # # Configuration file for the Sphinx documentation builder. # # This file does only contain a selection of the most common options. For a # full list see the documentation: # http://www.sphinx-doc.org/en/master/config # -- Path setup -------------------------------------------------------------- # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # # import os # import sys # sys.path.insert(0, os.path.abspath('.')) # -- Project information ----------------------------------------------------- project = 'Piuparts' copyright = '' author = '' # The short X.Y version version = '' # The full version, including alpha/beta/rc tags release = '0.98' # source_suffix = ['.rst', '.md'] source_suffix = '.txt' # The master toctree document. master_doc = 'index' # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'alabaster' # -- Options for HTMLHelp output --------------------------------------------- # Output file base name for HTML help builder. htmlhelp_basename = 'piupartsdoc' # -- Options for manual page output ------------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'piuparts_slave_run', 'start up piuparts-slave instance(s) in screen\n', '', 8) ] piuparts-1.1.1/docs/piuparts_slave_run/index.txt0000644000000000000000000000014113605426771017003 0ustar Manpage ======= .. toctree:: :maxdepth: 1 piuparts slave run piuparts-1.1.1/docs/piuparts_slave_run/piuparts_slave_run.8.txt0000644000000000000000000000304413605426771021774 0ustar .. only:: html piuparts_slave_run(8) ===================== :doctype: manpage :revdate: 2017-03-06 SYNOPSIS -------- *piuparts_slave_run* ['SLAVE-NUMBERS'] DESCRIPTION ----------- *piuparts_slave_run* runs piuparts-slave instance(s) in *screen* which run *piuparts* tests which test whether Debian packages handle installation, upgrading, and removal correctly. By default, *piuparts_slave_run* will start the number of slaves specified by the 'slave-count' parameter in the piuparts.conf file. If 'slave-count' is not defined, a single slave will be started. If a single argument is supplied to *piuparts_slave_run*, consisting of a space-separated list of numbers between 1 and 'slave-count', only those slaves will be started. OPTIONS ------- There are no options to this command currently. Note however that *this interface is under development* and may change. ENVIRONMENT ----------- Running piuparts in master-slave mode requires configuration in */etc/piuparts*. BUGS ---- *piuparts_slave_run* should be able to run slaves in different ways: 1. all, in a new screen (only this is implemented at the moment) 2. a specific one, in a specified screen slot in a (running) screen 3. append one to a (running) screen using a "random" slot 4. start slave in the current directory in the current shell in foreground. NOTES ----- Make sure to also read */usr/share/doc/piuparts-master/README_server.txt*. SEE ALSO -------- *piuparts* (1), *piuparts_slave_join* (8), *piuparts_slave_stop* (8), *screen* (1) AUTHOR ------ Holger Levsen (holger@layer-acht.org) piuparts-1.1.1/docs/piuparts_slave_stop/0000755000000000000000000000000013605426771015320 5ustar piuparts-1.1.1/docs/piuparts_slave_stop/conf.py0000644000000000000000000000237013605426771016621 0ustar # -*- coding: utf-8 -*- # # Configuration file for the Sphinx documentation builder. # # This file does only contain a selection of the most common options. For a # full list see the documentation: # http://www.sphinx-doc.org/en/master/config # -- Project information ----------------------------------------------------- project = 'Piuparts' copyright = '' author = '' # The short X.Y version version = '' # The full version, including alpha/beta/rc tags release = '0.98' # source_suffix = ['.rst', '.md'] source_suffix = '.txt' # The master toctree document. master_doc = 'index' # -- Options for HTML output ------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # html_theme = 'alabaster' # -- Options for HTMLHelp output --------------------------------------------- # Output file base name for HTML help builder. htmlhelp_basename = 'piupartsdoc' # -- Options for manual page output ------------------------------------------ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'piuparts_slave_stop', 'stop all running piuparts-slaves on a host\n', '', 8) ] piuparts-1.1.1/docs/piuparts_slave_stop/index.txt0000644000000000000000000000014313605426771017166 0ustar Manpage ======= .. toctree:: :maxdepth: 1 piuparts slave stop piuparts-1.1.1/docs/piuparts_slave_stop/piuparts_slave_stop.8.txt0000644000000000000000000000150713605426771022340 0ustar .. only:: html piuparts_slave_stop(8) ====================== :doctype: manpage :revdate: 2017-03-06 SYNOPSIS -------- *piuparts_slave_stop* DESCRIPTION ----------- *piuparts_slave_stop* gracefully stops all running piuparts-slaves sessions on a host. Gracefully means that it asks the slaves to stop running and will patiently wait until all the piuparts-slaves have done so. If more immediate action is required, use *piuparts_slave_join* and manual force instead. OPTIONS ------- There are no options to this command. ENVIRONMENT ----------- Running piuparts in master-slave mode requires configuration in */etc/piuparts*. NOTES ----- Make sure to also read */usr/share/doc/piuparts-master/README_server.txt*. SEE ALSO -------- *piuparts* (1), *piuparts_slave_join* (8) AUTHOR ------ Holger Levsen (holger@layer-acht.org) piuparts-1.1.1/helpers/0000755000000000000000000000000013266771660011727 5ustar piuparts-1.1.1/helpers/debiman-piuparts-distill/0000755000000000000000000000000013601665362016627 5ustar piuparts-1.1.1/helpers/debiman-piuparts-distill/atomically.go0000644000000000000000000000262313266771660021325 0ustar package main import ( "bufio" "compress/gzip" "io" "io/ioutil" "os" "path/filepath" ) func tempDir(dest string) string { tempdir := os.Getenv("TMPDIR") if tempdir == "" { // Convenient for development: decreases the chance that we // cannot move files due to /tmp being on a different file // system. tempdir = filepath.Dir(dest) } return tempdir } func writeAtomically(dest string, compress bool, write func(w io.Writer) error) (err error) { f, err := ioutil.TempFile(tempDir(dest), "debiman-") if err != nil { return err } defer func() { // Remove the tempfile if an error occurred if err != nil { os.Remove(f.Name()) } }() defer f.Close() bufw := bufio.NewWriter(f) w := io.Writer(bufw) var gzipw *gzip.Writer if compress { // NOTE(stapelberg): gzip’s decompression phase takes the same // time, regardless of compression level. Hence, we invest the // maximum CPU time once to achieve the best compression. gzipw, err = gzip.NewWriterLevel(bufw, gzip.BestCompression) if err != nil { return err } defer gzipw.Close() w = gzipw } if err := write(w); err != nil { return err } if compress { if err := gzipw.Close(); err != nil { return err } } if err := bufw.Flush(); err != nil { return err } if err := f.Chmod(0644); err != nil { return err } if err := f.Close(); err != nil { return err } return os.Rename(f.Name(), dest) } piuparts-1.1.1/helpers/debiman-piuparts-distill/piuparts.go0000644000000000000000000000750413334261717021032 0ustar // debiman-piuparts-distill extracts slave alternative links from // LOG-ALTERNATIVES lines found in piuparts logs. // // See https://github.com/Debian/debiman/issues/12 for more details. package main import ( "bufio" "encoding/json" "flag" "io" "log" "os" "path/filepath" "regexp" "sort" "strings" "sync" ) var ( logsDir = flag.String("logs_dir", "", "Directory containing piuparts logfiles") output = flag.String("output", "", "Path to write the (gzip-compressed, json-encoded) distilled links file to") parallel = flag.Int("parallel", 10, "Number of logfiles to read in parallel") ) var ( logAlternativesRe = regexp.MustCompile(`LOG-ALTERNATIVES: dpkg=([^:]+): piuparts=(?:[^:]+): (.*)`) slaveParamsRe = regexp.MustCompile(`--(?:install|slave) ([^ ]+) (?:[^ ]+) ([^ ]+)`) ) type link struct { Pkg string `json:"binpackage"` From string `json:"from"` To string `json:"to"` } // process reads the piuparts logfile at path. links are extracted from each // LOG-ALTERNATIVES line and written to the links channel. func process(path string, links chan<- link) error { f, err := os.Open(path) if err != nil { return err } defer f.Close() scanner := bufio.NewScanner(f) // Increase the maximum token size to 5 MB to handle logs such as // apprecommender_0.7.5-2.log, which has a very long line of interactive // xapian progress output. scanner.Buffer(nil, 5*1024*1024) for scanner.Scan() { line := strings.TrimSpace(scanner.Text()) if !strings.HasPrefix(line, "LOG-ALTERNATIVES: ") { continue } matches := logAlternativesRe.FindStringSubmatch(line) if matches == nil { continue } for _, param := range slaveParamsRe.FindAllStringSubmatch(line, -1) { links <- link{ Pkg: matches[1], From: param[1], To: param[2], } } } return scanner.Err() } // byPkg is a helper type for sorting the results slice by binary package. Once // Go 1.8 becomes available on piuparts.debian.org, we can switch to sort.Slice. type byPkg []link func (p byPkg) Len() int { return len(p) } func (p byPkg) Swap(i, j int) { p[i], p[j] = p[j], p[i] } func (p byPkg) Less(i, j int) bool { return p[i].Pkg < p[j].Pkg } func main() { flag.Parse() if *output == "" { log.Fatal("-output must be specified") } if *logsDir == "" { log.Fatal("-logs_dir must be specified") } // Spawn -parallel worker goroutines, waiting for work work := make(chan string) linksChan := make(chan link) var wg sync.WaitGroup for i := 0; i < *parallel; i++ { wg.Add(1) go func() { defer wg.Done() for path := range work { if err := process(path, linksChan); err != nil { log.Printf("error processing %q: %v", path, err) } } }() } // Collect results from all workers into linksMap linksMap := make(map[link]bool) // Channel for signaling that all results were collected collected := make(chan bool) go func() { for l := range linksChan { linksMap[l] = true } collected <- true }() // Walk through *logsDir, enqueue all .log files onto the work channel if err := filepath.Walk(*logsDir, func(path string, info os.FileInfo, err error) error { if err != nil { return err } if strings.HasSuffix(path, ".log") && info != nil && info.Mode().IsRegular() { work <- path } return nil }); err != nil { log.Fatal(err) } // Close the channel, signaling termination to the worker goroutines close(work) // Wait for the worker goroutines to terminate wg.Wait() close(linksChan) <-collected // Convert the unsorted linksMap into a slice for sorting links := make([]link, 0, len(linksMap)) for l := range linksMap { links = append(links, l) } // for easier debugging of the resulting file: sort.Stable(byPkg(links)) if err := writeAtomically(*output, true, func(w io.Writer) error { return json.NewEncoder(w).Encode(&links) }); err != nil { log.Fatal(err) } } piuparts-1.1.1/htdocs/0000755000000000000000000000000013605431717011542 5ustar piuparts-1.1.1/htdocs/bug_howto.tpl0000644000000000000000000001062713363064533014265 0ustar
How to file bugs based on tests run on piuparts.debian.org
This page aims to summarize how to file useful bugs fast. It assumes you are familar with reporting bugs in Debian.
First, of all, read the piuparts logfile and identify why piuparts testing failed.
Then, check the BTS for that package, to see if this issue was already filed as a bug. Often it's also useful to check the source packages bug page. Sometimes a bug already exists, describing the problem piuparts has found. More often, new bugs have to be filed.
Usertagging existing bugs to make them known to piuparts.debian.org
If there already is a bug describing the same problem you're seeing in the piuparts logfile, you can usertag it, so that the next piuparts-analyze run will be able to link the bug report with the logfile on piuparts.debian.org. (piuparts-analyze runs twice a day.) 
 User: debian-qa@lists.debian.org
 Usertags 987654 + piuparts
Filing new bugs
More often, there is no existing bug and you need to file one. To make this easy as well to have consistent quality bug reports, we collect templates for filing these bugs. Please use these templates! The following is an example bug report for illustration: 
 To: submit@bugs.debian.org
 Subject: $package: fails to upgrade from 'testing' - trying to overwrite ...

 Package: $package
 Version: $version
 Severity: serious
 User: debian-qa@lists.debian.org
 Usertags: piuparts

 Hi,

 during a test with piuparts I noticed your package fails to upgrade from
 'testing'. It installed fine in 'testing', then the upgrade to 'sid'
 fails because it tries to overwrite other packages files without
 declaring a replaces relation.

 See policy 7.6 at
 https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces

 From the attached log (scroll to the bottom...):

 $useful_except_from_logfile

 cheers,
        $your_name

 attachment: $failed_logfile
Please take care when filing bugs to file meaningful bugs and to not annoy maintainers. Don't nitpick or insist on severities, the important thing is to get the bug fixed, not the right severity. Optionally you can also send copies to the piuparts-devel mailinglist by adding X-debbugs-cc: piuparts-devel@alioth-lists.debian.net pseudo-headers.
Also, you should be aware that what you are doing can probably be seen as mass bug filing (even if you just file a few now, they are part of a series of bugs of one kind) and as such needs to be discussed on debian-devel@lists.d.o first! For many types of bugs this has already been done. This is or should be indicated in the summary web pages as well as the mail templates.
Marking bugs as affecting other packages
Sometimes there is a bug in another package which affects a package being tested. The following explains how to tell this to the BTS in a way piuparts-analyze will pick up: 
 # assume 987654 is our bug report in buggy-package,
 # but the problem only shows up when testing (upgrades of)
 # failing-package with piuparts:
 bts affects 987654 failing-package

 # and if failing-package is from a different source with a different
 # version number:
 bts found 987654 failing-package/$FAILED_VERSION
piuparts-1.1.1/htdocs/favicon.ico0000644000000000000000000001346613605426771013701 0ustar   F 0 h( @ %UֿUֿ@Uֿ@%@@@%@@@@Uֿ@Uֿ@@%@@@@@@V_Uֿ3͟V_3͟@ֿ%%@@@@3͟@@%%%V_@@V_@@V_%3͟V_@Uֿ@Uֿ%@ֿV_@V_@@V_ֿ%@V_Uֿ@Uֿ@@V_@%@@%@@@@@V_3͟@@V_3͟@@V_V_@@%V_@%V_ֿ@%@V_V_@%V_V_@@@V_%@@@3͟@ֿV_@@%@@@@%@@@3͟@@@V_@@@%@@@@@@@@@@@@@Uֿ%@@@@%@@@@@@@@@@@@@@@@@@@@@@@@@@%V_%@@%ֿUֿ3͟?|???( @??|?????|???(  V_%V_Uֿ%V_3͟%V_3͟%V_V_@@V_V_@V_Uֿ%@@UֿV_@Uֿ%%V_3͟@V_UֿUֿ3͟%3͟%V_@3͟@V_V_@3͟3͟Uֿ3͟Uֿ%V_Uֿ%V_V_@UֿV_%@V_Uֿ@V_V_%@UֿUֿ%@V_V_@@V_3?( 3939piuparts-1.1.1/htdocs/images/0000755000000000000000000000000013266762112013006 5ustar piuparts-1.1.1/htdocs/images/debian.png0000644000000000000000000000746713266762112014754 0ustar PNG  IHDR=i pHYsHHFk>IDATx tMǟJ%!5 b-U AKi"]J4֩RGH֒-bH,HOwry^4y;sg3ej^Z*$X@_)zo+V`%ہﴒa<{̂Pb"~x")))00k׮J߷o}qq(pFFNNÇϟߪU+k6l٪ɓ͛ WZUE2^)~|ʪY1t~~mMI7{߾};22r޼yժUަM]N2eGh(dӽV={.jQ\Msc!<g%ر"d.G}dj2 p&d6m%*.E'MTT>,,F^Ph_c&޽;o:d͵!>j(u"‘l|QfB2 b)2 yv>,!cʔ)$Cf#xeD.%ԩSf9Hk;친##// '*V2?[{J +2Y| KQ\\gqJ2$C233y~7@?T`_t{Çuk >uV=z$.EVL]:u*mUVyyy'RA6lIFaa! [еozt;wGmoooggfӧ,Y2tP2S͚5}||-[v9}AAO-XI& cU.\ rrr"!Ųx@O888h6<9s|(...[l1gϺ3򷵵Ezt!<|т#Fٳgztuuu0n8tMקO'N0Zɓx`J&E2L2RV{*[l1Zd@կ__ 1&䝧Hʢ͓Jۛ ֭$}QQ}*I hFAP #,1gq˗ߵkTXx,{=h]b%lX}^dpb!Ȑ!CqFQv48Ќ 2(zUz[nHJJT*UJ 򠨁 -fbb"'l#"r=M$A&O,7VRQuvv6Ȉ48he[\h~;>Q[J& uC hx/^<|UN2 fbn7L66h```Æ %T V1}t(ƣk׮Iܹs)aw$얃O Ehd$''KA 8q=F cFF@@mX F ># %xVĤ͚5C'1? FtŴa݈:\Au'o޼wezQhJ]3f ?0b8::~(oߞ?ʍ7O!\bMGaKE ޽9ɘ3gq@<}A=6d˫`_%,] _asQϜBBx9/Cc!F=a __~Q;DM|ڵ;~80UwUF,*7(~.]شi1=Ñ=ݙ1MWd d "ؙG8]&j7|C{H2 'EơCrt #((K { CȞW8!1aE2`AD"gkԨ!f ~P}ˢ5%ܹ^Aܓ“?їHF3hKV@ƶmT!Ɔ=$ȀPreb͹HN:O= y*zf#f%bY~~~ d.bڵksGb_|d8;;+P1sd)66,$$pL,?dt &n&o|}IALϜ9f͚J 3fBzD2hǼ?-Z28w9" ECԪU+h&-Cʕ+)Əoj2E2ϵT8FV̒A۪YJh׮1EC4*U_2KZEݻ7bjU̒1vXFdgf[l_mt1tPSΆ RKF^^^Չ6mQiP1j9ӑM/2NJL^ u2ԩ#bŊ5~2 =999Ru}EmAZ.?"IF׮]i6l2}-Ed?iv,ю"nԨqd5%RjՂ3C\!$D(h{E(>O$7``o 2  9j+]s/jJNUeXOOOkӇv}E||bI2[(,?Ns…Zj1="O>G.F=Clbaa1dВA|3QFFqdߟ 44ìYߟǽ7/[={hK{I,NcD4e !QA " _ ֜9stoYbh+s)&fPڨuuu%EhΝd!7Q"~iҤ Q:vȹ.j%wy)|4eF%rww7l"Gpd@<z@#CC+++Øa%~˘[-Q 42.yN".Ϫnp%i߾͛7_2a)/U ̙30._y8F>'O?uVƵRi&))I\ HЕnݺ6˱ܹs.0]KcϏf0ho-Ji80J< '+رcĈ'h{AݻoˆE5 Ɔ,"DZ`/w5G ~Fubbb@Ұ.簰0ٯ+W|-ׯ?F݋ogD" ٤IIxo̘1V3$ @!oܸmD#<ըQ#"KIIQ%AWa ю֭=ٳV{u۷o'LLLDǎ\bRSS=xI׉k׮_v =(|FHgO/2 m[rAIENDB`piuparts-1.1.1/htdocs/images/openlogo-nd-50.png0000644000000000000000000000136713266762112016166 0ustar PNG  IHDR2=طV3PLTE6Cꟳ@h` O0\p揧PubKGDH pHYs  ~tIMEGJIDATxVI DŽ $6:֩)CdX|\DxК[NSi`ͳ@М;W$@!IGt#jDЮY2Îa_ΰzεc󋛭"]O~.G2SEw bcF2LdxfaP[s>0ÁI`P?ȕ *r vy)*E{SklW=ͳpY9#Tm=!4+ŋ- {娔~Lm{-0x^_GY7XQ'J`WSX)*NZm]'F86h_X T3BR XFcL?߽kT^Y?I cWe-[,:moOoeȟ#]5b@~sMIb ?7_VOp>!Od;Ӝx7j7mU:@BK[Vw 7?sj~wɻOxRE^ZħpP<l5t59XpLI ֊{ߙJTȍ4]NJr JFz|$*+lG)VzE[IW}lU! o~qn'Y%I->x4~g$gKMOUr!,ZPUפ@T+j[U1A@kEwsO?m9|[YV PIK&z*&.Rj T!lHQ il)I r@0ֲPIYWYޒGZo !kIENDB`piuparts-1.1.1/htdocs/index.tpl0000644000000000000000000001033413605426771013377 0ustar
About piuparts.debian.org
piuparts is a tool for testing that .deb packages can be installed, upgraded, and removed without problems. piuparts is short for "package installation, upgrading and removal testing suite" and is a variant of something suggested by Tollef Fog Heen.
It does this by creating a minimal Debian installation in a chroot, and installing, upgrading, and removing packages in that environment, and comparing the state of the directory tree before and after. piuparts reports any files that have been added, removed, or modified during this process. piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive.
A quick introduction is available in the piuparts README, and all the options are listed on the piuparts manpage.
Besides all the information provided here, there is some more information available on wiki.debian.org:
The preferred method to reach us is via an email to the piuparts development mailinglist. Good ways to contribute are to provide patches via GIT pull requests and/or to file bugs based on piuparts runs.
Since 2005 piuparts is run systematically on all packages in Debian, at first only on unstable. Today piuparts.debian.org is testing many suites by running piuparts in master/slave mode. This setup currently consists of two hosts: pejacevic.debian.org and and piu-slave-ubc-01.debian.org:
  • pejacevic acts as the piuparts-master, which is responsible for scheduling test jobs to the slaves. The other main task is to generate the reports which are served via https://piuparts.debian.org.
  • piu-slave-ubc-01 runs four piuparts-slave instances, which then run piuparts itself.
To learn more about this setup, follow the "Documentation" links in the navigation menu on the left. Read those READMEs. The piuparts configuration for all the different suite(-combination)s that are currently being tested is also linked there.
These pages are updated two to three times a day.
Bugs submitted which were found using piuparts
piuparts-1.1.1/htdocs/news.tpl0000644000000000000000000010272613605431717013247 0ustar
News
2020-01-08 After maintaining piuparts.debian.org for more than a decade, Holger Levsen resigns from doing so and also steps back from maintaining the piuparts source package in Debian, after doing this for even more years. It was really a lot of fun, I learned a lot and am thankful for being able to contribute this service to improve Debian. Many thanks to all who contributed to it, especially Andreas Beckmann. And also many thanks to those, who will keep it running in the future!
2019-12-27 piuparts.debian.org switch to python3 done. Many thanks to all who contributed: Herbert Parentes Fortes Neto, Thomas Goirand, Mattia Rizzolo, Holger Levsen, Bastian Venthur, Nis Martensen and James Clarke.
2019-07-31 Add two new suites, bullseye and bullseye-rcmd, after the release of buster on July 6th.
2019-05-04 Release of piuparts 1.0.0 - may the force be with you! If everything goes as planned, this is also the last Python 2 release. Many thanks and kudos to Lars Wirzenius for starting to write piuparts 14 years ago.
2019-04-10 Another suite added: sid-merged-usr, to test package installation, removal and purge in sid in a --merged-usr environment.
2018-11-28: The output of various maintenance scripts is now available at https://piuparts.debian.org/logs/ instead of sending it via mails to the piuparts.d.o maintainers only.
2018-10-19: Three and a half years after stopping linking to packages.qa.debian.org (which was called PTS) we finally drop all references to the PTS and use tracker.d.o instead.
2018-08-28: Enable the logrotate test again to confirm #582630 has been fixed.
2018-04-22 Move git repo to salsa.debian.org. Many thanks to the alioth.debian.org admins for providing such nice services so long!
2017-08-14 Improve testing coverage of Wheezy LTS: wheezy2lts, squeeze2wheezy-lts and squeeze2bpo2wheezy now test upgrades up to wheezy-lts.
2017-04-28 Another suite added: stable2sid, to test package installation in stable (currently jessie), then upgrade directly to sid (always unstable), then removal and purge... This is useful to detect bugs before they reach testing.
2017-04-10 Another suite added: oldstable222sid, to test package installation in oldstable (currently wheezy), then upgrade to stable (currently jessie), then upgrade to testing (currently stretch), then upgrade to sid (always unstable), then removal and purge..
2017-04-07 Another suite added: stable22sid, to test package installation in stable (currently jessie), then upgrade to testing (currently stretch), then upgrade to sid (always unstable), then removal and purge..
2017-03-08 Changed cron to update the webpages two times a day instead of four. In the past updating took longer than six hours, so effectivly only two updates per day were done anyway.
2017-03-07 Another suite added: oldstable22testing, to test package installation in wheezy, then upgrade to jessie, then upgrade to testing, then removal and purge..
2017-02-27 Thanks again to DSA, pejacevic.debian.org is now a quad-core system instead of the single core system it was until now.
2017-02-25 To cope with the ever increasing Debian archive, a second slave node was added to the piuparts.debian.org setup, so that now there is pejacevic, the master host, plus piu-slave-bm-a, the old node, and piu-slave-ubc-01, the new node. All these nodes still only run the amd64 architecture, help to extend the code (and the web UI) to support testing several architectures would be greatly appriciated. Many thanks to DSA for maintaining the machines piuparts.debian.org is being run on!
2016-12-20 Another suite added: sid-strict, to test packages a bit stricter than usual: first instead of a simple install, an install, followed by a remove and then install is done and then file leftover after purge are also considered an error.
2016-12-04 piuparts results have been integrated with britney, the tool the release team uses for testing migrations.
2016-06-09 Two new suites were added: jessie2bpo, to test packages upgrades from jessie to jessie-backports and jessie2bpo2stretch, where these packages are also tested for upgrading to stretch.
2015-09-29 Another new suite was added: wheezy-pu, to only test packages in wheezy-proposed-updates.
2015-04-25 With the release of Jessie two new suites are being tested: jessie2stretch and stretch, which will become the next Debian release.
2015-02-04 Link to the new Debian Package Tracker (tracker.debian.org) instead to the old Package Tracker System (PTS).
2015-01-24 Another suite was added: jessie-rcmd, to test installations in jessie with --install-recommends.
2014-12-19 Two more new suites were added: jessie-pu, to only test packages in jessie-proposed-updates and wheezy2jessie-rcmd, to test package upgrades from wheezy to jessie with --install-recommends.
2014-12-05 In preparation of the jessie release, another new suite was added: jessie2proposed, testing installation in jessie, then upgrade to jessie-proposed-upgrades, ending in purge as usual. Web pages are now updated four times a day.
2014-05-30 Results from debsums on wheezy2jessie and wheezy2bpo2jessie are not being ignored anymore as #744398 has been fixed.
2014-05-22 Add squeeze-lts to the distros being testing (by testing squeeze2squeeze-lts upgrades).
2014-05-19 Add a graph to the startpage showing the number of RC and non-RC bugs filed due to running piuparts.
2014-05-11 Temporarily ignore debsums results for wheezy2jessie and wheezy2bpo2jessie due to #744398.
2014-02-26 A new JSON summary file is being published, showing package testing state, status URL, and the number of packages being blocked by failures, for each distribution.
2013-07-16 To better track bugs in piuparts.debian.org and piuparts itself, a new pseudo-package was created in the BTS: piuparts.debian.org, which will be used for tracking all issues with the piuparts.debian.org service.
2013-06-05 In preparation of the first wheezy point release, another new suite was added: squeeze2wheezy-proposed, testing installation in squeeze, then upgrade to wheezy-proposed-upgrades, ending in purge as usual.
2013-05-30 Another new suite added: wheezy2proposed, testing installation in wheezy, then upgrade to wheezy-proposed-upgrades, ending in purge as usual.
2013-05-29 Another new suite added: squeeze2bpo-sloppy, testing the upgrade from squeeze to squeeze-backports-sloppy, ending in purge as usual.
2013-05-22 The webpages served by https://piuparts.debian.org are updated twice a day now. Further changes which were applied last week: debsums failures have been reenabled, adequate is now run by piuparts (see #703902) and two new suites were added: experimental and sid-nodoc, which tests sid without files in /usr/share/doc/<package>.
2013-05-14 Thanks to the new "hardware", piu-slave-bm-a is running four slaves now. Plus, these slaves are also considerably faster than piatti. And there are two new suites being tested: wheezy2jessie and wheezy2bpo2jessie - whoohoo!
2013-05-13 piuparts.debian.org has been moved to a new hardware and hosting location, now running virtualized on this nice cluster at Bytemark. Thanks to the Debian System Administrators for their assistence in setting up the host and maintaining the Debian infrastructure! Also many thanks and kittos to the Department of Computer Science at the University of Helsinki, Finland, for hosting piatti.debian.org since 2005!
For maintaining this setup we used the *bikeshed* git branch.
2013-03-15 Among many other new features the 0.50 release offers much greater flexibility for configuring and selecting (partial) suites and different mirrors. Therefore it is possible to test nearly arbitrary upgrade pathes. On piuparts.debian.org this is now used for testing squeeze2bpo2wheezy and sid2experimental. Thanks to Andreas Beckmann for this great new feature!
2013-03-02 While the piuparts.git repo on Alioth (update 2018-04-23: moved to salsa) will continue to be the main repo, there is also a piuparts clone on github, for those who prefer to send pull requests that way.
2012-06-21 piuparts 0.45 has been released, featuring piuparts-master and piuparts-slave packages to ease installation of such a setup. If you run piuparts in master/slave mode, please let us know.
2012-06-04 Wheezy freeze is approaching and lots of uploads happening. Old piatti hardware has problems keeping up with the pace of uploads, number of packages and distros being tested! :-) Piatti is about six years old...
2012-03-31 Disable lenny2squeeze tests, as lenny has been archived.
2012-03-05: temporarily disabled this again until we've sorted out problems with it.
2012-02-20: piuparts-analyze now sends commands the BTS: if a bug has not been explicitly marked fixed in the new version, it can rather very savely be assumed it's still present.
2012-01-30: Add new suite to be tested, testing2sid, to catch upgrade problems before they reach testing.
2012-01-22: Since some weeks, piuparts-analyze is captable of moving logfiles from fail to bugged, if there is a bug report usertagged 'piuparts' against that package+version combination. Thus, since today there is a webpage, explaining how to file bugs based on tests run on piuparts.debian.org. So now the question how to help can easily be answered: read that page and start filing bugs!
2012-01-20: As squeeze2wheezy has been fully tested by today, re-enable rescheduling of old logs for sid, wheezy and squeezewheezy: 200 successful logs older than 90 days are rescheduled each day, plus 25 failed logs older than 30 days.
2011-12-20: Currently, while the machine is busy testing all of squeeze2wheeezy, all old log rescheduling has been disabled. Normally, these reschedulings happen for sid, wheezy and squeezewheezy: 200 successful logs old than 180 days are rescheduled each day, plus 25 failed logs older than 30 days.
2011-12-10: Finally, upgrades from squeeze to wheezy are also being tested. Yay!
2011-11-21: All mails created by the piuparts master-slave setup on piatti.d.o are now sent to the piuparts-reports mailinglist on alioth. Subcribe and learn more about the details of this setup!
2011-10-31: Re-create base.tgz's every week now, as they will only be replaced if the recreation was successful.
2011-10-23: Since today piuparts.debian.org is maintained in git, using the piatti branch.
2011-07-10: Since today dpkg is run with --force-unsafe-io by for all suites except lenny2squeeze, as dpkg from lenny doesn't support this option.
2011-07-10: systemd-sysv is the eighth package getting special treatment by piuparts as it needs removal of sysvinit before installation and installation of that package before removal...
2011-04-02: New daily cronjob to reschedule the oldest 200 logfiles of each sid and wheezy, if they are older then 180 days. IOW: make sure no logfile for sid and wheezy is older than half a year.
2011-02-22: piatti.debian.org has been upgraded to squeeze.
2011-02-07: Add wheezy! Whoohoo!
For now, the Wheezy distribution has just been added with the same testing options as Squeeze. In future, squeeze and lenny2squeeze will not be tested anymore, and squeeze2wheezy will also be added...
2011-01-25: Reschedule 27655 successfully tested packages in Squeeze, since they were tested before the deep freeze. Yesterday all 70 failed and bugged packages were rescheduled too, which surprisingly led to 6 successful tests, followed by a few more dependent packages also being tested.
2011-01-15: Reschedule 10123 successful and failed logs in lenny2squeeze for re-testing. Those are logs which have been tested before Squeeze was deep frozen or while there was still a bug in piuparts-slave, see last news entry for details.
2011-01-03: Reschedule 12306 successful and 8 bugged logs in lenny2squeeze for re-testing. Those are logs older than 148 days, which refers to when Squeeze was initially frozen (2010-08-06). Deep freeze was announced on 2010-12-13 and there are 3800 logs older then that too, but for future deletions it's better to use 2010-01-03 (=commit r857), which fixes a bug in piuparts-slave resulting in using the sid packages file for lenny2squeeze tests.
2010-11-28: debconf-english is the seventh package getting special treatment by piuparts: before removal, debconf-i18n is installed (see #539146 has the details and the news entry for 2010-11-25 lists the other six packages.)
2010-11-26: Schedule all 159 failed packages in lenny2squeeze for re-testing.
2010-11-25: Treat six packages specially: sudo (sensibly refuses removal if no root password is set), apt-listbugs (is called by apt and exists if there are RC buggy packages being upgraded), fai-nfsroot, ltsp-client-core (these two packages modify the installed system heavily and thus will only install if conditions are met), file-rc and upstart (these two replace essential packages and therefore apt needs to be told to do this).
2010-11-24: Disable the logrotate test until #582630 is fixed and reschedule all 51 packages in sid failed due to it.
2010-11-14: Schedule all 402 failed packages in sid for re-testing.
2010-11-12: Schedule all 108 failed packages in squeeze for re-testing. (Followup on 2010-09-04.)
2010-11-06: The lists of known circular depends is now taken from http://debian.semistable.com/debgraph.out.html and maintained separately (and manually) for each tested distribution in piuparts.conf - this is not optimal (which would be piuparts detecting them automatically) but much better than the hardcoded list which we had in the piuparts library since December 2009.
2010-09-04: Schedule all 27438 passed packages in squeeze for re-testing now that squeeze is frozen.
2009-07-24: #531349 has been fixed, piuparts results are now displayed in the PTS.
2010-05-18: From today on, broken logrotate scripts after purge are only reported in sid.
2010-05-16: Finally enabled testing of sid again. (Actually, sid was enabled on 2010-03-05, but piuparts.d.o was broken until today.)
2010-02-28: Due to #571925 testing of sid had to be disabled temporarily. On an unrelated note, testing of lenny2squeeze still has some issues atm...
2010-02-25: Since yesterday, squeeze and lenny2squeeze are being tested with "--warn-on-leftovers-after-purge" making piuparts only warn about leftover files after purge. This has two effects: an decrease in the number of failed logs to process, to better focus on more important problems and second, more packages will be tested, as less packages are (seen as) buggy. Today all failed packages in squeeze and lenny2squeeze have been rescheduled for testing.
2010-02-23: Since today, piuparts is able to detect broken logrotate scripts after purge, which will need retesting of all successfully tested packages eventually. The failed packages in squeeze also needs retesting, due to split into squeeze and lenny2squeeze last week.
2010-02-16: The squeeze test has been split into squeeze and lenny2squeeze, where squeeze means package installation in squeeze, removal and purge test, while lenny2squeeze means package installation in lenny, then upgrade to squeeze, then removal and purge test. This allows more issues to be found in squeeze since (potential) brokeness in lenny is not blurring the results in squeeze.
2010-01-05: Reschedule testing for 319 failed packages in sid and 544 in squeeze, since --warn-on-others is now used.
2009-12-24: Enable work-in-progress code to enable testing of packages with circular depends. This will allow testing of 5-6000 more packages in sid and squeeze, see #526046 and the 0.39 changelog for details. The list of packages with circular depends is currently hard-coded and will probably become a configuration option but not auto detected. But that's code yet to be written :-)
2009-12-21: So testing of 13398 in squeeze has taken 12 days, which is no big surprise as the squeeze tests are more complex. Today 499 failed packages from sid and 235 from squeeze have been rescheduled for testing, to catch broken symlinks in those too.
2009-12-12: After testing 14416 packages in sid in three days, reschedule 15944 packages in squeeze... see previous entry for an explanation why.
2009-12-09: Reschedule testing for 14287 successfully tested packages in sid, those in squeeze will be rescheduled once all testable package in sid have been tested again. This is because piuparts now creates and maintains chroots securily (using gpg signed Release files for both debootstrap and apt-get) and because it warns if broken symlinks are found in a package.
2009-12-05: Reschedule testing for ~400 failed packages in sid and ~600 in squeeze, to be followed by a rescheduling of all successful packages. This is because piuparts now warns if broken symlinks are found in a package.
2009-10-08: Reschedule testing for ~2000 failed packages in sid, which failed because of a problem when minimizing the chroot at the beginning of the piuparts tests. As of today, piuparts running on piuparts.debian.org does not minimize the chroots anymore.
2009-09-18: Reschedule testing for 17170 (successfully tested) packages in sid, to make sure they still install fine with dependency based booting enabled now in sid. Throwing away 42806 (successful) logfiles from those packages :-)
2009-09-16: Reschedule testing for 233 failing packages in sid which were affected by #545949. No packages in squeeze were affected.
2009-06-20: Failed logs are not grouped into (at the moment) seven types of known errors and one type of issues is detected in successful logs.
2009-06-06: Reschedule testing for 163 successful and 27 failing packages in sid which were affected by #530501. Once openssh 1:5.1p1-6 has reached squeeze, this will be done again with 194 packages there.
2009-05-27: Throw away all failed logs as there was a bug in piuparts leading to use a more uptodate mirror for getting the list of available packages and another for doing the tests. This lead to at least one fixed package which was incorrectly tested as failing, as an old version of the package was tested. To rule out some false positives about 1000 packages will be retested, but on this machine this will only take about a day :-)
2009-05-11: Filed #528266 and made piuparts ignore files in /tmp after purge. This got rid of 20 failures in sid and 14 in squeeze.
2009-05-06: Only believe statistics you faked yourself! Up until today piuparts used to include virtual packages (those only exist true the Provides: header) into the calculations of statistics of package states and the total number of packages. Suddenly, sid has 2444 packages less!
2009-05-01: All packages in squeeze and sid which can be tested have been tested. So it takes about one month to do a full piuparts run against one suite of the archive on this machine, that's almost 1000 packages tested per day.
2009-04-20: Deleted 86 more failed logfiles (out of 692 failures in total atm) which were due to broken packages, which most likely are temporarily uninstallable issues - a good indicator for this is that all of those failures happened in sid and none in squeeze. For the future there is a cronjob now, to notify the admins daily of such problems. In more distant future those issues should be detected and avoided.
2009-04-18: Deleted all 14 failed logfiles which complained about /var/games being present after purge, as this ain't an issue, see #524461.
2009-04-04: Deleted all failed logfiles so far for two reasons: until now, only three out of ten failure types where logged with a pattern to search for in the logfiles, now this is done for all ten types of failures. And second, the way of breaking circular dependencies was not bulletproof, thus there were false positives in the failures. Now it should be fine, though maybe this will lead to lots of untestable packages... we'll see.
2009-03-19: lenny2squeeze is not needed, so all logs for squeeze (as well as lenny2squeeze) were deleted. (As squeeze now includes two kinds of tests: installation and removal in squeeze, and installation in lenny, upgrade to squeeze, removal in squeeze.)
2009-02-28: Start maintaining piatti.debian.org via the piuparts svn repository on alioth.
2007-02-24: Holger puts piuparts source in svn.
2006-10-02: #390754 O: piuparts -- package installation, upgrading and removal testing tool"
2006-09-29: Lars seeks help maintaining piuparts.
2005-07-05: #317033 ITP: piuparts -- .deb package installation, upgrading, and removal testing tool
2005-06-19: Lars writes the first blog post about piuparts (version 0.4).
piuparts-1.1.1/htdocs/robots.txt0000644000000000000000000000003713266762112013612 0ustar User-agent: * Disallow: /fail/ piuparts-1.1.1/htdocs/style.css0000644000000000000000000001156013266762112013416 0ustar #main { border: none; padding-top: 10px; padding-bottom: 10px; padding-left: 10px; padding-right: 10px; vertical-align: top; background-color: #ddd; } hr { width: 100%; color:#d70751; background-color:#d70751; height:2px; margin: 0px; padding: 0px; } em { font-weight: bold; } p.note { font-family: sans-serif; color: #900; text-align: center; padding: 5px; font-size: 11px; font-weight: normal; } div.c1 {text-align: center} p.text { font-family: sans-serif; padding: 5px; font-weight: normal; } body { padding: 0px; margin: 0px; font-family: sans-serif; font-size: 90%; color: #000000; background-color: white; } #obeytoyourfriend { color: #000; background-color: white; border: 0px solid #000; border-collapse: collapse; margin-top: 0px; margin-bottom: 5px; padding-top: 0px; padding-left: 15px; font-family: serif; font-variant:small-caps; font-size: 140%; } p.validate { text-align: center; } table { color: #000; background-color: #000; border: 0px solid #000; border-collapse: separate; border-spacing: 1px; } h1 { font-size: 140%; text-align: center; color: #000; } h2 { font-size: 120%; text-align: center; color: #000; } h3 { font-size: 110%; text-align: center; color: #000; } p { font-size: 100%; text-align: justify; } tr { background-color: #FFF; } tr.odd { background-color: #FFFFFF; } tr.even { background-color: #e8e8e8; } td.sid { color: #000; text-align: left; } tr.experimental { color: #cc0000; } tr.unstable { color: #345677; } tr.sid_odd { color: #000; } td.exp { color: #cc0000; text-align: left; } tr.exp_odd { color: #900; } th { font-size: 120%; text-align: center; font-weight: bold; background-color: #BDF; border: 0px solid #000; padding-top: 10px; padding-bottom: 10px; padding-left: 6px; padding-right: 6px; } th.reject { font-size: 120%; text-align: center; font-weight: bold; background-color: #BDF; border: 0px solid #000; padding-top: 10px; padding-bottom: 10px; padding-left: 6px; padding-right: 6px; } td { font-size: 100%; border: 0px solid #000; padding: 4px; padding-left: 6px; padding-right: 6px; } a.needs-bugging { background-color:#e0c0d0; } span.needs-bugging { background-color:#e0c0d0; } #needs-bugging { background-color:#e0c0d0; } a.bugged { background-color:#c0c0f0; } a:link { color: #0000FF; text-decoration: none; } a:visited { color: #800080; text-decoration: none; } a:active { color: #FF0000; text-decoration: none; } a:hover { color: #0000FF; text-decoration: underline; } #header { padding: 3px; } h1.header { font-size: 32px; text-align: left; margin-bottom: 0px; margin-top: 0px; padding-top: 0px; padding-bottom: 0px; padding-left: 15px; } #footer { padding: 15px; font-size: 90%; text-align: center; } /* the following layout is taken from the PTS */ /* Tables */ table.containertable { clear: both; background-color: #ddd; border: none; } table.lefttable { width: 100%; border-collapse: collapse; border: 2px solid black; background-color: white; color: black; } table.righttable { width: 100%; border-collapse: collapse; border: 2px solid black; background-color: white; color: black; } td.containercell { background-color: #ddd; } td.titlecell { color: white; background-color: #d70751; font-weight: bold; text-align: center; padding: 0.2em 0.2em 0.1em 0.2em; border-top: 3px solid #999; border-bottom: 1px solid #999; } td.alerttitlecell { color: white; background-color: #0755d7; font-weight: bold; text-align: center; padding: 0.2em 0.2em 0.1em 0.2em; border-top: 3px solid #999; border-bottom: 1px solid #999; } td.labelcell { vertical-align: top; text-align: left; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; font-weight: bold; } td.alertlabelcell { color: white; background-color: #0755d7; vertical-align: top; text-align: left; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; font-weight: bold; } td.lightalertlabelcell { color: white; background-color: #90c0ff; vertical-align: top; text-align: left; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; font-weight: bold; } td.labelcell2 { padding: 0.2em 0.2em 0.1em 0.2em; border-top: 1px solid black; border-right: 1px dotted black; font-weight: bold; } td.contentcell { text-align: center; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; } td.lastcell { font-size: 80%; text-align: center; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; } td.contentcell2 { text-align: left; vertical-align: top; padding: 0.2em 0.3em 0.2em 0.3em; border-bottom: 1px dotted #999; border-right: 1px dotted #999; } piuparts-1.1.1/instances/0000755000000000000000000000000013605426771012251 5ustar piuparts-1.1.1/instances/Makefile0000644000000000000000000000120113605426771013703 0ustar ARCH = amd64 MIRROR = http://deb.debian.org/debian/ ARCH_piu-slave-1und1-01 = i386 MIRROR_piu-slave-1und1-01 = http://mirror.1und1.de/debian/ MIRROR_piu-slave-ubc-01 = http://mirror-ubc.debian.org/debian/ GENCONF = piuparts.conf.pejacevic GENCONF += piuparts.conf.piu-slave-1und1-01 GENCONF += piuparts.conf.piu-slave-ubc-01 all: $(GENCONF) clean: $(RM) $(GENCONF) piuparts.conf.%: piuparts.conf-template.pejacevic Makefile piuparts.conf-header.% cat piuparts.conf-header.$* > $@.tmp echo "" >> $@.tmp sed 's,@MIRROR@,$(or $(MIRROR_$*),$(MIRROR)),; s,@ARCH@,$(or $(ARCH_$*),$(ARCH)),;' < $< >> $@.tmp mv $@.tmp $@ piuparts-1.1.1/instances/README0000644000000000000000000000061013266762112013121 0ustar This directory contains live config files for real instances, usually with a .$HOSTNAME suffix. These are not intended for being used in the piuparts Debian packages (or only in /usr/share/doc/.../examples/). The update-piuparts-(master|slave)-setup scripts (run by piuparts(s|m)) use them when running and updating an instance directly from GIT. General config files are placed in ../conf/ piuparts-1.1.1/instances/piuparts.conf-header.pejacevic0000644000000000000000000000043513605426771020147 0ustar # # This is the configuration file for piuparts running in master-slave mode on # - pejacevic.debian.org (as master) and # - piu-slave-ubc-01.debian.org (as slave). # # For more information on this setup see https://salsa.debian.org/debian/piuparts/blob/develop/README_pejacevic.txt # piuparts-1.1.1/instances/piuparts.conf-header.piu-slave-1und1-010000644000000000000000000000076713605426771021277 0ustar # # This is the configuration file for piuparts running in master-slave mode on # - piu-slave-1und1-01.debian.org (as slave). # Except for the architecture being tested and the mirror configuration # this file is identical to the one for pejacevic and piu-slave-ubc-01. # # NOTE: However this slave does exist yet, as piuparts-master does not yet # support multiple architectures… # # For more information on this setup see https://salsa.debian.org/debian/piuparts/blob/develop/README_pejacevic.txt # piuparts-1.1.1/instances/piuparts.conf-header.piu-slave-ubc-010000644000000000000000000000054213605426771021107 0ustar # # This is the configuration file for piuparts running in master-slave mode on # - piu-slave-ubc-01.debian.org (as slave). # Except for the mirror configuration this file is identical to the one # for pejacevic and piu-slave-ubc-01. # # For more information on this setup see https://salsa.debian.org/debian/piuparts/blob/develop/README_pejacevic.txt # piuparts-1.1.1/instances/piuparts.conf-template.pejacevic0000644000000000000000000005400313605426771020532 0ustar [DEFAULT] # these are needed always flags-base = # default exceptions --scriptsdir /etc/piuparts/scripts # pejacevic's slaves are doing everything relevant on a ramdisk anyway --no-eatmydata # allow starting database servers --allow-database # default flags, only warning on leftover files flags-default = %(flags-base)s --warn-on-leftovers-after-purge # default flags for --merged-usr tests flags-merged-usr = %(flags-default)s # test in --merged-usr environment --merged-usr # these exist unowned and empty in an initial --merged-usr environment -i /usr/lib32/ -i /usr/libx32/ # like default flags, but failing on leftover files flags-leftovers = %(flags-base)s # restrict to problems in the package being tested --warn-on-others # perform some additional cleanup --scriptsdir /etc/piuparts/scripts-leftovers # aliases for the definitions below flags-start-testing = %(flags-start-bullseye)s flags-end-testing = %(flags-end-bullseye)s flags-start-stable = %(flags-start-stretch)s flags-end-stable = %(flags-end-stretch)s flags-start-oldstable = %(flags-start-jessie)s flags-end-oldstable = %(flags-end-jessie)s # common flags for tests starting in bullseye flags-start-bullseye = # no flags needed # common flags for tests ending in bullseye flags-end-bullseye = # no flags needed # common flags for tests starting in buster flags-start-buster = # debsums failures won't be fixed in buster, mostly related to # oldstyle packaging of aspell dictionaries --warn-on-debsums-errors # common flags for tests ending in buster flags-end-buster = # no flags needed # common flags for tests starting in stretch flags-start-stretch = # no flags needed # common flags for tests ending in stretch flags-end-stretch = # see #604807: --skip-logrotatefiles-test # debsums failures won't be fixed in stretch, mostly related to # oldstyle packaging of aspell dictionaries --warn-on-debsums-errors # common flags for tests starting in jessie flags-start-jessie = # no flags needed # common flags for tests ending in jessie flags-end-jessie = # extra fake-essential packages for successfully purging in jessie --scriptsdir /etc/piuparts/scripts-jessie # see #604807: --skip-logrotatefiles-test # debsums failures won't be fixed in jessie, mostly related to # obsolete/renamed conffiles that moved to different packages --warn-on-debsums-errors # won't be fixed in jessie --warn-on-install-over-symlink # common flags for tests starting in wheezy flags-start-wheezy = # no flags needed # common flags for tests ending in wheezy flags-end-wheezy = # extra fake-essential packages for successfully purging in wheezy --scriptsdir /etc/piuparts/scripts-wheezy # see #604807: --skip-logrotatefiles-test # debsums failures won't be fixed in wheezy --warn-on-debsums-errors # won't be fixed in wheezy --warn-on-install-over-symlink # common flags for tests starting in squeeze flags-start-squeeze = # up to squeeze a non-empty /etc/shells was shipped, actually installing # and removing a shell would remove its entry from /etc/shells -i /etc/shells # common flags for tests ending in squeeze flags-end-squeeze = # extra fake-essential packages for successfully purging in squeeze --scriptsdir /etc/piuparts/scripts-squeeze # see #604807: --skip-logrotatefiles-test # squeeze has been archived --warn-on-debsums-errors --warn-on-install-over-symlink # common flags for tests starting in lenny flags-start-lenny = # dpkg --force-unsafe-io was added in squeeze --dpkg-noforce-unsafe-io # up to squeeze a non-empty /etc/shells was shipped, actually installing # and removing a shell would remove its entry from /etc/shells -i /etc/shells # common flags for testing for broken symlinks flags-broken-symlinks = %(flags-default)s --fail-on-broken-symlinks --install-recommends --scriptsdir /etc/piuparts/scripts-broken-symlinks # --fake-essential-packages libjs-sphinxdoc # ignore intentionally broken symlinks -i :/usr/lib/libcxx/test/std/experimental/filesystem/Inputs/static_test_env/bad_symlink -i :/usr/share/gocode/src/github.com/Masterminds/glide/testdata/path/x/vendor -i :/usr/share/gocode/src/github.com/constabulary/gb/internal/fileutils/_testdata/copyfile/a/rick -i :/usr/share/gocode/src/github.com/hashicorp/atlas-go/archive/test-fixtures/archive-symlink/link/link -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/exe -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/ns/mnt -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/ns/net -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26232/cwd -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26232/root # ignore broken symlinks -i :/etc/systemd/system/display-manager.service # civicrm-common, #857601, wontfix, /usr/share/civicrm/packages/OpenFlashChart/open-flash-chart.swf -> ../../../../local/open-flash-chart.swf -i :/usr/share/civicrm/packages/OpenFlashChart/open-flash-chart.swf # musescore-common, #910145, wontfix, /usr/share/mscore-2.3/sound/{sf2,sfz} -> ../../sounds/{sf2,sfz} -I :/usr/share/mscore3?-.*/sound/sf[2z] # mender-client, #915680, /var/lib/mender -> /data/mender -i :/var/lib/mender # ignore broken symlinks (patterns) -I :/etc/.*/supervise -I /usr/include/python2\.7_d/.*\.h -I /usr/include/python3\..dm/.*\.h -I /usr/include/python3\..dm/cpython -I /usr/include/python3\..dm/internal -I :/usr/lib/jvm/java-.*-openjdk-.*/src\.zip -I :/usr/share/doc/kde/HTML/.*/common -I :/usr/share/man/man1/(.*-)?python.*-config\.1\.gz # ignore broken symlinks (third party game data) -I :/usr/lib/openjk/Jedi(Academy|Outcast)/base/assets.\.pk3 -I :/usr/lib/rtcw/main/.*\.(cfg|pk3) [global] basetgz-sections = tarball/sid tarball/sid-merged-usr tarball/bullseye tarball/buster tarball/stretch tarball/jessie tarball/wheezy tarball/squeeze tarball/lenny sections = experimental sid2experimental sid sid-strict sid-nodoc sid-merged-usr sid-broken-symlinks testing2sid bullseye bullseye-rcmd stable2sid stable22sid buster buster-rcmd buster-security buster-pu # buster-next buster2next stretch2buster stretch2Xbuster stretch2buster-rcmd stretch2Xbuster-rcmd stretch2bpo2buster stretch2bpo stretch stretch-rcmd stretch-security stretch-pu # stretch-next stretch2next oldstable222sid oldstable22testing jessie2stretch jessie2Xstretch jessie2stretch-rcmd jessie2Xstretch-rcmd jessie-lts2stretch jessie2bpo2stretch jessie2bpo # jessie-lts jessie2lts jessie jessie-rcmd jessie-security wheezy2jessie-lts wheezy2jessie wheezy2jessie-rcmd wheezy2bpo2jessie # wheezy-lts wheezy2lts wheezy wheezy-security squeeze2wheezy-lts squeeze2wheezy squeeze2bpo-sloppy squeeze2bpo2wheezy squeeze2squeeze-lts squeeze lenny2squeeze testing-suite = bullseye mirror = @MIRROR@ master-host = pejacevic.debian.org master-user = piupartsm bts-from = piuparts-devel@alioth-lists.debian.net master-command = /srv/piuparts.debian.org/share/piuparts/piuparts-master piuparts-command = sudo env PYTHONPATH=%(PYTHONPATH)s timeout -s INT -k 5m 80m /srv/piuparts.debian.org/sbin/piuparts PYTHONPATH = /srv/piuparts.debian.org/lib/python3/dist-packages master-directory = /srv/piuparts.debian.org/master slave-directory = /srv/piuparts.debian.org/slave basetgz-directory = /srv/piuparts.debian.org/slave/basetgz output-directory = /srv/piuparts.debian.org/htdocs backup-directory = /srv/piuparts.debian.org/backup tmpdir = /srv/piuparts.debian.org/tmp doc-root = / chroot-meta-directory = /srv/piuparts.debian.org/slave/refchroot chroot-meta-auto = reference-chroot-metadata.dat components = main arch = @ARCH@ area = main # the slave-count setting is for the slave(s) slave-count = 4 slave-flush-interval = 1800 # 3600s = 1h idle-sleep = 3600 max-tgz-age = 0 max-reserved = 100 # rescheduling settings reschedule-old-count = 250 reschedule-old-days = 90 expire-old-days = +30 reschedule-fail-count = 50 reschedule-fail-days = 10 expire-fail-days = +5 reschedule-untestable-days = 1 [tarball/sid] piuparts-flags = %(flags-default)s distro = None upgrade-test-distros = sid # 3 days (60*60*24*3) max-tgz-age = 259200 [tarball/sid-merged-usr] piuparts-flags = %(flags-merged-usr)s distro = None upgrade-test-distros = sid # 3 days (60*60*24*3) max-tgz-age = 259200 [tarball/bullseye] piuparts-flags = %(flags-default)s %(flags-start-bullseye)s distro = None upgrade-test-distros = bullseye # 1 week (60*60*24*7) max-tgz-age = 604800 [tarball/buster] piuparts-flags = %(flags-default)s %(flags-start-buster)s distro = None upgrade-test-distros = buster # 1 month (60*60*24*30) max-tgz-age = 2592000 [tarball/stretch] piuparts-flags = %(flags-default)s %(flags-start-stretch)s distro = None upgrade-test-distros = stretch # 1 month (60*60*24*30) max-tgz-age = 2592000 [tarball/jessie] piuparts-flags = %(flags-default)s %(flags-start-jessie)s distro = None upgrade-test-distros = jessie # 1 month (60*60*24*30) max-tgz-age = 2592000 [tarball/wheezy] piuparts-flags = %(flags-default)s %(flags-start-wheezy)s distro = None upgrade-test-distros = wheezy max-tgz-age = 0 [tarball/squeeze] piuparts-flags = %(flags-default)s %(flags-start-squeeze)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = squeeze max-tgz-age = 0 [tarball/lenny] piuparts-flags = %(flags-default)s %(flags-start-lenny)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = lenny max-tgz-age = 0 [experimental] precedence = 3 piuparts-flags = %(flags-default)s distro = experimental depends-sections = sid [sid2experimental] precedence = 3 piuparts-flags = %(flags-default)s depends-sections = sid distro = experimental upgrade-test-distros = sid experimental [sid] precedence = 1 piuparts-flags = --scriptsdir /etc/piuparts/scripts-log-alternatives %(flags-default)s distro = sid reschedule-fail-days = 2 expire-fail-days = +2 # we want packages in sid being retested sooner reschedule-old-count = 666 reschedule-old-days = 66 [sid-strict] precedence = 5 description = + Followed by another installation test. Also fails if there are leftover files after purge. piuparts-flags = --install-remove-install %(flags-leftovers)s # Once there are no packages left which leave files on purge behind, # --pedantic-purge-test should be added distro = sid [sid-nodoc] precedence = 7 description = + Testing without files in /usr/share/doc. piuparts-flags = %(flags-default)s --scriptsdir /etc/piuparts/scripts-no-usr-share-doc distro = sid [sid-merged-usr] precedence = 5 description = + In --merged-usr environment. json-sections = none piuparts-flags = %(flags-merged-usr)s distro = sid [sid-broken-symlinks] precedence = 9 description = + Failing on broken symlinks. json-sections = none piuparts-flags = %(flags-broken-symlinks)s distro = sid [testing2sid] precedence = 2 piuparts-flags = %(flags-default)s distro = testing upgrade-test-distros = testing sid reschedule-old-count = 0 [stable2sid] precedence = 4 description = + Find possible issues before the package migrates to testing. piuparts-flags = %(flags-default)s %(flags-start-stable)s distro = stable upgrade-test-distros = stable sid [stable22sid] precedence = 5 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stable)s distro = stable upgrade-test-distros = stable testing sid [oldstable222sid] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-oldstable)s distro = oldstable upgrade-test-distros = oldstable stable testing sid [oldstable22testing] precedence = 7 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-oldstable)s %(flags-end-testing)s distro = oldstable upgrade-test-distros = oldstable stable testing [bullseye] precedence = 3 piuparts-flags = --scriptsdir /etc/piuparts/scripts-log-alternatives %(flags-default)s %(flags-start-bullseye)s %(flags-end-bullseye)s distro = bullseye [bullseye-rcmd] precedence = 5 json-sections = none description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-bullseye)s %(flags-end-bullseye)s distro = bullseye [buster] precedence = 4 piuparts-flags = --scriptsdir /etc/piuparts/scripts-log-alternatives %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s distro = buster [buster-rcmd] precedence = 6 json-sections = none description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s distro = buster [buster-security] precedence = 2 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster distro = buster/updates [buster-pu] precedence = 3 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster distro = buster-proposed-updates [buster-next] precedence = 3 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s distro = buster-next [buster2next] precedence = 3 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s distro = buster upgrade-test-distros = buster buster-next [stretch2buster] precedence = 4 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s distro = stretch upgrade-test-distros = stretch buster [stretch2Xbuster] precedence = 5 json-sections = none description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2buster-rcmd] precedence = 6 json-sections = none description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s distro = stretch upgrade-test-distros = stretch buster [stretch2Xbuster-rcmd] precedence = 7 json-sections = none description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2bpo2buster] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s depends-sections = stretch distro = stretch-backports upgrade-test-distros = stretch stretch-backports buster [stretch2bpo] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = stretch distro = stretch-backports upgrade-test-distros = stretch stretch-backports [stretch] precedence = 5 piuparts-flags = --scriptsdir /etc/piuparts/scripts-log-alternatives %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch [stretch-rcmd] precedence = 6 json-sections = none description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch [stretch-security] precedence = 3 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch distro = stretch/updates [stretch-pu] precedence = 4 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch distro = stretch-proposed-updates [stretch-next] precedence = 4 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch-next [stretch2next] precedence = 4 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch upgrade-test-distros = stretch stretch-next [jessie2stretch] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s distro = jessie upgrade-test-distros = jessie stretch [jessie2Xstretch] precedence = 6 json-sections = none description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s distro = jessie upgrade-test-distros = jessie stretch exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie2stretch-rcmd] precedence = 7 json-sections = none description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s distro = jessie upgrade-test-distros = jessie stretch [jessie2Xstretch-rcmd] precedence = 7 json-sections = none description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s distro = jessie upgrade-test-distros = jessie stretch exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie-lts2stretch] precedence = 8 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s distro = jessie-lts upgrade-test-distros = jessie-lts stretch [jessie2bpo2stretch] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie distro = jessie-backports upgrade-test-distros = jessie jessie-backports stretch [jessie2bpo] precedence = 7 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie distro = jessie-backports upgrade-test-distros = jessie jessie-backports [jessie] precedence = 7 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie [jessie-rcmd] precedence = 70 json-sections = none description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie [jessie-security] precedence = 4 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie distro = jessie/updates [jessie-lts] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie-lts [jessie2lts] precedence = 6 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie upgrade-test-distros = jessie jessie-lts [wheezy2jessie-lts] precedence = 8 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s distro = wheezy upgrade-test-distros = wheezy jessie-lts [wheezy2jessie] precedence = 80 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s distro = wheezy upgrade-test-distros = wheezy jessie [wheezy2jessie-rcmd] precedence = 90 json-sections = none description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper distro = wheezy upgrade-test-distros = wheezy jessie [wheezy2bpo2jessie] precedence = 90 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy distro = wheezy-backports upgrade-test-distros = wheezy wheezy-backports jessie-lts [wheezy] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy reschedule-old-count = 0 [wheezy-lts] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy-lts reschedule-old-count = 0 [wheezy-security] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy distro = wheezy/updates reschedule-old-count = 0 [wheezy2lts] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy upgrade-test-distros = wheezy wheezy-lts reschedule-old-count = 0 [squeeze2wheezy-lts] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s distro = squeeze upgrade-test-distros = squeeze wheezy-lts reschedule-old-count = 0 [squeeze2wheezy] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2bpo2wheezy] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze distro = squeeze-backports upgrade-test-distros = squeeze squeeze-backports wheezy-lts reschedule-old-count = 0 [squeeze2bpo-sloppy] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze squeeze2bpo2wheezy upgrade-test-distros = squeeze squeeze-backports-sloppy reschedule-old-count = 0 [squeeze] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s distro = squeeze reschedule-old-count = 0 reschedule-fail-count = 0 [squeeze2squeeze-lts] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze distro = squeeze-lts upgrade-test-distros = squeeze squeeze-lts reschedule-old-count = 0 reschedule-fail-count = 0 [lenny2squeeze] precedence = 100 json-sections = none piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 reschedule-fail-count = 0 piuparts-1.1.1/instances/piuparts.conf.anbe0000644000000000000000000032446113605426771015705 0ustar # # This is the configuration file for piuparts in master-slave mode as run by AnBe. # [DEFAULT] flags-base = --no-eatmydata --scriptsdir /etc/piuparts/scripts --allow-database flags-default = %(flags-base)s --warn-on-leftovers-after-purge flags-merged-usr = --merged-usr -i /usr/lib32/ -i /usr/libx32/ %(flags-default)s flags-leftovers = --warn-on-others %(flags-base)s --scriptsdir /etc/piuparts/scripts-leftovers flags-start-testing = %(flags-start-bullseye)s flags-end-testing = %(flags-end-bullseye)s flags-start-stable = %(flags-start-stretch)s flags-start-bullseye = flags-end-bullseye = flags-start-buster = flags-end-buster = --warn-on-debsums-errors flags-start-stretch = flags-end-stretch = --skip-logrotatefiles-test --warn-on-debsums-errors flags-start-jessie = flags-end-jessie = --skip-logrotatefiles-test --warn-on-debsums-errors --warn-on-install-over-symlink --scriptsdir /etc/piuparts/scripts-jessie flags-start-wheezy = flags-end-wheezy = --skip-logrotatefiles-test --warn-on-debsums-errors --warn-on-install-over-symlink --scriptsdir /etc/piuparts/scripts-wheezy flags-start-squeeze = -i /etc/shells flags-end-squeeze = --skip-logrotatefiles-test --warn-on-debsums-errors --warn-on-install-over-symlink --scriptsdir /etc/piuparts/scripts-squeeze flags-start-lenny = --dpkg-noforce-unsafe-io -i /etc/shells flags-end-lenny = --skip-cronfiles-test --skip-logrotatefiles-test --warn-on-debsums-errors --warn-on-install-over-symlink --scriptsdir /etc/piuparts/scripts-lenny -i /etc/udev/ -i /etc/udev/rules.d/ flags-broken-symlinks = %(flags-default)s --fail-on-broken-symlinks --install-recommends --scriptsdir /etc/piuparts/scripts-broken-symlinks --fake-essential-packages libjs-sphinxdoc # ignore intentionally broken symlinks -i :/usr/lib/libcxx/test/std/experimental/filesystem/Inputs/static_test_env/bad_symlink -i :/usr/share/gocode/src/github.com/Masterminds/glide/testdata/path/x/vendor -i :/usr/share/gocode/src/github.com/constabulary/gb/internal/fileutils/_testdata/copyfile/a/rick -i :/usr/share/gocode/src/github.com/hashicorp/atlas-go/archive/test-fixtures/archive-symlink/link/link -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/exe -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/ns/mnt -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26231/ns/net -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26232/cwd -i :/usr/share/gocode/src/github.com/prometheus/procfs/fixtures/26232/root # ignore broken symlinks -i :/etc/systemd/system/display-manager.service # civicrm-common, #857601, wontfix, /usr/share/civicrm/packages/OpenFlashChart/open-flash-chart.swf -> ../../../../local/open-flash-chart.swf -i :/usr/share/civicrm/packages/OpenFlashChart/open-flash-chart.swf # musescore-common, #910145, wontfix, /usr/share/mscore-2.3/sound/{sf2,sfz} -> ../../sounds/{sf2,sfz} -I :/usr/share/mscore3?-.*/sound/sf[2z] # mender-client, #915680, /var/lib/mender -> /data/mender -i :/var/lib/mender # ignore broken symlinks (patterns) -I :/etc/.*/supervise -I /usr/include/python2\.7_d/.*\.h -I /usr/include/python3\..dm/.*\.h -I /usr/include/python3\..dm/cpython -I /usr/include/python3\..dm/internal -I :/usr/lib/jvm/java-.*-openjdk-.*/src\.zip -I :/usr/share/doc/kde/HTML/.*/common -I :/usr/share/man/man1/(.*-)?python.*-config\.1\.gz # ignore broken symlinks (third party game data) -I :/usr/lib/openjk/Jedi(Academy|Outcast)/base/assets.\.pk3 -I :/usr/lib/rtcw/main/.*\.(cfg|pk3) [global] # sections may be wrapped with continuation lines being indented # and may contain non-indented comments between continuation lines basetgz-sections = tarball/sid/amd64 tarball/sid/i386 tarball/sid-merged-usr/amd64 tarball/sid-merged-usr/i386 tarball/bullseye/amd64 tarball/bullseye/i386 tarball/buster/amd64 tarball/buster/i386 tarball/stretch/amd64 tarball/stretch/i386 tarball/stretch-merged-usr/amd64 tarball/stretch-merged-usr/i386 tarball/jessie/amd64 tarball/jessie/i386 tarball/wheezy/amd64 tarball/wheezy/i386 tarball/squeeze/amd64 tarball/squeeze/i386 tarball/lenny/amd64 tarball/lenny/i386 sections = experimental/main experimental_i386/main experimental-bl/main experimental-bl_i386/main sid/main sid-plus/main sid-rcmd/main sid-merged-usr/main sid-bl/main sid-bl_i386/main sid-ipi/main sid-debug/main testing-rcmd/main testing-nodocs/main buster/main buster-pu/main buster-debug/main stretch/main stretch-rcmd/main stretch-next/main stretch-security/main stretch-updates/main stretch-pu/main stretch-debug/main stretch-backports/main jessie/main jessie-rcmd/main jessie-lts/main jessie-security/main jessie-backports/main sid2experimental/main testing2sid/main stable2sid/main stable2testing2sid/main stretch2next/main stretch2buster/main stretch3buster/main stretch2buster_i386/main stretch3buster_i386/main stretch2buster-rcmd/main stretch3buster-rcmd/main stretch2buster-rcmd_i386/main stretch3buster-rcmd_i386/main stretch2buster-merged-usr/main stretch2buster-iri/main stretch2bpo2buster/main jessie2lts/main jessie2stretch/main jessie3stretch/main jessie2stretch_i386/main jessie3stretch_i386/main jessie2stretch-rcmd/main jessie3stretch-rcmd/main jessie2stretch-rcmd_i386/main jessie3stretch-rcmd_i386/main jessie2stretch-iri/main jessie2bpo2stretch/main jessie222testing/main jessie222testing_i386/main wheezy222testing/main wheezy222testing_i386/main squeeze222testing/main squeeze222testing_i386/main lenny222testing/main lenny222testing_i386/main # experimental/contrib experimental_i386/contrib experimental-bl/contrib experimental-bl_i386/contrib sid/contrib sid-plus/contrib sid-rcmd/contrib sid-merged-usr/contrib sid-bl/contrib sid-bl_i386/contrib sid-ipi/contrib sid-debug/contrib testing-rcmd/contrib testing-nodocs/contrib buster/contrib buster-pu/contrib buster-debug/contrib stretch/contrib stretch-rcmd/contrib stretch-next/contrib stretch-security/contrib stretch-updates/contrib stretch-pu/contrib stretch-debug/contrib stretch-backports/contrib jessie/contrib jessie-rcmd/contrib jessie-lts/contrib jessie-security/contrib jessie-backports/contrib sid2experimental/contrib testing2sid/contrib stable2sid/contrib stable2testing2sid/contrib stretch2next/contrib stretch2buster/contrib stretch3buster/contrib stretch2buster_i386/contrib stretch3buster_i386/contrib stretch2buster-rcmd/contrib stretch3buster-rcmd/contrib stretch2buster-rcmd_i386/contrib stretch3buster-rcmd_i386/contrib stretch2buster-merged-usr/contrib stretch2buster-iri/contrib stretch2bpo2buster/contrib jessie2lts/contrib jessie2stretch/contrib jessie3stretch/contrib jessie2stretch_i386/contrib jessie3stretch_i386/contrib jessie2stretch-rcmd/contrib jessie3stretch-rcmd/contrib jessie2stretch-rcmd_i386/contrib jessie3stretch-rcmd_i386/contrib jessie2stretch-iri/contrib jessie2bpo2stretch/contrib jessie222testing/contrib jessie222testing_i386/contrib wheezy222testing/contrib wheezy222testing_i386/contrib squeeze222testing/contrib squeeze222testing_i386/contrib lenny222testing/contrib lenny222testing_i386/contrib # experimental/non-free experimental_i386/non-free experimental-bl/non-free experimental-bl_i386/non-free sid/non-free sid-plus/non-free sid-rcmd/non-free sid-merged-usr/non-free sid-bl/non-free sid-bl_i386/non-free sid-ipi/non-free sid-debug/non-free testing-rcmd/non-free testing-nodocs/non-free buster/non-free buster-pu/non-free buster-debug/non-free stretch/non-free stretch-rcmd/non-free stretch-next/non-free stretch-security/non-free stretch-updates/non-free stretch-pu/non-free stretch-debug/non-free stretch-backports/non-free jessie/non-free jessie-rcmd/non-free jessie-lts/non-free jessie-security/non-free jessie-backports/non-free sid2experimental/non-free testing2sid/non-free stable2sid/non-free stable2testing2sid/non-free stretch2next/non-free stretch2buster/non-free stretch3buster/non-free stretch2buster_i386/non-free stretch3buster_i386/non-free stretch2buster-rcmd/non-free stretch3buster-rcmd/non-free stretch2buster-rcmd_i386/non-free stretch3buster-rcmd_i386/non-free stretch2buster-merged-usr/non-free stretch2buster-iri/non-free stretch2bpo2buster/non-free jessie2lts/non-free jessie2stretch/non-free jessie3stretch/non-free jessie2stretch_i386/non-free jessie3stretch_i386/non-free jessie2stretch-rcmd/non-free jessie3stretch-rcmd/non-free jessie2stretch-rcmd_i386/non-free jessie3stretch-rcmd_i386/non-free jessie2stretch-iri/non-free jessie2bpo2stretch/non-free jessie222testing/non-free jessie222testing_i386/non-free wheezy222testing/non-free wheezy222testing_i386/non-free squeeze222testing/non-free squeeze222testing_i386/non-free lenny222testing/non-free lenny222testing_i386/non-free # wheezy/main wheezy_i386/main wheezy-lts/main wheezy-security/main wheezy-backports/main wheezy2lts/main wheezy2jessie/main wheezy2jessie_i386/main # wheezy2jessie-rcmd/main # wheezy2jessie-rcmd_i386/main # wheezy2jessie-sysv/main # wheezy2jessie-apt1st/main wheezy2jessie-iri/main wheezy2bpo2jessie/main squeeze/main squeeze_i386/main squeeze-lts/main squeeze-backports/main squeeze-backports-sloppy/main squeeze2lts2wheezy/main squeeze2wheezy/main squeeze2wheezy_i386/main # squeeze2wheezy-rcmd/main # squeeze2wheezy-rcmd_i386/main squeeze2wheezy-lts/main squeeze2bpo2wheezy/main lenny/main lenny_i386/main lenny2squeeze/main lenny2squeeze_i386/main # wheezy/contrib wheezy_i386/contrib wheezy-lts/contrib # wheezy-security/contrib wheezy-backports/contrib wheezy2lts/contrib wheezy2jessie/contrib wheezy2jessie_i386/contrib # wheezy2jessie-rcmd/contrib # wheezy2jessie-rcmd_i386/contrib # wheezy2jessie-sysv/contrib # wheezy2jessie-apt1st/contrib wheezy2jessie-iri/contrib wheezy2bpo2jessie/contrib squeeze/contrib squeeze_i386/contrib squeeze-lts/contrib squeeze-backports/contrib # squeeze-backports-sloppy/contrib squeeze2lts2wheezy/contrib squeeze2wheezy/contrib squeeze2wheezy_i386/contrib # squeeze2wheezy-rcmd/contrib # squeeze2wheezy-rcmd_i386/contrib squeeze2wheezy-lts/contrib squeeze2bpo2wheezy/contrib lenny/contrib lenny_i386/contrib lenny2squeeze/contrib lenny2squeeze_i386/contrib # wheezy/non-free wheezy_i386/non-free wheezy-lts/non-free wheezy-security/non-free wheezy-backports/non-free wheezy2lts/non-free wheezy2jessie/non-free wheezy2jessie_i386/non-free # wheezy2jessie-rcmd/non-free # wheezy2jessie-rcmd_i386/non-free # wheezy2jessie-sysv/non-free # wheezy2jessie-apt1st/non-free wheezy2jessie-iri/non-free wheezy2bpo2jessie/non-free squeeze/non-free squeeze_i386/non-free squeeze-lts/non-free squeeze-backports/non-free # squeeze-backports-sloppy/non-free squeeze2lts2wheezy/non-free squeeze2wheezy/non-free squeeze2wheezy_i386/non-free # squeeze2wheezy-rcmd/non-free # squeeze2wheezy-rcmd_i386/non-free squeeze2wheezy-lts/non-free squeeze2bpo2wheezy/non-free lenny/non-free lenny_i386/non-free lenny2squeeze/non-free lenny2squeeze_i386/non-free # testing-suite = bullseye # json-sections = none # proxy = http://localhost:3128 # mirror = http://ftp.de.debian.org/debian # master-host = localhost # master-user = piupartsm master-command = /srv/piuparts/share/piuparts/piuparts-master piuparts-command = sudo nice env PYTHONPATH=%(PYTHONPATH)s timeout -s INT -k 5m 110m /srv/piuparts/sbin/piuparts PYTHONPATH = /srv/piuparts/lib/python3/dist-packages master-directory = /srv/piuparts/master slave-directory = /srv/piuparts/slave basetgz-directory = /srv/piuparts/slave/basetgz output-directory = /srv/piuparts/htdocs backup-directory = /srv/piuparts/backup tmpdir = /srv/piuparts/tmp chroot-meta-directory = /srv/piuparts/slave/refchroot chroot-meta-auto = reference-chroot-metadata.dat slave-load-max = 15.5 idle-sleep = 3600 slave-flush-interval = 1800 max-tgz-age = -1 max-reserved = 50 reschedule-old-count = 200 reschedule-old-days = 120 expire-old-days = +15 reschedule-fail-count = 100 reschedule-fail-days = 5 expire-fail-days = +3 reschedule-untestable-days = 1 slave-count = 12 ############################################################################ ### Tarball creation. ### ############################################################################ [tarball/sid/amd64] piuparts-flags = %(flags-default)s distro = None upgrade-test-distros = sid arch = amd64 area = main components = main # 60*60*24*2 max-tgz-age = 172800 [tarball/sid/i386] piuparts-flags = %(flags-default)s distro = None upgrade-test-distros = sid arch = i386 setarch = linux32 area = main components = main # 60*60*24*2 max-tgz-age = 172800 [tarball/sid-merged-usr/amd64] piuparts-flags = %(flags-merged-usr)s distro = None upgrade-test-distros = sid arch = amd64 area = main components = main # 60*60*24*2 max-tgz-age = 172800 [tarball/sid-merged-usr/i386] piuparts-flags = %(flags-merged-usr)s distro = None upgrade-test-distros = sid arch = i386 setarch = linux32 area = main components = main # 60*60*24*2 max-tgz-age = 172800 [tarball/bullseye/amd64] piuparts-flags = %(flags-default)s %(flags-start-bullseye)s distro = None upgrade-test-distros = bullseye arch = amd64 area = main components = main # 60*60*24*7 max-tgz-age = 604800 [tarball/bullseye/i386] piuparts-flags = %(flags-default)s %(flags-start-bullseye)s distro = None upgrade-test-distros = bullseye arch = i386 setarch = linux32 area = main components = main # 60*60*24*7 max-tgz-age = 604800 [tarball/buster/amd64] piuparts-flags = %(flags-default)s %(flags-start-buster)s distro = None upgrade-test-distros = buster arch = amd64 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/buster/i386] piuparts-flags = %(flags-default)s %(flags-start-buster)s distro = None upgrade-test-distros = buster arch = i386 setarch = linux32 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/stretch/amd64] piuparts-flags = %(flags-default)s %(flags-start-stretch)s distro = None upgrade-test-distros = stretch arch = amd64 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/stretch/i386] piuparts-flags = %(flags-default)s %(flags-start-stretch)s distro = None upgrade-test-distros = stretch arch = i386 setarch = linux32 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/stretch-merged-usr/amd64] piuparts-flags = %(flags-merged-usr)s %(flags-start-stretch)s distro = None upgrade-test-distros = stretch arch = amd64 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/stretch-merged-usr/i386] piuparts-flags = %(flags-merged-usr)s %(flags-start-stretch)s distro = None upgrade-test-distros = stretch arch = i386 setarch = linux32 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/jessie/amd64] piuparts-flags = %(flags-default)s %(flags-start-jessie)s distro = None upgrade-test-distros = jessie arch = amd64 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/jessie/i386] piuparts-flags = %(flags-default)s %(flags-start-jessie)s distro = None upgrade-test-distros = jessie arch = i386 setarch = linux32 area = main components = main # 60*60*24*30 max-tgz-age = 2592000 [tarball/wheezy/amd64] piuparts-flags = %(flags-default)s %(flags-start-wheezy)s distro = None upgrade-test-distros = wheezy arch = amd64 area = main components = main max-tgz-age = 0 [tarball/wheezy/i386] piuparts-flags = %(flags-default)s %(flags-start-wheezy)s distro = None upgrade-test-distros = wheezy arch = i386 setarch = linux32 area = main components = main max-tgz-age = 0 [tarball/squeeze/amd64] piuparts-flags = %(flags-default)s %(flags-start-squeeze)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = squeeze arch = amd64 area = main components = main max-tgz-age = 0 [tarball/squeeze/i386] piuparts-flags = %(flags-default)s %(flags-start-squeeze)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = squeeze arch = i386 setarch = linux32 area = main components = main max-tgz-age = 0 [tarball/lenny/amd64] piuparts-flags = %(flags-default)s %(flags-start-lenny)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = lenny arch = amd64 area = main components = main max-tgz-age = 0 [tarball/lenny/i386] piuparts-flags = %(flags-default)s %(flags-start-lenny)s --keyring /usr/share/keyrings/debian-archive-removed-keys.gpg distro = None upgrade-test-distros = lenny arch = i386 setarch = linux32 area = main components = main max-tgz-age = 0 ############################################################################ ### Install, remove, and purge tests. ### ############################################################################ [experimental/main] precedence = 40 piuparts-flags = %(flags-default)s depends-sections = sid/main distro = experimental arch = amd64 area = main components = main reschedule-old-days = 30 [experimental/contrib] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = experimental/main sid/main sid/contrib sid/non-free distro = experimental arch = amd64 area = contrib reschedule-old-days = 30 [experimental/non-free] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = experimental/main sid/main sid/contrib sid/non-free distro = experimental arch = amd64 area = non-free reschedule-old-days = 30 [experimental_i386/main] precedence = 40 piuparts-flags = %(flags-default)s depends-sections = sid-rcmd/main distro = experimental arch = i386 setarch = linux32 area = main components = main reschedule-old-days = 30 [experimental_i386/contrib] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = experimental_i386/main sid-rcmd/main sid-rcmd/contrib sid-rcmd/non-free distro = experimental arch = i386 setarch = linux32 area = contrib reschedule-old-days = 30 [experimental_i386/non-free] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = experimental_i386/main sid-rcmd/main sid-rcmd/contrib sid-rcmd/non-free distro = experimental arch = i386 setarch = linux32 area = non-free reschedule-old-days = 30 [experimental-bl/main] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid/main distro = experimental arch = amd64 area = main components = main [experimental-bl/contrib] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = experimental-bl/main sid/main sid/contrib sid/non-free distro = experimental arch = amd64 area = contrib [experimental-bl/non-free] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = experimental-bl/main sid/main sid/contrib sid/non-free distro = experimental arch = amd64 area = non-free [experimental-bl_i386/main] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid-rcmd/main distro = experimental arch = i386 setarch = linux32 area = main components = main [experimental-bl_i386/contrib] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = experimental-bl_i386/main sid-rcmd/main sid-rcmd/contrib sid-rcmd/non-free distro = experimental arch = i386 setarch = linux32 area = contrib [experimental-bl_i386/non-free] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = experimental-bl_i386/main sid-rcmd/main sid-rcmd/contrib sid-rcmd/non-free distro = experimental arch = i386 setarch = linux32 area = non-free [sid/main] precedence = 10 piuparts-flags = %(flags-default)s --scriptsdir /etc/piuparts/scripts-log-alternatives distro = sid arch = amd64 area = main components = main reschedule-old-days = 30 [sid/contrib] precedence = 45 piuparts-flags = %(flags-default)s --scriptsdir /etc/piuparts/scripts-log-alternatives depends-sections = sid/main sid/non-free distro = sid arch = amd64 area = contrib reschedule-old-days = 30 [sid/non-free] precedence = 45 piuparts-flags = %(flags-default)s --scriptsdir /etc/piuparts/scripts-log-alternatives depends-sections = sid/main sid/contrib distro = sid arch = amd64 area = non-free reschedule-old-days = 30 [sid-plus/main] precedence = 30 description = Debian %(distro)s / %(area)s: package installation, removal, installation, removal, and purge test. Failing on leftovers after purge. piuparts-flags = --install-remove-install %(flags-leftovers)s distro = sid arch = amd64 area = main components = main reschedule-old-days = 60 [sid-plus/contrib] precedence = 45 description = Debian %(distro)s / %(area)s: package installation, removal, installation, removal, and purge test. Failing on leftovers after purge. piuparts-flags = --install-remove-install %(flags-leftovers)s depends-sections = sid-plus/main sid-plus/non-free distro = sid arch = amd64 area = contrib reschedule-old-days = 60 [sid-plus/non-free] precedence = 45 description = Debian %(distro)s / %(area)s: package installation, removal, installation, removal, and purge test. Failing on leftovers after purge. piuparts-flags = --install-remove-install %(flags-leftovers)s depends-sections = sid-plus/main sid-plus/contrib distro = sid arch = amd64 area = non-free reschedule-old-days = 60 [sid-rcmd/main] precedence = 30 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s distro = sid arch = i386 setarch = linux32 area = main components = main reschedule-old-days = 60 [sid-rcmd/contrib] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s depends-sections = sid-rcmd/main sid-rcmd/non-free distro = sid arch = i386 setarch = linux32 area = contrib reschedule-old-days = 60 [sid-rcmd/non-free] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s depends-sections = sid-rcmd/main sid-rcmd/contrib distro = sid arch = i386 setarch = linux32 area = non-free reschedule-old-days = 60 [sid-merged-usr/main] precedence = 30 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s distro = sid arch = amd64 area = main components = main [sid-merged-usr/contrib] precedence = 45 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s depends-sections = sid-merged-usr/main sid-merged-usr/non-free distro = sid arch = amd64 area = contrib [sid-merged-usr/non-free] precedence = 45 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s depends-sections = sid-merged-usr/main sid-merged-usr/contrib distro = sid arch = amd64 area = non-free [sid-bl/main] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s distro = sid arch = amd64 area = main components = main [sid-bl/contrib] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid-bl/main sid-bl/non-free distro = sid arch = amd64 area = contrib [sid-bl/non-free] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid-bl/main sid-bl/contrib distro = sid arch = amd64 area = non-free [sid-bl_i386/main] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s distro = sid arch = i386 setarch = linux32 area = main components = main [sid-bl_i386/contrib] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid-bl_i386/main sid-bl_i386/non-free distro = sid arch = i386 setarch = linux32 area = contrib [sid-bl_i386/non-free] precedence = 130 description = + Failing on broken symlinks. piuparts-flags = %(flags-broken-symlinks)s depends-sections = sid-bl_i386/main sid-bl_i386/contrib distro = sid arch = i386 setarch = linux32 area = non-free [sid-debug/main] precedence = 40 piuparts-flags = %(flags-default)s depends-sections = sid/main distro = sid-debug arch = amd64 area = main components = main reschedule-old-count = 0 [sid-debug/contrib] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = sid/main sid/contrib sid/non-free distro = sid-debug arch = amd64 area = contrib reschedule-old-count = 0 [sid-debug/non-free] precedence = 45 piuparts-flags = %(flags-default)s depends-sections = sid/main sid/contrib sid/non-free distro = sid-debug arch = amd64 area = non-free reschedule-old-count = 0 [testing-rcmd/main] precedence = 30 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s distro = testing arch = i386 setarch = linux32 area = main components = main [testing-rcmd/contrib] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s depends-sections = testing-rcmd/main testing-rcmd/non-free distro = testing arch = i386 setarch = linux32 area = contrib [testing-rcmd/non-free] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s depends-sections = testing-rcmd/main testing-rcmd/contrib distro = testing arch = i386 setarch = linux32 area = non-free [testing-nodocs/main] precedence = 30 description = + Testing without files in /usr/share/doc. piuparts-flags = %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-no-usr-share-doc distro = testing arch = i386 setarch = linux32 area = main components = main [testing-nodocs/contrib] precedence = 45 description = + Testing without files in /usr/share/doc. piuparts-flags = %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-no-usr-share-doc depends-sections = testing-nodocs/main testing-nodocs/non-free distro = testing arch = i386 setarch = linux32 area = contrib [testing-nodocs/non-free] precedence = 45 description = + Testing without files in /usr/share/doc. piuparts-flags = %(flags-default)s %(flags-start-testing)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-no-usr-share-doc depends-sections = testing-nodocs/main testing-nodocs/contrib distro = testing arch = i386 setarch = linux32 area = non-free [buster/main] precedence = 10 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s distro = buster arch = amd64 area = main components = main reschedule-old-days = 60 [buster/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/non-free distro = buster arch = amd64 area = contrib reschedule-old-days = 60 [buster/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/contrib distro = buster arch = amd64 area = non-free reschedule-old-days = 60 [buster-pu/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main distro = buster-proposed-updates arch = amd64 area = main components = main [buster-pu/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/non-free distro = buster-proposed-updates arch = amd64 area = contrib [buster-pu/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/contrib distro = buster-proposed-updates arch = amd64 area = non-free [buster-debug/main] precedence = 40 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main distro = buster-debug arch = amd64 area = main components = main reschedule-old-count = 0 [buster-debug/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/contrib buster/non-free distro = buster-debug arch = amd64 area = contrib reschedule-old-count = 0 [buster-debug/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-buster)s %(flags-end-buster)s depends-sections = buster/main buster/contrib buster/non-free distro = buster-debug arch = amd64 area = non-free reschedule-old-count = 0 [stretch/main] precedence = 10 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch arch = amd64 area = main components = main reschedule-old-days = 60 [stretch/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/non-free distro = stretch arch = amd64 area = contrib reschedule-old-days = 60 [stretch/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib distro = stretch arch = amd64 area = non-free reschedule-old-days = 60 [stretch-rcmd/main] precedence = 40 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch arch = i386 setarch = linux32 area = main components = main reschedule-old-count = 100 [stretch-rcmd/contrib] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch-rcmd/main stretch-rcmd/non-free distro = stretch arch = i386 setarch = linux32 area = contrib [stretch-rcmd/non-free] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch-rcmd/main stretch-rcmd/contrib distro = stretch arch = i386 setarch = linux32 area = non-free [stretch-security/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch/updates arch = amd64 area = main components = main [stretch-security/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib distro = stretch/updates arch = amd64 area = contrib [stretch-security/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib distro = stretch/updates arch = amd64 area = non-free [stretch-updates/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-updates arch = amd64 area = main components = main [stretch-updates/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-updates arch = amd64 area = contrib [stretch-updates/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-updates arch = amd64 area = non-free [stretch-pu/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-proposed-updates arch = amd64 area = main components = main [stretch-pu/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/non-free distro = stretch-proposed-updates arch = amd64 area = contrib [stretch-pu/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib distro = stretch-proposed-updates arch = amd64 area = non-free [stretch-next/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s distro = stretch-next arch = amd64 area = main components = main reschedule-old-count = 500 reschedule-old-days = 25 [stretch-next/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch-next/main stretch-next/non-free distro = stretch-next arch = amd64 area = contrib reschedule-old-days = 25 [stretch-next/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch-next/main stretch-next/contrib distro = stretch-next arch = amd64 area = non-free reschedule-old-days = 25 [stretch-debug/main] precedence = 40 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-debug arch = amd64 area = main components = main reschedule-old-count = 0 [stretch-debug/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib stretch/non-free distro = stretch-debug arch = amd64 area = contrib reschedule-old-count = 0 [stretch-debug/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib stretch/non-free distro = stretch-debug arch = amd64 area = non-free reschedule-old-count = 0 [stretch-backports/main] precedence = 40 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main distro = stretch-backports arch = amd64 area = main components = main [stretch-backports/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib stretch/non-free stretch-backports/main stretch-backports/non-free distro = stretch-backports arch = amd64 area = contrib [stretch-backports/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch/main stretch/contrib stretch/non-free stretch-backports/main stretch-backports/contrib distro = stretch-backports arch = amd64 area = non-free [jessie/main] precedence = 10 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie arch = amd64 area = main components = main reschedule-old-count = 100 [jessie/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main jessie/non-free distro = jessie arch = amd64 area = contrib [jessie/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main jessie/contrib distro = jessie arch = amd64 area = non-free [jessie-rcmd/main] precedence = 40 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie arch = i386 setarch = linux32 area = main components = main reschedule-old-count = 100 [jessie-rcmd/contrib] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie-rcmd/main jessie-rcmd/non-free distro = jessie arch = i386 setarch = linux32 area = contrib [jessie-rcmd/non-free] precedence = 45 description = + With recommended packages. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie-rcmd/main jessie-rcmd/contrib distro = jessie arch = i386 setarch = linux32 area = non-free [jessie-security/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main distro = jessie/updates arch = amd64 area = main components = main [jessie-security/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main distro = jessie/updates arch = amd64 area = contrib [jessie-security/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main jessie/contrib distro = jessie/updates arch = amd64 area = non-free [jessie-lts/main] precedence = 20 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s distro = jessie-lts arch = amd64 area = main components = main reschedule-old-count = 500 reschedule-old-days = 25 [jessie-lts/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie-lts/main jessie-lts/non-free distro = jessie-lts arch = amd64 area = contrib reschedule-old-days = 25 [jessie-lts/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie-lts/main jessie-lts/contrib distro = jessie-lts arch = amd64 area = non-free reschedule-old-days = 25 [jessie-backports/main] precedence = 40 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main distro = jessie-backports arch = amd64 area = main components = main [jessie-backports/contrib] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main jessie/contrib jessie/non-free jessie-backports/main jessie-backports/non-free distro = jessie-backports arch = amd64 area = contrib [jessie-backports/non-free] precedence = 45 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie/main jessie/contrib jessie/non-free jessie-backports/main jessie-backports/contrib distro = jessie-backports arch = amd64 area = non-free [wheezy/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy arch = amd64 area = main components = main reschedule-old-count = 0 [wheezy/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main wheezy/non-free distro = wheezy arch = amd64 area = contrib reschedule-old-count = 0 [wheezy/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main wheezy/contrib distro = wheezy arch = amd64 area = non-free reschedule-old-count = 0 [wheezy_i386/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy arch = i386 setarch = linux32 area = main components = main reschedule-old-count = 0 [wheezy_i386/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy_i386/main wheezy_i386/non-free distro = wheezy arch = i386 setarch = linux32 area = contrib reschedule-old-count = 0 [wheezy_i386/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy_i386/main wheezy_i386/contrib distro = wheezy arch = i386 setarch = linux32 area = non-free reschedule-old-count = 0 [wheezy-security/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main distro = wheezy/updates arch = amd64 area = main components = main reschedule-old-count = 0 [wheezy-security/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main distro = wheezy/updates arch = amd64 area = contrib [wheezy-security/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main distro = wheezy/updates arch = amd64 area = non-free reschedule-old-count = 0 [wheezy-lts/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s distro = wheezy-lts arch = amd64 area = main components = main reschedule-old-count = 0 [wheezy-lts/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy-lts/main wheezy-lts/non-free distro = wheezy-lts arch = amd64 area = contrib reschedule-old-count = 0 [wheezy-lts/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy-lts/main wheezy-lts/contrib distro = wheezy-lts arch = amd64 area = non-free reschedule-old-count = 0 [wheezy-backports/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main distro = wheezy-backports arch = amd64 area = main components = main reschedule-old-count = 0 [wheezy-backports/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main wheezy/non-free wheezy-backports/main wheezy-backports/non-free distro = wheezy-backports arch = amd64 area = contrib reschedule-old-count = 0 [wheezy-backports/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy/main wheezy-backports/main wheezy-backports/contrib distro = wheezy-backports arch = amd64 area = non-free reschedule-old-count = 0 [squeeze/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s distro = squeeze arch = amd64 area = main components = main reschedule-old-count = 0 [squeeze/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze/non-free distro = squeeze arch = amd64 area = contrib reschedule-old-count = 0 [squeeze/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze/contrib distro = squeeze arch = amd64 area = non-free reschedule-old-count = 0 [squeeze_i386/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s distro = squeeze arch = i386 setarch = linux32 area = main components = main reschedule-old-count = 0 [squeeze_i386/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze_i386/main squeeze_i386/non-free distro = squeeze arch = i386 setarch = linux32 area = contrib reschedule-old-count = 0 [squeeze_i386/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze_i386/main squeeze_i386/contrib distro = squeeze arch = i386 setarch = linux32 area = non-free reschedule-old-count = 0 [squeeze-lts/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main distro = squeeze-lts arch = amd64 area = main components = main reschedule-old-count = 0 [squeeze-lts/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-lts/main squeeze/contrib squeeze/non-free squeeze-lts/non-free distro = squeeze-lts arch = amd64 area = contrib reschedule-old-count = 0 [squeeze-lts/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-lts/main squeeze/contrib squeeze-lts/contrib squeeze/non-free distro = squeeze-lts arch = amd64 area = non-free reschedule-old-count = 0 [squeeze-backports/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main distro = squeeze-backports arch = amd64 area = main components = main reschedule-old-count = 0 [squeeze-backports/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-backports/main squeeze-backports/non-free distro = squeeze-backports arch = amd64 area = contrib reschedule-old-count = 0 [squeeze-backports/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-backports/main squeeze-backports/contrib distro = squeeze-backports arch = amd64 area = non-free reschedule-old-count = 0 [squeeze-backports-sloppy/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-backports/main distro = squeeze-backports-sloppy arch = amd64 area = main components = main reschedule-old-count = 0 [squeeze-backports-sloppy/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-backports/main squeeze-backports/non-free distro = squeeze-backports-sloppy arch = amd64 area = contrib reschedule-old-count = 0 [squeeze-backports-sloppy/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-squeeze)s depends-sections = squeeze/main squeeze-backports/main squeeze-backports/contrib distro = squeeze-backports-sloppy arch = amd64 area = non-free reschedule-old-count = 0 [lenny/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s distro = lenny arch = amd64 area = main components = main reschedule-old-count = 0 [lenny/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s depends-sections = lenny/main lenny/non-free distro = lenny arch = amd64 area = contrib reschedule-old-count = 0 [lenny/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s depends-sections = lenny/main lenny/contrib distro = lenny arch = amd64 area = non-free reschedule-old-count = 0 [lenny_i386/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s distro = lenny arch = i386 setarch = linux32 area = main components = main reschedule-old-count = 0 [lenny_i386/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s depends-sections = lenny_i386/main lenny_i386/non-free distro = lenny arch = i386 setarch = linux32 area = contrib reschedule-old-count = 0 [lenny_i386/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-lenny)s depends-sections = lenny_i386/main lenny_i386/contrib distro = lenny arch = i386 setarch = linux32 area = non-free reschedule-old-count = 0 ############################################################################ ### Install, remove, purge, install, remove, and purge tests. ### ############################################################################ [sid-ipi/main] precedence = 175 description = Debian %(distro)s / %(area)s: package installation, purge, installation, removal, and purge test. piuparts-flags = --install-purge-install --warn-on-leftovers-after-purge -i /var/lib/ghc/package.conf.d/package.cache %(flags-leftovers)s distro = sid arch = amd64 area = main components = main reschedule-old-count = 100 [sid-ipi/contrib] precedence = 175 description = Debian %(distro)s / %(area)s: package installation, purge, installation, removal, and purge test. piuparts-flags = --install-purge-install --warn-on-leftovers-after-purge -i /var/lib/ghc/package.conf.d/package.cache %(flags-leftovers)s depends-sections = sid-ipi/main sid-ipi/non-free distro = sid arch = amd64 area = contrib [sid-ipi/non-free] precedence = 175 description = Debian %(distro)s / %(area)s: package installation, purge, installation, removal, and purge test. piuparts-flags = --install-purge-install --warn-on-leftovers-after-purge -i /var/lib/ghc/package.conf.d/package.cache %(flags-leftovers)s depends-sections = sid-ipi/main sid-ipi/contrib distro = sid arch = amd64 area = non-free ############################################################################ ### Install, distupgrade, remove, and purge tests. ### ############################################################################ [lenny2squeeze/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s -i /usr/local/share/octave/ -i /usr/local/share/octave/site-m/ -i /usr/share/octave/ls-R -i /usr/share/octave/packages/ arch = amd64 area = main components = main distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny2squeeze/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s depends-sections = lenny2squeeze/main lenny2squeeze/non-free arch = amd64 area = contrib distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny2squeeze/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s depends-sections = lenny2squeeze/main lenny2squeeze/contrib arch = amd64 area = non-free distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny2squeeze_i386/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s -i /usr/local/share/octave/ -i /usr/local/share/octave/site-m/ -i /usr/share/octave/ls-R -i /usr/share/octave/packages/ arch = i386 setarch = linux32 area = main components = main distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny2squeeze_i386/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s depends-sections = lenny2squeeze_i386/main lenny2squeeze_i386/non-free arch = i386 setarch = linux32 area = contrib distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny2squeeze_i386/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-squeeze)s depends-sections = lenny2squeeze_i386/main lenny2squeeze_i386/contrib arch = i386 setarch = linux32 area = non-free distro = lenny upgrade-test-distros = lenny squeeze reschedule-old-count = 0 [lenny222testing/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = amd64 area = main components = main distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 reschedule-fail-days = 12 expire-fail-days = +8 [lenny222testing/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = lenny222testing/main lenny222testing/non-free arch = amd64 area = contrib distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [lenny222testing/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = lenny222testing/main lenny222testing/contrib arch = amd64 area = non-free distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [lenny222testing_i386/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = i386 setarch = linux32 area = main components = main distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 reschedule-fail-days = 12 expire-fail-days = +8 [lenny222testing_i386/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = lenny222testing_i386/main lenny222testing_i386/non-free arch = i386 setarch = linux32 area = contrib distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [lenny222testing_i386/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-lenny)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = lenny222testing_i386/main lenny222testing_i386/contrib arch = i386 setarch = linux32 area = non-free distro = lenny upgrade-test-distros = lenny squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze2lts2wheezy/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main distro = squeeze-lts arch = i386 setarch = linux32 area = main components = main upgrade-test-distros = squeeze squeeze-lts wheezy-lts reschedule-old-count = 0 [squeeze2lts2wheezy/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main squeeze2lts2wheezy/main squeeze2lts2wheezy/non-free distro = squeeze-lts arch = i386 setarch = linux32 area = contrib upgrade-test-distros = squeeze squeeze-lts wheezy-lts reschedule-old-count = 0 [squeeze2lts2wheezy/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main squeeze2lts2wheezy/main squeeze2lts2wheezy/contrib distro = squeeze-lts arch = i386 setarch = linux32 area = non-free upgrade-test-distros = squeeze squeeze-lts wheezy-lts reschedule-old-count = 0 [squeeze2wheezy/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s arch = amd64 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy/main squeeze2wheezy/non-free arch = amd64 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy/main squeeze2wheezy/contrib arch = amd64 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-lts/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s arch = amd64 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy-lts reschedule-old-count = 0 [squeeze2wheezy-lts/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-lts/main squeeze2wheezy-lts/non-free arch = amd64 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy-lts reschedule-old-count = 0 [squeeze2wheezy-lts/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-lts/main squeeze2wheezy-lts/contrib arch = amd64 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy-lts reschedule-old-count = 0 [squeeze2wheezy_i386/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s arch = i386 setarch = linux32 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy_i386/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy_i386/main squeeze2wheezy_i386/non-free arch = i386 setarch = linux32 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy_i386/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy_i386/main squeeze2wheezy_i386/contrib arch = i386 setarch = linux32 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd/main] precedence = 1000 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s arch = amd64 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd/contrib] precedence = 1000 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-rcmd/main squeeze2wheezy-rcmd/non-free arch = amd64 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd/non-free] precedence = 1000 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-rcmd/main squeeze2wheezy-rcmd/contrib arch = amd64 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd_i386/main] precedence = 1000 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s arch = i386 setarch = linux32 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd_i386/contrib] precedence = 1000 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-rcmd_i386/main squeeze2wheezy-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2wheezy-rcmd_i386/non-free] precedence = 1000 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze2wheezy-rcmd_i386/main squeeze2wheezy-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy reschedule-old-count = 0 [squeeze2bpo2wheezy/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main distro = squeeze-backports arch = amd64 area = main components = main upgrade-test-distros = squeeze squeeze-backports wheezy-lts reschedule-old-count = 0 [squeeze2bpo2wheezy/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main squeeze2bpo2wheezy/main squeeze2bpo2wheezy/non-free distro = squeeze-backports arch = amd64 area = contrib upgrade-test-distros = squeeze squeeze-backports wheezy-lts reschedule-old-count = 0 [squeeze2bpo2wheezy/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-wheezy)s depends-sections = squeeze/main squeeze2bpo2wheezy/main squeeze2bpo2wheezy/contrib distro = squeeze-backports arch = amd64 area = non-free upgrade-test-distros = squeeze squeeze-backports wheezy-lts reschedule-old-count = 0 [squeeze222testing/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = amd64 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze222testing/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = squeeze222testing/main squeeze222testing/non-free arch = amd64 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze222testing/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = squeeze222testing/main squeeze222testing/contrib arch = amd64 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze222testing_i386/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = i386 setarch = linux32 area = main components = main distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze222testing_i386/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = squeeze222testing_i386/main squeeze222testing_i386/non-free arch = i386 setarch = linux32 area = contrib distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [squeeze222testing_i386/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-squeeze)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = squeeze222testing_i386/main squeeze222testing_i386/contrib arch = i386 setarch = linux32 area = non-free distro = squeeze upgrade-test-distros = squeeze wheezy jessie stretch buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy2lts/main] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s arch = amd64 area = main components = main distro = wheezy upgrade-test-distros = wheezy wheezy-lts reschedule-old-count = 0 [wheezy2lts/contrib] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy2lts/main wheezy2lts/non-free arch = amd64 area = contrib distro = wheezy upgrade-test-distros = wheezy wheezy-lts reschedule-old-count = 0 [wheezy2lts/non-free] precedence = 1000 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-wheezy)s depends-sections = wheezy2lts/main wheezy2lts/contrib arch = amd64 area = non-free distro = wheezy upgrade-test-distros = wheezy wheezy-lts reschedule-old-count = 0 [wheezy2jessie/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s arch = amd64 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie/main wheezy2jessie/non-free arch = amd64 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie/main wheezy2jessie/contrib arch = amd64 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie_i386/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s arch = i386 setarch = linux32 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie_i386/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie_i386/main wheezy2jessie_i386/non-free arch = i386 setarch = linux32 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie_i386/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie_i386/main wheezy2jessie_i386/contrib arch = i386 setarch = linux32 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd/main] precedence = 80 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper arch = amd64 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd/contrib] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper depends-sections = wheezy2jessie-rcmd/main wheezy2jessie-rcmd/non-free arch = amd64 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd/non-free] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper depends-sections = wheezy2jessie-rcmd/main wheezy2jessie-rcmd/contrib arch = amd64 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd_i386/main] precedence = 80 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper arch = i386 setarch = linux32 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd_i386/contrib] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper depends-sections = wheezy2jessie-rcmd_i386/main wheezy2jessie-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-rcmd_i386/non-free] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s -i /usr/lib/dbus-1.0/dbus-daemon-launch-helper depends-sections = wheezy2jessie-rcmd_i386/main wheezy2jessie-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-sysv/main] precedence = 80 description = + Keeping sysvinit instead of switching to systemd (if possible). piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-sysvinit arch = amd64 area = main components = main distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-sysv/contrib] precedence = 95 description = + Keeping sysvinit instead of switching to systemd (if possible). piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-sysvinit depends-sections = wheezy2jessie-sysv/main wheezy2jessie-sysv/non-free arch = amd64 area = contrib distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-sysv/non-free] precedence = 95 description = + Keeping sysvinit instead of switching to systemd (if possible). piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-sysvinit depends-sections = wheezy2jessie-sysv/main wheezy2jessie-sysv/contrib arch = amd64 area = non-free distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-apt1st/main] precedence = 80 description = + Upgrading apt before the rest of the system. piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-apt-first arch = amd64 area = main components = main distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-apt1st/contrib] precedence = 95 description = + Upgrading apt before the rest of the system. piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-apt-first depends-sections = wheezy2jessie-apt1st/main wheezy2jessie-apt1st/non-free arch = amd64 area = contrib distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-apt1st/non-free] precedence = 95 description = + Upgrading apt before the rest of the system. piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s --scriptsdir /etc/piuparts/scripts-apt-first depends-sections = wheezy2jessie-apt1st/main wheezy2jessie-apt1st/contrib arch = amd64 area = non-free distro = wheezy upgrade-test-distros = wheezy jessie reschedule-old-count = 0 [wheezy2jessie-iri/main] precedence = 90 description = Debian %(area)s: package installation and removal in wheezy, then upgrade to jessie, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s arch = amd64 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-iri/contrib] precedence = 95 description = Debian %(area)s: package installation and removal in wheezy, then upgrade to jessie, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie-iri/main wheezy2jessie-iri/non-free arch = amd64 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2jessie-iri/non-free] precedence = 95 description = Debian %(area)s: package installation and removal in wheezy, then upgrade to jessie, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy2jessie-iri/main wheezy2jessie-iri/contrib arch = amd64 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts [wheezy2bpo2jessie/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy/main distro = wheezy-backports arch = amd64 area = main components = main upgrade-test-distros = wheezy-lts wheezy-backports jessie-lts [wheezy2bpo2jessie/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy/main wheezy/non-free wheezy2bpo2jessie/main wheezy2bpo2jessie/non-free distro = wheezy-backports arch = amd64 area = contrib upgrade-test-distros = wheezy-lts wheezy-backports jessie-lts [wheezy2bpo2jessie/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-jessie)s depends-sections = wheezy/main wheezy2bpo2jessie/main wheezy2bpo2jessie/contrib distro = wheezy-backports arch = amd64 area = non-free upgrade-test-distros = wheezy-lts wheezy-backports jessie-lts [wheezy222testing/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = amd64 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy222testing/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = wheezy222testing/main wheezy222testing/non-free arch = amd64 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy222testing/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = wheezy222testing/main wheezy222testing/contrib arch = amd64 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy222testing_i386/main] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = i386 setarch = linux32 area = main components = main distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy222testing_i386/contrib] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = wheezy222testing_i386/main wheezy222testing_i386/non-free arch = i386 setarch = linux32 area = contrib distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [wheezy222testing_i386/non-free] precedence = 150 piuparts-flags = %(flags-default)s %(flags-start-wheezy)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = wheezy222testing_i386/main wheezy222testing_i386/contrib arch = i386 setarch = linux32 area = non-free distro = wheezy-lts upgrade-test-distros = wheezy-lts jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie2lts/main] precedence = 60 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s arch = amd64 area = main components = main distro = jessie upgrade-test-distros = jessie jessie-lts [jessie2lts/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie2lts/main jessie2lts/non-free arch = amd64 area = contrib distro = jessie upgrade-test-distros = jessie jessie-lts [jessie2lts/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-jessie)s depends-sections = jessie2lts/main jessie2lts/contrib arch = amd64 area = non-free distro = jessie upgrade-test-distros = jessie jessie-lts [jessie2stretch/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch/main jessie2stretch/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch/main jessie2stretch/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie3stretch/main] precedence = 80 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch/contrib] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch/main jessie3stretch/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch/non-free] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch/main jessie3stretch/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie2stretch_i386/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = i386 setarch = linux32 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch_i386/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch_i386/main jessie2stretch_i386/non-free arch = i386 setarch = linux32 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch_i386/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch_i386/main jessie2stretch_i386/contrib arch = i386 setarch = linux32 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie3stretch_i386/main] precedence = 80 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = i386 setarch = linux32 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch_i386/contrib] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch_i386/main jessie3stretch_i386/non-free arch = i386 setarch = linux32 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch_i386/non-free] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch_i386/main jessie3stretch_i386/contrib arch = i386 setarch = linux32 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie2stretch-rcmd/main] precedence = 80 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-rcmd/contrib] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-rcmd/main jessie2stretch-rcmd/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-rcmd/non-free] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-rcmd/main jessie2stretch-rcmd/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie3stretch-rcmd/main] precedence = 80 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch-rcmd/contrib] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch-rcmd/main jessie3stretch-rcmd/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch-rcmd/non-free] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch-rcmd/main jessie3stretch-rcmd/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie2stretch-rcmd_i386/main] precedence = 80 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = i386 setarch = linux32 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-rcmd_i386/contrib] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-rcmd_i386/main jessie2stretch-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-rcmd_i386/non-free] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-rcmd_i386/main jessie2stretch-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie3stretch-rcmd_i386/main] precedence = 80 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = i386 setarch = linux32 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch-rcmd_i386/contrib] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch-rcmd_i386/main jessie3stretch-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie3stretch-rcmd_i386/non-free] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie3stretch-rcmd_i386/main jessie3stretch-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [jessie2stretch-iri/main] precedence = 90 description = Debian %(area)s: package installation and removal in jessie, then upgrade to stretch, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-iri/contrib] precedence = 95 description = Debian %(area)s: package installation and removal in jessie, then upgrade to stretch, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-iri/main jessie2stretch-iri/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2stretch-iri/non-free] precedence = 95 description = Debian %(area)s: package installation and removal in jessie, then upgrade to stretch, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie2stretch-iri/main jessie2stretch-iri/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next [jessie2bpo2stretch/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie-lts/main distro = jessie-backports arch = amd64 area = main components = main upgrade-test-distros = jessie-lts jessie-backports stretch-next [jessie2bpo2stretch/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie-lts/main jessie-lts/contrib jessie-lts/non-free jessie2bpo2stretch/main jessie2bpo2stretch/non-free distro = jessie-backports arch = amd64 area = contrib upgrade-test-distros = jessie-lts jessie-backports stretch-next [jessie2bpo2stretch/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-stretch)s depends-sections = jessie-lts/main jessie-lts/contrib jessie-lts/non-free jessie2bpo2stretch/main jessie2bpo2stretch/contrib distro = jessie-backports arch = amd64 area = non-free upgrade-test-distros = jessie-lts jessie-backports stretch-next [jessie222testing/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = amd64 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie222testing/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = jessie222testing/main jessie222testing/non-free arch = amd64 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie222testing/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = jessie222testing/main jessie222testing/contrib arch = amd64 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie222testing_i386/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade arch = i386 setarch = linux32 area = main components = main distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie222testing_i386/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = jessie222testing_i386/main jessie222testing_i386/non-free arch = i386 setarch = linux32 area = contrib distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [jessie222testing_i386/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-jessie)s %(flags-end-testing)s --scriptsdir /etc/piuparts/scripts-multi-distro-upgrade depends-sections = jessie222testing_i386/main jessie222testing_i386/contrib arch = i386 setarch = linux32 area = non-free distro = jessie-lts upgrade-test-distros = jessie-lts stretch-next buster testing max-reserved = 25 reschedule-old-count = 100 [stretch2next/main] precedence = 60 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch stretch-next [stretch2next/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch2next/main stretch2next/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch stretch-next [stretch2next/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-stretch)s depends-sections = stretch2next/main stretch2next/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch stretch-next [stretch2buster/main] precedence = 50 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster/main stretch2buster/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster/main stretch2buster/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch3buster/main] precedence = 50 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster/contrib] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster/main stretch3buster/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster/non-free] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster/main stretch3buster/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2buster_i386/main] precedence = 50 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = i386 setarch = linux32 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster_i386/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster_i386/main stretch2buster_i386/non-free arch = i386 setarch = linux32 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster_i386/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster_i386/main stretch2buster_i386/contrib arch = i386 setarch = linux32 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch3buster_i386/main] precedence = 50 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = i386 setarch = linux32 area = main components = main distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster_i386/contrib] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster_i386/main stretch3buster_i386/non-free arch = i386 setarch = linux32 area = contrib distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster_i386/non-free] precedence = 95 description = + Testing two-stage upgrades. piuparts-flags = --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster_i386/main stretch3buster_i386/contrib arch = i386 setarch = linux32 area = non-free distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2buster-rcmd/main] precedence = 50 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster-rcmd/contrib] precedence = 95 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-rcmd/main stretch2buster-rcmd/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster-rcmd/non-free] precedence = 95 piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-rcmd/main stretch2buster-rcmd/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch3buster-rcmd/main] precedence = 50 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster-rcmd/contrib] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster-rcmd/main stretch3buster-rcmd/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster-rcmd/non-free] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster-rcmd/main stretch3buster-rcmd/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2buster-rcmd_i386/main] precedence = 50 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = i386 setarch = linux32 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster-rcmd_i386/contrib] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-rcmd_i386/main stretch2buster-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster-rcmd_i386/non-free] precedence = 95 description = + Testing with --install-recommends. piuparts-flags = --install-recommends %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-rcmd_i386/main stretch2buster-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch3buster-rcmd_i386/main] precedence = 50 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = i386 setarch = linux32 area = main components = main distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster-rcmd_i386/contrib] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster-rcmd_i386/main stretch3buster-rcmd_i386/non-free arch = i386 setarch = linux32 area = contrib distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch3buster-rcmd_i386/non-free] precedence = 95 description = + Testing two-stage upgrades with --install-recommends. piuparts-flags = --install-recommends --upgrade-before-dist-upgrade %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch3buster-rcmd_i386/main stretch3buster-rcmd_i386/contrib arch = i386 setarch = linux32 area = non-free distro = stretch upgrade-test-distros = stretch buster exclude-known-problems = packages_have_been_kept_back_error.conf packages_have_been_kept_back_issue.conf [stretch2buster-merged-usr/main] precedence = 90 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster-merged-usr/contrib] precedence = 95 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-merged-usr/main stretch2buster-merged-usr/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster-merged-usr/non-free] precedence = 95 description = + In --merged-usr environment. piuparts-flags = %(flags-merged-usr)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-merged-usr/main stretch2buster-merged-usr/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch2buster-iri/main] precedence = 90 description = Debian %(area)s: package installation and removal in stretch, then upgrade to buster, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stretch upgrade-test-distros = stretch buster [stretch2buster-iri/contrib] precedence = 95 description = Debian %(area)s: package installation and removal in stretch, then upgrade to buster, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-iri/main stretch2buster-iri/non-free arch = amd64 area = contrib distro = stretch upgrade-test-distros = stretch buster [stretch2buster-iri/non-free] precedence = 95 description = Debian %(area)s: package installation and removal in stretch, then upgrade to buster, installation, removal, and purge test. piuparts-flags = --install-remove-install %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stretch2buster-iri/main stretch2buster-iri/contrib arch = amd64 area = non-free distro = stretch upgrade-test-distros = stretch buster [stretch2bpo2buster/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s depends-sections = stretch/main distro = stretch-backports arch = amd64 area = main components = main upgrade-test-distros = stretch stretch-backports buster [stretch2bpo2buster/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s depends-sections = stretch/main stretch/contrib stretch/non-free stretch2bpo2buster/main stretch2bpo2buster/non-free distro = stretch-backports arch = amd64 area = contrib upgrade-test-distros = stretch stretch-backports buster [stretch2bpo2buster/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stretch)s %(flags-end-buster)s depends-sections = stretch/main stretch/contrib stretch/non-free stretch2bpo2buster/main stretch2bpo2buster/contrib distro = stretch-backports arch = amd64 area = non-free upgrade-test-distros = stretch stretch-backports buster [stable2sid/main] precedence = 70 piuparts-flags = %(flags-default)s %(flags-start-stable)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver arch = amd64 area = main components = main distro = stable upgrade-test-distros = stable sid [stable2sid/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stable)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stable2sid/main stable2sid/non-free arch = amd64 area = contrib distro = stable upgrade-test-distros = stable sid [stable2sid/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stable)s --scriptsdir /etc/piuparts/scripts-debug-problemresolver depends-sections = stable2sid/main stable2sid/contrib arch = amd64 area = non-free distro = stable upgrade-test-distros = stable sid [stable2testing2sid/main] precedence = 80 piuparts-flags = %(flags-default)s %(flags-start-stable)s arch = amd64 area = main components = main distro = stable upgrade-test-distros = stable testing sid reschedule-old-count = 0 [stable2testing2sid/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stable)s depends-sections = stable2testing2sid/main stable2testing2sid/non-free arch = amd64 area = contrib distro = stable upgrade-test-distros = stable testing sid reschedule-old-count = 0 [stable2testing2sid/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-stable)s depends-sections = stable2testing2sid/main stable2testing2sid/contrib arch = amd64 area = non-free distro = stable upgrade-test-distros = stable testing sid reschedule-old-count = 0 [testing2sid/main] precedence = 60 piuparts-flags = %(flags-default)s %(flags-start-testing)s arch = amd64 area = main components = main distro = testing upgrade-test-distros = testing sid reschedule-old-count = 0 [testing2sid/contrib] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-testing)s depends-sections = testing2sid/main testing2sid/non-free arch = amd64 area = contrib distro = testing upgrade-test-distros = testing sid reschedule-old-count = 0 [testing2sid/non-free] precedence = 95 piuparts-flags = %(flags-default)s %(flags-start-testing)s depends-sections = testing2sid/main testing2sid/contrib arch = amd64 area = non-free distro = testing upgrade-test-distros = testing sid reschedule-old-count = 0 [sid2experimental/main] precedence = 70 piuparts-flags = %(flags-default)s depends-sections = sid/main arch = amd64 area = main components = main upgrade-test-distros = sid experimental [sid2experimental/contrib] precedence = 95 piuparts-flags = %(flags-default)s depends-sections = sid2experimental/main sid/main sid/contrib sid/non-free arch = amd64 area = contrib upgrade-test-distros = sid experimental [sid2experimental/non-free] precedence = 95 piuparts-flags = %(flags-default)s depends-sections = sid2experimental/main sid/main sid/contrib sid/non-free arch = amd64 area = non-free upgrade-test-distros = sid experimental piuparts-1.1.1/known_problems/0000755000000000000000000000000013463366315013320 5ustar piuparts-1.1.1/known_problems/alternatives_after_purge_error.conf0000644000000000000000000000135413351442133022454 0ustar # # detect packages with unowned files in /etc/alternatives after purge (policy 6.8) # PATTERN='/etc/alternatives/.*not owned' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with leftover alternatives after purge' HELPTEXT='

Packages with unowned files in /etc/alternatives after purge (violating policy 6.8) see https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging

Alternatives are usually registered with update-alternatives in postinst configure and need to be unregistered again in prerm remove.

' piuparts-1.1.1/known_problems/alternatives_after_purge_issue.conf0000644000000000000000000000133413351442133022451 0ustar # # detect packages with unowned files in /etc/alternatives after purge (policy 6.8) # PATTERN='/etc/alternatives/.*not owned' WHERE='pass' ISSUE=1 HEADER='Packages with leftover alternatives after purge' HELPTEXT='

Packages with unowned files in /etc/alternatives after purge (violating policy 6.8) see https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging

Alternatives are usually registered with update-alternatives in postinst configure and need to be unregistered again in prerm remove.

' piuparts-1.1.1/known_problems/bin_or_sbin_binary_requires_usr_lib_library_inadequate_issue.conf0000644000000000000000000000076013266762112030607 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* bin-or-sbin-binary-requires-usr-lib-library' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'bin-or-sbin-binary-requires-usr-lib-library' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'bin-or-sbin-binary-requires-usr-lib-library' which indicates a bug.

" piuparts-1.1.1/known_problems/boring_broken_symlink_inadequate_issue.conf0000644000000000000000000000075413266762112024167 0ustar # # detect packages which have the string "Running adequate resulted in less interesting tags found" in their logs # PATTERN='(FAIL|WARN): Running adequate resulted in less interesting tags found: .* broken-symlink' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'broken-symlink' by adequate" HELPTEXT="

Some issues detected by adequate are also detected by piuparts, 'broken-symlink' is one of them. " piuparts-1.1.1/known_problems/boring_obsolete_conffile_inadequate_issue.conf0000644000000000000000000000076513266762112024624 0ustar # # detect packages which have the string "Running adequate resulted in less interesting tags found" in their logs # PATTERN='(FAIL|WARN): Running adequate resulted in less interesting tags found: .* obsolete-conffile' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'obsolete-conffile' by adequate" HELPTEXT="

Some issues detected by adequate are also detected by piuparts, 'obsolete-conffile' is one of them. " piuparts-1.1.1/known_problems/broken_binfmt_detector_inadequate_issue.conf0000644000000000000000000000076013266762112024306 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* broken-binfmt-detector' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'broken-binfmt-detector' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'broken-binfmt-detector' which indicates a bug: The detector registered with update-binfmts(8) does not exist.

" piuparts-1.1.1/known_problems/broken_binfmt_interpreter_inadequate_issue.conf0000644000000000000000000000077413266762112025045 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* broken-binfmt-interpreter' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'broken-binfmt-interpreter' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'broken-binfmt-interpreter' which indicates a bug: The interpreter registered with update-binfmts(8) does not exist.

" piuparts-1.1.1/known_problems/broken_symlinks_error.conf0000644000000000000000000000103613266762112020605 0ustar # # detect packages which have the string "Broken symlinks" in their logs # PATTERN='(WARN|FAIL): Broken symlink' WHERE='fail bugged affected' ISSUE=1 HEADER='Packages which have logs with the string "Broken symlinks"' HELPTEXT='

This is clearly an error, but as there are too many of this kind, piuparts can be configured to not fail if it detects broken symlinks. Another option is not to test for broken symlinks. See the piuparts manpage for details.

' piuparts-1.1.1/known_problems/broken_symlinks_issue.conf0000644000000000000000000000101613266762112020602 0ustar # # detect packages which have the string "Broken symlinks" in their logs # PATTERN='(WARN|FAIL): Broken symlink' WHERE='pass' ISSUE=1 HEADER='Packages which have logs with the string "Broken symlinks"' HELPTEXT='

This is clearly an error, but as there are too many of this kind, piuparts can be configured to not fail if it detects broken symlinks. Another option is not to test for broken symlinks. See the piuparts manpage for details.

' piuparts-1.1.1/known_problems/command_not_found_error.conf0000644000000000000000000000210213351442133021051 0ustar # # detect packages with miss a depends or use non-essential in purge # PATTERN='command not found|: not found' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs with the string "command not found"' HELPTEXT='

From the third paragraph about the meaning of the depends field in https://www.debian.org/doc/debian-policy/ch-relationships.html#binary-dependencies-depends-recommends-suggests-enhances-pre-depends 

The Depends field should also be used if the postinst, prerm or postrm scripts
require the package to be present in order to run. __Note, however, that the
postrm cannot rely on any non-essential packages to be present during the
purge phase__.

NOTE: it has not been verified that this error really caused the package to fail the piuparts test, but it did fail.
There are also successful logs with "command not found" (though not listed below).

' piuparts-1.1.1/known_problems/command_not_found_issue.conf0000644000000000000000000000163313351442133021060 0ustar # # detect packages which passed the piuparts test but have the string "command not found" in their logs # PATTERN='command not found|: not found' WHERE='pass' ISSUE=1 HEADER='Packages which passed the piuparts test but have logs with the string "command not found"' HELPTEXT='

From the third paragraph about the meaning of the depends field in https://www.debian.org/doc/debian-policy/ch-relationships.html#binary-dependencies-depends-recommends-suggests-enhances-pre-depends: 

The Depends field should also be used if the postinst, prerm or postrm scripts
require the package to be present in order to run. __Note, however, that the
postrm cannot rely on any non-essential packages to be present during the
purge phase__.
' piuparts-1.1.1/known_problems/conffile_prompt_error.conf0000644000000000000000000000234113351442133020553 0ustar # # detect packages with failed because they prompt due to a modified conffile # PATTERN='(EOF|end of file) on stdin at conffile prompt' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs with the string "EOF on stdin at conffile prompt"' HELPTEXT='

The piuparts upgrade test failed because dpkg detected a conffile as being modified and then prompted the user for an action. As there is no user input, this fails. But this is not the real problem, the real problem is that this prompt shows up in the first place, as there was nobody modifying this conffile at all, the package has just been installed and upgraded...

This is explained in detail in policy 10.7.3 at https://www.debian.org/doc/debian-policy/ch-files.html#behavior which says "[These scripts handling conffiles] must not ask unnecessary questions (particularly during upgrades), and must otherwise be good citizens."

According to the thread started at 200908191215.05079.holger@layer-acht.org these bugs are to be filed with severity serious.

' piuparts-1.1.1/known_problems/cron_error_after_removal_error.conf0000644000000000000000000000147313351442133022452 0ustar # # detect packages with cron errors after the package has been removed # PATTERN='(FAIL: Cron file .* has output with package removed|ERROR: Command failed.*./etc/cron\..*/.*.])' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because a cron file has output and/or exits with error after the package has been removed' HELPTEXT='

From https://www.debian.org/doc/debian-policy/ch-opersys.html#cron-jobs 

The scripts or crontab entries in these directories should check if all
necessary programs are installed before they try to execute them. Otherwise,
problems will arise when a package was removed but not purged since
configuration files are kept on the system in this situation.
' piuparts-1.1.1/known_problems/db_setup_error.conf0000644000000000000000000000161613266771660017215 0ustar # # detect packages with failed to install due to problems configuring the db - see #595652 # PATTERN='(warning: database package not installed|dbconfig-common: .* configure: (aborted|noninteractive fail|ignoring errors from here forwards)|updating database schema for .*...command failed with code 0|(postgres|psql): could not connect to server: No such file or directory|DBI connect.* failed: could not connect to server|pg_pconnect\(\): Unable to connect to PostgreSQL server|Non-default port in /etc/postgresql/.*/main/postgresql.conf|Unable to connect to MySQL server|unable to connect to mysql server)' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because installation failed because no database could be connected.' HELPTEXT='

Just how buggy these packages really are is discussed in #595652 at the moment.

' piuparts-1.1.1/known_problems/db_setup_issue.conf0000644000000000000000000000157613266771660017221 0ustar # # detect packages with failed to install due to problems configuring the db - see #595652 # PATTERN='(warning: database package not installed|dbconfig-common: .* configure: (aborted|noninteractive fail|ignoring errors from here forwards)|updating database schema for .*...command failed with code 0|(postgres|psql): could not connect to server: No such file or directory|DBI connect.* failed: could not connect to server|pg_pconnect\(\): Unable to connect to PostgreSQL server|Non-default port in /etc/postgresql/.*/main/postgresql.conf|Unable to connect to MySQL server|unable to connect to mysql server)' WHERE='pass' ISSUE=1 HEADER='Packages with failed logs because installation failed because no database could be connected.' HELPTEXT='

Just how buggy these packages really are is discussed in #595652 at the moment.

' piuparts-1.1.1/known_problems/debsums_mismatch_error.conf0000644000000000000000000000052313266771660020733 0ustar # # detect packages which modify shipped files # PATTERN='FAIL: debsums reports modifications inside the chroot|MISSING OBJECT' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with modified files before removal' HELPTEXT='

Modifying conffiles is forbidden by Policy 10.7.3. Modifying other shipped files is a stupid idea.

' piuparts-1.1.1/known_problems/debsums_mismatch_issue.conf0000644000000000000000000000050313266771660020730 0ustar # # detect packages which modify shipped files # PATTERN='FAIL: debsums reports modifications inside the chroot|MISSING OBJECT' WHERE='pass' ISSUE=1 HEADER='Packages with modified files before removal' HELPTEXT='

Modifying conffiles is forbidden by Policy 10.7.3. Modifying other shipped files is a stupid idea.

' piuparts-1.1.1/known_problems/dependency_error.conf0000644000000000000000000000103513266762112017511 0ustar # # detect packages with unsatisfied dependencies # PATTERN='E: Broken packages|E: Unable to correct problems, you have held broken packages.|E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because of unsatisfied dependencies' HELPTEXT='

Usually this is caused by some unsatisfied (versioned) Depends/Conflicts/Replaces. These packages will be automatically rescheduled for testing seven days after they failed.

' piuparts-1.1.1/known_problems/disappeared_files_after_purge_error.conf0000644000000000000000000000043513266762112023424 0ustar # # detect packages with disappeared files after purge # PATTERN='FAIL: After purging files have disappeared:' EXCLUDE_PATTERN=',|usr/local' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with files which disappeared after purge' HELPTEXT='

This is obviously broken.

' piuparts-1.1.1/known_problems/diversion_error.conf0000644000000000000000000000051613266762112017400 0ustar # # detect packages which mishandle diversions # PATTERN='ERROR: FAIL: (Existing|Installed) diversions' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages that leave a modified diversion state' HELPTEXT='

Packages need to remove all the diversions they introduced and may not modify diversions setup by other packages.

' piuparts-1.1.1/known_problems/files_in_usr_local_error.conf0000644000000000000000000000072613351442133021225 0ustar # # detect packages which leave stuff in /usr/local (see policy 9.1.2) # PATTERN='usr/local.+not owned' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with files and/or directories in /usr/local/ after purge' HELPTEXT='

This is a violation of policy 9.1.2: see https://www.debian.org/doc/debian-policy/ch-opersys.html#site-specific-programs.

' piuparts-1.1.1/known_problems/immediate_configuration_error.conf0000644000000000000000000000055713266762112022270 0ustar # # detect packages which failed because apt could not perform immediate configuration # PATTERN='E: Could not perform immediate configuration on' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because apt could not perform immediate configuration' HELPTEXT='

This is a bug in apt, but it has to be worked around in some packages.

' piuparts-1.1.1/known_problems/inadequate_exit_issue.conf0000644000000000000000000000074513266762112020552 0ustar # # detect packages which have the string "Exit code from adequate was" in their logs # PATTERN='(WARN|FAIL): Exit code from adequate was' WHERE='pass fail bugged affected' ISSUE=0 HEADER='Packages which have logs with the string "Exit code from adequate was"' HELPTEXT='

Running adequate resulted in an exit code not equal zero, which indicates a severe problem with adequate. Please investigate and report.

' piuparts-1.1.1/known_problems/incompatible_licenses_inadequate_issue.conf0000644000000000000000000000065613266762112024135 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* incompatible-licenses' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'incompatible-licenses' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'incompatible-licenses' which indicates a bug.

" piuparts-1.1.1/known_problems/initdscript_lsb_header_issue.conf0000644000000000000000000000261513266762112022103 0ustar # # detect packages with an update-rc.d warning # PATTERN='update-rc.d: warning.*do not match LSB|service.*already provided' WHERE='fail bugged affected pass' ISSUE=1 HEADER='Packages with logs with the string "update-rc.d: warning.*do not match LSB"' HELPTEXT='

Some packages have inconsistency between the init.d script headers used with dependency based boot sequencing and the runlevels specified on the update-rc.d command line and used by the legacy boot ordering. Such inconsistency is most likely a bug in the package, as the two ways of ordering init.d scripts should enable and disable the scripts in the same runlevels while Debian migrate to dependency based boot sequencing.

Such inconsinstency is reported like this when a postinst script call update-rc.d 

  update-rc.d: warning: initdscript start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (S)
  update-rc.d: warning: initdscript stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (none)

Such reports are most likely bugs in the package calling update-rc.d, and should be reported and fixed in the individual packages.

See the paragraph "How to solve migration problems" at https://wiki.debian.org/LSBInitScripts/DependencyBasedBoot for information how to fix these issues.

' piuparts-1.1.1/known_problems/insserv_error.conf0000644000000000000000000000241013266762112017062 0ustar # # detect packages with fail due to insserv error # PATTERN='insserv: exiting now|service.*already provided' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs with the string "insserv: exiting now"' HELPTEXT='

Some packages fail to install because their init.d scripts have headers with bugs. There are several classes of bugs.

Some packages fail to install because the init.d script have conflicting provide. This is normally reported like this: 

  insserv: script clvm: service lvm already provided!
  insserv: exiting now!

Other packages fail to install because their dependencies are missing. This is normally reported like this: 

  insserv: Service portmap has to be enabled to start service quotarpc
  insserv: exiting now!

Last, some packages introduce dependency loops, this is normally reported like this: 

  insserv: There is a loop between service script1 and script2 if started
  insserv: exiting without changing boot order!

See the paragraph "How to solve migration problems" at https://wiki.debian.org/LSBInitScripts/DependencyBasedBoot for information how to fix these issues.

' piuparts-1.1.1/known_problems/installs_over_symlink_error.conf0000644000000000000000000000123413403304304022012 0ustar # # detect packages which have the string "dirname part contains a symlink" in their logs # PATTERN='silently overwrites files via directory symlinks|installs objects over existing directory symlinks|dirname part contains a symlink' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages that install something over existing symlinks' HELPTEXT='

Installing anything over a symlink opens a can of worms - this causes problems on upgrades while switching between directory and symlink or if the symlink is ever changed.
Piuparts looks at all $pathname known to dpkg and checks for 

$(dirname $pathname) != $(readlink $(dirname $pathname))

' piuparts-1.1.1/known_problems/installs_over_symlink_issue.conf0000644000000000000000000000121413403304304022007 0ustar # # detect packages which have the string "dirname part contains a symlink" in their logs # PATTERN='silently overwrites files via directory symlinks|installs objects over existing directory symlinks|dirname part contains a symlink' WHERE='pass' ISSUE=1 HEADER='Packages that install something over existing symlinks' HELPTEXT='

Installing anything over a symlink opens a can of worms - this causes problems on upgrades while switching between directory and symlink or if the symlink is ever changed.
Piuparts looks at all $pathname known to dpkg and checks for 

$(dirname $pathname) != $(readlink $(dirname $pathname))

' piuparts-1.1.1/known_problems/ldd_inadequate_issue.conf0000644000000000000000000000062013266771660020344 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* ldd-failure' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'ldd-failure' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'ldd-failure' which indicates a bug.

" piuparts-1.1.1/known_problems/library_not_found_inadequate_issue.conf0000644000000000000000000000064213266762112023314 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* library-not-found' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'library-not-found' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'library-not-found' which indicates a bug.

" piuparts-1.1.1/known_problems/logrotate_error_after_removal_error.conf0000644000000000000000000000076513363064533023523 0ustar # # detect packages with logrotate errors after the package has been removed # PATTERN='(FAIL: Logrotate file .* has output with package removed|ERROR: Command failed.*'/etc/logrotate\.d/.*'])' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because a logrotate script has output and/or exits with error after the package has been removed' HELPTEXT='

The logrotate configuration causes output and/or exits with error when the package has been removed but not purged.

' piuparts-1.1.1/known_problems/maintainer_script_issue.conf0000644000000000000000000000055013266771660021116 0ustar # # detect packages in successfully-tested state that had maintainer script failures # PATTERN='(pre|post)-(installation|removal) script( subprocess)? returned error' WHERE='pass' ISSUE=0 HEADER='Packages in state successfully-tested but logfile contains a maintainer script failure' HELPTEXT='

This is a somewhat unclassified issue at the moment.

' piuparts-1.1.1/known_problems/missing-symbol-version-information_inadequate_issue.conf0000644000000000000000000000072513266762112026561 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* missing-symbol-version-information' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'missing-symbol-version-information' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'missing-symbol-version-information' which indicates a bug.

" piuparts-1.1.1/known_problems/missing_alternative_inadequate_issue.conf0000644000000000000000000000117413266771660023655 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* missing-alternative' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'missing-alternative' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'missing-alternative' which indicates a bug similar to this situation: a package is a provider of the virtual package 'x-terminal-emulator', but it doesn't register itself as an alternative for '/usr/bin/x-terminal-emulator'. See policy 11.8.3 and 11.8.4.

" piuparts-1.1.1/known_problems/missing_copyright_file_inadequate_issue.conf0000644000000000000000000000070213266762112024332 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .*missing-copyright-file|MISSING COPYRIGHT FILE' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'missing-copyright-file' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'missing-copyright-file' which indicates a bug. " piuparts-1.1.1/known_problems/missing_md5sums_error.conf0000644000000000000000000000040713266762112020523 0ustar # # detect packages that ship files without md5sums # PATTERN='(FILE WITHOUT MD5SUM|MD5SUM FILE NOT FOUND)' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages containing files without md5sum' HELPTEXT='

Packages that ship files that have no md5sum.

' piuparts-1.1.1/known_problems/missing_md5sums_issue.conf0000644000000000000000000000036713266762112020527 0ustar # # detect packages that ship files without md5sums # PATTERN='(FILE WITHOUT MD5SUM|MD5SUM FILE NOT FOUND)' WHERE='pass' ISSUE=1 HEADER='Packages containing files without md5sum' HELPTEXT='

Packages that ship files that have no md5sum.

' piuparts-1.1.1/known_problems/missing_pkgconfig-dependency_issue.conf0000644000000000000000000000067513266771660023227 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .*missing-pkgconfig-dependency' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'missing-pkgconfig-dependency' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'missing-pkgconfig-dependency' which indicates a bug. " piuparts-1.1.1/known_problems/modified_files_after_purge_error.conf0000644000000000000000000000042313266762112022720 0ustar # # detect packages with modified files after purge # PATTERN='FAIL: After purging files have been modified:' EXCLUDE_PATTERN=',|usr/local' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with modified files after purge' HELPTEXT='

This is obviously broken.

' piuparts-1.1.1/known_problems/module_build_error_error.conf0000644000000000000000000000052213463366315021254 0ustar # # detect dkms module source packages that fail to build for a current kernel # PATTERN='Error! Bad return status for module build on kernel' WHERE='fail bugged affected' ISSUE=0 HEADER="DKMS-packages failing to build a module for a default kernel" HELPTEXT="

Kernel module source may not be compatible with the latest kernel.

" piuparts-1.1.1/known_problems/module_build_error_issue.conf0000644000000000000000000000050213463366315021251 0ustar # # detect dkms module source packages that fail to build for a current kernel # PATTERN='Error! Bad return status for module build on kernel' WHERE='pass' ISSUE=1 HEADER="DKMS-packages failing to build a module for a default kernel" HELPTEXT="

Kernel module source may not be compatible with the latest kernel.

" piuparts-1.1.1/known_problems/needs_rebuild_issue.conf0000644000000000000000000000154713266771660020216 0ustar # # detect packages which have the string "Please rebuild the package" or "package ... should be rebuilt" in their logs # PATTERN="Please rebuild the package|should be rebuilt|should be rebuild|warning: maintainer scripts should not call install-info anymore|called emacs-package-install as a new-style add-on, but has no compat file" WHERE='pass' ISSUE=1 HEADER='Packages which have logs with the string "Please rebuild the package" or "package ... should be rebuilt"' HELPTEXT='

This is a recommendation to rebuild some packages with updated debhelper to enable new features, e.g. trigger support.

Please identify the correct package causing this warning and retest the rdepends after that package was fixed.

This issue may be a false positive during upgrade tests, since it can be triggered on an old package before (or while) it gets upgraded.

' piuparts-1.1.1/known_problems/obsolete_conffiles_error.conf0000644000000000000000000000100713266762112021236 0ustar # # detect packages that leave obsolete conffiles after upgrades # PATTERN='OBSOLETE CONFFILE' WHERE='fail bugged affected' ISSUE=1 HEADER='Packages leaving obsolete conffiles after upgrade' HELPTEXT='

Packages that leave obsolete conffiles after upgrade. Using 

dpkg-maintscript-helper rm_conffile
via dh_installdeb package.maintscript files is the recommended way to clean them up. There may be false positives, e.g. if a conffile was converted to a maintainer script managed configuration file.

' piuparts-1.1.1/known_problems/obsolete_conffiles_issue.conf0000644000000000000000000000076713266762112021251 0ustar # # detect packages that leave obsolete conffiles after upgrades # PATTERN='OBSOLETE CONFFILE' WHERE='pass' ISSUE=1 HEADER='Packages leaving obsolete conffiles after upgrade' HELPTEXT='

Packages that leave obsolete conffiles after upgrade. Using 

dpkg-maintscript-helper rm_conffile
via dh_installdeb package.maintscript files is the recommended way to clean them up. There may be false positives, e.g. if a conffile was converted to a maintainer script managed configuration file.

' piuparts-1.1.1/known_problems/overwrite_other_packages_files_error.conf0000644000000000000000000000157213351442133023641 0ustar # # detect packages which try to overwrite other packages files # PATTERN='trying to overwrite (.*) which is also in package' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because they tried to overwrite other packages files' HELPTEXT='

This is because the package tries to overwrite another packages files without declaring a replaces relation. See policy 7.6 at https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces.

According to the thread started at 200908071233.02813.holger@layer-acht.org these bugs are to be filed with severity serious.

' piuparts-1.1.1/known_problems/owned_files_after_purge_error.conf0000644000000000000000000000114613351442133022250 0ustar # # detect packages with owned files after purge (policy 6.8) # PATTERN='owned by:' EXCLUDE_PATTERN=',|usr/local/' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with owned files after purge' HELPTEXT='

WARNING: this list might contain false positives.
#316521: dpkg: incomplete cleanup of empty directories

Packages need to remove owned files after purge, see https://www.debian.org/doc/debian-policy/ch-files.html#behavior

' piuparts-1.1.1/known_problems/owned_files_after_purge_issue.conf0000644000000000000000000000106713351442133022251 0ustar # # detect packages with owned files after purge (policy 6.8) # PATTERN='owned by:' WHERE='pass' ISSUE=1 HEADER='Packages with owned files after purge' HELPTEXT='

WARNING: this list might contain false positives.
#316521: dpkg: incomplete cleanup of empty directories

Packages need to remove owned files after purge, see https://www.debian.org/doc/debian-policy/ch-files.html#behavior

' piuparts-1.1.1/known_problems/owned_files_by_many_packages_error.conf0000644000000000000000000000113613266762112023247 0ustar # # detect conffiles owned by two or more packages which are not removed by dpkg correctly # PATTERN='owned by.+,' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs which have conffiles owned by two or more packages which are not removed by dpkg correctly' HELPTEXT='

So these might be ok packages (or not, as there might be other problems), but we cannot know until #454694 in dpkg is fixed.
#316521: dpkg: incomplete cleanup of empty directories

' piuparts-1.1.1/known_problems/packages_have_been_kept_back_error.conf0000644000000000000000000000051513266762112023152 0ustar # # detect possible dependency issues # PATTERN='packages have been kept back' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with logs with the string "packages have been kept back"' HELPTEXT='

"packages have been kept back" usually indicates some dependency issue that caused apt to not fully upgrade the system.

' piuparts-1.1.1/known_problems/packages_have_been_kept_back_issue.conf0000644000000000000000000000047513266762112023156 0ustar # # detect possible dependency issues # PATTERN='packages have been kept back' WHERE='pass' ISSUE=1 HEADER='Packages with logs with the string "packages have been kept back"' HELPTEXT='

"packages have been kept back" usually indicates some dependency issue that caused apt to not fully upgrade the system.

' piuparts-1.1.1/known_problems/piuparts-depends-dummy_error.conf0000644000000000000000000000060413266771660022024 0ustar # # detect problems where piuparts-depends-dummy.deb was not installed # PATTERN='ERROR: Installation of piuparts-depends-dummy FAILED|ERROR: Command failed.*apt-get.*install.*piuparts-depends-dummy.deb' WHERE='fail bugged affected' ISSUE=0 HEADER='Problems installing piuparts-depends-dummy.deb' HELPTEXT='

apt chose to remove piuparts-depends-dummy.deb instead of fixing it.

' piuparts-1.1.1/known_problems/piuparts-depends-dummy_issue.conf0000644000000000000000000000046013266762112022013 0ustar # # detect problems where piuparts-depends-dummy.deb was not installed # PATTERN='ERROR: Installation of piuparts-depends-dummy FAILED' WHERE='pass' ISSUE=1 HEADER='Problems installing piuparts-depends-dummy.deb' HELPTEXT='

apt chose to remove piuparts-depends-dummy.deb instead of fixing it.

' piuparts-1.1.1/known_problems/post_installation_script_error.conf0000644000000000000000000000053713266771660022543 0ustar # # detect packages which failed because post-installation maintainer script failed # PATTERN='post-installation script( subprocess)? returned error' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because post-installation maintainer script failed' HELPTEXT='

This is a somewhat unclassified failure at the moment.

' piuparts-1.1.1/known_problems/post_removal_script_error.conf0000644000000000000000000000052013266771660021477 0ustar # # detect packages which failed because post-removal maintainer script failed # PATTERN='post-removal script( subprocess)? returned error' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because post-removal maintainer script failed' HELPTEXT='

This is a somewhat unclassified failure at the moment.

' piuparts-1.1.1/known_problems/pre_depends_error.conf0000644000000000000000000000142713266762112017670 0ustar # # detect packages which failed because of a problem with pre-depends # PATTERN='E: Couldn.t configure pre-depend .* for .*, probably a dependency cycle.' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because of a problem with pre-depends ' HELPTEXT='

The package(s) in question fail(s) to install or upgrade properly, because a pre-dependent package could not be configured. This is likely due to a dependency cycle.
Note that it is possible, that aptitude can deal with (some of) such situations (ie upgrades), (by removing the packages first), while apt-get cannot. While it can be argued this is a problem in apt-get, it is also a problem in the package(s) listed below, as both aptitude and apt-get can be used for installing packages.

' piuparts-1.1.1/known_problems/pre_installation_script_error.conf0000644000000000000000000000053413266771660022341 0ustar # # detect packages which failed because pre-installation maintainer script failed # PATTERN='pre-installation script( subprocess)? returned error' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because pre-installation maintainer script failed' HELPTEXT='

This is a somewhat unclassified failure at the moment.

' piuparts-1.1.1/known_problems/pre_removal_script_error.conf0000644000000000000000000000051513266771660021304 0ustar # # detect packages which failed because pre-removal maintainer script failed # PATTERN='pre-removal script( subprocess)? returned error' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because pre-removal maintainer script failed' HELPTEXT='

This is a somewhat unclassified failure at the moment.

' piuparts-1.1.1/known_problems/problems_and_no_force_error.conf0000644000000000000000000000135613266771660021730 0ustar # # detect packages with problems because of not enough force # PATTERN='E: There are problems and -y was used without --force-yes|E: Essential packages were removed and -y was used without --allow-remove-essential' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because of not enough force' HELPTEXT='

The piuparts logfile for these packages contains the string 

E: There are problems and -y was used without --force-yes
, which is usually an indication, that an essential package needs to be removed to install this package. As piuparts does not use that much force, the piuparts test fails.

This is usually not an error in the package and it needs to be seen how piuparts should deal with it.

' piuparts-1.1.1/known_problems/problems_and_no_force_issue.conf0000644000000000000000000000131513266771660021722 0ustar # # detect packages with problems because of not enough force # PATTERN='E: Essential packages were removed and -y was used without --allow-remove-essential' WHERE='pass' ISSUE=1 HEADER='Packages with logs reporting attempted removal of essential packages' HELPTEXT='

The piuparts logfile for these packages contains the string 

E: Essential packages were removed and -y was used without --allow-remove-essential
, which is usually an indication, that an essential package needs to be removed to install this package. As piuparts does not use that much force, the piuparts test fails.

This is usually not an error in the package and it needs to be seen how piuparts should deal with it.

' piuparts-1.1.1/known_problems/processes_running_error.conf0000644000000000000000000000213213351442133021131 0ustar # # detect packages which failed because of processes are running inside chroot # PATTERN='ERROR: FAIL: Processes are running inside chroot' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because of processes left behind' HELPTEXT='

There were processes running inside the chroot at the end of the piuparts run. This is probably due to directly calling /etc/rc.d/ scripts in packages maintainer scripts, which is a violation of policy 9.3.3.2 and must be replaced by using invoke-rc.d (which will respect an optionally existing policy-rc.d) - see https://www.debian.org/doc/debian-policy/ch-opersys.html#interfacing-with-init-systems, /usr/share/doc/sysv-rc/README.invoke-rc.d.gz and /usr/share/doc/sysv-rc/README.policy-rc.d.gz.

According to the thread started at 200908061127.35727.holger@layer-acht.org these bugs are to be filed with severity serious.

' piuparts-1.1.1/known_problems/program_name_collision_inadequate_issue.conf0000644000000000000000000000102713266771660024325 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* program-name-collision' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'program-name-collision' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'program-name-collision' which indicates that this package ships a program with the same name as another program. This is a violation of policy 10.1.

" piuparts-1.1.1/known_problems/py_file_not_bytecompiled_inadequate_issue.conf0000644000000000000000000000066713266762112024653 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* py-file-not-bytecompiled' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'py-file-not-bytecompiled' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'py-file-not-bytecompiled' which indicates a bug.

" piuparts-1.1.1/known_problems/pyshared_file_not_bytecompiled_inadequate_issue.conf0000644000000000000000000000071113266762112026030 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* pyshared-file-not-bytecompiled' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'pyshared-file-not-bytecompiled' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'pyshared-file-not-bytecompiled' which indicates a bug.

" piuparts-1.1.1/known_problems/resource_violation_error.conf0000644000000000000000000000067713266762112021321 0ustar # # detect packages violating resource limits during the test # PATTERN='Command was terminated after exceeding|Process KILLED - exceed maximum run time' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages where the test was aborted after exceeding time or output limit' HELPTEXT='

Usually caused by seriously misbehaving maintainer scripts that go into endless loops or try to get user input in DEBIAN_FRONTEND=noninteractive mode.

' piuparts-1.1.1/known_problems/symbol-size-mismatch_inadequate_issue.conf0000644000000000000000000000065313266762112023657 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* symbol-size-mismatch' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'symbol-size-mismatch' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'symbol-size-mismatch' which indicates a bug.

" piuparts-1.1.1/known_problems/test_was_skipped_issue.conf0000644000000000000000000000115613266771660020756 0ustar # # detect possible dependency issues # PATTERN='SKIP: All tests. Package cannot be tested with piuparts:' WHERE='pass' ISSUE=1 HEADER='Packages with logs with the string "SKIP: All tests. Package cannot be tested with piuparts"' HELPTEXT='

The package is not installable by piuparts and the test has therefore been skipped. This usually means that either the package depends on external ressources that are not (or no longer) publically available or that the package (from an archived release) is broken beyond repair. There is no point in doing distupgrade tests if the package cannot be installed initially.

' piuparts-1.1.1/known_problems/trigger_cycle_error.conf0000644000000000000000000000063113266771660020226 0ustar # # detect packages which failed because dpkg encountered trigger problems # PATTERN='dpkg: cycle found while processing triggers|dpkg: dependency problems prevent processing triggers' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because dpkg encountered trigger problems' HELPTEXT='

This is a bug in ???, but it may have to be worked around in some (unrelated) packages.

' piuparts-1.1.1/known_problems/unclassified_failures.conf0000644000000000000000000000047113266762112020530 0ustar # # report for failures that didn't match any known problem # PATTERN='QQQQQ_d195eab29de40b96b44ee8646c0d10d97171d454_dummy_pattern_QQQQQ' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with unclassified failures detected' HELPTEXT='

Please investigate and improve detection of known error types!

' piuparts-1.1.1/known_problems/undefined_symbol_inadequate_issue.conf0000644000000000000000000000063713266762112023127 0ustar # # detect packages with some inadequate tag from adequate # PATTERN='(FAIL|WARN): Running adequate resulted in .* undefined-symbol' WHERE='pass fail bugged affected' ISSUE=1 HEADER="Packages tagged 'undefined-symbol' by adequate" HELPTEXT="

Running adequate resulted in the package being tagged 'undefined-symbol' which indicates a bug.

" piuparts-1.1.1/known_problems/unknown_inadequate_issue.conf0000644000000000000000000000117713304500520021262 0ustar # # detect packages which have the string "Found unknown tags running adequate" in their logs # PATTERN='(FAIL|WARN): Found unknown tags running adequate' WHERE='pass fail bugged affected' ISSUE=1 HEADER='Packages which have logs with the string "Found unknown tags running adequate"' HELPTEXT='

Sometimes new types of problems are detected by adequate, which classifies them using tags. When this happens these new tags need to be made known to piuparts. Please notify piuparts-devel@alioth-lists.debian.net.

' piuparts-1.1.1/known_problems/unknown_purge_error.conf0000644000000000000000000000107413266762112020277 0ustar # # detect packages which try to overwrite other packages files # PATTERN='ERROR: Command failed .*.dpkg., .--purge., .--pending.]' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with failed logs because dpkg --purge --pending failed' HELPTEXT='

This is often because some dependencies have to be removed together and not seperatedly, and thus rather a bug in piuparts... but be careful, this list also includes failures due to "command not found"-error, which made the purge fail... so file those bugs first.

' piuparts-1.1.1/known_problems/unowned_files_after_purge_error.conf0000644000000000000000000000156013351442133022613 0ustar # # detect packages with unowned files after purge (policy 6.8) # PATTERN='not owned' EXCLUDE_PATTERN='/usr/share/mime/|usr/local/' WHERE='fail bugged affected' ISSUE=0 HEADER='Packages with unowned files after purge' HELPTEXT='

WARNING: this list might contain false positives. One group of them are packages with files in /usr/share/mime - those are bugs from shared-mime-info (#527063) and have been ignored for this list. There are probably others like this as well.

Packages with unowned files after purge (violating policy 6.8) see https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging

' piuparts-1.1.1/known_problems/unowned_files_after_purge_issue.conf0000644000000000000000000000154013351442133022610 0ustar # # detect packages with unowned files after purge (policy 6.8) # PATTERN='not owned' EXCLUDE_PATTERN="/usr/share/mime/|usr/local/" WHERE='pass' ISSUE=1 HEADER='Packages with unowned files after purge' HELPTEXT='

WARNING: this list might contain false positives. One group of them are packages with files in /usr/share/mime - those are bugs from shared-mime-info (#527063) and have been ignored for this list. There are probably others like this as well.

Packages with unowned files after purge (violating policy 6.8) see https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-removal-and-or-configuration-purging

' piuparts-1.1.1/known_problems/unowned_lib_symlink_error.conf0000644000000000000000000000034713266762112021453 0ustar # # detect unowned symlinks in library directories # PATTERN='UNOWNED SYMLINK' WHERE='fail bugged affected' ISSUE=0 HEADER='Unowned symlinks is library directories' HELPTEXT='

These should rather be shipped in packages.

' piuparts-1.1.1/known_problems/unowned_lib_symlink_issue.conf0000644000000000000000000000032713266762112021450 0ustar # # detect unowned symlinks in library directories # PATTERN='UNOWNED SYMLINK' WHERE='pass' ISSUE=1 HEADER='Unowned symlinks is library directories' HELPTEXT='

These should rather be shipped in packages.

' piuparts-1.1.1/known_problems/used_exception_issue.conf0000644000000000000000000000040613266762112020411 0ustar # # report packages that used a piuparts exception to pass a test # PATTERN='piuparts exception for package' WHERE='pass' ISSUE=1 HEADER='Packages that used a piuparts exception' HELPTEXT='

These packages needed a piuparts exception to pass the test.

' piuparts-1.1.1/lib/0000755000000000000000000000000013403304304011007 5ustar piuparts-1.1.1/lib/read_config.sh0000644000000000000000000000365713266771660013642 0ustar # Copyright © 2011, 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # Helper function for getting values from piuparts.conf. # Used by several master and slave scripts. # PIUPARTS_CONF=${PIUPARTS_CONF:-/etc/piuparts/piuparts.conf} [ -f "$PIUPARTS_CONF" ] || exit 0 # usage: get_config_value VARIABLE section key [default] get_config_value() { local section key value test -n "$1" && test "$1" = "$(echo "$1" | tr -c -d '[:alnum:]_')" || exit 1 section="$2" key="$3" # First select the [$section] block (\#^\[$section\]#) (use # as # marker because $section may contain slashes) up to the start of the # next section (/^\[/). The select the $key=value, this may be wrapped # with indented lines and comment lines embedded. The $key=value is # over once we hit the next key (or any line not starting with # or # whitespace. Throw away comments (/^#/d), the following key, remove # our $key= part, trim the value, remove empty lines, and print it. value="$(sed -rn '\#^\['"$section"'\]#,/^\[/ {/^'"$key"'\s*=/,/^[^ \t#]/ {/^#/d; /^'"$key"'\s*=|^\s/!d; s/^'"$key"'\s*=\s*//; s/^\s*//; s/\s*$//; /^$/d; p}}' "$PIUPARTS_CONF")" if [ -z "$value" ]; then if [ -n "${4+set}" ]; then value="$4" else echo "'$key' not set in section [$section] of $PIUPARTS_CONF, exiting." >&2 exit 1 fi fi eval "$1"='"$value"' } piuparts-1.1.1/lib/write_log.sh0000644000000000000000000000222213403304304013334 0ustar # Copyright © 2018 Holger Levsen (holger@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # Helper function for publishing logfiles # publish_logs() { local LOG_OUTPUT=$1 local LOG_PREFIX=$2 local HTDOCS="$3" local LOG=$4 local YEAR=$(date -u +%Y) local MONTH=$(date -u +%m) local DAY=$(date -u +%d) local DIR="$HTDOCS/logs/$YEAR/$MONTH/$DAY" mkdir -p "$DIR" if [ -n "$LOG_PREFIX" ] && [ ! -s "$DIR/$LOG.txt" ] ; then cat $LOG_PREFIX >> "$DIR/$LOG.txt" rm -f $LOG_PREFIX >/dev/null fi cat $LOG_OUTPUT >> "$DIR/$LOG.txt" rm -f $LOG_OUTPUT >/dev/null } piuparts-1.1.1/master-bin/0000755000000000000000000000000013605426771012323 5ustar piuparts-1.1.1/master-bin/archive_old_logs.in0000644000000000000000000000420213472727312016151 0ustar #!/bin/sh # Copyright 2009-2011 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections # # archive old log files # TOTAL=0 OLDPWD=$(pwd) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue cd $MASTER/$SECTION mkdir -p pass bugged affected fail mkdir -p archive/pass archive/bugged archive/affected archive/fail test -f archive/stamp || touch -d @0 archive/stamp # start at the epoch touch -d yesterday archive/stamp.new # look back one more day the next time we will be run OUTPUT="" # loop through all packages logs for PACKAGE in $(find pass/ fail/ bugged/ affected/ -name '*.log' -newer archive/stamp | cut -d"_" -f1 | cut -d"/" -f2 | sort -u) ; do # all logs except the last one (|sed '$d' deletes the last line) OLDLOGS=$( ls -tr1 --color=none bugged/${PACKAGE}_*.log affected/${PACKAGE}_*.log fail/${PACKAGE}_*.log pass/${PACKAGE}_*.log 2>/dev/null|sed '$d' ) if [ ! -z "$OLDLOGS" ] ; then if [ -z "$OUTPUT" ] ; then OUTPUT="yes" echo $SECTION fi for LOG in $OLDLOGS ; do TOTAL=$(($TOTAL + 1)) mv -v $LOG archive/$(echo $LOG|cut -d"/" -f1)/ done fi done find archive/ -name '*.log' | nice xargs -P $(nproc) -n 1 -r xz -f if [ -n "$OUTPUT" ] ; then echo echo fi mv archive/stamp.new archive/stamp cd "$OLDPWD" done if [ "$TOTAL" -gt "0" ]; then echo "Archived $TOTAL old logfiles." fi piuparts-1.1.1/master-bin/detect_archive_issues.in0000644000000000000000000000636013430261151017205 0ustar #!/bin/sh # Copyright © 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2017 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value HTDOCS global output-directory get_config_value SECTIONS global sections get_config_value URLBASE global urlbase https://piuparts.debian.org get_config_value DAYS global reschedule-untestable-days 7 ISSUE_LOG="$MASTER/archive_issues.txt" LOG_OUTPUT=$(mktemp) LOG_PREFIX=$(mktemp) STARTDATE=$(date -u +%s) # # detect packages which are/were untestable due to archive issue and mark them as such # LOGS=$(mktemp) URLS=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue URL_SED="s#$MASTER/$SECTION/fail#$URLBASE/$SECTION/untestable#" mkdir -p $MASTER/$SECTION/fail/ $MASTER/$SECTION/untestable/ $MASTER/$SECTION/recycle/ # '-mtime +0' actually means '24 hours or older' find $MASTER/$SECTION/fail -name '*.log' -mtime +0 | xargs -r \ grep -l -E "E: Broken packages|E: Unable to correct problems, you have held broken packages|E: Error, pkgProblemResolver::Resolve generated breaks" 2>/dev/null > $LOGS if [ -s $LOGS ]; then for package_log in $(cat $LOGS) do URL=$(echo "$package_log" | sed "$URL_SED") if ! grep -q "$URL" $ISSUE_LOG ; then # immediately recycle upon first appearance ln -f $package_log $MASTER/$SECTION/recycle/ fi mv $package_log $MASTER/$SECTION/untestable/ done sed "$URL_SED" $LOGS >> $URLS fi done if [ -s $URLS ]; then date >> $ISSUE_LOG cat $URLS >> $ISSUE_LOG FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "Broken packages detected! The following packages have been moved to" echo "$section/untestable and will be tested again in $DAYS days." echo echo "Broken packages are usually a temporary problem in the archive and are" echo "caught by other tools like britney or https://qa.debian.org/dose/debcheck.html" echo "If it is always the same package failing, it's likely to be an issue in the" echo "package." echo echo "Those problems were found in failed logs by grep'ing for these patterns:" echo " 'E: Broken packages'" echo " 'E: Unable to correct problems, you have held broken packages'" echo " 'E: Error, pkgProblemResolver::Resolve generated breaks'" echo ) > $LOG_PREFIX ( echo "$(date -u)" echo "Runtime: $RUNTIME" echo grep -f $URLS $ISSUE_LOG | sort | uniq -c | sort -rn echo ) > $LOG_OUTPUT publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS archive_issues fi rm $LOGS $URLS piuparts-1.1.1/master-bin/detect_network_issues.in0000644000000000000000000000710613553616457017277 0ustar #!/bin/sh # Copyright 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2017 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory STARTDATE=$(date -u +%s) LOG_OUTPUT=$(mktemp) LOG_PREFIX=$(mktemp) # # detect network/mirror problems # newline=" " tab=" " PATTERN= DISPLAY_PATTERN= add_pattern() { PATTERN="${PATTERN:+${PATTERN}|}($1)" DISPLAY_PATTERN="${DISPLAY_PATTERN:+${DISPLAY_PATTERN}${newline}}${tab}$1" } add_pattern "Cannot initiate the connection to" add_pattern "Hash Sum mismatch" add_pattern "Failed to fetch.*Could not resolve" add_pattern "Failed to fetch.*Something wicked happened resolving" add_pattern "Failed to fetch.*Unable to connect to" add_pattern "E: Method http has died unexpectedly" add_pattern "Some index files failed to download, they have been ignored, or old ones used instead." add_pattern "Some index files failed to download. They have been ignored, or old ones used instead." add_pattern "W: GPG error: .* Release: The following signatures were invalid: BADSIG" add_pattern "W: GPG error: .* InRelease: The following signatures were invalid: NODATA" add_pattern "W: GPG error: .* InRelease: Clearsigned file isn't valid, got 'NODATA'" add_pattern "E: The repository '.*' does no longer have a Release file." add_pattern "E: The repository '.*' no longer has a Release file." add_pattern "E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing" add_pattern "E: Version '.*' for '.*' was not found" add_pattern 'E: Method file has died unexpectedly!' add_pattern "E: Sub-process rred received a segmentation fault." add_pattern "ERROR: Command failed \(status=-7\):.*'apt-get', 'update'" add_pattern 'Package .* .* not found in .*(, .* is available)?' add_pattern "AppStream system cache was updated, but problems were found: Metadata files have errors" FILE=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue for subdir in fail bugged affected untestable ; do test -d $MASTER/$SECTION/$subdir || continue grep -r -l -E --include '*.log' "$PATTERN" $MASTER/$SECTION/$subdir >> $FILE done done if [ -s $FILE ] ; then FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "Network problems detected! The following logfiles have been deleted:" echo echo "Those problems were found in failed logs by grep'ing for these patterns:" echo "$DISPLAY_PATTERN" echo ) > $LOG_PREFIX ( echo "$(date -u)" echo "Runtime: $RUNTIME" echo for log in $(sort -u $FILE) do echo "$log" | cut -d "/" -f5- grep -E "$PATTERN" "$log" 2>/dev/null | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/" rm -f "$log" done echo ) > $LOG_OUTPUT publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS network_issues fi rm $FILE piuparts-1.1.1/master-bin/detect_piuparts_issues.in0000644000000000000000000001264013553616457017454 0ustar #!/bin/sh # Copyright © 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2019 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory STARTDATE=$(date -u +%s) LOG_OUTPUT=$(mktemp) LOG_PREFIX=$(mktemp) # # detect piuparts problems # newline=" " tab=" " PATTERN= DISPLAY_PATTERN= add_pattern() { PATTERN="${PATTERN:+${PATTERN}|}($1)" DISPLAY_PATTERN="${DISPLAY_PATTERN:+${DISPLAY_PATTERN}${newline}}${tab}$1" } add_pattern 'PIUPARTS OUTPUT INCOMPLETE' add_pattern "WARNING: History of available packages does not match - reference chroot may be outdated" add_pattern "tar( \(child\))?: .*.tar.gz: Cannot open: No such file or directory" add_pattern "tar: Error is not recoverable: exiting now" add_pattern "gzip: stdin: invalid compressed data--crc error" add_pattern "ERROR:.*:Temporary directory is not a directory" add_pattern "E: Sub-process Popen returned an error code \(2\)" add_pattern "chroot: failed to run command '.*': Permission denied" add_pattern "ERROR: Command failed \(status=(-7|100)\): .* 'apt-cache'" add_pattern 'update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument' add_pattern 'USERDEL: USER .* IS CURRENTLY LOGGED IN' add_pattern 'userdel: user .* is currently logged in' add_pattern 'mount: .* Cannot allocate memory' # Disk space issues. add_pattern "NO SPACE LEFT ON DEVICE" add_pattern "No space left on device" add_pattern "E: You don't have enough free space in /var/cache/apt/archives/" add_pattern "E: Failed to write temporary StateFile /var/lib/apt/extended_states.tmp" add_pattern "tar: .*: Wrote only .* of .* bytes" add_pattern "sh: echo: I/O error" add_pattern "passwd: Authentication token manipulation error" add_pattern "(chfn|groupadd|useradd): (cannot lock|failure while writing changes to) /etc/(group|gshadow|passwd|shadow|subuid)" add_pattern "(groupadd|useradd): (unable to lock|cannot rewrite) (group|password) file" # Database server issues. add_pattern 'INVOKE-RC.D: INITSCRIPT (MYSQL|POSTGRESQL|POSTGRESQL-8.3), ACTION "(RE)?START" FAILED.' add_pattern 'invoke-rc.d: initscript (mysql|postgresql|postgresql-8.3), action "(re)?start" failed.' add_pattern 'There is a MySQL server running, but we failed in our attempts to stop it.' add_pattern 'The PostgreSQL server failed to start.' add_pattern 'Is another postmaster already running on port 543.' add_pattern 'No database found online on port 5432' add_pattern 'unable to connect to postgresql server' add_pattern 'Failed to connect to postgres.' add_pattern 'PG::ConnectionBad: could not connect to server' add_pattern 'psql: FATAL: password authentication failed for user' add_pattern 'createuser: could not connect to database postgres: could not connect to server: No such file or directory' add_pattern 'Non-default port in /etc/postgresql/.*/main/postgresql.conf' add_pattern 'Firebird .* server already running.' # These are python errors from piuparts itself. Python errors from maintainer # scripts etc. would be indented by 2 spaces, so don't match here. add_pattern "^RuntimeError: maximum recursion depth exceeded while calling a Python object" add_pattern "^Traceback" add_pattern "^IndentationError:" add_pattern "^NameError:" add_pattern "^OSError:" add_pattern "^SyntaxError:" FILE=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue for subdir in fail bugged affected ; do test -d $MASTER/$SECTION/$subdir || continue grep -r -l -E --include '*.log' "$PATTERN" $MASTER/$SECTION/$subdir >> $FILE 2>/dev/null done for subdir in fail bugged affected untestable pass ; do test -d $MASTER/$SECTION/$subdir || continue find $MASTER/$SECTION/$subdir -name 'tmp??????' -mtime +1 -delete 2>/dev/null find $MASTER/$SECTION/$subdir -name '*.log' -size 0 >> $FILE done done if [ -s $FILE ] ; then FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "piuparts problems detected! Please review the following logfiles/packages." echo "If it is always the same package failing, it's likely to be an issue in the" echo "package." echo echo "Problems were found in failed logs by grep'ing for these patterns:" echo "$DISPLAY_PATTERN" echo ) > $LOG_PREFIX ( echo "$(date -u)" echo "Runtime: $RUNTIME" echo echo "The following logfiles have been deleted:" echo for log in $(sort -u $FILE) do echo "$log" | cut -d "/" -f5- test -s "$log" || echo "${tab}Empty logfile." grep -E "$PATTERN" "$log" 2>/dev/null | perl -e 'print grep !$seen{$_}++, <>;' | head -n 10 | sed "s/^/${tab}/" rm -f "$log" done echo ) > $LOG_OUTPUT publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS piuparts_issues fi rm $FILE piuparts-1.1.1/master-bin/detect_well_known_errors.py0000644000000000000000000001454213605426771020006 0ustar #!/usr/bin/python3 # -*- coding: utf-8 -*- # Copyright 2013 David Steele (dsteele@gmail.com) # Copyright © 2014 Andreas Beckmann (anbe@debian.org) # Copyright © 2017-2018 Holger Levsen (holger@layer-acht.org) # # This file is part of Piuparts # # Piuparts is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # Piuparts is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program; if not, see . import os import sys import time import logging import argparse import fcntl from collections import deque import piupartslib.conf from piupartslib.conf import MissingSection from piupartslib.dwke import * CONFIG_FILE = "/etc/piuparts/piuparts.conf" KPR_DIRS = ('pass', 'bugged', 'affected', 'fail', 'untestable') class Busy(Exception): def __init__(self): self.args = "section is locked by another process", class WKE_Config(piupartslib.conf.Config): """Configuration parameters for Well Known Errors""" def __init__(self, section="global", defaults_section=None): self.section = section piupartslib.conf.Config.__init__(self, section, { "sections": "report", "master-directory": ".", "known-problem-directory": "@sharedir@/piuparts/known_problems", "exclude-known-problems": None, }, defaults_section=defaults_section) def setup_logging(log_level): logger = logging.getLogger() logger.setLevel(log_level) handler = logging.StreamHandler(sys.stdout) logger.addHandler(handler) def process_section(section, config, problem_list, recheck=False, recheck_failed=False, pkgsdb=None): """ Update .bug and .kpr files for logs in this section """ sectiondir = os.path.join(config['master-directory'], section) workdirs = [os.path.join(sectiondir, x) for x in KPR_DIRS] if not os.access(sectiondir, os.F_OK): raise MissingSection("", section) with open(os.path.join(sectiondir, "master.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: raise Busy() [os.mkdir(x) for x in workdirs if not os.path.exists(x)] logdict = get_file_dict(workdirs, LOG_EXT) kprdict = get_file_dict(workdirs, KPR_EXT) bugdict = get_file_dict(workdirs, BUG_EXT) del_cnt = clean_cache_files(logdict, kprdict, recheck, recheck_failed) clean_cache_files(logdict, bugdict, skipnewer=True) kprdict = get_file_dict(workdirs, KPR_EXT) section_config = WKE_Config(section=section, defaults_section="global") section_config.read(CONFIG_FILE) if section_config['exclude-known-problems']: excluded = section_config['exclude-known-problems'].split() problem_list = [p for p in problem_list if p.name not in excluded] add_cnt = make_kprs(logdict, kprdict, problem_list) return (del_cnt, add_cnt) def detect_well_known_errors(sections, config, problem_list, recheck, recheck_failed): total_del = 0 total_add = 0 todo = deque([(s, 0) for s in sections]) while len(todo): (section, next_try) = todo.popleft() now = time.time() if (now < next_try): # sleeping, section has been tried recently time.sleep(max(30, next_try - now) + 30) try: (del_cnt, add_cnt) = \ process_section(section, config, problem_list, recheck, recheck_failed) total_del += del_cnt total_add += add_cnt current_time=time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime()) if del_cnt == 0 and add_cnt == 0: pass # nothing new else: logging.info("%s - %s: parsed logfiles: %d removed, %d added" % (current_time, section, del_cnt, add_cnt)) except Busy: # section is busy todo.append((section, time.time() + 300)) except MissingSection: pass current_time=time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime()) logging.info("%s - total parsed logfiles: %d removed, %d added" % (current_time, total_del, total_add)) logging.info("") if __name__ == '__main__': setup_logging(logging.DEBUG) parser = argparse.ArgumentParser( description="Detect well known errors", epilog=""" This script processes all log files against defined "known_problem" files, caching the problems found, by package, into ".kpr" files. """) parser.add_argument('sections', nargs='*', metavar='SECTION', help="limit processing to the listed SECTION(s)") parser.add_argument('--recheck', dest='recheck', action='store_true', help="recheck all log files (delete cache)") parser.add_argument('--recheck-failed', dest='recheck_failed', action='store_true', help="recheck failed log files (delete cache)") args = parser.parse_args() conf = WKE_Config() conf.read(CONFIG_FILE) with open(os.path.join(conf['master-directory'], "dwke.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: if sys.stdout.isatty(): sys.exit("another detect_well_known_errors process is already running") else: sys.exit(0) sections = args.sections if not sections: sections = conf['sections'].split() problem_list = create_problem_list(conf['known-problem-directory']) detect_well_known_errors(sections, conf, problem_list, args.recheck, args.recheck_failed) # vi:set et ts=4 sw=4 : piuparts-1.1.1/master-bin/distill_alternatives_log.in0000644000000000000000000000236413266771660017751 0ustar #!/bin/sh set -e # Copyright © 2017 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory mkdir -p ${HTDOCS}/for-manpages.d.o for SECTION in $SECTIONS ; do test -d $HTDOCS/$SECTION || continue get_config_value FLAGS $SECTION piuparts-flags "" if echo "$FLAGS" | grep -q scripts-log-alternatives ; then mkdir -p $(dirname ${HTDOCS}/for-manpages.d.o/${SECTION}) @libdir@/piuparts/debiman-piuparts-distill \ -logs_dir=${HTDOCS}/${SECTION} \ -output=${HTDOCS}/for-manpages.d.o/${SECTION}.json.gz fi done piuparts-1.1.1/master-bin/gather_bts_stats.in0000644000000000000000000000450613266771660016223 0ustar #!/bin/bash set -e # Copyright 2013,2014 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh get_config_value MASTER global master-directory BTS_STATS="$MASTER/bts_stats.txt" # exit if master-directory doesn't exist or if devscripts package is not installed test -n "$MASTER" || $(which bts) || exit 0 # "bts select" needs libsoap-lite-perl too dpkg -l libsoap-lite-perl >/dev/null 2>&1 || exit 0 # only run once a day TODAY=$(date +%Y%m%d) if $(grep -q ^$TODAY $BTS_STATS 2>/dev/null) ; then exit 0 fi # query bts RC="severity:serious severity:grave severity:critical" OTHER="severity:wishlist severity:minor severity:normal severity:important" DONE_RC=$(bts select usertag:piuparts users:debian-qa@lists.debian.org status:done archive:both $RC 2>/dev/null|wc -l) OPEN_RC=$(bts select usertag:piuparts users:debian-qa@lists.debian.org status:open status:forwarded $RC 2>/dev/null|wc -l) DONE_OTHER=$(bts select usertag:piuparts users:debian-qa@lists.debian.org status:done archive:both $OTHER 2>/dev/null|wc -l) OPEN_OTHER=$(bts select usertag:piuparts users:debian-qa@lists.debian.org status:open status:forwarded $OTHER 2>/dev/null|wc -l) # test if both values are integers if ! ( [[ $DONE_RC =~ ^-?[0-9]+$ ]] && [[ $OPEN_RC =~ ^-?[0-9]+$ ]] && [[ $DONE_OTHER =~ ^-?[0-9]+$ ]] && [[ $OPEN_OTHER =~ ^-?[0-9]+$ ]] ) ; then echo "Non-integer value detected, exiting." echo "DONE_RC: $DONE_RC OPEN_RC: $OPEN_RC" echo "DONE_OTHER: $DONE_OTHER OPEN_OTHER: $OPEN_OTHER" exit 1 fi # init file if needed if [ ! -f $BTS_STATS ] ; then echo "date, non-RC done, non-RC open, RC done, RC open" > $BTS_STATS fi # finally, write stats echo "$TODAY, $DONE_OTHER, $OPEN_OTHER, $DONE_RC, $OPEN_RC" >> $BTS_STATS piuparts-1.1.1/master-bin/generate_daily_report.in0000644000000000000000000001006513553616457017230 0ustar #!/bin/sh # Copyright © 2011-2013 Andreas Beckmann (anbe@debian.org) # Copyright © 2018-2019 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory get_config_value URLBASE global urlbase https://piuparts.debian.org get_config_value PYTHONPATH global PYTHONPATH '' get_config_value TESTING global testing-suite export PYTHONPATH LOCKFILE=$HTDOCS/daily.lock if [ -e $LOCKFILE ]; then echo "daily.lock exists:" ls -l $LOCKFILE exit 1 fi date > $LOCKFILE R_STARTDATE=$(date -u +%s) R_LOGDATEDIR="$HTDOCS/logs/$(date -u '+%Y/%m/%d')" # # various scripts to detect issues and cleanup to be run right before we generate the daily report # (this code block contains cruft and could be refactored) # OUTPUT=$(mktemp) LOG_OUTPUT=$(mktemp) for SCRIPT in archive_old_logs ; do STARTDATE=$(date -u +%s) @sharedir@/piuparts/master/$SCRIPT 2>&1 > $OUTPUT FINALDATE=$(date -u +%s) if [ -s $OUTPUT ] ; then if grep -q "Runtime: " $OUTPUT ; then cat $OUTPUT > $LOG_OUTPUT else RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "Runtime: $RUNTIME" ; echo ; cat $OUTPUT ) > $LOG_OUTPUT fi publish_logs $LOG_OUTPUT "" $HTDOCS $SCRIPT fi done rm $OUTPUT # # piuparts-report starts here # DAILYREPORT=$HTDOCS/dailyreport.txt >$DAILYREPORT # Failures of the day it should be, the rest is boring. FAILURESTAMP=$HTDOCS/last-failure-report.stamp test -f $FAILURESTAMP || touch -d @0 $FAILURESTAMP # start at the epoch touch $FAILURESTAMP.new # for the next report DAILYTMP=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue for DIRECTORY in fail bugged affected untestable ; do mkdir -p $MASTER/$SECTION/$DIRECTORY done find $MASTER/$SECTION/fail $MASTER/$SECTION/bugged $MASTER/$SECTION/affected $MASTER/$SECTION/untestable \ -type f -name '*.log' -newer $FAILURESTAMP -exec ls -1 {} + 2>/dev/null done | sed s#^$MASTER#$URLBASE# >> $DAILYTMP new_release_failures=$(cat $DAILYTMP | (egrep "/(sid|$TESTING|testing2sid)/" 2>/dev/null || echo) | wc -l) new_failures=$(cat $DAILYTMP | wc -l) new_failure_diff=$(( $new_failures - $new_release_failures )) echo -n "$new_failures new failures in total, " >> $DAILYREPORT echo "$new_release_failures new failures sid, $TESTING and testing2sid:" >> $DAILYREPORT # show failures (and show failures in sid, $TESTING and testing2sid first cat $DAILYTMP | (egrep "/(sid|$TESTING|testing2sid)/" 2>/dev/null || true) >> $DAILYREPORT echo >> $DAILYREPORT echo "$new_failure_diff other new failures:" >> $DAILYREPORT cat $DAILYTMP | (egrep -v "/(sid|$TESTING|testing2sid)/" 2>/dev/null || true) >> $DAILYREPORT rm $DAILYTMP echo "" >> $DAILYREPORT date >> $DAILYREPORT echo "Expiring old .html files:" >> $DAILYREPORT find $HTDOCS -name '* *.html' -mtime +30 -ls -delete >> $DAILYREPORT expire=$(mktemp) find $HTDOCS -name '*.html' -mtime +30 | head -n 500 > $expire if [ -s $expire ]; then ls -ld $(cat $expire) >> $DAILYREPORT rm -f $(cat $expire) >> $DAILYREPORT fi rm $expire FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $R_STARTDATE seconds" +%T) ( echo "Runtime: $RUNTIME" echo echo $R_LOGDATEDIR ls -lt $R_LOGDATEDIR echo cat $DAILYREPORT ) | mail -s generate_daily_report $LOGNAME mv $DAILYREPORT $R_LOGDATEDIR mv $FAILURESTAMP.new $FAILURESTAMP rm -f $LOCKFILE piuparts-1.1.1/master-bin/master_cleanup.in0000644000000000000000000000207413266771660015663 0ustar #!/bin/sh set -e # Copyright © 2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # cleanup $HTDOCS/daily.lock # # - this should only be run (automatically) on boot # . @sharedir@/piuparts/lib/read_config.sh get_config_value HTDOCS global output-directory LOCKFILE=$HTDOCS/daily.lock if [ -e $LOCKFILE ]; then if pgrep -f generate_daily_report || pgrep -f piuparts-report ; then echo "daily processing is running" else rm -f -v $LOCKFILE fi fi piuparts-1.1.1/master-bin/master_shell_runner.in0000644000000000000000000000162413403304304016710 0ustar #!/bin/sh # Copyright © 2018 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value HTDOCS global output-directory LOG_OUTPUT=$(mktemp) $1 >$LOG_OUTPUT 2>&1 publish_logs $LOG_OUTPUT "" $HTDOCS $1 piuparts-1.1.1/master-bin/prepare_backup.in0000644000000000000000000000370113436204472015631 0ustar #!/bin/sh set -e # Copyright 2009,2014,2019 Holger Levsen (holger@layer-acht.org) # Copyright © 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # backup statistiscs files to a single directory for DSA to backup # piuparts.d.o's state is not backupped # (currently it takes one month to run a full piuparts test on a suite # which produces a gigabyte of logfiles to be saved, for basically # not much value) # . @sharedir@/piuparts/lib/read_config.sh get_config_value SECTIONS global sections get_config_value MASTER global master-directory get_config_value HTDOCS global output-directory get_config_value BACKUPDIR global backup-directory '' get_config_value TESTING global testing-suite copy_to_backupdir() { if [ -f $MASTER/$1 ]; then cp $MASTER/$1 $BACKUPDIR/$1 elif [ "$(basename $1)" = "submissions.txt" ] && ([ "$(dirname $1)" = "${TESTING}-pu" ] || [ "$(dirname $1)" = "${TESTING}-security" ]) ; then : # during development these suites are created but stay empty else echo "Warning: $MASTER/$1 does not exist." # this produces a mail and that's a feature fi } test -n "$BACKUPDIR" || exit 0 copy_to_backupdir archive_issues.txt copy_to_backupdir bts_stats.txt for SECTION in $SECTIONS ; do mkdir -p $BACKUPDIR/$SECTION copy_to_backupdir $SECTION/submissions.txt copy_to_backupdir $SECTION/counts.txt done piuparts-1.1.1/master-bin/reclassify_bugged.in0000644000000000000000000000234013266771660016336 0ustar #!/bin/sh set -e # Copyright © 2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections OLDPWD=$(pwd) for SECTION in $SECTIONS ; do get_config_value KEEP_BUGGED $SECTION keep-bugged no if [ "$KEEP_BUGGED" = "no" ] && [ -d $MASTER/$SECTION/fail ] && [ -d $MASTER/$SECTION/bugged ]; then cd $MASTER/$SECTION mv bugged/*.log bugged/*.bug fail/ 2>/dev/null mv affected/*.log affected/*.bug fail/ 2>/dev/null cd "$OLDPWD" fi done @sharedir@/piuparts/master/report_newly_bugged_packages piuparts-1.1.1/master-bin/report_stale_reserved_packages.in0000644000000000000000000000323313472727312021111 0ustar #!/bin/sh # Copyright 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory # # find packages which have been reserved for more than one day and unschedule them # DAYS=1 LOGS=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION/reserved || continue find $MASTER/$SECTION/reserved/ -mtime +$DAYS -name "*.log" 2>/dev/null >> $LOGS done if [ -s $LOGS ] ; then LOG_OUTPUT=$(mktemp) echo "Stale reserved packages detected, which have been reserved more then $DAYS days ago!" > $LOG_OUTPUT echo "These packages have been unscheduled." >> $LOG_OUTPUT echo >> $LOG_OUTPUT for package_log in $(cat $LOGS) ; do mv -fv $package_log $(dirname $package_log)/../recycle/ >> $LOG_OUTPUT done publish_logs $LOG_OUTPUT "" $HTDOCS report_stale_reserved_packages fi rm $LOGS piuparts-1.1.1/master-bin/report_untestable_packages.in0000644000000000000000000000417513430261151020242 0ustar #!/bin/sh # Copyright © 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory get_config_value DAYS global reschedule-untestable-days 7 STARTDATE=$(date -u +%s) LOG_OUTPUT=$(mktemp) LOG_PREFIX=$(mktemp) # # find packages which have been in untestable for more than $DAYS days and reschedule them for testing # LOGS=$(mktemp) for SECTION in $SECTIONS ; do # only act on sections with precedence < 100 get_config_value PRECEDENCE $SECTION precedence 1 if [ $PRECEDENCE -gt 99 ] ; then continue ; fi # only act on sections with untestable packages test -d $MASTER/$SECTION/untestable || continue find $MASTER/$SECTION/untestable/ -mtime +$DAYS -name "*.log" 2>/dev/null >> $LOGS done if [ -s $LOGS ] ; then FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "Untestable packages detected, which have been tested more than $DAYS days ago!" echo "These packages have been rescheduled for piuparts testing." echo ) > $LOG_PREFIX ( echo "$(date -u)" echo "Runtime: $RUNTIME" echo for package_log in $(cat $LOGS) ; do rm -fv $package_log | sed "s#$MASTER/##g" | tr -d "'" done echo ) > $LOG_OUTPUT publish_logs $LOG_OUTPUT $LOG_PREFIX $HTDOCS report_untestable_packages fi rm $LOGS piuparts-1.1.1/master-bin/reschedule_oldest_logs.in0000644000000000000000000001503213472727312017372 0ustar #!/bin/sh # Copyright © 2009-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value HTDOCS global output-directory STARTDATE=$(date -u +%s) OUTPUT=$(mktemp) LOG_OUTPUT=$(mktemp) # # reschedule 200 oldest log files, if they are older than 180 days # reschedule 25 oldest fail log files, if they are older than 30 days # delete $reschedule-old-count oldest logfiles, if they are scheduled # for recycling and older than $expire-old-days # and $expire-old-days > $reschedule-old-days # get_config_value EXPIRE_AGE global expire-old-days 0 get_config_value AGE global reschedule-old-days 180 get_config_value COUNT global reschedule-old-count 200 get_config_value EXPIRE_FAIL_AGE global expire-fail-days 0 get_config_value FAIL_AGE global reschedule-fail-days 30 get_config_value FAIL_COUNT global reschedule-fail-count 25 get_config_value AUTO_RESCHEDULE global auto-reschedule yes if [ -n "$*" ]; then SECTIONS="$*" fi RECENT=$(mktemp) touch -d "2 days ago" "$RECENT" is_recent() { local stamp stamp="$1" if [ -f "$stamp" ] && [ "$stamp" -nt "$RECENT" ]; then return 0 fi return 1 } TMPLIST=$(mktemp) list_logs() { __AGE="$1" __COUNT="$2" shift 2 find "$@" -name "*.log" -mtime +$__AGE \ | tac \ | xargs --no-run-if-empty -n99999 -s1999999 ls -dt \ > $TMPLIST tail -n $__COUNT $TMPLIST } TOTAL=0 TOTAL_EXPIRED=0 UNSCHEDULE=0 TOTAL_QUEUED=0 LOGS=$(mktemp) OBSOLETE=$(mktemp) EXPIRED=$(mktemp) UNSORTED=$(mktemp) QUEUED=$(mktemp) OLDPWD=$(pwd) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue cd $MASTER/$SECTION mkdir -p pass fail bugged affected recycle recycle_was_idle= find recycle/ -name '*.log' > $QUEUED if [ ! -s $QUEUED ]; then recycle_was_idle="empty" elif [ -f "recycle.stamp" ] && [ "recycle.stamp" -nt "recycle" ]; then recycle_was_idle="idle" fi # Clean up obsolete rescheduling requests for log in $(find recycle/ -name '*.log' | cut -d"/" -f2) ; do for dir in pass bugged affected fail ; do test ! -e "$dir/$log" || continue 2 done echo "recycle/$log" done | sort > $OBSOLETE # Reschedule old logs >$LOGS >$EXPIRED get_config_value RESCHEDULE $SECTION auto-reschedule $AUTO_RESCHEDULE if [ "$RESCHEDULE" = "yes" ]; then get_config_value _EXPIRE_AGE $SECTION expire-old-days $EXPIRE_AGE get_config_value _AGE $SECTION reschedule-old-days $AGE get_config_value _COUNT $SECTION reschedule-old-count $COUNT get_config_value _EXPIRE_FAIL_AGE $SECTION expire-fail-days $EXPIRE_FAIL_AGE get_config_value _FAIL_AGE $SECTION reschedule-fail-days $FAIL_AGE get_config_value _FAIL_COUNT $SECTION reschedule-fail-count $FAIL_COUNT # FIXME: we ignore bugged here - ptyhon-bts is really the way to go >$UNSORTED case $_EXPIRE_AGE in +*) _EXPIRE_AGE=$(($_AGE$_EXPIRE_AGE)) ;; esac if [ "$_EXPIRE_AGE" -gt "$_AGE" ]; then if [ -n "$recycle_was_idle" ]; then list_logs $_EXPIRE_AGE $_COUNT pass fail affected >> $UNSORTED fi fi case $_EXPIRE_FAIL_AGE in +*) _EXPIRE_FAIL_AGE=$(($_FAIL_AGE$_EXPIRE_FAIL_AGE)) ;; esac if [ "$_EXPIRE_FAIL_AGE" -gt "$_FAIL_AGE" ]; then list_logs $_EXPIRE_FAIL_AGE $_FAIL_COUNT fail affected >> $UNSORTED fi for log in $(sort -u $UNSORTED) ; do # the log needs to be scheduled for recycling before it gets expired test -f "recycle/${log#*/}" && echo "$log" done > $EXPIRED >$UNSORTED if is_recent idle.stamp ; then list_logs $_AGE $_COUNT pass fail affected >> $UNSORTED fi list_logs $_FAIL_AGE $_FAIL_COUNT fail affected >> $UNSORTED for log in $(sort -u $UNSORTED) ; do # skip if already scheduled test -f "recycle/${log#*/}" || echo "$log" done > $LOGS fi find recycle/ -name '*.log' > $QUEUED if [ -s $LOGS ] || [ -s $OBSOLETE ] || [ -s $EXPIRED ] || [ -s $QUEUED ] ; then echo "#########################################################" >> $OUTPUT echo >> $OUTPUT RCOUNT=$(wc -l $LOGS | awk '{ print $1 }') TOTAL=$(($TOTAL + $RCOUNT)) ECOUNT=$(wc -l $EXPIRED | awk '{ print $1 }') TOTAL_EXPIRED=$(($TOTAL_EXPIRED + $ECOUNT)) UCOUNT=$(wc -l $OBSOLETE | awk '{ print $1 }') UNSCHEDULE=$(($UNSCHEDULE + $UCOUNT)) echo "$SECTION: $RCOUNT rescheduled, $ECOUNT expired, $UCOUNT obsolete${recycle_was_idle:+ (recycle-was-${recycle_was_idle})}" >> $OUTPUT if [ -s $LOGS ]; then ls -dtl $(cat $LOGS) >> $OUTPUT ln -f $(cat $LOGS) recycle/ fi if [ -s $EXPIRED ]; then ls -dtl $(cat $EXPIRED) >> $OUTPUT rm -fv $(cat $EXPIRED) >> $OUTPUT fi if [ -s $OBSOLETE ]; then rm -fv $(cat $OBSOLETE) >> $OUTPUT fi find recycle/ -name '*.log' > $QUEUED NUM_QUEUED=$(wc -l < $QUEUED) TOTAL_QUEUED=$(($TOTAL_QUEUED + $NUM_QUEUED)) echo "queued: $NUM_QUEUED$(is_recent idle.stamp || echo " not-idle")" >> $OUTPUT echo >> $OUTPUT fi cd $OLDPWD done rm $RECENT rm $TMPLIST rm $LOGS rm $OBSOLETE rm $EXPIRED rm $UNSORTED rm $QUEUED FINALDATE=$(date -u +%s) RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T) ( echo "$(date -u)" echo "Runtime: $RUNTIME" echo if [ "$TOTAL" -gt "0" ]; then echo "Rescheduled $TOTAL logs." fi if [ "$TOTAL_EXPIRED" -gt "0" ]; then echo "Deleted $TOTAL_EXPIRED expired logs." fi if [ "$UNSCHEDULE" -gt "0" ]; then echo "Cancelled $UNSCHEDULE outdated rescheduling requests." fi if [ "$TOTAL_QUEUED" -gt "0" ]; then echo "Queued logs: $TOTAL_QUEUED" fi echo cat $OUTPUT echo "##################################################################################################################" >> $OUTPUT echo "##################################################################################################################" >> $OUTPUT echo ) > $LOG_OUTPUT publish_logs $LOG_OUTPUT "" $HTDOCS reschedule_oldest_logs rm -f $OUTPUT piuparts-1.1.1/master-bin/reschedule_piuparts_tests.in0000644000000000000000000000565313553616457020164 0ustar #!/bin/sh set -e # Copyright © 2013-2017 Andreas Beckmann (anbe@debian.org) # Copyright © 2018 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . @sharedir@/piuparts/lib/read_config.sh get_config_value MASTER global master-directory get_config_value SECTIONS global sections get_config_value TESTING global testing-suite dryrun="" logfiles="" current="" longterm="" pass="" fail="fail" bugged="" affected="" untestable="untestable" for arg in "$@" do case "$arg" in --pass) shift pass="pass" ;; --affected) shift affected="affected" ;; --bugged) shift bugged="bugged" affected="affected" ;; --current) shift current="yes" ;; --longterm) shift longterm="yes" ;; --dryrun) shift dryrun="yes" ;; --logfiles) shift logfiles="yes" ;; *) break ;; esac done test -n "$1" || exit 1 TOTAL=0 recycle_logfiles() { local log recycle for log in "$@" do recycle="$(dirname "$(dirname "$log")")/recycle" if [ -f "$log" ] && [ ! -f "$recycle/$(basename $log)" ]; then TOTAL=$(( $TOTAL + 1 )) if [ -n "$dryrun" ]; then echo "candidate: $log" else ln -fv "$log" $recycle/ || true fi fi done } if [ -n "$logfiles" ]; then recycle_logfiles "$@" fi if [ -z "$logfiles" ]; then subdirs="$pass $fail $bugged $affected $untestable" pattern=' ('"$1"')\.conf$' # # mark logs matching kpr tag $1 for recycling # OLDPWD=$(pwd) LIST=$(mktemp) for SECTION in $SECTIONS ; do test -d $MASTER/$SECTION || continue go= if [ -z "$current$longterm" ]; then go=yes else case "$SECTION" in # don't catch old*stable22testing *$TESTING*|testing*|*sid*|*experimental*) if [ "$current" = "yes" ]; then go=yes fi ;; *22testing*) if [ "$longterm" = "yes" ]; then go=yes fi ;; esac fi if [ "$go" != "yes" ]; then echo "Skip: $SECTION" continue fi cd $MASTER searchdirs="" for dir in $subdirs ; do searchdirs="$searchdirs $SECTION/$dir" done mkdir -p $searchdirs $SECTION/recycle grep -rlE "$pattern" --include '*.kpr' $searchdirs | sed 's/\.kpr$/.log/' > $LIST recycle_logfiles $(cat $LIST) cd "$OLDPWD" done rm -f $LIST fi if [ "$TOTAL" -gt "0" ]; then if [ -n "$dryrun" ]; then echo "Found $TOTAL candidate logfiles for recycling." else echo "Marked $TOTAL logfiles for recycling." fi fi piuparts-1.1.1/master-bin/rotate_master_logs.in0000644000000000000000000000444613553616457016564 0ustar #!/bin/sh # # Copyright © 2017-2019 Andreas Beckmann (anbe@debian.org) # Copyright © 2018 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # * rotate master logfiles greater than 1 MB # * (delayed) compress rotated master logs # . @sharedir@/piuparts/lib/read_config.sh . @sharedir@/piuparts/lib/write_log.sh get_config_value SECTIONS global sections get_config_value MASTER global master-directory get_config_value HTDOCS global output-directory today=$(date +%Y%m%d) OUTPUT=$(mktemp) cd $MASTER for SECTION in $SECTIONS ; do mlog=$SECTION/master.log rlog=$SECTION/master.$today.log # rotate master.log larger than 1 MB if [ -f $mlog ] && [ ! -L $mlog ] && [ $(stat -c %s $mlog) -gt 1048576 ] && [ ! -e $rlog ]; then mv -v $mlog $rlog >> $OUTPUT fi # delayed compression of old master.log for log in $SECTION/master.*.log do if [ -f "$log" ] && [ "$log" != "$rlog" ]; then xz -v9 $log >>$OUTPUT 2>&1 fi done # delete master logs older than 90 days, but keep at least five old logs oldlogs=$(ls -t1 $SECTION/master.*.log.* 2>/dev/null | tail -n +5) if [ -n "$oldlogs" ]; then find $oldlogs -mtime +90 | xargs -r rm -fv >>$OUTPUT 2>&1 fi done LOGDATEDIR="$HTDOCS/logs/$(date -u '+%Y/%m/%d' -d '7 days ago')" if [ -d $LOGDATEDIR ] && [ -n "$(find $LOGDATEDIR -type f ! -name '*.xz')" ] ; then echo >> $OUTPUT echo "Compressing old piuparts.d.o logfiles." >> $OUTPUT echo >> $OUTPUT find $LOGDATEDIR -type f ! -name '*.xz' -exec xz -v9 {} \; >>$OUTPUT 2>&1 fi if [ -s $OUTPUT ] ; then LOG_OUTPUT=$(mktemp) echo "$(date -u)" > $LOG_OUTPUT echo >> $LOG_OUTPUT cat $OUTPUT >> $LOG_OUTPUT echo >> $LOG_OUTPUT publish_logs $LOG_OUTPUT "" $HTDOCS rotate_master_logs fi rm $OUTPUT piuparts-1.1.1/nosetests.cfg0000644000000000000000000000035113266762112012764 0ustar [nosetests] # create test coverage report with-coverage=1 with-coverage=1 cover-html=1 cover-html-dir=build/html cover-package=piupartslib,piuparts #cover-inclusive=1 # run doctests with-doctest=1 # catch all tests # all-modules=1 piuparts-1.1.1/piuparts-analyze.py0000644000000000000000000003316613605426771014155 0ustar #!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright 2011 Mika Pflüger (debian@mikapflueger.de) # Copyright © 2012-2017 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see """Analyze failed piuparts logs and move them around when the errors are known. This program looks at piuparts log files in ./fail, and queries the bts to find out if bugs have been filed already. If so, it moves them to ./bugged. It tries to detect if new versions of bugged packages are uploaded without solving the bug and will then update the bts with the new found versions, and copy the headers of the log in ./fail to the one in ./bugged and vice versa. It will then move the failed log to ./bugged as well. """ from __future__ import print_function import os import sys import time import re import shutil import subprocess import fcntl import debianbts import apt_pkg from signal import alarm, signal, SIGALRM from collections import deque import piupartslib.conf from piupartslib.conf import MissingSection CONFIG_FILE = "/etc/piuparts/piuparts.conf" apt_pkg.init_system() error_pattern = re.compile(r"(?<=\n).*error.*\n?", flags=re.IGNORECASE) chroot_pattern = re.compile(r"tmp/tmp.*?'") ############################################################################ class BTS_Timeout(Exception): pass def alarm_handler(signum, frame): raise BTS_Timeout class PiupartsBTS(): def __init__(self): self._bugs_usertagged_piuparts = None self._bugs_in_package = dict() self._bugs_affecting_package = dict() self._bug_versions = dict() self._queries = 0 self._misses = 0 def all_piuparts_bugs(self): if self._bugs_usertagged_piuparts is None: self._bugs_usertagged_piuparts = debianbts.get_usertag("debian-qa@lists.debian.org", ['piuparts'])['piuparts'] return self._bugs_usertagged_piuparts def bugs_in(self, package): if not package in self._bugs_in_package: self._misses += 1 signal(SIGALRM, alarm_handler) alarm(120) bugs = debianbts.get_bugs(package=package, bugs=self.all_piuparts_bugs(), archive='both') bugs += debianbts.get_bugs(package='src:' + package, bugs=self.all_piuparts_bugs(), archive='both') alarm(0) self._bugs_in_package[package] = sorted(set(bugs), reverse=True) self._queries += 1 return self._bugs_in_package[package] def bugs_affecting(self, package): if not package in self._bugs_affecting_package: self._misses += 1 signal(SIGALRM, alarm_handler) alarm(120) bugs = debianbts.get_bugs(affects=package, bugs=self.all_piuparts_bugs(), archive='both') bugs += debianbts.get_bugs(affects='src:' + package, bugs=self.all_piuparts_bugs(), archive='both') alarm(0) self._bugs_affecting_package[package] = sorted(set(bugs), reverse=True) self._queries += 1 return self._bugs_affecting_package[package] def bug_versions(self, bug): """Gets a list of only the version numbers for which the bug is found. Newest versions are returned first.""" # debianbts returns it in the format package/1.2.3 or 1.2.3 which will become 1.2.3 if not bug in self._bug_versions: self._misses += 1 signal(SIGALRM, alarm_handler) alarm(60) found_versions = debianbts.get_status([bug])[0].found_versions alarm(0) versions = [] for found_version in found_versions: v = found_version.rsplit('/', 1)[-1] if v == "None": # ignore $DISTRO/None versions pass else: versions.append(v) self._bug_versions[bug] = list(reversed(sorted(versions, cmp=apt_pkg.version_compare))) or ['~'] self._queries += 1 return self._bug_versions[bug] def print_stats(self): print("PiupartsBTS: %d queries, %d forwarded to debianbts" % (self._queries, self._misses)) piupartsbts = PiupartsBTS() ############################################################################ class Busy(Exception): def __init__(self): self.args = "section is locked by another process", class Config(piupartslib.conf.Config): def __init__(self, section="global", defaults_section=None): self.section = section piupartslib.conf.Config.__init__(self, section, { "sections": "report", "master-directory": ".", }, defaults_section=defaults_section) def find_logs(directory): """Returns list of logfiles sorted by age, newest first.""" logs = [os.path.join(directory, x) for x in os.listdir(directory) if x.endswith(".log")] return [y[1] for y in reversed(sorted([(os.path.getmtime(x), x) for x in logs]))] def find_bugged_logs(failed_log): package = package_name(failed_log) pat = "/" + package + "_" return [x for x in find_logs("bugged") + find_logs("affected") if pat in x] def package_name(log): return os.path.basename(log).split("_", 1)[0] def package_version(log): return os.path.basename(log).split("_", 1)[1].rstrip('.log') def package_source_version(log): version = package_version(log) possible_binnmu_part = version.rsplit('+', 1)[-1] if possible_binnmu_part.startswith('b') and possible_binnmu_part[1:].isdigit(): # the package version contains a binnmu-part which is not part of the source version # and therefore not accepted/tracked by the bts. Remove it. version = version.rsplit('+', 1)[0] return version def extract_errors(log): """This pretty stupid implementation is basically just 'grep -i error', and then removing the timestamps and the name of the chroot and the package version itself.""" with open(log, "r") as f: data = f.read() whole = '' pversion = package_version(log) for match in error_pattern.finditer(data): text = match.group() # Get rid of timestamps if text[:1].isdigit(): text = text.split(" ", 1)[1] # Get rid of chroot names if 'tmp/tmp' in text: text = re.sub(chroot_pattern, "chroot'", text) # Get rid of the package version text = text.replace(pversion, '') whole += text return whole def extract_headers(log): with open(log, "r") as f: data = f.read() headers = [] headers = data.partition("\nExecuting:")[0] if headers and not headers.endswith("\n"): headers += "\n" return headers def prepend_to_file(filename, data): with open(filename, "r") as f: old_data = f.read() with open(filename + ".tmp", "w") as f: f.write(data) f.write(old_data) shutil.copymode(filename, filename + ".tmp") os.rename(filename, filename + "~") os.rename(filename + ".tmp", filename) os.remove(filename + "~") def write_bug_file(failed_log, bugs): if bugs: with open(os.path.splitext(failed_log)[0] + '.bug', "w") as f: for bug in bugs: f.write('#%s\n' % (bug, bug)) def move_to_bugged(failed_log, bugged="bugged", bug=None): print("Moving %s to %s (#%s)" % (failed_log, bugged, bug)) os.rename(failed_log, os.path.join(bugged, os.path.basename(failed_log))) if bug is not None: write_bug_file(os.path.join(bugged, os.path.basename(failed_log)), [bug]) def mark_bugged_version(failed_log, bugged_log): """Copies the headers from the old log to the new log and vice versa and moves the new log to bugged. Removes the old log in bugged.""" bugged_headers = extract_headers(bugged_log) failed_headers = extract_headers(failed_log) prepend_to_file(failed_log, bugged_headers) prepend_to_file(bugged_log, failed_headers) move_to_bugged(failed_log) def bts_update_found(bugnr, newversion): if "DEBEMAIL" in os.environ and os.environ["DEBEMAIL"]: # subprocess.check_call(('bts', 'found', bugnr, newversion)) print(' '.join(('bts', 'found', str(bugnr), newversion))) def mark_logs_with_reported_bugs(): for failed_log in find_logs("fail") + find_logs("untestable"): try: pname = package_name(failed_log) pversion = package_source_version(failed_log) try: failed_errors = extract_errors(failed_log) except IOError: print('IOError while processing %s' % failed_log) continue moved = False abugs = piupartsbts.bugs_affecting(pname) bugs = piupartsbts.bugs_in(pname) for bug in abugs + bugs: if moved: break if bug in abugs: bugged = "affected" else: bugged = "bugged" found_versions = piupartsbts.bug_versions(bug) if pversion in found_versions: move_to_bugged(failed_log, bugged, bug) moved = True break for bug_version in found_versions: # print('DEBUG: %s/%s #%d %s' % (pname, pversion, bug, bug_version)) if apt_pkg.version_compare(pversion, bug_version) > 0: # pversion > bug_version bugged_logs = find_bugged_logs(failed_log) if not bugged_logs and not moved: print('%s/%s: Maybe the bug was filed earlier: https://bugs.debian.org/%d against %s/%s' % (pname, pversion, bug, pname, bug_version)) break for bugged_log in bugged_logs: old_pversion = package_source_version(bugged_log) bugged_errors = extract_errors(bugged_log) if (apt_pkg.version_compare(old_pversion, bug_version) == 0 # old_pversion == bug_version and failed_errors == bugged_errors): # a bug was filed for an old version of the package, # and the errors were the same back then - assume it is the same bug. if not moved: mark_bugged_version(failed_log, bugged_log) moved = True bts_update_found(bug, pversion) break if not moved: write_bug_file(failed_log, abugs + bugs) except KeyboardInterrupt: raise except: print('ERROR processing %s' % failed_log) print(sys.exc_info()[0]) alarm(0) def main(): conf = Config() conf.read(CONFIG_FILE) master_directory = conf["master-directory"] if len(sys.argv) > 1: sections = sys.argv[1:] else: sections = conf['sections'].split() with open(os.path.join(master_directory, "analyze.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: if sys.stdout.isatty(): sys.exit("another piuparts-analyze process is already running") else: sys.exit(0) todo = deque([(s, 0) for s in sections]) while len(todo): (section_name, next_try) = todo.popleft() now = time.time() if (now < next_try): print("Sleeping while section is busy") time.sleep(max(30, next_try - now) + 30) print(time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime())) print("%s:" % section_name) try: section_directory = os.path.join(master_directory, section_name) if not os.path.exists(section_directory): raise MissingSection("", section_name) with open(os.path.join(section_directory, "master.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: raise Busy() oldcwd = os.getcwd() os.chdir(section_directory) mark_logs_with_reported_bugs() os.chdir(oldcwd) except Busy: print("Section is busy") todo.append((section_name, time.time() + 300)) except MissingSection as e: print("Configuration Error in section '%s': %s" % (section_name, e)) print("") print(time.strftime("%a %b %2d %H:%M:%S %Z %Y", time.localtime())) piupartsbts.print_stats() if __name__ == "__main__": main() # vi:set et ts=4 sw=4 : piuparts-1.1.1/piuparts-master-backend.py0000644000000000000000000003525413605426771015372 0ustar #!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2011-2017 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see """Distributed piuparts processing, master program Lars Wirzenius """ import sys import logging import os import fcntl import time import random import piupartslib.conf import piupartslib.packagesdb from piupartslib.packagesdb import LogfileExists from piupartslib.conf import MissingSection from six.moves.urllib.error import URLError CONFIG_FILE = "/etc/piuparts/piuparts.conf" DISTRO_CONFIG_FILE = "/etc/piuparts/distros.conf" log_handler = None def setup_logging(log_level, log_file_name): logger = logging.getLogger() global log_handler logger.removeHandler(log_handler) if log_file_name: log_handler = logging.FileHandler(log_file_name) else: log_handler = logging.StreamHandler(sys.stderr) logger.addHandler(log_handler) logger.setLevel(log_level) def timestamp(): return time.strftime("[%Y-%m-%d %H:%M:%S]") class Config(piupartslib.conf.Config): def __init__(self, section="master", defaults_section=None): piupartslib.conf.Config.__init__(self, section, { "log-file": None, "master-directory": ".", "proxy": None, "mirror": None, "distro": None, "area": None, "arch": None, "upgrade-test-distros": None, "depends-sections": None, }, defaults_section=defaults_section) class CommandSyntaxError(Exception): def __init__(self, msg): self.args = msg, class ProtocolError(Exception): def __init__(self): self.args = "EOF, missing space in long part, or other protocol error", class Protocol: def __init__(self, myinput, output): self._input = myinput self._output = output def _readline(self): line = self._input.readline() logging.debug(">> " + line.rstrip()) return line def _writeline(self, line): logging.debug("<< " + line) self._output.write(line + "\n") self._output.flush() def _short_response(self, *words): self._writeline(" ".join(words)) def _read_long_part(self): lines = [] while True: line = self._input.readline() if not line: raise ProtocolError() if line == ".\n": break if line[0] != " ": raise ProtocolError() lines.append(line[1:]) return "".join(lines) class Master(Protocol): def __init__(self, myinput, output): Protocol.__init__(self, myinput, output) self._commands = { "section": self._switch_section, "recycle": self._recycle, "idle": self._idle, "status": self._status, "reserve": self._reserve, "unreserve": self._unreserve, "pass": self._pass, "fail": self._fail, "untestable": self._untestable, # debug commands, unstable and undocumented interface "_state": self._state, "_depends": self._depends, "_recursive-depends": self._recursive_depends, "_depcycle": self._depcycle, "_list": self._list, } self._section = None self._lock = None self._writeline("hello") def _init_section(self, section): if self._lock: self._lock.close() # clear all settings from a previous section and set defaults self._section = None self._lock = None self._recycle_mode = False self._idle_mode = None self._idle_stamp = os.path.join(section, "idle.stamp") self._package_databases = None self._binary_db = None config = Config(section=section, defaults_section="global") try: config.read(CONFIG_FILE) except MissingSection: return False if not os.path.exists(section): os.makedirs(section) self._lock = open(os.path.join(section, "master.lock"), "w") try: fcntl.flock(self._lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: return False self._section = section logging.debug(timestamp() + " switching logfile") logfile = config["log-file"] or os.path.join(section, "master.log") setup_logging(logging.DEBUG, logfile) logging.debug(timestamp() + " connected") # start with a dummy _binary_db (without Packages file), sufficient # for submitting finished logs self._binary_db = piupartslib.packagesdb.PackagesDB(prefix=section) return True def _init_db(self): if self._package_databases is not None: return self._package_databases = {} self._load_package_database(self._section) self._binary_db = self._package_databases[self._section] def _load_package_database(self, section): if section in self._package_databases: return config = Config(section=section, defaults_section="global") config.read(CONFIG_FILE) distro_config = piupartslib.conf.DistroConfig(DISTRO_CONFIG_FILE, config["mirror"]) db = piupartslib.packagesdb.PackagesDB(prefix=section) if self._recycle_mode and self._section == section: db.enable_recycling() self._package_databases[section] = db if config["depends-sections"]: deps = config["depends-sections"].split() for dep in deps: self._load_package_database(dep) db.set_dependency_databases([self._package_databases[dep] for dep in deps]) db.load_packages_urls( distro_config.get_packages_urls( config.get_distro(), config.get_area(), config.get_arch())) if config.get_distro() != config.get_final_distro(): # take version numbers (or None) from final distro db.load_alternate_versions_from_packages_urls( distro_config.get_packages_urls( config.get_final_distro(), config.get_area(), config.get_arch())) def _clear_idle(self): if not self._idle_mode is False: self._idle_mode = False if os.path.exists(self._idle_stamp): os.unlink(self._idle_stamp) def _set_idle(self): if not self._idle_mode is True: self._idle_mode = True open(self._idle_stamp, "w").close() os.utime(self._idle_stamp, (-1, self._binary_db._stamp)) def _get_idle_status(self): """ Returns number of seconds a cached idle status is still valid, or 0 if not known to be idle. """ if not os.path.exists(self._idle_stamp): return 0 stamp_mtime = os.path.getmtime(self._idle_stamp) ttl = stamp_mtime + 3600 - time.time() if ttl <= 0: return 0 # stamp expired if stamp_mtime < self._binary_db.get_mtime(): return 0 # stamp outdated return ttl + random.randrange(120) def do_transaction(self): line = self._readline() if line: parts = line.split() if len(parts) > 0: command = parts[0] args = parts[1:] if self._section is None and command != "section": raise CommandSyntaxError("Expected 'section' command, got %s" % command) if command in self._commands: self._commands[command](command, args) return True else: raise CommandSyntaxError("Unknown command %s" % command) return False def _check_args(self, count, command, args): if len(args) != count: raise CommandSyntaxError("Need exactly %d args: %s %s" % (count, command, " ".join(args))) def dump_pkgs(self): for st in self._binary_db.get_states(): for name in self._binary_db.get_pkg_names_in_state(st): logging.debug("%s : %s\n" % (st, name)) def _switch_section(self, command, args): self._check_args(1, command, args) if self._init_section(args[0]): self._short_response("ok") elif self._lock is None: # unknown section self._short_response("error") else: self._short_response("busy") def _recycle(self, command, args): self._check_args(0, command, args) if self._binary_db.enable_recycling(): self._idle_stamp = os.path.join(self._section, "recycle.stamp") self._recycle_mode = True self._short_response("ok") else: self._short_response("error") def _idle(self, command, args): self._check_args(0, command, args) self._short_response("ok", "%d" % self._get_idle_status()) def _status(self, command, args): self._check_args(0, command, args) self._init_db() stats = "" if self._binary_db._recycle_mode: stats += "(recycle) " total = 0 for state in self._binary_db.get_active_states(): count = len(self._binary_db.get_pkg_names_in_state(state)) total += count stats += "%s=%d " % (state, count) stats += "total=%d" % total self._short_response("ok", stats) def _reserve(self, command, args): self._check_args(0, command, args) self._init_db() package = self._binary_db.reserve_package() if package is None: self._set_idle() self._short_response("error") else: self._clear_idle() self._short_response("ok", package.name(), package.test_versions()) def _unreserve(self, command, args): self._check_args(2, command, args) self._binary_db.unreserve_package(args[0], args[1]) self._short_response("ok") def _pass(self, command, args): self._check_args(2, command, args) log = self._read_long_part() try: self._binary_db.pass_package(args[0], args[1], log) except LogfileExists: logging.info("Ignoring duplicate submission: %s %s %s" % ("pass", args[0], args[1])) self._short_response("ok") def _fail(self, command, args): self._check_args(2, command, args) log = self._read_long_part() try: self._binary_db.fail_package(args[0], args[1], log) except LogfileExists: logging.info("Ignoring duplicate submission: %s %s %s" % ("fail", args[0], args[1])) self._short_response("ok") def _untestable(self, command, args): self._check_args(2, command, args) log = self._read_long_part() try: self._binary_db.make_package_untestable(args[0], args[1], log) except LogfileExists: logging.info("Ignoring duplicate submission: %s %s %s" % ("untestable", args[0], args[1])) self._short_response("ok") # debug command def _state(self, command, args): self._check_args(1, command, args) self._short_response("ok", self._binary_db.get_package_state(args[0]), self._binary_db.get_test_versions(args[0])) # debug command def _depends(self, command, args): self._check_args(1, command, args) if self._binary_db.has_package(args[0]): package = self._binary_db.get_package(args[0]) self._short_response("ok", *package.dependencies()) else: self._short_response("error") # debug command def _recursive_depends(self, command, args): self._check_args(1, command, args) if self._binary_db.has_package(args[0]): package = self._binary_db.get_package(args[0]) self._short_response("ok", *self._binary_db._get_recursive_dependencies(package)) else: self._short_response("error") # debug command def _depcycle(self, command, args): self._check_args(1, command, args) if self._binary_db.has_package(args[0]): self._short_response("ok", *self._binary_db._get_dependency_cycle(args[0])) else: self._short_response("error") # debug command def _list(self, command, args): self._check_args(1, command, args) if args[0] in self._binary_db.get_states(): self._short_response("ok", *self._binary_db.get_pkg_names_in_state(args[0])) else: self._short_response("error") def main(): setup_logging(logging.INFO, None) global_config = Config(section="global") global_config.read(CONFIG_FILE) if global_config["proxy"]: os.environ["http_proxy"] = global_config["proxy"] master_directory = global_config["master-directory"] if not os.path.exists(master_directory): os.makedirs(master_directory) os.chdir(master_directory) m = Master(sys.stdin, sys.stdout) try: while m.do_transaction(): pass except URLError as e: logging.error("ABORT: URLError: " + str(e.reason)) except BrokenPipeError: logging.error("ABORT: BrokenPipeError") logging.debug(timestamp() + " disconnected") # https://docs.python.org/3/library/signal.html#note-on-sigpipe devnull = os.open(os.devnull, os.O_WRONLY) os.dup2(devnull, sys.stdout.fileno()) sys.exit(1) logging.debug(timestamp() + " disconnected") if __name__ == "__main__": main() # vi:set et ts=4 sw=4 : piuparts-1.1.1/piuparts-master.in0000755000000000000000000000233613266771660013764 0ustar #!/bin/sh set -e # Copyright © 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see # # wrapper for running piuparts-master-backend without needing to know # anything about the master setup and paths # . @sharedir@/piuparts/lib/read_config.sh get_config_value MASTER global master-directory get_config_value PYTHONPATH global PYTHONPATH '' get_config_value LOGFILE global log-file master-error.log export PYTHONPATH LOGFILE="$LOGFILE.$$" # put logfile in a deterministic location cd "$MASTER" trap "test -s ${LOGFILE} || rm -f ${LOGFILE}" EXIT timeout 15m @sharedir@/piuparts/piuparts-master-backend 2> "$LOGFILE" piuparts-1.1.1/piuparts-report.py0000644000000000000000000023322013605426771014016 0ustar #!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright 2009-2019 Holger Levsen (holger@layer-acht.org) # Copyright © 2011-2018 Andreas Beckmann (anbe@debian.org) # Copyright 2013 David Steele (dsteele@gmail.com) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see """Create HTML reports of piuparts log files Lars Wirzenius """ import os import sys import time import logging import shutil import re import string import yaml import hashlib import pickle import random import fcntl from collections import deque # if python-rpy2 ain't installed, we don't draw fancy graphs try: from rpy2 import robjects from rpy2.robjects.packages import importr except: pass import piupartslib.conf import piupartslib.packagesdb from piupartslib.conf import MissingSection from piupartslib.dwke import * import piupartslib.pkgsummary as pkgsummary from six.moves.urllib.error import HTTPError, URLError CONFIG_FILE = "/etc/piuparts/piuparts.conf" DISTRO_CONFIG_FILE = "/etc/piuparts/distros.conf" KPR_DIRS = ('pass', 'bugged', 'affected', 'fail') TPL_EXT = '.tpl' PIUPARTS_VERSION = "__PIUPARTS_VERSION__" HTML_HEADER = """ $page_title
$section_navigation
General information
About
News
FAQ
Contact us
Documentation
How to file bugs
using templates
Debian policy
piuparts.d.o configuration:
piuparts.conf,
distros.conf,
scripts and logs
README
README_server
piuparts manpage
Summaries
Bugs filed
Suites overview
src: piuparts
Source
piuparts.d.o bugs
piuparts bugs / ToDo
Other Debian QA efforts
Debian QA Group
Dose tools (former: EDOS)
Lintian
Debian Package Tracker
Ultimate Debian Database
jenkins.debian.net
ci.debian.net
Last update
$time
""" HTML_FOOTER = """
""" LOG_LIST_BODY_TEMPLATE = """ $logrows
$title in $section
$preface The list has $count packages.
""" STATE_BODY_TEMPLATE = """
Packages in state "$state" in $section $aside
    $list
""" SECTION_INDEX_BODY_TEMPLATE = """ $tablerows
$section statistics
$description
Binary packages per state
URL to Packages file
$packagesurl
""" MAINTAINER_BODY_TEMPLATE = """ $distrolinks $rows
$maintainer
""" BASIC_BODY_TEMPLATE = """ $rows
""" PROB_TPL = \ """ $HEADER in $SECTION, sorted by reverse dependency count. $HELPTEXT

The commandline to find these logs is: 

COMMAND='$COMMAND'

Please file bugs!
    $PACKAGE_LIST

Affected packages in $SECTION: $COUNT

""" PKG_ERROR_TPL = \ """
  • $RDEPS - $LOG$ARCH (piuparts.d.o) (tracker.d.o) (BTS) $BUG
    • """ title_by_dir = { "pass": "PASSED piuparts logs", "fail": "Failed UNREPORTED piuparts logs", "bugged": "Failed REPORTED piuparts logs", "affected": "Failed AFFECTED piuparts logs", "reserved": "RESERVED packages", "untestable": "UNTESTABLE packages", } desc_by_dir = { "pass": "Log files for packages that have PASSED testing.", "fail": "Log files for packages that have FAILED testing. " + "Bugs have not yet been reported.", "bugged": "Log files for packages that have FAILED testing. " + "Bugs have been reported, but not yet fixed.", "affected": "Log files for packages that have dependencies FAILED testing. " + "Bugs have been reported, but not yet fixed.", "reserved": "Packages that are RESERVED for testing on a node in a " + "distributed piuparts network.", "untestable": "Log files for packages that have are UNTESTABLE with " + "piuparts at the current time.", } state_by_dir = { "pass": "successfully-tested", "fail": "failed-testing", "bugged": "failed-testing", "affected": "failed-testing", "reserved": "waiting-to-be-tested", "untestable": "cannot-be-tested", } # better use XX_name.tpl and get the linktarget from the template # (its a substring of the of the that template # maintaining this list is errorprone and tiresome linktarget_by_template = [ ("initdscript_lsb_header_issue.tpl", "but logfile contains update-rc.d issues"), ("command_not_found_issue.tpl", "but logfile contains 'command not found'"), ("debsums_mismatch_issue.tpl", "but logfile contains modified conffiles or other shipped files"), ("alternatives_after_purge_issue.tpl", "but logfile contains forgotten alternatives"), ("owned_files_after_purge_issue.tpl", "but logfile contains owned files existing after purge"), ("unowned_files_after_purge_issue.tpl", "but logfile contains unowned files after purge"), ("maintainer_script_issue.tpl", "but logfile contains maintainer script failures"), ("db_setup_issue.tpl", "but logfile contains failure to setup a database"), ("problems_and_no_force_issue.tpl", "but logfile reports that not enough force was used"), ("installs_over_symlink_issue.tpl", "but package installs something over existing symlinks"), ("broken_symlinks_issue.tpl", "but logfile contains 'broken symlinks'"), ("unknown_inadequate_issue.tpl", "but logfile contains unknown inadequate issues"), ("boring_obsolete_conffile_file_inadequate_issue.tpl", "...and logfile contains tag from adequate 'obsolete-conffile-file'"), ("boring_broken_symlink_file_inadequate_issue.tpl", "...and logfile contains tag from adequate 'broken-symlink-file'"), ("bin_or_sbin_binary_requires_usr_lib_library_inadequate_issue.tpl", "but adequate found a binary in /bin or /sbin that requires a /usr/lib library"), ("incompatible_licenses_inadequate_issue.tpl", "but adequate found a license incompatibility"), ("broken_binfmt_detector_inadequate_issue.tpl", "but adequate did not find the detector registered with update_binfmts"), ("broken_binfmt_interpreter_inadequate_issue.tpl", "but adequate did not find the interpreter registered with update_binfmts"), ("missing_alternative_inadequate_issue.tpl", "but adequate found a missing alternative"), ("missing_copyright_file_inadequate_issue.tpl", "but adequate couldn't find a copyright file"), ("missing_pkgconfig-dependency_issue.tpl", "but adequate found a missing pkgconfig dependency"), ("program_name_collision_inadequate_issue.tpl", "but adequate found a program name collision"), ("py_file_not_bytecompiled_inadequate_issue.tpl", "but adequate found a .py file that is not byte-compiled"), ("pyshared_file_not_bytecompiled_inadequate_issue.tpl", "but adequate found a .py file in /usr/share/pyshared that is not byte-compiled"), ("ldd_inadequate_issue.tpl", "but adequate failed to run ldd on a binary or library"), ("library_not_found_inadequate_issue.tpl", "but adequate couldn't find a required library"), ("undefined_symbol_inadequate_issue.tpl", "but adequate found an undefined symbol"), ("symbol-size-mismatch_inadequate_issue.tpl", "but adequate found that a symbol has changed size since the package was built"), ("missing-symbol-version-information_inadequate_issue.tpl", "but adequate found that a library is missing symbol version information"), ("unknown_inadequate_issue.tpl", "but an unknown adequate issue was found"), ("inadequate_exit_issue.tpl", "but adequate exited inadequately"), ("packages_have_been_kept_back_issue.tpl", "but logfile contains 'packages have been kept back'"), ("needs_rebuild_issue.tpl", "but logfile recommends to rebuild some packages"), ("module_build_error_issue.tpl", "but logfile contains dkms module build failures"), ("obsolete_conffiles_issue.tpl", "but logfile reports obsolete conffiles"), ("missing_md5sums_issue.tpl", "but logfile reports missing md5sums"), ("unowned_lib_symlink_issue.tpl", "but logfile reports unowned lib symlinks"), ("piuparts-depends-dummy_issue.tpl", "but logfile reports piuparts-depends-dummy.deb could not be installed"), ("used_exception_issue.tpl", "but package used a piuparts exception"), ("test_was_skipped_issue.tpl", "but package test was skipped"), ("dependency_error.tpl", "due to unsatisfied dependencies"), ("packages_have_been_kept_back_error.tpl", "...and logfile also contains 'packages have been kept back'"), ("command_not_found_error.tpl", "due to a 'command not found' error"), ("files_in_usr_local_error.tpl", "due to files in /usr/local"), ("overwrite_other_packages_files_error.tpl", "due to overwriting other packages files"), ("debsums_mismatch_error.tpl", "due to modifying conffiles or other shipped files"), ("alternatives_after_purge_error.tpl", "due to forgotten alternatives after purge"), ("owned_files_by_many_packages_error.tpl", "due to owned files by many packages"), ("owned_files_after_purge_error.tpl", "due to owned files existing after purge"), ("unowned_files_after_purge_error.tpl", "due to unowned files after purge"), ("modified_files_after_purge_error.tpl", "due to files having been modified after purge"), ("disappeared_files_after_purge_error.tpl", "due to files having disappeared after purge"), ("diversion_error.tpl", "due to diversions being modified after purge"), ("processes_running_error.tpl", "due to leaving processes running behind"), ("resource_violation_error.tpl", "due to resource violation"), ("conffile_prompt_error.tpl", "due to prompting due to modified conffiles"), ("db_setup_error.tpl", "due to failing to setup a database"), ("insserv_error.tpl", "due to a problem with insserv"), ("problems_and_no_force_error.tpl", "due to not enough force being used"), ("trigger_cycle_error.tpl", "due to dpkg encountered trigger problems"), ("immediate_configuration_error.tpl", "due to apt could not perform immediate configuration"), ("pre_depends_error.tpl", "due to a problem with pre-depends"), ("pre_installation_script_error.tpl", "due to pre-installation maintainer script failed"), ("post_installation_script_error.tpl", "due to post-installation maintainer script failed"), ("pre_removal_script_error.tpl", "due to pre-removal maintainer script failed"), ("post_removal_script_error.tpl", "due to post-removal maintainer script failed"), ("unknown_purge_error.tpl", "due to purge failed due to an unknown reason"), ("cron_error_after_removal_error.tpl", "due to errors from cronjob after removal"), ("logrotate_error_after_removal_error.tpl", "due to errors from logrotate after removal"), ("installs_over_symlink_error.tpl", "...and package installs something over existing symlinks"), ("broken_symlinks_error.tpl", "...and logfile also contains 'broken symlinks'"), ("module_build_error_error.tpl", "...and logfile contains dkms module build failures"), ("obsolete_conffiles_error.tpl", "...and logfile reports obsolete conffiles"), ("missing_md5sums_error.tpl", "...and logfile reports missing md5sums"), ("unowned_lib_symlink_error.tpl", "...and logfile reports unowned lib symlinks"), ("piuparts-depends-dummy_error.tpl", "...and logfile reports piuparts-depends-dummy.deb could not be installed"), ("unclassified_failures.tpl", "due to unclassified failures"), ] class Config(piupartslib.conf.Config): def __init__(self, section="report", defaults_section=None): self.section = section piupartslib.conf.Config.__init__(self, section, { "sections": "report", "output-directory": "html", "master-directory": ".", "depends-sections": None, "description": "", "proxy": None, "mirror": None, "distro": None, "area": None, "arch": None, "upgrade-test-distros": None, "max-reserved": 1, "doc-root": "/", "known-problem-directory": "@sharedir@/piuparts/known_problems", "exclude-known-problems": None, "json-sections": "default", "precedence": 1, "web-host": "piuparts.debian.org", }, defaults_section=defaults_section) def setup_logging(log_level, log_file_name): logger = logging.getLogger() logger.setLevel(log_level) formatter = logging.Formatter(fmt="%(asctime)s %(message)s", datefmt="%H:%M:%S") handler = logging.StreamHandler(sys.stderr) handler.setFormatter(formatter) logger.addHandler(handler) if log_file_name: handler = logging.FileHandler(log_file_name) logger.addHandler(handler) def html_protect(vstr): vstr = "&".join(vstr.split("&")) vstr = "<".join(vstr.split("<")) vstr = ">".join(vstr.split(">")) vstr = """.join(vstr.split('"')) vstr = "'".join(vstr.split("'")) return vstr def is_bad_state(state): bad_states = [ #"successfully-tested", "failed-testing", "cannot-be-tested", # "essential-required", # obsolete #"waiting-to-be-tested", #"waiting-for-dependency-to-be-tested", "dependency-failed-testing", "dependency-cannot-be-tested", "dependency-does-not-exist", "circular-dependency", # obsolete "unknown", "unknown-preferred-alternative", # obsolete "no-dependency-from-alternatives-exists", # obsolete "outdated", #"foreign:*", "does-not-exist", ] return(state in bad_states) def emphasize_reason(reason): if is_bad_state(reason): reason = "<em>" + reason + "</em>" return reason def source_subdir(source): if source[:3] == "lib": return source[:4] else: return source[:1] def source_summary_url(web_host, doc_root, section, src_pkg): return("https://%s%s/%s/source/%s/%s.html" % ( web_host, doc_root, section, source_subdir(src_pkg), src_pkg, ) ) def maintainer_subdir(maintainer): return maintainer.lower()[:1] def find_files_with_suffix(vdir, suffix): pairs = [] # (mtime, name) for name in os.listdir(vdir): if name.endswith(suffix): try: if os.path.isfile(os.path.join(vdir, name)): mtime = os.path.getmtime(os.path.join(vdir, name)) pairs.append((mtime, name)) except OSError: pass # sort by mtime return [x[1] for x in sorted(pairs)] def update_file(source, target): if os.path.exists(target): try: aa = os.stat(source) bb = os.stat(target) except OSError: pass else: if aa.st_size == bb.st_size and aa.st_mtime <= bb.st_mtime: return try: os.remove(target) except: pass try: os.link(source, target) except OSError: try: shutil.copyfile(source, target) except IOError as xxx_todo_changeme: (errno, strerror) = xxx_todo_changeme.args logging.error("failed to copy %s to %s: I/O error(%d): %s" % (source, target, errno, strerror)) def copy_logs(logs_by_dir, output_dir): for vdir in logs_by_dir: fulldir = os.path.join(output_dir, vdir) if not os.path.exists(fulldir): os.makedirs(fulldir) for basename in logs_by_dir[vdir]: source = os.path.join(vdir, basename) target = os.path.join(fulldir, basename) update_file(source, target) def remove_old_logs(logs_by_dir, output_dir): for vdir in logs_by_dir: fulldir = os.path.join(output_dir, vdir) # convert logs_by_dir array to a dict to avoid linear search logs_dict = {} for log in logs_by_dir[vdir]: logs_dict[log] = 1 if os.path.exists(fulldir): for basename in os.listdir(fulldir): if not basename in logs_dict: os.remove(os.path.join(fulldir, basename)) def create_file(filename, contents): with open(filename, "w") as f: f.write(contents) def append_file(filename, contents): with open(filename, "a") as f: f.write(contents) def read_file(filename): with open(filename, "r") as f: return f.read() def readlines_file(filename): with open(filename, "r") as f: return f.readlines() def fileage(filename): mtime = os.path.getmtime(filename) now = time.time() return now - mtime def write_template_html(filename, body, mapping={}, defer_if_unmodified=False, md5cache=None): header = HTML_HEADER footer = HTML_FOOTER htmlpage = string.Template(header + body + footer) mapping = mapping.copy() mapping.update({ "content_md5": "", "piuparts_version": "", "time": "", }) content_md5 = hashlib.md5(htmlpage.safe_substitute(mapping).encode()).hexdigest() if md5cache is not None: md5cache['new'][filename] = content_md5 if defer_if_unmodified: if filename in md5cache['old'] and md5cache['old'][filename] == content_md5 and os.path.exists(filename): md5cache['unmodified'] += 1 # defer updating the file between 1 and 4 weeks minage = 1 * 7 * 86400 maxage = 4 * 7 * 86400 if fileage(filename) < random.randint(minage, maxage): return md5cache['refreshed'] += 1 if defer_if_unmodified and not filename in md5cache['old'] and os.path.exists(filename): # read the first 10 lines of the old file and look for a matching md5sum with open(filename, "r") as f: lineno = 0 for x in range(10): line = f.readline() if not line: break if line.startswith("<!-- ") and line[5:5 + len(content_md5)] == content_md5: md5cache['unmodified'] += 1 # defer updating the file between 1 and 4 weeks minage = 1 * 7 * 86400 maxage = 4 * 7 * 86400 if fileage(filename) < random.randint(minage, maxage): return md5cache['refreshed'] += 1 break mapping.update({ "content_md5": content_md5, "piuparts_version": PIUPARTS_VERSION, "time": time.strftime("%Y-%m-%d %H:%M %Z"), }) create_file(filename, htmlpage.safe_substitute(mapping)) if md5cache is not None: md5cache['written'] += 1 def create_section_navigation(section_names, current_section, doc_root): tablerows = "<tr class=\"titlerow\"><td class=\"alerttitlecell\">Suite: <a href='%s/%s'>%s</a></td></tr>" \ % (doc_root, current_section, current_section,) tablerows += "<tr><td class=\"contentcell\"><a href=\"%s/%s/maintainer/\">by maintainer / uploader</a></td></tr>\n" \ % (doc_root, current_section) tablerows += "<tr><td class=\"contentcell\"><a href=\"%s/%s/source/\">by source package</a></td></tr>\n" \ % (doc_root, current_section) tablerows += "<tr><td class=\"contentcell\">states <a href=\"%s/%s/states.png/\">graph</a></td></tr>\n" \ % (doc_root, current_section) tablerows += "<tr class=\"titlerow\"><td class=\"alerttitlecell\">all tested suites</td></tr>" for section in section_names: tablerows += ("<tr class=\"normalrow\"><td class=\"contentcell\"><a href='%s/%s'>%s</a></td></tr>\n") % \ (doc_root, html_protect(section), html_protect(section)) return tablerows; def get_email_address(maintainer): email = "INVALID maintainer address: %s" % (maintainer) try: m = re.match(r"(.+)(<)(.+@.+)(>)", maintainer) email = m.group(3) except: pass return email def package2id(package_name): # "+" is not a valid identifier char for id=... attributes return package_name.replace("+", "_") # return order preserving list of the first occurrence of an element def unique(stuff): # can't use set() because 'stuff' is a list of lists and list() is not hashable vlist = [] previtem = stuff for item in stuff: if item != previtem: vlist.append(item) previtem = item return vlist class Busy(Exception): def __init__(self): self.args = "section is locked by another process", class Section: def __init__(self, section, master_directory, doc_root, packagedb_cache={}): self._config = Config(section=section, defaults_section="global") self._config.read(CONFIG_FILE) self._distro_config = piupartslib.conf.DistroConfig( DISTRO_CONFIG_FILE, self._config["mirror"]) logging.debug("-------------------------------------------") logging.debug("Running section " + self._config.section) self._section_directory = os.path.abspath(os.path.join(master_directory, self._config.section)) if not os.path.exists(self._section_directory): logging.debug("Warning: %s did not exist, now created. Did you ever let the slave work?" % self._section_directory) os.makedirs(self._section_directory) self._doc_root = doc_root logging.debug("Loading and parsing Packages file") self._packagedb_cache = packagedb_cache self._package_databases = {} self._load_package_database(section, master_directory) self._binary_db = self._package_databases[section] self._source_db = piupartslib.packagesdb.PackagesDB(prefix=self._section_directory) self._source_db.load_packages_urls( self._distro_config.get_sources_urls( self._config.get_distro(), self._config.get_area())) if self._config.get_distro() != self._config.get_final_distro(): # take version numbers (or None) from final distro self._source_db.load_alternate_versions_from_packages_urls( self._distro_config.get_sources_urls( self._config.get_final_distro(), self._config.get_area())) self._log_name_cache = {} self._md5cache = { 'old': {}, 'new': {}, 'written': 0, 'unmodified': 0, 'refreshed': 0 } def _load_package_database(self, section, master_directory): if section in self._package_databases: return elif section in self._packagedb_cache: self._package_databases[section] = self._packagedb_cache[section] return config = Config(section=section, defaults_section="global") config.read(CONFIG_FILE) if not config["depends-sections"]: # this is a base database eligible for caching # only cache the most recent base database self._packagedb_cache.clear() sectiondir = os.path.join(master_directory, section) db = piupartslib.packagesdb.PackagesDB(prefix=sectiondir) self._package_databases[section] = db if config["depends-sections"]: deps = config["depends-sections"].split() for dep in deps: self._load_package_database(dep, master_directory) db.set_dependency_databases([self._package_databases[dep] for dep in deps]) else: # only cache the big base databases that don't have additional dependencies self._packagedb_cache[section] = db db.load_packages_urls( self._distro_config.get_packages_urls( config.get_distro(), config.get_area(), config.get_arch())) if config.get_distro() != config.get_final_distro(): # take version numbers (or None) from final distro db.load_alternate_versions_from_packages_urls( self._distro_config.get_packages_urls( config.get_final_distro(), config.get_area(), config.get_arch())) def _write_template_html(self, filename, body, mapping={}, defer_if_unmodified=False): mapping = mapping.copy() mapping.update({ "section_navigation": self._section_navigation, "doc_root": self._doc_root, "section": html_protect(self._config.section), }) write_template_html(filename, body, mapping, defer_if_unmodified=defer_if_unmodified, md5cache=self._md5cache) def write_log_list_page(self, filename, title, preface, logs): packages = {} for pathname, package, version in logs: packages[package] = packages.get(package, []) + [(pathname, version)] names = sorted(packages.keys()) lines = [] for package in names: versions = [] for pathname, version in packages[package]: cruft = "" bin_pkg = self._binary_db.get_package(package) if self._source_db.has_package(bin_pkg.source()) and \ bin_pkg.source_version() != self._source_db.get_version(bin_pkg.source()): cruft = " [cruft]" versions.append("<a href=\"%s\">%s</a>%s" % (html_protect(pathname), html_protect(version), cruft)) line = "<tr class=\"normalrow\"><td class=\"contentcell2\">%s</td><td class=\"contentcell2\">%s</td></tr>" % \ (html_protect(package), ", ".join(versions)) lines.append(line) if "FAIL" in preface: title_style = "alerttitlecell" else: title_style = "titlecell" self._write_template_html( filename, LOG_LIST_BODY_TEMPLATE, { "page_title": html_protect(title + " in " + self._config.section), "title": html_protect(title), "title_style": title_style, "preface": preface, "count": len(packages), "logrows": "".join(lines), }) def print_by_dir(self, output_directory, logs_by_dir): for vdir in logs_by_dir: vlist = [] for basename in logs_by_dir[vdir]: assert basename.endswith(".log") assert "_" in basename package, version = basename[:-len(".log")].split("_") vlist.append((os.path.join(vdir, basename), package, version)) self.write_log_list_page(os.path.join(output_directory, vdir + ".html"), title_by_dir[vdir], desc_by_dir[vdir], vlist) def find_links_to_logs(self, package_name, dirs, logs_by_dir): links = [] for vdir in dirs: # avoid linear search against log file names by caching in a dict # # this cache was added to avoid a very expensive linear search # against the arrays in logs_by_dir. Note that the use of this cache # assumes that the contents of logs_by_dir is invarient across calls # to find_links_to_logs() # if vdir not in self._log_name_cache: self._log_name_cache[vdir] = {} for basename in logs_by_dir[vdir]: if basename.endswith(".log"): package, version = basename[:-len(".log")].split("_") self._log_name_cache[vdir][package] = version if vdir == "fail": style = " class=\"needs-bugging\"" else: style = "" if package_name in self._log_name_cache[vdir]: basename = package_name \ + "_" \ + self._log_name_cache[vdir][package_name] \ + ".log" links.append("<a href=\"%s/%s\"%s>%s</a>" % ( self._doc_root, os.path.join(self._config.section, vdir, basename), style, html_protect(self._log_name_cache[vdir][package_name]), )) return links def link_to_maintainer_summary(self, maintainer): email = get_email_address(maintainer) return "<a href=\"%s/%s/maintainer/%s/%s.html\">%s</a>" \ % (self._doc_root, self._config.section, maintainer_subdir(email), email, html_protect(maintainer)) def link_to_uploaders(self, uploaders): link = "" for uploader in uploaders.split(","): link += self.link_to_maintainer_summary(uploader.strip()) + ", " return link[:-2] def link_to_source_summary(self, package_name): source_name = self._binary_db.get_package(package_name).source() link = "<a href=\"%s/%s/source/%s\">%s</a>" % ( self._doc_root, self._config.section, source_subdir(source_name) + "/" + source_name + ".html", html_protect(package_name)) return link def link_to_state_page(self, section, package_name, link_target): if self._binary_db.has_package(package_name): state = self._binary_db.get_package_state(package_name) link = "<a href=\"%s/%s/%s\">%s</a>" % ( self._doc_root, section, "state-" + state + ".html" + "#" + package_name, link_target) else: if link_target == package_name: link = html_protect(package_name) else: link = "unknown-package" return link def links_to_logs(self, package_name, state, logs_by_dir): link = "N/A" dirs = "" if state == "successfully-tested": dirs = ["pass"] elif state == "failed-testing": dirs = ["fail", "bugged", "affected"] elif state == "cannot-be-tested": dirs = ["untestable"] if dirs != "": links = self.find_links_to_logs(package_name, dirs, logs_by_dir) link = ", ".join(links) if "/bugged/" in link or "/affected/" in link: link += " - <a href=\"https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=" \ + package_name \ + "\" target=\"_blank\" class=\"bugged\"> bug filed </a>\n" return link def write_counts_summary(self): logging.debug("Writing counts.txt") header = "date" current_day = "%s" % time.strftime("%Y%m%d") counts = current_day total = 0 for state in self._binary_db.get_states(): count = len(self._binary_db.get_pkg_names_in_state(state)) header += ", %s" % state counts += ", %s" % count logging.debug("%s: %s" % (state, count)) total += count logging.debug("total: %s" % total) logging.debug("source: %s" % len(self._source_db.get_all_packages())) header += "\n" counts += "\n" countsfile = os.path.join(self._section_directory, "counts.txt") if not os.path.isfile(countsfile): logging.debug("writing new file: %s" % countsfile) create_file(countsfile, header) last_line = "" else: last_line = readlines_file(countsfile)[-1] if not current_day in last_line: append_file(countsfile, counts) logging.debug("appending line: %s" % counts.strip()) shutil.copy(countsfile, self._output_directory) return total def create_maintainer_summaries(self, maintainers, source_data): logging.debug("Writing %d maintainer summaries" % len(maintainers)) maintainer_dir = os.path.join(self._output_directory, "maintainer") if not os.path.exists(maintainer_dir): os.mkdir(maintainer_dir) states = ["fail", "unknown", "pass"] for maintainer in sorted(maintainers.keys()): sources = maintainers[maintainer] maintainer_subdir_path = os.path.join(maintainer_dir, maintainer_subdir(maintainer)) if not os.path.exists(maintainer_subdir_path): os.mkdir(maintainer_subdir_path) rows = "" package_rows = {} packages = {} for state in states: packages[state] = [] package_rows[state] = "" for source in sorted(sources): (state, sourcerows, binaryrows) = source_data[source] packages[state].append(source) package_rows[state] += sourcerows + binaryrows for state in states: if len(packages[state]) > 0: links = "" for package in packages[state]: links += "<a href=\"#%s\">%s</a> " % (package, package) else: links = " " rows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">%s:</td>" % state \ + "<td class=\"contentcell2\">%s</td>" % len(packages[state]) \ + "<td class=\"contentcell2\" colspan=\"4\">%s</td>" % links \ + "</tr>\n" distrolinks = "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">other distributions: </td>" \ + "<td class=\"contentcell2\" colspan=\"5\">" for section in self._section_names: if section != self._config.section: distrolinks += "<a href=\"" \ + self._doc_root \ + "/" \ + section \ + "/maintainer/" \ + maintainer_subdir(maintainer) \ + "/" \ + maintainer \ + ".html\">" \ + html_protect(section) \ + "</a> " distrolinks += "</td></tr>" self._write_template_html( os.path.join(maintainer_subdir_path, maintainer + ".html"), MAINTAINER_BODY_TEMPLATE, { "page_title": html_protect("Status of " + maintainer + " packages in " + self._config.section), "maintainer": html_protect(maintainer + " in " + self._config.section), "distrolinks": distrolinks, "rows": rows + "".join([package_rows[state] for state in states]), }, defer_if_unmodified=True) def create_source_summary(self, source, logs_by_dir): source_version = self._source_db.get_test_versions(source) binaries = self._source_db.get_control_header(source, "Binary") maintainer = self._source_db.get_control_header(source, "Maintainer") uploaders = self._source_db.get_control_header(source, "Uploaders") success = True failed = False binaryrows = "" for binary in sorted([x.strip() for x in binaries.split(",") if x.strip()]): if not self._binary_db.has_package(binary): # udebs or binary packages for other architectures # The latter is a FIXME which needs parsing the Packages files from other archs too binaryrows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">Binary:</td>" \ + "<td class=\"contentcell2\">%s</td>" \ % binary \ + "<td class=\"contentcell2\" colspan=\"4\">unknown package</td>" \ + "</tr>\n" continue state = self._binary_db.get_package_state(binary, recurse=False) if not "waiting" in state and "dependency" in state: state_style = "lightalertlabelcell" elif state == "failed-testing": state_style = "lightlabelcell" else: state_style = "labelcell" binary_version = self._binary_db.get_test_versions(binary) if self._binary_db.get_package(binary).source_version() != source_version: binary_version += " [cruft]" binary_arch = self._binary_db.get_control_header(binary, "Architecture") archsuffix = "" if binary_arch == "all": archsuffix = ":all" binaryrows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">Binary:</td>" \ + "<td class=\"contentcell2\">%s%s</td>" \ % (binary, archsuffix) \ + "<td class=\"%s\">piuparts-result:</td>" \ % state_style \ + "<td class=\"contentcell2\">%s %s</td>" \ % (self.link_to_state_page(self._config.section, binary, state), self.links_to_logs(binary, state, logs_by_dir)) \ + "<td class=\"labelcell\">Version:</td>" \ + "<td class=\"contentcell2\">%s</td>" \ % html_protect(binary_version) \ + "</tr>\n" if state not in ("successfully-tested", "essential-required"): success = False if state in ("failed-testing", "dependency-does-not-exist", "cannot-be-tested"): failed = True if binaryrows != "": source_state = "unknown" if success: source_state = "<img src=\"%s/images/sunny.png\" alt=\"success\">" % self._doc_root if failed: source_state = "<img src=\"%s/images/weather-severe-alert.png\" alt=\"failed\">" % self._doc_root sourcerows = "<tr class=\"titlerow\">" \ + "<td class=\"titlecell\" colspan=\"6\" id=\"%s\">%s in %s</td>" \ % (package2id(source), source, self._config.section) \ + "</tr>\n" sourcerows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">Source:</td>" \ + "<td class=\"contentcell2\">" \ + "<a href=\"https://tracker.debian.org/%s\" target=\"_blank\">%s</a>" \ % (source, html_protect(source)) \ + "</td>" \ + "<td class=\"labelcell\">piuparts summary:</td>" \ + "<td class=\"contentcell2\">%s</td>" \ % source_state \ + "<td class=\"labelcell\">Version:</td>" \ + "<td class=\"contentcell2\">%s</td>" \ % html_protect(source_version) \ + "</tr>\n" sourcerows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">Maintainer:</td>" \ + "<td class=\"contentcell2\" colspan=\"5\">%s</td>" \ % self.link_to_maintainer_summary(maintainer) \ + "</tr>\n" if uploaders: sourcerows += "<tr class=\"normalrow\">" \ + "<td class=\"labelcell\">Uploaders:</td>" \ + "<td class=\"contentcell2\" colspan=\"5\">%s</td>" \ % self.link_to_uploaders(uploaders) \ + "</tr>\n" source_summary_page_path = os.path.join(self._output_directory, "source", source_subdir(source)) if not os.path.exists(source_summary_page_path): os.makedirs(source_summary_page_path) self._write_template_html( os.path.join(source_summary_page_path, (source + ".html")), BASIC_BODY_TEMPLATE, { "page_title": html_protect("Status of source package " + source + " in " + self._config.section), "rows": sourcerows + binaryrows, }, defer_if_unmodified=True) # return parsable values if success: source_state = "pass" if failed: source_state = "fail" else: source_state = "udeb" sourcerows = "" return sourcerows, binaryrows, source_state, maintainer, uploaders def create_package_summaries(self, logs_by_dir): src_names = sorted(self._source_db.get_all_package_names()) logging.debug("Writing %d source summaries" % len(src_names)) maintainers = {} source_binary_rows = {} sources = "" sources_data = [] for source in src_names: (sourcerows, binaryrows, source_state, maintainer, uploaders) = \ self.create_source_summary(source, logs_by_dir) if source_state != "udeb": source_version = self._source_db.get_test_versions(source) source_data = {}; source_data["source"] = source source_data["state"] = source_state source_data["version"] = source_version sources_data.append(source_data) sources += "%s: %s\n" % (source, source_state) source_binary_rows[source] = (source_state, sourcerows, binaryrows) for maint in [maintainer] + uploaders.split(","): if maint.strip(): email = get_email_address(maint.strip()) if not "INVALID" in email: if not email in maintainers: maintainers[email] = [] maintainers[email].append(source) create_file(os.path.join(self._output_directory, "sources.txt"), sources) create_file(os.path.join(self._output_directory, "sources.yaml"), yaml.dump(sources_data, default_flow_style=False)) self.create_maintainer_summaries(maintainers, source_binary_rows) def make_section_stats_graph(self): countsfile = os.path.join(self._section_directory, "counts.txt") pngfile = os.path.join(self._output_directory, "states.png") grdevices = importr('grDevices') grdevices.png(file=pngfile, width=1600, height=900, pointsize=10, res=100, antialias="none") r = robjects.r states = self._binary_db.get_states() active_states = self._binary_db.get_active_states() r('nstate = %d' % len(states)) r('snames = c(%s)' % ','.join(['"%s"' % s if s in active_states else '"(%s)"' % s for s in states])) r('cnames = c("date",snames)') r('t <- (read.table("' + countsfile + '",sep=",",header=0,skip=1,col.names=cnames,row.names=1,fill=TRUE))') # here we define how many days we wants stats for (163=half a year) # r('v <- t[(nrow(t)-163):nrow(t),0:nstate]') # make graph since day 1 r('v <- t[0:nrow(t),0:nstate]') # thanks to http://tango.freedesktop.org/Generic_Icon_Theme_Guidelines for those nice colors r('palsize = 14') r('palette(c("#4e9a06", "#ef2929", "#d3d7cf", "#5c3566", "#c4a000", \ "#fce94f", "#a40000", "#888a85", "#2e3436", "#729fcf", \ "#3465a4", "#204a87", "#555753", "#ce5c00"))') r('barplot(t(v),col = 1:palsize, \ main="Binary packages per state in ' + self._config.section + '", \ xlab="", ylab="Number of binary packages", space=0, border=NA)') r('legend(x="bottom",legend=snames, ncol=2,fill=1:palsize,xjust=0.5,yjust=0,bty="n")') grdevices.dev_off() stats_html = "<tr class=\"normalrow\"> " \ + "<td class=\"contentcell2\" colspan=\"3\">" \ + "<a href=\"%s\">" \ % "states.png" \ + "<img src=\"%s\" width=\"100%%\" alt=\"Binary package states in %s\">" \ % ("states.png", self._config.section) \ + "</a>" \ + "<br>(<a href=\"counts.txt\">Source</a>)\n" \ + "</td></tr>\n" return stats_html def prepare_known_problem_reports(self, failures): self._problem_reports = {} for problem in self._problem_list: tpl_text = dwke_update_tpl(self._config.section, problem, failures.filtered(problem.name), PKG_ERROR_TPL, PROB_TPL, self._binary_db, self._source_db) if len(tpl_text): self._problem_reports[problem.name[:-5] + TPL_EXT] = tpl_text def create_and_link_to_analysises(self, state): link = "<ul>\n" for template, linktarget in linktarget_by_template: # successful logs only have issues and failed logs only have errors if (state == "failed-testing" and template[-9:] != "issue.tpl") \ or (state == "successfully-tested" and template[-9:] == "issue.tpl"): substats = "" if template in self._problem_reports: logging.debug("Writing analysis page for %s" % template) rows = self._problem_reports[template] self._write_template_html( os.path.join(self._output_directory, template[:-len(".tpl")] + ".html"), BASIC_BODY_TEMPLATE, { "page_title": html_protect("Packages in state " + state + " " + linktarget), "rows": rows, }) if state == "failed-testing": count_bugged = rows.count('"bugged/') count_affected = rows.count('"affected/') count_failed = rows.count('"fail/') sep = ": " if count_bugged > 0: substats += sep + "%s bugged" % count_bugged sep = ", " if count_affected > 0: substats += sep + "%s affected" % count_affected sep = ", " if count_failed > 0: substats += sep + "<span class=\"needs-bugging\">%s failed</span>" % count_failed else: count_passed = rows.count('"pass/') if count_passed > 0: substats += ": %s passed" % count_passed link += "<li><a href=%s>%s</a>%s</li>\n" % \ ( template[:-len(".tpl")] + ".html", linktarget, substats, ) link += "</ul>" if link == "<ul>\n</ul>": link = "" return link def write_section_index_page(self, dirs, total_packages): tablerows = "" for state in self._binary_db.get_active_states(): dir_link = "" analysis = "" for vdir in dirs: if vdir in ("pass", "fail", "bugged", "affected", "untestable") and state_by_dir[vdir] == state: dir_link += "<a href='%s.html'>%s</a> logs<br>" % (vdir, html_protect(vdir)) if state in ("successfully-tested", "failed-testing"): analysis = self.create_and_link_to_analysises(state) if analysis or len(self._binary_db.get_pkg_names_in_state(state)): tablerows += ("<tr class=\"normalrow\"><td class=\"contentcell2\"><a href='state-%s.html'>%s</a>%s</td>" + "<td class=\"contentcell2\">%d</td><td class=\"contentcell2\">%s</td></tr>\n") % \ (html_protect(state), html_protect(state), analysis, len(self._binary_db.get_pkg_names_in_state(state)), dir_link) try: tablerows += self.make_section_stats_graph(); except: logging.debug("Error generating the graph images, probably python-rpy2 is not installed, disabling graphs.") tablerows += "<tr class=\"normalrow\"> <td class=\"labelcell2\">Total</td> <td class=\"labelcell2\" colspan=\"2\">%d</td></tr>\n" % total_packages vendor = "Debian" if len(self._config.get_distros()) > 1: description = "%s %s: package installation in %s" % ( vendor, self._config.get_area(), self._config.get_start_distro()) for distro in self._config.get_distros()[1:]: description += ", dist-upgrade to %s" % distro description += ", removal, and purge test." else: description = "%s %s / %s: package installation, removal, and purge test." % ( vendor, self._config.get_distro(), self._config.get_area()) if self._config["description"].startswith("+"): description = description + " " + self._config["description"][1:] elif self._config["description"].endswith("+"): description = self._config["description"][:-1] + " " + description elif self._config["description"]: description = self._config["description"] self._write_template_html( os.path.join(self._output_directory, "index.html"), SECTION_INDEX_BODY_TEMPLATE, { "page_title": html_protect(self._config.section + " statistics"), "description": html_protect(description), "tablerows": tablerows, "packagesurl": "<br>".join([html_protect(url) for url in self._binary_db.get_urls()]), }) def _show_providers(self, dep): providers = self._binary_db.get_providers(dep) vlist = "" if providers: vlist += "\n<ul>\n" for provider in providers: vlist += "<li>provider %s is %s</li>\n" % \ (self.link_to_state_page(self._config.section, provider, provider), emphasize_reason(html_protect(self._binary_db.get_package_state(provider)))) vlist += "</ul>\n" return vlist def write_state_pages(self): for state in self._binary_db.get_active_states(): logging.debug("Writing page for %s" % state) vlist = "" if state in self._binary_db.get_error_states(): with_counts = True aside = " (reverse deps, blocked pkgs)" sort_key = lambda x: (-self._binary_db.block_count(x["Package"]), x["Package"]) else: with_counts = False aside = "" sort_key = lambda x: x["Package"] names = self._binary_db.get_pkg_names_in_state(state) packages = [self._binary_db.get_package(name) for name in names] packages.sort(key=sort_key) for package in packages: vlist += "<li id=\"%s\">%s" % ( package2id(package["Package"]), self.link_to_source_summary(package["Package"])) if package["Architecture"] == "all": vlist += ":all" if package.source_version() != self._source_db.get_version(package.source()): vlist += " [cruft]" if with_counts: vlist += " (%d, %d)" % (self._binary_db.rrdep_count(package["Package"]), self._binary_db.block_count(package["Package"])) vlist += " (%s)" % html_protect(package["Maintainer"]) all_deps = unique(package.all_dependencies()) if all_deps: vlist += "\n<ul>\n" for alternatives in all_deps: dep = alternatives[0] vlist += "<li>dependency %s is %s" % \ (self.link_to_state_page(self._config.section, dep, dep), emphasize_reason(html_protect(self._binary_db.get_package_state(dep, resolve_virtual=False)))) vlist += self._show_providers(dep) if len(alternatives) > 1: vlist += "\n<ul>\n" for dep in alternatives[1:]: vlist += "<li>alternative dependency %s is %s" % \ (self.link_to_state_page(self._config.section, dep, dep), emphasize_reason(html_protect(self._binary_db.get_package_state(dep, resolve_virtual=False)))) vlist += self._show_providers(dep) vlist += "</li>\n" vlist += "</ul>\n" vlist += "</li>\n" vlist += "</ul>\n" vlist += "</li>\n" self._write_template_html( os.path.join(self._output_directory, "state-%s.html" % state), STATE_BODY_TEMPLATE, { "page_title": html_protect("Packages in state " + state + " in " + self._config.section), "state": html_protect(state), "list": vlist, "aside": aside, }) def archive_logfile(self, vdir, log): archivedir = os.path.join("archive", vdir) if not os.path.exists(archivedir): os.makedirs(archivedir) try: os.rename(os.path.join(vdir, log), os.path.join("archive", vdir, log)) except OSError: logging.debug("OSError while archiving %s/%s" % (vdir, log)) def cleanup_removed_packages(self, logs_by_dir): vdirs = logs_by_dir.keys() vdirs = vdirs - {"reserved"} for vdir in vdirs: for log in sorted(logs_by_dir[vdir]): if log.endswith(".log"): package, version = log[:-len(".log")].split("_") if not self._binary_db.has_package(package): logging.debug("Archiving %s/%s, package was removed" % (vdir, log)) self.archive_logfile(vdir, log) logs_by_dir[vdir].remove(log) else: current = self._binary_db.get_test_versions(package) state = self._binary_db.get_package_state(package, resolve_virtual=False, recurse=False) if version != current: logging.debug("Archiving %s/%s, package is outdated (%s)" % (vdir, log, current)) self.archive_logfile(vdir, log) logs_by_dir[vdir].remove(log) elif state == "outdated": logging.debug("Archiving %s/%s, package is superseded" % (vdir, log)) self.archive_logfile(vdir, log) logs_by_dir[vdir].remove(log) def generate_html(self): md5cachefile = os.path.join(self._output_directory, '.md5cache') try: with open(md5cachefile, "rb") as f: self._md5cache['old'] = pickle.load(f, encoding='utf-8') except (IOError, EOFError): pass logging.debug("Finding log files") dirs = ["pass", "fail", "bugged", "affected", "reserved", "untestable"] logs_by_dir = {} for vdir in dirs: logs_by_dir[vdir] = find_files_with_suffix(vdir, ".log") logging.debug("Archiving logs of obsolete packages") self.cleanup_removed_packages(logs_by_dir) logging.debug("Copying log files") copy_logs(logs_by_dir, self._output_directory) logging.debug("Writing per-dir HTML pages") self.print_by_dir(self._output_directory, logs_by_dir) total_packages = self.write_counts_summary() self.create_package_summaries(logs_by_dir) logging.debug("Preparing known problem reports") failures = dwke_get_failures(self._binary_db, self._problem_list) self.prepare_known_problem_reports(failures) logging.debug("Writing section index page") self.write_section_index_page(dirs, total_packages) logging.debug("Writing state pages") self.write_state_pages() logging.debug("Wrote %d out of %d html files, refreshed %d out of %d unmodified files" % ( \ self._md5cache['written'], len(self._md5cache['new']), self._md5cache['refreshed'], self._md5cache['unmodified'])) with open(md5cachefile, "wb") as f: pickle.dump(self._md5cache['new'], f) logging.debug("Removing old log files") remove_old_logs(logs_by_dir, self._output_directory) def generate_summary(self, web_host): summary_path = os.path.join(self._output_directory, "summary.json") if os.path.isfile(summary_path): os.unlink(summary_path) reporting_sections = self._config['json-sections'].split() if not reporting_sections or reporting_sections[0] == 'default': reporting_sections = [self._config.get_std_distro()] if reporting_sections[0] == 'none': logging.debug("Skipping summary") else: logging.debug("Generating summary") summary = pkgsummary.new_summary() for reporting_section in reporting_sections: for binpkg in self._binary_db.get_all_packages(): pkgname = binpkg["Package"] state = self._binary_db.get_package_state(pkgname) flag = pkgsummary.get_flag(state) block_cnt = 0 if flag == 'F': block_cnt = self._binary_db.block_count(pkgname) srcpkg = self._binary_db.get_package(pkgname).source() url = source_summary_url( web_host, self._doc_root, self._config.section, srcpkg) pkgsummary.add_summary( summary, reporting_section, srcpkg, flag, block_cnt, url) pkgsummary.write_summary(summary, summary_path) def generate_output(self, output_directory, section_names, problem_list, web_host): # skip output generation for disabled sections if int(self._config["max-reserved"]) == 0: return self._section_names = section_names self._section_navigation = create_section_navigation( self._section_names, self._config.section, self._doc_root) self._output_directory = os.path.abspath(os.path.join(output_directory, self._config.section)) if not os.path.exists(self._output_directory): os.makedirs(self._output_directory) self._problem_list = problem_list if self._config['exclude-known-problems']: excluded = self._config['exclude-known-problems'].split() self._problem_list = [p for p in problem_list if p.name not in excluded] oldcwd = os.getcwd() os.chdir(self._section_directory) self.generate_html() os.chdir(oldcwd) self.generate_summary(web_host) def sections_by_precedence(sections): precedence = {} count = 0 for section in sections: config = Config(section=section, defaults_section="global") config.read(CONFIG_FILE) precedence[section] = (config["precedence"], count) count += 1 return sorted(sections, key=lambda x: precedence[x]) def generate_global_summary(dir, sections): json_name = "summary.json" logging.debug("Generating global summary") summary = pkgsummary.new_summary() for section in sections_by_precedence(sections): sec_path = os.path.join(dir, section, json_name) if os.path.isfile(sec_path): sec_summ = pkgsummary.read_summary(sec_path) summary = pkgsummary.merge_summary(summary, sec_summ) summary_path = os.path.join(dir, json_name) pkgsummary.write_summary(summary, summary_path) # START detect_well_known_errors def get_bug_text(logpath): bugpath = replace_ext(logpath, BUG_EXT) if os.path.exists(bugpath): return read_file(bugpath) return "" def populate_tpl(tmpl, vals): for key in vals: tmpl = re.sub("\$%s" % key, str(vals[key]), tmpl) return tmpl def dwke_update_tpl(section, problem, failures, ftpl, ptpl, pkgsdb, srcdb): pkg_text = "" bugged_section = False for failure in failures: bin_pkg = get_pkg(failure.pkgspec) src_pkg = pkgsdb.get_package(bin_pkg).source() rdep_cnt = pkgsdb.rrdep_count(bin_pkg) bin_arch = "" if pkgsdb.get_control_header(bin_pkg, "Architecture") == "all": bin_arch = " [all]" if pkgsdb.get_package(bin_pkg).source_version() != srcdb.get_version(src_pkg): bin_arch += " [cruft]" if bugged_section is False and failure.where != 'fail': bugged_section = True pkg_text += "</ul><ul>\n" log = os.path.join(failure.where, failure.pkgspec + LOG_EXT) pkg_text += populate_tpl(ftpl, { 'LOG': log, 'PACKAGE': bin_pkg, 'ARCH': bin_arch, 'BUG': get_bug_text(log), 'RDEPS': rdep_cnt, 'SPKG': src_pkg, 'SSUBDIR': source_subdir(src_pkg), 'SECTION': section, }) if len(pkg_text): return populate_tpl(ptpl, { 'HEADER': problem.HEADER, 'SECTION': section, 'HELPTEXT': problem.HELPTEXT, 'COMMAND': problem.get_command(), 'PACKAGE_LIST': pkg_text, 'COUNT': len(failures), }) return "" def dwke_get_failures(pkgsdb, problem_list): logdict = get_file_dict(KPR_DIRS, LOG_EXT) kprdict = get_file_dict(KPR_DIRS, KPR_EXT) del_cnt = clean_cache_files(logdict, kprdict) kprdict = get_file_dict(KPR_DIRS, KPR_EXT) add_cnt = make_kprs(logdict, kprdict, problem_list) failures = FailureManager(logdict) failures.sort_by_bugged_and_rdeps(pkgsdb) logging.info("parsed logfiles: %d removed, %d added" % (del_cnt, add_cnt)) for prob in problem_list: pcount = len(failures.filtered(prob.name)) if pcount: logging.info("%7d %s" % (pcount, prob.name)) return failures # END detect_well_known_errors def make_bts_stats_graph(master_dir, out_dir): countsfile = os.path.join(master_dir, "bts_stats.txt") pngfile = os.path.join(out_dir, "images", "bts_stats.png") grdevices = importr('grDevices') grdevices.png(file=pngfile, width=1600, height=900, pointsize=10, res=100) r = robjects.r r('t <- (read.table("' + countsfile + '",sep=",",header=1,row.names=1))') # make graph since day 1 r('v <- t[c(4, 2, 3, 1)]') # reorder columns # tango colors again: r('palette(c("#a40000", "#ef2929", "#4e9a06", "#8ae234"))') r('barplot(t(v),col = 1:4, \ main="Bugs with usertag=piuparts and user=debian-qa@lists.debian.org", \ xlab="", ylab="Total number of RC and non-RC bugs submitted and closed", space=0, border=NA)') r('legend("right", legend=rev(colnames(v)), fill=rev(1:4), inset=0.05, bty="n")') grdevices.dev_off() def main(): setup_logging(logging.DEBUG, None) global_config = Config(section="global") global_config.read(CONFIG_FILE) if global_config["proxy"]: os.environ["http_proxy"] = global_config["proxy"] section_names = global_config["sections"].split() process_section_names = section_names if len(sys.argv) > 1: process_section_names = sys.argv[1:] master_directory = global_config["master-directory"] output_directory = global_config["output-directory"] web_host = global_config["web-host"] doc_root = global_config["doc-root"].strip() if not doc_root.startswith("/"): doc_root = "/" + doc_root if doc_root.endswith("/"): doc_root = doc_root[:-1] problem_list = create_problem_list(global_config['known-problem-directory']) if not os.path.exists(master_directory): logging.debug("Warning: %s does not exist!?! Creating it for you now." % master_directory) os.makedirs(master_directory) return with open(os.path.join(master_directory, "report.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: if sys.stdout.isatty(): sys.exit("another piuparts-report process is already running") else: sys.exit(0) packagedb_cache = {} create_file(os.path.join(output_directory, "sections.yaml"), yaml.dump(section_names, default_flow_style=False)) todo = deque([(s, 0) for s in process_section_names]) while len(todo): (section_name, next_try) = todo.popleft() now = time.time() if (now < next_try): logging.info("Sleeping while section is busy") time.sleep(max(30, next_try - now) + 30) try: section_directory = os.path.join(master_directory, section_name) if not os.path.exists(section_directory): raise MissingSection("", section_name) with open(os.path.join(section_directory, "master.lock"), "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: raise Busy() section = Section(section_name, master_directory, doc_root, packagedb_cache=packagedb_cache) section.generate_output(output_directory, section_names, problem_list, web_host) except Busy: logging.info("Section is busy") todo.append((section_name, time.time() + 300)) except MissingSection as e: logging.error("Configuration Error in section '%s': %s" % (section_name, e)) except (HTTPError, URLError) as e: logging.error("Failed to fetch Packages files for section '%s' : %s" % (section_name, e)) # static pages logging.debug("Writing static pages") section_navigation = create_section_navigation(section_names, "sid", doc_root) for page in ("index", "news", "bug_howto"): tpl = os.path.join(output_directory, page + ".tpl") INDEX_BODY = read_file(tpl) if page == "index": page_title = "About piuparts.debian.org" elif page == "news": page_title = "News about piuparts.debian.org" else: page_title = "How to file bugs based on piuparts.d.o results" write_template_html( os.path.join(output_directory, page + ".html"), INDEX_BODY, { "page_title": page_title, "section_navigation": section_navigation, "doc_root": doc_root, }) # overview page rows = "<tr>" counter = 0 for section in section_names: counter += 1 rows += ("<td><a href=\"/%s/\"><img src=\"/%s/states.png\" width=\"99%%\" alt=\"Binary package states in %s\"></a></td>") % \ (section, section, section) if counter % 4 == 0: rows += "</tr><tr>" rows += "</tr>" write_template_html( os.path.join(output_directory, "overview.html"), BASIC_BODY_TEMPLATE, { "page_title": "Overview of suites tested by piuparts.debian.org", "section_navigation": section_navigation, "doc_root": doc_root, "rows": rows, }) # create graph about bugs filed try: make_bts_stats_graph(master_directory, output_directory); except: logging.debug("Error generating the graph images, probably python-rpy2 is not installed, disabling graphs.") generate_global_summary(output_directory, section_names) logging.debug("Done") if __name__ == "__main__": main() # vi:set et ts=4 sw=4 : ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piuparts-slave.py��������������������������������������������������������������������0000644�0000000�0000000�00000117722�13605426771�013625� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2011-2019 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> """Distributed piuparts processing, slave program Lars Wirzenius <liw@iki.fi> """ from __future__ import print_function import os import sys import stat import time import logging from signal import alarm, signal, SIGALRM, SIGINT, SIGKILL, SIGHUP, SIGUSR1 import subprocess import fcntl import random import apt_pkg import pipes import piupartslib.conf import piupartslib.packagesdb from piupartslib.conf import MissingSection apt_pkg.init_system() CONFIG_FILE = "/etc/piuparts/piuparts.conf" DISTRO_CONFIG_FILE = "/etc/piuparts/distros.conf" MAX_WAIT_TEST_RUN = 90 * 60 interrupted = False old_sigint_handler = None got_sighup = False got_sigusr1 = False def setup_logging(log_level, log_file_name): logger = logging.getLogger() logger.setLevel(log_level) formatter = logging.Formatter(fmt="%(asctime)s %(message)s", datefmt="%H:%M:%S") handler = logging.StreamHandler(sys.stderr) handler.setFormatter(formatter) logger.addHandler(handler) if log_file_name: handler = logging.FileHandler(log_file_name) logger.addHandler(handler) class Config(piupartslib.conf.Config): def __init__(self, section="slave", defaults_section=None): self.section = section piupartslib.conf.Config.__init__(self, section, { "sections": "slave", "basetgz-sections": "", "idle-sleep": 300, "max-tgz-age": 2592000, "min-tgz-retry-delay": 21600, "master-host": None, "master-user": None, "master-command": None, "proxy": None, "mirror": None, "setarch": None, "piuparts-command": "sudo piuparts", "piuparts-flags": "", "tmpdir": None, "distro": None, "area": None, "components": None, "chroot-tgz": None, "upgrade-test-distros": None, "basetgz-directory": ".", "chroot-meta-auto": None, "chroot-meta-directory": None, "max-reserved": 1, "debug": "no", "keep-sources-list": "no", "arch": None, "precedence": "1", "slave-load-max": None, "slave-flush-interval": 0, }, defaults_section=defaults_section) class Alarm(Exception): pass def alarm_handler(signum, frame): raise Alarm def sigint_handler(signum, frame): global interrupted interrupted = True print('\nSlave interrupted by the user, waiting for the current test to finish.') print('Press Ctrl-C again to abort now.') signal(SIGINT, old_sigint_handler) def sighup_handler(signum, frame): global got_sighup got_sighup = True print('SIGHUP: Will flush finished logs.') def sigusr1_handler(signum, frame): global got_sigusr1 global got_sighup got_sigusr1 = True got_sighup = True print('SIGUSR1: Will restart.') class MasterIsBusy(Exception): def __init__(self): self.args = "Master is busy, retry later", class MasterNotOK(Exception): def __init__(self): self.args = "Master did not respond with 'ok'", class MasterDidNotGreet(Exception): def __init__(self): self.args = "Master did not start with 'hello'", class MasterCommunicationFailed(Exception): def __init__(self): self.args = "Communication with master failed", class MasterIsCrazy(Exception): def __init__(self): self.args = "Master said something unexpected", class MasterCantRecycle(Exception): def __init__(self): self.args = "Master has nothing to recycle", class Slave: def __init__(self): self._to_master = None self._from_master = None self._master_host = None self._master_user = None self._master_command = None self._section = None def _readline(self): try: line = self._from_master.readline() except IOError: raise MasterCommunicationFailed() logging.debug("<< " + str(line.rstrip())) return line def _writeline(self, *words): line = " ".join(words) logging.debug(">> " + line) try: self._to_master.write(line + "\n") self._to_master.flush() except IOError: raise MasterCommunicationFailed() def set_master_host(self, host): logging.debug("Setting master host to %s" % host) if self._master_host != host: self.close() self._master_host = host def set_master_user(self, user): logging.debug("Setting master user to %s" % user) if self._master_user != user: self.close() self._master_user = user def set_master_command(self, cmd): logging.debug("Setting master command to %s" % cmd) if self._master_command != cmd: self.close() self._master_command = cmd def set_section(self, section): logging.debug("Setting section to %s" % section) self._section = section def connect_to_master(self): if not self._is_connected(): self._initial_connect() self._select_section() def _is_connected(self): return self._to_master and self._from_master def _initial_connect(self): logging.info("Connecting to %s" % self._master_host) ssh_command = ["ssh", "-x"] if self._master_user: ssh_command.extend(["-l", self._master_user]) ssh_command.append(self._master_host) ssh_command.append(self._master_command or "command-is-set-in-authorized_keys") p = subprocess.Popen(ssh_command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True) self._to_master = p.stdin self._from_master = p.stdout line = self._readline() if line != "hello\n": raise MasterDidNotGreet() def _select_section(self): self._writeline("section", self._section) line = self._readline() if line == "busy\n": raise MasterIsBusy() elif line != "ok\n": raise MasterNotOK() logging.debug("Connected to master") def close(self): if self._from_master is None and self._to_master is None: return logging.debug("Closing connection to master") if self._from_master is not None: self._from_master.close() if self._to_master is not None: self._to_master.close() self._from_master = self._to_master = None logging.info("Connection to master closed") def send_log(self, section, pass_or_fail, filename): logging.info("Sending log file %s/%s" % (section, filename)) basename = os.path.basename(filename) package, rest = basename.split("_", 1) version = rest[:-len(".log")] self._writeline(pass_or_fail, package, version) with open(filename, "r") as f: for line in f: if line.endswith("\n"): line = line[:-1] self._writeline(" " + line) self._writeline(".") line = self._readline() if line != "ok\n": raise MasterNotOK() def get_status(self, section): self._writeline("status") line = self._readline() words = line.split() if words and words[0] == "ok": logging.info("Master " + section + " status: " + " ".join(words[1:])) else: raise MasterIsCrazy() def enable_recycling(self): self._writeline("recycle") line = self._readline() words = line.split() if line != "ok\n": raise MasterCantRecycle() def get_idle(self): self._writeline("idle") line = self._readline() words = line.split() if words and words[0] == "ok" and len(words) == 2: return int(words[1]) else: raise MasterIsCrazy() def reserve(self): self._writeline("reserve") line = self._readline() words = line.split() if words and words[0] == "ok": logging.info("Reserved for us: %s %s" % (words[1], words[2])) self.remember_reservation(words[1], words[2]) return True elif words and words[0] == "error": logging.info("Master didn't reserve anything (more) for us") return False else: raise MasterIsCrazy() def unreserve(self, filename): basename = os.path.basename(filename) package, rest = basename.split("_", 1) version = rest[:-len(".log")] logging.info("Unreserve: %s %s" % (package, version)) self._writeline("unreserve", package, version) line = self._readline() if line != "ok\n": raise MasterNotOK() def _reserved_filename(self, name, version): return os.path.join("reserved", "%s_%s.log" % (name, version)) def remember_reservation(self, name, version): create_file(self._reserved_filename(name, version), "") def get_reserved(self): vlist = [] for basename in os.listdir("reserved"): if "_" in basename and basename.endswith(".log"): name, version = basename[:-len(".log")].split("_", 1) vlist.append((name, version)) return vlist def forget_reserved(self, name, version): try: os.remove(self._reserved_filename(name, version)) except os.error: pass class Section: def __init__(self, section, slave=None): self._config = Config(section=section, defaults_section="global") self._config.read(CONFIG_FILE) self._distro_config = piupartslib.conf.DistroConfig( DISTRO_CONFIG_FILE, self._config["mirror"]) self._error_wait_until = 0 self._idle_wait_until = 0 self._recycle_wait_until = 0 self._tarball_wait_until = 0 self._slave_directory = os.path.abspath(section) if not os.path.exists(self._slave_directory): os.makedirs(self._slave_directory) if self._config["debug"] in ["yes", "true"]: self._logger = logging.getLogger() self._logger.setLevel(logging.DEBUG) self._slave = slave or Slave() for rdir in ["new", "pass", "fail", "untestable", "reserved"]: rdir = os.path.join(self._slave_directory, rdir) if not os.path.exists(rdir): os.mkdir(rdir) if int(self._config["max-reserved"]) > 0: self._check_tarball() def _throttle_if_overloaded(self): global interrupted if interrupted or got_sighup: return if self._config["slave-load-max"] is None: return load_max = float(self._config["slave-load-max"]) if load_max < 1.0: return if os.getloadavg()[0] <= load_max: return load_resume = max(load_max - 1.0, 0.9) secs = random.randrange(30, 90) self._slave.close() while True: load = os.getloadavg()[0] if load <= load_resume: break logging.info("Sleeping due to high load (%.2f)" % load) try: time.sleep(secs) except KeyboardInterrupt: interrupted = True if interrupted or got_sighup: break if secs < 300: secs += random.randrange(30, 90) def _connect_to_master(self, recycle=False): self._slave.set_master_host(self._config["master-host"]) self._slave.set_master_user(self._config["master-user"]) self._slave.set_master_command(self._config["master-command"]) self._slave.set_section(self._config.section) self._slave.connect_to_master() if recycle: self._slave.enable_recycling() def _get_tarball(self): basetgz = self._config["chroot-tgz"] or \ self._distro_config.get_basetgz(self._config.get_start_distro(), self._config.get_arch(), merged_usr="--merged-usr" in self._config["piuparts-flags"]) return os.path.join(self._config["basetgz-directory"], basetgz) def _check_tarball(self): if int(self._config["max-tgz-age"]) < 0: return oldcwd = os.getcwd() os.chdir(self._slave_directory) tgz = self._get_tarball() max_tgz_age = int(self._config["max-tgz-age"]) min_tgz_retry_delay = int(self._config["min-tgz-retry-delay"]) ttl = 0 if max_tgz_age == 0: ttl = 86400 needs_update = not os.path.exists(tgz) if not needs_update and max_tgz_age > 0: # tgz exists and age is limited, so check age now = time.time() age = now - os.path.getmtime(tgz) ttl = max_tgz_age - age logging.info("Check-replace %s: age=%d vs. max=%d" % (tgz, age, max_tgz_age)) if ttl < 0: if os.path.exists(tgz + ".log"): age = now - os.path.getmtime(tgz + ".log") ttl = min_tgz_retry_delay - age logging.info("Limit-replace %s: last-retry=%d vs. min=%d" % (tgz, age, min_tgz_retry_delay)) if ttl < 0: needs_update = True logging.info("%s too old. Forcing re-creation" % tgz) if needs_update: self._slave.close() create_chroot(self._config, tgz, self._config.get_start_distro()) ttl = min_tgz_retry_delay self._tarball_wait_until = time.time() + ttl os.chdir(oldcwd) def _get_refchroot_metadata(self): if self._config["chroot-meta-auto"]: if self._config["chroot-meta-directory"]: path = os.path.join(self._config["chroot-meta-directory"], self._config.section) if not os.path.exists(path): os.makedirs(path) return os.path.join(path, self._config["chroot-meta-auto"]) return self._config["chroot-meta-auto"] return None def _check_refchroot_metadata(self): refchroot_metadata = self._get_refchroot_metadata() if refchroot_metadata: if os.path.exists(refchroot_metadata): try: age = time.time() - os.path.getmtime(refchroot_metadata) if age > 6 * 3600: os.unlink(refchroot_metadata) logging.info("Deleting old %s" % refchroot_metadata) except OSError: pass def _count_submittable_logs(self): files = 0 subdirs = ["pass", "fail", "untestable"] if interrupted: subdirs += ["reserved", "new"] for logdir in subdirs: for basename in os.listdir(os.path.join(self._slave_directory, logdir)): if basename.endswith(".log"): files += 1 return files def precedence(self): return int(self._config["precedence"]) def sleep_until(self, recycle=False): if recycle: return max(self._error_wait_until, self._recycle_wait_until) return max(self._error_wait_until, self._idle_wait_until) def run(self, do_processing=True, recycle=False): if time.time() < self.sleep_until(recycle=recycle): return 0 self._throttle_if_overloaded() self._config = Config(section=self._config.section, defaults_section="global") try: self._config.read(CONFIG_FILE) except MissingSection: logging.info("unknown section " + self._config.section) self._error_wait_until = time.time() + 3600 return 0 self._distro_config = piupartslib.conf.DistroConfig( DISTRO_CONFIG_FILE, self._config["mirror"]) if interrupted or got_sighup: do_processing = False if do_processing and time.time() > self._tarball_wait_until: self._check_tarball() if self._config.get_distro() == "None": # section is for tarball creation only self._idle_wait_until = self._tarball_wait_until + 60 self._recycle_wait_until = self._tarball_wait_until + 3600 return 0 if interrupted or got_sighup: do_processing = False if not do_processing and self._count_submittable_logs() == 0: return 0 logging.info("-------------------------------------------") action = "Running" if recycle: action = "Recycling" if not do_processing: action = "Flushing" logging.info("%s section %s (precedence=%d)" % (action, self._config.section, self.precedence())) if int(self._config["max-reserved"]) == 0: logging.info("disabled") self._error_wait_until = time.time() + 12 * 3600 return 0 if not self._config.get_distro() and not self._config.get_distros(): logging.error("neither 'distro' nor 'upgrade-test-distros' configured") self._error_wait_until = time.time() + 3600 return 0 with open(os.path.join(self._slave_directory, "slave.lock"), "w") as lock: oldcwd = os.getcwd() os.chdir(self._slave_directory) try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: logging.info("busy") self._error_wait_until = time.time() + 900 else: if do_processing: self._check_refchroot_metadata() if self._talk_to_master(fetch=do_processing, recycle=recycle, unreserve=interrupted): if do_processing: if not self._slave.get_reserved(): self._idle_wait_until = time.time() + int(self._config["idle-sleep"]) if recycle: self._recycle_wait_until = self._idle_wait_until + 3600 if do_processing and self._slave.get_reserved(): processed = self._process() if got_sighup and self._slave.get_reserved(): # keep this section at the front of the round-robin runnable queue self._idle_wait_until = 0 self._recycle_wait_until = 0 else: # put this section at the end of the round-robin runnable queue self._idle_wait_until = time.time() self._recycle_wait_until = time.time() return processed finally: os.chdir(oldcwd) return 0 def _talk_to_master(self, fetch=False, unreserve=False, recycle=False): flush = self._count_submittable_logs() > 0 fetch = fetch and not self._slave.get_reserved() if not flush and not fetch: return True try: self._connect_to_master(recycle=recycle) except KeyboardInterrupt: raise except MasterIsBusy: logging.error("master is busy") self._error_wait_until = time.time() + random.randrange(60, 180) except MasterCantRecycle: logging.error("master has nothing to recycle") self._recycle_wait_until = max(time.time(), self._idle_wait_until) + 3600 except (MasterDidNotGreet, MasterIsCrazy, MasterCommunicationFailed, MasterNotOK): logging.error("connection to master failed") self._error_wait_until = time.time() + 900 self._slave.close() else: try: for logdir in ["pass", "fail", "untestable"]: for basename in os.listdir(logdir): if basename.endswith(".log"): fullname = os.path.join(logdir, basename) self._slave.send_log(self._config.section, logdir, fullname) os.remove(fullname) if unreserve: for logdir in ["new", "reserved"]: for basename in os.listdir(logdir): if basename.endswith(".log"): fullname = os.path.join(logdir, basename) self._slave.unreserve(fullname) os.remove(fullname) if fetch: max_reserved = int(self._config["max-reserved"]) idle = self._slave.get_idle() if idle > 0: idle = min(idle, int(self._config["idle-sleep"])) logging.info("idle (%d)" % idle) if not recycle: self._idle_wait_until = time.time() + idle else: self._recycle_wait_until = time.time() + idle return 0 while len(self._slave.get_reserved()) < max_reserved and self._slave.reserve(): pass self._slave.get_status(self._config.section) except MasterNotOK: logging.error("master did not respond with 'ok'") self._error_wait_until = time.time() + 900 self._slave.close() except (MasterIsCrazy, MasterCommunicationFailed): logging.error("communication with master failed") self._error_wait_until = time.time() + 900 self._slave.close() else: return True return False def _process(self): global interrupted last_flush = time.time() packagenames = set([x[0] for x in self._slave.get_reserved()]) packages_files = {} for distro in [self._config.get_distro()] + self._config.get_distros(): if distro not in packages_files: try: pf = piupartslib.packagesdb.PackagesFile() pf.load_packages_urls( self._distro_config.get_packages_urls( distro, self._config.get_area(), self._config.get_arch()), packagenames) packages_files[distro] = pf except IOError: logging.error("failed to fetch packages file for %s" % distro) self._error_wait_until = time.time() + 900 return 0 except KeyboardInterrupt: interrupted = True del packagenames test_count = 0 self._check_tarball() if not os.path.exists(self._get_tarball()): self._error_wait_until = time.time() + 300 self._check_refchroot_metadata() for package_name, version in self._slave.get_reserved(): self._throttle_if_overloaded() if interrupted or got_sighup: break if int(self._config["slave-flush-interval"]): if time.time() - last_flush > int(self._config["slave-flush-interval"]): last_flush += 300 # throttle retries if self._talk_to_master(): last_flush = time.time() if not os.path.exists(self._get_tarball()): logging.error("Missing chroot-tgz %s" % self._get_tarball()) break test_count += 1 self._test_package(package_name, version, packages_files) self._slave.forget_reserved(package_name, version) self._talk_to_master(unreserve=interrupted) return test_count def _test_package(self, pname, pvers, packages_files): global old_sigint_handler old_sigint_handler = signal(SIGINT, sigint_handler) self._slave.close() logging.info("Testing package %s/%s %s" % (self._config.section, pname, pvers)) output_name = log_name(pname, pvers) logging.debug("Opening log file %s" % output_name) new_name = os.path.join("new", output_name) output = open(new_name, "w") output.write(time.strftime("Start: %Y-%m-%d %H:%M:%S %Z\n", time.gmtime())) distupgrade = len(self._config.get_distros()) > 1 command = [] if self._config["setarch"]: command.append("setarch") command.extend(self._config["setarch"].split()) command.extend(self._config["piuparts-command"].split()) if self._config["piuparts-flags"]: command.extend(self._config["piuparts-flags"].split()) if "http_proxy" in os.environ: command.extend(["--proxy", os.environ["http_proxy"]]) if self._config["mirror"]: mirror = self._config["mirror"] if self._config["components"]: mirror += " " + self._config["components"] command.extend(["--mirror", mirror]) if self._config["tmpdir"]: command.extend(["--tmpdir", self._config["tmpdir"]]) command.extend(["--arch", self._config.get_arch()]) command.extend(["-b", self._get_tarball()]) if not distupgrade: command.extend(["-d", self._config.get_distro()]) command.append("--no-upgrade-test") else: for distro in self._config.get_distros(): command.extend(["-d", distro]) if self._config["keep-sources-list"] in ["yes", "true"]: command.append("--keep-sources-list") if distupgrade and self._config["chroot-meta-auto"]: refchroot_metadata = self._get_refchroot_metadata() if not os.path.exists(refchroot_metadata): command.extend(["-S", refchroot_metadata]) else: command.extend(["-B", refchroot_metadata]) command.extend(["--apt", "%s=%s" % (pname, pvers)]) subdir = "fail" ret = 0 if not distupgrade: distro = self._config.get_distro() if not pname in packages_files[distro]: output.write("Package %s not found in %s\n" % (pname, distro)) ret = -10001 else: package = packages_files[distro][pname] if pvers != package["Version"]: output.write("Package %s %s not found in %s, %s is available\n" % (pname, pvers, distro, package["Version"])) ret = -10002 output.write("\n") package.dump(output) output.write("\n") else: distros = self._config.get_distros() if distros: # the package must exist somewhere for distro in distros: if pname in packages_files[distro]: break else: output.write("Package %s not found in any distribution\n" % pname) ret = -10003 # the package must have the correct version in the distupgrade target distro distro = distros[-1] if not pname in packages_files[distro]: # the package may "disappear" in the distupgrade target distro if pvers == "None": pass else: output.write("Package %s not found in %s\n" % (pname, distro)) ret = -10004 else: package = packages_files[distro][pname] if pvers != package["Version"]: output.write("Package %s %s not found in %s, %s is available\n" % (pname, pvers, distro, package["Version"])) ret = -10005 for distro in distros: output.write("\n[%s]\n" % distro) if pname in packages_files[distro]: packages_files[distro][pname].dump(output) output.write("\n") if ret == 0: prev = "~" for distro in distros: if pname in packages_files[distro]: v = packages_files[distro][pname]["Version"] if not apt_pkg.version_compare(prev, v) <= 0: output.write("Upgrade to %s requires downgrade: %s > %s\n" % (distro, prev, v)) ret = -10006 prev = v else: ret = -10010 if ret != 0: subdir = "untestable" if ret == 0: output.write("Executing: %s\n" % command2string(command)) ret, f = run_test_with_timeout(command, MAX_WAIT_TEST_RUN) if not f or f[-1] != '\n': f += '\n' output.write(f.replace('\033', '[ESC]')) lastline = f.split('\n')[-2] if ret < 0: output.write(" *** Process KILLED - exceed maximum run time ***\n") elif not "piuparts run ends" in lastline: ret += 1024 output.write(" *** PIUPARTS OUTPUT INCOMPLETE ***\n") elif distupgrade and self._config["chroot-meta-auto"]: try: refchroot_metadata = self._get_refchroot_metadata() if "History of available packages does not match - reference chroot may be outdated" in f: os.unlink(refchroot_metadata) logging.info("Deleting outdated %s" % refchroot_metadata) elif "Initial package selections do not match - ignoring loaded reference chroot state" in f: os.unlink(refchroot_metadata) logging.info("Deleting mismatching %s" % refchroot_metadata) except OSError: pass output.write("\n") output.write("ret=%d\n" % ret) output.write(time.strftime("End: %Y-%m-%d %H:%M:%S %Z\n", time.gmtime())) output.close() if ret == 0: subdir = "pass" os.rename(new_name, os.path.join(subdir, output_name)) logging.debug("Done with %s: %s (%d)" % (output_name, subdir, ret)) signal(SIGINT, old_sigint_handler) def log_name(package, version): return "%s_%s.log" % (package, version) def command2string(command): """Quote s.t. copy+paste from the logfile gives a runnable command in the shell.""" return " ".join([pipes.quote(arg) for arg in command]) def run_test_with_timeout(cmd, maxwait, kill_all=True): def terminate_subprocess(p, kill_all): pids = [p.pid] if kill_all: ps = subprocess.Popen(["ps", "--no-headers", "-o", "pid", "--ppid", "%d" % p.pid], stdout=subprocess.PIPE, universal_newlines=True) stdout, stderr = ps.communicate() pids.extend([int(pid) for pid in stdout.split()]) if p.poll() is None: print('Sending SIGINT...') try: os.killpg(os.getpgid(p.pid), SIGINT) except OSError: pass # piuparts has 30 seconds to clean up after Ctrl-C for i in range(60): time.sleep(0.5) if p.poll() is not None: break if p.poll() is None: print('Sending SIGTERM...') p.terminate() # piuparts has 5 seconds to clean up after SIGTERM for i in range(10): time.sleep(0.5) if p.poll() is not None: break if p.poll() is None: print('Sending SIGKILL...') p.kill() for pid in pids: if pid > 0: try: os.kill(pid, SIGKILL) print("Killed %d" % pid) except OSError: pass logging.debug("Executing: %s" % command2string(cmd)) stdout = "" p = subprocess.Popen(cmd, preexec_fn=os.setpgrp, universal_newlines=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) if maxwait > 0: signal(SIGALRM, alarm_handler) alarm(maxwait) try: stdout, stderr = p.communicate() alarm(0) except Alarm: terminate_subprocess(p, kill_all) return -1, stdout except KeyboardInterrupt: print('\nSlave interrupted by the user, cleaning up...') try: terminate_subprocess(p, kill_all) except KeyboardInterrupt: print('\nTerminating piuparts was interrupted... manual cleanup still neccessary.') raise raise ret = p.returncode if ret in [124, 137]: # process was terminated by the timeout command ret = -ret return ret, stdout def create_chroot(config, tarball, distro): command = [] if config["setarch"]: command.append("setarch") command.extend(config["setarch"].split()) command.extend(config["piuparts-command"].split()) if config["piuparts-flags"]: command.extend(config["piuparts-flags"].split()) if "http_proxy" in os.environ: command.extend(["--proxy", os.environ["http_proxy"]]) if config["mirror"]: mirror = config["mirror"] if config["components"]: mirror += " " + config["components"] command.extend(["--mirror", mirror]) if config["tmpdir"]: command.extend(["--tmpdir", config["tmpdir"]]) command.extend(["--arch", config.get_arch()]) command.extend(["-d", distro]) command.extend(["-s", tarball + ".new"]) command.extend(['--no-install-purge-test', '--no-upgrade-test']) command.extend(["--apt", "TARBALL"]) # dummy package name output_name = tarball + ".log" with open(output_name, "w") as output: try: fcntl.flock(output, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError: logging.info("Creation of tarball %s already in progress." % tarball) else: logging.info("Creating new tarball %s" % tarball) output.write(time.strftime("Start: %Y-%m-%d %H:%M:%S %Z\n\n", time.gmtime())) output.write("Executing: " + command2string(command) + "\n\n") logging.debug("Executing: " + command2string(command)) try: p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True) for line in p.stdout: output.write(line) logging.debug(">> " + line.rstrip()) p.wait() output.write(time.strftime("\nEnd: %Y-%m-%d %H:%M:%S %Z\n", time.gmtime())) if os.path.exists(tarball + ".new"): os.rename(tarball + ".new", tarball) else: logging.error("Tarball creation failed, see %s" % output_name) except IOError: output.write(time.strftime("\nFAIL: %Y-%m-%d %H:%M:%S %Z\n", time.gmtime())) logging.error("Tarball creation failed with IOError") def create_file(filename, contents): with open(filename, "w") as f: f.write(contents) def main(): setup_logging(logging.INFO, None) signal(SIGHUP, sighup_handler) signal(SIGUSR1, sigusr1_handler) # For supporting multiple architectures and suites, we take command-line # argument(s) referring to section(s) in the configuration file. # If no argument is given, the "sections" entry from the "global" section # is used. section_names = [] global_config = Config(section="global") global_config.read(CONFIG_FILE) if global_config["proxy"]: os.environ["http_proxy"] = global_config["proxy"] if len(sys.argv) > 1: section_names = sys.argv[1:] else: section_names = global_config["sections"].split() section_names += global_config["basetgz-sections"].split() persistent_connection = Slave() sections = [] for section_name in section_names: try: sections.append(Section(section_name, persistent_connection)) except MissingSection: # ignore unknown sections pass if not sections: logging.error("no sections found") return # flush logs from previous run for section in sections: section.run(do_processing=False) while True: global got_sighup test_count = 0 for section in sorted(sections, key=lambda section: (section.precedence(), section.sleep_until())): test_count += section.run(do_processing=(test_count == 0)) if got_sigusr1: logging.info("Restarting...") os.execv(__file__, sys.argv) if test_count == 0 and got_sighup: # clear SIGHUP state after flushing all sections got_sighup = False continue if test_count == 0: # try to recycle old logs # round robin recycling of all sections is ensured by the recycle_wait_until timestamps idle_until = min([section.sleep_until() for section in sections]) for section in sorted(sections, key=lambda section: section.sleep_until(recycle=True)): test_count += section.run(recycle=True) if test_count > 0 and idle_until < time.time(): break if interrupted: raise KeyboardInterrupt if test_count == 0 and not got_sighup: now = time.time() sleep_until = min([now + int(global_config["idle-sleep"])] + [section.sleep_until() for section in sections]) if (sleep_until > now): to_sleep = max(60, sleep_until - now) persistent_connection.close() logging.info("Nothing to do, sleeping for %d seconds." % to_sleep) time.sleep(to_sleep) if __name__ == "__main__": try: main() except KeyboardInterrupt: print('') print('Slave interrupted by the user, exiting...') sys.exit(1) # vi:set et ts=4 sw=4 : ����������������������������������������������piuparts-1.1.1/piuparts.py��������������������������������������������������������������������������0000644�0000000�0000000�00000406513�13605426771�012514� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/python3 # -*- coding: utf-8 -*- # # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2007-2018 Holger Levsen (holger@layer-acht.org) # Copyright © 2010-2019 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> """Debian package installation and uninstallation tester. This program sets up a minimal Debian system in a chroot, and installs and uninstalls packages and their dependencies therein, looking for problems. See the manual page (piuparts.1, generated from piuparts.1.txt) for more usage information. Lars Wirzenius <liw@iki.fi> """ from __future__ import print_function VERSION = "__PIUPARTS_VERSION__" import distro_info import time import logging import optparse import sys import tempfile import shutil import os import tarfile import stat import re import json import pickle import subprocess import traceback import uuid import apt_pkg import pipes from collections import namedtuple from signal import alarm, signal, SIGALRM, SIGTERM, SIGKILL try: from debian import deb822 except ImportError: from debian_bundle import deb822 import piupartslib.conf from six.moves import urllib import six apt_pkg.init_system() DISTRO_CONFIG_FILE = "/etc/piuparts/distros.conf" class Defaults: """Default settings which depend on flavor of Debian. Some settings, such as the default mirror and distribution, depend on which flavor of Debian we run under: Debian itself, or a derived distribution such as Ubuntu. This class abstracts away the defaults so that the rest of the code can just refer to the values defined herein. """ def get_components(self): """Return list of default components for a mirror.""" def get_mirror(self): """Return default mirror.""" def get_distribution(self): """Return default distribution.""" def get_keyring(self): """Return default keyring.""" class DebianDefaults(Defaults): def get_components(self): return ["main", "contrib", "non-free"] def get_mirror(self): return [("http://deb.debian.org/debian", self.get_components())] def get_distribution(self): return [distro_info.DebianDistroInfo().devel()] def get_keyring(self): return "/usr/share/keyrings/debian-archive-keyring.gpg" class UbuntuDefaults(Defaults): def get_components(self): return ["main", "universe", "restricted", "multiverse"] def get_mirror(self): return [("http://archive.ubuntu.com/ubuntu", self.get_components())] def get_distribution(self): return [distro_info.UbuntuDistroInfo().devel()] def get_keyring(self): return "/usr/share/keyrings/ubuntu-archive-keyring.gpg" class DefaultsFactory: """Instantiate the right defaults class.""" def guess_flavor(self): p = subprocess.Popen(["lsb_release", "-i", "-s"], stdout=subprocess.PIPE, universal_newlines=True) stdout, stderr = p.communicate() return stdout.strip().lower() def new_defaults(self): if not settings.defaults: settings.defaults = self.guess_flavor() if settings.defaults.lower() == "debian": return DebianDefaults() if settings.defaults.lower() == "ubuntu": return UbuntuDefaults() logging.error("Unknown set of defaults: %s" % settings.defaults) panic() class Settings: """Global settings for this program.""" def __init__(self): self.defaults = None self.tmpdir = None self.keep_env = False self.shell_on_error = False self.max_command_output_size = 8 * 1024 * 1024 # 8 MB (google-android-ndk-installer on install) (daptup on dist-upgrade) self.max_command_runtime = 60 * 60 # 60 minutes (texlive-full and blends metapackages on dist-upgrade) self.single_changes_list = False self.single_packages = False self.args_are_package_files = True # distro setup self.proxy = None self.debian_mirrors = [] self.extra_repos = [] self.testdebs_repo = None self.debian_distros = [] self.keep_sources_list = False self.keyring = None self.do_not_verify_signatures = False self.no_check_valid_until = False self.install_recommends = False self.install_suggests = False self.eatmydata = True self.dpkg_force_unsafe_io = True self.dpkg_force_confdef = False self.scriptsdirs = [] self.bindmounts = [] self.allow_database = False # chroot setup self.arch = None self.basetgz = None self.savetgz = None self.lvm_volume = None self.lvm_snapshot_size = "1G" self.existing_chroot = None self.hard_link = False self.schroot = None self.end_meta = None self.save_end_meta = None self.skip_minimize = True self.minimize = False self.debfoster_options = None self.docker_image = None self.merged_usr = False # tests and checks self.no_install_purge_test = False self.no_upgrade_test = False self.upgrade_before_dist_upgrade = False self.distupgrade_to_testdebs = False self.install_remove_install = False self.install_purge_install = False self.list_installed_files = False self.fake_essential_packages = [] self.extra_old_packages = [] self.skip_cronfiles_test = False self.skip_logrotatefiles_test = False self.adequate = True self.check_broken_diversions = True self.check_broken_symlinks = True self.warn_broken_symlinks = True self.warn_on_others = False self.warn_on_leftovers_after_purge = False self.warn_on_debsums_errors = False self.warn_on_install_over_symlink = False self.warn_if_inadequate = True self.pedantic_purge_test = False self.ignored_files = [ # /root/.rnd should *not* be listed here, see #750099 # piuparts state "/usr/sbin/policy-rc.d", # system state "/boot/grub/", "/etc/X11/", "/etc/X11/Xwrapper.config", #859929 "/etc/X11/default-display-manager", "/etc/aliases", "/etc/aliases.db", "/etc/crypttab", "/etc/group", "/etc/group-", "/etc/group.org", "/etc/gshadow", "/etc/gshadow-", "/etc/hosts", "/etc/inetd.conf", "/etc/inittab", "/etc/ld.so.cache", "/etc/machine-id", "/etc/mailname", "/etc/mtab", "/etc/network/interfaces", "/etc/news/", "/etc/news/organization", "/etc/news/server", "/etc/news/servers", "/etc/news/whoami", "/etc/nologin", "/etc/passwd", "/etc/passwd-", "/etc/passwd.org", "/etc/rc.local", "/etc/shadow", "/etc/shadow-", "/etc/shadow.org", "/etc/subgid", "/etc/subgid-", "/etc/subuid", "/etc/subuid-", "/usr/share/info/dir", "/usr/share/info/dir.old", "/var/cache/ldconfig/aux-cache", "/var/crash/", "/var/games/", # package management "/etc/apt/apt.conf.d/01autoremove-kernels", "/etc/apt/secring.gpg", "/etc/apt/trustdb.gpg", "/etc/apt/trusted.gpg", "/etc/apt/trusted.gpg~", "/var/cache/apt/archives/lock", "/var/cache/apt/archives/partial/", "/var/cache/apt/pkgcache.bin", "/var/cache/apt/srcpkgcache.bin", "/var/cache/debconf/", "/var/cache/debconf/config.dat", "/var/cache/debconf/config.dat.old", "/var/cache/debconf/config.dat-old", "/var/cache/debconf/passwords.dat", "/var/cache/debconf/passwords.dat.old", "/var/cache/debconf/templates.dat", "/var/cache/debconf/templates.dat.old", "/var/cache/debconf/templates.dat-old", "/var/lib/apt/daily_lock", "/var/lib/apt/extended_states", "/var/lib/cdebconf/", "/var/lib/cdebconf/passwords.dat", "/var/lib/cdebconf/questions.dat", "/var/lib/cdebconf/questions.dat-old", "/var/lib/cdebconf/templates.dat", "/var/lib/cdebconf/templates.dat-old", "/var/lib/dpkg/arch", "/var/lib/dpkg/available", "/var/lib/dpkg/available-old", "/var/lib/dpkg/diversions", "/var/lib/dpkg/diversions-old", "/var/lib/dpkg/lock", "/var/lib/dpkg/lock-frontend", "/var/lib/dpkg/status", "/var/lib/dpkg/status-old", "/var/lib/dpkg/statoverride", "/var/lib/dpkg/statoverride-old", "/var/log/alternatives.log", "/var/log/apt/eipp.log.xz", "/var/log/apt/history.log", "/var/log/apt/term.log", "/var/log/bootstrap.log", "/var/log/dbconfig-common/dbc.log", "/var/log/dpkg.log", # system logfiles "/var/log/auth.log", "/var/log/btmp", "/var/log/daemon.log", "/var/log/debug", "/var/log/faillog", "/var/log/kern.log", "/var/log/lastlog", "/var/log/lpr.log", "/var/log/mail.err", "/var/log/mail.info", "/var/log/mail.log", "/var/log/mail.warn", "/var/log/messages", "/var/log/news/", "/var/log/news/news.crit", "/var/log/news/news.err", "/var/log/news/news.notice", "/var/log/secure", "/var/log/syslog", "/var/log/tallylog", "/var/log/user.log", # application logfiles # actually, only modification should be permitted here, but not creation/removal "/var/log/fontconfig.log", # home directories of system accounts "/var/lib/debian-security-support/", # #749317 "/var/lib/gozerbot/", "/var/lib/nagios/", # nagios* (#668756) "/var/lib/onioncat/", # onioncat "/var/lib/pkcs11proxyd/", # caml-crush-server (#810703) "/var/lib/rbldns/", "/var/lib/sreview/", # sreview (#905500) "/var/spool/powerdns/", # pdns-server (#531134), pdns-recursor (#531135) # work around broken symlinks "/etc/modules-load.d/modules.conf", # -> ../modules (target obsoleted by modules-load.d) "/etc/sysctl.d/99-sysctl.conf", # -> ../sysctl.conf (target obsoleted by sysctl.d) "/usr/lib/python2.6/dist-packages/python-support.pth", # 635493 and #385775 "/usr/lib/python2.7/dist-packages/python-support.pth", "/usr/share/texmf/ls-R", # -> /var/lib/texmf/ls-R-TEXMFMAIN (link owned by tex-common, target created with mktexlsr from texlive-binaries) # work around #316521 dpkg: incomplete cleanup of empty directories "/etc/apache2/", "/etc/apache2/conf.d/", "/etc/clamav/", "/etc/cron.d/", "/etc/lighttpd/", "/etc/lighttpd/conf-available/", "/etc/modprobe.d/", "/etc/nagios-plugins/config/", "/etc/network/", "/etc/php/", "/etc/php/7.0/", "/etc/php/7.0/apache2/", "/etc/php/7.0/apache2/conf.d/", "/etc/php/7.0/cli/", "/etc/php/7.0/cli/conf.d/", "/etc/php/7.0/mods-available/", "/etc/php/7.0/phpdbg/", "/etc/php/7.0/phpdbg/conf.d/", "/etc/php5/", "/etc/php5/conf.d/", "/etc/php5/mods-available/", "/etc/sgml/", "/etc/ssl/", "/etc/ssl/certs/", "/etc/ssl/private/", "/etc/xml/", "/usr/share/dh-python/", "/usr/share/dh-python/dhpython/", "/usr/share/dh-python/dhpython/build/", "/usr/share/python3/", "/usr/share/python3/debpython/", "/var/lib/apache2/", # HACKS ] self.ignored_patterns = [ # system state "/dev/.*", "/etc/init.d/\.depend.*", "/run/.*", "/var/backups/.*", "/var/cache/man/.*", "/var/mail/.*", "/var/run/.*", # package management "/var/lib/apt/lists/.*", "/var/lib/dpkg/alternatives/.*", "/var/lib/dpkg/triggers/.*", "/var/lib/insserv/run.*.log", "/var/lib/ucf/.*", "/var/lib/update-rc.d/.*", # application data "/srv/.*", # 848186 "/var/lib/citadel/(data/.*)?", "/var/lib/mercurial-server/.*", "/var/lib/onak/.*", "/var/lib/openvswitch/(pki/.*)?", "/var/lib/vmm/(./.*)?", # 682184 "/var/log/exim/.*", "/var/log/exim4/.*", "/var/spool/exim/.*", "/var/spool/exim4/.*", "/var/spool/news/.*", "/var/spool/squid/(../.*)?", "/var/www/.*", # HACKS ":/lib/modules/([^/]*/(modules.*)?)?", ] self.non_pedantic_ignore_patterns = [ "/tmp/.*" ] settings = Settings() on_panic_hooks = {} counter = 0 def do_on_panic(hook): global counter cid = counter counter += 1 on_panic_hooks[cid] = hook return cid def dont_do_on_panic(id): del on_panic_hooks[id] class TimeOffsetFormatter(logging.Formatter): def __init__(self, fmt=None, datefmt=None): self.startup_time = time.time() logging.Formatter.__init__(self, fmt, datefmt) def formatTime(self, record, datefmt): t = time.time() - self.startup_time t_min = int(t / 60) t_sec = t % 60.0 return "%dm%.1fs" % (t_min, t_sec) DUMP = logging.DEBUG - 1 HANDLERS = [] def setup_logging(log_level, log_file_name): logging.addLevelName(DUMP, "DUMP") logger = logging.getLogger() logger.setLevel(log_level) formatter = TimeOffsetFormatter("%(asctime)s %(levelname)s: %(message)s") handler = logging.StreamHandler(sys.stdout) handler.setFormatter(formatter) logger.addHandler(handler) HANDLERS.append(handler) if log_file_name: handler = logging.FileHandler(log_file_name) handler.setFormatter(formatter) logger.addHandler(handler) HANDLERS.append(handler) def dump(msg): logger = logging.getLogger() logger.log(DUMP, msg) for handler in HANDLERS: handler.flush() def panic(exit=1): for i in reversed(range(counter)): if i in on_panic_hooks: on_panic_hooks[i]() logging.error("piuparts run ends.") sys.exit(exit) def indent_string(str): """Indent all lines in a string with two spaces and return result.""" return "\n".join([" " + line for line in str.split("\n")]) def command2string(command): """Quote s.t. copy+paste from the logfile gives a runnable command in the shell.""" return " ".join([pipes.quote(arg) for arg in command]) def unqualify(packages): if packages: return [p.split("=", 1)[0].strip() for p in packages] return packages class Alarm(Exception): pass def alarm_handler(signum, frame): raise Alarm def run(command, ignore_errors=False, timeout=0): """Run an external command and die with error message if it fails.""" def kill_subprocess(p, reason): logging.error("Terminating command due to %s" % reason) p.terminate() for i in range(10): time.sleep(0.5) if p.poll() is not None: break else: logging.error("Killing command due to %s" % reason) p.kill() p.wait() assert isinstance(command, type([])) logging.debug("Starting command: %s" % command) env = os.environ.copy() for var in ["LANG", "LANGUAGE", "LC_CTYPE", "LC_NUMERIC", "LC_TIME", "LC_COLLATE", "LC_MONETARY", "LC_MESSAGES", "LC_PAPER", "LC_NAME", "LC_ADDRESS", "LC_TELEPHONE", "LC_MEASUREMENT", "LC_IDENTIFICATION", "LC_ALL"]: if var in env: del env[var] env["PIUPARTS_OBJECTS"] = ' '.join(str(vobject) for vobject in settings.testobjects) devnull = open('/dev/null', 'r') p = subprocess.Popen(command, env=env, stdin=devnull, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True) output = "" excessive_output = False if timeout > 0: signal(SIGALRM, alarm_handler) alarm(timeout) try: while p.poll() is None: """Read 64 KB chunks, but depending on the output buffering behavior of the command we may get less even if more output is coming later. Abort after reading max_command_output_size bytes.""" output += p.stdout.read(1 << 16) if (len(output) > settings.max_command_output_size): excessive_output = True ignore_errors = False alarm(0) kill_subprocess(p, "excessive output") output += "\n\n***** Command was terminated after exceeding output limit (%.2f MB) *****\n" \ % (settings.max_command_output_size / 1024. / 1024.) break if not excessive_output: output += p.stdout.read(settings.max_command_output_size) alarm(0) except Alarm: ignore_errors = False kill_subprocess(p, "excessive runtime") output += "\n\n***** Command was terminated after exceeding runtime limit (%s s) *****\n" % timeout devnull.close() if output: dump("\n" + indent_string(output.rstrip("\n"))) if p.returncode == 0: logging.debug("Command ok: %s" % repr(command)) elif ignore_errors: logging.debug("Command failed (status=%d), but ignoring error: %s" % (p.returncode, repr(command))) else: logging.error("Command failed (status=%d): %s\n%s" % (p.returncode, repr(command), indent_string(output))) panic() return p.returncode, output def create_temp_file(): """Create a temporary file and return its full path.""" (fd, path) = tempfile.mkstemp(dir=settings.tmpdir) logging.debug("Created temporary file %s" % path) return (fd, path) def create_file(filename, contents): """Create a new file with the desired name and contents.""" try: with open(filename, "w") as f: f.write(contents) except IOError as detail: logging.error("Couldn't create file %s: %s" % (filename, detail)) panic() def readlines_file(filename): with open(filename, "r") as f: return f.readlines() def remove_files(filenames): """Remove some files.""" for filename in filenames: logging.debug("Removing %s" % filename) try: os.remove(filename) except OSError as detail: logging.error("Couldn't remove %s: %s" % (filename, detail)) panic() def make_metapackage(name, depends, conflicts, arch='all'): """Return the path to a .deb created just for satisfying dependencies Caller is responsible for removing the temporary directory containing the .deb when finished. """ # Inspired by pbuilder's pbuilder-satisfydepends-aptitude tmpdir = tempfile.mkdtemp(dir=settings.tmpdir) panic_handler_id = do_on_panic(lambda: shutil.rmtree(tmpdir)) create_file(os.path.join(tmpdir, ".piuparts.tmpdir"), "metapackage creation") old_umask = os.umask(0) os.makedirs(os.path.join(tmpdir, name, 'DEBIAN'), mode=0o755) os.umask(old_umask) control = deb822.Deb822() control['Package'] = name control['Version'] = '0.invalid.0' control['Architecture'] = arch control['Maintainer'] = ('piuparts developers team ' '<piuparts-devel@alioth-lists.debian.net>') control['Description'] = ('Dummy package to satisfy dependencies - ' 'created by piuparts\n' ' This package was created automatically by ' 'piuparts and can safely be removed') if depends: control['Depends'] = depends if conflicts: control['Conflicts'] = conflicts create_file(os.path.join(tmpdir, name, 'DEBIAN', 'control'), control.dump()) logging.debug("metapackage:\n" + indent_string(control.dump())) run(['dpkg-deb', '-b', '-Zgzip', '--nocheck', os.path.join(tmpdir, name)]) dont_do_on_panic(panic_handler_id) return os.path.join(tmpdir, name + '.deb') def split_path(pathname): parts = [] while pathname: (head, tail) = os.path.split(pathname) # print("split '%s' => '%s' + '%s'" % (pathname, head, tail)) if tail: parts.append(tail) elif not head: break elif head == pathname: parts.append(head) break pathname = head return parts def canonicalize_path(root, pathname, report_links=False): """Canonicalize a path name, simulating chroot at 'root'. When resolving the symlink, pretend (similar to chroot) that 'root' is the root of the filesystem. Also resolve '..' and '.' components. This should not escape the chroot below 'root', but for security concerns, use chroot and have the kernel resolve symlinks instead. Returns the final canonical path or a list of (path, target) tuples, one for each symlink encountered. """ # print("\nCANONICALIZE %s %s" % (root, pathname)) links = [] seen = [] parts = split_path(pathname) # print("PARTS ", list(reversed(parts))) path = "/" while parts: tag = "\n".join(parts + [path]) # print("TEST '%s' + " % path, list(reversed(parts))) if tag in seen or len(seen) > 1024: fullpath = os.path.join(path, *reversed(parts)) # print("LOOP %s" % fullpath) path = fullpath logging.error("ELOOP: Too many symbolic links in '%s'" % path) break seen.append(tag) part = parts.pop() # Using normpath() to cleanup '.', '..' and multiple slashes. # Removing a suffix 'foo/..' is safe here since it can't change the # meaning of 'path' because it contains no symlinks - they have been # resolved already. newpath = os.path.normpath(os.path.join(path, part)) rootedpath = os.path.join(root, newpath[1:]) if newpath == "/": path = "/" elif os.path.islink(rootedpath): target = os.readlink(rootedpath) # print("LINK to '%s'" % target) links.append((newpath, target)) if os.path.isabs(target): path = "/" parts.extend(split_path(target)) else: path = newpath # print("FINAL '%s'" % path) if report_links: return links return path def is_broken_symlink(root, dirpath, filename): """Is symlink dirpath+filename broken?""" if dirpath[:len(root)] == root: dirpath = dirpath[len(root):] pathname = canonicalize_path(root, os.path.join(dirpath, filename)) pathname = os.path.join(root, pathname[1:]) # The symlink chain, if any, has now been resolved. Does the target # exist? # print("EXISTS ", pathname, os.path.exists(pathname)) return not os.path.exists(pathname) FileInfo = namedtuple('FileInfo', ['st', 'target', 'user', 'group']) class Chroot: """A chroot for testing things in.""" def __init__(self): self.name = None self.bootstrapped = False self.mounts = [] self.initial_selections = None self.avail_md5_history = [] def create_temp_dir(self): """Create a temporary directory for the chroot.""" self.name = tempfile.mkdtemp(dir=settings.tmpdir) create_file(os.path.join(self.name, ".piuparts.tmpdir"), "chroot") os.chmod(self.name, 0o755) logging.debug("Created temporary directory %s" % self.name) def create(self, temp_tgz=None): """Create a chroot according to user's wishes.""" self.panic_handler_id = do_on_panic(self.remove) if not settings.schroot and not settings.docker_image: self.create_temp_dir() if temp_tgz: self.unpack_from_tgz(temp_tgz) elif settings.basetgz: self.unpack_from_tgz(settings.basetgz) elif settings.lvm_volume: self.setup_from_lvm(settings.lvm_volume) elif settings.existing_chroot: self.setup_from_dir(settings.existing_chroot) elif settings.schroot: self.setup_from_schroot(settings.schroot) elif settings.docker_image: self.setup_from_docker(settings.docker_image) else: self.setup_minimal_chroot() if not settings.schroot and not settings.docker_image: self.mount_proc() self.configure_chroot() # Copy scripts dirs into the chroot, merging all dirs together, # later files overwriting earlier ones. if settings.scriptsdirs: self.mkdir_p("tmp/scripts/") dest = self.relative("tmp/scripts/") for sdir in settings.scriptsdirs: logging.debug("Copying scriptsdir %s to %s" % (sdir, dest)) for sfile in os.listdir(sdir): if (sfile.startswith("post_") or sfile.startswith("pre_") or sfile.startswith("is_testable_")) \ and not ".dpkg-" in sfile \ and os.path.isfile(os.path.join(sdir, sfile)): shutil.copy(os.path.join(sdir, sfile), dest) # Run custom scripts after chroot has been unpacked/debootstrapped # Useful for adjusting apt configuration e.g. for internal mirror usage self.run_scripts("post_chroot_unpack") self.run(["apt-get", "update"]) if settings.basetgz or settings.docker_image or settings.schroot or settings.existing_chroot: self.run(["apt-get", "-yf", "dist-upgrade"]) self.minimize() self.remember_available_md5() # Run custom scripts after creating the chroot. self.run_scripts("post_setup") self.install_packages_by_name(settings.fake_essential_packages, with_scripts=False) if settings.savetgz and not temp_tgz: self.pack_into_tgz(settings.savetgz) def remove(self): """Remove a chroot and all its contents.""" if not settings.keep_env and os.path.exists(self.name): self.terminate_running_processes() self.unmount_all() if settings.lvm_volume: logging.debug('Unmounting and removing LVM snapshot %s' % self.lvm_snapshot_name) run(['umount', self.name]) run(['lvremove', '-f', self.lvm_snapshot]) if settings.schroot: logging.debug("Terminate schroot session '%s'" % self.name) run(['schroot', '--end-session', '--chroot', "session:" + self.schroot_session]) if settings.docker_image: logging.debug("Destroy docker container '%s'" % self.docker_container) run(['docker', 'rm', '-f', self.docker_container]) if not settings.schroot and not settings.docker_image: run(['rm', '-rf', '--one-file-system', self.name]) if os.path.exists(self.name): create_file(os.path.join(self.name, ".piuparts.tmpdir"), "removal failed") logging.debug("Removed directory tree at %s" % self.name) elif settings.keep_env: if settings.schroot: logging.debug("Keeping schroot session %s at %s" % (self.schroot_session, self.name)) elif settings.docker_image: logging.debug("Keeping container %s" % self.docker_container) else: logging.debug("Keeping directory tree at %s" % self.name) dont_do_on_panic(self.panic_handler_id) def was_bootstrapped(self): return self.bootstrapped def create_temp_tgz_file(self): """Return the path to a file to be used as a temporary tgz file""" # Yes, create_temp_file() would work just as well, but putting it in # the interface for Chroot allows the VirtServ hack to work. (fd, temp_tgz) = create_temp_file() os.close(fd) return temp_tgz def remove_temp_tgz_file(self, temp_tgz): """Remove the file that was used as a temporary tgz file""" # Yes, remove_files() would work just as well, but putting it in # the interface for Chroot allows the VirtServ hack to work. remove_files([temp_tgz]) def pack_into_tgz(self, result): """Tar and compress all files in the chroot.""" self.run(["apt-get", "clean"]) logging.debug("Saving %s to %s." % (self.name, result)) (fd, tmpfile) = tempfile.mkstemp(dir=os.path.dirname(result)) os.close(fd) cleanup_tmpfile = lambda: os.remove(tmpfile) panic_handler_id = do_on_panic(cleanup_tmpfile) run(['tar', '-czf', tmpfile, '--one-file-system', '--exclude', 'tmp/scripts', '-C', self.name, './']) os.chmod(tmpfile, 0o644) os.rename(tmpfile, result) dont_do_on_panic(panic_handler_id) def unpack_from_tgz(self, tarball): """Unpack a tarball to a chroot.""" logging.debug("Unpacking %s into %s" % (tarball, self.name)) prefix = [] if settings.eatmydata and os.path.isfile('/usr/bin/eatmydata'): prefix.append('eatmydata') run(prefix + ["tar", "-C", self.name, "-zxf", tarball]) def setup_from_schroot(self, schroot): self.schroot_session = schroot.split(":", 1)[-1] + "-" + str(uuid.uuid1()) + "-piuparts" run(['schroot', '--begin-session', '--chroot', schroot, '--session-name', self.schroot_session]) ret_code, output = run(['schroot', '--chroot', "session:" + self.schroot_session, '--location']) self.name = output.strip() logging.info("New schroot session in '%s'" % self.name) @staticmethod def check_if_docker_storage_driver_is_supported(): ret_code, output = run(['docker', 'info']) if 'overlay2' not in output: logging.error('Only overlay2 storage driver is supported') panic() def setup_from_docker(self, docker_image): self.check_if_docker_storage_driver_is_supported() ret_code, output = run(['docker', 'run', '-d', '-it', docker_image, 'bash']) if ret_code != 0: logging.error("Couldn't start the container from '%s'" % docker_image) panic() self.docker_container = output.strip() ret_code, output = run(['docker', 'inspect', self.docker_container]) container_data = json.loads(output)[0] self.name = container_data['GraphDriver']['Data']['MergedDir'] logging.info("New container created '%s'" % self.docker_container) def setup_from_lvm(self, lvm_volume): """Create a chroot by creating an LVM snapshot.""" self.lvm_base = os.path.dirname(lvm_volume) self.lvm_vol_name = os.path.basename(lvm_volume) self.lvm_snapshot_name = self.lvm_vol_name + "-" + str(uuid.uuid1()) self.lvm_snapshot = os.path.join(self.lvm_base, self.lvm_snapshot_name) logging.debug("Creating LVM snapshot %s from %s" % (self.lvm_snapshot, lvm_volume)) run(['lvcreate', '-n', self.lvm_snapshot, '-s', lvm_volume, '-L', settings.lvm_snapshot_size]) logging.info("Mounting LVM snapshot to %s" % self.name) run(['mount', self.lvm_snapshot, self.name]) def setup_from_dir(self, dirname): """Create chroot from an existing one.""" # if on same device, make hard link cmd = ["cp"] if settings.hard_link and os.stat(dirname).st_dev == os.stat(self.name).st_dev: cmd += ["-al"] logging.debug("Hard linking %s to %s" % (dirname, self.name)) else: cmd += ["-ax"] logging.debug("Copying %s into %s" % (dirname, self.name)) for name in os.listdir(dirname): src = os.path.join(dirname, name) dst = os.path.join(self.name, name) run(cmd + [src, dst]) def interactive_shell(self): logging.info("Entering interactive shell in %s" % self.name) env = os.environ.copy() env['debian_chroot'] = "piuparts:%s" % self.name try: subprocess.call(['chroot', self.name, 'bash', '-l'], env=env) except: pass def run(self, command, ignore_errors=False): prefix = [] if settings.eatmydata and os.path.isfile(os.path.join(self.name, 'usr/bin/eatmydata')): prefix.append('eatmydata') if settings.schroot: return run( ["schroot", "--preserve-environment", "--run-session", "--chroot", "session:" + self.schroot_session, "--directory", "/", "-u", "root", "--"] + prefix + command, ignore_errors=ignore_errors, timeout=settings.max_command_runtime) elif settings.docker_image: return run( ['docker', 'exec', self.docker_container,] + prefix + command, ignore_errors=ignore_errors, timeout=settings.max_command_runtime, ) else: return run(["chroot", self.name] + prefix + command, ignore_errors=ignore_errors, timeout=settings.max_command_runtime) def mkdir_p(self, path): fullpath = self.relative(path) if not os.path.isdir(fullpath): os.makedirs(fullpath) def create_apt_sources(self, distro): """Create an /etc/apt/sources.list with a given distro.""" lines = [] lines.extend(settings.distro_config.get_deb_lines( distro, settings.debian_mirrors[0][1])) for mirror, components in settings.debian_mirrors[1:]: lines.append("deb %s %s %s" % (mirror, distro, " ".join(components))) for repo in settings.extra_repos: lines.append(repo) create_file(self.relative("etc/apt/sources.list"), "\n".join(lines) + "\n") logging.debug("sources.list:\n" + indent_string("\n".join(lines))) def enable_testdebs_repo(self, update=True): if settings.testdebs_repo: if settings.testdebs_repo.startswith("deb"): debline = settings.testdebs_repo elif settings.testdebs_repo.startswith("/"): debline = "deb [ trusted=yes ] file://%s ./" % settings.testdebs_repo else: debline = "deb [ trusted=yes ] %s ./" % settings.testdebs_repo logging.debug("enabling testdebs repository '%s'" % debline) create_file(self.relative("etc/apt/sources.list.d/piuparts-testdebs-repo.list"), debline + "\n") if update: self.run(["apt-get", "update"]) def disable_testdebs_repo(self): if settings.testdebs_repo: logging.debug("disabling testdebs repository") remove_files([self.relative("etc/apt/sources.list.d/piuparts-testdebs-repo.list")]) def create_apt_conf(self): """Create /etc/apt/apt.conf.d/piuparts inside the chroot.""" lines = ['APT::Get::Assume-Yes "yes";\n'] lines.append('APT::Install-Recommends "%d";\n' % int(settings.install_recommends)) lines.append('APT::Install-Suggests "%d";\n' % int(settings.install_suggests)) lines.append('APT::Get::AllowUnauthenticated "%s";\n' % settings.apt_unauthenticated) lines.append('Acquire::PDiffs "false";\n') if settings.no_check_valid_until: lines.append('Acquire::Check-Valid-Until "false";\n') if settings.proxy: proxy = settings.proxy elif "http_proxy" in os.environ: proxy = os.environ["http_proxy"] else: proxy = None pat = re.compile(r"^Acquire::http::Proxy\s+\"([^\"]+)\"", re.I) p = subprocess.Popen(["apt-config", "dump"], stdout=subprocess.PIPE, universal_newlines=True) stdout, _ = p.communicate() if stdout: for line in stdout.split("\n"): m = re.match(pat, line) if proxy is None and m: proxy = m.group(1) if proxy: lines.append('Acquire::http::Proxy "%s";\n' % proxy) if settings.dpkg_force_unsafe_io: lines.append('Dpkg::Options {"--force-unsafe-io";};\n') if settings.dpkg_force_confdef: lines.append('Dpkg::Options {"--force-confdef";};\n') create_file(self.relative("etc/apt/apt.conf.d/piuparts"), "".join(lines)) def create_dpkg_conf(self): """Create /etc/dpkg/dpkg.cfg.d/piuparts inside the chroot.""" lines = [] if settings.dpkg_force_unsafe_io: lines.append('force-unsafe-io\n') if settings.dpkg_force_confdef: lines.append('force-confdef\n') logging.info("Warning: dpkg has been configured to use the force-confdef option. This will hide problems, see #466118.") if lines: self.mkdir_p("etc/dpkg/dpkg.cfg.d") create_file(self.relative("etc/dpkg/dpkg.cfg.d/piuparts"), "".join(lines)) def create_policy_rc_d(self): """Create a policy-rc.d that prevents daemons from running.""" full_name = self.relative("usr/sbin/policy-rc.d") policy = "#!/bin/sh\n" if settings.allow_database: policy += 'test "$1" = "mysql" && exit 0\n' policy += 'test "$1" = "postgresql" && exit 0\n' policy += 'test "$1" = "postgresql-8.3" && exit 0\n' policy += 'test "$1" = "firebird2.5-super" && exit 0\n' policy += 'test "$1" = "firebird3.0" && exit 0\n' policy += "exit 101\n" create_file(full_name, policy) os.chmod(full_name, 0o755) logging.debug("Created policy-rc.d and chmodded it.") def create_resolv_conf(self): """Update resolv.conf based on the current configuration in the host system. Strip comments and whitespace.""" if settings.docker_image: # Docker takes care of this return full_name = self.relative("etc/resolv.conf") resolvconf = "" with open("/etc/resolv.conf", "r") as f: for line in f: if line.strip() and not line.startswith(('#', ';')): resolvconf += line.strip() + '\n' create_file(full_name, resolvconf) os.chmod(full_name, 0o644) logging.debug("Created resolv.conf.") def setup_minimal_chroot(self): """Set up a minimal Debian system in a chroot.""" logging.debug("Setting up minimal chroot for %s at %s." % (settings.debian_distros[0], self.name)) prefix = [] if settings.eatmydata and os.path.isfile('/usr/bin/eatmydata'): prefix.append('eatmydata') options = [] if settings.do_not_verify_signatures: logging.info("Warning: not using --keyring option when running debootstrap!") else: options.append("--keyring=%s" % settings.keyring) if settings.eatmydata: options.append('--include=eatmydata') if settings.merged_usr: options.append('--merged-usr') else: options.append('--no-merged-usr') options.append('--components=%s' % ','.join(settings.debian_mirrors[0][1])) if settings.arch: options.append('--arch=%s' % settings.arch) run(prefix + ["debootstrap", "--variant=minbase"] + options + [settings.debian_distros[0], self.name, settings.distro_config.get_mirror(settings.debian_distros[0])]) self.bootstrapped = True def minimize(self): """Minimize a chroot by removing (almost all) unnecessary packages""" if settings.skip_minimize or not settings.minimize: return self.run(["apt-get", "install", "debfoster"]) debfoster_command = ["debfoster"] + settings.debfoster_options if settings.eatmydata: debfoster_command.append("eatmydata") self.run(debfoster_command) remove_files([self.relative("var/lib/debfoster/keepers")]) self.run(["dpkg", "--purge", "debfoster"]) def configure_chroot(self): """Configure a chroot according to current settings""" os.environ["PIUPARTS_DISTRIBUTION"] = settings.distro_config.get_distribution(settings.debian_distros[0]) if not settings.keep_sources_list: self.create_apt_sources(settings.debian_distros[0]) self.create_apt_conf() self.create_dpkg_conf() self.create_policy_rc_d() self.create_resolv_conf() for bindmount in settings.bindmounts: self.mount(bindmount, bindmount, opts="bind") if not os.path.exists(self.name + '/dev/null'): run(['mknod', '-m' ,'666', self.name + '/dev/null', 'c', '1', '3']) def remember_available_md5(self): """Keep a history of 'apt-cache dumpavail | md5sum' after initial setup and each dist-upgrade step to notice outdated reference chroot metadata""" errorcode, avail_md5 = self.run(["sh", "-c", "apt-cache dumpavail | md5sum"]) self.avail_md5_history.append(avail_md5.split()[0]) def remember_initial_selections(self): """Remember initial selections to easily recognize mismatching chroot metadata""" self.initial_selections = self.get_selections() def upgrade_to_distros(self, distros, packages, apt_get_upgrade=False): """Upgrade a chroot installation to each successive distro.""" for distro in distros: logging.debug("Upgrading %s to %s" % (self.name, distro)) os.environ["PIUPARTS_DISTRIBUTION_NEXT"] = settings.distro_config.get_distribution(distro) self.create_apt_sources(distro) # Run custom scripts before upgrade self.run_scripts("pre_distupgrade") self.run(["apt-get", "update"]) if apt_get_upgrade: self.run(["apt-get", "-y", "upgrade"]) self.run(["apt-get", "-yf", "dist-upgrade"]) self.remember_available_md5() os.environ["PIUPARTS_DISTRIBUTION_PREV"] = os.environ["PIUPARTS_DISTRIBUTION"] os.environ["PIUPARTS_DISTRIBUTION"] = settings.distro_config.get_distribution(distro) # Sometimes dist-upgrade won't upgrade the packages we want # to test because the new version depends on a newer library, # and installing that would require removing the old version # of the library, and we've told apt-get not to remove # packages. So, we force the installation like this. if packages: known_packages = self.get_known_packages(packages + settings.extra_old_packages) self.install_packages_by_name(known_packages) # Run custom scripts after upgrade self.run_scripts("post_distupgrade") self.check_for_no_processes() def get_known_packages(self, packages): """Does apt-get (or apt-cache) know about a set of packages?""" known_packages = [] new_packages = [] for name in packages: (status, output) = self.run(["apt-cache", "show", "--no-all-versions", name], ignore_errors=True) # apt-cache reports status for some virtual packages and packages # in status config-files-remaining state without installation # candidate -- but only real packages have Filename/MD5sum/SHA* if status != 0 or re.search(r'^(Filename|MD5sum|SHA1|SHA256):', output, re.M) is None: new_packages.append(name) else: known_packages.append(name) if not known_packages: logging.info("apt-cache does not know about any of the requested packages") else: logging.info("apt-cache knows about the following packages: " + ", ".join(known_packages)) if new_packages: logging.info("the following packages are not in the archive: " + ", ".join(new_packages)) return known_packages def copy_files(self, source_names, target_name): """Copy files in 'source_name' to file/dir 'target_name', relative to the root of the chroot.""" target_name = self.relative(target_name) logging.debug("Copying %s to %s" % (", ".join(source_names), target_name)) for source_name in source_names: try: shutil.copy(source_name, target_name) except IOError as detail: logging.error("Error copying %s to %s: %s" % (source_name, target_name, detail)) panic() def list_installed_files(self, pre_info, post_info): """List the new files installed, removed and modified between two dir trees. Actually, it is a nice output of the funcion diff_meta_dat.""" (new, removed, modified) = diff_meta_data(pre_info, post_info) file_owners = self.get_files_owned_by_packages() if new: logging.debug("New installed files on system:\n" + file_list(new, file_owners)) else: logging.debug("The package did not install any new file.\n") if removed: logging.debug("The following files have disappeared:\n" + file_list(removed, file_owners)) if modified: logging.debug("The following files have been modified:\n" + file_list(modified, file_owners)) else: logging.debug("The package did not modify any file.\n") def is_installed(self, packages): if not packages: return True retcode, output = self.run(["dpkg-query", "-f", "${Package} ${Status}\n", "-W"] + packages, ignore_errors=True) if retcode != 0: return False installed = True for line in output.splitlines(): pkg, desired, whatever, status = line.split() if status != 'installed': logging.error("Installation of %s failed", pkg) installed = False return installed def install_packages(self, package_files, packages, with_scripts=True, reinstall=False): if package_files: self.install_package_files(package_files, packages, with_scripts=with_scripts) else: self.install_packages_by_name(packages, with_scripts=with_scripts, reinstall=reinstall) def install_package_files(self, package_files, packages=None, with_scripts=False): if packages and settings.testdebs_repo: self.install_packages_by_name(packages, with_scripts=with_scripts) return if package_files: # Check whether apt-get can install debs (supported since apt 1.1) # # If it can, this is preferable to the traditional # `dpkg -i foo.deb && apt-get -yf install` # approach since 'apt-get -yf install' can 'resolve' dependency # problems by removing the package we are trying to install (status, output) = self.run(["dpkg-query", "-f", "${Version}\n", "-W", "apt"], ignore_errors=True) apt_can_install_debs = apt_pkg.version_compare(output.strip(), "1.1") >= 0 # This must look like a local path so that apt-get can # distinguish it from a 'package/suite' request. self.copy_files(package_files, "tmp") tmp_files = [os.path.join("./tmp", os.path.basename(a)) for a in package_files] if with_scripts: self.run_scripts("pre_install") if apt_can_install_debs: # --allow-downgrades is also required in order to permit # installing a deb with the same version as that already # installed apt_get_install = ["apt-get", "-y", "--allow-downgrades"] else: apt_get_install = ["apt-get", "-yf"] apt_get_install.extend(settings.distro_config.get_target_flags( os.environ["PIUPARTS_DISTRIBUTION"])) apt_get_install.append("install") if settings.list_installed_files: pre_info = self.get_tree_meta_data() if apt_can_install_debs: self.run(apt_get_install + tmp_files) else: (ret, out) = self.run(["dpkg", "-i"] + tmp_files, ignore_errors=True) if ret != 0: if "dependency problems - leaving unconfigured" in out: pass else: logging.error("Installation failed") panic() if settings.list_installed_files: self.list_installed_files(pre_info, self.get_tree_meta_data()) self.run(apt_get_install) if settings.list_installed_files: self.list_installed_files(pre_info, self.get_tree_meta_data()) if not self.is_installed(unqualify(packages)): logging.error("Could not install %s.", " ".join(unqualify(packages))) panic() logging.info("Installation of %s ok", tmp_files) if with_scripts: self.run_scripts("post_install") remove_files([self.relative(name) for name in tmp_files]) def install_packages_by_name(self, packages, with_scripts=True, reinstall=False): if packages: if with_scripts: self.run_scripts("pre_install") self.run(["apt-cache", "policy"]) self.run(["apt-cache", "policy"] + unqualify(packages)) if settings.list_installed_files: pre_info = self.get_tree_meta_data() target_flags = settings.distro_config.get_target_flags(os.environ["PIUPARTS_DISTRIBUTION"]) self.apt_get_install(to_install=packages, flags=target_flags, reinstall=reinstall) if settings.list_installed_files: self.list_installed_files(pre_info, self.get_tree_meta_data()) if with_scripts: self.run_scripts("post_install") def apt_get_install(self, to_install=[], to_remove=[], to_purge=[], flags=[], reinstall=False): command = ["apt-get", "-y"] + flags + ["install"] if reinstall: command.append("--reinstall") command.extend(to_install) command.extend(["%s-" % x for x in unqualify(to_remove)]) command.extend(["%s_" % x for x in unqualify(to_purge)]) self.run(command) def get_selections(self): """Get current package selections in a chroot.""" # "${Status}" emits three columns, e.g. "install ok installed" # "${binary:Package}" requires a multi-arch dpkg, so fall back to "${Package}" on older versions (status, output) = self.run(["dpkg-query", "-W", "-f", "${Status}\\t${binary:Package}\\t${Package}\\t${Version}\\n"]) vdict = {} for line in [line for line in output.split("\n") if line.strip()]: token = line.split() status = token[0] name = token[3] if status == "install": version = token[-1] else: version = None vdict[name] = (status, version) return vdict def get_diversions(self): """Get current dpkg-divert --list in a chroot.""" if not settings.check_broken_diversions: return (status, output) = self.run(["dpkg-divert", "--list"]) return output.split("\n") def get_modified_diversions(self, pre_install_diversions, post_install_diversions=None): """Check that diversions in chroot are identical (though potentially reordered).""" if post_install_diversions is None: post_install_diversions = self.get_diversions() removed = [ln for ln in pre_install_diversions if not ln in post_install_diversions] added = [ln for ln in post_install_diversions if not ln in pre_install_diversions] return (removed, added) def check_debsums(self): (status, output) = run(["debsums", "--root", self.name, "-ac", "--ignore-obsolete"], ignore_errors=True) if status != 0: logging.error("FAIL: debsums reports modifications inside the chroot:\n%s" % indent_string(output.replace(self.name, ""))) if not settings.warn_on_debsums_errors: panic() def check_adequate(self, packages): """Run adequate and categorize output according to our needs. """ packages = unqualify([p for p in packages if not p.endswith("=None")]) if packages and settings.adequate and os.path.isfile('/usr/bin/adequate'): (status, output) = run(["dpkg-query", "-f", "${Version}\n", "-W", "adequate"], ignore_errors=True) logging.info("Running adequate version %s now." % output.strip()) adequate_tags = [ 'bin-or-sbin-binary-requires-usr-lib-library', 'broken-binfmt-detector', 'broken-binfmt-interpreter', 'incompatible-licenses', 'ldd-failure', 'library-not-found', 'missing-alternative', 'missing-copyright-file', 'missing-pkgconfig-dependency', 'missing-symbol-version-information', 'program-name-collision', 'py-file-not-bytecompiled', 'pyshared-file-not-bytecompiled', 'symbol-size-mismatch', 'undefined-symbol', ] boring_tags = [ 'obsolete-conffile', 'broken-symlink', ] ignored_tags = [] (status, output) = run(["adequate", "--root", self.name] + packages, ignore_errors=True) for tag in ignored_tags: # ignore some tags _regex = '^[^:]+: ' + tag + ' .*\n' output = re.compile(_regex, re.MULTILINE).sub('', output) if output: inadequate_results = '' boring_results = '' for tag in adequate_tags: if ' ' + tag + ' ' in output: inadequate_results += ' ' + tag + ' ' for tag in boring_tags: if ' ' + tag + ' ' in output: boring_results += ' ' + tag + ' ' if settings.warn_if_inadequate: error_code = 'WARN' else: error_code = 'FAIL' logging.error("%s: Inadequate results from running adequate!\n%s" % (error_code, indent_string(output.replace(self.name, "")))) if inadequate_results: logging.error("%s: Running adequate resulted in inadequate tags found: %s" % (error_code, inadequate_results)) if boring_results: logging.error("%s: Running adequate resulted in less interesting tags found: %s" % (error_code, boring_results)) if not boring_results and not inadequate_results: logging.error("%s: Found unknown tags running adequate." % error_code) if status != 0: logging.error("%s: Exit code from adequate was %s!" % (error_code, status)) if not settings.warn_if_inadequate: panic() def list_paths_with_symlinks(self): file_owners = self.get_files_owned_by_packages() bad = [] overwrites = False usrmerge = set() for f in sorted(file_owners.keys()): dn, fn = os.path.split(f) dc = canonicalize_path(self.name, dn) if dn != dc: # Allow the /usr merge to have taken place. For example, if # f (the file recorded in the dpkg database) is /bin/cat, # then dn is /bin, and it's OK for /bin to have become a # symlink to /usr/bin. Similarly /sbin, /lib, /libQUAL # (/lib32 etc.) or any subdirectory of /lib or /libQUAL # can be /usr-merged. if dc == '/usr' + dn and (dn in ('/bin', '/sbin') or dn.startswith('/lib')): # Only report each directory once if dn not in usrmerge: usrmerge.add(dn) logging.info('%s converted to %s by /usr merge', dn, dc) continue fc = os.path.join(dc, fn) of = ", ".join(file_owners[f]) if fc in file_owners: overwrites = True ofc = ", ".join(file_owners[fc]) else: ofc = "?" bad.append("%s (%s) != %s (%s)" % (f, of, fc, ofc)) for (link, target) in canonicalize_path(self.name, dn, report_links=True): bad.append(" %s -> %s" % (link, target)) if bad: if overwrites: msg = "FAIL: silently overwrites files via directory symlinks:\n" else: msg = "installs objects over existing directory symlinks:\n" msg += indent_string("\n".join(bad)) if not settings.warn_on_install_over_symlink: logging.error(msg) panic() else: logging.info(msg) def remove_packages(self, packages, ignore_errors=False): """Remove packages in a chroot.""" if packages: self.run(["apt-get", "remove"] + ["%s-" % x if x.endswith('+') else x for x in unqualify(packages)], ignore_errors=ignore_errors) def purge_packages(self, packages, ignore_errors=False): """Purge packages in a chroot.""" if packages: self.run(["dpkg", "--purge"] + unqualify(packages), ignore_errors=ignore_errors) def restore_selections(self, reference_chroot_state, packages_qualified): """Restore package selections in a chroot to the state in 'reference_chroot_state'.""" if reference_chroot_state["avail_md5"] != self.avail_md5_history: logging.warn("History of available packages does not match - reference chroot may be outdated") logging.debug(" reference: %s" % " ".join(reference_chroot_state["avail_md5"])) logging.debug(" current : %s" % " ".join(self.avail_md5_history)) self.list_paths_with_symlinks() self.check_debsums() self.check_adequate(packages_qualified) # Run custom scripts before removing all packages. self.run_scripts("pre_remove") selections = reference_chroot_state["selections"] packages = unqualify(packages_qualified) changes = diff_selections(self, selections) deps = {} nondeps = {} for name, state_version in six.iteritems(changes): if name in packages: nondeps[name] = state_version else: deps[name] = state_version deps_to_remove = [name for name, (state, version) in six.iteritems(deps) if state == "remove"] deps_to_purge = [name for name, (state, version) in six.iteritems(deps) if state == "purge"] nondeps_to_remove = [name for name, (state, version) in six.iteritems(nondeps) if state == "remove"] nondeps_to_purge = [name for name, (state, version) in six.iteritems(nondeps) if state == "purge"] all_to_remove = deps_to_remove + deps_to_purge + nondeps_to_remove + nondeps_to_purge all_to_install = [(name, version) for name, (state, version) in six.iteritems(deps) if state == "install"] all_to_install += [(name, version) for name, (state, version) in six.iteritems(nondeps) if state == "install"] # First remove all packages (and reinstall missing ones). self.remove_packages(deps_to_remove) if all_to_install: version_qualified = [name for (name, version) in all_to_install if version is None] version_qualified += ["%s=%s" % (name, version) for (name, version) in all_to_install if version is not None] self.apt_get_install(to_remove=all_to_remove, to_install=version_qualified, flags=["--no-install-recommends", "--force-yes"]) # reinstall potentially downgraded packages, they may have # missing files in case of unfortunate Breaks+Replaces timing self.apt_get_install(to_install=version_qualified, flags=["--no-install-recommends", "--reinstall"]) else: self.remove_packages(all_to_remove) # Run custom scripts after removing all packages. self.run_scripts("post_remove") if not settings.skip_cronfiles_test: cronfiles = self.check_if_cronfiles(packages) if cronfiles: self.check_output_cronfiles(cronfiles) if not settings.skip_logrotatefiles_test: logrotatefiles = self.check_if_logrotatefiles(packages) if logrotatefiles: installed = self.install_logrotate() self.check_output_logrotatefiles(logrotatefiles) self.purge_packages(installed) # Then purge all packages being depended on. self.purge_packages(deps_to_purge) # Finally, purge actual packages. self.purge_packages(nondeps_to_purge) # Run custom scripts after purge all packages. self.run_scripts("post_purge") # Now do a final run to see that everything worked. self.run(["dpkg", "--purge", "--pending"]) self.run(["dpkg", "--remove", "--pending"]) def get_tree_meta_data(self): """Return the filesystem meta data for all objects in the chroot.""" self.run(["apt-get", "clean"]) logging.debug("Recording chroot state") root = self.relative(".") uidmap = {} with open(self.relative("etc/passwd"), "r") as passwd: for line in passwd: (usr, x, uid) = line.split(":")[0:3] uidmap[int(uid)] = usr gidmap = {} with open(self.relative("etc/group"), "r") as group: for line in group: (grp, x, gid) = line.split(":")[0:3] gidmap[int(gid)] = grp vdict = {} proc = os.path.join(root, "proc") devpts = os.path.join(root, "dev/pts") for dirpath, dirnames, filenames in os.walk(root): assert dirpath[:len(root)] == root if dirpath[:len(proc) + 1] in [proc, proc + "/"]: continue if dirpath[:len(devpts) + 1] in [devpts, devpts + "/"]: continue for name in [dirpath] + \ [os.path.join(dirpath, f) for f in filenames]: st = os.lstat(name) if stat.S_ISLNK(st.st_mode): target = os.readlink(name) else: target = None if stat.S_ISDIR(st.st_mode): name += "/" if st.st_uid in uidmap: user = uidmap[st.st_uid] else: user = "#%d" % st.st_uid if st.st_gid in gidmap: group = gidmap[st.st_gid] else: group = "#%d" % st.st_gid vdict[name[len(root):]] = FileInfo(st, target, user, group) return vdict def get_state_meta_data(self): chroot_state = {} chroot_state["initial_selections"] = self.initial_selections chroot_state["avail_md5"] = self.avail_md5_history chroot_state["tree"] = self.get_tree_meta_data() chroot_state["selections"] = self.get_selections() chroot_state["diversions"] = self.get_diversions() return chroot_state def relative(self, pathname): if pathname.startswith('/'): return os.path.join(self.name, pathname[1:]) return os.path.join(self.name, pathname) def get_files_owned_by_packages(self): """Return dict[filename] = [packagenamelist].""" vdir = self.relative("var/lib/dpkg/info") vdict = {} for basename in os.listdir(vdir): if basename.endswith(".list"): pkg = basename[:-len(".list")] for line in readlines_file(os.path.join(vdir, basename)): pathname = line.strip() if pathname in vdict: vdict[pathname].append(pkg) else: vdict[pathname] = [pkg] return vdict def check_for_no_processes(self, fail=None): """Check there are no processes running inside the chroot.""" if settings.docker_image: (status, output) = run(["docker", "top", self.docker_container]) count = len(output.strip().split("\n")) - 2 # header + bash launched on container creation else: (status, output) = run(["lsof", "-w", "+D", self.name], ignore_errors=True) count = len(output.split("\n")) - 1 if count > 0: if fail is None: fail = not settings.allow_database logging.error("%s: Processes are running inside chroot:\n%s" % ("FAIL" if fail else "WARN", indent_string(output))) if fail: self.terminate_running_processes() panic() def terminate_running_processes(self): """Terminate all processes running in the chroot.""" if settings.docker_image: # Docker takes care of this return seen = [] while True: p = subprocess.Popen(["lsof", "-t", "+D", self.name], stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True) stdout, _ = p.communicate() if not stdout: break pidlist = reversed([int(pidstr) for pidstr in stdout.split("\n") if len(pidstr) and int(pidstr) > 0]) if not pidlist: break for pid in pidlist: try: signo = (SIGTERM, SIGKILL)[pid in seen] os.kill(pid, signo) seen.append(pid) logging.debug("kill -%d %d" % (signo, pid)) time.sleep(0.25) except OSError: pass time.sleep(5) # If /selinux is present, assume that this is the only supported # location by libselinux. Otherwise use the new location. # /selinux was shipped by the libselinux package until wheezy. def selinuxfs_path(self): if os.path.isdir(self.relative('/selinux')): return '/selinux' else: return '/sys/fs/selinux' def mount(self, source, path, fstype=None, opts=None, no_mkdir=False): """Mount something into the chroot and remember it for unmount_all().""" path = canonicalize_path(self.name, path) if not no_mkdir: self.mkdir_p(path) fullpath = self.relative(path) command = ["mount"] if fstype is not None: command.extend(["-t", fstype]) if opts is not None: command.extend(["-o", opts]) command.extend([source, fullpath]) run(command) self.mounts.append(fullpath) def unmount_all(self): """Unmount everything we mount()ed into the chroot.""" # Workaround to unmount /proc/sys/fs/binfmt_misc which is mounted by # update-binfmts but never unmounted # This workaround can be removed once #847788 is fixed binfmt_misc = self.relative("/proc/sys/fs/binfmt_misc") if os.path.ismount(binfmt_misc): self.mounts.append(binfmt_misc) for mountpoint in reversed(self.mounts): run(["umount", mountpoint], ignore_errors=True) def mount_proc(self): """Mount /proc etc. inside chroot.""" self.mount("proc", "/proc", fstype="proc") etcmtab = self.relative("etc/mtab") if not os.path.lexists(etcmtab): os.symlink("../proc/mounts", etcmtab) self.mount("devpts", "/dev/pts", fstype="devpts", opts="newinstance,noexec,nosuid,gid=5,mode=0620,ptmxmode=0666") dev_ptmx_rel_path = self.relative("dev/ptmx") if not os.path.islink(dev_ptmx_rel_path): if not os.path.exists(dev_ptmx_rel_path): os.mknod(dev_ptmx_rel_path, 0o0666 | stat.S_IFCHR, os.makedev(5, 2)) self.mount(self.relative("dev/pts/ptmx"), "/dev/ptmx", opts="bind", no_mkdir=True) p = subprocess.Popen(["tty"], stdout=subprocess.PIPE, universal_newlines=True) stdout, _ = p.communicate() current_tty = stdout.strip() if p.returncode == 0 and os.path.exists(current_tty): dev_console = self.relative("/dev/console") if not os.path.exists(dev_console): os.mknod(dev_console, 0o0600, os.makedev(5, 1)) self.mount(current_tty, "/dev/console", opts="bind", no_mkdir=True) self.mount("tmpfs", "/dev/shm", fstype="tmpfs", opts="size=65536k") if selinux_enabled(): self.mount("/sys/fs/selinux", self.selinuxfs_path(), opts="bind,ro") def is_ignored(self, pathname, info="PATH"): """Is a file (or dir or whatever) to be ignored?""" if pathname in settings.ignored_files: return True if ':' + pathname in settings.ignored_files: logging.info("IGNORED %s: %s" % (info, pathname)) return True for pattern in settings.ignored_patterns: if pattern[0] == ':': verbose = True pattern = pattern[1:] else: verbose = False if re.search('^' + pattern + '$', pathname): if verbose: logging.info("IGNORED %s: %s" % (info, pathname)) return True return False def check_for_broken_symlinks(self, warn_only=None, file_owners={}): """Check that all symlinks in chroot are non-broken.""" if not settings.check_broken_symlinks: return broken = [] for dirpath, dirnames, filenames in os.walk(self.name): # Remove /proc within chroot to avoid lots of spurious errors. if dirpath == self.name and "proc" in dirnames: dirnames.remove("proc") for filename in filenames: full_name = name = os.path.join(dirpath, filename) if name.startswith(self.name): name = name[len(self.name):] ret = is_broken_symlink(self.name, dirpath, filename) if ret and not self.is_ignored(name, info="broken symlink"): try: target = os.readlink(full_name) except os.error: target = "<unknown>" entry = "%s -> %s" % (name, target) if name in file_owners: entry += " (%s)" % ", ".join(file_owners[name]) broken.append(entry) if broken: if settings.warn_broken_symlinks or warn_only: logging.error("WARN: Broken symlinks:\n%s" % indent_string("\n".join(broken))) else: logging.error("FAIL: Broken symlinks:\n%s" % indent_string("\n".join(broken))) panic() else: logging.debug("No broken symlinks as far as we can find.") def check_if_cronfiles(self, packages): """Check if the packages have cron files under /etc/cron.d and in case positive, it returns the list of files. """ # FIXME! Does not work for M-A: same packages vdir = self.relative("var/lib/dpkg/info") vlist = [] for p in packages: basename = p + ".list" if not os.path.exists(os.path.join(vdir, basename)): continue for line in readlines_file(os.path.join(vdir, basename)): pathname = line.strip() if pathname.startswith("/etc/cron."): if os.path.isfile(self.relative(pathname.strip("/"))): st = os.lstat(self.relative(pathname.strip("/"))) mode = st[stat.ST_MODE] # XXX /etc/cron.d/ files are NOT executables if (mode & stat.S_IEXEC): vlist.append(pathname) logging.info("Package " + p + " contains cron file: " + pathname) return vlist def check_output_cronfiles(self, list): """Check if a given list of cronfiles has any output. Executes cron file as cron would do (except for SHELL)""" failed = False for vfile in list: if not os.path.exists(self.relative(vfile.strip("/"))): continue (retval, output) = self.run([vfile]) if output: failed = True logging.error("FAIL: Cron file %s has output with package removed" % vfile) if failed: panic() def check_if_logrotatefiles(self, packages): """Check if the packages have logrotate files under /etc/logrotate.d and in case positive, it returns the list of files. """ # FIXME! Does not work for M-A: same packages vdir = self.relative("var/lib/dpkg/info") vlist = [] for p in packages: basename = p + ".list" if not os.path.exists(os.path.join(vdir, basename)): continue for line in readlines_file(os.path.join(vdir, basename)): pathname = line.strip() if os.path.dirname(pathname) == "/etc/logrotate.d": if os.path.isfile(self.relative(pathname.strip("/"))): vlist.append(pathname) logging.info("Package " + p + " contains logrotate file: " + pathname) return vlist def install_logrotate(self): """Install logrotate for check_output_logrotatefiles, and return the list of packages that were installed""" old_selections = self.get_selections() self.run(['apt-get', 'install', '-y', 'logrotate']) diff = diff_selections(self, old_selections) return diff.keys() def check_output_logrotatefiles(self, list): """Check if a given list of logrotatefiles has any output. Executes logrotate file as logrotate would do from cron (except for SHELL)""" failed = False for vfile in list: if not os.path.exists(self.relative(vfile.strip("/"))): continue (retval, output) = self.run(['/usr/sbin/logrotate', vfile]) if output or retval != 0: failed = True logging.error("FAIL: Logrotate file %s exits with error or has output with package removed" % vfile) if failed: panic() def run_scripts(self, step, ignore_errors=False): """ Run custom scripts to given step post-install|remove|purge""" errorcodes = 0 if not settings.scriptsdirs: return errorcodes logging.info("Running scripts " + step) basepath = self.relative("tmp/scripts/") if not os.path.exists(basepath): logging.error("Scripts directory %s does not exist" % basepath) panic() list_scripts = sorted(os.listdir(basepath)) for vfile in list_scripts: if vfile.startswith(step): script = os.path.join("tmp/scripts", vfile) errorcode, output = self.run([script], ignore_errors=ignore_errors) errorcodes = errorcodes | errorcode return errorcodes def selinux_enabled(enabled_test="/usr/sbin/selinuxenabled"): if os.access(enabled_test, os.X_OK): retval, output = run([enabled_test], ignore_errors=True) if retval == 0: return True else: return False def objects_are_different(obj1, obj2): """Are filesystem objects different based on their meta data?""" if (obj1.st.st_mode != obj2.st.st_mode or obj1.user != obj2.user or obj1.group != obj2.group or obj1.target != obj2.target): return True if stat.S_ISREG(obj1.st.st_mode): return obj1.st.st_size != obj2.st.st_size # or obj1.st.st_mtime != obj2.st.st_mtime return False def format_object_attributes(obj): st = obj.st ft = "" if stat.S_ISDIR(st.st_mode): ft += "d" if stat.S_ISCHR(st.st_mode): ft += "c" if stat.S_ISBLK(st.st_mode): ft += "b" if stat.S_ISREG(st.st_mode): ft += "-" if stat.S_ISFIFO(st.st_mode): ft += "p" if stat.S_ISLNK(st.st_mode): ft += "l" if stat.S_ISSOCK(st.st_mode): ft += "s" res = "(%s, %s, %s %o, %d, %s)" % ( obj.user, obj.group, ft, st.st_mode, st.st_size, obj.target) return res def diff_meta_data(tree1, tree2, quiet=False): """Compare two dir trees and return list of new files (only in 'tree2'), removed files (only in 'tree1'), and modified files.""" tree1_c = tree1.copy() tree2_c = tree2.copy() for name in settings.ignored_files: if name[0] == ':': verbose = not quiet name = name[1:] else: verbose = False if name in tree1: if verbose: logging.info("IGNORED PATH@1: %s" % name) del tree1_c[name] if name in tree2: if verbose: logging.info("IGNORED PATH@2: %s" % name) del tree2_c[name] for pattern in settings.ignored_patterns: if pattern[0] == ':': verbose = not quiet pattern = pattern[1:] else: verbose = False pat = re.compile(pattern) for name in tree1.keys(): m = pat.search(name) if m: if verbose: logging.info("IGNORED PATH@1: %s" % name) del tree1_c[name] for name in tree2.keys(): m = pat.search(name) if m: if verbose: logging.info("IGNORED PATH@2: %s" % name) del tree2_c[name] modified = [] for name in tree1.keys(): if name in tree2_c: if objects_are_different(tree1[name], tree2[name]): if not quiet: logging.debug("Modified(user, group, mode, size, target): %s expected%s != found%s" % (name, format_object_attributes(tree1[name]), format_object_attributes(tree2[name]))) modified.append((name, tree1[name])) del tree1_c[name] del tree2_c[name] removed = [x for x in six.iteritems(tree1_c)] new = [x for x in six.iteritems(tree2_c)] # fix for #586793 # prune rc?.d symlinks renamed by insserv pat1 = re.compile(r"^(/etc/rc.\.d/)[SK][0-9]{2}(.*)$") for name1, data1 in removed[:]: m = pat1.search(name1) if m: pat2 = re.compile(r"^" + m.group(1) + r"[SK][0-9]{2}" + m.group(2) + r"$") for name2, data2 in new[:]: m = pat2.search(name2) if m: logging.debug("File was renamed: %s\t=> %s" % (name1, name2)) removed.remove((name1, data1)) new.remove((name2, data2)) # this is again special casing due to the behaviour of a single package :( # general tracking of moved files would be the better approach, probably. return new, removed, modified def file_list(meta_infos, file_owners): """Return list of indented filenames.""" meta_infos = sorted(meta_infos[:]) vlist = [] for name, obj in meta_infos: info = "" if obj.target is not None: info = " -> %s" % obj.target vlist.append(" %s%s\t" % (name, info)) key = name if key.endswith('/'): key = key[:-1] if key in file_owners: vlist.append(" owned by: %s\n" % ", ".join(file_owners[key])) else: vlist.append(" not owned\n") return "".join(vlist) def offending_packages(meta_infos, file_owners): """Return a Set of offending packages.""" pkgset = set() for name, data in meta_infos: if name in file_owners: for pkg in file_owners[name]: pkgset.add(pkg) return pkgset def prune_files_list(files, depsfiles): """Remove elements from 'files' that are in 'depsfiles', and return the list of removed elements. """ warn = [] depfiles_names = [x[0] for x in depsfiles] for vfile in files[:]: if vfile[0] in depfiles_names: files.remove(vfile) warn.append(vfile) return warn def diff_selections(chroot, selections): """Compare original and current package selection. Return dict where dict[package_name] = original_status, that is, the value in the dict is the state that the package needs to be set to to restore original selections.""" changes = {} current = chroot.get_selections() for name, (value, version) in six.iteritems(current): if name not in selections: changes[name] = ("purge", None) elif selections[name][0] != value and \ selections[name][0] in ["purge", "install"]: changes[name] = selections[name] for name, (value, version) in six.iteritems(selections): if name not in current or \ current[name][1] != version: changes[name] = selections[name] return changes def get_package_names_from_package_files(package_files): """Return list of package names given list of package file names.""" vlist = [] for filename in package_files: (status, output) = run(["dpkg", "--info", filename]) p = None v = None for line in [line.lstrip() for line in output.split("\n")]: if line.startswith("Package:"): p = line.split(":", 1)[1].strip() if line.startswith("Version:"): v = line.split(":", 1)[1].strip() if p is not None: if v is not None: vlist.append(p + "=" + v) else: vlist.append(p) return vlist # Method to process a changes file, returning a list of all the .deb packages # from the 'Files' stanza. def process_changes(changes): # Determine the path to the changes file, then check if it's readable. dir_path = "" changes_path = "" if not os.path.dirname(changes): changes_path = os.path.basename(changes) else: dir_path = os.path.dirname(changes) + "/" changes_path = os.path.abspath(changes) if not os.access(changes_path, os.R_OK): logging.warn(changes_path + " is not readable. Skipping.") return # Determine the packages in the changes file through the 'Files' stanza. field = 'Files' pattern = re.compile( r'^' + field + r':' + r''' # The field we want the contents from (.*?) # The contents of the field \n([^ ]|$) # Start of a new field or EOF ''', re.MULTILINE | re.DOTALL | re.VERBOSE) with open(changes_path, "r") as f: file_text = f.read() matches = pattern.split(file_text) # Append all the packages found in the changes file to a package list. package_list = [] newline_p = re.compile('\n') package_p = re.compile('.*?([^ ]+\.deb)$') for line in newline_p.split(matches[1]): if package_p.match(line): package = dir_path + package_p.split(line)[1] package_list.append(package) # Return the list. return package_list def check_results(chroot, chroot_state, file_owners, deps_info=None): """Check that current chroot state matches 'chroot_state'. If settings.warn_on_others is True and deps_info is not None, then only print a warning rather than failing if the current chroot contains files that are in deps_info but not in chroot_state["tree"]. (In this case, deps_info should be the result of chroot.get_tree_meta_data() right after the dependencies are installed, but before the actual packages to test are installed.) """ reference_info = chroot_state["tree"] ok = True if settings.check_broken_diversions: (removed, added) = chroot.get_modified_diversions(chroot_state["diversions"]) if added: logging.error("FAIL: Installed diversions (dpkg-divert) not removed by purge:\n%s" % indent_string("\n".join(added))) ok = False if removed: logging.error("FAIL: Existing diversions (dpkg-divert) removed/modified:\n%s" % indent_string("\n".join(removed))) ok = False current_info = chroot.get_tree_meta_data() if settings.warn_on_others and deps_info is not None: (new, removed, modified) = diff_meta_data(reference_info, current_info) (depsnew, depsremoved, depsmodified) = diff_meta_data(reference_info, deps_info, quiet=True) warnnew = prune_files_list(new, depsnew) warnremoved = prune_files_list(removed, depsremoved) warnmodified = prune_files_list(modified, depsmodified) else: (new, removed, modified) = diff_meta_data(reference_info, current_info) if new: if settings.warn_on_leftovers_after_purge: logging.info("Warning: Package purging left files on system:\n" + file_list(new, file_owners)) else: logging.error("FAIL: Package purging left files on system:\n" + file_list(new, file_owners)) ok = False if removed: logging.error("FAIL: After purging files have disappeared:\n" + file_list(removed, file_owners)) ok = False if modified: logging.error("FAIL: After purging files have been modified:\n" + file_list(modified, file_owners)) ok = False if settings.warn_on_others and deps_info is not None: if warnnew: msg = ("Warning: Package purging left files on system:\n" + file_list(warnnew, file_owners) + "These files seem to have been left by dependencies rather " "than by packages\nbeing explicitly tested.\n") logging.info(msg) if warnremoved: msg = ("After purging files have disappeared:\n" + file_list(warnremoved, file_owners) + "This seems to have been caused by dependencies rather " "than by packages\nbeing explicitly tested.\n") logging.info(msg) if warnmodified: msg = ("After purging files have been modified:\n" + file_list(warnmodified, file_owners) + "This seems to have been caused by dependencies rather " "than by packages\nbeing explicitly tested.\n") logging.info(msg) return ok def install_purge_test(chroot, chroot_state, package_files, packages, extra_packages): """Do an install-purge test. Return True if successful, False if not. Assume 'root' is a directory already populated with a working chroot, with packages in states given by 'selections'.""" deps_info = None os.environ["PIUPARTS_TEST"] = "install" chroot.run_scripts("pre_test") # Install packages into the chroot. os.environ["PIUPARTS_PHASE"] = "install" chroot.enable_testdebs_repo() chroot.check_for_no_processes(fail=True) chroot.check_for_broken_symlinks() chroot.run_scripts("pre_install") chroot.install_packages([], extra_packages, with_scripts=False) if settings.warn_on_others or settings.install_purge_install: # Create a metapackage with dependencies from the given packages if package_files: control_infos = [] # We were given package files, so let's get the Depends and # Conflicts directly from the .debs for deb in package_files: returncode, output = run(["dpkg", "-f", deb]) control = deb822.Deb822(output) control_infos.append(control) else: # We have package names. Use apt to get all their control # information. apt_cache_args = ["apt-cache", "show", "--no-all-versions"] if os.environ["PIUPARTS_DISTRIBUTION"] in ["lenny"]: # apt-cache in lenny does not accept version-qualified packages apt_cache_args.extend(unqualify(packages)) else: apt_cache_args.extend(packages) returncode, output = chroot.run(apt_cache_args) control_infos = deb822.Deb822.iter_paragraphs(output.splitlines()) depends = [] conflicts = [] provides = [] arch = 'all' for control in control_infos: if control.get("pre-depends"): depends.extend([x.strip() for x in control["pre-depends"].split(',')]) if control.get("depends"): depends.extend([x.strip() for x in control["depends"].split(',')]) if control.get("conflicts"): conflicts.extend([x.strip() for x in control["conflicts"].split(',')]) if control.get("provides"): provides.extend([x.strip() for x in control["provides"].split(',')]) if control.get("architecture"): a = control["architecture"] if arch == 'all': arch = a if arch != a: logging.info("architecture mismatch: %s != %s)" % (arch, a)) for provided in provides: if provided in conflicts: conflicts.remove(provided) all_depends = ", ".join(depends) all_conflicts = ", ".join(conflicts) metapackage = make_metapackage("piuparts-depends-dummy", depends=all_depends, conflicts=all_conflicts, arch=arch) cleanup_metapackage = lambda: shutil.rmtree(os.path.dirname(metapackage)) panic_handler_id = do_on_panic(cleanup_metapackage) # Install the metapackage chroot.install_package_files([metapackage], with_scripts=False) # Check whether it got installed, the 'dpkg -i p-d-d.deb && apt-get -yf install' approach # may not have installed it, cannot happen with 'apt-get install p-d-d.deb' (since stretch) if not chroot.is_installed(["piuparts-depends-dummy"]): logging.error("Installation of piuparts-depends-dummy FAILED") # don't panic(), too many problems on old distros # Now remove it metapackagename = os.path.basename(metapackage)[:-4] chroot.purge_packages([metapackagename]) cleanup_metapackage() dont_do_on_panic(panic_handler_id) # Save the file ownership information so we can tell which # modifications were caused by the actual packages we are testing, # rather than by their dependencies. deps_info = chroot.get_tree_meta_data() if settings.install_purge_install: # save chroot state with all deps installed chroot_state_with_deps = chroot.get_state_meta_data() chroot.check_for_no_processes() chroot.check_for_broken_symlinks(warn_only=True) # warn only since no scripts could fix up things after installing the dependencies chroot.install_packages(package_files, packages, with_scripts=False) chroot.run_scripts("post_install") if settings.install_purge_install: file_owners = chroot.get_files_owned_by_packages() chroot.restore_selections(chroot_state_with_deps, packages) logging.info("Validating chroot after purge") chroot.check_debsums() chroot.check_for_no_processes() chroot.check_for_broken_symlinks(file_owners=file_owners) if not check_results(chroot, chroot_state_with_deps, file_owners, deps_info=deps_info): return False logging.info("Reinstalling after purge") chroot.install_packages(package_files, packages, with_scripts=True) if settings.install_remove_install: chroot.remove_packages(packages, ignore_errors=True) logging.info("Reinstalling after remove") chroot.install_packages(package_files, packages, with_scripts=True) chroot.install_packages(package_files, packages, with_scripts=True, reinstall=True) chroot.disable_testdebs_repo() file_owners = chroot.get_files_owned_by_packages() chroot.check_for_no_processes() chroot.check_for_broken_symlinks(file_owners=file_owners) # Remove all packages from the chroot that weren't there initially. chroot.restore_selections(chroot_state, packages) chroot.run_scripts("post_test") chroot.check_for_no_processes(fail=True) chroot.check_for_broken_symlinks(file_owners=file_owners) return check_results(chroot, chroot_state, file_owners, deps_info=deps_info) def install_upgrade_test(chroot, chroot_state, package_files, packages, old_packages): """Install old_packages via apt-get, then upgrade from package files. Return True if successful, False if not.""" os.environ["PIUPARTS_TEST"] = "upgrade" chroot.run_scripts("pre_test") # First install via apt-get. os.environ["PIUPARTS_PHASE"] = "install" chroot.install_packages_by_name(old_packages) chroot.check_for_no_processes() chroot.check_for_broken_symlinks() if settings.install_remove_install: chroot.remove_packages(packages, ignore_errors=True) # Then from the package files. os.environ["PIUPARTS_PHASE"] = "upgrade" chroot.enable_testdebs_repo() chroot.install_packages(package_files, packages) chroot.disable_testdebs_repo() file_owners = chroot.get_files_owned_by_packages() chroot.check_for_no_processes() chroot.check_for_broken_symlinks(file_owners=file_owners) # Remove all packages from the chroot that weren't there initially. chroot.restore_selections(chroot_state, packages) chroot.run_scripts("post_test") chroot.check_for_no_processes(fail=True) chroot.check_for_broken_symlinks(file_owners=file_owners) return check_results(chroot, chroot_state, file_owners) def save_meta_data(filename, chroot_state): """Save directory tree meta data into a file for fast access later.""" logging.debug("Saving chroot meta data to %s" % filename) with open(filename, "wb") as f: pickle.dump(chroot_state, f) def load_meta_data(filename): """Load meta data saved by 'save_meta_data'.""" logging.debug("Loading chroot meta data from %s" % filename) with open(filename, "rb") as f: return pickle.load(f) def install_and_upgrade_between_distros(package_files, packages_qualified): """Install package and upgrade it between distributions, then remove. Return True if successful, False if not.""" # this function is a bit confusing at first, because of what it does by default: # 1. create chroot with source distro # 2. upgrade chroot to target distro # 3. remove chroot and recreate chroot with source distro # 4. install depends in chroot # 5. install package in chroot # 6. upgrade chroot to target distro # 7. remove package and depends # 8. compare results # # sounds silly, or? # well, it is a reasonable default (see below for why), but # step 2+3 can be done differently by using --save-end-meta once and # then --end-meta for all following runs - until the target distro # changes again... # # Under normal circumstances the target distro can change anytime, ie. at # the next mirror pulse, so unless the target distro is frozen, this is # a reasonable default behaviour for distro upgrade tests, which are not # done by default anyway. os.environ["PIUPARTS_TEST"] = "distupgrade" packages = unqualify(packages_qualified) chroot = get_chroot() chroot.create() chroot.remember_initial_selections() chroot_state = None if settings.end_meta: if os.path.exists(settings.end_meta): chroot_state = load_meta_data(settings.end_meta) else: logging.info("Cannot load chroot state from %s - generating it on-the-fly." % settings.end_meta) if chroot_state is not None: if chroot.initial_selections != chroot_state["initial_selections"]: logging.warn("Initial package selections do not match - ignoring loaded reference chroot state") refsel = [(s, p, v) for p, (s, v) in six.iteritems(chroot_state["initial_selections"])] cursel = [(s, p, v) for p, (s, v) in six.iteritems(chroot.initial_selections)] rsel = [x for x in refsel if not x in cursel] csel = [x for x in cursel if not x in refsel] [logging.debug(" -%s" % " ".join(x)) for x in rsel] [logging.debug(" +%s" % " ".join(x)) for x in csel] chroot_state = None if chroot_state is None: temp_tgz = None if chroot.was_bootstrapped(): temp_tgz = chroot.create_temp_tgz_file() panic_handler_id = do_on_panic(lambda: chroot.remove_temp_tgz_file(temp_tgz)) chroot.pack_into_tgz(temp_tgz) chroot.upgrade_to_distros(settings.debian_distros[1:], []) chroot.check_for_no_processes(fail=True) chroot_state = chroot.get_state_meta_data() if settings.save_end_meta: save_meta_data(settings.save_end_meta, chroot_state) chroot.remove() # leave indication in logfile why we do what we do logging.info( "Notice: package selections and meta data from target distro saved, now starting over from source distro. See the description of --save-end-meta and --end-meta to learn why this is neccessary and how to possibly avoid it.") chroot = get_chroot() if temp_tgz is None: chroot.create() else: chroot.create(temp_tgz) chroot.remove_temp_tgz_file(temp_tgz) dont_do_on_panic(panic_handler_id) chroot.check_for_no_processes(fail=True) cannot_test = chroot.run_scripts("is_testable", ignore_errors=True) if cannot_test != 0: if cannot_test & 2: logging.info("FAIL: All tests. Package cannot be tested with piuparts: %s.", " ".join(packages)) retval = False else: logging.info("SKIP: All tests. Package cannot be tested with piuparts: %s.", " ".join(packages)) retval = True testable = False chroot.remove() return retval if settings.shell_on_error: panic_handler_id = do_on_panic(lambda: chroot.interactive_shell()) chroot.run_scripts("pre_test") os.environ["PIUPARTS_PHASE"] = "install" distupgrade_packages = packages known_packages = chroot.get_known_packages(packages + settings.extra_old_packages) chroot.install_packages_by_name(known_packages) if settings.install_remove_install: chroot.remove_packages(packages, ignore_errors=True) distupgrade_packages = [] chroot.check_for_no_processes() os.environ["PIUPARTS_PHASE"] = "distupgrade" chroot.upgrade_to_distros(settings.debian_distros[1:-1], distupgrade_packages, settings.upgrade_before_dist_upgrade) if settings.distupgrade_to_testdebs: chroot.enable_testdebs_repo(update=False) chroot.upgrade_to_distros(settings.debian_distros[-1:], distupgrade_packages, settings.upgrade_before_dist_upgrade) chroot.check_for_no_processes() os.environ["PIUPARTS_PHASE"] = "upgrade" if not settings.distupgrade_to_testdebs: chroot.enable_testdebs_repo() chroot.install_packages(package_files, [p for p in packages_qualified if not p.endswith("=None")]) chroot.disable_testdebs_repo() file_owners = chroot.get_files_owned_by_packages() chroot.check_for_no_processes() # Remove all packages from the chroot that weren't in the reference chroot. chroot.restore_selections(chroot_state, packages_qualified) chroot.run_scripts("post_test") chroot.check_for_no_processes(fail=True) result = check_results(chroot, chroot_state, file_owners) if settings.shell_on_error: dont_do_on_panic(panic_handler_id) if not result: chroot.interactive_shell() chroot.remove() return result def parse_mirror_spec(str, defaultcomponents=[]): """Parse a mirror specification from the --mirror option argument. Return (mirror, componentslist).""" parts = str.split() return parts[0], parts[1:] or defaultcomponents[:] def find_default_debian_mirrors(): """Find the default Debian mirrors.""" mirrors = [] try: for line in readlines_file("/etc/apt/sources.list"): line = re.sub('\[arch=.*\]', '', line) parts = line.split() if len(parts) > 2 and parts[0] == "deb": mirrors.append((parts[1], parts[3:])) break # Only use the first one, at least for now. except IOError: return None return mirrors def forget_ignores(option, opt, value, parser, *args, **kwargs): settings.bindmounts = [] parser.values.ignore = [] parser.values.ignore_regex = [] settings.ignored_files = [] settings.ignored_patterns = [] def set_basetgz_to_pbuilder(option, opt, value, parser, *args, **kwargs): parser.values.basetgz = "/var/cache/pbuilder/base.tgz" def parse_command_line(): """Parse the command line, change global settings, return non-options.""" parser = optparse.OptionParser(usage="%prog [options] package ...", version="piuparts %s" % VERSION) parser.add_option("-a", "--apt", action="store_true", default=False, help="Command line arguments are package names " + "to be installed via apt.") parser.add_option("--arch", metavar="ARCH", action="store", help="Create chroot and run tests for (non-default) architecture ARCH.") parser.add_option("-b", "--basetgz", metavar="TARBALL", help="Use TARBALL as the contents of the initial " + "chroot, instead of building a new one with " + "debootstrap.") parser.add_option("--bindmount", action="append", metavar="DIR", default=[], help="Directory to be bind-mounted inside the chroot.") parser.add_option("-d", "--distribution", action="append", metavar="NAME", help="Which Debian distribution to use: a code name " + "(for example buster, bullseye, sid) or experimental. The " + "default is sid (=unstable).") parser.add_option("-D", "--defaults", action="store", help="Choose which set of defaults to use " "(debian/ubuntu).") parser.add_option("--debfoster-options", default="-o MaxPriority=required -o UseRecommends=no -f -n apt debfoster", help="Run debfoster with different parameters (default: -o MaxPriority=required -o UseRecommends=no -f -n apt debfoster).") parser.add_option("--no-eatmydata", default=False, action='store_true', help="Default is to use libeatmydata in the chroot") parser.add_option("--dpkg-noforce-unsafe-io", default=False, action='store_true', help="Default is to run dpkg with --force-unsafe-io option, which causes dpkg to skip certain file system syncs known to cause substantial performance degradation on some filesystems. This option turns that off and dpkg will use safe I/O operations.") parser.add_option("--dpkg-force-confdef", default=False, action='store_true', help="Make dpkg use --force-confdef, which lets dpkg always choose the default action when a modified conffile is found. This option will make piuparts ignore errors it was designed to report and therefore should only be used to hide problems in depending packages. (See #466118.)") parser.add_option("--do-not-verify-signatures", default=False, action='store_true', help="Do not verify signatures from the Release files when running debootstrap.") parser.add_option("--no-check-valid-until", default=False, action='store_true', help="Set apt option Acquire::Check-Valid-Until=false for testing archived releases.") parser.add_option("--allow-database", default=False, action='store_true', help="Allow database servers (MySQL, PostgreSQL) to be started in the chroot.") parser.add_option("--distupgrade-to-testdebs", default=False, action='store_true', help="Use the testdebs repository as distupgrade target.") parser.add_option("--distupgrade-to-testdebs-from", metavar="DIR", help="Shorthand for --bindmount DIR --testdebs-repo DIR --distupgrade-to-testdebs.") parser.add_option("-e", "--existing-chroot", metavar="DIR", help="Use DIR as the contents of the initial " + "chroot, instead of building a new one with " + "debootstrap") parser.add_option("--hard-link", default=False, action='store_true', help="When using --existing-chroot, and the source dir is on the same" "filesystem, hard-link files instead of copying them.") parser.add_option("-i", "--ignore", action="append", metavar="FILENAME", default=[], help="Add FILENAME to list of filenames to be " + "ignored when comparing changes to chroot." "FILENAMES prefixed with ':' will be reported verbosely.") parser.add_option("-I", "--ignore-regex", "--ignore-regexp", action="append", metavar="REGEX", default=[], help="Add REGEX to list of Perl compatible regular " + "expressions for filenames to be " + "ignored when comparing changes to chroot." "Patterns prefixed with ':' will report all matches verbosely.") parser.add_option("--install-recommends", action="store_true", default=False, help="Enable the installation of Recommends.") parser.add_option("--install-suggests", action="store_true", default=False, help="Enable the installation of Suggests.") def keep_env_parser(option, opt_str, value, parser): setattr(parser.values, option.dest, True) if "--keep-tmpdir" == opt_str: print('WARNING `--keep-tmpdir` is deprecated, use `--keep-env` ' 'instead') parser.add_option("-k", "--keep-env", "--keep-tmpdir", action="callback", callback=keep_env_parser, default=False, dest='keep_env', help="Keep the environment used for testing after " "the program ends.") parser.add_option("-K", "--keyring", action="store", metavar="FILE", help="Use FILE as the keyring to use with debootstrap when creating chroots.") parser.add_option("--keep-sources-list", action="store_true", default=False, help="Don't modify the chroot's " + "etc/apt/sources.list.") parser.add_option("-l", "--log-file", "--logfile", metavar="FILENAME", help="Write log file to FILENAME in addition to " + "the standard output.") parser.add_option("--list-installed-files", action="store_true", default=False, help="List files added to the chroot after the " + "installation of the package.") parser.add_option("--lvm-volume", metavar="LVM-VOL", action="store", help="Use LVM-VOL as source for the chroot, instead of building " + "a new one with debootstrap. This creates a snapshot of the " + "given LVM volume and mounts it to the chroot path") parser.add_option("--lvm-snapshot-size", metavar="SNAPSHOT-SIZE", action="store", default="1G", help="Use SNAPSHOT-SIZE as snapshot size when creating " + "a new LVM snapshot (default: 1G)") parser.add_option("--schroot", metavar="SCHROOT-NAME", action="store", help="Use schroot session named SCHROOT-NAME for the " "testing environment, instead of building a new one " "with debootstrap.") parser.add_option("--docker-image", metavar="DOCKER-IMAGE", action="store", help="Use a container created from the docker image " "DOCKER-IMAGE for the testing environment, instead of " "building a new one with debootstrap.") parser.add_option("--merged-usr", default=False, action='store_true', help="Create a --merged-usr chroot (if using debootstrap).") parser.add_option("-m", "--mirror", action="append", metavar="URL", default=[], help="Which Debian mirror to use.") parser.add_option("--extra-repo", action="append", default=[], help="Additional (unparsed) lines to be appended to sources.list, e.g. " + "'deb <URL> <distrib> <components>' or 'deb file://</bind/mount> ./'") parser.add_option("--testdebs-repo", help="A repository that contains the packages to be tested, e.g. " + "'deb <URL> <distrib> <components>...' or 'deb file://</bind/mount> ./'," + "plain URLs or local paths are permitted, too.") parser.add_option("--no-adequate", default=False, action='store_true', help="Don't run adequate after installation.") parser.add_option("--no-diversions", action="store_true", default=False, help="Don't check for broken diversions.") parser.add_option("-n", "--no-ignores", action="callback", callback=forget_ignores, help="Forget all ignores set so far, including " + "built-in ones.") parser.add_option("-N", "--no-symlinks", action="store_true", default=False, help="Don't check for broken symlinks.") parser.add_option("--upgrade-before-dist-upgrade", action="store_true", default=False, help="Perform two-stage upgrades: 'apt-get upgrade && apt-get dist-upgrade'") parser.add_option("--no-upgrade-test", action="store_true", default=False, help="Skip testing the upgrade from an existing version " + "in the archive.") parser.add_option("--no-install-purge-test", action="store_true", default=False, help="Skip install and purge test.") parser.add_option("--install-purge-install", action="store_true", default=False, help="Purge package after installation and reinstall.") parser.add_option("--install-remove-install", action="store_true", default=False, help="Remove package after installation and reinstall. For testing installation in config-files-remaining state.") parser.add_option("--fake-essential-packages", action="append", default=[], help="Install additional packages in the base chroot that are not removed after the test. " + "Takes a comma separated list of package names and can be given multiple times. " + "Useful for packages that can be used during purge of the package to be tested " + "or to test whether the package to be tested mishandles these packages.") parser.add_option("--extra-old-packages", action="append", default=[], help="Install these additional packages along with the old packages from the archive. " + "Useful to test Conflicts/Replaces of packages that will disappear during the update. " + "Takes a comma separated list of package names and can be given multiple times. " + "For install/purge tests these packages will be installed before the package that is to be tested.") parser.add_option("-p", "--pbuilder", action="callback", callback=set_basetgz_to_pbuilder, help="Use /var/cache/pbuilder/base.tgz as the base " + "tarball.") parser.add_option("--pedantic-purge-test", action="store_true", default=False, help="Be pedantic when checking if a purged package leaves files behind. If this option is not set, files left in /tmp are ignored.") parser.add_option("--proxy", metavar="URL", help="Use the proxy at URL for accessing the mirrors.") parser.add_option("-s", "--save", metavar="FILENAME", help="Save the chroot into FILENAME.") parser.add_option("-B", "--end-meta", metavar="FILE", help="Load chroot package selection and file meta data from FILE. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests.") parser.add_option("-S", "--save-end-meta", metavar="FILE", help="Save chroot package selection and file meta data in FILE for later use. See the function install_and_upgrade_between_distros() in piuparts.py for defaults. Mostly useful for large scale distro upgrade tests.") parser.add_option("--single-changes-list", default=False, action="store_true", help="test all packages from all changes files together.") parser.add_option("--single-packages", default=False, action="store_true", help="test all packages from the command line individually.") parser.add_option("--skip-cronfiles-test", action="store_true", default=False, help="Skip testing the output from the cron files.") parser.add_option("--skip-logrotatefiles-test", action="store_true", default=False, help="Skip testing the output from the logrotate files.") parser.add_option("--skip-minimize", action="store_true", default=True, help="Skip minimize chroot step. This is the default now.") parser.add_option("--minimize", action="store_true", default=False, help="Minimize chroot with debfoster. This used to be the default until #539142 was fixed.") parser.add_option("--scriptsdir", metavar="DIR", action="append", default=[], help="Directory where are placed the custom scripts. Can be given multiple times.") parser.add_option("--shell-on-error", default=False, action='store_true', help="Execute an interactive shell in the chroot if an error occurred.") parser.add_option("-t", "--tmpdir", metavar="DIR", help="Use DIR for temporary storage. Default is " + "$TMPDIR or /tmp.") parser.add_option("-v", "--verbose", action="store_true", default=False, help="No meaning anymore.") parser.add_option("--warn-on-others", action="store_true", default=False, help="Print a warning rather than failing if " "files are left behind, modified, or removed " "by a package that was not given on the " "command-line. Behavior with multiple packages " "given could be problematic, particularly if the " "dependency tree of one package in the list " "includes another in the list. Therefore, it is " "recommended to use this option with one package " "at a time.") parser.add_option("--warn-on-leftovers-after-purge", action="store_true", default=False, help="Print a warning rather than failing if " "files are left behind after purge.") parser.add_option("--warn-on-debsums-errors", action="store_true", default=False, help="Print a warning rather than failing if " "debsums reports modified files.") parser.add_option("--warn-on-install-over-symlink", action="store_true", default=False, help="Print a warning rather than failing if " "files are installed over existing symlinks.") parser.add_option("--fail-if-inadequate", action="store_true", default=False, help="Fail on inadequate results from running adequate.") parser.add_option("--fail-on-broken-symlinks", action="store_true", default=False, help="Fail if broken symlinks are detected.") parser.add_option("--log-level", action="store", metavar='LEVEL', default="dump", help="Displays messages from LEVEL level, possible values are: error, info, dump, debug. The default is dump.") parser.add_option("--max-command-output-size", action="store", metavar='SIZE', default=0, help="Set maximum permitted command output to SIZE (in MB).") (opts, args) = parser.parse_args() # expand combined options if opts.distupgrade_to_testdebs_from: opts.bindmount.append(opts.distupgrade_to_testdebs_from) opts.testdebs_repo = opts.distupgrade_to_testdebs_from opts.distupgrade_to_testdebs = True settings.defaults = opts.defaults defaults = DefaultsFactory().new_defaults() settings.tmpdir = opts.tmpdir settings.keep_env = opts.keep_env settings.shell_on_error = opts.shell_on_error if opts.max_command_output_size: settings.max_command_output_size = int(opts.max_command_output_size) * 1024 * 1024; settings.single_changes_list = opts.single_changes_list settings.single_packages = opts.single_packages settings.args_are_package_files = not opts.apt # distro setup settings.proxy = opts.proxy if settings.proxy: os.environ["http_proxy"] = settings.proxy settings.debian_mirrors = [parse_mirror_spec(x, defaults.get_components()) for x in opts.mirror] settings.extra_repos = opts.extra_repo settings.testdebs_repo = opts.testdebs_repo settings.debian_distros = opts.distribution settings.keep_sources_list = opts.keep_sources_list if opts.keyring: settings.keyring = opts.keyring else: settings.keyring = defaults.get_keyring() settings.do_not_verify_signatures = opts.do_not_verify_signatures if settings.do_not_verify_signatures: settings.apt_unauthenticated = "Yes" else: settings.apt_unauthenticated = "No" settings.no_check_valid_until = opts.no_check_valid_until settings.install_recommends = opts.install_recommends settings.install_suggests = opts.install_suggests settings.eatmydata = not opts.no_eatmydata settings.dpkg_force_unsafe_io = not opts.dpkg_noforce_unsafe_io settings.dpkg_force_confdef = opts.dpkg_force_confdef settings.scriptsdirs = opts.scriptsdir settings.bindmounts += opts.bindmount settings.allow_database = opts.allow_database # chroot setup settings.arch = opts.arch settings.basetgz = opts.basetgz settings.savetgz = opts.save settings.lvm_volume = opts.lvm_volume settings.lvm_snapshot_size = opts.lvm_snapshot_size settings.existing_chroot = opts.existing_chroot settings.hard_link = opts.hard_link settings.schroot = opts.schroot settings.end_meta = opts.end_meta settings.save_end_meta = opts.save_end_meta settings.skip_minimize = opts.skip_minimize settings.minimize = opts.minimize if settings.minimize: settings.skip_minimize = False settings.debfoster_options = opts.debfoster_options.split() settings.docker_image = opts.docker_image settings.merged_usr = opts.merged_usr # tests and checks settings.no_install_purge_test = opts.no_install_purge_test settings.no_upgrade_test = opts.no_upgrade_test settings.upgrade_before_dist_upgrade = opts.upgrade_before_dist_upgrade settings.distupgrade_to_testdebs = opts.distupgrade_to_testdebs settings.install_purge_install = opts.install_purge_install settings.install_remove_install = opts.install_remove_install settings.list_installed_files = opts.list_installed_files [settings.fake_essential_packages.extend([i.strip() for i in csv.split(",")]) for csv in opts.fake_essential_packages] [settings.extra_old_packages.extend([i.strip() for i in csv.split(",")]) for csv in opts.extra_old_packages] settings.skip_cronfiles_test = opts.skip_cronfiles_test settings.skip_logrotatefiles_test = opts.skip_logrotatefiles_test settings.adequate = not opts.no_adequate settings.check_broken_diversions = not opts.no_diversions settings.check_broken_symlinks = not opts.no_symlinks settings.warn_broken_symlinks = not opts.fail_on_broken_symlinks settings.warn_on_others = opts.warn_on_others settings.warn_on_leftovers_after_purge = opts.warn_on_leftovers_after_purge settings.warn_on_debsums_errors = opts.warn_on_debsums_errors settings.warn_on_install_over_symlink = opts.warn_on_install_over_symlink settings.warn_if_inadequate = not opts.fail_if_inadequate settings.pedantic_purge_test = opts.pedantic_purge_test settings.ignored_files += opts.ignore settings.ignored_patterns += opts.ignore_regex if not settings.pedantic_purge_test: settings.ignored_patterns += settings.non_pedantic_ignore_patterns log_file_name = opts.log_file if opts.log_level == "error": setup_logging(logging.ERROR, log_file_name) elif opts.log_level == "info": setup_logging(logging.INFO, log_file_name) elif opts.log_level == "debug": setup_logging(logging.DEBUG, log_file_name) else: setup_logging(DUMP, log_file_name) exitcode = None if not settings.tmpdir: if "TMPDIR" in os.environ: settings.tmpdir = os.environ["TMPDIR"] else: settings.tmpdir = "/tmp" if not os.path.isdir(settings.tmpdir): logging.error("Temporary directory is not a directory: %s" % settings.tmpdir) panic() for sdir in settings.scriptsdirs: if not os.path.isdir(sdir): logging.error("Scripts directory is not a directory: %s" % sdir) panic() if not settings.debian_distros: settings.debian_distros = defaults.get_distribution() if not settings.debian_mirrors: if opts.defaults: settings.debian_mirrors = defaults.get_mirror() else: settings.debian_mirrors = find_default_debian_mirrors() if not settings.debian_mirrors: settings.debian_mirrors = defaults.get_mirror() settings.distro_config = piupartslib.conf.DistroConfig( DISTRO_CONFIG_FILE, settings.debian_mirrors[0][0]) if settings.keep_sources_list and len(settings.debian_distros) > 1: logging.error("--keep-sources-list only makes sense " "with only one distribution") exitcode = 1 if not args: logging.error("Need command line arguments: " + "names of packages or package files") exitcode = 1 settings.testobjects = args if exitcode is not None: sys.exit(exitcode) return args def get_chroot(): return Chroot() # Process the packages given in a list def process_packages(package_list): # Find the names of packages. if settings.args_are_package_files: packages = get_package_names_from_package_files(package_list) package_files = package_list else: packages = package_list package_files = [] if len(settings.debian_distros) == 1: chroot = get_chroot() chroot.create() if settings.shell_on_error: panic_handler_id = do_on_panic(lambda: chroot.interactive_shell()) chroot_state = chroot.get_state_meta_data() testable = True cannot_test = chroot.run_scripts("is_testable", ignore_errors=True) if cannot_test != 0: testable = False if cannot_test & 2: logging.info("FAIL: All tests. Package cannot be tested with piuparts: %s.", " ".join(packages)) panic() else: logging.info("SKIP: All tests. Package cannot be tested with piuparts: %s.", " ".join(packages)) if testable and not settings.no_install_purge_test: extra_packages = chroot.get_known_packages(settings.extra_old_packages) if not install_purge_test(chroot, chroot_state, package_files, packages, extra_packages): logging.error("FAIL: Installation and purging test.") panic() logging.info("PASS: Installation and purging test.") if testable and not settings.no_upgrade_test: if not settings.args_are_package_files and not settings.testdebs_repo: logging.info("Can't test upgrades: -a or --apt option used.") else: packages_to_query = unqualify(packages) packages_to_query.extend(settings.extra_old_packages) known_packages = chroot.get_known_packages(packages_to_query) if not known_packages: logging.info("Can't test upgrade: packages not known by apt-get.") elif install_upgrade_test(chroot, chroot_state, package_files, packages, known_packages): logging.info("PASS: Installation, upgrade and purging tests.") else: logging.error("FAIL: Installation, upgrade and purging tests.") panic() if settings.shell_on_error: dont_do_on_panic(panic_handler_id) chroot.remove() else: if install_and_upgrade_between_distros(package_files, packages): logging.info("PASS: Upgrading between Debian distributions.") else: logging.error("FAIL: Upgrading between Debian distributions.") panic() def main(): """Main program. But you knew that.""" args = parse_command_line() # check if user has root privileges if os.getuid(): print('You need to be root to use piuparts.') sys.exit(1) logging.info("-" * 78) logging.info("To quickly glance what went wrong, scroll down to the bottom of this logfile.") logging.info("FAQ available at https://wiki.debian.org/piuparts/FAQ") logging.info("The FAQ also explains how to contact us in case you think piuparts is wrong.") logging.info("-" * 78) logging.info("piuparts version %s starting up." % VERSION) logging.info("Command line arguments: %s" % command2string(sys.argv)) logging.info("Running on: %s %s %s %s %s" % os.uname()) # Make sure debconf does not ask questions and stop everything. # Packages that don't use debconf will lose. os.environ["DEBIAN_FRONTEND"] = "noninteractive" # Reduce the amount of ESC-codes in the logfile. os.environ["DPKG_COLORS"] = "never" if "DISPLAY" in os.environ: del os.environ["DISPLAY"] changes_packages_list = [] regular_packages_list = [] changes_p = re.compile('.*\.changes$') for arg in args: if changes_p.match(arg): package_list = process_changes(arg) if settings.single_changes_list: for package in package_list: regular_packages_list.append(package) else: changes_packages_list.append(package_list) else: regular_packages_list.append(arg) if changes_packages_list: for package_list in changes_packages_list: process_packages(package_list) if regular_packages_list: if settings.single_packages: for package in regular_packages_list: process_packages([package]) else: process_packages(regular_packages_list) logging.info("PASS: All tests.") logging.info("piuparts run ends.") if __name__ == "__main__": try: main() except KeyboardInterrupt: print('') print('Piuparts interrupted by the user, exiting...') panic(1) sys.exit(1) except SystemExit: raise except: print('') print('Piuparts caught exception, exiting...') print('-'*60) traceback.print_exc(file=sys.stdout) print('-'*60) panic(1) raise # vi:set et ts=4 sw=4 : �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/�������������������������������������������������������������������������0000755�0000000�0000000�00000000000�13607334167�012617� 5����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/__init__.py��������������������������������������������������������������0000644�0000000�0000000�00000007433�13605426771�014740� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2013-2015 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> import bz2 import lzma import zlib from six.moves import urllib class DecompressedStream(): def __init__(self, fileobj, decompressor=None): self._input = fileobj self._decompressor = decompressor self._buffer = "" self._line_buffer = [] self._i = 0 self._end = 0 self._undecbuf = b'' def _split_decode(self, myb): lmyb = len(myb) for end in range(lmyb, max(lmyb-6, -1), -1): try: return myb[:end].decode(), myb[end:] except UnicodeDecodeError: pass # not returned yet? We have a problem raise UnicodeDecodeError def _refill(self): if self._input is None: return False while True: # repeat until decompressor yields some output or input is exhausted chunk = self._input.read(4096) if not chunk: self.close() return False if self._decompressor: chunk = self._decompressor.decompress(chunk) if isinstance(chunk, bytes): chunk = self._undecbuf + chunk chunk, self._undecbuf = self._split_decode(chunk) self._buffer = self._buffer + chunk if chunk: return True def readline(self): while not self._i < self._end: self._i = self._end = 0 self._line_buffer = None empty = not self._refill() if not self._buffer: break self._line_buffer = self._buffer.splitlines(True) self._end = len(self._line_buffer) self._buffer = "" if not self._line_buffer[-1].endswith("\n") and not empty: self._buffer = self._line_buffer[-1] self._end = self._end - 1 if self._i < self._end: self._i = self._i + 1 return self._line_buffer[self._i - 1] return "" def close(self): if self._input: self._input.close() self._input = self._decompressor = None def open_packages_url(url): """Open a Packages.bz2 file pointed to by a URL""" socket = None error = None for ext in ['.xz', '.bz2', '.gz', '']: try: socket = urllib.request.urlopen(url + ext) except urllib.error.HTTPError as e: error = e else: break else: raise error url = socket.geturl() if ext == '.bz2': decompressor = bz2.BZ2Decompressor() decompressed = DecompressedStream(socket, decompressor) elif ext == '.gz': decompressor = zlib.decompressobj(16 + zlib.MAX_WBITS) decompressed = DecompressedStream(socket, decompressor) elif ext == '.xz': decompressor = lzma.LZMADecompressor() decompressed = DecompressedStream(socket, decompressor) elif ext == '': decompressed = socket else: raise Exception('Unknown compression: {}'.format(ext)) return (url, decompressed) # vi:set et ts=4 sw=4 : �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/conf.py������������������������������������������������������������������0000644�0000000�0000000�00000021654�13605426771�014127� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2012-2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> # # NOTA BENE: This module MUST NOT use the logging module, since it can and # will be used before logging is set up (because logging can be configured # in a configuration file, that's why). # try: import UserDict except ImportError: from collections import UserDict import subprocess import collections import re import distro_info from functools import reduce import six from six.moves import configparser from six.moves import reduce class MissingSection(Exception): def __init__(self, filename, section): self.args = "Section %s not defined in configuration file %s" % \ (section, filename), class MissingMandatorySetting(Exception): def __init__(self, filename, key): self.args = "Value for %s not set in configuration file %s" % \ (key, filename), class Config(UserDict): def __init__(self, section, defaults, mandatory=[], defaults_section=None): UserDict.__init__(self) self._section = section self._defaults_section = defaults_section for key, value in six.iteritems(defaults): self[key] = value self._mandatory = mandatory def read(self, filename): cp = configparser.ConfigParser() cp.read(filename) if not cp.has_section(self._section): raise MissingSection(filename, self._section) for key in self.keys(): if cp.has_option(self._section, key): self[key] = cp.get(self._section, key) elif self._defaults_section and cp.has_option(self._defaults_section, key): self[key] = cp.get(self._defaults_section, key) elif key in self._mandatory: raise MissingMandatorySetting(filename, key) def get_mirror(self, distro=None): if self["mirror"] is not None: return self["mirror"] return "http://deb.debian.org/debian" def get_distros(self): if self["upgrade-test-distros"] is not None: return self["upgrade-test-distros"].split() return [] def get_distro(self): if self["distro"]: return self["distro"] distros = self.get_distros() if distros: return distros[-1] return None def get_start_distro(self): distros = self.get_distros() if distros: return distros[0] return self["distro"] def get_final_distro(self): distros = self.get_distros() if distros: return distros[-1] return self["distro"] def _get_distmap(self): debdist = distro_info.DebianDistroInfo() # start with e.g. "sid" -> "unstable" distmap = collections.defaultdict(lambda: "unknown", [ (debdist.old(), "oldstable"), (debdist.devel(), "unstable"), (debdist.stable(), "stable"), (debdist.testing(), "testing"), ("experimental", "experimental"), ("rc", "experimental"), ]) # add mappings for e.g. "oldstable" -> "oldstable" distmap.update(dict([(val, val) for key, val in six.iteritems(distmap)])) # map e.g. "Debian6" -> "oldstable" where debdist.old(result="fullname") # currently returns 'Debian 6.0 "Squeeze"' dkey = lambda x: "Debian" + re.split('[ \.]', x(result="fullname"))[1] dfuncs = [debdist.old, debdist.stable, debdist.testing] distmap.update(dict([(dkey(x), distmap[x()]) for x in dfuncs])) return distmap def _map_distro(self, distro): distro_root = re.split("[\-\.]", distro)[0] distmap = self._get_distmap() return distmap[distro_root] def get_std_distro(self, distrolist=[]): if not distrolist: distrolist = [self.get_distro()] + self.get_distros() mappedlist = [self._map_distro(x) for x in distrolist] return reduce(lambda x, y: y if y != "unknown" else x, mappedlist) def get_area(self): if self["area"] is not None: return self["area"] return "main" def get_arch(self): if not self["arch"]: # Try to figure it out ourselves, using dpkg p = subprocess.Popen(["dpkg", "--print-architecture"], stdout=subprocess.PIPE) self["arch"] = p.stdout.read().decode().rstrip() return self["arch"] class DistroConfig(UserDict): def __init__(self, filename, mirror): UserDict.__init__(self) self._mirror = mirror self._defaults = { "uri": None, "distribution": None, "components": None, "target-release": None, "depends": None, "candidates": None, } cp = configparser.SafeConfigParser() cp.read(filename) for section in cp.sections(): self[section] = dict(self._defaults) for key in self._defaults.keys(): if cp.has_option(section, key): self[section][key] = cp.get(section, key) def get(self, section, key=None): if not section in self.keys(): self[section] = dict(self._defaults, distribution=section) if not key is None: return self[section][key] return self[section] def _is_virtual(self, distro): uri = self.get(distro, "uri") return uri is not None and uri == "None" def get_mirror(self, distro): if self._is_virtual(distro): distro = self._expand_depends(distro)[0] return self.get(distro, "uri") or self._mirror def get_distribution(self, distro): if self._is_virtual(distro): distro = self._expand_depends(distro)[0] return self.get(distro, "distribution") or distro def get_candidates(self, distro): return (self.get(distro, "candidates") or "").split() or [distro] def _get_packages_url(self, distro, area, arch): return "%s/dists/%s/%s/binary-%s/Packages" % ( self.get_mirror(distro), self.get_distribution(distro), area, arch) def get_packages_urls(self, distro, area, arch): return [self._get_packages_url(d, area, arch) for d in self.get_candidates(distro)] def _get_sources_url(self, distro, area): return "%s/dists/%s/%s/source/Sources" % ( self.get_mirror(distro), self.get_distribution(distro), area) def get_sources_urls(self, distro, area): return [self._get_sources_url(d, area) for d in self.get_candidates(distro)] def get_target_flags(self, distro): tr = self.get(distro, "target-release") if tr: return ["-t", tr] return [] def _expand_depends(self, distro, include_virtual=False): todo = [distro] done = [] seen = [] while todo: curr = todo[0] todo = todo[1:] if not curr in seen: seen.append(curr) todo = (self.get(curr, "depends") or "").split() + [curr] + todo elif not curr in done: if include_virtual or not self._is_virtual(curr): done.append(curr) assert(len(done) > 0) return done def get_deb_lines(self, distro, components): lines = [] for d in self._expand_depends(distro): for c in components: if self[d]["components"] is None or c in self[d]["components"].split(): lines.append("deb %s %s %s" % ( self.get_mirror(d), self.get_distribution(d), c)) return lines def get_basetgz(self, distro, arch, merged_usr=False): # look for the first base distribution for d in self._expand_depends(distro): if self.get(d, "depends"): next # skip partial distro return "%s%s_%s.tar.gz" % (self.get_distribution(d), "-merged-usr" if merged_usr else "", arch) return None # vi:set et ts=4 sw=4 : ������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/dependencyparser.py������������������������������������������������������0000644�0000000�0000000�00000025425�13605426771�016535� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2012 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> """Parser for Debian package relationship strings This module contains the class DependencyParser, which parses Debian package relationship strings (e.g., the Depends header). The class raises the DependencySyntaxError exception on syntactic errors. The result uses SimpleDependency objects. Lars Wirzenius <liw@iki.fi> """ import re class DependencySyntaxError(Exception): """Syntax error in package dependency declaration""" def __init__(self, msg, cursor): self._msg = "Error: %s: %s (text at error: '%s', full text being parsed: '%s')" % \ (cursor.get_position(), msg, cursor.get_text(10), cursor.get_full_text()) def __str__(self): return self._msg def __repr__(self): return self._msg class _Cursor: """Store an input string and a movable location in it""" def __init__(self, myinput): self._input = myinput self._len = len(self._input) self._pos = 0 def skip_whitespace(self): while self._pos < self._len and self._input[self._pos].isspace(): self.mynext() def at_end(self): """Are we at the end of the input?""" self.skip_whitespace() return self._pos >= self._len def mynext(self): """Move to the next character""" if self._pos < self._len: self._pos += 1 def get_char(self): """Return current character, None if at end""" if self._pos >= self._len: return None else: return self._input[self._pos] def get_full_text(self): return self._input def get_text(self, length): """Return up to length characters from the current position""" if self._pos >= self._len: return "" else: return self._input[self._pos:self._pos + length] def match(self, regexp): """Match a regular expression against the current position The cursor is advanced by the length of the match, if any. """ m = regexp.match(self._input[self._pos:]) if m: self._pos += len(m.group()) return m def match_literal(self, literal): """Match a literal string against the current position. Return True and move position if there is a match, else return False. """ if self.get_text(len(literal)) == literal: self._pos += len(literal) return True else: return False def get_position(self): """Return current position, as string""" return "pos %d" % self._pos class SimpleDependency: """Express simple dependency towards another package""" def __init__(self, name, operator, version, arch): self.name = name self.operator = operator self.version = version self.arch = arch def __repr__(self): return "<DEP: %s, %s, %s, %s>" % (self.name, self.operator, self.version, self.arch) class DependencyParser: """Parse Debian package relationship strings Debian packages have a rich language for expressing their relationships. See the Debian Policy Manual, chapter 7 ("Declaring relationships between packages"). This Python module implements a parser for strings expressing such relationships. Syntax of dependency fields (Pre-Depends, Depends, Recommends, Suggests, Conflicts, Provides, Replaces, Enhances, Build-Depends, Build-Depends-Indep, Build-Conflicts, Build-Conflicts-Indep), in a BNF-like form: depends-field ::= EMPTY | dependency ("," dependency)* dependency ::= possible-dependency ("|" possible-dependency)* possible-dependency ::= package-name version-dependency? arch-restriction? version-dependency ::= "(" relative-operator version-number ")" relative-operator ::= "<<" | "<=" | "=" | ">=" | ">>" | "<" | ">" version-number ::= epoch? upstream-version debian-revision? arch-restriction ::= "[" arch-name arch-name* "]" | "[" "!" arch-name ("!" arch-name)* "]" package-name ::= alphanumeric name-char name-char* ":any"? epoch ::= integer ":" upstream-version ::= alphanumeric version-char* -- policy says "should start with digit", but not all packages do debian-revision ::= "-" debian-version-char debian-version-char* arch-name ::= alphanumeric alphanumeric* EMPTY ::= "" integer ::= digit digit* alphanumeric ::= "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" | digit digit ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" name-char ::= alphanumeric | "+" | "-" | "." | "_" version-char ::= alphanumeric | "." | "+" | "-" | ":" | "~" debian-version-char ::= alphanumeric | "." | "+" White space can occur between any tokens except inside package-name, version-number, or arch-name. Some of the headers restrict the syntax somewhat, e.g., Provides does not allow version-dependency, but this is not included in the syntax for simplicity. Note: Added "_" to name-char, because some packages (type-handling in particular) use Provides: headers with bogus package names. Note: Added upper case letters to name pattern, since it some of the Mozilla localization packages use or used them. """ def __init__(self, input_string): self._cursor = _Cursor(input_string) self._list = self._parse_dependencies() def get_dependencies(self): """Return parsed dependencies The result is a list of lists of SimpleDependency objects. Let's try that again. The result is a list of dependencies, corresponding to the comma-separated items in the dependency list. Each dependency is also a list, or SimpleDependency objects, representing alternative ways to fulfill the dependency; in other words, items separated by the vertical bar (|). For example, "foo, bar | foobar" would result in the following list: [[foo], [bar, foobar]]. """ return self._list def _parse_dependencies(self): vlist = [] dep = self._parse_dependency() while dep: vlist.append(dep) self._cursor.skip_whitespace() if self._cursor.at_end(): break if not self._cursor.match_literal(","): raise DependencySyntaxError("Expected comma", self._cursor) dep = self._parse_dependency() return vlist def _parse_dependency(self): vlist = [] dep = self._parse_possible_dependency() while dep: vlist.append(dep) self._cursor.skip_whitespace() if not self._cursor.match_literal("|"): break dep = self._parse_possible_dependency() return vlist def _parse_possible_dependency(self): name = self._parse_package_name() if not name: return None (op, version) = self._parse_version_dependency() arch = self._parse_arch_restriction() return SimpleDependency(name, op, version, arch) _name_pat = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9+._-]+") # The MultiArch spec supports an ":any" modifier. Loosen the # accepted arch's, to avoid crashing. _any_suffix_pat = re.compile(r":[a-zA-Z0-9-]+") def _parse_package_name(self): self._cursor.skip_whitespace() if self._cursor.at_end(): return None m = self._cursor.match(self._name_pat) if not m: raise DependencySyntaxError("Expected a package name", self._cursor) if self._cursor.match(self._any_suffix_pat): pass return m.group() _op_pat = re.compile(r"(<<|<=|=|>=|>>|<(?![<=])|>(?![>=]))") _version_pat = re.compile(r"(?P<epoch>\d+:)?" + r"(?P<upstream>[a-zA-Z0-9+][a-zA-Z0-9.+:~-]*)" + r"(?P<debian>-[a-zA-Z0-9.+]+)?") def _parse_version_dependency(self): self._cursor.skip_whitespace() if self._cursor.get_char() == "(": self._cursor.mynext() self._cursor.skip_whitespace() opm = self._cursor.match(self._op_pat) if not opm: raise DependencySyntaxError("Expected a version relation " + "operator", self._cursor) operator = opm.group() if operator == "<": operator = "<=" elif operator == ">": operator = ">=" self._cursor.skip_whitespace() verm = self._cursor.match(self._version_pat) if not verm: raise DependencySyntaxError("Expected a version number", self._cursor) self._cursor.skip_whitespace() if self._cursor.get_char() != ")": raise DependencySyntaxError("Expected ')'", self._cursor) self._cursor.mynext() return opm.group(), verm.group() else: return None, None _arch_pat = re.compile(r"!?[a-zA-Z0-9-]+") def _parse_arch_restriction(self): self._cursor.skip_whitespace() if self._cursor.get_char() == "[": self.mynext() vlist = [] while True: self._cursor.skip_whitespace() if self._cursor.get_char() == "]": self._cursor.mynext() break m = self._cursor.match(self._arch_pat) if not m: raise DependencySyntaxError("Expected architecture name", self._cursor) vlist.append(m.group()) return vlist else: return None # vi:set et ts=4 sw=4 : �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/dwke.py������������������������������������������������������������������0000644�0000000�0000000�00000020333�13607334167�014124� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright 2013 David Steele (dsteele@gmail.com) # Copyright © 2014 Andreas Beckmann (anbe@debian.org) # # This file is part of Piuparts # # Piuparts is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # Piuparts is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program; if not, see <http://www.gnu.org/licenses/>. # dwke.py is used by master-bin/detect_well_known_errors.py import os import logging import re from collections import namedtuple KPR_EXT = '.kpr' BUG_EXT = '.bug' LOG_EXT = '.log' class Problem(): """ Encapsulate a particular known problem """ required_tags = ["PATTERN", "WHERE", "ISSUE", "HEADER", "HELPTEXT"] optional_tags = ["EXCLUDE_PATTERN", "EXPLAIN", "PRIORITY"] def __init__(self, probpath): """probpath is the path to the problem definition file""" self.probpath = probpath self.name = os.path.basename(probpath) self.short_name = os.path.splitext(self.name)[0] self.tags_are_valid = True self.init_problem() for tag in self.required_tags: if not tag in self.__dict__: self.tags_are_valid = False if "PATTERN" in self.__dict__: self.inc_re = re.compile(self.PATTERN) else: self.inc_re = None if "EXCLUDE_PATTERN" in self.__dict__: self.exc_re = re.compile(self.EXCLUDE_PATTERN) else: self.exc_re = None def valid(self): return self.tags_are_valid def init_problem(self): """Load problem file parameters (HELPTEXT="foo" -> self.HELPTEXT)""" with open(self.probpath, 'r') as pb: probbody = pb.read() tagged = re.sub("^([A-Z_]+=)", "<hdr>\g<0>", probbody, 0, re.MULTILINE) for chub in re.split('<hdr>', tagged)[1:]: (name, value) = re.split("=", chub, 1, re.MULTILINE) while value[-1] == '\n': value = value[:-1] if re.search("^\'.+\'$", value, re.MULTILINE | re.DOTALL) \ or re.search('^\".+\"$', value, re.MULTILINE | re.DOTALL): value = value[1:-1] if name in self.required_tags or name in self.optional_tags: self.__dict__[name] = value else: self.tags_are_valid = False self.WHERE = self.WHERE.split(" ") def has_problem(self, logbody, where): """Does the log text 'logbody' contain this known problem?""" if where in self.WHERE: if self.inc_re.search(logbody, re.MULTILINE): for line in logbody.splitlines(): if self.inc_re.search(line): if self.exc_re is None \ or not self.exc_re.search(line): return True return False def get_command(self): cmd = "grep -E \"%s\"" % self.PATTERN if "EXCLUDE_PATTERN" in self.__dict__: cmd += " | grep -v -E \"%s\"" % self.EXCLUDE_PATTERN return cmd class FailureManager(): """Class to track known failures encountered, by package, where (e.g. 'fail'), and known problem type""" def __init__(self, logdict): """logdict is {pkgspec: fulllogpath} across all log files""" self.logdict = logdict self.failures = [] self.load_failures() def load_failures(self): """Collect failures across all kpr files, as named tuples""" for pkgspec in self.logdict: logpath = self.logdict[pkgspec] try: with open(get_kpr_path(logpath), 'r') as kp: for line in kp: (where, problem) = self.parse_kpr_line(line) self.failures.append(make_failure(where, problem, pkgspec)) except IOError: logging.error("Error processing %s" % get_kpr_path(logpath)) def parse_kpr_line(self, line): """Parse a line in a kpr file into where (e.g. 'pass') and problem name""" m = re.search("^([a-z]+)/.+ (.+)$", line) return (m.group(1), m.group(2)) def sort_by_path(self): self.failures.sort(key=lambda x: self.logdict[x.pkgspec]) def sort_by_bugged_and_rdeps(self, pkgsdb): self.pkgsdb = pkgsdb def keyfunc(x, pkgsdb=self.pkgsdb, logdict=self.logdict): rdeps = pkgsdb.rrdep_count(get_pkg(x.pkgspec)) is_failed = get_where(logdict[x.pkgspec]) == "fail" return (not is_failed, -rdeps, logdict[x.pkgspec]) self.failures.sort(key=keyfunc) def filtered(self, problem): return [x for x in self.failures if problem == x.problem] def make_failure(where, problem, pkgspec): return (namedtuple('Failure', 'where problem pkgspec')(where, problem, pkgspec)) def get_where(logpath): """Convert a path to a log file to the 'where' component (e.g. 'pass')""" return logpath.split('/')[-2] def replace_ext(fpath, newext): basename = os.path.splitext(os.path.split(fpath)[1])[0] return '/'.join(fpath.split('/')[:-1] + [basename + newext]) def get_pkg(pkgspec): return pkgspec.split('_')[0] def get_kpr_path(logpath): """Return the kpr file path for a particular log path""" return replace_ext(logpath, KPR_EXT) def get_file_dict(workdirs, ext): """For files in [workdirs] with extension 'ext', create a dict of <pkgname>_<version>: <path>""" return {os.path.splitext(os.path.basename(fl))[0]: os.path.join(d, fl) for d in workdirs for fl in os.listdir(d) if os.path.splitext(fl)[1] == ext} def create_problem_list(pdir): plist = [] pdir_list = os.listdir(pdir) pdir_list.sort() for pfile in [x for x in pdir_list if x.endswith(".conf")]: prob = Problem(os.path.join(pdir, pfile)) if prob.valid(): plist.append(prob) else: logging.error("Keyword error in %s - skipping" % pfile) return plist def clean_cache_files(logdict, cachedict, recheck=False, recheck_failed=False, skipnewer=False): """Delete files in cachedict if the corresponding logdict file is missing or newer""" count = 0 for pkgspec in cachedict: try: if pkgspec not in logdict \ or (os.path.getmtime(logdict[pkgspec]) > os.path.getmtime(cachedict[pkgspec]) and not skipnewer)\ or get_where(logdict[pkgspec]) != get_where(cachedict[pkgspec])\ or recheck\ or (recheck_failed and not get_where(cachedict[pkgspec]) in ['pass']): os.remove(cachedict[pkgspec]) count = count + 1 except (IOError, OSError): # logfile may have disappeared pass return count def make_kprs(logdict, kprdict, problem_list): """Create kpr files, as necessary, so every log file has one kpr entries are e.g. fail/xorg-docs_1:1.6-1.log broken_symlinks_error.conf""" needs_kpr = set(logdict.keys()).difference(set(kprdict.keys())) for pkg_spec in needs_kpr: logpath = logdict[pkg_spec] try: with open(logpath, 'r', errors='backslashreplace') as lb: logbody = lb.read() where = get_where(logpath) kprs = ["%s/%s.log %s\n" % (where, pkg_spec, problem.name) for problem in problem_list if problem.has_problem(logbody, where)] kprs = ''.join(kprs) if where != 'pass' and not kprs: kprs = "%s/%s.log %s\n" % (where, pkg_spec, "unclassified_failures.conf") with open(get_kpr_path(logpath), 'w') as f: f.write(kprs) except IOError: logging.error("File error processing %s" % logpath) return len(needs_kpr) # vi:set et ts=4 sw=4 : �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/packagesdb.py������������������������������������������������������������0000644�0000000�0000000�00000115303�13605426771�015261� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # Copyright 2005 Lars Wirzenius (liw@iki.fi) # Copyright © 2011-2018 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along with # this program. If not, see <https://www.gnu.org/licenses/> """Packages database for distributed piuparts processing This module contains tools for keeping track of which packages have been tested, the test results, and for determining what to test next. Lars Wirzenius <liw@iki.fi> """ import logging import os import random import stat import tempfile import time try: import UserDict except ImportError: from collections import UserDict import apt_pkg import piupartslib from piupartslib.dependencyparser import DependencyParser import six apt_pkg.init_system() def rfc822_like_header_parse(myinput): headers = [] while True: line = myinput.readline() if not line or line in ["\r\n", "\n"]: break if headers and line and line[0].isspace(): headers[-1] = headers[-1] + line else: headers.append(line) return headers class Package(UserDict): def __init__(self, headers): UserDict.__init__(self) self.headers = headers for header in headers: name, value = header.split(":", 1) self[name.strip()] = value.strip() self._parsed_deps = {} self._parsed_alt_deps = {} self.rrdep_cnt = None self.block_cnt = None self.waiting_cnt = None self.rdep_chain_len = None def name(self): return self["Package"] def version(self): return self["Version"] def source(self): # Binary packages built from the source package with the same name # and version don't have a Source header. if "Source" in self: # If source and binary version differ (e.g. for binNMUs), the # source version is given as the second element in the "Source" # entry. Strip off the optional source version. return self["Source"].split(" ")[0] return self["Package"] def source_version(self): # Binary packages built from the source package with the same name # and version don't have a Source header. if "Source" in self: # If source and binary version differ (e.g. for binNMUs), the # source version is given as the second element in the "Source" # entry. Strip off the parentheses around the source version. if " " in self["Source"]: return self["Source"].split(" ")[1][1:-1] return self["Version"] def set_test_versions(self, tv): self["TestVersions"] = tv def test_versions(self): """ToDo: for distupgrade tests test_versions() should return a list of versions in all distros in the upgrade path""" if "TestVersions" in self: return self["TestVersions"] return self.version() def _parse_dependencies(self, header_name): if header_name in self._parsed_deps: depends = self._parsed_deps[header_name] else: parser = DependencyParser(self[header_name]) depends = parser.get_dependencies() depends = [alternatives[0].name for alternatives in depends] self._parsed_deps[header_name] = depends return depends def _parse_alternative_dependencies(self, header_name): if header_name in self._parsed_alt_deps: depends = self._parsed_alt_deps[header_name] else: parser = DependencyParser(self[header_name]) depends = parser.get_dependencies() depends = [[alt.name for alt in alternatives] for alternatives in depends] self._parsed_alt_deps[header_name] = depends return depends # first alternative only - [package_name...] def dependencies(self): vlist = [] for header in ["Depends", "Pre-Depends"]: if header in self: vlist += self._parse_dependencies(header) return vlist # all alternatives - [[package_name...]...] def all_dependencies(self, header_name=None): headers = ["Depends", "Pre-Depends"] if header_name is not None: headers = [header_name] vlist = [] for header in headers: if header in self: vlist += self._parse_alternative_dependencies(header) return vlist def prefer_alt_depends(self, header_name, dep_idx, dep): if header_name in self: if header_name not in self._parsed_deps: self._parse_dependencies(header_name) if self._parsed_deps[header_name][dep_idx]: self._parsed_deps[header_name][dep_idx] = dep def provides(self): vlist = [] for header in ["Provides"]: if header in self: vlist += self._parse_dependencies(header) return vlist def dump(self, output_file): output_file.write("".join(self.headers)) class PackagesFile(UserDict): def __init__(self): UserDict.__init__(self) self._urllist = [] def load_packages_urls(self, urls, restrict_packages=None): for url in urls: logging.debug("Opening %s.*" % url) (url, stream) = piupartslib.open_packages_url(url) logging.debug("Fetching %s" % url) self._read_file(stream, restrict_packages=restrict_packages) stream.close() self._urllist.append(url) def _read_file(self, myinput, restrict_packages=None): """Parse a Packages file and add its packages to us-the-dict""" while True: headers = rfc822_like_header_parse(myinput) if not headers: break p = Package(headers) if p["Package"] in self: q = self[p["Package"]] if apt_pkg.version_compare(p["Version"], q["Version"]) <= 0: # there is already a newer version continue if restrict_packages is not None: if p["Package"] not in restrict_packages: # unwanted package continue self[p["Package"]] = p def get_urls(self): return self._urllist class LogDB: def exists(self, pathname): try: cache = self.exists_cache except AttributeError: self.exists_cache = {} cache = self.exists_cache if pathname not in cache: cache[pathname] = os.path.exists(pathname) return cache[pathname] def _evict(self, pathname): try: cache = self.exists_cache if pathname in cache: del cache[pathname] except AttributeError: pass def bulk_load_dir(self, dirname): try: cache = self.exists_cache except AttributeError: self.exists_cache = {} cache = self.exists_cache for basename in os.listdir(dirname): if basename.endswith(".log"): cache[os.path.join(dirname, basename)] = True def remove_file(self, pathname): os.remove(pathname) def _log_name(self, package, version): return "%s_%s.log" % (package, version) def _kpr_name(self, package, version): return "%s_%s.kpr" % (package, version) def log_exists2(self, package, version, subdirs): log_name = self._log_name(package, version) for subdir in subdirs: if self.exists(os.path.join(subdir, log_name)): return True return False def log_exists(self, package, subdirs): return self.log_exists2(package.name(), package.test_versions(), subdirs) def create(self, subdir, package, version, contents): (fd, temp_name) = tempfile.mkstemp(dir=subdir) contents = contents.encode() if os.write(fd, contents) != len(contents): raise Exception("Partial write?") os.close(fd) # tempfile.mkstemp sets the file mode to be readable only by owner. # Let's make it follow the umask. umask = os.umask(0) os.umask(umask) os.chmod(temp_name, 0o666 & ~umask) full_name = os.path.join(subdir, self._log_name(package, version)) try: os.link(temp_name, full_name) except OSError: return False finally: os.remove(temp_name) self._evict(full_name) return True def remove(self, subdir, package, version): full_name = os.path.join(subdir, self._log_name(package, version)) if self.exists(full_name): self.remove_file(full_name) self._evict(full_name) def remove_kpr(self, subdir, package, version): full_name = os.path.join(subdir, self._kpr_name(package, version)) if os.path.exists(full_name): self.remove_file(full_name) def stat(self, subdir, package, version): full_name = os.path.join(subdir, self._log_name(package, version)) return os.stat(full_name) class LogfileExists(Exception): def __init__(self, path, package, version): self.args = (path, package, version) class PackagesDB: # these packages are uses as dependencies but are only available # from foreign architectures # HACK: this hardcoded list should be moved to some data file _foreign_packages = { "ia32-libs-i386": "i386", "ia32-libs-gtk-i386": "i386", "libnss-mdns-i386": "i386", } # these packages are used as dependencies but are only available on # some architectures or from third-party repositories # HACK: this hardcoded list should be moved to some data file _ignored_missing_dependencies = [ # non-linux "kbdcontrol", "vidcontrol", # jessie "mediawiki", # stretch "llvm-3.7-dev", # third-party "oracle-instantclient-basic", "oracle-instantclient11.2-basic", "oracle-instantclient12.1-basic", ] # keep in sync with piuparts-report.py: emphasize_reason() # FIXME: can we reorder this list or remove entries without breaking the counts.txt for the plot? _states = [ "successfully-tested", "failed-testing", "cannot-be-tested", "essential-required", # obsolete "waiting-to-be-tested", "waiting-for-dependency-to-be-tested", "dependency-failed-testing", "dependency-cannot-be-tested", "dependency-does-not-exist", "circular-dependency", # obsolete "unknown", "unknown-preferred-alternative", # obsolete "no-dependency-from-alternatives-exists", # obsolete "outdated", # "foreign:*", # can only happen as query result for a dependency # "does-not-exist", # can only happen as query result for a dependency # "ignore-does-not-exist", # can only happen as query result for a dependency ] _good_states = [ "successfully-tested", "essential-required", "ignore-does-not-exist", ] + ["foreign:%s" % arch for arch in set(_foreign_packages.values())] _obsolete_states = [ "essential-required", "circular-dependency", "unknown-preferred-alternative", "no-dependency-from-alternatives-exists", ] _propagate_error_state = { "failed-testing": "dependency-failed-testing", "cannot-be-tested": "dependency-cannot-be-tested", "dependency-failed-testing": "dependency-failed-testing", "dependency-cannot-be-tested": "dependency-cannot-be-tested", "dependency-does-not-exist": "dependency-cannot-be-tested", "does-not-exist": "dependency-does-not-exist", } _propagate_waiting_state = { "waiting-to-be-tested": "waiting-for-dependency-to-be-tested", "waiting-for-dependency-to-be-tested": "waiting-for-dependency-to-be-tested", } def __init__(self, logdb=None, prefix=None): self.prefix = prefix self._packages_files = [] self._ready_for_testing = None self._logdb = logdb or LogDB() self._packages = None self._in_state = None self._package_state = {} self._dependency_databases = [] self._recycle_mode = False self._candidates_for_testing = None self._rdeps = None self.set_subdirs(ok="pass", fail="fail", evil="untestable", reserved="reserved", morefail=["bugged", "affected"], recycle="recycle") self.create_subdirs() def set_subdirs(self, ok=None, fail=None, evil=None, reserved=None, morefail=None, recycle=None): # Prefix all the subdirs with the prefix if self.prefix: pformat = self.prefix + "/%s" else: pformat = "%s" self._submissions = pformat % "submissions.txt" self._all = [] if ok: self._ok = pformat % ok self._all.append(self._ok) if fail: self._fail = pformat % fail self._all.append(self._fail) if evil: self._evil = pformat % evil self._all.append(self._evil) if reserved: self._reserved = pformat % reserved self._all.append(self._reserved) if morefail: self._morefail = [pformat % s for s in morefail] self._all.extend(self._morefail) if recycle: self._recycle = pformat % recycle self._all.append(self._recycle) self._most = [x for x in self._all if x not in [self._reserved, self._recycle]] def create_subdirs(self): for sdir in self._all: if not os.path.exists(sdir): os.makedirs(sdir) def enable_recycling(self): if self._recycle_mode: return True if self._packages is not None: logging.info("too late for recycling") return False for basename in os.listdir(self._recycle): if basename.endswith(".log"): self._recycle_mode = True return True logging.info("nothing to recycle") return False def get_mtime(self): return max([os.path.getmtime(sdir) for sdir in self._all]) def load_packages_urls(self, urls): pf = PackagesFile() pf.load_packages_urls(urls) self._packages_files.append(pf) self._packages = None def load_alternate_versions_from_packages_urls(self, urls): # take version numbers (or None) from alternate URLs pf2 = PackagesFile() pf2.load_packages_urls(urls) for package in self.get_all_packages(): if package.name() in pf2: package.set_test_versions(pf2[package.name()].version()) else: package.set_test_versions("None") def get_urls(self): urls = [] for pf in self._packages_files: urls.extend(pf.get_urls()) return urls def set_dependency_databases(self, dependency_databases=[]): self._dependency_databases = list(dependency_databases) def _find_all_packages(self): if self._packages is None: self._packages = {} self._virtual_packages = {} for pf in self._packages_files: for p in pf.values(): self._packages[p["Package"]] = p for p in self._packages.values(): for provided in p.provides(): if provided != p["Package"]: if provided not in self._virtual_packages: self._virtual_packages[provided] = [] self._virtual_packages[provided].append(p["Package"]) def _get_recursive_dependencies(self, package): assert self._packages is not None deps = [] more = package.dependencies() while more: dep = more[0] more = more[1:] if dep not in deps: deps.append(dep) dep_pkg = self.get_package(dep, recurse=True, resolve_virtual=True) if dep_pkg is not None: more += dep_pkg.dependencies() return deps def _get_dependency_cycle(self, package_name): deps = [] circular = [] more = [package_name] while more: dep = more[0] more = more[1:] if dep not in deps: deps.append(dep) dep_pkg = self.get_package(dep, recurse=True, resolve_virtual=True) if dep_pkg is not None and package_name in self._get_recursive_dependencies(dep_pkg): circular.append(dep) more += dep_pkg.dependencies() return circular def _is_successfully_tested(self, package): # a pass/ log exists but no corresponding recycle/ log exists if self._logdb.log_exists(package, [self._ok]): if not (self._recycle_mode and self._logdb.log_exists(package, [self._recycle])): return True return False def _lookup_package_state(self, package, use_cached_success, check_outdated): if check_outdated: # Check if dependency databases have a newer version of this package. # Use the actual package versions, not the target versions. curr_ver = package.version() for db in self._dependency_databases: dep_ver = db.get_version(package.name()) if dep_ver is not None and apt_pkg.version_compare(curr_ver, dep_ver) < 0: #logging.info("[%s] outdated: %s %s < %s @[%s]" % (self.prefix, package.name(), curr_ver, dep_ver, db.prefix)) return "outdated"; if self._recycle_mode: if self._logdb.log_exists(package, [self._reserved]): return "waiting-to-be-tested" if self._logdb.log_exists(package, [self._recycle]): return "unknown" if self._logdb.log_exists(package, [self._ok]): success = True if not use_cached_success: # if a pass/ log exists but any dependency may be not # trivially satisfiable do not skip dependency resolution for dep in package.dependencies(): if not self.get_package(dep, resolve_virtual=True): success = False break if success: return "successfully-tested" if self._logdb.log_exists(package, [self._fail] + self._morefail): return "failed-testing" if self._logdb.log_exists(package, [self._evil]): return "cannot-be-tested" if self._logdb.log_exists(package, [self._reserved]): return "waiting-to-be-tested" return "unknown" def _compute_package_state(self, package): # First attempt to resolve (still-unresolved) multiple alternative depends # Definitely sub-optimal, but improvement over blindly selecting first one # Select the first alternative in the highest of the following states: # 1) "essential-required" # 2) "successfully-tested" # 3) "waiting-to-be-tested" / "waiting-for-dependency-to-be-tested" # 4) "unknown" (retry later) # and update the preferred alternative of that dependency. # If no alternative is in any of these states we retry later ("unknown") # or set "dependency-does-not-exist". # # Problems: # a) We will test and fail when >=1 "successfully-tested" but another # that failed is selected by apt during test run # b) We may report a status of "waiting-for-dependency-to-be-tested" # instead of "waiting-to-be-tested" depending on the order the # package states get resolved. for header in ["Depends", "Pre-Depends"]: alt_deps = package.all_dependencies(header) for d in range(len(alt_deps)): if len(alt_deps[d]) > 1: alt_found = 0 prefer_alt_score = -1 prefer_alt = None for alternative in alt_deps[d]: altdep_state = self.get_best_package_state(alternative) if altdep_state != "does-not-exist": alt_found += 1 if prefer_alt_score < 3 and altdep_state == "essential-required": prefer_alt = alternative prefer_alt_score = 3 elif prefer_alt_score < 2 and altdep_state == "successfully-tested": prefer_alt = alternative prefer_alt_score = 2 elif prefer_alt_score < 1 and \ altdep_state in ["waiting-to-be-tested", "waiting-for-dependency-to-be-tested"]: prefer_alt = alternative prefer_alt_score = 1 elif prefer_alt_score < 0 and altdep_state == "unknown": prefer_alt = alternative prefer_alt_score = 0 if alt_found == 0: return "dependency-does-not-exist" if prefer_alt_score >= 0: package.prefer_alt_depends(header, d, prefer_alt) dep_states = [(dep, self.get_best_package_state(dep)) for dep in package.dependencies()] for dep, dep_state in dep_states: if dep_state in self._propagate_error_state: return self._propagate_error_state[dep_state] testable = True for dep, dep_state in dep_states: if dep_state not in self._good_states: testable = False break if testable: if self._is_successfully_tested(package): return "successfully-tested" return "waiting-to-be-tested" # treat circular-dependencies as testable (for the part of the circle) circular_deps = self._get_dependency_cycle(package["Package"]) if package["Package"] in circular_deps: testable = True for dep, dep_state in dep_states: if dep in circular_deps: # allow any non-error dep_state on the cycle for testing # (error states are handled by the error propagation above) pass elif dep_state not in self._good_states: # non-circular deps must have passed before testing circular deps testable = False break if testable: if self._is_successfully_tested(package): return "successfully-tested" return "waiting-to-be-tested" for dep, dep_state in dep_states: if dep_state in self._propagate_waiting_state: return self._propagate_waiting_state[dep_state] return "unknown" def _initialize_package_states(self, use_cached_success, check_outdated): self._find_all_packages() self._package_state = {} self._in_state = {} for state in self._states: self._in_state[state] = [] todo = [] for package_name, package in six.iteritems(self._packages): state = self._lookup_package_state(package, use_cached_success, check_outdated) assert state in self._states self._package_state[package_name] = state if state == "unknown": todo.append(package_name) else: self._in_state[state].append(package_name) return todo def _compute_package_states(self, use_cached_success=False): if self._in_state is not None: return self._stamp = time.time() for subdir in self._all: self._logdb.bulk_load_dir(subdir) todo = self._initialize_package_states(use_cached_success=use_cached_success, check_outdated=False) for db in self._dependency_databases: db._compute_package_states(use_cached_success=True) if self._dependency_databases: # redo the initialization to properly resolve "outdated" packages after the dependency databases have been initialized todo = self._initialize_package_states(use_cached_success=use_cached_success, check_outdated=True) while todo: package_names = todo todo = [] done = [] for package_name in package_names: if self._package_state[package_name] == "unknown": state = self._compute_package_state(self._packages[package_name]) assert state in self._states if state == "unknown": todo.append(package_name) else: self._in_state[state].append(package_name) self._package_state[package_name] = state done.append(package_name) if not done: # If we didn't do anything this time, we sure aren't going # to do anything the next time either. break self._in_state["unknown"] = todo for state in self._states: self._in_state[state].sort() def get_states(self): return self._states def get_active_states(self): return [x for x in self._states if not x in self._obsolete_states] def get_error_states(self): return [x for x in self._propagate_error_state.keys() if x in self._states] def get_waiting_states(self): return [x for x in self._propagate_waiting_state.keys() if x in self._states] def get_pkg_names_in_state(self, state): self._compute_package_states() return set(self._in_state[state]) def has_package(self, name): self._find_all_packages() return name in self._packages def get_package(self, name, recurse=False, resolve_virtual=False): self._find_all_packages() if name in self._packages: return self._packages[name] if recurse: for db in self._dependency_databases: if db.has_package(name): return db.get_package(name) if resolve_virtual: providers = self.get_providers(name, recurse=recurse) if providers: return self.get_package(providers[0], recurse=recurse, resolve_virtual=False) return None def get_version(self, name): self._find_all_packages() if name in self._packages: return self._packages[name].version() return None def get_test_versions(self, name): self._find_all_packages() if name in self._packages: return self._packages[name].test_versions() return None def get_providers(self, name, recurse=True): self._find_all_packages() providers = [] if name in self._virtual_packages: providers.extend(self._virtual_packages[name]) if recurse: for db in self._dependency_databases: providers.extend(db.get_providers(name, recurse=False)) return providers def get_all_packages(self): self._find_all_packages() return self._packages.values() def get_all_package_names(self): self._find_all_packages() return self._packages.keys() def get_control_header(self, package_name, header): self._find_all_packages() if header == "Uploaders": # not all (source) packages have an Uploaders header uploaders = "" try: uploaders = self._packages[package_name][header] except: pass return uploaders else: return self._packages[package_name][header] def get_package_state(self, package_name, resolve_virtual=True, recurse=True): self._compute_package_states() if package_name in self._package_state: if recurse and self._package_state[package_name] == "outdated": for db in self._dependency_databases: state = db.get_package_state(package_name, resolve_virtual=resolve_virtual, recurse=False) if state not in ["does-not-exist", "outdated"]: return state return self._package_state[package_name] if package_name in self._virtual_packages: if resolve_virtual: provider = self._virtual_packages[package_name][0] return self._package_state[provider] else: return "virtual" if recurse: for db in self._dependency_databases: state = db.get_package_state(package_name, resolve_virtual=resolve_virtual, recurse=False) if state != "does-not-exist": return state if package_name in self._foreign_packages: return "foreign:%s" % self._foreign_packages[package_name] if package_name in self._ignored_missing_dependencies: return "ignore-does-not-exist" return "does-not-exist" def get_best_package_state(self, package_name, resolve_virtual=True, recurse=True): package_state = self.get_package_state(package_name, resolve_virtual=resolve_virtual, recurse=recurse) if package_state in self._good_states: return package_state providers = [] if resolve_virtual: providers = self.get_providers(package_name, recurse=recurse) if not providers: return package_state states = [self.get_package_state(name, resolve_virtual=False, recurse=recurse) for name in [package_name] + providers] for state in self._good_states + list(self._propagate_waiting_state.keys()) + list(self._propagate_error_state.keys()): if state in states: return state return package_state def _get_package_weight(self, p): # compute the priority of a package that needs testing # result will be used as a reverse sorting key, so higher is earlier waiting_count = self.waiting_count(p["Package"]) rdep_chain_len = self.rdep_chain_len(p["Package"]) if not self._recycle_mode: return ( min(rdep_chain_len, waiting_count), waiting_count, ) try: statobj = self._logdb.stat(self._recycle, p.name(), p.test_versions()) ctime = statobj[stat.ST_CTIME] # last inode modification = time of linking into recycle/ mtime = statobj[stat.ST_MTIME] except OSError: ctime = 0 mtime = 0 return ( min(rdep_chain_len, waiting_count), waiting_count, not self._logdb.log_exists(p, [self._ok]), # prefer problematic logs -ctime / 3600, # prefer older, at 1 hour granularity to allow randomization -mtime / 3600, # prefer older, at 1 hour granularity to allow randomization ) def _find_packages_ready_for_testing(self): if self._candidates_for_testing is None: self._candidates_for_testing = [self.get_package(pn) for pn in self.get_pkg_names_in_state("waiting-to-be-tested")] self._candidates_for_testing = [p for p in self._candidates_for_testing if not self._logdb.log_exists(p, [self._reserved]) or self._logdb.log_exists(p, [self._recycle])] if len(self._candidates_for_testing) > 1: tuples = [(self._get_package_weight(p), random.random(), p) for p in self._candidates_for_testing] self._candidates_for_testing = [x[-1] for x in sorted(tuples, reverse=True)] return self._candidates_for_testing[:] def _remove_unavailable_candidate(self, p): self._candidates_for_testing.remove(p) def reserve_package(self): for p in self._find_packages_ready_for_testing(): if self._logdb.log_exists(p, [self._reserved]): self._remove_unavailable_candidate(p) continue if self._recycle_mode and self._logdb.log_exists(p, [self._recycle]): for vdir in [x for x in self._most if x != self._ok]: if self._logdb.log_exists(p, [vdir]): self._logdb.remove(vdir, p.name(), p.test_versions()) logging.info("Recycled %s %s %s" % (vdir, p.name(), p.test_versions())) elif self._logdb.log_exists(p, self._most): self._remove_unavailable_candidate(p) continue if self._logdb.log_exists(p, [self._recycle]): self._logdb.remove(self._recycle, p.name(), p.test_versions()) if self._logdb.create(self._reserved, p.name(), p.test_versions(), ""): return p return None def _check_for_acceptability_as_filename(self, str): if "/" in str: raise Exception("'/' in (partial) filename: %s" % str) def _record_submission(self, category, package, version): with open(self._submissions, "a") as submissions: submissions.write("%d %s %s %s\n" % (time.time(), category, package, version)) def _remove_logs_if_reserved(self, package, version): if self._logdb.log_exists2(package, version, [self._reserved]): for vdir in self._most: if self._logdb.log_exists2(package, version, [vdir]): self._logdb.remove(vdir, package, version) logging.info("Recycled %s %s %s" % (vdir, package, version)) self._logdb.remove(self._reserved, package, version) def unreserve_package(self, package, version): self._check_for_acceptability_as_filename(package) self._check_for_acceptability_as_filename(version) if self._logdb.log_exists2(package, version, [self._reserved]): if not self._logdb.log_exists2(package, version, [self._recycle]): # restore possible recycle marker if self._logdb.log_exists2(package, version, self._most): self._logdb.create(self._recycle, package, version, "") self._logdb.remove(self._reserved, package, version) def _process_log(self, package, version, log, subdir, result): self._check_for_acceptability_as_filename(package) self._check_for_acceptability_as_filename(version) self._remove_logs_if_reserved(package, version) if self._logdb.create(subdir, package, version, log): self._record_submission(result, package, version) self._logdb.remove_kpr(subdir, package, version) else: raise LogfileExists(subdir, package, version) def pass_package(self, package, version, log): self._process_log(package, version, log, self._ok, "pass") def fail_package(self, package, version, log): self._process_log(package, version, log, self._fail, "fail") def make_package_untestable(self, package, version, log): self._process_log(package, version, log, self._evil, "untestable") def _get_rdep_dict(self): """Return dict of one-level reverse dependencies by package""" if self._rdeps is None: self._rdeps = {} for pkg_name in self.get_all_package_names(): for dep in self.get_package(pkg_name).dependencies(): dep_pkg = self.get_package(dep, recurse=True, resolve_virtual=True) if dep_pkg is not None: dep = dep_pkg["Package"] if not dep in self._rdeps: self._rdeps[dep] = set() self._rdeps[dep].add(pkg_name) return self._rdeps def _calc_rrdep_pkg_counts(self, pkg): pkg_name = pkg['Package'] self._compute_package_states() # populate _package_state # calc full recursive reverse dependency package set rrdep_set = set() rdeps = self._get_rdep_dict() next_level = set([pkg_name]) chain_len = 0 while next_level: chain_len += 1 rrdep_set |= next_level new_pkgs = next_level next_level = set([y for x in new_pkgs if x in rdeps for y in rdeps[x]]) next_level -= rrdep_set rrdep_set.remove(pkg_name) # calculate and set the metrics pkg.rrdep_cnt = len(rrdep_set) error_states = self.get_error_states() if self._package_state[pkg_name] in error_states: block_list = [x for x in rrdep_set if self._package_state[x] in error_states] pkg.block_cnt = len(block_list) else: pkg.block_cnt = 0 waiting_states = self.get_waiting_states() if self._package_state[pkg_name] in waiting_states: waiting_list = [x for x in rrdep_set if self._package_state[x] in waiting_states] pkg.waiting_cnt = len(waiting_list) else: pkg.waiting_cnt = 0 pkg.rdep_chain_len = chain_len def block_count(self, name): pkg = self.get_package(name) if pkg is None: return -1 if pkg.block_cnt is None: self._calc_rrdep_pkg_counts(pkg) return pkg.block_cnt def rrdep_count(self, name): pkg = self.get_package(name) if pkg is None: return -1 if pkg.rrdep_cnt is None: self._calc_rrdep_pkg_counts(pkg) return pkg.rrdep_cnt def waiting_count(self, name): pkg = self.get_package(name) if pkg is None: return -1 if pkg.waiting_cnt is None: self._calc_rrdep_pkg_counts(pkg) return pkg.waiting_cnt def rdep_chain_len(self, name): pkg = self.get_package(name) if pkg is None: return -1 if pkg.rdep_chain_len is None: self._calc_rrdep_pkg_counts(pkg) return pkg.rdep_chain_len # vi:set et ts=4 sw=4 : �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/piupartslib/pkgsummary.py������������������������������������������������������������0000644�0000000�0000000�00000016147�13605426771�015402� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/python3 # Copyright 2014 David Steele (dsteele@gmail.com) # # This file is part of Piuparts # # Piuparts is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # Piuparts is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program; if not, see <http://www.gnu.org/licenses/>. # Piuparts summary generation module # # This module is used to create exportable section and global package testing # result summaries. # # The canonical location for section summaries is at # # https://piuparts.debian.org/<section>/summary.json # # The global summary is at # # https://piuparts.debian.org/summary.json # # Example output: # # summary.json # { # "_comment": "Debian Piuparts Package Results - https://piuparts.debian.org/...", # "_date": "Wed Feb 26 01:48:43 UTC 2014", # "_id": "Piuparts Package Test Results Summary", # "_type": "source", # "_version": "1.0", # "packages": { # "0ad": { # "overall": [ # "X", # 0, # "http://localhost/piuparts/sid-fail-broken-symlinks/source/0/0ad.html" # ], # "stable": [ # "P", # 0, # "http://localhost/piuparts/wheezy/source/0/0ad.html" # ], # "unstable": [ # "X", # 0, # "http://localhost/piuparts/sid-fail-broken-symlinks/source/0/0ad.html" # ] # }, # "0ad-data": { # ... # } # # # The packages are listed by source package. E.g. "unstable" here is a # json-section (see README_server.txt). The single character flags are # defined below. The number is the number of packages which # are blocked from testing due to a failed package. The URL is a human # friendly page for inspecting the results for that package/distribution. # # Binary package results are combined into source package results. The 'worst' # flag in the group is reported ("F" is worst overall). # # For the global summary, the packages 'worst' result across json-sections # is used. In the case of a tie, the more-important-precedence # section/json-section result is used. # # The global file also includes an 'overall' json-section, which contains # the 'worst' result across the other json-sections. from __future__ import print_function import json import datetime from collections import namedtuple, defaultdict import six class SummaryException(Exception): pass SUMMID = "Piuparts Package Test Results Summary" SUMMVER = "1.0" DEFSEC = 'overall' FlagInfo = namedtuple('FlagInfo', ['word', 'priority', 'states']) flaginfo = { 'F': FlagInfo('Failed', 0, ["failed-testing"]), 'X': FlagInfo('Blocked', 1, [ "cannot-be-tested", "dependency-failed-testing", "dependency-cannot-be-tested", "dependency-does-not-exist", ]), 'W': FlagInfo('Waiting', 2, [ "waiting-to-be-tested", "waiting-for-dependency-to-be-tested", ]), 'P': FlagInfo('Passed', 3, [ "essential-required", "successfully-tested", ]), '-': FlagInfo('Unknown', 4, [ "does-not-exist", "unknown", ]), } state2flg = dict([(y, x[0]) for x in six.iteritems(flaginfo) for y in x[1].states]) def worst_flag(*flags): try: flag = min(*flags, key=lambda x: flaginfo[x].priority) except KeyError: raise SummaryException("Unknown flag in " + flags.__repr__()) return(flag) def get_flag(state): try: flag = state2flg[state] except KeyError: raise SummaryException("Unknown state - " + state) return(flag) def new_summary(): cdate_array = datetime.datetime.utcnow().ctime().split() utcdate = " ".join(cdate_array[:-1] + ["UTC"] + [cdate_array[-1]]) # define the packages struct. The default should never be the one added dfltentry = ['-', 0, 'invalid url'] pkgstruct = defaultdict(lambda: defaultdict(lambda: dfltentry)) return({ "_id": SUMMID, "_version": SUMMVER, "_date": utcdate, "_comment": "Debian Piuparts Package Results - " "https://salsa.debian.org/debian/piuparts/raw/" "develop/piupartslib/pkgsummary.py", "_type": "source", "packages": pkgstruct, }) def add_summary(summary, rep_sec, pkg, flag, block_cnt, url): if not flag in flaginfo or not isinstance(block_cnt, int) \ or not url.startswith('http'): raise SummaryException("Invalid summary argument") pdict = summary["packages"] [old_flag, old_cnt, old_url] = pdict[pkg][rep_sec] block_cnt = max(block_cnt, old_cnt) if old_flag != worst_flag(old_flag, flag): pdict[pkg][rep_sec] = [flag, block_cnt, url] else: pdict[pkg][rep_sec] = [old_flag, block_cnt, old_url] return summary def merge_summary(gbl_summ, sec_summ): spdict = sec_summ["packages"] for pkg in spdict: for rep_sec in spdict[pkg]: flag, block_cnt, url = spdict[pkg][rep_sec] add_summary(gbl_summ, rep_sec, pkg, flag, block_cnt, url) add_summary(gbl_summ, DEFSEC, pkg, flag, block_cnt, url) return gbl_summ def tooltip(summary, pkg): """Returns e.g. "Failed in testing and stable, blocking 5 packages".""" tip = '' pkgdict = summary['packages'] if pkg in pkgdict: flag, block_cnt, url = pkgdict[pkg][DEFSEC] sections = [x for x in pkgdict[pkg] if x != DEFSEC] applicable = [x for x in sections if pkgdict[pkg][x][0] == flag] tip = flaginfo[flag].word if len(applicable) > 2: tip += ' in ' + ', '.join(applicable[:-1]) + ' and ' + applicable[-1] elif len(applicable) == 2: tip += ' in ' + ' and '.join(applicable) elif len(applicable) == 1: tip += ' in ' + applicable[0] if block_cnt: tip += ", blocking %d packages" % block_cnt tip += '.' return tip def write_summary(summary, fname): with open(fname, 'w') as fl: json.dump(summary, fl, sort_keys=True, indent=1) def read_summary(fname): with open(fname, 'r') as fl: result = json.load(fl) if result["_id"] != SUMMID or result["_version"] != SUMMVER: raise SummaryException('Summary JSON header mismatch') return result if __name__ == '__main__': import sys # read a global summary file and return DDPO info by package summary = read_summary(sys.argv[1]) for pkg in summary['packages']: flag, blocked, url = summary['packages'][pkg][DEFSEC] print(pkg, flag, url, tooltip(summary, pkg)) �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/���������������������������������������������������������������������������0000755�0000000�0000000�00000000000�13605426771�012142� 5����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/detect_leftover_processes.in�����������������������������������������������0000644�0000000�0000000�00000003641�13266771660�017745� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009 Holger Levsen (holger@layer-acht.org) # Copyright © 2012-2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # find processes running in deleted chroots # . @sharedir@/piuparts/lib/read_config.sh # outputs file age in seconds (or 0 if the file does not exist) file_age() { if [ -e "$1" ]; then local ctime now ctime=$(stat -c %Z "$1" 2>/dev/null || echo 0) now=$(date +%s) echo $(($now - $ctime)) else echo "0" fi } get_config_value SLAVEROOT global slave-directory get_config_value PIUPARTS_TMPDIR global tmpdir STATEFILE=$SLAVEROOT/leftover_processes # clear the statefile daily and whine again test $(file_age $STATEFILE) -lt 86000 || rm -f $STATEFILE OUTPUT="$(sudo ls --color=never -lad /proc/*/root 2>/dev/null | grep "$PIUPARTS_TMPDIR" | grep "(deleted)")" if [ -z "$OUTPUT" ]; then rm -f $STATEFILE elif [ "$(cat $STATEFILE 2>/dev/null)" != "$OUTPUT" ]; then echo "Found processes running with a deleted chroot in $PIUPARTS_TMPDIR" echo "This is usually because of 'FAIL: Processes are running inside chroot' which" echo "usually means the package violates 'must use invoke-rc.d (policy 9.3.3.2)'." echo echo "$OUTPUT" echo echo "Please cleanup manually." echo "Since #522918 has been fixed this should no longer happen." echo "$OUTPUT" > $STATEFILE fi �����������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/detect_slave_problems.in���������������������������������������������������0000644�0000000�0000000�00000004042�13371014657�017033� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009-2010 Holger Levsen (holger@layer-acht.org) # Copyright © 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # this scripts monitors the output of piuparts-slave # when running in screen started by ~piupartss/bin/slave_run # . @sharedir@/piuparts/lib/read_config.sh # outputs file age in seconds (or 0 if the file does not exist) file_age() { if [ -e "$1" ]; then local ctime now ctime=$(stat -c %Z "$1" 2>/dev/null || echo 0) now=$(date +%s) echo $(($now - $ctime)) else echo "0" fi } get_config_value SLAVEROOT global slave-directory get_config_value IDLE_SLEEP global idle-sleep 1800 SCREENLOG=$SLAVEROOT/screenlog.1 STATEFILE=$SLAVEROOT/slave-problems # clear the statefile daily and whine again test $(file_age $STATEFILE) -lt 86000 || rm -f $STATEFILE # Only complain if screenlog is older than $IDLE_SLEEP + 30 minutes # and the problem is new or was not reported within the previous 24 hours. if [ $(file_age $SCREENLOG) -le $(($IDLE_SLEEP + 1800)) ]; then rm -f $STATEFILE elif [ ! -f $STATEFILE ]; then { echo "Either a test is running for a very long time (more than $(( ($IDLE_SLEEP + 1800)/60 )) minutes)," echo "or piuparts-slave hangs or is not running at all or wasn't started with" echo "~piupartss/bin/slave_run - please investigate and take appropriate measures!" echo tail $SCREENLOG } | mail -s "problem with piuparts-slave detected" piupartss touch $STATEFILE fi ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/detect_tmp_cruft.in��������������������������������������������������������0000644�0000000�0000000�00000003166�13266771660�016036� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009 Holger Levsen (holger@layer-acht.org) # Copyright © 2012-2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # check for stale mounts and chroots in $PIUPARTS_TMPDIR # . @sharedir@/piuparts/lib/read_config.sh get_config_value PIUPARTS_TMPDIR global tmpdir MOUNTS="$(mktemp)" cp /proc/mounts "$MOUNTS" if [ "$(grep -v trash "$MOUNTS" | grep -c "$PIUPARTS_TMPDIR")" -gt 1 ] ; then echo "More than one mountpoint below $PIUPARTS_TMPDIR detected!" echo grep "$PIUPARTS_TMPDIR" "$MOUNTS" echo echo "Zero or one mountpoint is normal for piuparts operation, more is not." echo "Please investigate and cleanup." echo fi rm "$MOUNTS" LS_TMP=$(ls --color=never -l "$PIUPARTS_TMPDIR") if [ "$(echo "$LS_TMP" | wc -l)" -gt 12 ] ; then echo "More than ten directories in $PIUPARTS_TMPDIR detected!" echo echo "$LS_TMP" echo echo "$(du -shx "$PIUPARTS_TMPDIR" 2>/dev/null)" echo echo "One is normal for piuparts operation, more is not." echo "Please investigate and cleanup." echo fi ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/slave_cleanup.in�����������������������������������������������������������0000644�0000000�0000000�00000003031�13266771660�015313� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2012 Holger Levsen (holger@layer-acht.org) # Copyright © 2012-2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # cleanup old chroots etc. in $PIUPARTS_TMPDIR # # - this should only be run (automatically) on boot # . @sharedir@/piuparts/lib/read_config.sh get_config_value PIUPARTS_TMPDIR global tmpdir # do nothing if piuparts-slave is running if pgrep -f share/piuparts/piuparts-slave >/dev/null ; then exit 0 fi # umount all mount points (should be none on boot, but this script can also be called at other times) for MOUNTPOINT in $(tac /proc/mounts | cut -d " " -f 2 | grep "$PIUPARTS_TMPDIR/") do sudo umount "$MOUNTPOINT" done # cleanup tmp mkdir -p "$PIUPARTS_TMPDIR" for dir in $(ls -d1 "$PIUPARTS_TMPDIR"/*/ 2>/dev/null || true) do if sudo test -f "$dir/.piuparts.tmpdir"; then sudo rm -rf --one-file-system "$dir" test ! -d "$dir" || sudo touch "$dir/.piuparts.tmpdir" fi done �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/slave_join�����������������������������������������������������������������0000755�0000000�0000000�00000002306�13266771660�014225� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009-2010 Holger Levsen (holger@layer-acht.org) # Copyright © 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # reattach to an existing slave session # SESSIONNAME=piuparts_slave_screen if screen -ls $SESSIONNAME | grep -q "No Sockets found" ; then echo "piuparts-slave not running!" echo echo "ps faxu | grep piuparts" ps faxu | grep piuparts echo echo "screen -ls" screen -ls exit 1 fi if [ -w $(tty) ]; then screen -x -S $SESSIONNAME else # use script hack to get a new writable tty script -q -c "screen -x -S $SESSIONNAME" /dev/null fi ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/slave_reexec.in������������������������������������������������������������0000644�0000000�0000000�00000001723�13605426771�015142� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright © 2019 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # Send all running piuparts-slaves SIGUSR1 to reexec themselves, # picking up new code and sections. # for pid in $(pgrep --full '/usr/bin/python3 @sharedir@/piuparts/piuparts-slave') do echo "Sending SIGUSR1 to ${pid}..." kill -USR1 $pid sleep 5 done echo "Done." ���������������������������������������������piuparts-1.1.1/slave-bin/slave_run.in���������������������������������������������������������������0000644�0000000�0000000�00000004403�13266771660�014474� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009-2013 Holger Levsen (holger@layer-acht.org) # Copyright © 2013 Andreas Beckmann (anbe@debian.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # Run (several) piuparts-slave instance(s) in screen to allow interactive # control later on. # . @sharedir@/piuparts/lib/read_config.sh get_config_value PYTHONPATH global PYTHONPATH '' get_config_value SLAVEROOT global slave-directory get_config_value PIUPARTS_TMPDIR global tmpdir get_config_value SLAVECOUNT global slave-count 1 export PYTHONPATH SESSIONNAME=piuparts_slave_screen SCREENLOG=$SLAVEROOT/screenlog if ! screen -ls $SESSIONNAME | grep -q "No Sockets found" ; then echo "piuparts-slave is already running!" echo screen -ls exit 1 fi # cleanup cruft from previous runs @sharedir@/piuparts/slave/slave_cleanup rm -f $SCREENLOG.* # ensure the temporary directory exists mkdir -p $PIUPARTS_TMPDIR mkdir -p $SLAVEROOT cd $SLAVEROOT # Ensure the screen session exists, run normal shell in screen 0 SHELL=/bin/bash screen -S $SESSIONNAME -d -m echo "Started screen session '$SESSIONNAME'." # run this on a single slave or a sequence of slaves # (use value given, if given and an integer and <= $SLAVECOUNT) if [ "$1" != "" ] && [ "$1" -eq "$1" ] && [ "$1" -le "$SLAVECOUNT" ] ; then SLAVES="$(seq $1)" else SLAVES="$(seq $SLAVECOUNT)" fi for SLAVENUM in $SLAVES do mkdir -p $SLAVENUM SLAVEDIR=$(readlink -f $SLAVENUM) # Ensure there is a window for this slave. screen -S $SESSIONNAME -X screen -t slave$SLAVENUM -L $SLAVENUM # Launch the slave. screen -S $SESSIONNAME -p $SLAVENUM -X stuff " cd $SLAVEDIR && @sharedir@/piuparts/piuparts-slave " echo "piuparts-slave $SLAVENUM has been started." done �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/slave-bin/slave_stop.in��������������������������������������������������������������0000644�0000000�0000000�00000003730�13605426771�014654� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2017-2019 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # Stop all running piuparts-slaves gracefully and watch… # . @sharedir@/piuparts/lib/read_config.sh get_config_value SLAVEROOT global slave-directory get_config_value SLAVECOUNT global slave-count 1 SESSIONNAME=piuparts_slave_screen cd $SLAVEROOT LOGS= for i in $(seq 1 $SLAVECOUNT) ; do LOGS="$LOGS screenlog.$i" done tail -f $LOGS & TAILPID=$! trap "kill $TAILPID" INT TERM EXIT sleep 0.5 pkill --signal INT --full '/usr/bin/python3 @sharedir@/piuparts/piuparts-slave' i=1 while pgrep --full '/usr/bin/python3 @sharedir@/piuparts/piuparts-slave' > /dev/null ; do echo -n "$(date -u +%T) - " pgrep --full '/usr/bin/python3 @sharedir@/piuparts/piuparts-slave' | xargs -r echo -n "slaves running: " SLEEP=$(( $i * $i )) BUSY=$(ps fax | grep -v grep | grep '/usr/bin/python3 @sbindir@/piuparts' | awk '{print $NF}') if [ -n "$BUSY" ] ; then # really/meaningful busy echo echo "slaves are busy with:" echo "$BUSY" fi echo -n " sleeping ${SLEEP}sec now..." for j in $(seq 1 $SLEEP) ; do sleep 1 echo -n "." done echo i=$(( $i + 1 )) done echo -n "$(date -u +%T) - " pgrep --full '/usr/bin/python3 @sharedir@/piuparts/piuparts-slave' | xargs echo "slaves running: (should list none) " trap - INT TERM EXIT kill $TAILPID pkill --full "SCREEN -S $SESSIONNAME -d -m" ����������������������������������������piuparts-1.1.1/tests/�������������������������������������������������������������������������������0000755�0000000�0000000�00000000000�13605426771�011424� 5����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/tests/README�������������������������������������������������������������������������0000644�0000000�0000000�00000001221�13266762112�012273� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������To get discovered by nosetests the python files in this directory must be named test_<sourcefile>.py. The test case classes should be named 'NameOfTheClassUnderTestTests' and the test functions should be named 'test_functionundertest_expected_bahavior' e.g.: # foobar.py ----- class FooBar(object): def baz(self): return 'baz ----- # tests/test_foobar.py ----- class FooBarTests(unittest.Testcase): def setUp(self): self.fb = FooBar() def test_baz_should_return_baz(self): ret_val = self.fb.baz() self.assertEqual(ret_val, 'baz') ----- Use pymox as mocking framework if needed. apt-get install python-mox3 �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/tests/test_config.py�����������������������������������������������������������������0000644�0000000�0000000�00000006350�13532216515�014276� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������import unittest import piupartslib.conf as conf import distro_info class ConfStdDistroTests(unittest.TestCase): def setUp(self): self.cobj = conf.Config('notimportant', {}) debdist = distro_info.DebianDistroInfo() self.stable = debdist.stable() self.unstable = debdist.devel() self.oldstable = debdist.old() self.testing = debdist.testing() self.experimental = 'experimental' def testConfStdDistroNames(self): self.assertEqual(self.unstable, 'sid') self.assertEqual(self.experimental, 'experimental') def testConfMapDistro(self): self.assertEqual(self.cobj._map_distro('bogus'), 'unknown') self.assertEqual(self.cobj._map_distro(self.oldstable), 'oldstable') self.assertEqual(self.cobj._map_distro(self.stable), 'stable') self.assertEqual(self.cobj._map_distro(self.testing), 'testing') self.assertEqual(self.cobj._map_distro(self.unstable), 'unstable') self.assertEqual(self.cobj._map_distro(self.experimental), 'experimental') self.assertEqual(self.cobj._map_distro('oldstable'), 'oldstable') self.assertEqual(self.cobj._map_distro('stable'), 'stable') self.assertEqual(self.cobj._map_distro('testing'), 'testing') self.assertEqual(self.cobj._map_distro('unstable'), 'unstable') self.assertEqual(self.cobj._map_distro('experimental'), 'experimental') def testConfMapProposedDistro(self): self.assertEqual( self.cobj._map_distro('stable-proposed'), 'stable') self.assertEqual( self.cobj._map_distro(self.stable + '-proposed'), 'stable') def testConfMapRemainingDistros(self): self.assertEqual(self.cobj._map_distro('rc-buggy'), 'experimental') self.assertEqual( self.cobj._map_distro('Debian6.0.9'), self.cobj._map_distro('squeeze')) self.assertEqual( self.cobj._map_distro('Debian7.4'), self.cobj._map_distro('wheezy')) self.assertEqual( self.cobj._map_distro('Debian8'), self.cobj._map_distro('jessie')) self.assertEqual( self.cobj._map_distro('Debian8.1'), self.cobj._map_distro('jessie')) self.assertEqual( self.cobj._map_distro('Debian9'), self.cobj._map_distro('stretch')) def testConfGetStdDistro(self): for std in [ 'oldstable', 'stable', 'testing', 'unstable', 'experimental']: self.assertEqual( self.cobj.get_std_distro([self.__dict__[std]]), std) self.assertEqual( self.cobj.get_std_distro([self.__dict__[std], 'unknown']), std) self.assertEqual( self.cobj.get_std_distro(['unknown', self.__dict__[std]]), std) self.assertEqual( self.cobj.get_std_distro( ['unknown', 'unknown', self.__dict__[std]]), std) self.assertEqual( self.cobj.get_std_distro( [self.__dict__[std], 'unknown', 'unknown']), std) self.assertEqual(self.cobj.get_std_distro(['unknown']), 'unknown') self.assertEqual( self.cobj.get_std_distro(['unknown', 'unknown']), 'unknown') ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/tests/test_dependencyparser.py�������������������������������������������������������0000644�0000000�0000000�00000001701�13520654773�016370� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������import unittest import piupartslib.dependencyparser class DependencyParserTests(unittest.TestCase): """Tests for module dependencyparser.""" def parse(self, str): parser = piupartslib.dependencyparser.DependencyParser(str) deps = parser.get_dependencies() names = [] for dep in deps: names.append([]) for simpledep in dep: names[-1].append(simpledep.name) return deps, names def testEmpty(self): deps, names = self.parse("") self.failUnlessEqual(deps, []) def testSingle(self): deps, names = self.parse("foo") self.failUnlessEqual(names, [["foo"]]) def testTwo(self): deps, names = self.parse("foo, bar") self.failUnlessEqual(names, [["foo"], ["bar"]]) def testAlternatives(self): deps, names = self.parse("foo, bar | foobar") self.failUnlessEqual(names, [["foo"], ["bar", "foobar"]]) ���������������������������������������������������������������piuparts-1.1.1/tests/test_piuparts.py���������������������������������������������������������������0000644�0000000�0000000�00000017770�13520654773�014721� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������import unittest from mox3 import mox import os import shutil import piuparts from piuparts import is_broken_symlink class DefaultsFactoryTests(unittest.TestCase): def setUp(self): self.mox = mox.Mox() self.df = piuparts.DefaultsFactory() piuparts.settings = piuparts.Settings() def tearDown(self): self.mox.UnsetStubs() def test_new_defaults_return_debian_defaults(self): # mock the guess_flavor function as it runs lsb_release in a subprocess self.mox.StubOutWithMock(self.df, 'guess_flavor') self.df.guess_flavor().AndReturn('debian') self.mox.ReplayAll() defaults = self.df.new_defaults() self.mox.VerifyAll() self.assertEqual(defaults.get_keyring(), '/usr/share/keyrings/debian-archive-keyring.gpg') self.assertEqual(defaults.get_components(), ["main", "contrib", "non-free"]) self.assertEqual(defaults.get_mirror(), [("http://deb.debian.org/debian", ["main", "contrib", "non-free"])]) self.assertEqual(defaults.get_distribution(), ['sid']) def test_new_defaults_return_ubuntu_defaults(self): # mock the guess_flavor function as it runs lsb_release in a subprocess self.mox.StubOutWithMock(self.df, 'guess_flavor') self.df.guess_flavor().AndReturn('ubuntu') self.mox.ReplayAll() defaults = self.df.new_defaults() self.mox.VerifyAll() self.assertEqual(defaults.get_keyring(), '/usr/share/keyrings/ubuntu-archive-keyring.gpg') self.assertEqual(defaults.get_components(), ["main", "universe", "restricted", "multiverse"]) self.assertEqual(defaults.get_mirror(), [("http://archive.ubuntu.com/ubuntu", ["main", "universe", "restricted", "multiverse"])]) def test_new_defaults_panics_with_unknown_flavor(self): # mock the guess_flavor function as it runs lsb_release in a subprocess # and the panic function as it would use sys.exit() self.mox.StubOutWithMock(self.df, 'guess_flavor') self.df.guess_flavor().AndReturn('centos') self.mox.StubOutWithMock(piuparts, 'panic') piuparts.panic().AndReturn('Oh dear! Its CentOS!') self.mox.ReplayAll() defaults = self.df.new_defaults() self.mox.VerifyAll() # panic() would cause sys.exit() so no Defaults object would # ever be returned self.assertEqual(defaults, None) class IsBrokenSymlinkTests(unittest.TestCase): testdir = "is-broken-symlink-testdir" def symlink(self, target, name): pathname = os.path.join(self.testdir, name) os.symlink(target, pathname) self.symlinks.append(pathname) def setUp(self): self.symlinks = [] os.mkdir(self.testdir) self.symlink("notexist", "relative-broken") self.symlink("relative-broken", "relative-broken-to-symlink") self.symlink(".", "relative-works") self.symlink("relative-works", "relative-works-to-symlink") self.symlink("/etc", "absolute-broken") self.symlink("absolute-broken", "absolute-broken-to-symlink") self.symlink("/", "absolute-works") self.symlink("/absolute-works", "absolute-works-to-symlink") os.mkdir(os.path.join(self.testdir, "dir")) self.symlink("dir", "dir-link") os.mkdir(os.path.join(self.testdir, "dir/subdir")) self.symlink("subdir", "dir/subdir-link") self.symlink("notexist/", "trailing-slash-broken") self.symlink("dir/", "trailing-slash-works") self.symlink("selfloop", "selfloop") self.symlink("/absolute-selfloop", "absolute-selfloop") self.symlink("../dir/selfloop", "dir/selfloop") self.symlink("../dir-link/selfloop", "dir/selfloop1") self.symlink("../../dir/subdir/selfloop", "dir/subdir/selfloop") self.symlink("../../dir-link/subdir/selfloop", "dir/subdir/selfloop1") self.symlink("../../link/subdir-link/selfloop", "dir/subdir/selfloop2") self.symlink("../../dir-link/subdir-link/selfloop", "dir/subdir/selfloop3") self.symlink("explode/bomb", "explode") def tearDown(self): shutil.rmtree(self.testdir) def testRelativeBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "relative-broken")) def testRelativeBrokenToSymlink(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "relative-broken-to-symlink")) def testAbsoluteBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "absolute-broken")) def testAbsoluteBrokenToSymlink(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "absolute-broken-to-symlink")) def testTrailingSlashBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "trailing-slash-broken")) def testSelfLoopBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "selfloop")) def testExpandingSelfLoopBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "explode")) def testAbsoluteSelfLoopBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "absolute-selfloop")) def testSubdirSelfLoopBroken(self): self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/selfloop")) self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/selfloop1")) self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/subdir/selfloop")) self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/subdir/selfloop1")) self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/subdir/selfloop2")) self.failUnless(is_broken_symlink(self.testdir, self.testdir, "dir/subdir/selfloop3")) def testRelativeWorks(self): self.failIf(is_broken_symlink(self.testdir, self.testdir, "relative-works")) def testRelativeWorksToSymlink(self): self.failIf(is_broken_symlink(self.testdir, self.testdir, "relative-works-to-symlink")) def testAbsoluteWorks(self): self.failIf(is_broken_symlink(self.testdir, self.testdir, "absolute-works")) def testAbsoluteWorksToSymlink(self): self.failIf(is_broken_symlink(self.testdir, self.testdir, "absolute-works-to-symlink")) def testTrailingSlashWorks(self): self.failIf(is_broken_symlink(self.testdir, self.testdir, "trailing-slash-works")) def testMultiLevelNestedSymlinks(self): # target/first-link -> ../target/second-link -> ../target os.mkdir(os.path.join(self.testdir, "target")) self.symlink("../target", "target/second-link") self.symlink("../target/second-link", "target/first-link") self.failIf(is_broken_symlink(self.testdir, self.testdir, "target/first-link")) def testMultiLevelNestedAbsoluteSymlinks(self): # first-link -> /second-link/final-target # second-link -> /target-dir os.mkdir(os.path.join(self.testdir, "final-dir")) os.mkdir(os.path.join(self.testdir, "final-dir/final-target")) self.symlink("/second-link/final-target", "first-link") self.symlink("/final-dir", "second-link") self.failIf(is_broken_symlink(self.testdir, self.testdir, "first-link")) ��������piuparts-1.1.1/tests/test_pkgsummary.py�������������������������������������������������������������0000644�0000000�0000000�00000012571�13266762112�015235� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������import unittest import datetime import shutil import tempfile import os import json import piupartslib.pkgsummary as pkgsummary class PkgSummaryTests(unittest.TestCase): def testSummFlaginfoStateDups(self): finfo = pkgsummary.flaginfo states = sorted([y for x in finfo for y in finfo[x].states]) nodups = sorted(list(set(states))) self.assertTrue('successfully-tested' in states) self.assertEqual(states, nodups) def testSummGetFlag(self): self.assertEqual('F', pkgsummary.get_flag('failed-testing')) self.assertEqual('X', pkgsummary.get_flag('dependency-does-not-exist')) self.assertEqual('P', pkgsummary.get_flag('successfully-tested')) self.assertEqual('W', pkgsummary.get_flag('waiting-to-be-tested')) with self.assertRaises(pkgsummary.SummaryException): pkgsummary.get_flag('bogus-state') def testSummWorstFlag(self): self.assertEqual('F', pkgsummary.worst_flag('F')) self.assertEqual('P', pkgsummary.worst_flag('P')) self.assertEqual('F', pkgsummary.worst_flag('P', 'F')) self.assertEqual('F', pkgsummary.worst_flag('F', 'F')) self.assertEqual('W', pkgsummary.worst_flag('W', 'P')) self.assertEqual('F', pkgsummary.worst_flag('W', 'P', 'F', 'X', '-')) with self.assertRaises(pkgsummary.SummaryException): pkgsummary.worst_flag('Z') class PkgSummaryAddTests(unittest.TestCase): def setUp(self): self.summ = pkgsummary.new_summary() def testSummNewSumm(self): # Verify any parameters which are depended on downstream self.assertEqual("Piuparts Package Test Results Summary", self.summ['_id']) self.assertEqual("1.0", self.summ['_version']) self.assertEqual({}, self.summ['packages']) thedate = datetime.datetime.strptime(self.summ['_date'], "%a %b %d %H:%M:%S UTC %Y") def testSummAddArgValidation(self): with self.assertRaises(pkgsummary.SummaryException): pkgsummary.add_summary( self.summ, 'foodist', 'foopkg', 'Z', 0, 'http://foo') with self.assertRaises(pkgsummary.SummaryException): pkgsummary.add_summary( self.summ, 'foodist', 'foopkg', 'X', 'bogus', 'http://foo') with self.assertRaises(pkgsummary.SummaryException): pkgsummary.add_summary( self.summ, 'foodist', 'foopkg', 'X', 1, 'ittp://foo') pkgsummary.add_summary( self.summ, 'foodist', 'foopkg', 'X', 1, 'http://foo') def testSummAddArgStorageFormat(self): # store non-overlapping entries pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'X', 0, 'http://foo') pkgsummary.add_summary( self.summ, 'dist', 'pkg2', 'W', 1, 'http://foo2') pkgsummary.add_summary( self.summ, 'dist2', 'pkg3', 'P', 2, 'http://foo3') self.assertEqual( ['X', 0, 'http://foo'], self.summ['packages']['pkg']['dist']) self.assertEqual( ['W', 1, 'http://foo2'], self.summ['packages']['pkg2']['dist']) self.assertEqual( ['P', 2, 'http://foo3'], self.summ['packages']['pkg3']['dist2']) def testSummAddOverwriteFlag(self): pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'X', 0, 'http://foo') pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'P', 0, 'http://foo2') self.assertEqual('X', self.summ['packages']['pkg']['dist'][0]) self.assertEqual('http://foo', self.summ['packages']['pkg']['dist'][2]) pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'F', 0, 'http://foo3') self.assertEqual('F', self.summ['packages']['pkg']['dist'][0]) self.assertEqual('http://foo3', self.summ['packages']['pkg']['dist'][2]) def testSummAddBlockCount(self): pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'X', 0, 'http://foo') pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'P', 1, 'http://foo') self.assertEqual(1, self.summ['packages']['pkg']['dist'][1]) pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'F', 2, 'http://foo') self.assertEqual(2, self.summ['packages']['pkg']['dist'][1]) def testSummMerge(self): pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'X', 0, 'http://foo') mergesumm = pkgsummary.new_summary() pkgsummary.merge_summary(mergesumm, self.summ) self.assertEqual(mergesumm['packages']['pkg']['dist'], self.summ['packages']['pkg']['dist']) self.assertEqual(mergesumm['packages']['pkg']['dist'], mergesumm['packages']['pkg']['overall']) class PkgSummaryStorageTests(unittest.TestCase): def setUp(self): self.summ = pkgsummary.new_summary() pkgsummary.add_summary(self.summ, 'dist', 'pkg', 'X', 0, 'http://foo') self.tmpdir = tempfile.mkdtemp() self.tmpfilename = os.path.join(self.tmpdir, "foo.json") pkgsummary.write_summary(self.summ, self.tmpfilename) def tearDown(self): shutil.rmtree(self.tmpdir) def testSummFileRead(self): summ2 = pkgsummary.read_summary(self.tmpfilename) self.assertEqual(self.summ, summ2) def testSummFileStorage(self): with open(self.tmpfilename, 'r') as fl: summ2 = json.load(fl) self.assertEqual(self.summ, summ2) ���������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/tests/unittests.py�������������������������������������������������������������������0000644�0000000�0000000�00000003375�13605426771�014050� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- import os import unittest import piupartslib.packagesdb import six class FakeLogDB(piupartslib.packagesdb.LogDB): """A fake version of the LogDB class, for testing This version simulates filesystem actions so that there is no need to do actual I/O. Cleaner, although not quite as thorough. """ def __init__(self): self.dict = { "pass": [], "fail": [], "untestable": [], "reserved": [], "bugged": [], "affected": [], } def listdir(self, dirname): return self.dict[dirname] def _parse(self, pathname): return os.path.dirname(pathname), os.path.basename(pathname) def exists(self, pathname): vdir, base = self._parse(pathname) return base in self.dict[vdir] def remove_file(self, pathname): vdir, base = self._parse(pathname) if base in self.dict[vdir]: del self.dict[vdir] def create(self, subdir, package, version, contents): return True class PackagesDbTests(unittest.TestCase): def new_db(self, packages_file_contents): db = piupartslib.packagesdb.PackagesDB(FakeLogDB()) db.read_packages_file(six.StringIO(packages_file_contents)) return db def reserve(self, packages_file_contents): db = self.new_db(packages_file_contents) return db.reserve_package() def testNoPackages(self): p = self.reserve("") self.failUnlessEqual(p, None) def testNoDeps(self): p = self.reserve("""\ Package: foo Version: 1.0-1 """) self.failIfEqual(p, None) self.failUnlessEqual(p["Package"], "foo") if __name__ == "__main__": unittest.main() # vi:set et ts=4 sw=4 : �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/update-piuparts-master-setup���������������������������������������������������������0000755�0000000�0000000�00000005265�13271133277�015771� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009-2018 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # update piuparts master setup from git (eg. used on pejacevic.debian.org) # if [ "`id -n -u`" != "piupartsm" ] ; then echo please run this script as piupartsm user exit 1 fi PIUPARTS_PREFIX=/srv/piuparts.debian.org PIUPARTS_HTDOCS=$PIUPARTS_PREFIX/htdocs PIUPARTS_TMPDIR=$PIUPARTS_PREFIX/tmp # # create $PIUPARTS_PREFIX # if [ ! -d $PIUPARTS_PREFIX ] ; then sudo mkdir -p $PIUPARTS_PREFIX sudo chown piupartsm:piuparts $PIUPARTS_PREFIX sudo chmod 0775 $PIUPARTS_PREFIX fi # # update source # if [ ! -d $PIUPARTS_PREFIX/src ] ; then mkdir -p $PIUPARTS_PREFIX/src chmod 0755 $PIUPARTS_PREFIX/src cd $PIUPARTS_PREFIX/src git clone https://salsa.debian.org/debian/piuparts.git cd piuparts git checkout develop fi cd $PIUPARTS_PREFIX/src/piuparts pwd # git checkout branch if $1 is given if [ ! -z "$1" ] ; then git checkout $1 fi # git fetch+pull if $2 is given if [ ! -z "$2" ] ; then git fetch $2 git pull $2 $1 fi # # install everything from GIT into PIUPARTS_PREFIX # make clean make prefix=$PIUPARTS_PREFIX \ build-master build-doc make prefix=$PIUPARTS_PREFIX \ docdir=$PIUPARTS_HTDOCS/doc \ htdocsdir=$PIUPARTS_HTDOCS \ install-master install-doc install-conf-4-running-from-git make clean # # install copies of the weather icons # to avoid needing FollowSymlinks in the apache config # for icon in weather-severe-alert.png sunny.png do if [ -e /usr/share/icons/Tango/24x24/status/$icon ] ; then cp -f /usr/share/icons/Tango/24x24/status/$icon $PIUPARTS_HTDOCS/images/$icon fi done # # update $PIUPARTS_PREFIX # cd $PIUPARTS_PREFIX pwd mkdir -p master backup # to support multiple hosts with this setup cd etc/piuparts HOSTNAME=$(hostname) for f in piuparts.conf do ln -sf $f.$HOSTNAME $f done # # create working dir # mkdir -p $PIUPARTS_TMPDIR # tell DSA not to back it up touch $PIUPARTS_TMPDIR/.nobackup # # update master home # cd pwd ln -sf $PIUPARTS_PREFIX/share/piuparts/master bin crontab $PIUPARTS_PREFIX/etc/piuparts/crontab-master echo "Update finished." �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������piuparts-1.1.1/update-piuparts-slave-setup����������������������������������������������������������0000755�0000000�0000000�00000005252�13271133277�015604� 0����������������������������������������������������������������������������������������������������ustar ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh set -e # Copyright 2009-2018 Holger Levsen (holger@layer-acht.org) # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see <https://www.gnu.org/licenses/> # # update piuparts slave setup from git (eg. used on piu-slave-*.debian.org) # if [ "`id -n -u`" != "piupartss" ] ; then echo please run this script as piupartss user exit 1 fi PIUPARTS_PREFIX=/srv/piuparts.debian.org PIUPARTS_HTDOCS=$PIUPARTS_PREFIX/htdocs PIUPARTS_TMPDIR=$PIUPARTS_PREFIX/tmp # # create $PIUPARTS_PREFIX # if [ ! -d $PIUPARTS_PREFIX ] ; then sudo mkdir -p $PIUPARTS_PREFIX sudo chown piupartss:piuparts $PIUPARTS_PREFIX sudo chmod 0775 $PIUPARTS_PREFIX fi # # update source # if [ ! -d $PIUPARTS_PREFIX/src ] ; then mkdir -p $PIUPARTS_PREFIX/src chmod 0755 $PIUPARTS_PREFIX/src cd $PIUPARTS_PREFIX/src git clone https://salsa.debian.org/debian/piuparts.git cd piuparts git checkout develop fi cd $PIUPARTS_PREFIX/src/piuparts pwd # git checkout branch if $1 is given if [ ! -z "$1" ] ; then git checkout $1 fi # git fetch+pull if $2 is given if [ ! -z "$2" ] ; then git fetch $2 git pull $2 $1 fi # # install everything from GIT into PIUPARTS_PREFIX # make clean make prefix=$PIUPARTS_PREFIX \ build-slave build-doc make prefix=$PIUPARTS_PREFIX \ docdir=$PIUPARTS_HTDOCS/doc \ htdocsdir=$PIUPARTS_HTDOCS \ install-slave install-doc install-conf-4-running-from-git make clean # # update $PIUPARTS_PREFIX # cd $PIUPARTS_PREFIX pwd mkdir -p slave slave/basetgz # to support multiple hosts with this setup cd etc/piuparts HOSTNAME=$(hostname) for f in piuparts.conf do ln -sf $f.$HOSTNAME $f done # # create working dir # if [ ! -d $PIUPARTS_TMPDIR ] ; then if ! [ -L $PIUPARTS_TMPDIR ] && [ -d /srv/piuparts-workspace ] ; then ln -sf /srv/piuparts-workspace $PIUPARTS_TMPDIR fi fi if [ ! -d $PIUPARTS_TMPDIR ] && [ ! -L $PIUPARTS_TMPDIR ] ; then echo "$PIUPARTS_TMPDIR is neither a directory nor link, please fix." exit 1 fi # tell DSA not to back it up touch $PIUPARTS_TMPDIR/.nobackup # # update slave home # cd pwd ln -sf $PIUPARTS_PREFIX/share/piuparts/slave bin crontab $PIUPARTS_PREFIX/etc/piuparts/crontab-slave echo "Update finished." ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������